Volatile journals is an anti-pattern: Documentation is lacking

[worikgh]

Friends

I am here at the end of a long struggle to determine why my logs were not persisting across reboots. Deleting logs on reboot is unnecessary, or is there a reason I cannot see?

The config file /etc/systemd/journald.conf has Storage set properly, but it was over ridden by this package.

There is no documentation I can see with the package about how to change that

In addition I can see no rational or reason for this package. I imagine there are a lot of reasons for this package existing, they are not bundled with it

I am having to delete this package - I wish there were a better way

[MichaIng]

Do this to enable persistent journal logs:

sudo mkdir -p /etc/systemd/journal.conf.d
cat << '_EOF_' | sudo tee /etc/systemd/journal.conf.d/99-persistent.conf
[Journal]
Storage=persistent
SystemMaxFiles=7
MaxFileSec=2day
_EOF_

or whichever number/lifetime of log files you prefer.

It generally makes sense to use dedicated override files in *.conf.d directories, instead of the base config file. It assures settings in /usr/lib/systemd/*.conf.d are overridden (notably /usr/lib/systemd/journald.conf.d/40-rpi-volatile-storage.conf in this case), /etc/systemd/journal.conf can be auto-updated by dpkg, and you can easier revert or review your custom changes.

[worikgh]

Thank you for the attention

I do not think the specific problem I have with this package disabling persistent logs, disabling logging from my PoV, is the main point. I think I choose a bad title for this issue.

My point is there is no indication what this package is for and why it was installed. This Github repository is the home page from apt show raspberrypi-sys-mods and I cannot find anything here to help (except your reply, thank you, but I do not want to be wasting your time like this).

The only effect I have found is that it breaks logging. (Why? It seems the opposite of "Various modifications to improve the performance or user experience." What am I missing?)

My specific question does not matter - I have deleted this package and now I have logs again - my point is more meta: I cannot tell what good things this package might do.

thanks

[MichaIng]

Persistent (on disk) journal logging is slower, especially on the older RPi models with very limited MMC speed. Until Debian Bookworm, volatile journal logging was the default, only with Trixie, that default was changed at Debian, with their images shipping with the /var/log/journal dir pre-installed, in which case the default Storage=auto uses that automatically. So pros and cons are debatable, and obviously RPi devs chose performance and SD card conservation over persistent journal logs.

Instead of adding /usr/lib/systemd/journald.conf.d/40-rpi-volatile-storage.conf with this package, it might be easier to just remove /var/log/journal in pi-gen after debootstrap, and leave Storage=auto in place. This is how we do it in our project. But either way, one needs to know how to enable persistent journal, where it logs to, or its config file structure.

Generally, this package mostly works with config files. So if you know the respective config file structure, you know how to override anything it configures. You can browse the files here on the repo, which moreless reflects where they are located on the system. Or:

dpkg -L raspberrypi-sys-mods

One bigger chunk of things it does is the udev rules it adds, to grant GPIO and graphics interface access to specific system groups, and certain nodes at consistent paths via symlinks, like this one: https://github.com/RPi-Distro/raspberrypi-sys-mods/blob/pios/trixie/usr/lib/udev/rules.d/99-com.rules

Also much is related to first run setup, applying the pre-configuration done with the Raspberry Pi Imager.

I imagine it would be tedious to maintain a documentation for this package, given the large number of small things it does, which also change somewhat regularly, and between Debian versions. Browsing this repository, reading the commit texts, can answer most of the questions. But there is also quite some legacy stuff, and also newer things I do not understand either (and I am not exactly a newbe).

If you remove the package, expect a few things to break, and others to not behave as documented in the RPi docs anymore. E.g. if you use the serial console, it won't work anymore, since the /dev/serial* symlinks won't be created anymore. So you'd need to change the respective /boot/firmware/cmdline.txt entry to use the true device name, like /dev/ttyS0 or whatever the primary UART is, depending on the RPi model and in case config.txt settings. Similar with I2C and SPI nodes, if used, where missing permissions can come on top. Means: if you remove the package, before you create a bug report for something not working as expected, verify whether it is related to the missing package.

[pelwell]

The use of persistent logging was found to be a cause of significant SD card wear, leading to eventual failure. The raspi-config utility has an option to control logging - it's under Advanced Options/Logging.