test: prefer reading journalctl transport=syslog over /var/log/syslog#6896
test: prefer reading journalctl transport=syslog over /var/log/syslog#6896blackboxsw wants to merge 3 commits into
Conversation
Recent versions of journalctl and rsyslog avoid mirroring /dev/console messages to /var/log/syslog. Prefer to ask journalctl directly about _TRANSPORT=syslog type messages to track what cloud-init writes to console as this approach works in both legacy and current rsyslog/journald behavior.
blackboxsw
force-pushed
the
ci-read-syslog-from-journalctl
branch
from
e84aeae to
3e6a674
Compare
| # Prefer syslog transport categorized messages over presence of | ||
| # /var/log/syslog as recent versions of rsyslog will avoid mirroring | ||
| # /dev/console to syslog. | ||
| if client.execute(["which", "journalctl"]).ok: |
There was a problem hiding this comment.
Any distro running systemd should have journalctl. I don't think we need this check.
There was a problem hiding this comment.
I've dropped this which check as we only support systemd environments at the moment in integration tests. I also renamed get_syslog_or_console to get_journal_syslog and reduced complexity for handling minimal images which don't contain rsyslogd. I re-ran integration tests successfully on minimal and generic cloud images for tests/integration_tests/modules/test_ssh_auth_key_fingerprints.py and tests/integration_tests/modules/test_keys_to_console.py with success on both jammy (systemd v 249) and noble (systemd v 255).
| return get_console_log(client) | ||
| else: | ||
| # Prefer syslog transport categorized messages over presence of | ||
| # /var/log/syslog as recent versions of rsyslog will avoid mirroring |
There was a problem hiding this comment.
Is this caused by rsyslog behavior changes across versions or is this just a config change of default behavior? I would assume the latter.
There was a problem hiding this comment.
This actually is due to systemd v255 changes via systemd-executor process sandboxes which means the direct writes to /dev/console are not automatically mirrored as events to rsyslog any more. See system-executor details in v255 Changelog. It seems a side-effect of this changeset now only initially persists the /dev/console messages directly to the binary journal instead of also refecting those events to rsyslog for event handling. The option we have, which was introducted in systemd v205, is to strictly consume the systemd event tags that are populated for _TRANSPORT=syslog. This does get us a structured syslog-only set of logs emiteed by cloud-init pre and post systemd v255.
I'll update this comment pointing at systemd-executor v255 instead of journalctl/rsyslog specifically.
There was a problem hiding this comment.
Pull request overview
This PR updates integration tests to read syslog-relevant output from journalctl instead of /var/log/syslog, addressing SRU-era non-regression failures on newer Ubuntu images (e.g., noble) where traditional syslog file mirroring can be incomplete.
Changes:
- Replaced the
/var/log/syslog-based helper with a newget_journal_syslog()helper in integration test utilities. - Updated affected integration tests to use
get_journal_syslog()instead ofget_syslog_or_console(). - Simplified
tests/integration_tests/util.pyimports by removing the unusedOS_IMAGE_TYPEimport.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
tests/integration_tests/util.py |
Replaces syslog/console helper with a journalctl-based log retrieval helper. |
tests/integration_tests/modules/test_ssh_auth_key_fingerprints.py |
Switches fingerprint assertions to use the new journal-based helper. |
tests/integration_tests/modules/test_keys_to_console.py |
Switches key-console blacklist assertions to use the new journal-based helper. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def get_journal_syslog(client: "IntegrationInstance") -> str: | ||
| """Syslog events are categorized _TRANSPORT=syslog from systemd v205.""" | ||
| # Prefer syslog transport categorized messages over presence of | ||
| # /var/log/syslog as systemd v255 introduced systemd-executor | ||
| # which sandboxes unit processes resulting in direct writes to | ||
| # /dev/console being logged directly to journal binary instead | ||
| # of mirrored as rsyslog events. | ||
| return client.execute(["journalctl", "_TRANSPORT=syslog", "-b", "0"]) |
|
Resolved copilot review comments. |
Proposed Commit Message
Additional Context
Non-regression integration-test failure during 26.1 SRU testing. Affects 25.3 and earlier as well on noble and newer images.
Test Steps
Merge type