From 79332a165245357d7e7b6ae906822a052ba4b286 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sat, 9 May 2026 20:40:08 +0100 Subject: [PATCH 1/6] feat: S08.03 slice 5 BDD harness pointed at FreeRTOS-on-QEMU target Slice 5 of S08.03 (#304). Establishes the harness that lets BDD scenarios run against the FreeRTOS-on-QEMU target alongside the existing Linux/Windows runs. - target_driver.py module abstracts the example-spawn behind a BDD_TARGET env var; FreeRTOS dispatches to qemu-system-arm with the cross-built ELF, Linux/Windows keep the existing native subprocess flow. Same prompt-protocol over stdin/stdout (ExampleInteractive prints SolidSyslog> on every target) so wait_for_prompt / send_command are unchanged. - Per-target oracle pairs in ci/docker-compose.bdd.yml and .devcontainer/docker-compose.yml: syslog-ng-linux + behave-linux, syslog-ng-freertos + behave-freertos. Pairs never run together; network_mode: service:syslog-ng-freertos gives the FreeRTOS pair a shared netns so QEMU's slirp gateway 10.0.2.2 NATs to the pair's loopback. Both oracles alias as syslog-ng for backwards-compat. - bdd-freertos-qemu CI job depends on build-freertos-target's new ELF artifact upload; runs the freertos compose pair via --exit-code-from. Branch protection list and summary quality monitor updated. - Scenarios that don't pass on FreeRTOS today (hardcoded TEST_* hostname/PROCID/timestamp; no meta/origin/timeQuality SD wired; scenarios using cmdline args) tagged @freertoswip; the bdd-freertos CI command filters --tags='not @freertoswip and @udp'. Linux filter unchanged. Single passing scenario today is header_fields::App name matches the example program. - The cpputest-freertos-cross image bump (Python + behave) is prepared in CppUTestFreertosDocker but not yet pushed; the behave-freertos compose service inline-installs Python+behave until the image SHA advances. DEVLOG entry calls out the follow-up bump PR. Local verification: - behave-linux: 21 features / 46 scenarios pass, 0 failed. - behave-freertos: 1 feature / 1 scenario pass, 0 failed, 45 @freertoswip skipped. Closes #304. --- .devcontainer/docker-compose.yml | 63 +++++++++++-- .github/workflows/ci.yml | 69 +++++++++++++- Bdd/README.md | 96 +++++++++++++++++++ Bdd/features/buffered.feature | 1 + Bdd/features/environment.py | 21 +++-- Bdd/features/header_fields.feature | 2 + Bdd/features/message_fields.feature | 1 + Bdd/features/origin.feature | 1 + Bdd/features/prival.feature | 1 + Bdd/features/steps/syslog_steps.py | 26 ++---- Bdd/features/steps/target_driver.py | 111 ++++++++++++++++++++++ Bdd/features/structured_data.feature | 1 + Bdd/features/syslog.feature | 1 + Bdd/features/time_quality.feature | 1 + Bdd/features/timestamp.feature | 1 + Bdd/features/udp_mtu.feature | 1 + CLAUDE.md | 2 +- DEVLOG.md | 132 +++++++++++++++++++++++++++ ci/docker-compose.bdd.yml | 91 ++++++++++++++++-- docs/containers.md | 29 ++++-- 20 files changed, 600 insertions(+), 51 deletions(-) create mode 100644 Bdd/README.md create mode 100644 Bdd/features/steps/target_driver.py diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index cd0ab773..92e66c7a 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -17,15 +17,19 @@ services: working_dir: /workspaces/SolidSyslog command: sleep infinity depends_on: - - syslog-ng + - syslog-ng-linux - syslog-ng: + # Linux pair — syslog-ng oracle + behave runner, used by gcc/clang dev + # work and the bdd-linux-syslog-ng CI job. Each BDD target gets its + # own oracle + runner pair so they don't interfere; see + # ci/docker-compose.bdd.yml for the rationale. + syslog-ng-linux: image: balabit/syslog-ng:latest volumes: - ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf - ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro - ../Bdd/output:/var/log/syslog-ng - - syslog-ng-ctl:/var/lib/syslog-ng + - syslog-ng-ctl-linux:/var/lib/syslog-ng ports: - "5514:5514/udp" - "5514:5514/tcp" @@ -51,22 +55,27 @@ services: sleep 0.1 done ) & wait $$pid + networks: + default: + aliases: + - syslog-ng - behave: + behave-linux: image: ghcr.io/davidcozens/behave:sha-3faff14 volumes: - ..:/workspaces/SolidSyslog:cached - ../Bdd/output:/workspaces/SolidSyslog/Bdd/output - - syslog-ng-ctl:/var/lib/syslog-ng + - syslog-ng-ctl-linux:/var/lib/syslog-ng - ${SSH_AUTH_SOCK:-/dev/null}:/ssh-agent - ${HOME:-/tmp}/.claude:/home/developer/.claude - ${HOME:-/tmp}/.config/gh:/home/developer/.config/gh environment: - SSH_AUTH_SOCK=/ssh-agent + - BDD_TARGET=linux user: developer working_dir: /workspaces/SolidSyslog depends_on: - - syslog-ng + - syslog-ng-linux command: sleep infinity clang: @@ -105,10 +114,44 @@ services: working_dir: /workspaces/SolidSyslog command: sleep infinity + # FreeRTOS pair — syslog-ng oracle + the cross devcontainer that hosts + # both QEMU and Behave. freertos-target shares syslog-ng-freertos's + # network namespace so QEMU's slirp gateway 10.0.2.2 NATs to the pair's + # loopback, where syslog-ng-freertos is listening on 0.0.0.0:5514. + syslog-ng-freertos: + image: balabit/syslog-ng:latest + volumes: + - ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf + - ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro + - ../Bdd/output:/var/log/syslog-ng + - syslog-ng-ctl-freertos:/var/lib/syslog-ng + working_dir: /workspaces/SolidSyslog + entrypoint: ["bash", "-c"] + command: + - | + syslog-ng -F & + pid=$$! + deadline=$$((SECONDS + 30)) + while [ ! -S /var/lib/syslog-ng/syslog-ng.ctl ]; do + kill -0 "$$pid" 2>/dev/null || { echo "syslog-ng exited unexpectedly"; exit 1; } + [ "$$SECONDS" -ge "$$deadline" ] && { echo "Timed out waiting for syslog-ng control socket"; kill "$$pid" 2>/dev/null; exit 1; } + sleep 0.1 + done + ( while kill -0 "$$pid" 2>/dev/null; do + chmod 0777 /var/lib/syslog-ng/syslog-ng.ctl 2>/dev/null + sleep 0.1 + done ) & + wait $$pid + networks: + default: + aliases: + - syslog-ng + freertos-target: image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae volumes: - ..:/workspaces/SolidSyslog:cached + - syslog-ng-ctl-freertos:/var/lib/syslog-ng - ${SSH_AUTH_SOCK:-/dev/null}:/ssh-agent - ${HOME:-/tmp}/.claude:/home/developer/.claude - ${HOME:-/tmp}/.config/gh:/home/developer/.config/gh @@ -116,12 +159,18 @@ services: - SYS_PTRACE security_opt: - seccomp:unconfined + network_mode: service:syslog-ng-freertos environment: - SSH_AUTH_SOCK=/ssh-agent - BUILD_PRESET=freertos-cross + - BDD_TARGET=freertos + - EXAMPLE_BINARY=build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf user: developer working_dir: /workspaces/SolidSyslog + depends_on: + - syslog-ng-freertos command: sleep infinity volumes: - syslog-ng-ctl: + syslog-ng-ctl-linux: + syslog-ng-ctl-freertos: diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7d4415b2..d7a8a196 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -630,7 +630,8 @@ jobs: - name: Run BDD tests run: > docker compose -f ci/docker-compose.bdd.yml - up --build --abort-on-container-exit --exit-code-from behave + up --abort-on-container-exit --exit-code-from behave-linux + behave-linux syslog-ng-linux - name: BDD Test Report uses: dorny/test-reporter@v3 @@ -696,8 +697,8 @@ jobs: - name: Configure (ARM cross) run: cmake --preset freertos-cross - - name: Cross-build hello-world ELF - run: cmake --build --preset freertos-cross --target SolidSyslogFreeRtosHelloWorld + - name: Cross-build ELFs + run: cmake --build --preset freertos-cross --target SolidSyslogFreeRtosHelloWorld SolidSyslogFreeRtosSingleTask # QEMU smoke: run the ELF for a few seconds, capture stdout, assert the # expected greeting. timeout returns 124 when it kills QEMU, which is @@ -734,9 +735,64 @@ jobs: [ "$RC" -eq 124 ] || { echo "Expected timeout exit 124, got $RC"; exit 1; } echo "$OUT" | grep -q "hello from FreeRTOS on QEMU mps2-an385" + - name: Upload SingleTask ELF + uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0 + with: + name: solid-syslog-freertos-singletask + path: build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf + retention-days: 1 + compression-level: 0 + + bdd-freertos-qemu: + needs: build-freertos-target + runs-on: ubuntu-latest + permissions: + contents: read + checks: write + + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + + - uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0 + with: + name: solid-syslog-freertos-singletask + path: build/freertos-cross/Example/FreeRtos/SingleTask/ + + - name: Make ELF readable + run: chmod +r build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf + + - name: Prepare BDD output directory + run: mkdir -p Bdd/junit && chmod 777 Bdd/junit + + - name: Run BDD tests + run: > + docker compose -f ci/docker-compose.bdd.yml + up --abort-on-container-exit --exit-code-from behave-freertos + behave-freertos syslog-ng-freertos + + - name: BDD Test Report (FreeRTOS) + uses: dorny/test-reporter@v3 + if: success() || failure() + with: + name: Test Results (BDD FreeRTOS) + path: Bdd/junit/TESTS-*.xml + reporter: java-junit + + - name: Upload JUnit XML + if: success() || failure() + uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0 + with: + name: junit-bdd-freertos-qemu + path: Bdd/junit/TESTS-*.xml + retention-days: 1 + + - name: Compose logs on failure + if: failure() + run: docker compose -f ci/docker-compose.bdd.yml logs --no-color + summary: if: always() && github.event_name == 'pull_request' - needs: [build-linux-gcc, build-linux-clang, sanitize-linux-gcc, coverage-linux-gcc, analyze-tidy, analyze-cppcheck, analyze-format, analyze-iwyu, bdd-linux-syslog-ng, build-windows-msvc, bdd-windows-otel, integration-linux-openssl, integration-windows-openssl, build-freertos-host-tdd, build-freertos-target] + needs: [build-linux-gcc, build-linux-clang, sanitize-linux-gcc, coverage-linux-gcc, analyze-tidy, analyze-cppcheck, analyze-format, analyze-iwyu, bdd-linux-syslog-ng, build-windows-msvc, bdd-windows-otel, integration-linux-openssl, integration-windows-openssl, build-freertos-host-tdd, build-freertos-target, bdd-freertos-qemu] runs-on: ubuntu-latest permissions: contents: read @@ -819,6 +875,11 @@ jobs: "name": "bdd-windows-otel", "pattern": "**/junit-bdd-windows-otel/TESTS-*.xml" }, + { + "id": "junit", + "name": "bdd-freertos-qemu", + "pattern": "**/junit-bdd-freertos-qemu/TESTS-*.xml" + }, { "id": "junit", "name": "build-windows-msvc", diff --git a/Bdd/README.md b/Bdd/README.md new file mode 100644 index 00000000..1ba276e6 --- /dev/null +++ b/Bdd/README.md @@ -0,0 +1,96 @@ +# BDD scenarios + +Gherkin features under `features/` are driven by [Behave](https://behave.readthedocs.io/) +against a `syslog-ng` oracle (Linux + FreeRTOS) or `otelcol-contrib` (Windows). The +same feature files run on every target — the active runner is selected by the +`BDD_TARGET` environment variable, which `features/environment.py` reads in +`before_all` and dispatches via `features/steps/target_driver.py`. + +| `BDD_TARGET` | Example-under-test | Oracle | +|---|---|---| +| `linux` (default) | `build/debug/Example/SolidSyslogExample` (native subprocess) | `syslog-ng-linux` | +| `windows` | `build/msvc-debug/Example/Debug/SolidSyslogExample.exe` (native subprocess) | `otelcol-contrib` | +| `freertos` | `build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf` driven through `qemu-system-arm` | `syslog-ng-freertos` | + +Each BDD target pairs with its own oracle service so jobs (and developers +switching containers) never interfere — see [`docs/containers.md`](../docs/containers.md) +for the compose layout. + +## Tags + +| Tag | Meaning | +|---|---| +| `@udp` / `@tcp` / `@tls` / `@mtls` | Transport-specific scenario; CI jobs filter by these. | +| `@windows_wip` | Skipped on the Windows runner (typically OS-specific behaviour the OTel oracle doesn't model). | +| `@freertoswip` | Skipped on the FreeRTOS-on-QEMU runner. The FreeRTOS SingleTask example bakes hard-coded `TEST_*` values for hostname, PROCID, and timestamp; consumes config via the interactive `set NAME VALUE` command rather than getopt; and does not yet wire `meta` / `origin` / `timeQuality` SD. Scenarios that depend on any of these are tagged today and the tag is removed slice-by-slice as each gap is closed. | +| `@wip` | Globally skipped on every runner. | + +## Running locally + +### Linux (host development) + +From inside the `gcc` devcontainer: + +```bash +behave Bdd/features/syslog.feature # one feature +behave --tags='not @wip' Bdd/features/ # full Linux suite +``` + +The example binary and oracle path default to the values +`environment.before_all` sets for `BDD_TARGET=linux`; no env vars needed. + +### FreeRTOS-on-QEMU + +Switch the devcontainer service to `freertos-target` (see +[`docs/containers.md`](../docs/containers.md) — switch the `service:` line in +`.devcontainer/devcontainer.json` and rebuild). After rebuild, build the ELF +once and run Behave from inside the container: + +```bash +cmake --preset freertos-cross +cmake --build --preset freertos-cross --target SolidSyslogFreeRtosSingleTask +behave --tags='not @wip and not @freertoswip and @udp' Bdd/features/ +``` + +`BDD_TARGET=freertos` and `EXAMPLE_BINARY=build/freertos-cross/...` are +preset on the service so Behave dispatches to the QEMU driver +automatically. The driver spawns `qemu-system-arm -M mps2-an385 -serial +stdio …` and pipes `set` / `send` / `quit` over the UART; the QEMU +guest's slirp gateway `10.0.2.2` reaches the paired `syslog-ng-freertos` +oracle on the shared loopback. + +#### Fault-finding tips + +- **No prompt within 30 seconds**: the `SolidSyslog>` prompt is printed by + the FreeRTOS interactive task, which the IP-network event hook only + spawns once `eNetworkUp` fires. Add `-d guest_errors` to the QEMU args + (in `target_driver.py::_QEMU_BASE_ARGS`) to surface CPU exceptions, or + detach the same QEMU command from Behave and drive it manually: + ```bash + qemu-system-arm -M mps2-an385 -m 16M -display none -serial stdio \ + -icount shift=auto,sleep=off,align=off \ + -netdev user,id=net0 -net nic,netdev=net0,model=lan9118 \ + -kernel build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf + ``` +- **Oracle is silent**: confirm the syslog-ng-freertos service is + healthy (`docker compose ps`), and that the host you're testing + resolves under the shared netns. Slice 3b.1.5's ARP-priming work + means the very first datagram is delivered, but a stale slirp NAT + table on a long-running compose network has been observed to + swallow packets — `docker compose down -v` then up. +- **Want a quick frame inspection**: `cat Bdd/output/received.log` + inside the freertos pair gives the parsed frame in + `key=value` form (the syslog-ng template in + `Bdd/syslog-ng/syslog-ng.conf`). + +## Running in CI + +| Job | Compose pair | +|---|---| +| `bdd-linux-syslog-ng` | `syslog-ng-linux` + `behave-linux` | +| `bdd-freertos-qemu` | `syslog-ng-freertos` + `behave-freertos` (cross image w/ QEMU + Behave) | +| `bdd-windows-otel` | runner-direct (no compose); spawns `otelcol-contrib.exe` directly | + +`bdd-freertos-qemu` depends on `build-freertos-target` to upload the +SingleTask ELF as an artifact; the new job downloads it before +`docker compose up`. diff --git a/Bdd/features/buffered.feature b/Bdd/features/buffered.feature index 4c02386c..ff183d33 100644 --- a/Bdd/features/buffered.feature +++ b/Bdd/features/buffered.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: Buffered message delivery An example wired with a real (non-Null) buffer drives messages through a service thread that drains the buffer and sends to the diff --git a/Bdd/features/environment.py b/Bdd/features/environment.py index 17cd2242..7cc65423 100644 --- a/Bdd/features/environment.py +++ b/Bdd/features/environment.py @@ -92,12 +92,21 @@ def otel_start_oracle(): def before_all(context): - # Configurable via env so the same step code drives Linux (syslog-ng) and - # Windows (OTel Collector) runners with different binary paths and oracle - # output formats. - context.example_binary = os.environ.get( - "EXAMPLE_BINARY", "build/debug/Example/SolidSyslogExample" - ) + # Configurable via env so the same step code drives Linux (syslog-ng), + # Windows (OTel Collector), and FreeRTOS-on-QEMU runners with + # different binary paths and oracle output formats. BDD_TARGET picks + # how the example is spawned (native subprocess vs qemu-system-arm) + # — see Bdd/features/steps/target_driver.py. + context.target = os.environ.get("BDD_TARGET", "linux") + + if context.target == "freertos": + default_binary = ( + "build/freertos-cross/Example/FreeRtos/SingleTask/" + "SolidSyslogFreeRtosSingleTask.elf" + ) + else: + default_binary = "build/debug/Example/SolidSyslogExample" + context.example_binary = os.environ.get("EXAMPLE_BINARY", default_binary) context.received_log = os.environ.get( "RECEIVED_LOG", "Bdd/output/received.log" ) diff --git a/Bdd/features/header_fields.feature b/Bdd/features/header_fields.feature index ad7b2b56..b84410bc 100644 --- a/Bdd/features/header_fields.feature +++ b/Bdd/features/header_fields.feature @@ -3,6 +3,7 @@ Feature: Message header fields The library includes hostname, app-name, and process ID in the RFC 5424 message header. + @freertoswip Scenario: Hostname matches the system hostname Given the syslog oracle is running When the example program sends a syslog message @@ -13,6 +14,7 @@ Feature: Message header fields When the example program sends a syslog message Then the app name is "SolidSyslogExample" + @freertoswip Scenario: Process ID matches the example program PID Given the syslog oracle is running When the example program sends a syslog message diff --git a/Bdd/features/message_fields.feature b/Bdd/features/message_fields.feature index acbd6b9a..5d9c22c0 100644 --- a/Bdd/features/message_fields.feature +++ b/Bdd/features/message_fields.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: Message ID and message body The library includes message ID and message body in the RFC 5424 message. diff --git a/Bdd/features/origin.feature b/Bdd/features/origin.feature index 7b3b54c2..ca8623fb 100644 --- a/Bdd/features/origin.feature +++ b/Bdd/features/origin.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: Structured data — origin The library includes origin metadata identifying the software component. diff --git a/Bdd/features/prival.feature b/Bdd/features/prival.feature index f53d994d..b9e61d46 100644 --- a/Bdd/features/prival.feature +++ b/Bdd/features/prival.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: PRIVAL encoding The library calculates RFC 5424 PRIVAL from facility and severity. The example program accepts --facility and --severity flags, diff --git a/Bdd/features/steps/syslog_steps.py b/Bdd/features/steps/syslog_steps.py index e8040b0c..0f60aaba 100644 --- a/Bdd/features/steps/syslog_steps.py +++ b/Bdd/features/steps/syslog_steps.py @@ -22,6 +22,7 @@ otel_kill_oracle, otel_start_oracle, ) +from target_driver import spawn_example_process, stop_example_process PER_TRANSPORT_LOG_SYSLOG_NG = { "udp": RECEIVED_UDP_LOG, @@ -399,17 +400,7 @@ def _run_with_prompt_protocol(context, binary, label, extra_args, expected_messa f"{label} binary not found at {binary} — build with cmake first" ) - cmd = [os.path.abspath(binary)] - if extra_args: - cmd.extend(extra_args) - - process = subprocess.Popen( - cmd, - stdin=subprocess.PIPE, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - ) + process = spawn_example_process(context, extra_args=extra_args, binary=binary) context.example_pid = process.pid try: @@ -417,11 +408,14 @@ def _run_with_prompt_protocol(context, binary, label, extra_args, expected_messa send_command(process, f"send {expected_messages}") wait_for_messages(context, expected_messages) - process.stdin.write("quit\n") - process.stdin.flush() - process.wait(timeout=10) - assert process.returncode == 0, ( - f"{label} failed with exit code {process.returncode}" + returncode = stop_example_process(process, context.target) + # FreeRTOS keeps the QEMU VM alive after `quit` (the scheduler + # idles so a GDB attach works), so stop_example_process kills + # QEMU and returns None — no exit code to assert. Linux/Windows + # binaries exit cleanly on `quit` and their return code is + # meaningful. + assert returncode in (0, None), ( + f"{label} failed with exit code {returncode}" ) finally: # Don't let an intermediate exception leak the helper into later diff --git a/Bdd/features/steps/target_driver.py b/Bdd/features/steps/target_driver.py new file mode 100644 index 00000000..529045d3 --- /dev/null +++ b/Bdd/features/steps/target_driver.py @@ -0,0 +1,111 @@ +"""Target-driver abstraction for spawning the example-under-test. + +The same Behave feature files run on multiple target platforms — Linux, +Windows, FreeRTOS-on-QEMU. The platforms differ only in how the +example-under-test is spawned: a native subprocess on Linux/Windows, or +qemu-system-arm with the cross-built ELF on FreeRTOS. Both backends +expose the same `SolidSyslog> ` prompt protocol over stdin/stdout +(printf-to-UART for FreeRTOS, printf-to-pipe for the native binaries), +so the existing wait_for_prompt / send_command helpers in +syslog_steps.py work unchanged once the spawn returns a Popen-like +object. + +The active target is selected by the BDD_TARGET environment variable, +read in environment.before_all and stashed on context.target. Steps +call spawn_example_process(context, extra_args=...) instead of +subprocess.Popen directly. +""" + +import os +import subprocess + + +# QEMU machine + UART config matches build-freertos-target's smoke run +# in .github/workflows/ci.yml: mps2-an385 / Cortex-M3 / 16 MiB / LAN9118 +# NIC / -serial stdio for the polled CMSDK UART. -icount keeps virtual +# time decoupled from host load so timing-sensitive scenarios are +# deterministic under CI noise. +_QEMU_BASE_ARGS = [ + "qemu-system-arm", + "-M", "mps2-an385", + "-m", "16M", + "-display", "none", + "-serial", "stdio", + "-icount", "shift=auto,sleep=off,align=off", + "-netdev", "user,id=net0", + "-net", "nic,netdev=net0,model=lan9118", +] + + +def spawn_example_process(context, extra_args=None, binary=None): + """Spawn the example-under-test for the active target. + + Returns a subprocess.Popen with text-mode stdin/stdout/stderr pipes + so the caller can use the existing prompt-protocol helpers + (wait_for_prompt / send_command) without modification. + + extra_args are appended to the binary's command line on Linux and + Windows (where the example-runner parses argv via getopt). FreeRTOS + has no getopt port and consumes its config via interactive `set` + commands instead; passing extra_args on FreeRTOS therefore raises — + such scenarios should be tagged @freertoswip until a follow-up + slice teaches the FreeRTOS driver to translate cmdline flags into + the equivalent set commands. + + binary defaults to context.example_binary (the standard SingleTask + binary or its FreeRTOS .elf equivalent); callers that drive a + different example (e.g. the Linux Threaded binary) pass it + explicitly. The FreeRTOS path only ever sees the SingleTask .elf + today — threaded scenarios are tagged @freertoswip until a + FreeRTOS threaded example exists. + """ + target = getattr(context, "target", "linux") + if binary is None: + binary = context.example_binary + + if target == "freertos": + if extra_args: + raise NotImplementedError( + "FreeRTOS target does not accept argv; tag the scenario " + "@freertoswip until the driver translates flags into " + "interactive `set` commands. Got: " + repr(extra_args) + ) + cmd = list(_QEMU_BASE_ARGS) + ["-kernel", os.path.abspath(binary)] + else: + cmd = [os.path.abspath(binary)] + if extra_args: + cmd.extend(extra_args) + + return subprocess.Popen( + cmd, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + + +def stop_example_process(process, target, timeout=10): + """Tear down the example process for the active target. + + Linux/Windows: the example exits cleanly on `quit`, so wait for the + return code. FreeRTOS: `quit` deletes the interactive task but the + QEMU VM keeps idling (the scheduler stays alive so a GDB attach + works), so kill QEMU after a short grace period — the BDD scenario + has already verified the oracle received the frame, the QEMU exit + code carries no useful signal. + """ + if target == "freertos": + try: + process.stdin.write("quit\n") + process.stdin.flush() + except (BrokenPipeError, OSError): + pass + process.kill() + process.wait(timeout=timeout) + return None + + process.stdin.write("quit\n") + process.stdin.flush() + process.wait(timeout=timeout) + return process.returncode diff --git a/Bdd/features/structured_data.feature b/Bdd/features/structured_data.feature index d4c21f04..2692d4a5 100644 --- a/Bdd/features/structured_data.feature +++ b/Bdd/features/structured_data.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: Structured data — meta SD-ELEMENT The library populates the IANA-registered "meta" SD-ELEMENT with sequenceId, sysUpTime, and language per RFC 5424 §7.3. diff --git a/Bdd/features/syslog.feature b/Bdd/features/syslog.feature index ab11b332..9646fcbb 100644 --- a/Bdd/features/syslog.feature +++ b/Bdd/features/syslog.feature @@ -3,6 +3,7 @@ Feature: Walking skeleton end-to-end The example program sends an RFC 5424 message via UDP. syslog-ng receives it and writes the parsed fields to a log file. + @freertoswip Scenario: SolidSyslog sends a valid RFC 5424 message to syslog-ng Given the syslog oracle is running When the example program sends a syslog message diff --git a/Bdd/features/time_quality.feature b/Bdd/features/time_quality.feature index fa493073..6d34fb88 100644 --- a/Bdd/features/time_quality.feature +++ b/Bdd/features/time_quality.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: Structured data — time quality The library includes time quality metadata in structured data. diff --git a/Bdd/features/timestamp.feature b/Bdd/features/timestamp.feature index 130e4bb7..4e8f8d0e 100644 --- a/Bdd/features/timestamp.feature +++ b/Bdd/features/timestamp.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: Timestamp encoding The library captures the timestamp at raise-time via an injected clock and formats it as RFC 5424 FULL-DATE "T" FULL-TIME. diff --git a/Bdd/features/udp_mtu.feature b/Bdd/features/udp_mtu.feature index 3004c1b7..75454c39 100644 --- a/Bdd/features/udp_mtu.feature +++ b/Bdd/features/udp_mtu.feature @@ -1,4 +1,5 @@ @udp +@freertoswip Feature: UDP datagram path-MTU clipping When a UDP message exceeds the path MTU the sender clips it to the path-MTU's safe payload, walking back over any partial UTF-8 diff --git a/CLAUDE.md b/CLAUDE.md index 80bf14e7..eab08478 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -25,7 +25,7 @@ becomes the single commit message — so the PR title must follow Conventional C **Branch protection rules (configured on GitHub):** - Direct pushes to `main` are blocked -- PRs require all status checks to pass before merging: build-linux-gcc, build-linux-clang, build-windows-msvc, sanitize-linux-gcc, coverage-linux-gcc, analyze-tidy, analyze-cppcheck, analyze-format, analyze-iwyu, integration-linux-openssl, integration-windows-openssl, bdd-linux-syslog-ng, bdd-windows-otel, build-freertos-host-tdd, build-freertos-target, summary +- PRs require all status checks to pass before merging: build-linux-gcc, build-linux-clang, build-windows-msvc, sanitize-linux-gcc, coverage-linux-gcc, analyze-tidy, analyze-cppcheck, analyze-format, analyze-iwyu, integration-linux-openssl, integration-windows-openssl, bdd-linux-syslog-ng, bdd-windows-otel, bdd-freertos-qemu, build-freertos-host-tdd, build-freertos-target, summary - Squash merge only — other merge strategies are disabled - Branches are deleted automatically after merge diff --git a/DEVLOG.md b/DEVLOG.md index 9c52c51e..5749cdd1 100644 --- a/DEVLOG.md +++ b/DEVLOG.md @@ -1,5 +1,137 @@ # Dev Log +## 2026-05-09 — S08.03 slice 5 — BDD harness pointed at FreeRTOS-on-QEMU target (#304) + +Slice 4 made the FreeRTOS example's identity, transport endpoint, and PRIVAL +fields mutable in-RAM via `set NAME VALUE` over the existing UART command +channel. The slice-4 smoke run was a hand-rolled Behave-equivalent (Python +listener + QEMU subprocess + UART heredoc); slice 5 promotes that flow into +real Behave + the existing syslog-ng oracle, and adds the CI matrix entry. + +### Decisions + +- **Behave runs alongside QEMU in `cpputest-freertos-cross`.** Spawning QEMU + as a subprocess and piping stdin/stdout to its UART (via `-serial stdio`) + is only natural inside the QEMU process's parent — putting Behave in a + different container would mean cross-container PTY plumbing. The cross + image was the cheapest place to add Python + Behave (one image bump, no + new image, no apt-installs in CI). Bump prepared in CppUTestFreertosDocker + (`dockerfile.cross` adds `python3 + python3-pip + behave==1.3.3`); a + follow-up PR pushes it and bumps SolidSyslog's SHA refs. Until then the + compose service does an inline `apt-get install python3-pip; pip install` + which is documented to be removed once the bump lands. +- **Per-target oracle pairs.** `ci/docker-compose.bdd.yml` and + `.devcontainer/docker-compose.yml` rename `syslog-ng` → `syslog-ng-linux` + / `behave` → `behave-linux` and add `syslog-ng-freertos` + the + `behave-freertos` runner. Each pair gets its own `syslog-ng` instance + with its own ctl socket and output volume; pairs never run together + (CI scopes services per job; devcontainer's `depends_on` only starts + the active pair). Adding a future target is one more block in the same + shape. +- **Shared netns over slirp NAT.** `behave-freertos` uses + `network_mode: service:syslog-ng-freertos`, so QEMU's slirp gateway + `10.0.2.2` NATs to the pair's loopback where syslog-ng listens on + `0.0.0.0:5514`. No port forwarding to the runner / host, no cross- + container DNS for the data path. +- **DNS alias for backwards compat.** Both `syslog-ng-` services + alias as the bare `syslog-ng` on their network. The Linux example wiring + (`Example/Common/Example*Config.c::host = "syslog-ng"`) and the BDD step + helpers (`wait_for_tcp_port_open(host="syslog-ng")`) keep resolving + unchanged. Pairs never co-exist, so the "two services aliasing the same + name" pattern is benign. +- **`target_driver.py` (a single Python module) abstracts the spawn.** + Same prompt protocol (`SolidSyslog>` over stdin/stdout — already printed + by `Example/Common/ExampleInteractive.c` on every target) means + `wait_for_prompt` / `send_command` work unchanged; only the spawn and + the teardown differ. `spawn_example_process(context, extra_args, binary)` + branches on `context.target` (set from `BDD_TARGET` in `before_all`); + `stop_example_process` returns `None` on FreeRTOS (kills QEMU after + `quit` because the scheduler keeps idling) and the example's exit code + on Linux/Windows. +- **`extra_args` is unsupported on FreeRTOS today.** The FreeRTOS example + has no getopt port; cmdline flags are meaningless to it. Calling + `spawn_example_process` with `extra_args` while `BDD_TARGET=freertos` + raises `NotImplementedError` so a misconfigured scenario fails loudly + rather than silently hitting QEMU's argv-parsing path. Translating + cmdline → `set NAME VALUE` is a follow-up; for slice 5, scenarios that + use cmdline args are tagged `@freertoswip`. +- **`@freertoswip` tags scenarios that don't pass on FreeRTOS yet.** + Behaves as a target-specific `@wip`. The `bdd-freertos-qemu` CI job runs + `--tags='not @wip and not @freertoswip and @udp'`; Linux runs + `--tags='not @wip'` so existing Linux behaviour is unchanged. Tagged + features today and the reason each one is tagged: + | Feature | Why | + |---|---| + | `syslog.feature` (single scenario) | Composite assertion includes "system hostname", "current timestamp", and "PID of example program" — FreeRTOS uses the literal `FreeRtosExample` hostname, the RFC 5424 publication-date placeholder, and PROCID `1`. | + | `header_fields.feature` (hostname, PID scenarios) | Same hardcoded values. App-name scenario passes (untagged) because the FreeRTOS example sets app name to `SolidSyslogExample`, matching. | + | `timestamp.feature` | Hardcoded TEST timestamp (RFC 5424 publication date 2009-03-23). | + | `structured_data.feature` | FreeRTOS example wires no `meta` SD (no sequenceId, sysUpTime, language). | + | `origin.feature` | FreeRTOS example wires no origin SD. | + | `time_quality.feature` | FreeRTOS example wires no timeQuality SD. | + | `prival.feature` | Uses `--facility` / `--severity` cmdline args. | + | `message_fields.feature` | Uses `--message-id` / `--message` cmdline args. | + | `udp_mtu.feature` | Uses `--message` cmdline args with very long bodies. | + | `buffered.feature` | Drives the Linux Threaded binary (no FreeRTOS equivalent yet). | +- **Walking-skeleton acceptance is "the harness lands, with at least one + scenario passing."** That scenario is `header_fields.feature::App name + matches the example program`. Every other current `@udp` scenario is + tagged `@freertoswip`. Untagging them is the work of follow-up slices — + some will come for free as soon as `Example/FreeRtos/SingleTask/main.c` + wires SD or accepts a real clock callback; others need the + cmdline-flag → `set` translation in the FreeRTOS driver. +- **CI structurally mirrors `bdd-linux-syslog-ng`.** `build-freertos-target` + gains an artifact upload step for `SolidSyslogFreeRtosSingleTask.elf`; + the new `bdd-freertos-qemu` job downloads it and runs the freertos + compose pair. The summary quality monitor surfaces FreeRTOS BDD JUnit + alongside the existing two BDD jobs. Branch protection list updated in + CLAUDE.md. + +### Local verification + +- `behave-linux` against `syslog-ng-linux`: 21 features, 46 scenarios + passing, 0 failed, 0 skipped — pre-rename behaviour preserved. +- `behave-freertos` against `syslog-ng-freertos`: 1 feature, 1 scenario + passing, 0 failed, 45 `@freertoswip` skipped. + +### Deferred + +- **Image bump in CppUTestFreertosDocker.** Prepared locally; David to + push when convinced. Once the new SHA is published, bump the SHA refs + in `.devcontainer/docker-compose.yml`, `.github/workflows/ci.yml`, + `ci/docker-compose.bdd.yml`, `docs/containers.md` (single follow-up + PR, `chore: bump container image`-style), and drop the inline + `apt-get / pip install` from the `behave-freertos` compose command. +- **Removing `@freertoswip` tags.** Each tagged feature/scenario is a + follow-up slice. Likely order: structured-data wiring (origin, + time_quality, sequenceId — needs a small example-side change to wire + the SDs), then real RTC callback (timestamp), then cmdline-flag → `set` + translation in the driver (covers prival, message_fields, udp_mtu), + then a FreeRTOS Threaded example for `buffered.feature`. +- **Behave runs both pairs simultaneously locally.** Today only one + devcontainer service is active at a time, so only one pair starts. If + a future workflow needs both pairs concurrently in the same compose + project, the `syslog-ng` DNS alias collides — switch to per-target + hostname overrides via env var (the `SOLIDSYSLOG_BDD_*_HOST` pattern + is already in place for Windows) and remove the alias. +- **Refactoring `bdd-windows-otel` to use the new `target_driver` + abstraction.** Windows still works through the older + `oracle_format`-branched code path. The two patterns subsume each + other; rolling Windows in is cosmetic and a follow-up cleanup. + +### Open questions + +- Should we surface the `@freertoswip` count somewhere visible (a + tag-count line in the summary job, a comment on PRs)? Today it's only + visible to a developer running `behave -v`; a CI-side surface would + make the "we're tagging slowly being removed" story legible at a + glance. Probably yes, separate small PR. +- Is shared-netns the right primitive vs. host networking long-term? + Shared netns isolates the BDD pair from the runner, which is nicer, + but it depends on a docker-compose feature that doesn't exist on + Windows hosts. If a future target needs to run on a Windows runner, + we'll revisit; for now Linux runners host both Linux and FreeRTOS BDD + jobs and the feature works. + ## 2026-05-09 — S08.03 slice 4 — `set` command for in-RAM configuration injection (#302) Slice 3b.2 hardcoded the FreeRTOS example's identity, message body, and diff --git a/ci/docker-compose.bdd.yml b/ci/docker-compose.bdd.yml index 08bbdffc..974ddbae 100644 --- a/ci/docker-compose.bdd.yml +++ b/ci/docker-compose.bdd.yml @@ -1,11 +1,11 @@ services: - syslog-ng: + syslog-ng-linux: image: balabit/syslog-ng:latest volumes: - ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf - ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro - - bdd-output:/var/log/syslog-ng - - syslog-ng-ctl:/var/lib/syslog-ng + - bdd-output-linux:/var/log/syslog-ng + - syslog-ng-ctl-linux:/var/lib/syslog-ng entrypoint: ["bash", "-c"] command: - | @@ -19,25 +19,98 @@ services: done chmod 0777 /var/lib/syslog-ng/syslog-ng.ctl wait + networks: + default: + aliases: + - syslog-ng healthcheck: test: ["CMD-SHELL", "test -S /var/lib/syslog-ng/syslog-ng.ctl"] interval: 2s timeout: 5s retries: 10 - behave: + behave-linux: image: ghcr.io/davidcozens/behave:sha-3faff14 user: "0" volumes: - ..:/workspaces/SolidSyslog - - bdd-output:/workspaces/SolidSyslog/Bdd/output - - syslog-ng-ctl:/var/lib/syslog-ng + - bdd-output-linux:/workspaces/SolidSyslog/Bdd/output + - syslog-ng-ctl-linux:/var/lib/syslog-ng working_dir: /workspaces/SolidSyslog depends_on: - syslog-ng: + syslog-ng-linux: condition: service_healthy + environment: + - BDD_TARGET=linux command: behave --junit --junit-directory Bdd/junit --tags='not @wip' Bdd/features/ + # Each BDD target pairs with its own syslog-ng instance so jobs don't + # interfere. The behave- service shares the syslog-ng- + # network namespace via network_mode so QEMU's slirp gateway 10.0.2.2 + # NATs to the pair's loopback, where syslog-ng- is listening on + # 0.0.0.0:5514. Adding more targets is then just another paired service + # block. + syslog-ng-freertos: + image: balabit/syslog-ng:latest + volumes: + - ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf + - ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro + - bdd-output-freertos:/var/log/syslog-ng + - syslog-ng-ctl-freertos:/var/lib/syslog-ng + entrypoint: ["bash", "-c"] + command: + - | + syslog-ng -F & + pid=$$! + deadline=$$((SECONDS + 30)) + while [ ! -S /var/lib/syslog-ng/syslog-ng.ctl ]; do + kill -0 "$$pid" 2>/dev/null || { echo "syslog-ng exited unexpectedly"; exit 1; } + [ "$$SECONDS" -ge "$$deadline" ] && { echo "Timed out waiting for syslog-ng control socket"; kill "$$pid" 2>/dev/null; exit 1; } + sleep 0.1 + done + chmod 0777 /var/lib/syslog-ng/syslog-ng.ctl + wait + networks: + default: + aliases: + - syslog-ng + healthcheck: + test: ["CMD-SHELL", "test -S /var/lib/syslog-ng/syslog-ng.ctl"] + interval: 2s + timeout: 5s + retries: 10 + + behave-freertos: + # Cross image carries qemu-system-arm + python3 + behave (post-image + # bump in CppUTestFreertosDocker). Until that bump lands, behave is + # pip-installed at runtime; remove the inline install once the image + # tag advances. + image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae + user: "0" + volumes: + - ..:/workspaces/SolidSyslog + - bdd-output-freertos:/workspaces/SolidSyslog/Bdd/output + - syslog-ng-ctl-freertos:/var/lib/syslog-ng + working_dir: /workspaces/SolidSyslog + network_mode: service:syslog-ng-freertos + depends_on: + syslog-ng-freertos: + condition: service_healthy + environment: + - BDD_TARGET=freertos + - EXAMPLE_BINARY=build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf + entrypoint: ["bash", "-c"] + command: + - | + # Temporary install — once the cpputest-freertos-cross image + # gains python3-pip + behave (CppUTestFreertosDocker bump), drop + # this line and the `apt-get install` block below. + apt-get update >/dev/null && apt-get install -y --no-install-recommends python3-pip >/dev/null + pip install --no-cache-dir --break-system-packages -r Bdd/requirements.txt + behave --junit --junit-directory Bdd/junit --tags='not @wip and not @freertoswip and @udp' Bdd/features/ + volumes: - bdd-output: - syslog-ng-ctl: + bdd-output-linux: + bdd-output-freertos: + syslog-ng-ctl-linux: + syslog-ng-ctl-freertos: diff --git a/docs/containers.md b/docs/containers.md index 20496f4a..32c4ac63 100644 --- a/docs/containers.md +++ b/docs/containers.md @@ -7,7 +7,7 @@ | `ghcr.io/davidcozens/cpputest` | `sha-18f19e1` | devcontainer (`gcc` service), most CI jobs | | `ghcr.io/davidcozens/cpputest-clang` | `sha-7eac3ab` | `clang` compose service, `build-linux-clang` CI job, `analyze-iwyu` CI job | | `ghcr.io/davidcozens/cpputest-freertos` | `sha-44efeae` | `freertos-host` compose service, `build-freertos-host-tdd` CI job — adds FreeRTOS-Kernel / Plus-TCP / Plus-FAT / Mbed TLS sources for host-TDD of FreeRTOS adapters against fakes | -| `ghcr.io/davidcozens/cpputest-freertos-cross` | `sha-44efeae` | `freertos-target` compose service, `build-freertos-target` CI job — adds `gcc-arm-none-eabi`, `libnewlib-arm-none-eabi`, `gdb-multiarch` (aliased as `arm-none-eabi-gdb`), `qemu-system-arm` for cross builds and on-QEMU runs | +| `ghcr.io/davidcozens/cpputest-freertos-cross` | `sha-44efeae` | `freertos-target` compose service, `build-freertos-target` CI job, `behave-freertos` BDD service, `bdd-freertos-qemu` CI job — adds `gcc-arm-none-eabi`, `libnewlib-arm-none-eabi`, `gdb-multiarch` (aliased as `arm-none-eabi-gdb`), `qemu-system-arm`, `python3` + `behave` for cross builds, on-QEMU runs, and BDD scenarios driving a QEMU target | | `balabit/syslog-ng` | `latest` | `syslog-ng` service — BDD test oracle | | `ghcr.io/davidcozens/behave` | `sha-3faff14` | `behave` service — Debian trixie + Python 3.12 + Behave for BDD scenarios | @@ -17,12 +17,25 @@ The devcontainer uses Docker Compose (`.devcontainer/docker-compose.yml`). VS Code connects to the `gcc` service (GCC). The `clang` service is on-demand only — it starts when you explicitly run a command against it and stops when done. -The `syslog-ng` and `behave` services support BDD testing. The `gcc` service depends on -`syslog-ng`, so it starts automatically with the devcontainer. The `behave` service is -on-demand — it runs when BDD scenarios are executed. See [BDD testing](bdd.md) for details. +BDD testing pairs each target with its own `syslog-ng` oracle so jobs running +in parallel (or developers switching containers) never interfere: -As cross-compilation targets are added, each gets its own service in the compose file, -following the same pattern. +| Target | Behave runner | Oracle | +|---|---|---| +| Linux | `behave-linux` | `syslog-ng-linux` | +| FreeRTOS | inside `freertos-target` (image carries both QEMU and Behave) | `syslog-ng-freertos` | + +The `gcc` service depends on `syslog-ng-linux` so it starts automatically with the +devcontainer; the `freertos-target` service depends on `syslog-ng-freertos` and shares +its network namespace via `network_mode: service:syslog-ng-freertos`, so QEMU's +slirp gateway `10.0.2.2` NATs to the pair's loopback where `syslog-ng-freertos` is +listening on `0.0.0.0:5514`. Both oracles also alias as the bare hostname `syslog-ng` +on their network so the existing example wiring (`Example/Common/Example*Config.c`, +the BDD step helpers) keeps resolving without per-target host overrides. The +pairs never run together, so the alias collision is academic. + +As more cross-compilation targets are added, each gets its own oracle pair in the +same shape (`syslog-ng-` + a runner service or in-container Behave). ## Running the clang build locally @@ -69,8 +82,8 @@ The available services and the build preset each one drives: | `gcc` | Primary C/C++ development (default) | `debug` | | `clang` | Clang-specific debugging / portability | `clang-debug` | | `freertos-host` | TDD of FreeRTOS adapters against host-side fakes | `debug` | -| `freertos-target` | Cross builds, on-QEMU runs, GDB attach (Cortex-M3, mps2-an385) | `freertos-cross` | -| `behave` | BDD scenario development (Python + Behave) | (none — cmake skipped) | +| `freertos-target` | Cross builds, on-QEMU runs, GDB attach (Cortex-M3, mps2-an385), BDD against the QEMU target | `freertos-cross` | +| `behave-linux` | Linux BDD scenario development (Python + Behave) | (none — cmake skipped) | To switch: From 6188890d4abf6cd89b194ab573795ac12e2876e5 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sat, 9 May 2026 20:57:14 +0100 Subject: [PATCH 2/6] fix: scope syslog-ng alias to per-pair bridge networks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit flagged that both syslog-ng-linux and syslog-ng-freertos sharing the bare `syslog-ng` alias on the default network gives docker's embedded DNS multiple A records when both pairs run together — behave-linux's wait_for_tcp_port_open(host="syslog-ng") and the example wiring's hardcoded "syslog-ng" host could non-deterministically hit either oracle. CI is currently safe (each job scopes services to its own pair) but a developer running plain `docker compose up` from the host, or a future matrix run, would trigger the collision. Each pair now lives on its own bridge network (linux / freertos); the `syslog-ng` alias is scoped to that network so the same name resolves to the right oracle from inside each pair, and pairs running together cannot see each other's services. Local smoke confirms no regressions: behave-linux 46/46 pass, behave-freertos 1/1 pass (45 @freertoswip skipped). --- .devcontainer/docker-compose.yml | 17 +++++++++++++---- ci/docker-compose.bdd.yml | 25 +++++++++++++++++-------- 2 files changed, 30 insertions(+), 12 deletions(-) diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 92e66c7a..9381aa76 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -15,13 +15,16 @@ services: - BUILD_PRESET=debug user: developer working_dir: /workspaces/SolidSyslog + networks: + - linux command: sleep infinity depends_on: - syslog-ng-linux # Linux pair — syslog-ng oracle + behave runner, used by gcc/clang dev - # work and the bdd-linux-syslog-ng CI job. Each BDD target gets its - # own oracle + runner pair so they don't interfere; see + # work and the bdd-linux-syslog-ng CI job. Each pair lives on its own + # bridge network so the `syslog-ng` DNS alias is scoped per-pair, even + # if a future workflow runs both pairs simultaneously; see # ci/docker-compose.bdd.yml for the rationale. syslog-ng-linux: image: balabit/syslog-ng:latest @@ -56,7 +59,7 @@ services: done ) & wait $$pid networks: - default: + linux: aliases: - syslog-ng @@ -74,6 +77,8 @@ services: - BDD_TARGET=linux user: developer working_dir: /workspaces/SolidSyslog + networks: + - linux depends_on: - syslog-ng-linux command: sleep infinity @@ -143,7 +148,7 @@ services: done ) & wait $$pid networks: - default: + freertos: aliases: - syslog-ng @@ -174,3 +179,7 @@ services: volumes: syslog-ng-ctl-linux: syslog-ng-ctl-freertos: + +networks: + linux: + freertos: diff --git a/ci/docker-compose.bdd.yml b/ci/docker-compose.bdd.yml index 974ddbae..36612650 100644 --- a/ci/docker-compose.bdd.yml +++ b/ci/docker-compose.bdd.yml @@ -20,7 +20,7 @@ services: chmod 0777 /var/lib/syslog-ng/syslog-ng.ctl wait networks: - default: + linux: aliases: - syslog-ng healthcheck: @@ -37,6 +37,8 @@ services: - bdd-output-linux:/workspaces/SolidSyslog/Bdd/output - syslog-ng-ctl-linux:/var/lib/syslog-ng working_dir: /workspaces/SolidSyslog + networks: + - linux depends_on: syslog-ng-linux: condition: service_healthy @@ -44,12 +46,15 @@ services: - BDD_TARGET=linux command: behave --junit --junit-directory Bdd/junit --tags='not @wip' Bdd/features/ - # Each BDD target pairs with its own syslog-ng instance so jobs don't - # interfere. The behave- service shares the syslog-ng- - # network namespace via network_mode so QEMU's slirp gateway 10.0.2.2 - # NATs to the pair's loopback, where syslog-ng- is listening on - # 0.0.0.0:5514. Adding more targets is then just another paired service - # block. + # Each BDD target pairs with its own syslog-ng instance on its own + # bridge network so the `syslog-ng` DNS alias is scoped per-pair — + # if both pairs ever run together (today they don't, by design), + # neither side accidentally resolves the other's oracle. The + # behave- service shares the syslog-ng- network + # namespace via network_mode so QEMU's slirp gateway 10.0.2.2 NATs + # to the pair's loopback, where syslog-ng- is listening on + # 0.0.0.0:5514. Adding more targets is one more pair + one more + # network block. syslog-ng-freertos: image: balabit/syslog-ng:latest volumes: @@ -71,7 +76,7 @@ services: chmod 0777 /var/lib/syslog-ng/syslog-ng.ctl wait networks: - default: + freertos: aliases: - syslog-ng healthcheck: @@ -114,3 +119,7 @@ volumes: bdd-output-freertos: syslog-ng-ctl-linux: syslog-ng-ctl-freertos: + +networks: + linux: + freertos: From 12e0970c85cb16245c530404e8c09521e21a2576 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sat, 9 May 2026 21:07:30 +0100 Subject: [PATCH 3/6] docs: FreeRTOS baseline across README, build/BDD/CI/container docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Establishes the FreeRTOS doc baseline so a reader landing on README can navigate to either build, BDD, or example detail and find current information. Designed to be extended slice-by-slice as more FreeRTOS features land. - README.md: Status section names FreeRTOS Cortex-M3 (mps2-an385) as in-progress (E08 #268); examples list grows to four with the new Example/FreeRtos/SingleTask line; public-headers section lists the FreeRTOS adapter headers (SolidSyslogFreeRtosDatagram.h, SolidSyslogFreeRtosStaticResolver.h). - docs/bdd.md: Architecture section replaces the legacy two-service picture with a per-target table (Linux / FreeRTOS / Windows) and links to Bdd/README.md for the per-target compose details. Tag table gains @freertoswip with the rollout-marker explanation. Runner filter table gains the FreeRTOS row. Local-running commands renamed to behave-linux / syslog-ng-linux. - docs/ci.md: Job table picks up build-freertos-host-tdd, build-freertos-target, bdd-freertos-qemu, plus the pre-existing miss of integration-windows-openssl. Forward-looking sentence about ARM cross-compile / FreeRTOS QEMU oracle now matches reality. - docs/builds.md: New 'FreeRTOS cross — freertos-cross' section pointing at Example/FreeRtos/README.md and Bdd/README.md. BDD section renamed to behave-linux. - docs/containers.md: Top image table corrected to the per-target syslog-ng-linux / syslog-ng-freertos service names and the new behave-linux runner; cpputest-freertos-cross row already covered this PR's earlier doc edits. - Example/FreeRtos/README.md: Top-level table now distinguishes HelloWorld (minimal bring-up smoke for cross-compile / scheduler / UART / newlib regression detection) from SingleTask (canonical syslog example, BDD-driven). Notes that HelloWorld will be retired once SingleTask provides sufficient diagnostic confidence. --- Example/FreeRtos/README.md | 13 ++++++++---- README.md | 9 +++++++- docs/bdd.md | 42 +++++++++++++++++++++++++------------- docs/builds.md | 32 ++++++++++++++++++++++++----- docs/ci.md | 6 +++++- docs/containers.md | 4 ++-- 6 files changed, 79 insertions(+), 27 deletions(-) diff --git a/Example/FreeRtos/README.md b/Example/FreeRtos/README.md index e7665c79..bdc72508 100644 --- a/Example/FreeRtos/README.md +++ b/Example/FreeRtos/README.md @@ -1,13 +1,18 @@ -# FreeRTOS example +# FreeRTOS examples Cross-compiled FreeRTOS demos that run under `qemu-system-arm` with the `mps2-an385` (Cortex-M3) machine. | Subdir | Purpose | |---|---| -| `HelloWorld/` | Single FreeRTOS task that prints `hello` via semihosting. Bring-up smoke test for E08; no syslog content yet. | - -Future stories under E08 will add UDP / TCP / TLS examples alongside. +| `HelloWorld/` | Minimal FreeRTOS-on-QEMU bring-up smoke. Single task printing one line over the CMSDK UART; no IP stack, no syslog. CI runs this under QEMU as a fast pre-flight — when it goes red the cross-compile / scheduler / UART / newlib chain is broken before any higher layer is implicated. Will be retired once `SingleTask`'s coverage and confidence make the diagnostic separation unnecessary. | +| `SingleTask/` | Canonical FreeRTOS-on-QEMU example: SolidSyslog with NullBuffer + UdpSender over FreeRTOS-Plus-TCP, the `SolidSyslogFreeRtosStaticResolver` pinned to the QEMU slirp gateway (`10.0.2.2`), and the `Example/Common/ExampleInteractive` runner. Drive identity / endpoint / PRIVAL fields over the UART command channel with `set NAME VALUE`, emit N RFC 5424 datagrams with `send N`, exit cleanly with `quit`. The BDD harness drives this binary by piping the same commands through `qemu-system-arm`'s stdio UART; see [`Bdd/README.md`](../../Bdd/README.md). | + +Both examples share `Common/` (CMSDK UART driver, newlib syscalls, mps2-an385 +linker script, startup) and `cmake/` (the `arm-none-eabi.cmake` toolchain +file). The `freertos-cross` CMake preset and the `freertos-target` devcontainer +service ([`docs/containers.md`](../../docs/containers.md)) carry everything +needed to build and run them. ## In VS Code diff --git a/README.md b/README.md index 945d7dd2..9f6a2d76 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,11 @@ formatting, UDP / TCP / TLS / mTLS transport, asynchronous buffering, rotating block store-and-forward with CRC-16 integrity, and the full [IEC 62443 SL1–SL4 component set](docs/iec62443.md). +FreeRTOS support is in active development on Cortex-M3 (mps2-an385 under QEMU): +UDP transport via FreeRTOS-Plus-TCP, host-TDD'd adapters, an interactive +SingleTask example, and BDD scenarios driven through QEMU's UART +([epic E08 #268](https://github.com/DavidCozens/solid-syslog/issues/268)). + **Not yet production-ready**, and no API stability guarantee yet. Known gaps: - Public API may evolve — sender implementations currently use static @@ -75,12 +80,14 @@ Public headers are split by audience (Interface Segregation Principle): - **`SolidSyslogTimeQualitySd.h`** — timeQuality structured data (RFC 5424 §7.1): tzKnown, isSynced, syncAccuracy - **`SolidSyslogOriginSd.h`** — origin structured data (RFC 5424 §7.2): software, swVersion, enterpriseId, ip - **`SolidSyslogPosixClock.h`** / **`SolidSyslogPosixHostname.h`** / **`SolidSyslogPosixProcessId.h`** / **`SolidSyslogPosixSysUpTime.h`** — POSIX helpers +- **`SolidSyslogFreeRtosDatagram.h`** / **`SolidSyslogFreeRtosStaticResolver.h`** — FreeRTOS-Plus-TCP UDP datagram adapter and a hardcoded-IPv4 resolver for FreeRTOS targets -Three example programs demonstrate usage: +Four example programs demonstrate usage: - **`Example/SingleTask/`** — POSIX, NullBuffer, single-task bare-metal model - **`Example/Threaded/`** — POSIX, PosixMessageQueueBuffer, two pthreads (logger + service), SwitchingSender over UDP + TCP + TLS + mTLS (TLS build required for the last two); `--transport` sets the initial transport, `switch ` flips it at runtime - **`Example/Windows/`** — Windows, CircularBuffer + WindowsMutex, Win32 service thread (`_beginthreadex`) draining the buffer, Winsock UDP / TCP, with the Windows clock / hostname / process-id / sysUpTime helpers +- **`Example/FreeRtos/SingleTask/`** — FreeRTOS-on-QEMU (Cortex-M3, mps2-an385), NullBuffer, UDP via FreeRTOS-Plus-TCP, interactive `set NAME VALUE` / `send N` / `quit` command channel over the CMSDK UART; BDD-driven against syslog-ng. See [`Example/FreeRtos/README.md`](Example/FreeRtos/README.md) ## Compliance diff --git a/docs/bdd.md b/docs/bdd.md index f540879b..818f04f3 100644 --- a/docs/bdd.md +++ b/docs/bdd.md @@ -37,16 +37,25 @@ fail here. Bdd/output/received*.log ``` -Two Docker Compose services collaborate: - -| Service | Role | -|---|---| -| `syslog-ng` | Receives syslog on UDP / TCP 5514, TLS 6514, and mTLS 6515; parses RFC 5424; writes parsed fields to per-transport log files | -| `behave` | Runs Gherkin scenarios that invoke the example binary and assert on the parsed output | - -The example binary is built in the `gcc` container but executed by Behave via `subprocess.run`. -Both services share the workspace mount, so `Bdd/output/received.log` is visible to both -syslog-ng (writer) and Behave (reader) without any network file transfer. +BDD targets pair with their own oracle so jobs and developers +switching containers never interfere. Each pair lives on its own +bridge network so the `syslog-ng` DNS alias is scoped per-pair — +see [`Bdd/README.md`](../Bdd/README.md) for the per-target compose +layout, the `BDD_TARGET` env-var contract, and FreeRTOS local +fault-finding tips. + +| Target | Behave runner | Oracle | +|---|---|---| +| Linux | `behave-linux` | `syslog-ng-linux` | +| FreeRTOS | inside `freertos-target` (cross image carries QEMU + Behave) | `syslog-ng-freertos` (shared netns; QEMU slirp `10.0.2.2` reaches it on loopback) | +| Windows | `behave` on the runner | `otelcol-contrib` (no compose; runner-direct) | + +The Linux example binary is built in the `gcc` container but executed by Behave via +`subprocess.run`. Both services share the workspace mount, so `Bdd/output/received.log` +is visible to both syslog-ng (writer) and Behave (reader) without any network file +transfer. The FreeRTOS ELF is built in the cross image and run under +`qemu-system-arm` from inside the `freertos-target` service; Behave drives it +through the QEMU UART (`-serial stdio`). ## Key files @@ -125,18 +134,20 @@ etc.) without rewriting feature tags. | `@mtls` | Needs mutual TLS (client cert + key) | | `@buffered` | Needs a Linux-only buffered example capability beyond a basic ring buffer + service thread — file-backed block store, switching sender between transports, or syslog-ng reload via the UNIX control socket. The cross-platform "single message via UDP/TCP through a real buffer" path is *not* `@buffered` post-S13.18; TLS and mTLS dropped `@buffered` in S13.19 once the OTel oracle gained TLS receivers (Windows otelcol-contrib listens on 6514 / 6515 with `client_ca_file` for mTLS). | -Two rollout markers are also used (temporary; remove once the scenario passes): +Three rollout markers are also used (temporary; remove once the scenario passes): | Tag | Meaning | | --- | --- | | `@wip` | Skip everywhere — work in progress | | `@windows_wip` | Skip on Windows only — should work but not yet verified | +| `@freertoswip` | Skip on FreeRTOS only — currently fails or errors on the FreeRTOS-on-QEMU target. Each tagged scenario is a follow-up: usually because the FreeRTOS SingleTask example bakes hardcoded `TEST_*` values (hostname, PROCID, timestamp), wires no `meta` / `origin` / `timeQuality` SD yet, or because the scenario uses cmdline args that the FreeRTOS driver doesn't translate to `set NAME VALUE` | Runner tag filters: | Runner | Filter | | --- | --- | | Linux (syslog-ng) | `not @wip` | +| FreeRTOS (syslog-ng-freertos via QEMU) | `not @wip and not @freertoswip and @udp` | | Windows (OTel Collector) | `not @wip and not @windows_wip and not @buffered` | ## Two oracles, one step file @@ -199,8 +210,8 @@ This avoids restarting the syslog-ng container between scenarios. From WSL or a host terminal (not inside the devcontainer): ```bash -# Start syslog-ng -docker compose -f .devcontainer/docker-compose.yml up -d syslog-ng +# Start the Linux oracle +docker compose -f .devcontainer/docker-compose.yml up -d syslog-ng-linux # Build the example binary (inside the gcc container) docker compose -f .devcontainer/docker-compose.yml exec gcc \ @@ -209,10 +220,13 @@ docker compose -f .devcontainer/docker-compose.yml exec gcc \ cmake --build --preset debug # Run Behave -docker compose -f .devcontainer/docker-compose.yml run --rm behave \ +docker compose -f .devcontainer/docker-compose.yml run --rm behave-linux \ behave Bdd/features/ ``` +For the FreeRTOS pair (cross-build the ELF, then run Behave via QEMU +inside `freertos-target`), see [`Bdd/README.md`](../Bdd/README.md). + ## Verifying syslog-ng manually To send a test message and check that syslog-ng is receiving and parsing correctly: diff --git a/docs/builds.md b/docs/builds.md index 04d19255..02fd5175 100644 --- a/docs/builds.md +++ b/docs/builds.md @@ -95,6 +95,23 @@ cmake --preset release cmake --build --preset release --target junit ``` +## FreeRTOS cross — `freertos-cross` + +ARM cross-build for FreeRTOS targets running under `qemu-system-arm` +(Cortex-M3, mps2-an385). Uses the `freertos-target` devcontainer service +or a host with `arm-none-eabi-gcc` + `qemu-system-arm` on `PATH`. + +```bash +cmake --preset freertos-cross +cmake --build --preset freertos-cross --target SolidSyslogFreeRtosSingleTask +``` + +The ELF lands at +`build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf`. +See [`Example/FreeRtos/README.md`](../Example/FreeRtos/README.md) for run / +GDB-attach instructions and [`Bdd/README.md`](../Bdd/README.md) for driving +it under Behave + the syslog-ng oracle. + ## Installing the library ```bash @@ -107,17 +124,22 @@ This installs the static library to `lib/` and the public headers to `include/`. ## BDD tests — Behave -End-to-end tests run inside the `behave` devcontainer service. Switch to it by changing -`"service": "behave"` in `.devcontainer/devcontainer.json` and rebuilding, or run from the -gcc container: +End-to-end tests run against per-target oracle pairs. The Linux pair uses the +`behave-linux` devcontainer service; switch to it by changing +`"service": "behave-linux"` in `.devcontainer/devcontainer.json` and +rebuilding, or run from the gcc container: ```bash behave Bdd/features/ ``` -In the behave container, **Ctrl+Shift+B** runs `behave Bdd/features/` automatically. +In the behave-linux container, **Ctrl+Shift+B** runs `behave Bdd/features/` automatically. + +For the FreeRTOS pair (cross-build the SingleTask ELF, then drive QEMU through +Behave inside `freertos-target`), see [`Bdd/README.md`](../Bdd/README.md). -See [BDD testing](bdd.md) for architecture details. +See [BDD testing](bdd.md) for architecture details and the `BDD_TARGET` / +`@freertoswip` contract. ## JUnit XML output diff --git a/docs/ci.md b/docs/ci.md index 3947d7f4..92f86834 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -20,8 +20,12 @@ without renaming what's already there. | `analyze-format` | — | clang-format dry-run; fails if any file needs reformatting | | `analyze-iwyu` | `iwyu` | include-what-you-use; fails on missing or unused `#include` directives | | `integration-linux-openssl` | `debug` | Runs the in-process TLS integration tests against libssl (no network oracle) | -| `bdd-linux-syslog-ng` | — | End-to-end BDD test via Docker Compose (syslog-ng + Behave), Linux runner | +| `integration-windows-openssl` | `msvc-debug` | Same TLS integration tests on `windows-latest` against libssl from vcpkg | +| `bdd-linux-syslog-ng` | — | End-to-end BDD test via Docker Compose (`syslog-ng-linux` + `behave-linux`), Linux runner | | `bdd-windows-otel` | — | Windows-eligible BDD scenarios driven against an OTel Collector oracle | +| `build-freertos-host-tdd` | `debug` | Host-TDD of FreeRTOS adapters against fakes; runs inside `cpputest-freertos` (FreeRTOS upstream sources at fixed paths) | +| `build-freertos-target` | `freertos-cross` | ARM cross-build (Cortex-M3, mps2-an385); QEMU bring-up smoke for HelloWorld; uploads the SingleTask ELF as an artifact for `bdd-freertos-qemu` | +| `bdd-freertos-qemu` | — | Pulls the SingleTask ELF artifact, brings up the freertos compose pair (`syslog-ng-freertos` + `behave-freertos`); Behave drives the example through `qemu-system-arm`'s UART | ## Branch protection diff --git a/docs/containers.md b/docs/containers.md index 32c4ac63..9f36419e 100644 --- a/docs/containers.md +++ b/docs/containers.md @@ -8,8 +8,8 @@ | `ghcr.io/davidcozens/cpputest-clang` | `sha-7eac3ab` | `clang` compose service, `build-linux-clang` CI job, `analyze-iwyu` CI job | | `ghcr.io/davidcozens/cpputest-freertos` | `sha-44efeae` | `freertos-host` compose service, `build-freertos-host-tdd` CI job — adds FreeRTOS-Kernel / Plus-TCP / Plus-FAT / Mbed TLS sources for host-TDD of FreeRTOS adapters against fakes | | `ghcr.io/davidcozens/cpputest-freertos-cross` | `sha-44efeae` | `freertos-target` compose service, `build-freertos-target` CI job, `behave-freertos` BDD service, `bdd-freertos-qemu` CI job — adds `gcc-arm-none-eabi`, `libnewlib-arm-none-eabi`, `gdb-multiarch` (aliased as `arm-none-eabi-gdb`), `qemu-system-arm`, `python3` + `behave` for cross builds, on-QEMU runs, and BDD scenarios driving a QEMU target | -| `balabit/syslog-ng` | `latest` | `syslog-ng` service — BDD test oracle | -| `ghcr.io/davidcozens/behave` | `sha-3faff14` | `behave` service — Debian trixie + Python 3.12 + Behave for BDD scenarios | +| `balabit/syslog-ng` | `latest` | `syslog-ng-linux` and `syslog-ng-freertos` services — BDD test oracles, one per target pair | +| `ghcr.io/davidcozens/behave` | `sha-3faff14` | `behave-linux` service — Debian trixie + Python 3.12 + Behave for Linux BDD scenarios. The FreeRTOS BDD runner uses the `cpputest-freertos-cross` image instead (which carries QEMU + Behave). | ## Docker Compose setup From d02560fbe6307c0bd1bc8a520b96219e4b9c4054 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sat, 9 May 2026 21:08:19 +0100 Subject: [PATCH 4/6] fix: validate BDD_TARGET against the known target set CodeRabbit flagged that the previous environment.py treated every non-freertos value as Linux, so a typo or BDD_TARGET=windows silently misrouted execution rather than failing loudly. Replace the implicit fallback with an explicit `linux | windows | freertos` dict, lowercase the input so case differences are tolerated, and raise ValueError on any other value with the allowed set in the message. Adds the Windows default binary path so windows is now a first-class entry. bdd-windows-otel CI job sets BDD_TARGET=windows explicitly so the new validation accepts it; previously it relied on the default-Linux fallback while overriding EXAMPLE_BINARY. Linux and FreeRTOS jobs are already explicit via the compose file's environment block. --- .github/workflows/ci.yml | 1 + Bdd/features/environment.py | 20 ++++++++++++++------ 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d7a8a196..03c2bbda 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -574,6 +574,7 @@ jobs: - name: Run BDD tests env: + BDD_TARGET: windows EXAMPLE_BINARY: build/msvc-debug/Example/Debug/SolidSyslogExample.exe RECEIVED_LOG: Bdd/output/received.jsonl ORACLE_FORMAT: otel-jsonl diff --git a/Bdd/features/environment.py b/Bdd/features/environment.py index 7cc65423..b5819763 100644 --- a/Bdd/features/environment.py +++ b/Bdd/features/environment.py @@ -97,16 +97,24 @@ def before_all(context): # different binary paths and oracle output formats. BDD_TARGET picks # how the example is spawned (native subprocess vs qemu-system-arm) # — see Bdd/features/steps/target_driver.py. - context.target = os.environ.get("BDD_TARGET", "linux") + context.target = os.environ.get("BDD_TARGET", "linux").lower() - if context.target == "freertos": - default_binary = ( + default_binaries = { + "linux": "build/debug/Example/SolidSyslogExample", + "windows": "build/msvc-debug/Example/Debug/SolidSyslogExample.exe", + "freertos": ( "build/freertos-cross/Example/FreeRtos/SingleTask/" "SolidSyslogFreeRtosSingleTask.elf" + ), + } + if context.target not in default_binaries: + raise ValueError( + f"Unsupported BDD_TARGET={context.target!r}. " + f"Expected one of: {sorted(default_binaries)}." ) - else: - default_binary = "build/debug/Example/SolidSyslogExample" - context.example_binary = os.environ.get("EXAMPLE_BINARY", default_binary) + context.example_binary = os.environ.get( + "EXAMPLE_BINARY", default_binaries[context.target] + ) context.received_log = os.environ.get( "RECEIVED_LOG", "Bdd/output/received.log" ) From 3294f411d4a81f0dd9a2ecbe20dc68fe49414b1e Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sat, 9 May 2026 21:10:38 +0100 Subject: [PATCH 5/6] chore: bump cpputest-freertos[-cross] to sha-10a14ae and drop inline install MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit flagged the inline `apt-get install python3-pip; pip install` in the behave-freertos compose service as redundant — the image bump that adds python3 + behave to the cross image has now landed (CppUTestFreertosDocker `feat: add python3 + behave to cross image`, SHA 10a14ae, published to GHCR). Bump every reference and drop the inline install along with its `entrypoint: bash -c` wrapper, falling back to the simpler `command: behave ...` form mirroring behave-linux. Files updated: - .devcontainer/docker-compose.yml — freertos-host + freertos-target - .github/workflows/ci.yml — build-freertos-host-tdd + build-freertos-target - ci/docker-compose.bdd.yml — behave-freertos image + simplified command - docs/containers.md — the cpputest-freertos / cross image table rows Local smoke confirms behave-freertos runtime drops from ~45s (apt+pip install on every run) to ~2.4s; same 1 scenario passes, 45 @freertoswip skipped. --- .devcontainer/docker-compose.yml | 4 ++-- .github/workflows/ci.yml | 4 ++-- ci/docker-compose.bdd.yml | 17 +++-------------- docs/containers.md | 4 ++-- 4 files changed, 9 insertions(+), 20 deletions(-) diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 9381aa76..67352e72 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -102,7 +102,7 @@ services: command: sleep infinity freertos-host: - image: ghcr.io/davidcozens/cpputest-freertos:sha-44efeae + image: ghcr.io/davidcozens/cpputest-freertos:sha-10a14ae volumes: - ..:/workspaces/SolidSyslog:cached - ${SSH_AUTH_SOCK:-/dev/null}:/ssh-agent @@ -153,7 +153,7 @@ services: - syslog-ng freertos-target: - image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae + image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-10a14ae volumes: - ..:/workspaces/SolidSyslog:cached - syslog-ng-ctl-freertos:/var/lib/syslog-ng diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 03c2bbda..1a75aa7e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -660,7 +660,7 @@ jobs: contents: read checks: write container: - image: ghcr.io/davidcozens/cpputest-freertos:sha-44efeae + image: ghcr.io/davidcozens/cpputest-freertos:sha-10a14ae options: --user root env: GIT_CONFIG_COUNT: 1 @@ -685,7 +685,7 @@ jobs: permissions: contents: read container: - image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae + image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-10a14ae options: --user root env: GIT_CONFIG_COUNT: 1 diff --git a/ci/docker-compose.bdd.yml b/ci/docker-compose.bdd.yml index 36612650..da61eb58 100644 --- a/ci/docker-compose.bdd.yml +++ b/ci/docker-compose.bdd.yml @@ -86,11 +86,8 @@ services: retries: 10 behave-freertos: - # Cross image carries qemu-system-arm + python3 + behave (post-image - # bump in CppUTestFreertosDocker). Until that bump lands, behave is - # pip-installed at runtime; remove the inline install once the image - # tag advances. - image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae + # Cross image carries qemu-system-arm + python3 + behave. + image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-10a14ae user: "0" volumes: - ..:/workspaces/SolidSyslog @@ -104,15 +101,7 @@ services: environment: - BDD_TARGET=freertos - EXAMPLE_BINARY=build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf - entrypoint: ["bash", "-c"] - command: - - | - # Temporary install — once the cpputest-freertos-cross image - # gains python3-pip + behave (CppUTestFreertosDocker bump), drop - # this line and the `apt-get install` block below. - apt-get update >/dev/null && apt-get install -y --no-install-recommends python3-pip >/dev/null - pip install --no-cache-dir --break-system-packages -r Bdd/requirements.txt - behave --junit --junit-directory Bdd/junit --tags='not @wip and not @freertoswip and @udp' Bdd/features/ + command: behave --junit --junit-directory Bdd/junit --tags='not @wip and not @freertoswip and @udp' Bdd/features/ volumes: bdd-output-linux: diff --git a/docs/containers.md b/docs/containers.md index 9f36419e..24a35b2e 100644 --- a/docs/containers.md +++ b/docs/containers.md @@ -6,8 +6,8 @@ |---|---|---| | `ghcr.io/davidcozens/cpputest` | `sha-18f19e1` | devcontainer (`gcc` service), most CI jobs | | `ghcr.io/davidcozens/cpputest-clang` | `sha-7eac3ab` | `clang` compose service, `build-linux-clang` CI job, `analyze-iwyu` CI job | -| `ghcr.io/davidcozens/cpputest-freertos` | `sha-44efeae` | `freertos-host` compose service, `build-freertos-host-tdd` CI job — adds FreeRTOS-Kernel / Plus-TCP / Plus-FAT / Mbed TLS sources for host-TDD of FreeRTOS adapters against fakes | -| `ghcr.io/davidcozens/cpputest-freertos-cross` | `sha-44efeae` | `freertos-target` compose service, `build-freertos-target` CI job, `behave-freertos` BDD service, `bdd-freertos-qemu` CI job — adds `gcc-arm-none-eabi`, `libnewlib-arm-none-eabi`, `gdb-multiarch` (aliased as `arm-none-eabi-gdb`), `qemu-system-arm`, `python3` + `behave` for cross builds, on-QEMU runs, and BDD scenarios driving a QEMU target | +| `ghcr.io/davidcozens/cpputest-freertos` | `sha-10a14ae` | `freertos-host` compose service, `build-freertos-host-tdd` CI job — adds FreeRTOS-Kernel / Plus-TCP / Plus-FAT / Mbed TLS sources for host-TDD of FreeRTOS adapters against fakes | +| `ghcr.io/davidcozens/cpputest-freertos-cross` | `sha-10a14ae` | `freertos-target` compose service, `build-freertos-target` CI job, `behave-freertos` BDD service, `bdd-freertos-qemu` CI job — adds `gcc-arm-none-eabi`, `libnewlib-arm-none-eabi`, `gdb-multiarch` (aliased as `arm-none-eabi-gdb`), `qemu-system-arm`, `python3` + `behave` for cross builds, on-QEMU runs, and BDD scenarios driving a QEMU target | | `balabit/syslog-ng` | `latest` | `syslog-ng-linux` and `syslog-ng-freertos` services — BDD test oracles, one per target pair | | `ghcr.io/davidcozens/behave` | `sha-3faff14` | `behave-linux` service — Debian trixie + Python 3.12 + Behave for Linux BDD scenarios. The FreeRTOS BDD runner uses the `cpputest-freertos-cross` image instead (which carries QEMU + Behave). | From ad8a6930de39540b062cfac633143d3ddd9a2fe3 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sat, 9 May 2026 21:11:43 +0100 Subject: [PATCH 6/6] refactor: address CodeRabbit nitpicks on slice 5 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Bdd/features/steps/target_driver.py — drop the redundant `quit\n` write in the FreeRTOS branch of stop_example_process. The docstring said "kills after a short grace period" but the code immediately followed `quit\n` with `process.kill()`, so the write was best-effort and misleading. The BDD scenario has already verified the oracle received the frame; just kill QEMU and move on. Docstring rewritten to match. - DEVLOG.md — blank lines added around the @freertoswip rationale table to satisfy markdownlint MD058. - Bdd/README.md — note added to the FreeRTOS fault-finding tips explaining that syslog-ng-freertos exposes no host ports (unlike syslog-ng-linux) and how to send a probe frame from inside the container. --- Bdd/README.md | 5 +++++ Bdd/features/steps/target_driver.py | 16 ++++++---------- DEVLOG.md | 2 ++ 3 files changed, 13 insertions(+), 10 deletions(-) diff --git a/Bdd/README.md b/Bdd/README.md index 1ba276e6..ee5eccd4 100644 --- a/Bdd/README.md +++ b/Bdd/README.md @@ -82,6 +82,11 @@ oracle on the shared loopback. inside the freertos pair gives the parsed frame in `key=value` form (the syslog-ng template in `Bdd/syslog-ng/syslog-ng.conf`). +- **`nc localhost 5514` from the host won't reach `syslog-ng-freertos`.** + Unlike `syslog-ng-linux`, the freertos oracle exposes no host ports + — `freertos-target` reaches it via the shared netns on the pair's + loopback. To send a probe frame manually, run inside the container: + `docker compose exec freertos-target nc -u 127.0.0.1 5514`. ## Running in CI diff --git a/Bdd/features/steps/target_driver.py b/Bdd/features/steps/target_driver.py index 529045d3..281a48c0 100644 --- a/Bdd/features/steps/target_driver.py +++ b/Bdd/features/steps/target_driver.py @@ -89,18 +89,14 @@ def stop_example_process(process, target, timeout=10): """Tear down the example process for the active target. Linux/Windows: the example exits cleanly on `quit`, so wait for the - return code. FreeRTOS: `quit` deletes the interactive task but the - QEMU VM keeps idling (the scheduler stays alive so a GDB attach - works), so kill QEMU after a short grace period — the BDD scenario - has already verified the oracle received the frame, the QEMU exit - code carries no useful signal. + return code. FreeRTOS: `quit` only deletes the interactive task — + the QEMU VM keeps idling (the FreeRTOS scheduler stays alive so a + GDB attach works), so the only way to terminate is to kill QEMU + directly. The BDD scenario has already verified the oracle + received the frame, so the QEMU exit code carries no useful + signal — return None for that path so callers don't assert on it. """ if target == "freertos": - try: - process.stdin.write("quit\n") - process.stdin.flush() - except (BrokenPipeError, OSError): - pass process.kill() process.wait(timeout=timeout) return None diff --git a/DEVLOG.md b/DEVLOG.md index 5749cdd1..2ce9962e 100644 --- a/DEVLOG.md +++ b/DEVLOG.md @@ -60,6 +60,7 @@ real Behave + the existing syslog-ng oracle, and adds the CI matrix entry. `--tags='not @wip and not @freertoswip and @udp'`; Linux runs `--tags='not @wip'` so existing Linux behaviour is unchanged. Tagged features today and the reason each one is tagged: + | Feature | Why | |---|---| | `syslog.feature` (single scenario) | Composite assertion includes "system hostname", "current timestamp", and "PID of example program" — FreeRTOS uses the literal `FreeRtosExample` hostname, the RFC 5424 publication-date placeholder, and PROCID `1`. | @@ -72,6 +73,7 @@ real Behave + the existing syslog-ng oracle, and adds the CI matrix entry. | `message_fields.feature` | Uses `--message-id` / `--message` cmdline args. | | `udp_mtu.feature` | Uses `--message` cmdline args with very long bodies. | | `buffered.feature` | Drives the Linux Threaded binary (no FreeRTOS equivalent yet). | + - **Walking-skeleton acceptance is "the harness lands, with at least one scenario passing."** That scenario is `header_fields.feature::App name matches the example program`. Every other current `@udp` scenario is