Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 67 additions & 9 deletions .devcontainer/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,24 @@ services:
- BUILD_PRESET=debug
user: developer
working_dir: /workspaces/SolidSyslog
networks:
- linux
command: sleep infinity
depends_on:
- syslog-ng
- syslog-ng-linux

syslog-ng:
# Linux pair — syslog-ng oracle + behave runner, used by gcc/clang dev
# work and the bdd-linux-syslog-ng CI job. Each pair lives on its own
# bridge network so the `syslog-ng` DNS alias is scoped per-pair, even
# if a future workflow runs both pairs simultaneously; see
# ci/docker-compose.bdd.yml for the rationale.
syslog-ng-linux:
image: balabit/syslog-ng:latest
volumes:
- ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro
- ../Bdd/output:/var/log/syslog-ng
- syslog-ng-ctl:/var/lib/syslog-ng
- syslog-ng-ctl-linux:/var/lib/syslog-ng
ports:
- "5514:5514/udp"
- "5514:5514/tcp"
Expand All @@ -51,22 +58,29 @@ services:
sleep 0.1
done ) &
wait $$pid
networks:
linux:
aliases:
- syslog-ng
Comment thread
coderabbitai[bot] marked this conversation as resolved.

behave:
behave-linux:
image: ghcr.io/davidcozens/behave:sha-3faff14
volumes:
- ..:/workspaces/SolidSyslog:cached
- ../Bdd/output:/workspaces/SolidSyslog/Bdd/output
- syslog-ng-ctl:/var/lib/syslog-ng
- syslog-ng-ctl-linux:/var/lib/syslog-ng
- ${SSH_AUTH_SOCK:-/dev/null}:/ssh-agent
- ${HOME:-/tmp}/.claude:/home/developer/.claude
- ${HOME:-/tmp}/.config/gh:/home/developer/.config/gh
environment:
- SSH_AUTH_SOCK=/ssh-agent
- BDD_TARGET=linux
user: developer
working_dir: /workspaces/SolidSyslog
networks:
- linux
depends_on:
- syslog-ng
- syslog-ng-linux
command: sleep infinity

clang:
Expand All @@ -88,7 +102,7 @@ services:
command: sleep infinity

freertos-host:
image: ghcr.io/davidcozens/cpputest-freertos:sha-44efeae
image: ghcr.io/davidcozens/cpputest-freertos:sha-10a14ae
volumes:
- ..:/workspaces/SolidSyslog:cached
- ${SSH_AUTH_SOCK:-/dev/null}:/ssh-agent
Expand All @@ -105,23 +119,67 @@ services:
working_dir: /workspaces/SolidSyslog
command: sleep infinity

# FreeRTOS pair — syslog-ng oracle + the cross devcontainer that hosts
# both QEMU and Behave. freertos-target shares syslog-ng-freertos's
# network namespace so QEMU's slirp gateway 10.0.2.2 NATs to the pair's
# loopback, where syslog-ng-freertos is listening on 0.0.0.0:5514.
syslog-ng-freertos:
image: balabit/syslog-ng:latest
volumes:
- ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro
- ../Bdd/output:/var/log/syslog-ng
- syslog-ng-ctl-freertos:/var/lib/syslog-ng
working_dir: /workspaces/SolidSyslog
entrypoint: ["bash", "-c"]
command:
- |
syslog-ng -F &
pid=$$!
deadline=$$((SECONDS + 30))
while [ ! -S /var/lib/syslog-ng/syslog-ng.ctl ]; do
kill -0 "$$pid" 2>/dev/null || { echo "syslog-ng exited unexpectedly"; exit 1; }
[ "$$SECONDS" -ge "$$deadline" ] && { echo "Timed out waiting for syslog-ng control socket"; kill "$$pid" 2>/dev/null; exit 1; }
sleep 0.1
done
( while kill -0 "$$pid" 2>/dev/null; do
chmod 0777 /var/lib/syslog-ng/syslog-ng.ctl 2>/dev/null
sleep 0.1
done ) &
wait $$pid
networks:
freertos:
aliases:
- syslog-ng

freertos-target:
image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae
image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-10a14ae
volumes:
- ..:/workspaces/SolidSyslog:cached
- syslog-ng-ctl-freertos:/var/lib/syslog-ng
- ${SSH_AUTH_SOCK:-/dev/null}:/ssh-agent
- ${HOME:-/tmp}/.claude:/home/developer/.claude
- ${HOME:-/tmp}/.config/gh:/home/developer/.config/gh
cap_add:
- SYS_PTRACE
security_opt:
- seccomp:unconfined
network_mode: service:syslog-ng-freertos
environment:
- SSH_AUTH_SOCK=/ssh-agent
- BUILD_PRESET=freertos-cross
- BDD_TARGET=freertos
- EXAMPLE_BINARY=build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf
user: developer
working_dir: /workspaces/SolidSyslog
depends_on:
- syslog-ng-freertos
command: sleep infinity

volumes:
syslog-ng-ctl:
syslog-ng-ctl-linux:
syslog-ng-ctl-freertos:

networks:
linux:
freertos:
74 changes: 68 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,7 @@ jobs:

- name: Run BDD tests
env:
BDD_TARGET: windows
EXAMPLE_BINARY: build/msvc-debug/Example/Debug/SolidSyslogExample.exe
RECEIVED_LOG: Bdd/output/received.jsonl
ORACLE_FORMAT: otel-jsonl
Expand Down Expand Up @@ -630,7 +631,8 @@ jobs:
- name: Run BDD tests
run: >
docker compose -f ci/docker-compose.bdd.yml
up --build --abort-on-container-exit --exit-code-from behave
up --abort-on-container-exit --exit-code-from behave-linux
behave-linux syslog-ng-linux

- name: BDD Test Report
uses: dorny/test-reporter@v3
Expand Down Expand Up @@ -658,7 +660,7 @@ jobs:
contents: read
checks: write
container:
image: ghcr.io/davidcozens/cpputest-freertos:sha-44efeae
image: ghcr.io/davidcozens/cpputest-freertos:sha-10a14ae
options: --user root
env:
GIT_CONFIG_COUNT: 1
Expand All @@ -683,7 +685,7 @@ jobs:
permissions:
contents: read
container:
image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-44efeae
image: ghcr.io/davidcozens/cpputest-freertos-cross:sha-10a14ae
options: --user root
env:
GIT_CONFIG_COUNT: 1
Expand All @@ -696,8 +698,8 @@ jobs:
- name: Configure (ARM cross)
run: cmake --preset freertos-cross

- name: Cross-build hello-world ELF
run: cmake --build --preset freertos-cross --target SolidSyslogFreeRtosHelloWorld
- name: Cross-build ELFs
run: cmake --build --preset freertos-cross --target SolidSyslogFreeRtosHelloWorld SolidSyslogFreeRtosSingleTask

# QEMU smoke: run the ELF for a few seconds, capture stdout, assert the
# expected greeting. timeout returns 124 when it kills QEMU, which is
Expand Down Expand Up @@ -734,9 +736,64 @@ jobs:
[ "$RC" -eq 124 ] || { echo "Expected timeout exit 124, got $RC"; exit 1; }
echo "$OUT" | grep -q "hello from FreeRTOS on QEMU mps2-an385"

- name: Upload SingleTask ELF
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: solid-syslog-freertos-singletask
path: build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf
retention-days: 1
compression-level: 0

bdd-freertos-qemu:
needs: build-freertos-target
runs-on: ubuntu-latest
permissions:
contents: read
checks: write

steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

- uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
with:
name: solid-syslog-freertos-singletask
path: build/freertos-cross/Example/FreeRtos/SingleTask/

- name: Make ELF readable
run: chmod +r build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf

- name: Prepare BDD output directory
run: mkdir -p Bdd/junit && chmod 777 Bdd/junit

- name: Run BDD tests
run: >
docker compose -f ci/docker-compose.bdd.yml
up --abort-on-container-exit --exit-code-from behave-freertos
behave-freertos syslog-ng-freertos

- name: BDD Test Report (FreeRTOS)
uses: dorny/test-reporter@v3
if: success() || failure()
with:
name: Test Results (BDD FreeRTOS)
path: Bdd/junit/TESTS-*.xml
reporter: java-junit

- name: Upload JUnit XML
if: success() || failure()
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: junit-bdd-freertos-qemu
path: Bdd/junit/TESTS-*.xml
retention-days: 1

- name: Compose logs on failure
if: failure()
run: docker compose -f ci/docker-compose.bdd.yml logs --no-color

summary:
if: always() && github.event_name == 'pull_request'
needs: [build-linux-gcc, build-linux-clang, sanitize-linux-gcc, coverage-linux-gcc, analyze-tidy, analyze-cppcheck, analyze-format, analyze-iwyu, bdd-linux-syslog-ng, build-windows-msvc, bdd-windows-otel, integration-linux-openssl, integration-windows-openssl, build-freertos-host-tdd, build-freertos-target]
needs: [build-linux-gcc, build-linux-clang, sanitize-linux-gcc, coverage-linux-gcc, analyze-tidy, analyze-cppcheck, analyze-format, analyze-iwyu, bdd-linux-syslog-ng, build-windows-msvc, bdd-windows-otel, integration-linux-openssl, integration-windows-openssl, build-freertos-host-tdd, build-freertos-target, bdd-freertos-qemu]
runs-on: ubuntu-latest
permissions:
contents: read
Expand Down Expand Up @@ -819,6 +876,11 @@ jobs:
"name": "bdd-windows-otel",
"pattern": "**/junit-bdd-windows-otel/TESTS-*.xml"
},
{
"id": "junit",
"name": "bdd-freertos-qemu",
"pattern": "**/junit-bdd-freertos-qemu/TESTS-*.xml"
},
{
"id": "junit",
"name": "build-windows-msvc",
Expand Down
101 changes: 101 additions & 0 deletions Bdd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# BDD scenarios

Gherkin features under `features/` are driven by [Behave](https://behave.readthedocs.io/)
against a `syslog-ng` oracle (Linux + FreeRTOS) or `otelcol-contrib` (Windows). The
same feature files run on every target — the active runner is selected by the
`BDD_TARGET` environment variable, which `features/environment.py` reads in
`before_all` and dispatches via `features/steps/target_driver.py`.

| `BDD_TARGET` | Example-under-test | Oracle |
|---|---|---|
| `linux` (default) | `build/debug/Example/SolidSyslogExample` (native subprocess) | `syslog-ng-linux` |
| `windows` | `build/msvc-debug/Example/Debug/SolidSyslogExample.exe` (native subprocess) | `otelcol-contrib` |
| `freertos` | `build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf` driven through `qemu-system-arm` | `syslog-ng-freertos` |

Each BDD target pairs with its own oracle service so jobs (and developers
switching containers) never interfere — see [`docs/containers.md`](../docs/containers.md)
for the compose layout.

## Tags

| Tag | Meaning |
|---|---|
| `@udp` / `@tcp` / `@tls` / `@mtls` | Transport-specific scenario; CI jobs filter by these. |
| `@windows_wip` | Skipped on the Windows runner (typically OS-specific behaviour the OTel oracle doesn't model). |
| `@freertoswip` | Skipped on the FreeRTOS-on-QEMU runner. The FreeRTOS SingleTask example bakes hard-coded `TEST_*` values for hostname, PROCID, and timestamp; consumes config via the interactive `set NAME VALUE` command rather than getopt; and does not yet wire `meta` / `origin` / `timeQuality` SD. Scenarios that depend on any of these are tagged today and the tag is removed slice-by-slice as each gap is closed. |
| `@wip` | Globally skipped on every runner. |

## Running locally

### Linux (host development)

From inside the `gcc` devcontainer:

```bash
behave Bdd/features/syslog.feature # one feature
behave --tags='not @wip' Bdd/features/ # full Linux suite
```

The example binary and oracle path default to the values
`environment.before_all` sets for `BDD_TARGET=linux`; no env vars needed.

### FreeRTOS-on-QEMU

Switch the devcontainer service to `freertos-target` (see
[`docs/containers.md`](../docs/containers.md) — switch the `service:` line in
`.devcontainer/devcontainer.json` and rebuild). After rebuild, build the ELF
once and run Behave from inside the container:

```bash
cmake --preset freertos-cross
cmake --build --preset freertos-cross --target SolidSyslogFreeRtosSingleTask
behave --tags='not @wip and not @freertoswip and @udp' Bdd/features/
```

`BDD_TARGET=freertos` and `EXAMPLE_BINARY=build/freertos-cross/...` are
preset on the service so Behave dispatches to the QEMU driver
automatically. The driver spawns `qemu-system-arm -M mps2-an385 -serial
stdio …` and pipes `set` / `send` / `quit` over the UART; the QEMU
guest's slirp gateway `10.0.2.2` reaches the paired `syslog-ng-freertos`
oracle on the shared loopback.

#### Fault-finding tips

- **No prompt within 30 seconds**: the `SolidSyslog>` prompt is printed by
the FreeRTOS interactive task, which the IP-network event hook only
spawns once `eNetworkUp` fires. Add `-d guest_errors` to the QEMU args
(in `target_driver.py::_QEMU_BASE_ARGS`) to surface CPU exceptions, or
detach the same QEMU command from Behave and drive it manually:
```bash
qemu-system-arm -M mps2-an385 -m 16M -display none -serial stdio \
-icount shift=auto,sleep=off,align=off \
-netdev user,id=net0 -net nic,netdev=net0,model=lan9118 \
-kernel build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf
```
- **Oracle is silent**: confirm the syslog-ng-freertos service is
healthy (`docker compose ps`), and that the host you're testing
resolves under the shared netns. Slice 3b.1.5's ARP-priming work
means the very first datagram is delivered, but a stale slirp NAT
table on a long-running compose network has been observed to
swallow packets — `docker compose down -v` then up.
- **Want a quick frame inspection**: `cat Bdd/output/received.log`
inside the freertos pair gives the parsed frame in
`key=value` form (the syslog-ng template in
`Bdd/syslog-ng/syslog-ng.conf`).
- **`nc localhost 5514` from the host won't reach `syslog-ng-freertos`.**
Unlike `syslog-ng-linux`, the freertos oracle exposes no host ports
— `freertos-target` reaches it via the shared netns on the pair's
loopback. To send a probe frame manually, run inside the container:
`docker compose exec freertos-target nc -u 127.0.0.1 5514`.

## Running in CI

| Job | Compose pair |
|---|---|
| `bdd-linux-syslog-ng` | `syslog-ng-linux` + `behave-linux` |
| `bdd-freertos-qemu` | `syslog-ng-freertos` + `behave-freertos` (cross image w/ QEMU + Behave) |
| `bdd-windows-otel` | runner-direct (no compose); spawns `otelcol-contrib.exe` directly |

`bdd-freertos-qemu` depends on `build-freertos-target` to upload the
SingleTask ELF as an artifact; the new job downloads it before
`docker compose up`.
1 change: 1 addition & 0 deletions Bdd/features/buffered.feature
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
@udp
@freertoswip
Feature: Buffered message delivery
An example wired with a real (non-Null) buffer drives messages
through a service thread that drains the buffer and sends to the
Expand Down
Loading
Loading