Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .devcontainer/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ services:
# if a future workflow runs both pairs simultaneously; see
# ci/docker-compose.bdd.yml for the rationale.
syslog-ng-linux:
image: balabit/syslog-ng:latest
image: balabit/syslog-ng:4.8.2
volumes:
- ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro
Expand Down Expand Up @@ -124,7 +124,7 @@ services:
# network namespace so QEMU's slirp gateway 10.0.2.2 NATs to the pair's
# loopback, where syslog-ng-freertos is listening on 0.0.0.0:5514.
syslog-ng-freertos:
image: balabit/syslog-ng:latest
image: balabit/syslog-ng:4.8.2
volumes:
- ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -580,7 +580,7 @@ jobs:
ORACLE_FORMAT: otel-jsonl
run: >
behave --junit --junit-directory Bdd/junit
--tags='not @wip and not @windows_wip'
--tags='not @wip and not @windows_wip and not @no_rtc'
Bdd/features/

- name: BDD Test Report (Windows)
Expand Down
6 changes: 4 additions & 2 deletions Bdd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ for the compose layout.
|---|---|
| `@udp` / `@tcp` / `@tls` / `@mtls` | Transport-specific scenario; CI jobs filter by these. |
| `@windows_wip` | Skipped on the Windows runner (typically OS-specific behaviour the OTel oracle doesn't model). |
| `@freertoswip` | Skipped on the FreeRTOS-on-QEMU runner. The FreeRTOS SingleTask example bakes hard-coded `TEST_*` values for hostname, PROCID, and timestamp; consumes config via the interactive `set NAME VALUE` command rather than getopt; and does not yet wire `meta` / `origin` / `timeQuality` SD. Scenarios that depend on any of these are tagged today and the tag is removed slice-by-slice as each gap is closed. |
| `@freertoswip` | Skipped on the FreeRTOS-on-QEMU runner. The FreeRTOS SingleTask example bakes hard-coded `TEST_*` values for hostname and PROCID and consumes config via the interactive `set NAME VALUE` command rather than getopt. Scenarios that depend on any of these are tagged today and the tag is removed slice-by-slice as each gap is closed. |
| `@rtc` | Scenario assumes the device has an RTC and synchronised wall-clock time. Run on Linux/Windows (which have both). Skipped on FreeRTOS, which models a no-RTC product per RFC 5424 §6.2.3.1. |
| `@no_rtc` | Scenario asserts the no-RTC product behaviour over the wire (`tzKnown="0"`, `isSynced="0"`). Run on FreeRTOS. Skipped on Linux/Windows. The NILVALUE TIMESTAMP itself is not asserted via the oracle — syslog-ng silently substitutes receipt time for `${ISODATE}` / `${S_ISODATE}` when the wire timestamp is NILVALUE — so that case is covered by formatter unit tests. |
| `@wip` | Globally skipped on every runner. |

## Running locally
Expand All @@ -49,7 +51,7 @@ once and run Behave from inside the container:
```bash
cmake --preset freertos-cross
cmake --build --preset freertos-cross --target SolidSyslogFreeRtosSingleTask
behave --tags='not @wip and not @freertoswip and @udp' Bdd/features/
behave --tags='not @wip and not @freertoswip and not @rtc and @udp' Bdd/features/
```

`BDD_TARGET=freertos` and `EXAMPLE_BINARY=build/freertos-cross/...` are
Expand Down
14 changes: 13 additions & 1 deletion Bdd/features/origin.feature
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Feature: Structured data — origin
When the example program sends a syslog message
Then the structured data contains ip "192.0.2.1"

@freertoswip
@rtc
Scenario: All standard structured data present
Given the syslog oracle is running
When the example program sends a syslog message
Expand All @@ -32,3 +32,15 @@ Feature: Structured data — origin
And the structured data contains swVersion "0.7.0"
And the structured data contains enterpriseId "1.3.6.1.4.1.99999"
And the structured data contains ip "192.0.2.1"

@no_rtc
Scenario: All standard structured data present (no RTC)
Given the syslog oracle is running
When the example program sends a syslog message
Then the structured data contains sequenceId "1"
And the structured data contains tzKnown "0"
And the structured data contains isSynced "0"
And the structured data contains software "SolidSyslogExample"
And the structured data contains swVersion "0.7.0"
And the structured data contains enterpriseId "1.3.6.1.4.1.99999"
And the structured data contains ip "192.0.2.1"
24 changes: 22 additions & 2 deletions Bdd/features/time_quality.feature
Original file line number Diff line number Diff line change
@@ -1,16 +1,36 @@
@udp
@freertoswip
Feature: Structured data — time quality
The library includes time quality metadata in structured data.
The library includes time quality metadata in structured data. RTC-equipped
targets (Linux, Windows) report tzKnown="1" and isSynced="1"; embedded
targets without an RTC report tzKnown="0" and isSynced="0", per the
no-RTC product stance modelled in the FreeRTOS reference example.

@rtc
Scenario: Time quality appears in structured data
Given the syslog oracle is running
When the example program sends a syslog message
Then the structured data contains tzKnown "1"
And the structured data contains isSynced "1"

@rtc
Scenario: Time quality and sequence ID coexist
Given the syslog oracle is running
When the example program sends a syslog message
Then the structured data contains sequenceId "1"
And the structured data contains tzKnown "1"
And the structured data contains isSynced "1"

@no_rtc
Scenario: Time quality reflects no RTC
Given the syslog oracle is running
When the example program sends a syslog message
Then the structured data contains tzKnown "0"
And the structured data contains isSynced "0"

@no_rtc
Scenario: Time quality and sequence ID coexist (no RTC)
Given the syslog oracle is running
When the example program sends a syslog message
Then the structured data contains sequenceId "1"
And the structured data contains tzKnown "0"
And the structured data contains isSynced "0"
9 changes: 7 additions & 2 deletions Bdd/features/timestamp.feature
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
@udp
@freertoswip
@rtc
Feature: Timestamp encoding
The library captures the timestamp at raise-time via an injected clock
and formats it as RFC 5424 FULL-DATE "T" FULL-TIME.
and formats it as RFC 5424 FULL-DATE "T" FULL-TIME. The no-RTC product
shape (config.clock = NULL -> NILVALUE TIMESTAMP per RFC 5424 §6.2.3.1)
is covered by formatter unit tests; syslog-ng silently substitutes
receipt time for ${ISODATE} and ${S_ISODATE} when the wire TIMESTAMP
is NILVALUE, so a BDD assertion against the oracle would re-test what
the unit tests already cover, with no extra coverage.

Scenario: Timestamp is received by syslog-ng
Given the syslog oracle is running
Expand Down
48 changes: 23 additions & 25 deletions Example/FreeRtos/SingleTask/main.c
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@
#include "SolidSyslogOriginSd.h"
#include "SolidSyslogPrival.h"
#include "SolidSyslogStdAtomicOps.h"
#include "SolidSyslogTimestamp.h"
#include "SolidSyslogTimeQuality.h"
#include "SolidSyslogTimeQualitySd.h"
#include "SolidSyslogUdpSender.h"

#include <FreeRTOS.h>
Expand Down Expand Up @@ -123,19 +124,6 @@ static struct SolidSyslogMessage g_message = {
.msg = g_msg,
};

/* RFC 5424 publication date — placeholder until S08.03 slice 4+ injects a
* real RTC-backed clock callback. */
static const struct SolidSyslogTimestamp TEST_TIMESTAMP = {
.year = 2009U,
.month = 3U,
.day = 23U,
.hour = 0U,
.minute = 0U,
.second = 0U,
.microsecond = 0U,
.utcOffsetMinutes = 0,
};

/* Plus-TCP requires the network interface descriptor and its endpoint(s)
* to outlive the IP stack. */
static NetworkInterface_t networkInterface;
Expand Down Expand Up @@ -203,9 +191,17 @@ static void GetProcessId(struct SolidSyslogFormatter* formatter)
SolidSyslogFormatter_BoundedString(formatter, g_processId, strlen(g_processId));
}

static void GetTimestamp(struct SolidSyslogTimestamp* timestamp)
/* No RTC and no time-sync on this reference target — the example models an
* embedded device that has no concept of wall-clock time. RFC 5424 §6.2.3.1
* mandates NILVALUE TIMESTAMP in that case, and the timeQuality SD reports
* tzKnown=0, isSynced=0. SolidSyslogConfig.clock=NULL drops through to the
* library's NilClock; the resulting all-zero SolidSyslogTimestamp fails
* TimestampIsValid in Core/Source/SolidSyslog.c and emits "-" on the wire. */
static void GetTimeQuality(struct SolidSyslogTimeQuality* timeQuality)
{
*timestamp = TEST_TIMESTAMP;
timeQuality->tzKnown = false;
timeQuality->isSynced = false;
timeQuality->syncAccuracyMicroseconds = SOLIDSYSLOG_SYNC_ACCURACY_OMIT;
}

static void GetEndpoint(struct SolidSyslogEndpoint* endpoint)
Expand Down Expand Up @@ -345,21 +341,22 @@ static void InteractiveTask(void* argument)
.getSysUpTime = SolidSyslogFreeRtosSysUpTime_Get,
.getLanguage = ExampleLanguage_Get,
};
struct SolidSyslogStructuredData* metaSd = SolidSyslogMetaSd_Create(&metaConfig);
struct SolidSyslogOriginSdConfig originConfig = {
.software = "SolidSyslogExample",
.swVersion = "0.7.0",
.enterpriseId = EXAMPLE_ENTERPRISE_ID,
.getIpCount = ExampleIps_Count,
.getIpAt = ExampleIps_At,
struct SolidSyslogStructuredData* metaSd = SolidSyslogMetaSd_Create(&metaConfig);
struct SolidSyslogStructuredData* timeQualitySd = SolidSyslogTimeQualitySd_Create(GetTimeQuality);
struct SolidSyslogOriginSdConfig originConfig = {
.software = "SolidSyslogExample",
.swVersion = "0.7.0",
.enterpriseId = EXAMPLE_ENTERPRISE_ID,
.getIpCount = ExampleIps_Count,
.getIpAt = ExampleIps_At,
};
struct SolidSyslogStructuredData* originSd = SolidSyslogOriginSd_Create(&originConfig);
struct SolidSyslogStructuredData* sdList[] = {metaSd, originSd};
struct SolidSyslogStructuredData* sdList[] = {metaSd, timeQualitySd, originSd};

struct SolidSyslogConfig config = {
.buffer = buffer,
.sender = NULL,
.clock = GetTimestamp,
.clock = NULL,
.getHostname = GetHostname,
.getAppName = GetAppName,
.getProcessId = GetProcessId,
Expand All @@ -373,6 +370,7 @@ static void InteractiveTask(void* argument)

SolidSyslog_Destroy();
SolidSyslogOriginSd_Destroy();
SolidSyslogTimeQualitySd_Destroy();
SolidSyslogMetaSd_Destroy();
SolidSyslogAtomicCounter_Destroy();
SolidSyslogStdAtomicOps_Destroy();
Expand Down
8 changes: 4 additions & 4 deletions ci/docker-compose.bdd.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
services:
syslog-ng-linux:
image: balabit/syslog-ng:latest
image: balabit/syslog-ng:4.8.2
volumes:
- ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro
Expand Down Expand Up @@ -44,7 +44,7 @@ services:
condition: service_healthy
environment:
- BDD_TARGET=linux
command: behave --junit --junit-directory Bdd/junit --tags='not @wip' Bdd/features/
command: behave --junit --junit-directory Bdd/junit --tags='not @wip and not @no_rtc' Bdd/features/

# Each BDD target pairs with its own syslog-ng instance on its own
# bridge network so the `syslog-ng` DNS alias is scoped per-pair —
Expand All @@ -56,7 +56,7 @@ services:
# 0.0.0.0:5514. Adding more targets is one more pair + one more
# network block.
syslog-ng-freertos:
image: balabit/syslog-ng:latest
image: balabit/syslog-ng:4.8.2
volumes:
- ../Bdd/syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- ../Bdd/syslog-ng/tls:/etc/syslog-ng/tls:ro
Expand Down Expand Up @@ -101,7 +101,7 @@ services:
environment:
- BDD_TARGET=freertos
- EXAMPLE_BINARY=build/freertos-cross/Example/FreeRtos/SingleTask/SolidSyslogFreeRtosSingleTask.elf
command: behave --junit --junit-directory Bdd/junit --tags='not @wip and not @freertoswip and @udp' Bdd/features/
command: behave --junit --junit-directory Bdd/junit --tags='not @wip and not @freertoswip and not @rtc and @udp' Bdd/features/

volumes:
bdd-output-linux:
Expand Down
2 changes: 1 addition & 1 deletion docs/containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
| `ghcr.io/davidcozens/cpputest-clang` | `sha-7eac3ab` | `clang` compose service, `build-linux-clang` CI job, `analyze-iwyu` CI job |
| `ghcr.io/davidcozens/cpputest-freertos` | `sha-10a14ae` | `freertos-host` compose service, `build-freertos-host-tdd` CI job — adds FreeRTOS-Kernel / Plus-TCP / Plus-FAT / Mbed TLS sources for host-TDD of FreeRTOS adapters against fakes |
| `ghcr.io/davidcozens/cpputest-freertos-cross` | `sha-10a14ae` | `freertos-target` compose service, `build-freertos-target` CI job, `behave-freertos` BDD service, `bdd-freertos-qemu` CI job — adds `gcc-arm-none-eabi`, `libnewlib-arm-none-eabi`, `gdb-multiarch` (aliased as `arm-none-eabi-gdb`), `qemu-system-arm`, `python3` + `behave` for cross builds, on-QEMU runs, and BDD scenarios driving a QEMU target |
| `balabit/syslog-ng` | `latest` | `syslog-ng-linux` and `syslog-ng-freertos` services — BDD test oracles, one per target pair |
| `balabit/syslog-ng` | `4.8.2` | `syslog-ng-linux` and `syslog-ng-freertos` services — BDD test oracles, one per target pair. Pinned to the 4.8 LTS line; 4.11.0 (`latest` as of 2026-02-24) regressed by aborting on `STATS` over the control socket, which crashed the oracle and cascaded to the dev-container network when `freertos-target` shares the namespace. |
| `ghcr.io/davidcozens/behave` | `sha-3faff14` | `behave-linux` service — Debian trixie + Python 3.12 + Behave for Linux BDD scenarios. The FreeRTOS BDD runner uses the `cpputest-freertos-cross` image instead (which carries QEMU + Behave). |

## Docker Compose setup
Expand Down
Loading