Skip to content

Unify header information across all output formats - #106

Merged
maebli merged 1 commit into
mainfrom
fix/output-format-consistency
Jul 23, 2026
Merged

Unify header information across all output formats#106
maebli merged 1 commit into
mainfrom
fix/output-format-consistency

Conversation

@maebli

@maebli maebli commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Problem

Each output format (table, csv, json, yaml) re-implemented frame parsing, decryption and field extraction independently (~1,100 duplicated lines in mbus_data.rs), and the copies had drifted apart:

  • CSV, wireless frames: header columns were only filled from a long TPL header. Wireless frames with a short TPL header or no TPL header — the common wM-Bus case — produced empty columns, even though the table showed identification number, manufacturer, access number, status and security mode.
  • JSON/YAML: only raw internals were exposed (configuration_field: raw: 9504, manufacturer_code: code: ['E','L','S'], status: '') with no decoded security mode or readable header fields.
  • Encrypted frames without a key: wired CSV emitted garbage data points parsed from ciphertext.

Fix

One shared parse-and-decrypt path now builds a normalized FrameSummary (frame type, function, address, identification number, manufacturer incl. name lookup, access number, status, security mode, version, device type, encryption state, records). Header fields resolve long TPL header → short TPL header → wireless link layer.

  • table and csv render from the summary, so they can no longer disagree. CSV gains the missing wireless header fields, a Manufacturer Name column and one unified column set for all frame types.
  • json/yaml keep their full structured dump (backwards compatible; existing shape tests unchanged) and gain an embedded summary section with the decoded header and record values:
summary:
  frame_type: Wireless
  function: SndNk
  identification_number: '12345678'
  manufacturer:
    code: ELS
    name: Elster Group
  access_number: 42
  status: No Error(s)
  security_mode: AES-CBC-128; IV ≠ 0
  version: 51
  device_type: Gas Meter
  encrypted: true
  decrypted: true
  records:
  - value: (2850427)e-2[m³](Volume)
    data_information: 0,Inst,BCD 8-digit
    header_hex: 0C 14
    data_hex: 27 04 85 02
  • Encrypted frames without a key consistently report header info plus encrypted_payload_hex instead of garbage records.
  • Decryption logic is shared with the hexview path instead of being repeated five times.

Since CLI, wasm and Python all route through serialize_mbus_data, every deployment picks this up without changes.

Behavioral changes

  • CSV: new Manufacturer Name column; wireless rows now share the wired column set (Function/Address columns, Address empty for wireless); wireless header fields are filled.
  • Table: wireless link-layer + TPL tables merged into one; always-false Is globally Unique Id row dropped; Function uses Display instead of Debug (SndNk instead of SndNk { prm: false }); identification numbers render as digits instead of IdentificationNumber { number: … }.
  • JSON/YAML: additive summary key; manufacturer_info now also appears for wired frames when the code resolves via the short-TPL/link-layer path.

Possible follow-ups (not in this PR)

  • Deprecate the top-level manufacturer_info key in JSON/YAML in favor of summary.manufacturer (kept for now for compatibility).
  • Split summary.records[].value into structured value/exponent/unit/quantity fields instead of the table-style string, once the display format is settled.
  • Consider serializing frame.data byte arrays as hex strings in the raw dump (breaking change for JSON consumers).

Testing

  • cargo test --features std,decryption and --features std — all green (34 tests, incl. 6 new: wireless CSV header fields, encrypted-without-key CSV, decrypted CSV records, summary presence in JSON/YAML, and a cross-format agreement test that asserts every header value appears in all four formats for wired, short-TPL wireless and no-TPL wireless frames).
  • cargo check --workspace (cli, wasm, python), no-default-features build, clippy and fmt clean.

All output formats (table, csv, json, yaml) previously extracted frame
and header information independently, each with its own copy of the
parse-and-decrypt logic. The copies had drifted apart: wireless frames
with a short TPL header (the common wM-Bus case) produced empty header
columns in CSV even though the table showed the link-layer address,
manufacturer, access number, status and security mode, and the JSON and
YAML outputs exposed only raw internals with no decoded security mode
or readable header fields.

Replace the per-format duplication with a single parse path that builds
a normalized FrameSummary (frame type, function, address,
identification number, manufacturer with name lookup, access number,
status, security mode, version, device type, encryption state and data
records). Header fields resolve from the long TPL header, then the
short TPL header, then the wireless link layer.

- table and csv now render from the shared summary, so they can no
  longer disagree; csv gains the previously missing wireless header
  fields, a Manufacturer Name column and a unified column set for all
  frame types
- json and yaml keep their full structured dump for backwards
  compatibility and gain an embedded summary section with the decoded,
  human-readable header and record values
- encrypted frames without a key now consistently report the encrypted
  payload instead of emitting garbage records in csv
- decryption logic is shared with the hexview path instead of being
  repeated five times

All deployments (cli, wasm, python) route through serialize_mbus_data
and pick this up without changes.
@maebli
maebli merged commit 47c800a into main Jul 23, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant