From 00b76becaa8e7bfecf23645810e4bed6704c6566 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:17:15 +0200 Subject: [PATCH 01/13] docs: spec for always-including /tf and /tf_static in data_load_mcap Users narrowing their topic selection can leave out /tf and /tf_static, silently breaking Scene3D transform-tree rendering. Design keeps the plugin format-agnostic: a generic (topic, encoding) whitelist forces these channels selected and locked in the picker, verified against real ROS 2 bags (message_encoding=cdr, not the schema's ros2msg tag). Co-Authored-By: Claude Sonnet 5 --- ...026-07-06-mcap-always-include-tf-design.md | 153 ++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md diff --git a/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md b/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md new file mode 100644 index 0000000..bc84744 --- /dev/null +++ b/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md @@ -0,0 +1,153 @@ +# data_load_mcap: always-include /tf and /tf_static + +## Problem + +`data_load_mcap`'s topic-picker dialog treats every MCAP channel identically: +the user checks/unchecks rows, and only checked topics get parsed and pushed +into the ObjectStore. When a user narrows their selection instead of using +"Select All" — the common case when they only care about a handful of +signals — `/tf` and `/tf_static` can end up unchecked. Since PJ4's Scene3D +needs the transform tree to place `PointCloud`/`Mesh3D`/other 3D objects, +this silently breaks 3D rendering with no obvious cause. + +The plugin is meant to be format-agnostic — it must not simply hardcode +`if topic == "/tf"` as a ROS special case baked into otherwise-generic +loading logic. + +## Mechanism: generic always-include whitelist, keyed by (topic, encoding) + +A small static table of `{topic, encoding}` pairs, matched against +`ChannelInfo{topic, encoding}` — fields already populated in `analyzeFile()` +directly from the MCAP channel summary (`channel_ptr->topic`, +`channel_ptr->messageEncoding`). No parser instantiation, no schema +introspection, no ROS awareness in the mechanism itself: + +```cpp +struct AlwaysIncludeRule { + std::string topic; + std::string encoding; +}; + +const std::vector kAlwaysIncludeRules = { + {"/tf", "cdr"}, + {"/tf_static", "cdr"}, +}; +``` + +**Encoding value, verified against real ROS 2 bags:** MCAP's `Channel` +record carries `message_encoding` (the wire/serialization format of the +message bytes) separately from the `Schema` record's `encoding` (the +schema-definition language). For `tf2_msgs/msg/TFMessage` on real rosbag2 +mcap recordings (checked with the `mcap` CLI against +`~/ws_plotjuggler/DATA/amcl_test_bag.mcap` and others): + +- Channel `messageEncoding` = `cdr` — this is what `ChannelInfo.encoding` + actually stores (`mcap_dialog.hpp` prefers `channel_ptr->messageEncoding` + over the schema's encoding whenever the channel's own field is non-empty). +- Schema `encoding` = `ros2msg` — the message-definition-language tag shown + by `mcap info`'s bracketed suffix; not what `ChannelInfo.encoding` holds + for a normally-written ROS 2 bag. + +So the whitelist rule matches on `"cdr"`, not `"ros2msg"`. + +The mechanism itself is generic: `kAlwaysIncludeRules` is just data. A +future convention (a different middleware's own transform-tree topic, or +some other rendering-dependency) could add a row without touching the +enforcement logic below. ROS's `/tf`/`/tf_static` are simply the first (and +for now, only) entries. + +## Enforcement: pre-checked, cannot uncheck + +Channels matching a whitelist rule (with `msg_count > 0`) are: + +1. Unconditionally re-inserted into `selected_topics_` at every mutation + point: `analyzeFile()`'s default-select path, `onSelectionChanged`, and + both `btnSelectAll`/`btnDeselectAll` handlers in `onClicked`. +2. Rendered in `widget_data()` as both selected and disabled — reusing the + existing `setSelectedRows`/`setDisabledRows` calls already used to gray + out zero-message channels — so the checkbox/row cannot be toggled by the + user. +3. Given a `setCellTooltip` on the topic-name cell explaining why: "Always + loaded — required for 3D transform rendering." + +The correctness guarantee lives at the model layer (`selected_topics_`), +not the view: even if the widget's visual state were ever wrong, every +handler re-asserts whitelist membership before the dialog's state is +considered final, and `mcap_source.cpp`'s existing `topicFilter` (unchanged) +reads only from `selected_topics_`. + +## Known limitation, not fixed here + +`tableWidget` has `sortingEnabled=true`. Prior work already found that +`setSelectedRows`/`setDisabledRows` are index-based and desync visually +after a native column-header sort (the QUiLoader-loaded table reorders rows +without notifying the plugin), because `data_load_mcap` was deferred from +the fix applied to the ROS2/WebRTC pickers (which moved to the text-keyed +`setSelectedItems`, an SDK method that already exists). A full fix for the +disabled-rows side needs a new text-keyed `setDisabledItems` at the SDK +level (`plotjuggler_sdk` + PJ4), a larger cross-repo change. + +Because this feature's guarantee is enforced in `selected_topics_` +regardless of what the view displays, a post-sort desync can only cause a +**cosmetic** glitch (the locked look landing on the wrong row momentarily) +— never a dropped `/tf`/`/tf_static` topic. This design bundles the cheap, +already-available half of the fix (switching *selection* restore from +`setSelectedRows` to `setSelectedItems`, matching what PR #188 did for +ros2/webrtc) since it's directly relevant to a newly-added kind of +"selected" state, but leaves the disabled-rows index limitation as a +documented, pre-existing gap rather than pulling in the larger SDK change. + +## Data flow + +1. `analyzeFile()` builds `all_channels_` from the MCAP summary. +2. Any channel matching `kAlwaysIncludeRules` (with `msg_count > 0`) is + force-inserted into `selected_topics_`. +3. `widget_data()` marks those rows selected + disabled every render, with + a tooltip. +4. `onSelectionChanged` / `onClicked` (`btnSelectAll`/`btnDeselectAll`) + re-assert whitelist membership after applying the user's requested + change. +5. `loadConfig()` (restoring a previously-saved layout) also re-asserts + whitelist membership after populating `selected_topics_` from the saved + JSON, so an old saved config that predates this feature (and therefore + omits `/tf`) still gets it forced back in. +6. `mcap_source.cpp`'s `topicFilter` (unchanged) reads from + `selected_topics_` — whitelisted topics are always parsed and pushed, + feeding Scene3D via the existing `kFrameTransforms` classification in + `parser_ros`. + +## Testing + +`mcap_dialog_test.cpp` was deleted in PR #144 when its old tests (which +asserted the presence of widgets removed in that PR) went stale, and was +never replaced — `McapDialog` currently has zero direct unit coverage. The +CMake wiring for the removed target is recoverable from git history +(`git show 7f024e2 -- data_load_mcap/CMakeLists.txt`) and will be restored +with the same shape (links `plotjuggler_sdk::plugin_sdk`, `nlohmann_json`, +`LZ4::lz4_static`, `zstd::libzstd_static`, `GTest::gtest_main`). + +New coverage needed: + +- Whitelist matching: exact `(topic, encoding)` hits match; near-misses + (right topic wrong encoding, right encoding wrong topic) do not. +- `analyzeFile()` on a synthetic MCAP with a `cdr`-encoded `/tf` channel: + the topic ends up in `selected_topics_` even when `loadConfig()` supplies + a saved selection that omits it. +- `onClicked(btnDeselectAll)` and `onSelectionChanged` with `/tf` excluded + from the reported `selected` list still leave it in `selected_topics_`. +- Zero-msg-count `/tf` channel (edge case) is *not* force-included — matches + the existing "nothing to load" rule for empty channels. +- A small fixture MCAP with a `cdr`-encoded `/tf` topic will be added to + `test_data/`, following the existing pattern in + `generate_verification_mcaps.py`. + +## Non-goals + +- `data_stream_ros2` / `data_stream_foxglove_bridge`'s demand-driven + per-topic subscription work (`feat/topic-subscription`, unmerged) has the + same underlying risk for live streaming — a "required" topic may never be + dragged into a plot, so it's never subscribed. That's a separate + plugin/branch not yet merged; flagged as a related follow-up, out of + scope here. +- No glob/regex topic-name matching — exact string match only. +- No user-facing UI to edit the whitelist — it's a compiled-in table. From 7176460980b18474580a9656723f4a656175ad50 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:30:47 +0200 Subject: [PATCH 02/13] docs: switch always-include whitelist to (schema, encoding) keying MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Topic-name matching (/tf, /tf_static) breaks on real Foxglove-recorded files, which have no fixed topic-name convention for transforms (verified with the mcap CLI: a real file uses topic "tf", no leading slash). Key the whitelist on message type instead — tf2_msgs/msg/TFMessage (cdr) and foxglove.FrameTransform (protobuf), both already classified as kFrameTransforms in parser_ros/parser_protobuf — so it works regardless of topic naming. Co-Authored-By: Claude Sonnet 5 --- ...026-07-06-mcap-always-include-tf-design.md | 81 ++++++++++++------- 1 file changed, 51 insertions(+), 30 deletions(-) diff --git a/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md b/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md index bc84744..f11cd04 100644 --- a/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md +++ b/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md @@ -14,47 +14,68 @@ The plugin is meant to be format-agnostic — it must not simply hardcode `if topic == "/tf"` as a ROS special case baked into otherwise-generic loading logic. -## Mechanism: generic always-include whitelist, keyed by (topic, encoding) +## Mechanism: generic always-include whitelist, keyed by (schema name, encoding) -A small static table of `{topic, encoding}` pairs, matched against -`ChannelInfo{topic, encoding}` — fields already populated in `analyzeFile()` -directly from the MCAP channel summary (`channel_ptr->topic`, -`channel_ptr->messageEncoding`). No parser instantiation, no schema -introspection, no ROS awareness in the mechanism itself: +A small static table of `{schema_name, encoding}` pairs, matched against +`ChannelInfo{schema, encoding}` — fields already populated in `analyzeFile()` +directly from the MCAP channel/schema summary (`schema_it->second->name`, +`channel_ptr->messageEncoding`). No parser instantiation, no ROS awareness in +the mechanism itself — just the two fields the dialog already reads per +channel: ```cpp struct AlwaysIncludeRule { - std::string topic; + std::string schema_name; std::string encoding; }; const std::vector kAlwaysIncludeRules = { - {"/tf", "cdr"}, - {"/tf_static", "cdr"}, + {"tf2_msgs/msg/TFMessage", "cdr"}, // ROS 2 transform tree + {"foxglove.FrameTransform", "protobuf"}, // Foxglove-style transform tree }; ``` -**Encoding value, verified against real ROS 2 bags:** MCAP's `Channel` -record carries `message_encoding` (the wire/serialization format of the -message bytes) separately from the `Schema` record's `encoding` (the -schema-definition language). For `tf2_msgs/msg/TFMessage` on real rosbag2 -mcap recordings (checked with the `mcap` CLI against -`~/ws_plotjuggler/DATA/amcl_test_bag.mcap` and others): - -- Channel `messageEncoding` = `cdr` — this is what `ChannelInfo.encoding` - actually stores (`mcap_dialog.hpp` prefers `channel_ptr->messageEncoding` - over the schema's encoding whenever the channel's own field is non-empty). -- Schema `encoding` = `ros2msg` — the message-definition-language tag shown - by `mcap info`'s bracketed suffix; not what `ChannelInfo.encoding` holds - for a normally-written ROS 2 bag. - -So the whitelist rule matches on `"cdr"`, not `"ros2msg"`. - -The mechanism itself is generic: `kAlwaysIncludeRules` is just data. A -future convention (a different middleware's own transform-tree topic, or -some other rendering-dependency) could add a row without touching the -enforcement logic below. ROS's `/tf`/`/tf_static` are simply the first (and -for now, only) entries. +**Why schema (message type), not topic name.** The initial draft of this +design keyed on topic name (`/tf`, `/tf_static`) + encoding. Two pieces of +evidence overturned that: + +1. Foxglove's own transform message, `foxglove.FrameTransform` + (`parser_protobuf/foxglove_object_codecs.hpp`, already classified as + `kFrameTransforms` in `parser_protobuf/protobuf_parser.cpp:829-830`), is + explicitly designed to be published under *any* topic name — there is no + fixed convention the way ROS fixes `/tf`. +2. Checked with the `mcap` CLI against real Foxglove-recorded files + (`~/ws_plotjuggler/DATA/example-023-av-argo-2.mcap`): its transform + channel is topic `tf` — no leading slash — schema `foxglove.FrameTransform`, + channel `messageEncoding` = `protobuf`. A topic-name-based rule (`/tf`) + would have silently missed this real file. Matching on schema name sidesteps + topic-naming variance entirely. + +**Encoding values, verified against real bags with the `mcap` CLI:** MCAP's +`Channel` record carries `message_encoding` (the wire/serialization format of +the message bytes) separately from the `Schema` record's `encoding` (the +schema-definition language) — these are two different fields, and it's easy +to reach for the wrong one: + +- ROS 2 (`~/ws_plotjuggler/DATA/amcl_test_bag.mcap` and others): schema + `tf2_msgs/msg/TFMessage` has channel `messageEncoding` = `cdr` (what + `ChannelInfo.encoding` actually stores — `mcap_dialog.hpp` prefers + `channel_ptr->messageEncoding` over the schema's encoding whenever the + channel's own field is non-empty) and schema `encoding` = `ros2msg` (the + bracketed tag `mcap info` displays, and *not* what `ChannelInfo.encoding` + holds). +- Foxglove (`~/ws_plotjuggler/DATA/example-012-av-ds.mcap`, + `example-016-av-waymo-ds.mcap`, `example-023-av-argo-2.mcap`): schema + `foxglove.FrameTransform` has channel `messageEncoding` = `protobuf`. + +So the whitelist rules match on `"cdr"` and `"protobuf"` respectively, not +on `"ros2msg"`. + +The mechanism itself is generic: `kAlwaysIncludeRules` is just data, keyed +by message type rather than any particular naming convention. A future +message type with the same rendering-dependency problem could add a row +without touching the enforcement logic below. ROS's `tf2_msgs/msg/TFMessage` +and Foxglove's `foxglove.FrameTransform` are simply the first two entries. ## Enforcement: pre-checked, cannot uncheck From 94a87b432a4169b242f5150dc557161b5ab827f1 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:30:48 +0200 Subject: [PATCH 03/13] docs: implementation plan for always-include /tf and /tf_static Co-Authored-By: Claude Sonnet 5 --- .../2026-07-06-mcap-always-include-tf.md | 842 ++++++++++++++++++ 1 file changed, 842 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md diff --git a/docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md b/docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md new file mode 100644 index 0000000..d0c66c9 --- /dev/null +++ b/docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md @@ -0,0 +1,842 @@ +# data_load_mcap: always-include /tf and /tf_static — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make `data_load_mcap` force-include any channel whose `(schema name, encoding)` matches a known transform-tree message type (ROS 2's `tf2_msgs/msg/TFMessage` over `cdr`, Foxglove's `foxglove.FrameTransform` over `protobuf`) — pre-checked and locked in the topic picker — so 3D rendering never silently breaks because a user narrowed their topic selection, while the loader itself stays format-agnostic (the mechanism is a generic, data-driven whitelist; these two entries are just its current contents). + +**Architecture:** All changes live in `data_load_mcap/mcap_dialog.hpp` (dialog-side selection/model logic) plus one small refactor to make the existing selection-restore sort-safe. `mcap_source.cpp` needs no changes — it already reads exclusively from `McapDialog::selectedTopics()`. Enforcement lives at the model layer (`selected_topics_`), re-asserted after every mutation point, so the guarantee doesn't depend on the view staying in sync. + +**Tech Stack:** C++20, GTest, nlohmann::json, the vendored header-only `mcap` library, Python (`pip install mcap`) for generating test fixtures. + +**Spec:** `docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md` + +--- + +## Task 1: Add the whitelist test fixture + +**Files:** +- Modify: `data_load_mcap/test_data/generate_verification_mcaps.py` +- Create (generated binary artifact, to be committed): `data_load_mcap/test_data/test_dialog_whitelist.mcap` + +This fixture needs five channels covering every case the design calls out: a ROS 2 `/tf` match, a Foxglove-style match under a *non-conventional* topic name (proving the whitelist isn't topic-name-based), a whitelisted schema/encoding with zero messages (must NOT force-include), a near-miss (right schema, wrong encoding), and an ordinary unaffected channel. + +- [ ] **Step 1: Add schema constants and the generator function** + +Add near the top of `generate_verification_mcaps.py`, after the existing `SCHEMA_NESTED` constant (around line 70): + +```python +SCHEMA_TF2_MSGS = ( + b"geometry_msgs/TransformStamped[] transforms\n" + b"================\n" + b"MSG: geometry_msgs/TransformStamped\n" + b"Header header\n" + b"string child_frame_id\n" + b"Transform transform\n" + b"================\n" + b"MSG: std_msgs/Header\n" + b"int32 sec\n" + b"uint32 nanosec\n" + b"string frame_id\n" + b"================\n" + b"MSG: geometry_msgs/Transform\n" + b"Vector3 translation\n" + b"Quaternion rotation\n" +) + +SCHEMA_FOXGLOVE_FRAME_TRANSFORM_PROTO = ( + b'syntax = "proto3";\n' + b"package foxglove;\n" + b"message FrameTransform {\n" + b" string parent_frame_id = 2;\n" + b" string child_frame_id = 3;\n" + b"}\n" +) +``` + +Add the generator function after `gen_embedded_timestamp()` (around line 146, before the "Test 3: Large arrays" section): + +```python +# --------------------------------------------------------------------------- +# Test: always-include whitelist (schema + encoding based, not topic name) +# --------------------------------------------------------------------------- + +def gen_dialog_whitelist(): + path = OUT / "test_dialog_whitelist.mcap" + n_msgs = 5 + dt_ns = 100_000_000 + + with open(path, "wb") as f: + w = Writer(f) + w.start(profile="ros2", library="pj-verification") + + # 1. ROS 2 /tf -- whitelisted (schema tf2_msgs/msg/TFMessage, encoding cdr). + schema_tf2 = w.register_schema( + name="tf2_msgs/msg/TFMessage", encoding="ros2msg", data=SCHEMA_TF2_MSGS, + ) + ch_tf = w.register_channel(topic="/tf", message_encoding="cdr", schema_id=schema_tf2) + + # 2. Foxglove transform under a non-conventional topic name -- whitelisted + # (schema foxglove.FrameTransform, encoding protobuf). Proves the match + # is on message type, not on topic naming: real Foxglove-recorded files + # use topic names like plain "tf" (no leading slash) or arbitrary names. + schema_foxglove = w.register_schema( + name="foxglove.FrameTransform", encoding="protobuf", + data=SCHEMA_FOXGLOVE_FRAME_TRANSFORM_PROTO, + ) + ch_foxglove = w.register_channel( + topic="transforms", message_encoding="protobuf", schema_id=schema_foxglove, + ) + + # 3. /tf_static -- same whitelisted schema/encoding, but zero messages: + # must NOT be force-included (nothing to load). + _ = w.register_channel(topic="/tf_static", message_encoding="cdr", schema_id=schema_tf2) + + # 4. Near miss: right schema name, wrong encoding -- must NOT be force-included. + schema_tf2_json = w.register_schema( + name="tf2_msgs/msg/TFMessage", encoding="jsonschema", data=SCHEMA_TF2_MSGS, + ) + ch_near_miss_encoding = w.register_channel( + topic="/near_miss_encoding", message_encoding="json", schema_id=schema_tf2_json, + ) + + # 5. Ordinary channel, unaffected by the whitelist. + schema_float = w.register_schema( + name="std_msgs/Float64", encoding="ros2msg", data=SCHEMA_FLOAT64, + ) + ch_ordinary = w.register_channel( + topic="/sensor/value2", message_encoding="cdr", schema_id=schema_float, + ) + + for i in range(n_msgs): + ts = i * dt_ns + w.add_message(channel_id=ch_tf, log_time=ts, publish_time=ts, data=cdr_float64(1.0)) + # Dummy protobuf bytes -- doesn't need to be valid, the dialog never + # decodes message content, only reads channel/schema metadata. + w.add_message(channel_id=ch_foxglove, log_time=ts, publish_time=ts, data=b"\x00" * 8) + w.add_message( + channel_id=ch_near_miss_encoding, log_time=ts, publish_time=ts, data=b'{"x":1}', + ) + w.add_message(channel_id=ch_ordinary, log_time=ts, publish_time=ts, data=cdr_float64(2.0)) + + w.finish() + print(f"[OK] {path.name:45s} {path.stat().st_size:>8} bytes") +``` + +- [ ] **Step 2: Call the generator from `__main__`** + +In the `if __name__ == "__main__":` block, add the call (order doesn't matter; place after `gen_embedded_timestamp()`): + +```python + gen_publish_vs_log_time() + gen_embedded_timestamp() + gen_dialog_whitelist() + gen_large_arrays() +``` + +- [ ] **Step 3: Generate the fixture and verify its shape** + +Run: +```bash +cd data_load_mcap/test_data +python3 generate_verification_mcaps.py +``` +Expected: a line `[OK] test_dialog_whitelist.mcap ...` among the output, and the file exists. + +Then verify the channel table with the `mcap` CLI: +```bash +~/Apps/mcap-linux-amd64 info test_dialog_whitelist.mcap +``` +Expected: 5 channels — `/tf` (`tf2_msgs/msg/TFMessage [ros2msg]`), `transforms` (`foxglove.FrameTransform [protobuf]`), `/tf_static` (0 msgs), `/near_miss_encoding`, `/sensor/value2` — each with 5 messages except `/tf_static` (0). + +- [ ] **Step 4: Commit** + +```bash +cd /home/davide/ws_plotjuggler/pj-official-plugins/.worktrees/mcap-always-include-tf +git add data_load_mcap/test_data/generate_verification_mcaps.py data_load_mcap/test_data/test_dialog_whitelist.mcap +git commit -m "test(data_load_mcap): add whitelist fixture (ROS2 tf + Foxglove FrameTransform)" +``` + +--- + +## Task 2: Restore the `mcap_dialog_test` CMake target + +**Files:** +- Modify: `data_load_mcap/CMakeLists.txt` +- Create: `data_load_mcap/tests/mcap_dialog_test.cpp` + +`McapDialog` currently has zero direct unit coverage — its test target was deleted in PR #144 when the tests it had went stale (they asserted widgets that PR removed). The CMake wiring is recoverable verbatim from git history. + +- [ ] **Step 1: Write the initial test file** + +Create `data_load_mcap/tests/mcap_dialog_test.cpp`: + +```cpp +#define MCAP_IMPLEMENTATION +#include "mcap_dialog.hpp" + +#include + +#include +#include + +namespace { + +TEST(McapDialogTest, DefaultConstructs) { + McapDialog dialog; + EXPECT_TRUE(dialog.selectedTopics().empty()); + EXPECT_TRUE(dialog.analyzeError().empty()); +} + +} // namespace +``` + +- [ ] **Step 2: Restore the CMake target** + +In `data_load_mcap/CMakeLists.txt`, insert this block between the `mcap_helpers_test` block and the `message_byte_store_test` block (i.e. right after the `add_test(NAME mcap_helpers_test COMMAND mcap_helpers_test)` line, before the `# --- Unit test for the MessageByteStore` comment): + +```cmake +# --- Unit test for MCAP dialog model/config behavior --- +add_executable(mcap_dialog_test tests/mcap_dialog_test.cpp) +target_compile_features(mcap_dialog_test PRIVATE cxx_std_20) +target_compile_options(mcap_dialog_test PRIVATE ${PJ_WARNING_FLAGS}) +target_compile_definitions(mcap_dialog_test PRIVATE + MCAP_TEST_DATA_DIR="${CMAKE_CURRENT_SOURCE_DIR}/test_data" +) +target_include_directories(mcap_dialog_test PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR} + ${CMAKE_CURRENT_BINARY_DIR}/generated +) +target_include_directories(mcap_dialog_test SYSTEM PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/contrib) +target_link_libraries(mcap_dialog_test PRIVATE + plotjuggler_sdk::plugin_sdk nlohmann_json::nlohmann_json LZ4::lz4_static zstd::libzstd_static GTest::gtest_main +) +add_test(NAME mcap_dialog_test COMMAND mcap_dialog_test) +``` + +- [ ] **Step 3: Build and run** + +Run: +```bash +cd /home/davide/ws_plotjuggler/pj-official-plugins/.worktrees/mcap-always-include-tf +./build.sh data_load_mcap +ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: build succeeds, `mcap_dialog_test` passes (`DefaultConstructs`). + +- [ ] **Step 4: Commit** + +```bash +git add data_load_mcap/CMakeLists.txt data_load_mcap/tests/mcap_dialog_test.cpp +git commit -m "test(data_load_mcap): restore mcap_dialog_test target with a baseline smoke test" +``` + +--- + +## Task 3: Make selection restore sort-safe (text-keyed, not index-keyed) + +**Files:** +- Modify: `data_load_mcap/mcap_dialog.hpp:89-111` (inside `widget_data()`) +- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` + +`tableWidget` has `sortingEnabled=true`; restoring selection by row index (`setSelectedRows`) desyncs the moment the host's native column sort reorders rows, because the plugin's row vector stays in its own order. `setSelectedItems` (text-keyed, matched by the host against column-0 text) already exists in the SDK and was just applied to `data_stream_ros2`/`data_stream_webrtc`'s pickers on the (unmerged) `feat/unify-picker-tables` branch — same fix, applied here directly since the SDK method itself is already available. This also gives the whitelist feature (next tasks) a sort-safe foundation to lock rows on top of. + +- [ ] **Step 1: Write the failing test** + +Append to `data_load_mcap/tests/mcap_dialog_test.cpp` (inside the anonymous namespace): + +```cpp +TEST(McapDialogTest, WidgetDataSelectsRowsByTextNotIndex) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_publish_vs_log_time.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + const auto data = nlohmann::json::parse(dialog.widget_data()); + ASSERT_TRUE(data.contains("tableWidget")); + const auto& tbl = data["tableWidget"]; + + ASSERT_TRUE(tbl.contains("selected_items")); + std::vector selected_items = tbl["selected_items"].get>(); + EXPECT_EQ(selected_items, (std::vector{"/sensor/value"})); + + EXPECT_FALSE(tbl.contains("selected_rows")); +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: FAIL on `WidgetDataSelectsRowsByTextNotIndex` — `tbl` has `"selected_rows"`, not `"selected_items"`. + +- [ ] **Step 3: Implement the refactor** + +In `data_load_mcap/mcap_dialog.hpp`, inside `widget_data()`, replace: + +```cpp + std::vector> rows; + std::vector selected_row_indices; + std::vector disabled_row_indices; + rows.reserve(filtered.size()); + + for (size_t i = 0; i < filtered.size(); ++i) { + const auto& ch = *filtered[i]; + rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); + if (selected_topics_.count(ch.topic) > 0) { + selected_row_indices.push_back(static_cast(i)); + } + if (ch.msg_count == 0) { + disabled_row_indices.push_back(static_cast(i)); + } + } + wd.setTableRows("tableWidget", rows); + wd.setDisabledRows("tableWidget", disabled_row_indices); + wd.setSelectedRows("tableWidget", selected_row_indices); +``` + +with: + +```cpp + std::vector> rows; + std::vector selected_topic_names; + std::vector disabled_row_indices; + rows.reserve(filtered.size()); + + for (size_t i = 0; i < filtered.size(); ++i) { + const auto& ch = *filtered[i]; + rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); + if (selected_topics_.count(ch.topic) > 0) { + selected_topic_names.push_back(ch.topic); + } + if (ch.msg_count == 0) { + disabled_row_indices.push_back(static_cast(i)); + } + } + wd.setTableRows("tableWidget", rows); + wd.setDisabledRows("tableWidget", disabled_row_indices); + // Restore selection by first-column text (the topic name), not row index: + // the host matches items by text, which is sort-agnostic, so the selection + // survives the table's built-in column sorting (sortingEnabled=true). + wd.setSelectedItems("tableWidget", selected_topic_names); +``` + +(`disabled_row_indices` stays index-based — there is no text-keyed `setDisabledItems` in the SDK yet; see the design doc's "Known limitation" section.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: PASS, all tests green. + +- [ ] **Step 5: Commit** + +```bash +git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp +git commit -m "fix(data_load_mcap): restore selection by text, sort-safe like ros2/webrtc pickers" +``` + +--- + +## Task 4: Whitelist matching helper + +**Files:** +- Modify: `data_load_mcap/mcap_dialog.hpp:14-25` (anonymous namespace, after `ChannelInfo`) +- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` + +- [ ] **Step 1: Write the failing tests** + +Append to `data_load_mcap/tests/mcap_dialog_test.cpp`: + +```cpp +TEST(AlwaysIncludeRuleTest, MatchesRos2Tf) { + ChannelInfo ch; + ch.topic = "/tf"; + ch.schema = "tf2_msgs/msg/TFMessage"; + ch.encoding = "cdr"; + ch.msg_count = 10; + EXPECT_TRUE(isAlwaysIncluded(ch)); +} + +TEST(AlwaysIncludeRuleTest, MatchesFoxgloveFrameTransformUnderAnyTopicName) { + ChannelInfo ch; + ch.topic = "transforms"; // deliberately not "/tf" + ch.schema = "foxglove.FrameTransform"; + ch.encoding = "protobuf"; + ch.msg_count = 10; + EXPECT_TRUE(isAlwaysIncluded(ch)); +} + +TEST(AlwaysIncludeRuleTest, RejectsRightSchemaWrongEncoding) { + ChannelInfo ch; + ch.topic = "/tf"; + ch.schema = "tf2_msgs/msg/TFMessage"; + ch.encoding = "json"; + ch.msg_count = 10; + EXPECT_FALSE(isAlwaysIncluded(ch)); +} + +TEST(AlwaysIncludeRuleTest, RejectsRightEncodingWrongSchema) { + ChannelInfo ch; + ch.topic = "/sensor/value"; + ch.schema = "std_msgs/Float64"; + ch.encoding = "cdr"; + ch.msg_count = 10; + EXPECT_FALSE(isAlwaysIncluded(ch)); +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: FAIL to compile — `isAlwaysIncluded` is not defined. + +- [ ] **Step 3: Implement the whitelist** + +In `data_load_mcap/mcap_dialog.hpp`, right after the `ChannelInfo` struct definition (after line 25, before `class McapDialog`), add: + +```cpp +struct AlwaysIncludeRule { + std::string schema_name; + std::string encoding; +}; + +/// Channels whose (schema, encoding) match a rule here are always loaded, +/// regardless of user selection -- 3D rendering (Scene3D's transform tree) +/// silently breaks without them. Keyed on message type rather than topic +/// name because topic naming isn't consistent across producers: Foxglove's +/// own foxglove.FrameTransform has no fixed topic-name convention the way +/// ROS fixes /tf (real Foxglove-recorded files use topic names like plain +/// "tf", no leading slash, or something else entirely). The mechanism here +/// is generic -- this table is just its (currently two-entry) data. +const std::vector kAlwaysIncludeRules = { + {"tf2_msgs/msg/TFMessage", "cdr"}, + {"foxglove.FrameTransform", "protobuf"}, +}; + +bool isAlwaysIncluded(const ChannelInfo& ch) { + for (const auto& rule : kAlwaysIncludeRules) { + if (ch.schema == rule.schema_name && ch.encoding == rule.encoding) { + return true; + } + } + return false; +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: PASS, all tests green. + +- [ ] **Step 5: Commit** + +```bash +git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp +git commit -m "feat(data_load_mcap): add (schema, encoding) always-include whitelist" +``` + +--- + +## Task 5: Force-include whitelisted channels in `analyzeFile()` + +**Files:** +- Modify: `data_load_mcap/mcap_dialog.hpp:326-335` (end of `analyzeFile()`) +- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` + +This is the core guarantee: whitelisted channels end up in `selected_topics_` even when a saved config's `selected_topics` predates this feature and omits them. Because `loadConfig()` already calls `analyzeFile()` as its last step, this single change also covers the `loadConfig()` path from the design doc. + +- [ ] **Step 1: Write the failing test** + +Append to `data_load_mcap/tests/mcap_dialog_test.cpp`: + +```cpp +TEST(McapDialogTest, ForceIncludesOnlyWhitelistedNonEmptyChannels) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + // Saved selection predates this feature and omits everything except one + // ordinary topic -- as if loading an old layout. + cfg["selected_topics"] = std::vector{"/sensor/value2"}; + + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + const auto& selected = dialog.selectedTopics(); + EXPECT_TRUE(selected.count("/tf") > 0); // whitelisted, has messages -> forced back in + EXPECT_TRUE(selected.count("transforms") > 0); // whitelisted (any topic name) -> forced back in + EXPECT_EQ(selected.count("/tf_static"), 0u); // whitelisted schema/encoding, zero messages -> NOT forced + EXPECT_EQ(selected.count("/near_miss_encoding"), 0u); // right schema, wrong encoding -> NOT forced + EXPECT_TRUE(selected.count("/sensor/value2") > 0); // explicitly saved by the user -> stays +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: FAIL — `/tf` and `transforms` are absent from `selectedTopics()` (the saved selection omitted them and nothing adds them back yet). + +- [ ] **Step 3: Implement `reassertAlwaysIncluded()` and call it from `analyzeFile()`** + +In `data_load_mcap/mcap_dialog.hpp`, change the end of `analyzeFile()` from: + +```cpp + // If no previous selection, select all channels with messages + if (selected_topics_.empty()) { + for (const auto& ch : all_channels_) { + if (ch.msg_count > 0) { + selected_topics_.insert(ch.topic); + } + } + } + } +``` + +to: + +```cpp + // If no previous selection, select all channels with messages + if (selected_topics_.empty()) { + for (const auto& ch : all_channels_) { + if (ch.msg_count > 0) { + selected_topics_.insert(ch.topic); + } + } + } + + reassertAlwaysIncluded(); + } + + /// Channels matching kAlwaysIncludeRules are always loaded: 3D rendering + /// depends on them even when the user narrows their selection to a handful + /// of unrelated topics. Idempotent -- safe to call after any mutation of + /// selected_topics_. + void reassertAlwaysIncluded() { + for (const auto& ch : all_channels_) { + if (ch.msg_count > 0 && isAlwaysIncluded(ch)) { + selected_topics_.insert(ch.topic); + } + } + } +``` + +(Place `reassertAlwaysIncluded()` as a new private method right after `analyzeFile()`, before `filteredChannels()`.) + +- [ ] **Step 4: Run test to verify it passes** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: PASS, all tests green. + +- [ ] **Step 5: Commit** + +```bash +git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp +git commit -m "feat(data_load_mcap): force-include whitelisted channels after analyzeFile()" +``` + +--- + +## Task 6: Lock whitelisted rows in the picker (disabled + tooltip) + +**Files:** +- Modify: `data_load_mcap/mcap_dialog.hpp:89-111` (inside `widget_data()`, the loop introduced/changed in Task 3) +- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` + +- [ ] **Step 1: Write the failing test** + +Append to `data_load_mcap/tests/mcap_dialog_test.cpp` (add `#include ` to the top of the file alongside the other includes): + +```cpp +TEST(McapDialogTest, WhitelistedRowsAreDisabledAndTooltippedInWidgetData) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + const auto data = nlohmann::json::parse(dialog.widget_data()); + const auto& rows = data["tableWidget"]["rows"]; + + int tf_row = -1; + int ordinary_row = -1; + for (size_t i = 0; i < rows.size(); ++i) { + if (rows[i][0] == "/tf") { + tf_row = static_cast(i); + } + if (rows[i][0] == "/sensor/value2") { + ordinary_row = static_cast(i); + } + } + ASSERT_NE(tf_row, -1); + ASSERT_NE(ordinary_row, -1); + + const auto& disabled = data["tableWidget"]["disabled_rows"]; + EXPECT_NE(std::find(disabled.begin(), disabled.end(), tf_row), disabled.end()); + EXPECT_EQ(std::find(disabled.begin(), disabled.end(), ordinary_row), disabled.end()); + + ASSERT_TRUE(data["tableWidget"].contains("cell_tooltips")); + const auto& tooltips = data["tableWidget"]["cell_tooltips"]; + EXPECT_TRUE(tooltips.contains(std::to_string(tf_row) + ",0")); +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: FAIL — the `/tf` row index is absent from `disabled_rows`, and `cell_tooltips` is missing. + +- [ ] **Step 3: Implement the lock** + +In `data_load_mcap/mcap_dialog.hpp`, inside `widget_data()`, change the per-row loop from: + +```cpp + for (size_t i = 0; i < filtered.size(); ++i) { + const auto& ch = *filtered[i]; + rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); + if (selected_topics_.count(ch.topic) > 0) { + selected_topic_names.push_back(ch.topic); + } + if (ch.msg_count == 0) { + disabled_row_indices.push_back(static_cast(i)); + } + } +``` + +to: + +```cpp + for (size_t i = 0; i < filtered.size(); ++i) { + const auto& ch = *filtered[i]; + rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); + if (selected_topics_.count(ch.topic) > 0) { + selected_topic_names.push_back(ch.topic); + } + bool locked_always_included = ch.msg_count > 0 && isAlwaysIncluded(ch); + if (ch.msg_count == 0 || locked_always_included) { + disabled_row_indices.push_back(static_cast(i)); + } + if (locked_always_included) { + wd.setCellTooltip("tableWidget", static_cast(i), 0, + "Always loaded — required for 3D transform rendering."); + } + } +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: PASS, all tests green. + +- [ ] **Step 5: Commit** + +```bash +git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp +git commit -m "feat(data_load_mcap): lock whitelisted rows (disabled + tooltip) in the picker" +``` + +--- + +## Task 7: Survive `Deselect All` and host-reported selection changes + +**Files:** +- Modify: `data_load_mcap/mcap_dialog.hpp:182-208` (`onSelectionChanged`, `onClicked`) +- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` + +`onSelectionChanged` fully replaces `selected_topics_` with whatever the host reports, and `btnDeselectAll` clears it outright. Both need to re-assert whitelist membership afterward. (`btnSelectAll` doesn't need this: it only *adds* to `selected_topics_`, and whitelisted topics are already present from `reassertAlwaysIncluded()`, so the union can't drop them.) + +- [ ] **Step 1: Write the failing tests** + +Append to `data_load_mcap/tests/mcap_dialog_test.cpp`: + +```cpp +TEST(McapDialogTest, DeselectAllKeepsWhitelistedTopicsSelected) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + EXPECT_TRUE(dialog.onClicked("btnDeselectAll")); + + const auto& selected = dialog.selectedTopics(); + EXPECT_TRUE(selected.count("/tf") > 0); + EXPECT_TRUE(selected.count("transforms") > 0); + EXPECT_EQ(selected.count("/sensor/value2"), 0u); +} + +TEST(McapDialogTest, HostReportedSelectionOmittingWhitelistIsOverridden) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + // Simulate the host reporting a selection that omits /tf and transforms -- + // the dialog must add them back regardless. + EXPECT_TRUE(dialog.onSelectionChanged("tableWidget", {"/sensor/value2"})); + + const auto& selected = dialog.selectedTopics(); + EXPECT_TRUE(selected.count("/tf") > 0); + EXPECT_TRUE(selected.count("transforms") > 0); + EXPECT_TRUE(selected.count("/sensor/value2") > 0); +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: FAIL — both tests find `/tf` and `transforms` missing from `selectedTopics()`. + +- [ ] **Step 3: Re-assert after both mutation points** + +In `data_load_mcap/mcap_dialog.hpp`, change: + +```cpp + bool onSelectionChanged(std::string_view widget_name, const std::vector& selected) override { + if (widget_name == "tableWidget") { + selected_topics_.clear(); + for (const auto& topic : selected) { + selected_topics_.insert(topic); + } + return true; // update OK button state + } + return false; + } + + bool onClicked(std::string_view widget_name) override { + if (widget_name == "btnSelectAll") { + auto filtered = filteredChannels(); + for (const auto* ch : filtered) { + if (ch->msg_count > 0) { + selected_topics_.insert(ch->topic); + } + } + return true; + } + if (widget_name == "btnDeselectAll") { + selected_topics_.clear(); + return true; + } + return false; + } +``` + +to: + +```cpp + bool onSelectionChanged(std::string_view widget_name, const std::vector& selected) override { + if (widget_name == "tableWidget") { + selected_topics_.clear(); + for (const auto& topic : selected) { + selected_topics_.insert(topic); + } + reassertAlwaysIncluded(); + return true; // update OK button state + } + return false; + } + + bool onClicked(std::string_view widget_name) override { + if (widget_name == "btnSelectAll") { + auto filtered = filteredChannels(); + for (const auto* ch : filtered) { + if (ch->msg_count > 0) { + selected_topics_.insert(ch->topic); + } + } + return true; + } + if (widget_name == "btnDeselectAll") { + selected_topics_.clear(); + reassertAlwaysIncluded(); + return true; + } + return false; + } +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: +```bash +./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V +``` +Expected: PASS, all tests green. + +- [ ] **Step 5: Commit** + +```bash +git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp +git commit -m "fix(data_load_mcap): keep whitelisted topics selected through Deselect All / host selection changes" +``` + +--- + +## Task 8: Document the behavior and run the full suite + +**Files:** +- Modify: `data_load_mcap/README.md` + +- [ ] **Step 1: Add a documentation section** + +In `data_load_mcap/README.md`, after the "## Channel discovery" section (after line 76, before "## How parsers see this loader"), add: + +```markdown +## Always-included channels + +Channels whose `(schema name, encoding)` matches a small built-in whitelist +are always loaded, pre-checked and locked in the picker, regardless of the +user's selection — because PJ4's Scene3D needs them to place other 3D +objects (`PointCloud`/`Mesh3D`) and silently breaks without them. Today the +whitelist covers ROS 2's transform tree (`tf2_msgs/msg/TFMessage` over +`cdr`) and Foxglove's (`foxglove.FrameTransform` over `protobuf`). The match +is on message type, not topic name — Foxglove's convention has no fixed +topic name the way ROS fixes `/tf`. This keeps the loader itself +format-agnostic: the whitelist is just data (`kAlwaysIncludeRules` in +`mcap_dialog.hpp`), not a ROS-specific code path. +``` + +- [ ] **Step 2: Run the full plugin test suite** + +Run: +```bash +cd /home/davide/ws_plotjuggler/pj-official-plugins/.worktrees/mcap-always-include-tf +./build.sh data_load_mcap +ctest --test-dir build -R mcap -V +``` +Expected: `mcap_dialog_test`, `mcap_helpers_test`, `message_byte_store_test` all PASS. + +- [ ] **Step 3: Commit** + +```bash +git add data_load_mcap/README.md +git commit -m "docs(data_load_mcap): document the always-include whitelist" +``` + +--- + +## Self-Review Notes + +- **Spec coverage:** Problem/mechanism (Tasks 4-5), enforcement/pre-checked-locked (Tasks 5-6-7), known limitation (Task 3 covers the cheap half; the disabled-rows index limitation is intentionally left as-is, matching the design doc's non-goal), data flow items 1-6 (Tasks 5-7, `mcap_source.cpp` untouched per item 6), testing (Tasks 1-7 fixture + coverage), non-goals (nothing added for ROS1, glob matching, or a whitelist-editing UI — none of the tasks touch those). +- **Type consistency:** `AlwaysIncludeRule{schema_name, encoding}`, `kAlwaysIncludeRules`, `isAlwaysIncluded(const ChannelInfo&)`, and `reassertAlwaysIncluded()` are named identically everywhere they're introduced (Task 4, 5) and consumed (Task 6, 7). +- **No placeholders:** every step shows the exact code to write; no "add appropriate handling" steps. From 488803ca4358ccc4d9e0c0c125d7adacd075bcfb Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:37:57 +0200 Subject: [PATCH 04/13] test(data_load_mcap): add whitelist fixture (ROS2 tf + Foxglove FrameTransform) Co-Authored-By: Claude Sonnet 5 --- .../test_data/generate_verification_mcaps.py | 97 ++++++++++++++++++ .../test_data/test_dialog_whitelist.mcap | Bin 0 -> 2682 bytes 2 files changed, 97 insertions(+) create mode 100644 data_load_mcap/test_data/test_dialog_whitelist.mcap diff --git a/data_load_mcap/test_data/generate_verification_mcaps.py b/data_load_mcap/test_data/generate_verification_mcaps.py index d1e6af1..68c8220 100644 --- a/data_load_mcap/test_data/generate_verification_mcaps.py +++ b/data_load_mcap/test_data/generate_verification_mcaps.py @@ -7,6 +7,9 @@ Files generated: test_publish_vs_log_time.mcap -- Test 1: publish_time != log_time (offset 500ms) test_embedded_timestamp.mcap -- Test 2: Header.stamp differs from MCAP timestamp (+2s) + test_dialog_whitelist.mcap -- Test: always-include whitelist (tf2_msgs/TFMessage + + foxglove.FrameTransform matches, empty channel, near + miss, and ordinary channel) test_large_arrays.mcap -- Test 3: float64[1000] arrays for clamp/skip testing test_empty_channel.mcap -- Test 6: one channel with 0 messages test_unsupported_encoding.mcap -- Test 7: channel with encoding "protobuf" (no parser) @@ -69,6 +72,33 @@ def cdr_float64_fixed_array(values: list) -> bytes: b"string frame_id\n" ) +SCHEMA_TF2_MSGS = ( + b"geometry_msgs/TransformStamped[] transforms\n" + b"================\n" + b"MSG: geometry_msgs/TransformStamped\n" + b"Header header\n" + b"string child_frame_id\n" + b"Transform transform\n" + b"================\n" + b"MSG: std_msgs/Header\n" + b"int32 sec\n" + b"uint32 nanosec\n" + b"string frame_id\n" + b"================\n" + b"MSG: geometry_msgs/Transform\n" + b"Vector3 translation\n" + b"Quaternion rotation\n" +) + +SCHEMA_FOXGLOVE_FRAME_TRANSFORM_PROTO = ( + b'syntax = "proto3";\n' + b"package foxglove;\n" + b"message FrameTransform {\n" + b" string parent_frame_id = 2;\n" + b" string child_frame_id = 3;\n" + b"}\n" +) + OUT = Path(__file__).parent @@ -145,6 +175,72 @@ def gen_embedded_timestamp(): print(f"[OK] {path.name:45s} {path.stat().st_size:>8} bytes") +# --------------------------------------------------------------------------- +# Test: always-include whitelist (schema + encoding based, not topic name) +# --------------------------------------------------------------------------- + +def gen_dialog_whitelist(): + path = OUT / "test_dialog_whitelist.mcap" + n_msgs = 5 + dt_ns = 100_000_000 + + with open(path, "wb") as f: + w = Writer(f) + w.start(profile="ros2", library="pj-verification") + + # 1. ROS 2 /tf -- whitelisted (schema tf2_msgs/msg/TFMessage, encoding cdr). + schema_tf2 = w.register_schema( + name="tf2_msgs/msg/TFMessage", encoding="ros2msg", data=SCHEMA_TF2_MSGS, + ) + ch_tf = w.register_channel(topic="/tf", message_encoding="cdr", schema_id=schema_tf2) + + # 2. Foxglove transform under a non-conventional topic name -- whitelisted + # (schema foxglove.FrameTransform, encoding protobuf). Proves the match + # is on message type, not on topic naming: real Foxglove-recorded files + # use topic names like plain "tf" (no leading slash) or arbitrary names. + schema_foxglove = w.register_schema( + name="foxglove.FrameTransform", encoding="protobuf", + data=SCHEMA_FOXGLOVE_FRAME_TRANSFORM_PROTO, + ) + ch_foxglove = w.register_channel( + topic="transforms", message_encoding="protobuf", schema_id=schema_foxglove, + ) + + # 3. /tf_static -- same whitelisted schema/encoding, but zero messages: + # must NOT be force-included (nothing to load). + _ = w.register_channel(topic="/tf_static", message_encoding="cdr", schema_id=schema_tf2) + + # 4. Near miss: right schema name, wrong encoding -- must NOT be force-included. + schema_tf2_json = w.register_schema( + name="tf2_msgs/msg/TFMessage", encoding="jsonschema", data=SCHEMA_TF2_MSGS, + ) + ch_near_miss_encoding = w.register_channel( + topic="/near_miss_encoding", message_encoding="json", schema_id=schema_tf2_json, + ) + + # 5. Ordinary channel, unaffected by the whitelist. + schema_float = w.register_schema( + name="std_msgs/Float64", encoding="ros2msg", data=SCHEMA_FLOAT64, + ) + ch_ordinary = w.register_channel( + topic="/sensor/value2", message_encoding="cdr", schema_id=schema_float, + ) + + for i in range(n_msgs): + ts = i * dt_ns + w.add_message(channel_id=ch_tf, log_time=ts, publish_time=ts, data=cdr_float64(1.0)) + # Dummy protobuf bytes -- doesn't need to be valid, the dialog never + # decodes message content, only reads channel/schema metadata. + w.add_message(channel_id=ch_foxglove, log_time=ts, publish_time=ts, data=b"\x00" * 8) + w.add_message( + channel_id=ch_near_miss_encoding, log_time=ts, publish_time=ts, data=b'{"x":1}', + ) + w.add_message(channel_id=ch_ordinary, log_time=ts, publish_time=ts, data=cdr_float64(2.0)) + + w.finish() + print(f"[OK] {path.name:45s} {path.stat().st_size:>8} bytes") + + # --------------------------------------------------------------------------- # Test 3: Large arrays # --------------------------------------------------------------------------- @@ -354,6 +450,7 @@ def gen_large_file(): gen_publish_vs_log_time() gen_embedded_timestamp() + gen_dialog_whitelist() gen_large_arrays() gen_empty_channel() gen_unsupported_encoding() diff --git a/data_load_mcap/test_data/test_dialog_whitelist.mcap b/data_load_mcap/test_data/test_dialog_whitelist.mcap new file mode 100644 index 0000000000000000000000000000000000000000..a744138035172336a30c1fd4d6b19cc12b8bb402 GIT binary patch literal 2682 zcmeHJi&Io(6u&)!WLOP6HE$Z=6P#*-8XhGtdcfV3CQb5e}OWHFGOg|_)sh*6Lr+*iZk^86Uy z(CGGm2=~9na>lhNR?(*Xd);}z-9cD<&+0{)#rpztBNyNLW}*H}Y-KemZ|NI2esQt$ z*Ziz-3mKdm6CdQ5GwJT;|WA~Qt^LJdBemehXnmgm3+pqTX z<@I~i2huc$R-bF`Xmm_z{;2ct*oh&Uvy!xoNW7&LFm_e`dB?UAJs=)qq_*ht2&xi0Fv>`pVW=anv$mhcr4}apvwd zkvs0$uS)Qou2&8x?0+?EXR=s5M;`jltkPp|R?II>`}hyKV?ROI9~50O$wLd(iawy){g{{9dXvXnUl-x~U^s*R&z<^vk07gpi(HD*eIm8?9S{ z>s%ds-P>LaN}btDWL)f?Tu`N$RQEn-zFU5-x}h?Zl3PBAY2CiDwPE5hjekP@tbzx1 zc2)G_fIdx5WzB)8m5Oh#geE+vQ-P^0$ZCR`~ ztLwx*X4RNHzV9#*mWftS)WRNw)c;ThMY4y3MaP{0sNWRy`u+(}ukeE$5NhXW)XrXb z@hJFCA9$+(hv3Hxa0vdWpdXAN9ND~%LQY3-`0~ZkrIA>DntgM+;0;bH>W>73knkl#uP8ML) zfN(QKTPcol@sYU{+UqZu`yPl;#l;#m%oqZk_)!30BdyiL*s>kUYZ!5M6#OU)&;lbWxEkLGO_14F0rB>kvyeGaFAmWOWRmOA!Q~1Z<8nW_w1kVN1o0aD4CpfDI2Xq84n4oD1s@ZN^s*O6TZ zBc#(H2P6SVFccu^SlGyhk$|&3F)&pL77U|o6v^nU7M9gfHUn)0?=KuWus>j=Gy-x! zGO!~MNF__zSej8f$r1;p8i5=j{7-~<;^7Ix)Pn}pm%I;6QH94YM2Wre!#ibOn|}$v zb(kZNxyZ6Z_!jbjo}etJhIh@QFu$PYK^zfu6rE;FM_5B)*Jx zdOgx`coBd)d4YL70XLCH=j-uL8Oxg-z8?G& Date: Mon, 6 Jul 2026 20:40:25 +0200 Subject: [PATCH 05/13] test(data_load_mcap): restore mcap_dialog_test target with a baseline smoke test Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/CMakeLists.txt | 17 +++++++++++++++++ data_load_mcap/tests/mcap_dialog_test.cpp | 17 +++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 data_load_mcap/tests/mcap_dialog_test.cpp diff --git a/data_load_mcap/CMakeLists.txt b/data_load_mcap/CMakeLists.txt index e596d34..6609d36 100644 --- a/data_load_mcap/CMakeLists.txt +++ b/data_load_mcap/CMakeLists.txt @@ -44,6 +44,23 @@ target_include_directories(mcap_helpers_test SYSTEM PRIVATE ${CMAKE_CURRENT_SOUR target_link_libraries(mcap_helpers_test PRIVATE LZ4::lz4_static zstd::libzstd_static GTest::gtest_main) add_test(NAME mcap_helpers_test COMMAND mcap_helpers_test) +# --- Unit test for MCAP dialog model/config behavior --- +add_executable(mcap_dialog_test tests/mcap_dialog_test.cpp) +target_compile_features(mcap_dialog_test PRIVATE cxx_std_20) +target_compile_options(mcap_dialog_test PRIVATE ${PJ_WARNING_FLAGS}) +target_compile_definitions(mcap_dialog_test PRIVATE + MCAP_TEST_DATA_DIR="${CMAKE_CURRENT_SOURCE_DIR}/test_data" +) +target_include_directories(mcap_dialog_test PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR} + ${CMAKE_CURRENT_BINARY_DIR}/generated +) +target_include_directories(mcap_dialog_test SYSTEM PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/contrib) +target_link_libraries(mcap_dialog_test PRIVATE + plotjuggler_sdk::plugin_sdk nlohmann_json::nlohmann_json LZ4::lz4_static zstd::libzstd_static GTest::gtest_main +) +add_test(NAME mcap_dialog_test COMMAND mcap_dialog_test) + # --- Unit test for the MessageByteStore (hot/cold lazy byte layer) --- find_package(Threads REQUIRED) add_executable(message_byte_store_test tests/message_byte_store_test.cpp) diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp new file mode 100644 index 0000000..00d73d2 --- /dev/null +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -0,0 +1,17 @@ +#define MCAP_IMPLEMENTATION +#include "mcap_dialog.hpp" + +#include + +#include +#include + +namespace { + +TEST(McapDialogTest, DefaultConstructs) { + McapDialog dialog; + EXPECT_TRUE(dialog.selectedTopics().empty()); + EXPECT_TRUE(dialog.analyzeError().empty()); +} + +} // namespace From df01a08cb0e4dd9f73d211fbfa272485be7d85d6 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:42:54 +0200 Subject: [PATCH 06/13] fix(data_load_mcap): restore selection by text, sort-safe like ros2/webrtc pickers Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/mcap_dialog.hpp | 9 ++++++--- data_load_mcap/tests/mcap_dialog_test.cpp | 17 +++++++++++++++++ 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/data_load_mcap/mcap_dialog.hpp b/data_load_mcap/mcap_dialog.hpp index 4113b76..6e923ac 100644 --- a/data_load_mcap/mcap_dialog.hpp +++ b/data_load_mcap/mcap_dialog.hpp @@ -92,7 +92,7 @@ class McapDialog : public PJ::DialogPluginTyped { wd.setTableHeaders("tableWidget", headers); std::vector> rows; - std::vector selected_row_indices; + std::vector selected_topic_names; std::vector disabled_row_indices; rows.reserve(filtered.size()); @@ -100,7 +100,7 @@ class McapDialog : public PJ::DialogPluginTyped { const auto& ch = *filtered[i]; rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); if (selected_topics_.count(ch.topic) > 0) { - selected_row_indices.push_back(static_cast(i)); + selected_topic_names.push_back(ch.topic); } if (ch.msg_count == 0) { disabled_row_indices.push_back(static_cast(i)); @@ -108,7 +108,10 @@ class McapDialog : public PJ::DialogPluginTyped { } wd.setTableRows("tableWidget", rows); wd.setDisabledRows("tableWidget", disabled_row_indices); - wd.setSelectedRows("tableWidget", selected_row_indices); + // Restore selection by first-column text (the topic name), not row index: + // the host matches items by text, which is sort-agnostic, so the selection + // survives the table's built-in column sorting (sortingEnabled=true). + wd.setSelectedItems("tableWidget", selected_topic_names); wd.setShortcut("btnSelectAll", "Ctrl+A"); wd.setShortcut("btnDeselectAll", "Ctrl+Shift+A"); diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp index 00d73d2..ab3376d 100644 --- a/data_load_mcap/tests/mcap_dialog_test.cpp +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -14,4 +14,21 @@ TEST(McapDialogTest, DefaultConstructs) { EXPECT_TRUE(dialog.analyzeError().empty()); } +TEST(McapDialogTest, WidgetDataSelectsRowsByTextNotIndex) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_publish_vs_log_time.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + const auto data = nlohmann::json::parse(dialog.widget_data()); + ASSERT_TRUE(data.contains("tableWidget")); + const auto& tbl = data["tableWidget"]; + + ASSERT_TRUE(tbl.contains("selected_items")); + std::vector selected_items = tbl["selected_items"].get>(); + EXPECT_EQ(selected_items, (std::vector{"/sensor/value"})); + + EXPECT_FALSE(tbl.contains("selected_rows")); +} + } // namespace From e37a097c5454d9f37767a95050952e844fcd66ff Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:45:38 +0200 Subject: [PATCH 07/13] feat(data_load_mcap): add (schema, encoding) always-include whitelist Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/mcap_dialog.hpp | 27 +++++++++++++++++ data_load_mcap/tests/mcap_dialog_test.cpp | 36 +++++++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/data_load_mcap/mcap_dialog.hpp b/data_load_mcap/mcap_dialog.hpp index 6e923ac..dfdff1e 100644 --- a/data_load_mcap/mcap_dialog.hpp +++ b/data_load_mcap/mcap_dialog.hpp @@ -24,6 +24,33 @@ struct ChannelInfo { uint64_t msg_count = 0; }; +struct AlwaysIncludeRule { + std::string schema_name; + std::string encoding; +}; + +/// Channels whose (schema, encoding) match a rule here are always loaded, +/// regardless of user selection -- 3D rendering (Scene3D's transform tree) +/// silently breaks without them. Keyed on message type rather than topic +/// name because topic naming isn't consistent across producers: Foxglove's +/// own foxglove.FrameTransform has no fixed topic-name convention the way +/// ROS fixes /tf (real Foxglove-recorded files use topic names like plain +/// "tf", no leading slash, or something else entirely). The mechanism here +/// is generic -- this table is just its (currently two-entry) data. +const std::vector kAlwaysIncludeRules = { + {"tf2_msgs/msg/TFMessage", "cdr"}, + {"foxglove.FrameTransform", "protobuf"}, +}; + +[[maybe_unused]] bool isAlwaysIncluded(const ChannelInfo& ch) { + for (const auto& rule : kAlwaysIncludeRules) { + if (ch.schema == rule.schema_name && ch.encoding == rule.encoding) { + return true; + } + } + return false; +} + class McapDialog : public PJ::DialogPluginTyped { using PJ::DialogPluginTyped::onValueChanged; diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp index ab3376d..a54b758 100644 --- a/data_load_mcap/tests/mcap_dialog_test.cpp +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -31,4 +31,40 @@ TEST(McapDialogTest, WidgetDataSelectsRowsByTextNotIndex) { EXPECT_FALSE(tbl.contains("selected_rows")); } +TEST(AlwaysIncludeRuleTest, MatchesRos2Tf) { + ChannelInfo ch; + ch.topic = "/tf"; + ch.schema = "tf2_msgs/msg/TFMessage"; + ch.encoding = "cdr"; + ch.msg_count = 10; + EXPECT_TRUE(isAlwaysIncluded(ch)); +} + +TEST(AlwaysIncludeRuleTest, MatchesFoxgloveFrameTransformUnderAnyTopicName) { + ChannelInfo ch; + ch.topic = "transforms"; // deliberately not "/tf" + ch.schema = "foxglove.FrameTransform"; + ch.encoding = "protobuf"; + ch.msg_count = 10; + EXPECT_TRUE(isAlwaysIncluded(ch)); +} + +TEST(AlwaysIncludeRuleTest, RejectsRightSchemaWrongEncoding) { + ChannelInfo ch; + ch.topic = "/tf"; + ch.schema = "tf2_msgs/msg/TFMessage"; + ch.encoding = "json"; + ch.msg_count = 10; + EXPECT_FALSE(isAlwaysIncluded(ch)); +} + +TEST(AlwaysIncludeRuleTest, RejectsRightEncodingWrongSchema) { + ChannelInfo ch; + ch.topic = "/sensor/value"; + ch.schema = "std_msgs/Float64"; + ch.encoding = "cdr"; + ch.msg_count = 10; + EXPECT_FALSE(isAlwaysIncluded(ch)); +} + } // namespace From 4148b0a1ea3a98162d8965d14004b87dcc88c63c Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 20:48:28 +0200 Subject: [PATCH 08/13] feat(data_load_mcap): force-include whitelisted channels after analyzeFile() Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/mcap_dialog.hpp | 16 +++++++++++++++- data_load_mcap/tests/mcap_dialog_test.cpp | 18 ++++++++++++++++++ 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/data_load_mcap/mcap_dialog.hpp b/data_load_mcap/mcap_dialog.hpp index dfdff1e..e8dfd7e 100644 --- a/data_load_mcap/mcap_dialog.hpp +++ b/data_load_mcap/mcap_dialog.hpp @@ -42,7 +42,7 @@ const std::vector kAlwaysIncludeRules = { {"foxglove.FrameTransform", "protobuf"}, }; -[[maybe_unused]] bool isAlwaysIncluded(const ChannelInfo& ch) { +bool isAlwaysIncluded(const ChannelInfo& ch) { for (const auto& rule : kAlwaysIncludeRules) { if (ch.schema == rule.schema_name && ch.encoding == rule.encoding) { return true; @@ -362,6 +362,20 @@ class McapDialog : public PJ::DialogPluginTyped { } } } + + reassertAlwaysIncluded(); + } + + /// Channels matching kAlwaysIncludeRules are always loaded: 3D rendering + /// depends on them even when the user narrows their selection to a handful + /// of unrelated topics. Idempotent -- safe to call after any mutation of + /// selected_topics_. + void reassertAlwaysIncluded() { + for (const auto& ch : all_channels_) { + if (ch.msg_count > 0 && isAlwaysIncluded(ch)) { + selected_topics_.insert(ch.topic); + } + } } std::vector filteredChannels() const { diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp index a54b758..0f09db4 100644 --- a/data_load_mcap/tests/mcap_dialog_test.cpp +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -67,4 +67,22 @@ TEST(AlwaysIncludeRuleTest, RejectsRightEncodingWrongSchema) { EXPECT_FALSE(isAlwaysIncluded(ch)); } +TEST(McapDialogTest, ForceIncludesOnlyWhitelistedNonEmptyChannels) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + // Saved selection predates this feature and omits everything except one + // ordinary topic -- as if loading an old layout. + cfg["selected_topics"] = std::vector{"/sensor/value2"}; + + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + const auto& selected = dialog.selectedTopics(); + EXPECT_TRUE(selected.count("/tf") > 0); // whitelisted, has messages -> forced back in + EXPECT_TRUE(selected.count("transforms") > 0); // whitelisted (any topic name) -> forced back in + EXPECT_EQ(selected.count("/tf_static"), 0u); // whitelisted schema/encoding, zero messages -> NOT forced + EXPECT_EQ(selected.count("/near_miss_encoding"), 0u); // right schema, wrong encoding -> NOT forced + EXPECT_TRUE(selected.count("/sensor/value2") > 0); // explicitly saved by the user -> stays +} + } // namespace From 138b5f05de27d051ab25b7158a79aba755f2d9a4 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 21:37:18 +0200 Subject: [PATCH 09/13] feat(data_load_mcap): lock whitelisted rows (disabled + tooltip) in the picker Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/mcap_dialog.hpp | 7 ++++- data_load_mcap/tests/mcap_dialog_test.cpp | 32 +++++++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/data_load_mcap/mcap_dialog.hpp b/data_load_mcap/mcap_dialog.hpp index e8dfd7e..49ca070 100644 --- a/data_load_mcap/mcap_dialog.hpp +++ b/data_load_mcap/mcap_dialog.hpp @@ -129,9 +129,14 @@ class McapDialog : public PJ::DialogPluginTyped { if (selected_topics_.count(ch.topic) > 0) { selected_topic_names.push_back(ch.topic); } - if (ch.msg_count == 0) { + bool locked_always_included = ch.msg_count > 0 && isAlwaysIncluded(ch); + if (ch.msg_count == 0 || locked_always_included) { disabled_row_indices.push_back(static_cast(i)); } + if (locked_always_included) { + wd.setCellTooltip( + "tableWidget", static_cast(i), 0, "Always loaded — required for 3D transform rendering."); + } } wd.setTableRows("tableWidget", rows); wd.setDisabledRows("tableWidget", disabled_row_indices); diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp index 0f09db4..970530d 100644 --- a/data_load_mcap/tests/mcap_dialog_test.cpp +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -3,6 +3,7 @@ #include +#include #include #include @@ -85,4 +86,35 @@ TEST(McapDialogTest, ForceIncludesOnlyWhitelistedNonEmptyChannels) { EXPECT_TRUE(selected.count("/sensor/value2") > 0); // explicitly saved by the user -> stays } +TEST(McapDialogTest, WhitelistedRowsAreDisabledAndTooltippedInWidgetData) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + const auto data = nlohmann::json::parse(dialog.widget_data()); + const auto& rows = data["tableWidget"]["rows"]; + + int tf_row = -1; + int ordinary_row = -1; + for (size_t i = 0; i < rows.size(); ++i) { + if (rows[i][0] == "/tf") { + tf_row = static_cast(i); + } + if (rows[i][0] == "/sensor/value2") { + ordinary_row = static_cast(i); + } + } + ASSERT_NE(tf_row, -1); + ASSERT_NE(ordinary_row, -1); + + const auto& disabled = data["tableWidget"]["disabled_rows"]; + EXPECT_NE(std::find(disabled.begin(), disabled.end(), tf_row), disabled.end()); + EXPECT_EQ(std::find(disabled.begin(), disabled.end(), ordinary_row), disabled.end()); + + ASSERT_TRUE(data["tableWidget"].contains("cell_tooltips")); + const auto& tooltips = data["tableWidget"]["cell_tooltips"]; + EXPECT_TRUE(tooltips.contains(std::to_string(tf_row) + ",0")); +} + } // namespace From 563e005562f5d3c50989f68201389e9353207806 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 21:40:09 +0200 Subject: [PATCH 10/13] fix(data_load_mcap): keep whitelisted topics selected through Deselect All / host selection changes Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/mcap_dialog.hpp | 2 ++ data_load_mcap/tests/mcap_dialog_test.cpp | 30 +++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/data_load_mcap/mcap_dialog.hpp b/data_load_mcap/mcap_dialog.hpp index 49ca070..faf2427 100644 --- a/data_load_mcap/mcap_dialog.hpp +++ b/data_load_mcap/mcap_dialog.hpp @@ -220,6 +220,7 @@ class McapDialog : public PJ::DialogPluginTyped { for (const auto& topic : selected) { selected_topics_.insert(topic); } + reassertAlwaysIncluded(); return true; // update OK button state } return false; @@ -237,6 +238,7 @@ class McapDialog : public PJ::DialogPluginTyped { } if (widget_name == "btnDeselectAll") { selected_topics_.clear(); + reassertAlwaysIncluded(); return true; } return false; diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp index 970530d..4eca857 100644 --- a/data_load_mcap/tests/mcap_dialog_test.cpp +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -117,4 +117,34 @@ TEST(McapDialogTest, WhitelistedRowsAreDisabledAndTooltippedInWidgetData) { EXPECT_TRUE(tooltips.contains(std::to_string(tf_row) + ",0")); } +TEST(McapDialogTest, DeselectAllKeepsWhitelistedTopicsSelected) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + EXPECT_TRUE(dialog.onClicked("btnDeselectAll")); + + const auto& selected = dialog.selectedTopics(); + EXPECT_TRUE(selected.count("/tf") > 0); + EXPECT_TRUE(selected.count("transforms") > 0); + EXPECT_EQ(selected.count("/sensor/value2"), 0u); +} + +TEST(McapDialogTest, HostReportedSelectionOmittingWhitelistIsOverridden) { + McapDialog dialog; + nlohmann::json cfg; + cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; + ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + + // Simulate the host reporting a selection that omits /tf and transforms -- + // the dialog must add them back regardless. + EXPECT_TRUE(dialog.onSelectionChanged("tableWidget", {"/sensor/value2"})); + + const auto& selected = dialog.selectedTopics(); + EXPECT_TRUE(selected.count("/tf") > 0); + EXPECT_TRUE(selected.count("transforms") > 0); + EXPECT_TRUE(selected.count("/sensor/value2") > 0); +} + } // namespace From c34c8ed535c8005761678cda1aec4f9929e09c8c Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 21:41:35 +0200 Subject: [PATCH 11/13] docs(data_load_mcap): document the always-include whitelist Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/data_load_mcap/README.md b/data_load_mcap/README.md index 722e8da..b100db3 100644 --- a/data_load_mcap/README.md +++ b/data_load_mcap/README.md @@ -75,6 +75,19 @@ so the user can select which topics to import. Per-import options: - **Embedded timestamp** — extract from message headers when supported by the parser (e.g. ROS `header.stamp`). +## Always-included channels + +Channels whose `(schema name, encoding)` matches a small built-in whitelist +are always loaded, pre-checked and locked in the picker, regardless of the +user's selection — because PJ4's Scene3D needs them to place other 3D +objects (`PointCloud`/`Mesh3D`) and silently breaks without them. Today the +whitelist covers ROS 2's transform tree (`tf2_msgs/msg/TFMessage` over +`cdr`) and Foxglove's (`foxglove.FrameTransform` over `protobuf`). The match +is on message type, not topic name — Foxglove's convention has no fixed +topic name the way ROS fixes `/tf`. This keeps the loader itself +format-agnostic: the whitelist is just data (`kAlwaysIncludeRules` in +`mcap_dialog.hpp`), not a ROS-specific code path. + ## How parsers see this loader Because the loader speaks only `pushMessage` with a fetcher, any From c3230d4f680c7f5bb741e7b4a8698fa159e57cd0 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Mon, 6 Jul 2026 21:52:49 +0200 Subject: [PATCH 12/13] test(data_load_mcap): pin fixture channels so negative whitelist checks can't pass vacuously Co-Authored-By: Claude Sonnet 5 --- data_load_mcap/tests/mcap_dialog_test.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/data_load_mcap/tests/mcap_dialog_test.cpp b/data_load_mcap/tests/mcap_dialog_test.cpp index 4eca857..4d71874 100644 --- a/data_load_mcap/tests/mcap_dialog_test.cpp +++ b/data_load_mcap/tests/mcap_dialog_test.cpp @@ -5,6 +5,7 @@ #include #include +#include #include namespace { @@ -78,6 +79,16 @@ TEST(McapDialogTest, ForceIncludesOnlyWhitelistedNonEmptyChannels) { ASSERT_TRUE(dialog.loadConfig(cfg.dump())); + // Pin the fixture: all five channels must be present in the analyzed table, + // otherwise the negative assertions below would pass vacuously. + const auto data = nlohmann::json::parse(dialog.widget_data()); + std::set row_topics; + for (const auto& row : data["tableWidget"]["rows"]) { + row_topics.insert(row[0].get()); + } + EXPECT_EQ( + row_topics, (std::set{"/near_miss_encoding", "/sensor/value2", "/tf", "/tf_static", "transforms"})); + const auto& selected = dialog.selectedTopics(); EXPECT_TRUE(selected.count("/tf") > 0); // whitelisted, has messages -> forced back in EXPECT_TRUE(selected.count("transforms") > 0); // whitelisted (any topic name) -> forced back in From dba3811d42a855f1982f9a3dd192c4697a3443e2 Mon Sep 17 00:00:00 2001 From: Davide Faconti Date: Tue, 7 Jul 2026 10:00:36 +0200 Subject: [PATCH 13/13] chore: drop working plan/spec documents from the PR Planning artifacts stay local; the PR carries only the shippable change. Co-Authored-By: Claude Fable 5 --- .../2026-07-06-mcap-always-include-tf.md | 842 ------------------ ...026-07-06-mcap-always-include-tf-design.md | 174 ---- 2 files changed, 1016 deletions(-) delete mode 100644 docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md delete mode 100644 docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md diff --git a/docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md b/docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md deleted file mode 100644 index d0c66c9..0000000 --- a/docs/superpowers/plans/2026-07-06-mcap-always-include-tf.md +++ /dev/null @@ -1,842 +0,0 @@ -# data_load_mcap: always-include /tf and /tf_static — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Make `data_load_mcap` force-include any channel whose `(schema name, encoding)` matches a known transform-tree message type (ROS 2's `tf2_msgs/msg/TFMessage` over `cdr`, Foxglove's `foxglove.FrameTransform` over `protobuf`) — pre-checked and locked in the topic picker — so 3D rendering never silently breaks because a user narrowed their topic selection, while the loader itself stays format-agnostic (the mechanism is a generic, data-driven whitelist; these two entries are just its current contents). - -**Architecture:** All changes live in `data_load_mcap/mcap_dialog.hpp` (dialog-side selection/model logic) plus one small refactor to make the existing selection-restore sort-safe. `mcap_source.cpp` needs no changes — it already reads exclusively from `McapDialog::selectedTopics()`. Enforcement lives at the model layer (`selected_topics_`), re-asserted after every mutation point, so the guarantee doesn't depend on the view staying in sync. - -**Tech Stack:** C++20, GTest, nlohmann::json, the vendored header-only `mcap` library, Python (`pip install mcap`) for generating test fixtures. - -**Spec:** `docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md` - ---- - -## Task 1: Add the whitelist test fixture - -**Files:** -- Modify: `data_load_mcap/test_data/generate_verification_mcaps.py` -- Create (generated binary artifact, to be committed): `data_load_mcap/test_data/test_dialog_whitelist.mcap` - -This fixture needs five channels covering every case the design calls out: a ROS 2 `/tf` match, a Foxglove-style match under a *non-conventional* topic name (proving the whitelist isn't topic-name-based), a whitelisted schema/encoding with zero messages (must NOT force-include), a near-miss (right schema, wrong encoding), and an ordinary unaffected channel. - -- [ ] **Step 1: Add schema constants and the generator function** - -Add near the top of `generate_verification_mcaps.py`, after the existing `SCHEMA_NESTED` constant (around line 70): - -```python -SCHEMA_TF2_MSGS = ( - b"geometry_msgs/TransformStamped[] transforms\n" - b"================\n" - b"MSG: geometry_msgs/TransformStamped\n" - b"Header header\n" - b"string child_frame_id\n" - b"Transform transform\n" - b"================\n" - b"MSG: std_msgs/Header\n" - b"int32 sec\n" - b"uint32 nanosec\n" - b"string frame_id\n" - b"================\n" - b"MSG: geometry_msgs/Transform\n" - b"Vector3 translation\n" - b"Quaternion rotation\n" -) - -SCHEMA_FOXGLOVE_FRAME_TRANSFORM_PROTO = ( - b'syntax = "proto3";\n' - b"package foxglove;\n" - b"message FrameTransform {\n" - b" string parent_frame_id = 2;\n" - b" string child_frame_id = 3;\n" - b"}\n" -) -``` - -Add the generator function after `gen_embedded_timestamp()` (around line 146, before the "Test 3: Large arrays" section): - -```python -# --------------------------------------------------------------------------- -# Test: always-include whitelist (schema + encoding based, not topic name) -# --------------------------------------------------------------------------- - -def gen_dialog_whitelist(): - path = OUT / "test_dialog_whitelist.mcap" - n_msgs = 5 - dt_ns = 100_000_000 - - with open(path, "wb") as f: - w = Writer(f) - w.start(profile="ros2", library="pj-verification") - - # 1. ROS 2 /tf -- whitelisted (schema tf2_msgs/msg/TFMessage, encoding cdr). - schema_tf2 = w.register_schema( - name="tf2_msgs/msg/TFMessage", encoding="ros2msg", data=SCHEMA_TF2_MSGS, - ) - ch_tf = w.register_channel(topic="/tf", message_encoding="cdr", schema_id=schema_tf2) - - # 2. Foxglove transform under a non-conventional topic name -- whitelisted - # (schema foxglove.FrameTransform, encoding protobuf). Proves the match - # is on message type, not on topic naming: real Foxglove-recorded files - # use topic names like plain "tf" (no leading slash) or arbitrary names. - schema_foxglove = w.register_schema( - name="foxglove.FrameTransform", encoding="protobuf", - data=SCHEMA_FOXGLOVE_FRAME_TRANSFORM_PROTO, - ) - ch_foxglove = w.register_channel( - topic="transforms", message_encoding="protobuf", schema_id=schema_foxglove, - ) - - # 3. /tf_static -- same whitelisted schema/encoding, but zero messages: - # must NOT be force-included (nothing to load). - _ = w.register_channel(topic="/tf_static", message_encoding="cdr", schema_id=schema_tf2) - - # 4. Near miss: right schema name, wrong encoding -- must NOT be force-included. - schema_tf2_json = w.register_schema( - name="tf2_msgs/msg/TFMessage", encoding="jsonschema", data=SCHEMA_TF2_MSGS, - ) - ch_near_miss_encoding = w.register_channel( - topic="/near_miss_encoding", message_encoding="json", schema_id=schema_tf2_json, - ) - - # 5. Ordinary channel, unaffected by the whitelist. - schema_float = w.register_schema( - name="std_msgs/Float64", encoding="ros2msg", data=SCHEMA_FLOAT64, - ) - ch_ordinary = w.register_channel( - topic="/sensor/value2", message_encoding="cdr", schema_id=schema_float, - ) - - for i in range(n_msgs): - ts = i * dt_ns - w.add_message(channel_id=ch_tf, log_time=ts, publish_time=ts, data=cdr_float64(1.0)) - # Dummy protobuf bytes -- doesn't need to be valid, the dialog never - # decodes message content, only reads channel/schema metadata. - w.add_message(channel_id=ch_foxglove, log_time=ts, publish_time=ts, data=b"\x00" * 8) - w.add_message( - channel_id=ch_near_miss_encoding, log_time=ts, publish_time=ts, data=b'{"x":1}', - ) - w.add_message(channel_id=ch_ordinary, log_time=ts, publish_time=ts, data=cdr_float64(2.0)) - - w.finish() - print(f"[OK] {path.name:45s} {path.stat().st_size:>8} bytes") -``` - -- [ ] **Step 2: Call the generator from `__main__`** - -In the `if __name__ == "__main__":` block, add the call (order doesn't matter; place after `gen_embedded_timestamp()`): - -```python - gen_publish_vs_log_time() - gen_embedded_timestamp() - gen_dialog_whitelist() - gen_large_arrays() -``` - -- [ ] **Step 3: Generate the fixture and verify its shape** - -Run: -```bash -cd data_load_mcap/test_data -python3 generate_verification_mcaps.py -``` -Expected: a line `[OK] test_dialog_whitelist.mcap ...` among the output, and the file exists. - -Then verify the channel table with the `mcap` CLI: -```bash -~/Apps/mcap-linux-amd64 info test_dialog_whitelist.mcap -``` -Expected: 5 channels — `/tf` (`tf2_msgs/msg/TFMessage [ros2msg]`), `transforms` (`foxglove.FrameTransform [protobuf]`), `/tf_static` (0 msgs), `/near_miss_encoding`, `/sensor/value2` — each with 5 messages except `/tf_static` (0). - -- [ ] **Step 4: Commit** - -```bash -cd /home/davide/ws_plotjuggler/pj-official-plugins/.worktrees/mcap-always-include-tf -git add data_load_mcap/test_data/generate_verification_mcaps.py data_load_mcap/test_data/test_dialog_whitelist.mcap -git commit -m "test(data_load_mcap): add whitelist fixture (ROS2 tf + Foxglove FrameTransform)" -``` - ---- - -## Task 2: Restore the `mcap_dialog_test` CMake target - -**Files:** -- Modify: `data_load_mcap/CMakeLists.txt` -- Create: `data_load_mcap/tests/mcap_dialog_test.cpp` - -`McapDialog` currently has zero direct unit coverage — its test target was deleted in PR #144 when the tests it had went stale (they asserted widgets that PR removed). The CMake wiring is recoverable verbatim from git history. - -- [ ] **Step 1: Write the initial test file** - -Create `data_load_mcap/tests/mcap_dialog_test.cpp`: - -```cpp -#define MCAP_IMPLEMENTATION -#include "mcap_dialog.hpp" - -#include - -#include -#include - -namespace { - -TEST(McapDialogTest, DefaultConstructs) { - McapDialog dialog; - EXPECT_TRUE(dialog.selectedTopics().empty()); - EXPECT_TRUE(dialog.analyzeError().empty()); -} - -} // namespace -``` - -- [ ] **Step 2: Restore the CMake target** - -In `data_load_mcap/CMakeLists.txt`, insert this block between the `mcap_helpers_test` block and the `message_byte_store_test` block (i.e. right after the `add_test(NAME mcap_helpers_test COMMAND mcap_helpers_test)` line, before the `# --- Unit test for the MessageByteStore` comment): - -```cmake -# --- Unit test for MCAP dialog model/config behavior --- -add_executable(mcap_dialog_test tests/mcap_dialog_test.cpp) -target_compile_features(mcap_dialog_test PRIVATE cxx_std_20) -target_compile_options(mcap_dialog_test PRIVATE ${PJ_WARNING_FLAGS}) -target_compile_definitions(mcap_dialog_test PRIVATE - MCAP_TEST_DATA_DIR="${CMAKE_CURRENT_SOURCE_DIR}/test_data" -) -target_include_directories(mcap_dialog_test PRIVATE - ${CMAKE_CURRENT_SOURCE_DIR} - ${CMAKE_CURRENT_BINARY_DIR}/generated -) -target_include_directories(mcap_dialog_test SYSTEM PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/contrib) -target_link_libraries(mcap_dialog_test PRIVATE - plotjuggler_sdk::plugin_sdk nlohmann_json::nlohmann_json LZ4::lz4_static zstd::libzstd_static GTest::gtest_main -) -add_test(NAME mcap_dialog_test COMMAND mcap_dialog_test) -``` - -- [ ] **Step 3: Build and run** - -Run: -```bash -cd /home/davide/ws_plotjuggler/pj-official-plugins/.worktrees/mcap-always-include-tf -./build.sh data_load_mcap -ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: build succeeds, `mcap_dialog_test` passes (`DefaultConstructs`). - -- [ ] **Step 4: Commit** - -```bash -git add data_load_mcap/CMakeLists.txt data_load_mcap/tests/mcap_dialog_test.cpp -git commit -m "test(data_load_mcap): restore mcap_dialog_test target with a baseline smoke test" -``` - ---- - -## Task 3: Make selection restore sort-safe (text-keyed, not index-keyed) - -**Files:** -- Modify: `data_load_mcap/mcap_dialog.hpp:89-111` (inside `widget_data()`) -- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` - -`tableWidget` has `sortingEnabled=true`; restoring selection by row index (`setSelectedRows`) desyncs the moment the host's native column sort reorders rows, because the plugin's row vector stays in its own order. `setSelectedItems` (text-keyed, matched by the host against column-0 text) already exists in the SDK and was just applied to `data_stream_ros2`/`data_stream_webrtc`'s pickers on the (unmerged) `feat/unify-picker-tables` branch — same fix, applied here directly since the SDK method itself is already available. This also gives the whitelist feature (next tasks) a sort-safe foundation to lock rows on top of. - -- [ ] **Step 1: Write the failing test** - -Append to `data_load_mcap/tests/mcap_dialog_test.cpp` (inside the anonymous namespace): - -```cpp -TEST(McapDialogTest, WidgetDataSelectsRowsByTextNotIndex) { - McapDialog dialog; - nlohmann::json cfg; - cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_publish_vs_log_time.mcap"; - ASSERT_TRUE(dialog.loadConfig(cfg.dump())); - - const auto data = nlohmann::json::parse(dialog.widget_data()); - ASSERT_TRUE(data.contains("tableWidget")); - const auto& tbl = data["tableWidget"]; - - ASSERT_TRUE(tbl.contains("selected_items")); - std::vector selected_items = tbl["selected_items"].get>(); - EXPECT_EQ(selected_items, (std::vector{"/sensor/value"})); - - EXPECT_FALSE(tbl.contains("selected_rows")); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: FAIL on `WidgetDataSelectsRowsByTextNotIndex` — `tbl` has `"selected_rows"`, not `"selected_items"`. - -- [ ] **Step 3: Implement the refactor** - -In `data_load_mcap/mcap_dialog.hpp`, inside `widget_data()`, replace: - -```cpp - std::vector> rows; - std::vector selected_row_indices; - std::vector disabled_row_indices; - rows.reserve(filtered.size()); - - for (size_t i = 0; i < filtered.size(); ++i) { - const auto& ch = *filtered[i]; - rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); - if (selected_topics_.count(ch.topic) > 0) { - selected_row_indices.push_back(static_cast(i)); - } - if (ch.msg_count == 0) { - disabled_row_indices.push_back(static_cast(i)); - } - } - wd.setTableRows("tableWidget", rows); - wd.setDisabledRows("tableWidget", disabled_row_indices); - wd.setSelectedRows("tableWidget", selected_row_indices); -``` - -with: - -```cpp - std::vector> rows; - std::vector selected_topic_names; - std::vector disabled_row_indices; - rows.reserve(filtered.size()); - - for (size_t i = 0; i < filtered.size(); ++i) { - const auto& ch = *filtered[i]; - rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); - if (selected_topics_.count(ch.topic) > 0) { - selected_topic_names.push_back(ch.topic); - } - if (ch.msg_count == 0) { - disabled_row_indices.push_back(static_cast(i)); - } - } - wd.setTableRows("tableWidget", rows); - wd.setDisabledRows("tableWidget", disabled_row_indices); - // Restore selection by first-column text (the topic name), not row index: - // the host matches items by text, which is sort-agnostic, so the selection - // survives the table's built-in column sorting (sortingEnabled=true). - wd.setSelectedItems("tableWidget", selected_topic_names); -``` - -(`disabled_row_indices` stays index-based — there is no text-keyed `setDisabledItems` in the SDK yet; see the design doc's "Known limitation" section.) - -- [ ] **Step 4: Run test to verify it passes** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: PASS, all tests green. - -- [ ] **Step 5: Commit** - -```bash -git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp -git commit -m "fix(data_load_mcap): restore selection by text, sort-safe like ros2/webrtc pickers" -``` - ---- - -## Task 4: Whitelist matching helper - -**Files:** -- Modify: `data_load_mcap/mcap_dialog.hpp:14-25` (anonymous namespace, after `ChannelInfo`) -- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` - -- [ ] **Step 1: Write the failing tests** - -Append to `data_load_mcap/tests/mcap_dialog_test.cpp`: - -```cpp -TEST(AlwaysIncludeRuleTest, MatchesRos2Tf) { - ChannelInfo ch; - ch.topic = "/tf"; - ch.schema = "tf2_msgs/msg/TFMessage"; - ch.encoding = "cdr"; - ch.msg_count = 10; - EXPECT_TRUE(isAlwaysIncluded(ch)); -} - -TEST(AlwaysIncludeRuleTest, MatchesFoxgloveFrameTransformUnderAnyTopicName) { - ChannelInfo ch; - ch.topic = "transforms"; // deliberately not "/tf" - ch.schema = "foxglove.FrameTransform"; - ch.encoding = "protobuf"; - ch.msg_count = 10; - EXPECT_TRUE(isAlwaysIncluded(ch)); -} - -TEST(AlwaysIncludeRuleTest, RejectsRightSchemaWrongEncoding) { - ChannelInfo ch; - ch.topic = "/tf"; - ch.schema = "tf2_msgs/msg/TFMessage"; - ch.encoding = "json"; - ch.msg_count = 10; - EXPECT_FALSE(isAlwaysIncluded(ch)); -} - -TEST(AlwaysIncludeRuleTest, RejectsRightEncodingWrongSchema) { - ChannelInfo ch; - ch.topic = "/sensor/value"; - ch.schema = "std_msgs/Float64"; - ch.encoding = "cdr"; - ch.msg_count = 10; - EXPECT_FALSE(isAlwaysIncluded(ch)); -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: FAIL to compile — `isAlwaysIncluded` is not defined. - -- [ ] **Step 3: Implement the whitelist** - -In `data_load_mcap/mcap_dialog.hpp`, right after the `ChannelInfo` struct definition (after line 25, before `class McapDialog`), add: - -```cpp -struct AlwaysIncludeRule { - std::string schema_name; - std::string encoding; -}; - -/// Channels whose (schema, encoding) match a rule here are always loaded, -/// regardless of user selection -- 3D rendering (Scene3D's transform tree) -/// silently breaks without them. Keyed on message type rather than topic -/// name because topic naming isn't consistent across producers: Foxglove's -/// own foxglove.FrameTransform has no fixed topic-name convention the way -/// ROS fixes /tf (real Foxglove-recorded files use topic names like plain -/// "tf", no leading slash, or something else entirely). The mechanism here -/// is generic -- this table is just its (currently two-entry) data. -const std::vector kAlwaysIncludeRules = { - {"tf2_msgs/msg/TFMessage", "cdr"}, - {"foxglove.FrameTransform", "protobuf"}, -}; - -bool isAlwaysIncluded(const ChannelInfo& ch) { - for (const auto& rule : kAlwaysIncludeRules) { - if (ch.schema == rule.schema_name && ch.encoding == rule.encoding) { - return true; - } - } - return false; -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: PASS, all tests green. - -- [ ] **Step 5: Commit** - -```bash -git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp -git commit -m "feat(data_load_mcap): add (schema, encoding) always-include whitelist" -``` - ---- - -## Task 5: Force-include whitelisted channels in `analyzeFile()` - -**Files:** -- Modify: `data_load_mcap/mcap_dialog.hpp:326-335` (end of `analyzeFile()`) -- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` - -This is the core guarantee: whitelisted channels end up in `selected_topics_` even when a saved config's `selected_topics` predates this feature and omits them. Because `loadConfig()` already calls `analyzeFile()` as its last step, this single change also covers the `loadConfig()` path from the design doc. - -- [ ] **Step 1: Write the failing test** - -Append to `data_load_mcap/tests/mcap_dialog_test.cpp`: - -```cpp -TEST(McapDialogTest, ForceIncludesOnlyWhitelistedNonEmptyChannels) { - McapDialog dialog; - nlohmann::json cfg; - cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; - // Saved selection predates this feature and omits everything except one - // ordinary topic -- as if loading an old layout. - cfg["selected_topics"] = std::vector{"/sensor/value2"}; - - ASSERT_TRUE(dialog.loadConfig(cfg.dump())); - - const auto& selected = dialog.selectedTopics(); - EXPECT_TRUE(selected.count("/tf") > 0); // whitelisted, has messages -> forced back in - EXPECT_TRUE(selected.count("transforms") > 0); // whitelisted (any topic name) -> forced back in - EXPECT_EQ(selected.count("/tf_static"), 0u); // whitelisted schema/encoding, zero messages -> NOT forced - EXPECT_EQ(selected.count("/near_miss_encoding"), 0u); // right schema, wrong encoding -> NOT forced - EXPECT_TRUE(selected.count("/sensor/value2") > 0); // explicitly saved by the user -> stays -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: FAIL — `/tf` and `transforms` are absent from `selectedTopics()` (the saved selection omitted them and nothing adds them back yet). - -- [ ] **Step 3: Implement `reassertAlwaysIncluded()` and call it from `analyzeFile()`** - -In `data_load_mcap/mcap_dialog.hpp`, change the end of `analyzeFile()` from: - -```cpp - // If no previous selection, select all channels with messages - if (selected_topics_.empty()) { - for (const auto& ch : all_channels_) { - if (ch.msg_count > 0) { - selected_topics_.insert(ch.topic); - } - } - } - } -``` - -to: - -```cpp - // If no previous selection, select all channels with messages - if (selected_topics_.empty()) { - for (const auto& ch : all_channels_) { - if (ch.msg_count > 0) { - selected_topics_.insert(ch.topic); - } - } - } - - reassertAlwaysIncluded(); - } - - /// Channels matching kAlwaysIncludeRules are always loaded: 3D rendering - /// depends on them even when the user narrows their selection to a handful - /// of unrelated topics. Idempotent -- safe to call after any mutation of - /// selected_topics_. - void reassertAlwaysIncluded() { - for (const auto& ch : all_channels_) { - if (ch.msg_count > 0 && isAlwaysIncluded(ch)) { - selected_topics_.insert(ch.topic); - } - } - } -``` - -(Place `reassertAlwaysIncluded()` as a new private method right after `analyzeFile()`, before `filteredChannels()`.) - -- [ ] **Step 4: Run test to verify it passes** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: PASS, all tests green. - -- [ ] **Step 5: Commit** - -```bash -git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp -git commit -m "feat(data_load_mcap): force-include whitelisted channels after analyzeFile()" -``` - ---- - -## Task 6: Lock whitelisted rows in the picker (disabled + tooltip) - -**Files:** -- Modify: `data_load_mcap/mcap_dialog.hpp:89-111` (inside `widget_data()`, the loop introduced/changed in Task 3) -- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` - -- [ ] **Step 1: Write the failing test** - -Append to `data_load_mcap/tests/mcap_dialog_test.cpp` (add `#include ` to the top of the file alongside the other includes): - -```cpp -TEST(McapDialogTest, WhitelistedRowsAreDisabledAndTooltippedInWidgetData) { - McapDialog dialog; - nlohmann::json cfg; - cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; - ASSERT_TRUE(dialog.loadConfig(cfg.dump())); - - const auto data = nlohmann::json::parse(dialog.widget_data()); - const auto& rows = data["tableWidget"]["rows"]; - - int tf_row = -1; - int ordinary_row = -1; - for (size_t i = 0; i < rows.size(); ++i) { - if (rows[i][0] == "/tf") { - tf_row = static_cast(i); - } - if (rows[i][0] == "/sensor/value2") { - ordinary_row = static_cast(i); - } - } - ASSERT_NE(tf_row, -1); - ASSERT_NE(ordinary_row, -1); - - const auto& disabled = data["tableWidget"]["disabled_rows"]; - EXPECT_NE(std::find(disabled.begin(), disabled.end(), tf_row), disabled.end()); - EXPECT_EQ(std::find(disabled.begin(), disabled.end(), ordinary_row), disabled.end()); - - ASSERT_TRUE(data["tableWidget"].contains("cell_tooltips")); - const auto& tooltips = data["tableWidget"]["cell_tooltips"]; - EXPECT_TRUE(tooltips.contains(std::to_string(tf_row) + ",0")); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: FAIL — the `/tf` row index is absent from `disabled_rows`, and `cell_tooltips` is missing. - -- [ ] **Step 3: Implement the lock** - -In `data_load_mcap/mcap_dialog.hpp`, inside `widget_data()`, change the per-row loop from: - -```cpp - for (size_t i = 0; i < filtered.size(); ++i) { - const auto& ch = *filtered[i]; - rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); - if (selected_topics_.count(ch.topic) > 0) { - selected_topic_names.push_back(ch.topic); - } - if (ch.msg_count == 0) { - disabled_row_indices.push_back(static_cast(i)); - } - } -``` - -to: - -```cpp - for (size_t i = 0; i < filtered.size(); ++i) { - const auto& ch = *filtered[i]; - rows.push_back({ch.topic, ch.schema, ch.encoding, std::to_string(ch.msg_count)}); - if (selected_topics_.count(ch.topic) > 0) { - selected_topic_names.push_back(ch.topic); - } - bool locked_always_included = ch.msg_count > 0 && isAlwaysIncluded(ch); - if (ch.msg_count == 0 || locked_always_included) { - disabled_row_indices.push_back(static_cast(i)); - } - if (locked_always_included) { - wd.setCellTooltip("tableWidget", static_cast(i), 0, - "Always loaded — required for 3D transform rendering."); - } - } -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: PASS, all tests green. - -- [ ] **Step 5: Commit** - -```bash -git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp -git commit -m "feat(data_load_mcap): lock whitelisted rows (disabled + tooltip) in the picker" -``` - ---- - -## Task 7: Survive `Deselect All` and host-reported selection changes - -**Files:** -- Modify: `data_load_mcap/mcap_dialog.hpp:182-208` (`onSelectionChanged`, `onClicked`) -- Test: `data_load_mcap/tests/mcap_dialog_test.cpp` - -`onSelectionChanged` fully replaces `selected_topics_` with whatever the host reports, and `btnDeselectAll` clears it outright. Both need to re-assert whitelist membership afterward. (`btnSelectAll` doesn't need this: it only *adds* to `selected_topics_`, and whitelisted topics are already present from `reassertAlwaysIncluded()`, so the union can't drop them.) - -- [ ] **Step 1: Write the failing tests** - -Append to `data_load_mcap/tests/mcap_dialog_test.cpp`: - -```cpp -TEST(McapDialogTest, DeselectAllKeepsWhitelistedTopicsSelected) { - McapDialog dialog; - nlohmann::json cfg; - cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; - ASSERT_TRUE(dialog.loadConfig(cfg.dump())); - - EXPECT_TRUE(dialog.onClicked("btnDeselectAll")); - - const auto& selected = dialog.selectedTopics(); - EXPECT_TRUE(selected.count("/tf") > 0); - EXPECT_TRUE(selected.count("transforms") > 0); - EXPECT_EQ(selected.count("/sensor/value2"), 0u); -} - -TEST(McapDialogTest, HostReportedSelectionOmittingWhitelistIsOverridden) { - McapDialog dialog; - nlohmann::json cfg; - cfg["filepath"] = std::string(MCAP_TEST_DATA_DIR) + "/test_dialog_whitelist.mcap"; - ASSERT_TRUE(dialog.loadConfig(cfg.dump())); - - // Simulate the host reporting a selection that omits /tf and transforms -- - // the dialog must add them back regardless. - EXPECT_TRUE(dialog.onSelectionChanged("tableWidget", {"/sensor/value2"})); - - const auto& selected = dialog.selectedTopics(); - EXPECT_TRUE(selected.count("/tf") > 0); - EXPECT_TRUE(selected.count("transforms") > 0); - EXPECT_TRUE(selected.count("/sensor/value2") > 0); -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: FAIL — both tests find `/tf` and `transforms` missing from `selectedTopics()`. - -- [ ] **Step 3: Re-assert after both mutation points** - -In `data_load_mcap/mcap_dialog.hpp`, change: - -```cpp - bool onSelectionChanged(std::string_view widget_name, const std::vector& selected) override { - if (widget_name == "tableWidget") { - selected_topics_.clear(); - for (const auto& topic : selected) { - selected_topics_.insert(topic); - } - return true; // update OK button state - } - return false; - } - - bool onClicked(std::string_view widget_name) override { - if (widget_name == "btnSelectAll") { - auto filtered = filteredChannels(); - for (const auto* ch : filtered) { - if (ch->msg_count > 0) { - selected_topics_.insert(ch->topic); - } - } - return true; - } - if (widget_name == "btnDeselectAll") { - selected_topics_.clear(); - return true; - } - return false; - } -``` - -to: - -```cpp - bool onSelectionChanged(std::string_view widget_name, const std::vector& selected) override { - if (widget_name == "tableWidget") { - selected_topics_.clear(); - for (const auto& topic : selected) { - selected_topics_.insert(topic); - } - reassertAlwaysIncluded(); - return true; // update OK button state - } - return false; - } - - bool onClicked(std::string_view widget_name) override { - if (widget_name == "btnSelectAll") { - auto filtered = filteredChannels(); - for (const auto* ch : filtered) { - if (ch->msg_count > 0) { - selected_topics_.insert(ch->topic); - } - } - return true; - } - if (widget_name == "btnDeselectAll") { - selected_topics_.clear(); - reassertAlwaysIncluded(); - return true; - } - return false; - } -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: -```bash -./build.sh data_load_mcap && ctest --test-dir build -R mcap_dialog_test -V -``` -Expected: PASS, all tests green. - -- [ ] **Step 5: Commit** - -```bash -git add data_load_mcap/mcap_dialog.hpp data_load_mcap/tests/mcap_dialog_test.cpp -git commit -m "fix(data_load_mcap): keep whitelisted topics selected through Deselect All / host selection changes" -``` - ---- - -## Task 8: Document the behavior and run the full suite - -**Files:** -- Modify: `data_load_mcap/README.md` - -- [ ] **Step 1: Add a documentation section** - -In `data_load_mcap/README.md`, after the "## Channel discovery" section (after line 76, before "## How parsers see this loader"), add: - -```markdown -## Always-included channels - -Channels whose `(schema name, encoding)` matches a small built-in whitelist -are always loaded, pre-checked and locked in the picker, regardless of the -user's selection — because PJ4's Scene3D needs them to place other 3D -objects (`PointCloud`/`Mesh3D`) and silently breaks without them. Today the -whitelist covers ROS 2's transform tree (`tf2_msgs/msg/TFMessage` over -`cdr`) and Foxglove's (`foxglove.FrameTransform` over `protobuf`). The match -is on message type, not topic name — Foxglove's convention has no fixed -topic name the way ROS fixes `/tf`. This keeps the loader itself -format-agnostic: the whitelist is just data (`kAlwaysIncludeRules` in -`mcap_dialog.hpp`), not a ROS-specific code path. -``` - -- [ ] **Step 2: Run the full plugin test suite** - -Run: -```bash -cd /home/davide/ws_plotjuggler/pj-official-plugins/.worktrees/mcap-always-include-tf -./build.sh data_load_mcap -ctest --test-dir build -R mcap -V -``` -Expected: `mcap_dialog_test`, `mcap_helpers_test`, `message_byte_store_test` all PASS. - -- [ ] **Step 3: Commit** - -```bash -git add data_load_mcap/README.md -git commit -m "docs(data_load_mcap): document the always-include whitelist" -``` - ---- - -## Self-Review Notes - -- **Spec coverage:** Problem/mechanism (Tasks 4-5), enforcement/pre-checked-locked (Tasks 5-6-7), known limitation (Task 3 covers the cheap half; the disabled-rows index limitation is intentionally left as-is, matching the design doc's non-goal), data flow items 1-6 (Tasks 5-7, `mcap_source.cpp` untouched per item 6), testing (Tasks 1-7 fixture + coverage), non-goals (nothing added for ROS1, glob matching, or a whitelist-editing UI — none of the tasks touch those). -- **Type consistency:** `AlwaysIncludeRule{schema_name, encoding}`, `kAlwaysIncludeRules`, `isAlwaysIncluded(const ChannelInfo&)`, and `reassertAlwaysIncluded()` are named identically everywhere they're introduced (Task 4, 5) and consumed (Task 6, 7). -- **No placeholders:** every step shows the exact code to write; no "add appropriate handling" steps. diff --git a/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md b/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md deleted file mode 100644 index f11cd04..0000000 --- a/docs/superpowers/specs/2026-07-06-mcap-always-include-tf-design.md +++ /dev/null @@ -1,174 +0,0 @@ -# data_load_mcap: always-include /tf and /tf_static - -## Problem - -`data_load_mcap`'s topic-picker dialog treats every MCAP channel identically: -the user checks/unchecks rows, and only checked topics get parsed and pushed -into the ObjectStore. When a user narrows their selection instead of using -"Select All" — the common case when they only care about a handful of -signals — `/tf` and `/tf_static` can end up unchecked. Since PJ4's Scene3D -needs the transform tree to place `PointCloud`/`Mesh3D`/other 3D objects, -this silently breaks 3D rendering with no obvious cause. - -The plugin is meant to be format-agnostic — it must not simply hardcode -`if topic == "/tf"` as a ROS special case baked into otherwise-generic -loading logic. - -## Mechanism: generic always-include whitelist, keyed by (schema name, encoding) - -A small static table of `{schema_name, encoding}` pairs, matched against -`ChannelInfo{schema, encoding}` — fields already populated in `analyzeFile()` -directly from the MCAP channel/schema summary (`schema_it->second->name`, -`channel_ptr->messageEncoding`). No parser instantiation, no ROS awareness in -the mechanism itself — just the two fields the dialog already reads per -channel: - -```cpp -struct AlwaysIncludeRule { - std::string schema_name; - std::string encoding; -}; - -const std::vector kAlwaysIncludeRules = { - {"tf2_msgs/msg/TFMessage", "cdr"}, // ROS 2 transform tree - {"foxglove.FrameTransform", "protobuf"}, // Foxglove-style transform tree -}; -``` - -**Why schema (message type), not topic name.** The initial draft of this -design keyed on topic name (`/tf`, `/tf_static`) + encoding. Two pieces of -evidence overturned that: - -1. Foxglove's own transform message, `foxglove.FrameTransform` - (`parser_protobuf/foxglove_object_codecs.hpp`, already classified as - `kFrameTransforms` in `parser_protobuf/protobuf_parser.cpp:829-830`), is - explicitly designed to be published under *any* topic name — there is no - fixed convention the way ROS fixes `/tf`. -2. Checked with the `mcap` CLI against real Foxglove-recorded files - (`~/ws_plotjuggler/DATA/example-023-av-argo-2.mcap`): its transform - channel is topic `tf` — no leading slash — schema `foxglove.FrameTransform`, - channel `messageEncoding` = `protobuf`. A topic-name-based rule (`/tf`) - would have silently missed this real file. Matching on schema name sidesteps - topic-naming variance entirely. - -**Encoding values, verified against real bags with the `mcap` CLI:** MCAP's -`Channel` record carries `message_encoding` (the wire/serialization format of -the message bytes) separately from the `Schema` record's `encoding` (the -schema-definition language) — these are two different fields, and it's easy -to reach for the wrong one: - -- ROS 2 (`~/ws_plotjuggler/DATA/amcl_test_bag.mcap` and others): schema - `tf2_msgs/msg/TFMessage` has channel `messageEncoding` = `cdr` (what - `ChannelInfo.encoding` actually stores — `mcap_dialog.hpp` prefers - `channel_ptr->messageEncoding` over the schema's encoding whenever the - channel's own field is non-empty) and schema `encoding` = `ros2msg` (the - bracketed tag `mcap info` displays, and *not* what `ChannelInfo.encoding` - holds). -- Foxglove (`~/ws_plotjuggler/DATA/example-012-av-ds.mcap`, - `example-016-av-waymo-ds.mcap`, `example-023-av-argo-2.mcap`): schema - `foxglove.FrameTransform` has channel `messageEncoding` = `protobuf`. - -So the whitelist rules match on `"cdr"` and `"protobuf"` respectively, not -on `"ros2msg"`. - -The mechanism itself is generic: `kAlwaysIncludeRules` is just data, keyed -by message type rather than any particular naming convention. A future -message type with the same rendering-dependency problem could add a row -without touching the enforcement logic below. ROS's `tf2_msgs/msg/TFMessage` -and Foxglove's `foxglove.FrameTransform` are simply the first two entries. - -## Enforcement: pre-checked, cannot uncheck - -Channels matching a whitelist rule (with `msg_count > 0`) are: - -1. Unconditionally re-inserted into `selected_topics_` at every mutation - point: `analyzeFile()`'s default-select path, `onSelectionChanged`, and - both `btnSelectAll`/`btnDeselectAll` handlers in `onClicked`. -2. Rendered in `widget_data()` as both selected and disabled — reusing the - existing `setSelectedRows`/`setDisabledRows` calls already used to gray - out zero-message channels — so the checkbox/row cannot be toggled by the - user. -3. Given a `setCellTooltip` on the topic-name cell explaining why: "Always - loaded — required for 3D transform rendering." - -The correctness guarantee lives at the model layer (`selected_topics_`), -not the view: even if the widget's visual state were ever wrong, every -handler re-asserts whitelist membership before the dialog's state is -considered final, and `mcap_source.cpp`'s existing `topicFilter` (unchanged) -reads only from `selected_topics_`. - -## Known limitation, not fixed here - -`tableWidget` has `sortingEnabled=true`. Prior work already found that -`setSelectedRows`/`setDisabledRows` are index-based and desync visually -after a native column-header sort (the QUiLoader-loaded table reorders rows -without notifying the plugin), because `data_load_mcap` was deferred from -the fix applied to the ROS2/WebRTC pickers (which moved to the text-keyed -`setSelectedItems`, an SDK method that already exists). A full fix for the -disabled-rows side needs a new text-keyed `setDisabledItems` at the SDK -level (`plotjuggler_sdk` + PJ4), a larger cross-repo change. - -Because this feature's guarantee is enforced in `selected_topics_` -regardless of what the view displays, a post-sort desync can only cause a -**cosmetic** glitch (the locked look landing on the wrong row momentarily) -— never a dropped `/tf`/`/tf_static` topic. This design bundles the cheap, -already-available half of the fix (switching *selection* restore from -`setSelectedRows` to `setSelectedItems`, matching what PR #188 did for -ros2/webrtc) since it's directly relevant to a newly-added kind of -"selected" state, but leaves the disabled-rows index limitation as a -documented, pre-existing gap rather than pulling in the larger SDK change. - -## Data flow - -1. `analyzeFile()` builds `all_channels_` from the MCAP summary. -2. Any channel matching `kAlwaysIncludeRules` (with `msg_count > 0`) is - force-inserted into `selected_topics_`. -3. `widget_data()` marks those rows selected + disabled every render, with - a tooltip. -4. `onSelectionChanged` / `onClicked` (`btnSelectAll`/`btnDeselectAll`) - re-assert whitelist membership after applying the user's requested - change. -5. `loadConfig()` (restoring a previously-saved layout) also re-asserts - whitelist membership after populating `selected_topics_` from the saved - JSON, so an old saved config that predates this feature (and therefore - omits `/tf`) still gets it forced back in. -6. `mcap_source.cpp`'s `topicFilter` (unchanged) reads from - `selected_topics_` — whitelisted topics are always parsed and pushed, - feeding Scene3D via the existing `kFrameTransforms` classification in - `parser_ros`. - -## Testing - -`mcap_dialog_test.cpp` was deleted in PR #144 when its old tests (which -asserted the presence of widgets removed in that PR) went stale, and was -never replaced — `McapDialog` currently has zero direct unit coverage. The -CMake wiring for the removed target is recoverable from git history -(`git show 7f024e2 -- data_load_mcap/CMakeLists.txt`) and will be restored -with the same shape (links `plotjuggler_sdk::plugin_sdk`, `nlohmann_json`, -`LZ4::lz4_static`, `zstd::libzstd_static`, `GTest::gtest_main`). - -New coverage needed: - -- Whitelist matching: exact `(topic, encoding)` hits match; near-misses - (right topic wrong encoding, right encoding wrong topic) do not. -- `analyzeFile()` on a synthetic MCAP with a `cdr`-encoded `/tf` channel: - the topic ends up in `selected_topics_` even when `loadConfig()` supplies - a saved selection that omits it. -- `onClicked(btnDeselectAll)` and `onSelectionChanged` with `/tf` excluded - from the reported `selected` list still leave it in `selected_topics_`. -- Zero-msg-count `/tf` channel (edge case) is *not* force-included — matches - the existing "nothing to load" rule for empty channels. -- A small fixture MCAP with a `cdr`-encoded `/tf` topic will be added to - `test_data/`, following the existing pattern in - `generate_verification_mcaps.py`. - -## Non-goals - -- `data_stream_ros2` / `data_stream_foxglove_bridge`'s demand-driven - per-topic subscription work (`feat/topic-subscription`, unmerged) has the - same underlying risk for live streaming — a "required" topic may never be - dragged into a plot, so it's never subscribed. That's a separate - plugin/branch not yet merged; flagged as a related follow-up, out of - scope here. -- No glob/regex topic-name matching — exact string match only. -- No user-facing UI to edit the whitelist — it's a compiled-in table.