feat(device): read the live capability document and merge it into DeviceCapabilities (closes #390)

docs/adr/0001-firmware-feature-gating.md

@@ -2,7 +2,7 @@
 
 - **Status:** Accepted (2026-06-19)
 - **Issue:** [#251](https://github.com/daqifi/daqifi-core/issues/251)
-- **Follow-ups:** [#254](https://github.com/daqifi/daqifi-core/issues/254) (floor + `-113` backstop), [#255](https://github.com/daqifi/daqifi-core/issues/255) (dead-code removal), [#256](https://github.com/daqifi/daqifi-core/issues/256) (version table + `Supports()` seam; #327 reader still deferred)
+- **Follow-ups:** [#254](https://github.com/daqifi/daqifi-core/issues/254) (floor + `-113` backstop), [#255](https://github.com/daqifi/daqifi-core/issues/255) (dead-code removal), [#256](https://github.com/daqifi/daqifi-core/issues/256) (version table + `Supports()` seam), [#390](https://github.com/daqifi/daqifi-core/issues/390) (capability-document reader)
 - **Supersedes:** —
 
 > **Note on the evidence section.** §"Context — firmware audit" below is a *living*
@@ -170,12 +170,13 @@ forward-looking third:
 3. **`DeviceFeature` version table (deferred → built)** — introduce only when we start consuming
    a command newer than the floor. That happened with SD-over-WiFi (≥ v3.7.0), so the table now
    exists; see the implementation note below.
-4. **#327 capability document (later, non-blocking)** — when a device answers
-   `CONFigure:CAPabilities:APIVersion?` ≥ 1, populate `DeviceCapabilities` from the live
-   `CONFigure:CAPabilities:JSON?` document. The floor/board logic remains the bootstrap and
-   the fallback for anything that predates or omits the query. Note: #327 the *issue* is
-   closed, but the capability *commands* shipped in v3.5.0 and are maintained (#548), so this
-   is a real growth path — we just don't lean on "#327 the framework."
+4. **Capability document (deferred → built)** — when a device answers
+   `CONFigure:CAPabilities:APIVersion?` with a schema version this parser understands, populate
+   `DeviceCapabilities` from the live `CONFigure:CAPabilities:JSON?` document. The floor/board
+   logic remains the bootstrap and the fallback for anything that predates or omits the query.
+   Built in [#390](https://github.com/daqifi/daqifi-core/issues/390); see the implementation note
+   below. Note: firmware #327 the *issue* is closed, but the capability *commands* shipped in
+   v3.5.0 and are maintained (#548) — we depend on the commands, not on "#327 the framework."
 
 ### Evaluate `Supports` lazily — don't cache version-derived flags
 
@@ -264,7 +265,54 @@ from the firmware audit table above. Both previously hand-rolled gates — the S
 transport gate and the SD-storage-query `-113` backstop — now resolve their required version and
 board through it; `EnsureSdFileTransferSupportedOnTransport()` retains only the transport
 predicate (which feature applies over WiFi vs. USB) and delegates the support question to the
-seam. Part 2 — the `CONFigure:CAPabilities:JSON?` / `:APIVersion?` reader — remains deferred.
+seam. Part 2 — the `CONFigure:CAPabilities:JSON?` / `:APIVersion?` reader — followed in #390; see
+the next note.
+
+### Implementation note (2026-07-29, issue [#390](https://github.com/daqifi/daqifi-core/issues/390))
+
+Decision 2 item 4 is **built**. `DaqifiDevice.ReadCapabilityDocumentAsync()` issues
+`CONFigure:CAPabilities:APIVersion?` and `CONFigure:CAPabilities:JSON?`, and
+[`CapabilityDocumentParser`](../../src/Daqifi.Core/Device/Capabilities/CapabilityDocumentParser.cs)
+turns the reply into a
+[`CapabilityDocument`](../../src/Daqifi.Core/Device/Capabilities/CapabilityDocument.cs) kept on
+`DeviceMetadata.CapabilityDocument`. Four details worth recording:
+
+- **Two gates, not one.** The read is skipped entirely unless
+  `Supports(DeviceFeature.CapabilityDocument)` (already in the table at v3.5.0), and the document
+  is only trusted when the reported schema version falls within
+  `MinimumCapabilityDocumentApiVersion..MaximumCapabilityDocumentApiVersion` (1..2 today). The
+  upper bound is deliberate: firmware bumps that byte *only* on a breaking change, so a higher
+  version is a document whose fields may no longer mean what this parser assumes — and the board
+  table is a better answer than a plausible-but-wrong number. Raise the constant together with the
+  parser when adopting a newer schema. Both queries go out in **one** text exchange: swapping the
+  protobuf consumer out and back costs far more than either reply does, and measured on the bench
+  a second exchange cost ~1 s of connect latency to avoid transferring a document that takes
+  ~25 ms. The version still gates whether the document is *trusted*, which is the point of the
+  gate.
+- **Merge, never replace.** `CapabilityDocument.MergeInto` overlays only the fields the document
+  actually states; everything else keeps its `FromDeviceType` value. Every parsed field is
+  nullable, so "omitted" can never read as "absent" — which is what preserves the `Unknown`-board
+  rule above. `HasWincWifiModule` and `SupportsStreaming` are never overlaid: the schema carries no
+  chipset facts by design, and it emits the `streaming` block unconditionally.
+- **`UpdateFromProtobuf` had to stop replacing.** It rebuilt `Capabilities` from the board table on
+  *every* status message carrying a part number, which would have discarded the overlay mid-session
+  (and was already discarding channel counts). It now rebuilds only when the detected board
+  actually changes, and re-applies the document overlay last, so the device's own answer outranks
+  both the board table and the status message's port counts.
+- **The refresh point is the connect sequence, not status processing.** The document is a real SCPI
+  round-trip — it does not ride along in the protobuf status message, and reading it pauses the
+  protobuf consumer — so `InitializeAsync` reads it once after the device reports its board and
+  firmware version (the version gate fails closed, so an earlier read would skip on every device)
+  and before `OnDeviceInitializingAsync`. A failed read is absorbed: the device keeps its
+  board-derived capabilities, so nothing that works today starts failing. Callers needing a fresh
+  `current_max_rate_hz`, which tracks the enabled channel set, call `ReadCapabilityDocumentAsync()`
+  again. Measured cost on the bench NQ1 over USB: connect+initialize goes from ~3.16 s to ~4.29 s.
+
+Bench-validated against the NQ1 on firmware 3.7.2: the document agrees with `FromDeviceType` on
+every flag it states (SD, WiFi, USB) and supplies the channel counts (16 analog in, 0 analog out,
+16 digital). The single disagreement is `MaxSamplingRate` — the table's hardcoded `1000` against
+the device's `22000` — and it is the disagreement this reader exists to resolve: that literal
+predates firmware v3.5.0 removing the Type-2 muxed scan-rate cap (firmware #528).
 
 ## Alternatives considered
 
@@ -305,12 +353,17 @@ their place.**
    firmware SCPI table. Public-API removal, separate from the #253 fix.
 3. [#256](https://github.com/daqifi/daqifi-core/issues/256) — `DeviceFeature` version table +
    lazy `Supports(...)`: **done** (triggered by SD-over-WiFi @ v3.7.0; see the implementation
-   note under Decision 2). The `CONFigure:CAPabilities:JSON?` reader (#327 growth path) is
-   **still deferred** and tracked separately.
+   note under Decision 2).
+4. [#390](https://github.com/daqifi/daqifi-core/issues/390) — `CONFigure:CAPabilities:JSON?` /
+   `:APIVersion?` reader populating `DeviceCapabilities`: **done** (triggered by
+   daqifi-desktop [#118](https://github.com/daqifi/daqifi-desktop/issues/118), which needs a
+   per-configuration streaming ceiling the static table cannot express; see the implementation
+   note under Decision 2).
 
 ## Out of scope
 
-The full #327 capability-document reader; the version table (until a post-floor command needs
-it); version-selected command emit (obviated by the floor). This ADR covers the
-investigation, the strategy decision, the v3.5.0 floor, and the minimal board-gate + typed
-`-113` backstop.
+The version table (until a post-floor command needs it) and version-selected command emit
+(obviated by the floor) were both out of scope for this ADR's original decision; the table has
+since been built under #256, and the capability-document reader — deferred at the time of writing
+— under #390. This ADR covers the investigation, the strategy decision, the v3.5.0 floor, and the
+minimal board-gate + typed `-113` backstop.

src/Daqifi.Core.Tests/Communication/Producers/ScpiMessageProducerTests.cs

@@ -32,6 +32,28 @@ public void GetDeviceInfo_ReturnsCorrectCommand()
         AssertMessageFormat(message);
     }
 
+    [Fact]
+    public void GetCapabilitiesApiVersion_ReturnsCorrectCommand()
+    {
+        // Act
+        var message = ScpiMessageProducer.GetCapabilitiesApiVersion;
+
+        // Assert
+        Assert.Equal("CONFigure:CAPabilities:APIVersion?", message.Data);
+        AssertMessageFormat(message);
+    }
+
+    [Fact]
+    public void GetCapabilitiesJson_ReturnsCorrectCommand()
+    {
+        // Act
+        var message = ScpiMessageProducer.GetCapabilitiesJson;
+
+        // Assert
+        Assert.Equal("CONFigure:CAPabilities:JSON?", message.Data);
+        AssertMessageFormat(message);
+    }
+
     [Fact]
     public void DisableDeviceEcho_ReturnsCorrectCommand()
     {

src/Daqifi.Core.Tests/Device/Capabilities/CapabilityDocumentMergeTests.cs

@@ -0,0 +1,157 @@
+using System;
+using Daqifi.Core.Device;
+using Daqifi.Core.Device.Capabilities;
+
+namespace Daqifi.Core.Tests.Device.Capabilities;
+
+/// <summary>
+/// Tests that a capability document <i>overlays</i> board-derived
+/// <see cref="DeviceCapabilities"/> rather than replacing them (ADR 0001, Decision 2 item 4).
+/// </summary>
+public class CapabilityDocumentMergeTests
+{
+    private static CapabilityDocument BenchDocument()
+    {
+        Assert.True(CapabilityDocumentParser.TryParse(
+            CapabilityDocumentSamples.Nyquist1Firmware372, out var document));
+        return document!;
+    }
+
+    private static CapabilityDocument Parse(string json)
+    {
+        Assert.True(CapabilityDocumentParser.TryParse(json, out var document));
+        return document!;
+    }
+
+    [Fact]
+    public void MergeInto_RealNyquist1Document_AgreesWithTheBoardTable()
+    {
+        // The bench check the issue asks for, pinned as a regression test: for a board the static
+        // table already describes, the device's own answer must not contradict it.
+        var boardDerived = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+        var merged = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+
+        BenchDocument().MergeInto(merged);
+
+        Assert.Equal(boardDerived.HasSdCard, merged.HasSdCard);
+        Assert.Equal(boardDerived.HasWiFi, merged.HasWiFi);
+        Assert.Equal(boardDerived.HasUsb, merged.HasUsb);
+        Assert.Equal(boardDerived.SupportsStreaming, merged.SupportsStreaming);
+    }
+
+    [Fact]
+    public void MergeInto_RealNyquist1Document_FillsChannelCountsAndRaisesTheRateCeiling()
+    {
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+
+        BenchDocument().MergeInto(capabilities);
+
+        Assert.Equal(16, capabilities.AnalogInputChannels);
+        Assert.Equal(0, capabilities.AnalogOutputChannels);
+        Assert.Equal(16, capabilities.DigitalChannels);
+
+        // The one field where the device disagrees with the table, and the reason this reader
+        // exists: the hardcoded 1000 Hz predates firmware v3.5.0 removing the muxed scan-rate cap.
+        Assert.Equal(22000, capabilities.MaxSamplingRate);
+    }
+
+    [Fact]
+    public void MergeInto_LeavesFieldsTheSchemaDoesNotCarry()
+    {
+        // The schema publishes what a client can do, not what parts are fitted, so it has no
+        // WINC-module fact and no "streaming supported" boolean. Both stay board-derived.
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+
+        BenchDocument().MergeInto(capabilities);
+
+        Assert.True(capabilities.HasWincWifiModule);
+        Assert.True(capabilities.SupportsStreaming);
+    }
+
+    [Fact]
+    public void MergeInto_DocumentThatStatesNothing_LeavesEveryBoardValueIntact()
+    {
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist3);
+        capabilities.AnalogInputChannels = 8;
+        capabilities.AnalogOutputChannels = 2;
+        capabilities.DigitalChannels = 16;
+
+        Parse("{\"schema_version\":2}").MergeInto(capabilities);
+
+        Assert.True(capabilities.HasSdCard);
+        Assert.True(capabilities.HasWiFi);
+        Assert.True(capabilities.HasUsb);
+        Assert.True(capabilities.HasWincWifiModule);
+        Assert.Equal(8, capabilities.AnalogInputChannels);
+        Assert.Equal(2, capabilities.AnalogOutputChannels);
+        Assert.Equal(16, capabilities.DigitalChannels);
+        Assert.Equal(1000, capabilities.MaxSamplingRate);
+    }
+
+    [Fact]
+    public void MergeInto_PartialDocument_OverlaysOnlyWhatItStates()
+    {
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+        capabilities.AnalogInputChannels = 16;
+
+        // States storage only. Everything else — transports, channels, streaming — must fall back.
+        Parse("{\"schema_version\":2,\"storage\":{\"sd_supported\":false}}").MergeInto(capabilities);
+
+        Assert.False(capabilities.HasSdCard);
+        Assert.True(capabilities.HasWiFi);
+        Assert.True(capabilities.HasUsb);
+        Assert.Equal(16, capabilities.AnalogInputChannels);
+        Assert.Equal(1000, capabilities.MaxSamplingRate);
+    }
+
+    [Fact]
+    public void MergeInto_ChannelArrayIsOverlaidAsASet()
+    {
+        // channels[] is the board's complete channel list, so once it is present a count of zero
+        // for a kind is a real answer rather than a gap to fall back on.
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+        capabilities.AnalogInputChannels = 99;
+        capabilities.AnalogOutputChannels = 99;
+        capabilities.DigitalChannels = 99;
+
+        Parse("{\"schema_version\":2,\"channels\":[{\"id\":0,\"kind\":\"analog-input\"}]}")
+            .MergeInto(capabilities);
+
+        Assert.Equal(1, capabilities.AnalogInputChannels);
+        Assert.Equal(0, capabilities.AnalogOutputChannels);
+        Assert.Equal(0, capabilities.DigitalChannels);
+    }
+
+    [Fact]
+    public void MergeInto_NonPositiveMaximumSampleRate_DoesNotLowerTheCeiling()
+    {
+        // A 0 or negative ceiling would make every streaming frequency invalid; keep the board's.
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Nyquist1);
+
+        Parse("{\"schema_version\":2,\"streaming\":{\"sample_rate_range_hz\":{\"min\":1,\"max\":0}}}")
+            .MergeInto(capabilities);
+
+        Assert.Equal(1000, capabilities.MaxSamplingRate);
+    }
+
+    [Fact]
+    public void MergeInto_UnknownBoard_DoesNotTurnHardwareFlagsOffThatTheDocumentOmits()
+    {
+        // Preserves the ADR 0001 rule that all-false capabilities on an Unknown board mean
+        // "not yet known", not "hardware absent": an omitted field never flips a flag.
+        var capabilities = DeviceCapabilities.FromDeviceType(DeviceType.Unknown);
+
+        Parse("{\"schema_version\":2,\"transports\":{\"usb\":{\"supported\":true}}}")
+            .MergeInto(capabilities);
+
+        Assert.True(capabilities.HasUsb);
+        Assert.False(capabilities.HasSdCard);
+        Assert.False(capabilities.HasWiFi);
+    }
+
+    [Fact]
+    public void MergeInto_NullCapabilities_Throws()
+    {
+        Assert.Throws<ArgumentNullException>(() => BenchDocument().MergeInto(null!));
+    }
+}