feat(epcis): async capture, private-by-default, partition-aware events

[zsculac]

Summary

Rewires POST /api/epcis/capture and GET /api/epcis/events to be private-by-default, per-request context graph, partition-aware, async-only. Bare EPCIS documents now publish into the private partition by default; only a dkg:privateDataAnchor "true" triple per root entity is left in the public partition. Captures and queries accept contextGraphId + subGraphName per request (with a config fallback for single-CG deployments). The events query exposes a ?finalized=true|false boolean to switch between the canonical (durable) and shared-working-memory (in-flight) public partitions; in both modes the response merges the matching private partition for events the local node can see.

A new CLI tree dkg epcis {capture,status,query} wraps the HTTP API. A multi-node devnet smoke-test (scripts/epcis-smoke-test.sh) verifies the full privacy and authorization contract end-to-end.

Why private-by-default

EPCIS supply-chain events carry commercially sensitive data: bizSteps reveal operational stages, EPC lists reveal materials in transit, partner identifiers live in bizTransactionList. The previous async capture path wrapped a bare document as { public: doc } before publishing, forcing every captured event to be visible to all CG subscribers. Operators capturing real supply-chain data had to choose between leaking everything or skipping EPCIS.

Bare document → private partition is the smallest, hardest-to-misuse default that preserves discoverability (anchors leak) while protecting payload (full triples stay local). Callers who want a fully public capture send { public: <doc> }; hybrid captures send { public, private }. The agent layer's existing defaultVisibility: 'private' is now honored end-to-end.

Alternatives considered:

• Public default (status quo). Rejected: leaks sensitive data by default; forgetting a flag has high blast radius.
• Required envelope ({ public, private } always). Rejected: hostile to the public-only and private-only common cases; breaks every existing client that posts bare EPCIS documents.
• Per-event privacy flags inside the EPCIS document. Rejected: not in GS1 EPCIS 2.0 spec; non-standard extension; complicates the partition-aware query.
• Public default + private=true query param. Rejected: out-of-band metadata for what is fundamentally a payload-shape decision.

Wire contract

Capture

POST /api/epcis/capture
{
  "contextGraphId"?: string,           // optional, falls back to config.epcis.contextGraphId
  "subGraphName"?: string,             // optional, no fallback
  "epcisDocument": <bareDoc | { public, private } | { public } | { private }>,
  "publishOptions"?: {
    "accessPolicy"?: "public" | "ownerOnly" | "allowList",
    "allowedPeers"?: string[]
  }
}

→ 202 Accepted { captureID, receivedAt, eventCount, status: "accepted" }
→ 400 InvalidContent           validation, missing CG with no fallback, bad CG/subGraph
→ 404 ContextGraphNotFound     CG not subscribed locally
→ 503 PublisherDisabled / PublisherUnavailable / EnqueueFailed

Bare documents wrap as { private: doc } internally. The validator runs against the document the publisher will actually publish (bare doc, envelope's public, or envelope's private — whichever is present).

Status

GET /api/epcis/capture/:captureID
→ 200 { captureID, state, receivedAt, finalizedAt, error }
→ 404 CaptureNotFound

state reflects the lift queue's machine: accepted | claimed | validated | broadcast | included | finalized | failed.

Events query

GET /api/epcis/events?
  contextGraphId=...        # optional, falls back to config
  &subGraphName=...         # optional, no fallback
  &finalized=true|false     # default true (canonical, durable view)
  &epc=...&bizStep=...&from=...&to=... &eventID=... &eventType=...
  &parentID=... &childEPC=... &inputEPC=... &outputEPC=... &anyEPC=...
  &action=... &disposition=... &readPoint=... &bizLocation=...
  &perPage=... &nextPageToken=...

Response stays a flat GS1 EPCIS 2.0-conformant EPCISQueryDocument. The SPARQL builder unions:

• Pure-public events: full EPCIS triples in the chosen public partition (canonical when finalized=true, SWM when finalized=false).
• Anchored-private events: dkg:privateDataAnchor "true" in the chosen public partition AND the matching event's payload in <cg>/_private.

Orphan private payloads (no matching anchor in the chosen public partition) are excluded — an integrity guard.

CLI

dkg epcis capture <document.json> [--context-graph-id <id>] [--sub-graph-name <name>]
                                  [--access-policy public|ownerOnly|allowList]
                                  [--allowed-peer <peerId>]    # repeatable
dkg epcis status <captureID>
dkg epcis query [--context-graph-id <id>] [--sub-graph-name <name>] [--finalized true|false]
                [--epc <epc>] [--biz-step <step>] [--from <ts>] [--to <ts>]
                [--event-id <id>] [...] [--all]

Exit codes: 0 success, 1 unexpected error, 2 client error (4xx), 3 service unavailable (503), 4 not found (404).

Slices

This branch is the integration of seven slices. Each was developed on its own slice branch, devnet-verified where applicable, and merged here.

#	Slice	Highlights
01	capture private-by-default + drop sync handler	bare EPCIS doc → { private: doc }; sync handleCapture removed
02	capture per-request contextGraphId + subGraphName	body fields with config fallback for contextGraphId
03	events query ?finalized partition selector + private merge	SPARQL UNION public partition + private partition by anchor
04	events query per-request contextGraphId + subGraphName	mirrors slice 02 on the query side; bundled SPARQL builder fix for cross-GRAPH joins
03b	SWM anchor subject IRI fix	publisher's lift validator was rewriting subjects to a synthesized IRI; SWM anchor and _private payload disagreed and never joined. Fix: keep source IRI through validation.
05	CLI dkg epcis {capture,status,query} subcommands	wraps the HTTP API; pagination via --all follows Link: rel="next"; full exit-code matrix
06	multi-node devnet privacy + auth-gate smoke test	curated CG with explicit publisher-wallet auth; 11 scenarios; all PASS

Testing

Three layers of verification, each addressing a different failure mode.

Unit (per-package)

• EPCIS query SPARQL builder: table-driven over partition × subGraphName × representative filter sets.
• Capture content resolver: bare → { private } wrap; envelope passthrough; envelope rejection.

Handler / route (in-process HTTP stub)

• Capture: bare → publisher receives { private }; envelope passthrough; access-policy + allowed-peers propagation; per-request CG + fallback; CG/subgraph validation.
• Events query: ?finalized=true|false selects the right public graph URI; private payload merging; orphan exclusion; subgraph variant; back-compat for no-contextGraphId callers.
• CLI: 32 spawn tests covering happy path, flag-to-body translation, envelope parsing, --all pagination, full exit-code matrix.

Suite results on this branch:

• pnpm -F @origintrail-official/dkg-epcis test — 142/142 pass
• pnpm -F @origintrail-official/dkg-cli test (epcis subset) — 49/49 pass
• npx tsc --noEmit — clean across packages/{epcis,publisher,cli,agent}

Devnet end-to-end (multi-node)

scripts/epcis-smoke-test.sh boots a 6-node devnet and runs an 11-scenario verification on top of a freshly created curated context graph with the publisher wallet explicitly registered as authorized. Scenarios cover:

1. Capture bare doc → 202 + captureID.
2. Poll status → reaches finalized (proves wallet auth + on-chain publish lifecycle).
   3-4. Events query in both ?finalized= modes returns the captured event with full private payload populated.
   5-6. Non-allowed observer node (N3) sees an empty event list and a verifiably-empty <cg>/_private partition (privacy contract honored at the API and at raw SPARQL).
   7-8. Allow-list capture (informational — see Known gaps).
3. Repeat raw SPARQL probe on N3 — still empty.
4. Mixed envelope capture: N3 sees public-only, owner sees both halves joined.
5. New auth-gate scenario: capture from an unauthorized node — chain or network gate fires; capture does not finalize. Proves the curated CG's authorization gate is real and not silently no-op'd.

Latest devnet result on this branch: 11/11 PASS (1 informational). Report committed at docs/epcis/devnet-results-2026-05-05.md. Per-slice live e2e reports also committed under docs/epcis/.

Known gaps tracked separately

• EPCIS: allow-list private payload propagation has no receiver-side pull #409 — allow-list private payload propagation has no receiver-side pull. The publisher correctly authorizes peers in allowedPeers (server-side AccessHandler enforces it), but no code on the receiver's side ever calls AccessClient.requestAccess. The PRD's "allowed peers receive private payload" promise is structurally incomplete in this branch. The privacy gate is verifiably correct (non-allowed peers cannot fetch); the delivery requires a follow-up. Scenario 8 in the smoke test is therefore informational.

• Three pre-existing publishJsonLd agent tests assert that agent.publishAsync populates <cg>/_private immediately on return; in the actual model publishAsync writes to staging only, and the lift runner moves to <cg>/_private later. The contract is exercised live by slice/03b's devnet probe and by slice/06's smoke test. The stale unit tests should be either rewritten to drive a real lift runner or moved to integration tests in a separate cleanup.

Out of scope

• New auth scopes specific to EPCIS — existing daemon bearer-token auth gates these endpoints unchanged.
• New ontology terms beyond dkg:privateDataAnchor, which already existed.
• Per-event access policies (the unit of access policy stays the lift job / KC).
• EPCIS XML — endpoints accept JSON-LD only, as today.
• Receiver-side allow-list pull — tracked in EPCIS: allow-list private payload propagation has no receiver-side pull #409.

Test plan

• pnpm install && pnpm typecheck clean
• pnpm -F @origintrail-official/dkg-epcis test passes
• pnpm -F @origintrail-official/dkg-cli test passes (epcis subset green; pre-existing PROD-BUG audit failures are unrelated)
• ./scripts/devnet.sh start && ./scripts/epcis-smoke-test.sh exits 0 (11/11 PASS)
• dkg epcis capture <doc> --context-graph-id <cg> and dkg epcis query --context-graph-id <cg> smoke-tested manually against the devnet

🤖 Generated with Claude Code

[github-actions]

Codex review skipped: filtered diff is 5658 lines (cap: 5,000). Please consider splitting this into smaller PRs for reviewability.

[github-actions]

Codex review skipped: filtered diff is 37474 lines (cap: 5,000). Please consider splitting this into smaller PRs for reviewability.

[github-actions]

Codex review skipped: filtered diff is 39547 lines (cap: 5,000). Please consider splitting this into smaller PRs for reviewability.