Skip to content

feat(1474): expose ledger-emitted events from the node#1849

Open
m2ux wants to merge 15 commits into
mainfrom
feat/1474-ledger-events-v2
Open

feat(1474): expose ledger-emitted events from the node#1849
m2ux wants to merge 15 commits into
mainfrom
feat/1474-ledger-events-v2

Conversation

@m2ux

@m2ux m2ux commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Expose the ledger's per-transaction events as additive Substrate runtime events so the indexer and other consumers read them from the node instead of re-executing every transaction to reconstruct them.

🐛 Issue 📐 Engineering 📐 ADR 🧪 Test plan


Motivation

When the node applies a transaction the ledger emits a stream of events describing what happened, and the node immediately discards it. Every consumer that needs those events — the indexer above all — must re-run each transaction locally to reconstruct them, duplicating compute on every operator's machine and slowing time-to-visibility for wallets and explorers. Issue #1474 also asks whether publishing is worth the network cost; it is — deposited events live in the frame_system::Events state trie (cleared each block), not the gossiped block body, so there is no incremental peer-to-peer traffic.


Changes

  • pallet-midnight / pallet-midnight-system - additive Event::LedgerEvent (index 8 / index 1); one deposited per ledger event. Existing variant indices unchanged.
  • Ledger v7/v8/v9 + Bridge - stop discarding the event stream at the apply sites; a single build_ledger_events helper serialises each event as the ledger's own tagged bytes into an additive events field on both apply-return structs. Version-uniform; the v9 payload divergence is absorbed opaquely by the tag.
  • Pricing / benchmark - emission left unpriced (matches FRAME whitelisted-event convention); bench_block_full_of_events ships as the worst-case block-weight guardrail.
  • Docs / change file - docs/ledger-events.md; changes/runtime/added/expose-ledger-events.md.
  • Tests - unit/pallet coverage for per-event deposit, payload round-trip, the last-variant index, and the failed / dry-run emit-nothing paths; an invalidated regression test was updated. Remaining cases (partial-success, integration) are deferred — see the TODO below.

🤖 AI Assistance

  • Assistant / Model: Claude Code / claude-opus-4-8 (1M context)
  • Context scope: mixed
  • Prompt classes: code-generation, test-writing, refactoring, docs
  • Provenance log: 08-provenance-log.md

📌 Submission Checklist

  • Changes are backward-compatible (or flagged if breaking)
  • Pull request description explains why the change is needed
  • Self-reviewed the diff
  • I have included a change file, or skipped for this reason: changes/runtime/added/expose-ledger-events.md
  • If the changes introduce a new feature, I have bumped the node minor version
  • Update documentation (if relevant)
  • No new todos introduced

🔱 Fork Strategy

  • Node Runtime Update
  • Node Client Update
  • Other
  • N/A

🗹 TODO before merging

  • Build and test the change (it was not compiled on the authoring machine) — the steps are in the validation checklist: build the ledger crate and both pallets, run the event-emission benchmark, and run fmt/clippy
  • Rebuild the runtime metadata — the new event variants change it, and metadata-pinned tooling must be regenerated
  • Bump the node minor version for this new feature
  • Confirm with stakeholders that the indexer is the only service that needs to read these events directly from the node (asked on the issue thread)
  • Add the remaining cases from the test plan — the written subset covers per-event deposit, payload round-trip and the emit-nothing paths; the partial-success and integration cases remain (pallet-midnight-system needs a test harness first)
  • Confirm additive-field decode-safety against historical block replay, and measure the per-event serialisation cost on the transaction-apply path — both unverified by the current tests (see the test plan)
  • Ready for review

Scaffold commit to open the draft PR for midnight-node#1474.
Implementation follows per the work package plan.

Signed-off-by: Mike Clay <mike.clay@shielded.io>
@m2ux m2ux self-assigned this Jul 8, 2026
@m2ux m2ux linked an issue Jul 8, 2026 that may be closed by this pull request
m2ux and others added 11 commits July 8, 2026 12:17
…ht-node#1474)

Stop discarding the per-transaction Vec<Event<D>> that the ledger already
produces. Ledger::apply_verified_transaction now returns the event stream
alongside its applied-stage classification, and Ledger::apply_system_tx keeps
the event vec from its success tuple instead of dropping it.

Because versions/common/ compiles once per ledger version via module
parameterization, this single source edit lands in v7, v8 and v9. The two
Bridge call sites destructure the vec but discard it for now; T3 consumes it.
Failure semantics are unchanged: TransactionResult::Failure still returns
LedgerApiError::Transaction(Invalid(..)) and emits no events.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…eld (midnight-node#1474)

Introduce the SCALE wire shape that carries ledger events across the host
boundary, using the routing-header hybrid from the plan (section 3):

- LedgerEventSource mirrors the ledger's EventSource (tag event-source[v1],
  stable across v7/v8/v9) field-for-field, so consumers can route on the
  (transaction_hash, logical_segment, physical_segment) triple.
- LedgerEvent { source, content_tagged_bytes } keeps the variable/divergent
  EventDetails<D> payload opaque as the ledger's own tagged bytes. The tag is
  self-describing, so version divergence (event-details[v9] vs [v14], the v9
  ContractLog.logged_item change) needs no node-side branching.

Both host-return structs gain an additive events: Vec<LedgerEvent> field,
appended last (design decision D1). Appending keeps the field decode-safe
against historical replay: a decoder built with the field defaults it to an
empty vec on bytes encoded before it existed, so no new #[version(N)] host
function is required.

Error plumbing: a new SerializationError::EventDetails variant and its
SerializableError impl route tagged-serialise failures on EventDetails<D>
through the existing LedgerApiError::Serialization channel. The two Bridge
constructors get empty-vec placeholders; T3 populates them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…ts (midnight-node#1474)

Consume the event stream that T1 surfaced in Bridge::apply_transaction and
Bridge::apply_system_transaction. A shared build_ledger_events helper mirrors
each EventSource into the SCALE LedgerEventSource header and tagged-serialises
the EventDetails<D> payload into content_tagged_bytes, then populates the
additive events field on the return struct.

The helper is uniform across all three ledger versions: the tag is embedded in
the serialised bytes, so the v9 divergence (event-details[v14],
ContractLog.logged_item change) is absorbed opaquely with no per-version
conversion code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…idnight (midnight-node#1474)

Add Event::LedgerEvent(LedgerEvent) to the pallet-midnight event enum, appended
last so the existing eight variants keep their positional indices. In
send_mn_transaction, iterate the new result.events field and deposit one runtime
event per ledger event (design decision D2), preserving per-event
(pallet_index, variant_index) filtering for subxt / Polkadot.js consumers.

The existing deposit sites are byte-identical. Emission is non-consensus
narration and is not weighed (accept-unpriced, D3); the T7 benchmark is the
guardrail for that decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…ght-node#1474)

Apply the symmetric system-tx change (design decision D4). Add a sibling
Event::LedgerEvent(LedgerEvent) variant to pallet-midnight-system, appended
last so the existing SystemTransactionApplied index is unchanged.

Both system-tx apply paths — the send_mn_system_transaction dispatchable and
the MidnightSystemTransactionExecutor::execute_system_transaction trait impl —
now carry the new result.events field out of the mut_ledger_state closure and
deposit one runtime event per ledger event beside the existing
SystemTransactionApplied deposit. This surfaces ParamChange, DustInitialUtxo
and dust-generation events that were previously invisible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…dnight-node#1474)

Add the worst-case event-emission benchmark that discharges GO-gate condition
C1 / acceptance metric M8. It fills a block with up to MAX_BENCH_EVENTS ledger
events, each carrying a deploy-heavy ~4 KiB opaque payload, so the total event
volume (~1 MiB) tracks the ledger's per-block bytes_churned ceiling — the
code-grounded worst-case magnitude from the comprehension artifact (DD11).

The benchmark measures the runtime-side deposit cost (the state-trie write into
frame_system::Events) that the accept-unpriced decision (D3) deliberately
leaves unweighed. The human runs it and compares the reported weight against
BlockWeights headroom; this converts the accept-unpriced choice into a decision
backed by measurement.

Compilation and execution are deferred to the human runner (cargo bench /
runtime-benchmarks harness).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…node#1474)

Add docs/ledger-events.md covering the new runtime-event surface:

- What is emitted: the per-event LedgerEvent variant on pallet-midnight and its
  sibling on pallet-midnight-system, and the routing-header-plus-opaque-bytes
  wire shape.
- Indexer-author guidance: consume from frame_system::Events via
  subscribeStorage / getStorage, and decode content_tagged_bytes with the
  matching ledger version's tagged_deserialize (the tag self-identifies the
  version).
- Contract-author guidance: (address, entry_point) is the contract-event
  namespace; no separate node-side topic field.
- Pricing: the accept-unpriced decision, why it is safe, and the
  bench_block_full_of_events guardrail plus the per-event-weight-term fallback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
Add changes/runtime/added/expose-ledger-events.md per the in-tree change-file
convention (component-tagged runtime addition), describing the new
LedgerEvent runtime-event surface and pointing at docs/ledger-events.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…node#1474)

Trim agent-authored doc blocks and inline comments on the new ledger-event
surface down to terse one-liners, matching the proportionality of the
surrounding code. Comment-only; no logic change.

Ref: #1474

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…1474)

Add PR1849-TC-01/TC-02 payload round-trip and TC-11/TC-14 wire-type
SCALE round-trip. Strengthen the apply_verified_transaction test helper
to return and assert the emitted events instead of discarding them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
…de#1474)

Add PR1849-TC-01 per-event deposit with tx-hash routing, TC-15 failed-tx
emits nothing, TC-17 dry-run emits nothing, and TC-11 last-variant index
check. Update the existing send_mn_transaction assertion for the new
interleaved LedgerEvent records.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
@datadog-official

datadog-official Bot commented Jul 8, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 1 Pipeline job failed

+test | Run tests   View in Datadog   GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 61c9c34 | Docs | Give us feedback!

@m2ux m2ux marked this pull request as ready for review July 13, 2026 10:46
@m2ux m2ux requested a review from a team as a code owner July 13, 2026 10:46
@jina-simulation

jina-simulation Bot commented Jul 13, 2026

Copy link
Copy Markdown

Jina Review

Jina has completed this review.

Review: https://app.usejina.com/reviews/b10b40ce-bde6-4742-8c1c-d58aa41f7ed1

Item Status
Review Completed
Findings No issues found

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@jina-simulation jina-simulation Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime Review - Merge Readiness 5/5

0 issues found - Ready to merge

There are no accepted candidate issues, so Jina has no evidence-based blocking finding.

The metadata rebuild is an important release follow-up but is recorded as a non-blocking technical observation because no investigation candidate fingerprint exists.

Final review: No investigation candidates were provided, and direct source review found no adjudicable runtime correctness issue.

Generated metadata still needs rebuilding before downstream consumers can decode the new events.

Review Comments

  • Rebuild generated runtime metadata

    The checked-in metadata files do not contain the new LedgerEvent variants, while the PR changes both pallet event enums.

    The change documentation correctly notes that a metadata rebuild is required; downstream static subxt consumers will not see or decode these events until metadata/static is regenerated.

    Evidence:

    • git diff origin/main...HEAD contains no metadata/static changes
    • strings metadata/static/midnight_metadata_2.0.0.scale shows SystemTransactionApplied but no LedgerEvent
    • metadata/src/lib.rs generates the latest bindings from metadata/static/midnight_metadata_2.0.0.scale

    Related files: metadata/static/midnight_metadata_2.0.0.scale, metadata/src/lib.rs, pallets/midnight/src/lib.rs, pallets/midnight-system/src/lib.rs

Issues

No medium/high-confidence runtime issues were found.

Summary

  • Status: warned
  • Areas investigated: 6
  • Tasks/probes performed: 0
  • Issues reported: 0
  • Publishable issues: 0
  • Inline comments: 0
  • File-level comments: 0
  • Review comments: 1
  • Unanchored publishable findings posted as file-level comments: 0
  • Blocked validations: 0
  • Dashboard: full runtime review
  • Commit: ae03405
  • Changed files: 11
  • Tool calls: none

…es (midnight-node#1474)

The event-stream work added `Vec<Event<D>>` to the return tuples of
`apply_verified_transaction` and `apply_system_tx`, pushing both past clippy's
type-complexity threshold under `-D warnings` on the Rust 1.95 toolchain
(merged from main). The tuple shapes are intentional and read clearly at the
call sites, so annotate both methods with `#[allow(clippy::type_complexity)]`.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Mike Clay <mike.clay@shielded.io>
@jina-simulation

jina-simulation Bot commented Jul 13, 2026

Copy link
Copy Markdown

Jina Review

Jina has completed this review.

Review: https://app.usejina.com/reviews/3a2f0b56-6216-4e4e-9753-b3d76255801f

Item Status
Review Completed
Findings No issues found

@jina-simulation jina-simulation Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime Review - Merge Readiness 5/5

0 issues found - Ready to merge

Ready to merge based on the accepted-issue rubric: there are no accepted issues.

The stale generated metadata observation is non-blocking and was already reported in the PR thread.

Final review: No investigation candidates were provided or substantiated.

No accepted runtime issues remain; the metadata rebuild observation is already covered by prior PR context.

Issues

No medium/high-confidence runtime issues were found.

Summary

  • Status: warned
  • Areas investigated: 5
  • Tasks/probes performed: 0
  • Issues reported: 0
  • Publishable issues: 0
  • Inline comments: 0
  • File-level comments: 0
  • Review comments: 0
  • Unanchored publishable findings posted as file-level comments: 0
  • Blocked validations: 0
  • Dashboard: full runtime review
  • Commit: 84169dd
  • Changed files: 11
  • Tool calls: none

The ledger-event work deposits LedgerEvent from pallet-midnight and
pallet-midnight-system, changing the runtime event surface, so the checked-in
static metadata no longer matches the runtime (failing the metadata check and
the toolkit's static codegen). Regenerate midnight_metadata.scale and the
2.0.0 snapshot via `subxt metadata -f bytes` (subxt 0.50.0) against the node
image built for this branch.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Mike Clay <mike.clay@shielded.io>
@jina-simulation

jina-simulation Bot commented Jul 14, 2026

Copy link
Copy Markdown

Jina Review

Jina has completed this review.

Review: https://app.usejina.com/reviews/0e13345f-1343-4d38-939a-77e1f5d81e44

Item Status
Review Completed
Findings Issues found

@jina-simulation jina-simulation Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime Review - Merge Readiness 1/5

8 issues found - Strongly do not merge

Strongly do not merge.

The review found multiple high-risk, medium-likelihood issues affecting runtime upgrade safety, validator compatibility, bridge accounting, and indexer correctness.

In particular, an incompatible mixed-binary upgrade can halt execution, while swallowed system-transaction failures can cause permanent cross-chain accounting loss.

The unpriced and under-benchmarked event stream adds additional production reliability risk, and release automation can publish...

[truncated]

Final review: The review investigated user and system transaction event forwarding, ledger persistence and failure semantics, bridge and observation integrations, host ABI compatibility, metadata and SCALE compatibility, upgrade automation, event-volume limits, and runtime release identity.

Source tracing, CodeGraph queries, metadata comparisons, and bounded SCALE/size probes were performed; Rust tests, benchmarks, node execution, and RPC checks could not run because cargo/rustc and a built node were unavailable.

The issues that must be addressed before merge are: "Root governance SystemTransactionApplied uses the state root instead of the transaction hash" because it breaks indexer correlation between system-transaction and LedgerEvent records; "System LedgerEvent payload growth is not included in caller weight" because bridge and operational system paths can accept variable event work without corresponding weight protection; "New runtime cannot decode ledger host responses from an old node binary" because mixed-version validators can fail transaction execution after activation; "C2M bridge discards pending accounting state when system transaction execution fai...

[truncated]

Issues

Inline comments posted

  • Root governance SystemTransactionApplied uses the state root instead of the transaction hash

    Location: pallets/midnight-system/src/lib.rs:144; Risk / impact: high; Evidence confidence: high; Production likelihood: medium; Category: integration; Validation: hybrid

    The root send_mn_system_transaction path returns result.state_root as the variable named hash, then emits SystemTransactionApplied { hash }. LedgerEvent.source.transaction_hash is built from the ledger transaction hash, so the two events cannot be correlated by hash. The internal executor has the same tuple bug for its returned hash.

  • New runtime cannot decode ledger host responses from an old node binary

    Location: ledger/src/common/types.rs:71; Risk / impact: high; Evidence confidence: high; Production likelihood: medium; Category: compatibility; Validation: hybrid

    The runtime-facing TransactionAppliedStateRoot and SystemTransactionAppliedStateRoot structs gained trailing events fields, but the ledger 7/8/9 host APIs still use these shared types. A runtime containing this change, executed by an older node binary whose host function returns the previous struct encoding, attempts to decode a missing events Vec and fails.

  • Event benchmark covers 1 MiB while ledger permits 50 MiB of churn

    Location: pallets/midnight/src/benchmarking.rs:29; Risk / impact: medium; Evidence confidence: medium; Production likelihood: medium; Category: performance; Validation: hybrid

    The event guardrail benchmark assumes MAX_BENCH_EVENTS * BENCH_EVENT_PAYLOAD_BYTES is approximately 1 MiB, but deployed ledger configuration permits 50,000,000 bytesChurned per block. The benchmark therefore does not cover the effective configured upper bound of event-bearing execution.

File-level or unanchored issues

  • C2M bridge discards pending accounting state when system transaction execution fails

    Location: pallets/c2m-bridge/src/lib.rs:305; Risk / impact: high; Evidence confidence: medium; Production likelihood: medium; Category: integration; Validation: hybrid

    The bridge treats a failed Midnight system transaction as handled. execute_serialized_tx swallows executor errors, while handle_subminimal_transfer unconditionally deletes SubminimalTransfers afterward. The same pattern consumes ApprovedMcTxHashes before execution for approved user transfers, so an executor failure can also permanently lose the approval.

  • Runtime activation has no validator binary compatibility gate

    Location: local-environment/src/commands/federatedRuntimeUpgrade.ts:94; Risk / impact: high; Evidence confidence: medium; Production likelihood: medium; Category: compatibility; Validation: hybrid

    The runtime-upgrade flow authorizes and applies a new Wasm runtime after client image rollout health checks, but it never verifies that every validator is running a binary capable of decoding the new host response structs.

  • Image rollout ignores the documented include/exclude service filters

    Location: local-environment/src/commands/imageUpgrade.ts:78; Risk / impact: medium; Evidence confidence: high; Production likelihood: medium; Category: compatibility; Validation: execution

    The CLI advertises --include and --exclude controls and fullUpgrade passes them to imageUpgrade, but imageUpgrade does not apply either filter before recreating services.

  • Release automation can publish the changed runtime ABI under spec_version 2.0.0

    Location: .github/workflows/release-image.yml:142; Risk / impact: medium; Evidence confidence: high; Production likelihood: medium; Category: compatibility; Validation: hybrid

    The PR changes System.Events metadata while runtime/src/lib.rs remains at spec_version 002_000_000. release-image.yml reads that value and assigns runtime tag 2.0.0 before the later archive-changes job optionally bumps the version. Nothing in the release workflow rejects a changed runtime ABI whose spec_version equals the base runtime.

  • System LedgerEvent payload growth is not included in caller weight

    Location: pallets/cnight-observation/src/lib.rs:643; Risk / impact: medium; Evidence confidence: medium; Production likelihood: medium; Category: performance; Validation: source trace

    process_tokens executes a system transaction that now deposits an unbounded number of opaque LedgerEvent payloads, but its mandatory weight is based only on UTXO count and fixed storage operations. The root system transaction path likewise uses a fixed configurable weight. The added event benchmark measures deposit cost separately but does not feed that cost into either caller's weight.

Areas Investigated

  • Committed user-transaction event fidelity and privacy - 4 task(s), 0 issue(s) found
  • System-transaction events across governance and bridge integrations - 4 task(s), 2 issue(s) found
  • Native host boundary, deterministic execution, and runtime upgrade compatibility - 3 task(s), 1 issue(s) found
  • SCALE metadata, RPC retrieval, and version-aware indexer compatibility - 4 task(s), 0 issue(s) found
  • Unpriced event volume, block limits, and node reliability - 3 task(s), 0 issue(s) found
  • Conclusive system transaction hash contract validation - 4 task(s), 0 issue(s) found
  • Outer integration atomicity when event serialization fails - 3 task(s), 1 issue(s) found
  • Prove or disprove production enforcement of the required binary-first upgrade - 2 task(s), 2 issue(s) found
  • Host ABI and Wasm memory amplification at maximum event volume - 3 task(s), 1 issue(s) found
  • Runtime spec-version and metadata cache invalidation for the changed event ABI - 3 task(s), 1 issue(s) found

Review Details

  • Status: issues found
  • Areas investigated: 10
  • Tasks/probes performed: 33
  • Issues found: 8
  • Inline comments: 3
  • File-level comments: 5
  • Full investigation detail (areas, tasks, evidence): view on the Jina dashboard
  • Commit: 61c9c34
  • Changed files: 13

)
.map_err(Error::<T>::from)?;
Ok::<(Vec<u8>, (Hash, Vec<LedgerEvent>)), Error<T>>((
result.state_root,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root governance SystemTransactionApplied uses the state root instead of the transaction hash

Risk: high
Confidence: high
Likelihood: medium
Grounding: runtime review, hybrid

What happens

The root send_mn_system_transaction path returns result.state_root as the variable named hash, then emits SystemTransactionApplied { hash }. LedgerEvent.source.transaction_hash is built from the ledger transaction hash, so the two events cannot be correlated by hash. The internal executor has the same tuple bug for its returned hash.

Root cause

The closure returns a tuple whose first element is result.state_root, but that first element is later treated as the transaction hash.

Why it matters

The feature's primary consumer contract is event correlation. Governance, parameter, dust, and bridge-related ledger effects can become orphaned or attributed to the wrong operation.

Evidence

  • pallets/midnight-system/src/lib.rs:143-145 returns (result.state_root, (result.tx_hash, result.events)).
  • pallets/midnight-system/src/lib.rs:149-152 emits SystemTransactionApplied with hash.
  • ledger/src/versions/common/mod.rs:551-555 stores tx_hash separately and builds events from ledger event sources.
  • pallets/midnight-system/src/lib.rs:198 returns Ok(hash), confirming the wrongly selected first tuple element is externally exposed.

Reproduction or trace

For any successfully applied system transaction, let result.state_root = R and result.tx_hash = H. The root dispatchable binds hash = R, while each LedgerEvent source is derived from H. Therefore SystemTransactionApplied.hash != LedgerEvent.source.transaction_hash whenever R != H, which is the normal case.

Suggested fix

Bind and return result.tx_hash as the event/returned hash, while continuing to persist result.state_root as the ledger state key.

View full runtime review audit trail

pub claim_rewards: Vec<u128>,
pub unshielded_utxos_created: Vec<UtxoInfo>,
pub unshielded_utxos_spent: Vec<UtxoInfo>,
pub events: Vec<LedgerEvent>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New runtime cannot decode ledger host responses from an old node binary

Risk: high
Confidence: high
Likelihood: medium
Grounding: runtime review, hybrid

What happens

The runtime-facing TransactionAppliedStateRoot and SystemTransactionAppliedStateRoot structs gained trailing events fields, but the ledger 7/8/9 host APIs still use these shared types. A runtime containing this change, executed by an older node binary whose host function returns the previous struct encoding, attempts to decode a missing events Vec and fails.

Root cause

The host ABI changed through shared SCALE return structs without introducing separately versioned return types or a compatibility guard. Appending fields is only safe for old decoders consuming new responses; new decoders cannot decode old responses with missing trailing fields.

Why it matters

A mixed-version deployment can stop validators from executing blocks after runtime activation. The PR documentation does not state the required binary-first upgrade ordering or enforce it.

Evidence

  • ledger/src/common/types.rs:61-79 appends events fields to both host return structs.
  • ledger/src/host_api/ledger_7.rs, ledger_8.rs, and ledger_9.rs return those shared structs through AllocateAndReturnByCodec.
  • node/src/service.rs:212-221 registers all three host function sets in the node executor.
  • Executed SCALE probe: old_len=41, new_len=42, new_suffix=00; new decoder on old bytes reports EOF.
  • cargo execution was unavailable in the VM, so the repository test suite could not run.

Reproduction or trace

Encode the pre-change TransactionAppliedStateRoot layout with empty vectors, then decode it as the new layout. The decoder consumes all old fields and fails when reading the appended events Vec. Encoding the new layout and decoding it as the old layout succeeds because the old decoder ignores trailing bytes.

Suggested fix

Keep legacy host methods returning legacy structs and add explicitly versioned host methods/return types for event-bearing responses, or enforce and document a binary-first compatible-node requirement before runtime activation. Add an integration test covering a new runtime against an old node host.

View full runtime review audit trail

const BENCH_EVENT_PAYLOAD_BYTES: usize = 4 * 1024;

/// Upper bound on the number of ledger events deposited in a single block for
/// the worst-case benchmark. `MAX_BENCH_EVENTS * BENCH_EVENT_PAYLOAD_BYTES`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Event benchmark covers 1 MiB while ledger permits 50 MiB of churn

Risk: medium
Confidence: medium
Likelihood: medium
Grounding: runtime review, hybrid

What happens

The event guardrail benchmark assumes MAX_BENCH_EVENTS * BENCH_EVENT_PAYLOAD_BYTES is approximately 1 MiB, but deployed ledger configuration permits 50,000,000 bytesChurned per block. The benchmark therefore does not cover the effective configured upper bound of event-bearing execution.

Root cause

The benchmark hard-codes 256 synthetic 4 KiB events, while the documented safety argument relies on bytesChurned. No event-byte cap or runtime-side event weight term couples the actual emitted event bytes to the benchmarked bound.

Why it matters

A Wasm allocation failure during block execution can reject an otherwise ledger-valid transaction and threaten block execution reliability. Even without a trap, large unpriced event streams can create substantial CPU, heap, and state-write costs.

Evidence

  • pallets/midnight/src/benchmarking.rs:26-32 defines 4 KiB payloads and 256 events.
  • res/dev/ledger-parameters-config.json:162 sets bytesChurned to 50000000.
  • The sizing probe calculated 1,058,306 bytes for the benchmark response lower bound and a 47.68x gap to the configured bytesChurned value.
  • The same probe calculated approximately 100.9 MiB for two simultaneous 50 MiB representations and approximately 201.9 MiB for four.
  • ledger/src/host_api/ledger_9.rs:114 and 142 return event-bearing structs through AllocateAndReturnByCodec.
  • pallets/midnight/src/lib.rs:427 and pallets/midnight-system/src/lib.rs:156-158,194-196 deposit the returned events into runtime storage.

Reproduction or trace

The bounded size probe models one LedgerEvent as 32-byte hash + two u16 fields + SCALE vector length + payload. With 256 events of 4096 bytes, the encoded event vector is 1,058,305 bytes. The configured 50,000,000-byte ledger limit is 47.68 times larger.

Suggested fix

Derive the benchmark bound from the actual maximum event bytes and run it through the complete host/Wasm path, or introduce an explicit event-byte/count limit enforced before host allocation. Add event serialization, host-return, Wasm decode, and System.Events storage costs to the benchmark.

View full runtime review audit trail

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C2M bridge discards pending accounting state when system transaction execution fails

Risk: high
Confidence: medium
Likelihood: medium
Grounding: runtime review, hybrid

What happens

The bridge treats a failed Midnight system transaction as handled. execute_serialized_tx swallows executor errors, while handle_subminimal_transfer unconditionally deletes SubminimalTransfers afterward. The same pattern consumes ApprovedMcTxHashes before execution for approved user transfers, so an executor failure can also permanently lose the approval.

Root cause

execute_serialized_tx has no Result return value and suppresses executor failures. Callers therefore cannot distinguish successful ledger commitment from construction/execution failure before mutating or deleting bridge bookkeeping state.

Why it matters

A rare event-payload serialization failure introduced by this PR can become permanent cross-chain accounting loss rather than a retriable failed transfer.

Evidence

  • pallets/c2m-bridge/src/lib.rs:266-280 logs executor Err without returning it.
  • pallets/c2m-bridge/src/lib.rs:296-305 kills SubminimalTransfers after the helper regardless of result.
  • pallets/c2m-bridge/src/lib.rs:343 removes ApprovedMcTxHashes before invoking the executor.
  • ledger/src/versions/common/mod.rs:555-586 makes EventDetails serialization a fallible part of system transaction execution.
  • cargo test probe could not run because cargo is unavailable.

Reproduction or trace

Set SubminimalTransfersConfiguration threshold to 250, process two 90-unit transfers, then process a third 90-unit transfer while the executor returns Err. The helper logs the error, and handle_subminimal_transfer executes SubminimalTransfers::kill(), leaving no pending state for retry.

Suggested fix

Return Result from execute_serialized_tx and only kill accumulators or consume approvals after confirmed executor success. Alternatively wrap the full bridge handling path transactionally and propagate failure so caller state rolls back.

View full runtime review audit trail

This issue references a file outside this PR diff, so Jina posted it on this changed file to make the thread resolvable.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime activation has no validator binary compatibility gate

Risk: high
Confidence: medium
Likelihood: medium
Grounding: runtime review, hybrid

What happens

The runtime-upgrade flow authorizes and applies a new Wasm runtime after client image rollout health checks, but it never verifies that every validator is running a binary capable of decoding the new host response structs.

Root cause

The upgrade tooling treats container health and rollout completion as proof of compatibility. authorizeUpgrade checks runtime authorization/spec-version state, not the host binary version of every validator.

Why it matters

A mixed validator set can fail only when ledger transactions execute, risking stalled authoring/import and difficult-to-diagnose network degradation after governance activation.

Evidence

  • fullUpgrade.ts:31-49 performs imageUpgrade followed by federatedRuntimeUpgrade.
  • federatedRuntimeUpgrade.ts:94-96 calls authorizeUpgrade or authorizeUpgradeWithoutChecks.
  • federatedRuntimeUpgrade.ts:162 calls applyAuthorizedUpgrade.
  • No validator discovery, binary-version query, peer handshake compatibility check, or minimum compatible node version is implemented.
  • node/src/service.rs:203-216 registers ledger host functions in the node executor.
  • Prior round execution established that decoding an old host response with the new struct reaches EOF at the appended events field.

Reproduction or trace

Trace fullUpgrade -> imageUpgrade -> health polling -> federatedRuntimeUpgrade -> authorizeUpgrade -> applyAuthorizedUpgrade. There is no branch that queries validator versions or blocks activation based on compatibility.

Suggested fix

Add an explicit pre-authorization compatibility check over all validators, publish and enforce a minimum node version, and fail closed if any validator is unreachable or below that version. Add a mixed-binary integration test with an empty block followed by a transaction-bearing block.

View full runtime review audit trail

This issue references a file outside this PR diff, so Jina posted it on this changed file to make the thread resolvable.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image rollout ignores the documented include/exclude service filters

Risk: medium
Confidence: high
Likelihood: medium
Grounding: runtime review, execution

What happens

The CLI advertises --include and --exclude controls and fullUpgrade passes them to imageUpgrade, but imageUpgrade does not apply either filter before recreating services.

Root cause

imageUpgrade.ts selects opts.services or listServices(...) and immediately iterates the result; includePattern and excludePattern are never read.

Why it matters

Operators cannot rely on documented service-selection controls during a sensitive binary-first upgrade, making staged rollout rehearsals misleading and increasing the chance of an unintended mixed or broad deployment.

Evidence

  • index.ts:309-310 documents --include and --exclude.
  • fullUpgrade.ts:38-39 forwards includePattern and excludePattern.
  • imageUpgrade.ts:78 selects services without filtering.
  • imageUpgrade.ts:95 iterates every selected service.
  • The scoped executable probe reported no includePattern/excludePattern references in imageUpgrade.ts.

Reproduction or trace

Invoke full-upgrade with --exclude 'node6'. The implementation still obtains the complete compose service list and loops over every service because no filter operation exists.

Suggested fix

Apply includePattern and excludePattern to the discovered or explicit service list before rollout, and log the final filtered list. Add tests covering include, exclude, and an empty filtered result.

View full runtime review audit trail

This issue references a file outside this PR diff, so Jina posted it on this changed file to make the thread resolvable.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release automation can publish the changed runtime ABI under spec_version 2.0.0

Risk: medium
Confidence: high
Likelihood: medium
Grounding: runtime review, hybrid

What happens

The PR changes System.Events metadata while runtime/src/lib.rs remains at spec_version 002_000_000. release-image.yml reads that value and assigns runtime tag 2.0.0 before the later archive-changes job optionally bumps the version. Nothing in the release workflow rejects a changed runtime ABI whose spec_version equals the base runtime.

Root cause

Runtime version extraction is informational and tag-generating only; the optional bump is performed asynchronously after release preparation and is not an invariant or gate.

Why it matters

A production runtime ABI change can be distributed with the same identity used by old metadata caches and historical decoders, causing event decoding failures or silent misinterpretation by indexers and applications.

Evidence

  • runtime/src/lib.rs:283 contains spec_version: 002_000_000.
  • The base and PR both report spec_version 002_000_000.
  • The base metadata SHA is c68fffda80949d98a27051416a30798960539c59744a5f67b6e68a8d04011932 while the PR metadata SHA is cbbb09d7b1d7bd2b3862abf4c0451ced04f1fb1334d3eec1ebcb58e2bdc27e02.
  • Executing the release-image parsing logic produced release_image_runtime_tag=2.0.0.
  • release-image.yml runs archive-changes only after prepare-release and finalize-release.
  • archive-changes.yml performs the runtime bump only when inputs.bump-version is true.

Reproduction or trace

The changed checkout reports the same runtime spec_version as origin/main but a different generated metadata artifact. The release workflow's own parsing logic maps the changed checkout to runtime-2.0.0.

Suggested fix

Make release validation fail when runtime ABI/metadata changes without a spec_version change, or require the bump before prepare-release and build/tag generation. Add an automated check comparing the PR/base runtime identity when runtime metadata or pallet event definitions change.

View full runtime review audit trail

This issue references a file outside this PR diff, so Jina posted it on this changed file to make the thread resolvable.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System LedgerEvent payload growth is not included in caller weight

Risk: medium
Confidence: medium
Likelihood: medium
Grounding: runtime review, source trace

What happens

process_tokens executes a system transaction that now deposits an unbounded number of opaque LedgerEvent payloads, but its mandatory weight is based only on UTXO count and fixed storage operations. The root system transaction path likewise uses a fixed configurable weight. The added event benchmark measures deposit cost separately but does not feed that cost into either caller's weight.

Root cause

Event emission is deliberately unpriced, while event payload size is opaque and no event-count or byte-size limit is enforced at the system pallet boundary.

Why it matters

Production block authors and validators receive no weight-based protection against the additional event serialization and storage work introduced by this PR.

Evidence

  • pallets/cnight-observation/src/lib.rs:643 assigns process_tokens a capacity-based mandatory weight.
  • pallets/cnight-observation/src/lib.rs:729-737 invokes the system executor internally.
  • pallets/cnight-observation/src/lib.rs:752-754 reports weight only as process_tokens(utxo_count).
  • pallets/cnight-observation/src/weights.rs:48-59 has no serialized-byte or event-payload term.
  • pallets/midnight-system/src/lib.rs:114-120 uses a fixed system transaction weight.
  • pallets/midnight-system/src/lib.rs:193-195 deposits every ledger event without an event cap.
  • 1 more evidence item(s) recorded.

Reproduction or trace

Trace process_tokens with an input producing N ledger events: its reported weight remains the existing count-based process_tokens(utxo_count), while execute_system_transaction loops over all returned events and deposits each opaque Vec. Increasing payload bytes changes storage work without changing the weight formula.

Suggested fix

Add an explicit event count/byte bound and charge the corresponding weight in every entrypoint, or ensure ledger synthetic-cost limits strictly and demonstrably bound emitted event bytes.

View full runtime review audit trail

This issue references a file outside this PR diff, so Jina posted it on this changed file to make the thread resolvable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Node should expose events emitted by the Ledger

1 participant