ci: fail open on sccache when R2 secrets are absent#475
Open
mfw78 wants to merge 53 commits into
Open
Conversation
Drop the nexum:intent use from nexum:host/types: the intent-status event now carries an opaque versioned status body and an opaque receipt, so nexum:host is a leaf package. The core bindgen resolves against wit/nexum-host alone, module worlds pull the intent packages only with the pool capability, and the intent vocabulary generates in the adapter bindgen. The body codec lives in the new nexum-status-body crate: a leading u8 version tag, then the version's borsh payload (v1: lifecycle status, optional settlement proof, optional fail reason). Unknown tags fail closed, a body is never empty, and goldens pin the wire form. The pool router lowers an adapter-reported status through it (settled is fulfilled plus proof, failed is cancelled plus reason) and keepers decode via nexum_sdk::status_body.
Rename the pre-release intent packages into the videre namespace and
reshape them to the pinned 0.1.0 surface: nexum:value-flow becomes
videre:value-flow, nexum:intent splits into videre:types and the
two-faced videre:venue (worker client, provider adapter), and
nexum:adapter retires with its venue-adapter world folded into
videre:venue. nexum:host and shepherd:cow are untouched.
The 0.1 shapes are EVM-only and thin: single-asset gives/wants,
auth-scheme {eip1271, eip712}, settlement {chain}, uint amounts
(big-endian, minimal-length), asset {native, erc20{token}}, plain-enum
intent-status, the seven-case venue-error with rate-limit, unsigned-tx
{chain, to, value, data}, and the quotation record. Dropped cases
return as additive 0.2+ variants; quote functions land separately.
Rust follows the wire: PoolRouter is VenueRegistry, AdapterActor is
VenueActor, GuardPolicy/AllowAllGuard collapse into EgressGuard (the
unit guard allows every egress), IntentPool is VenueClient, PoolQuota
is SubmitQuota, and a VenueId newtype keys the registry. Quota
exhaustion now reports rate-limited with the window; adapter traps
project onto unavailable. The module capability is client; goldens
and the conformance mirrors follow the single-asset header.
Reset nexum:host and shepherd:cow package and use versions to 0.1.0, the shared baseline every package now semvers from independently. wasi:* keeps its own 0.2.x. Drop the 0.1-to-0.2 migration guide and its references. Closes #371
A local command (scripts/check-venue-agnostic.sh, just check-venue-agnostic) and a non-blocking CI job assert nexum-runtime is venue-agnostic: its crate graph reaches no videre/intent/venue/cow crate, its sources carry no venue symbol, and nexum:host resolves as a leaf WIT package. The symbol scan is red by design until the physical host cut lands; the flip to a blocking gate is tracked for M2.
…cannot slip the guard
A deny verdict at the venue-registry guard seam now logs a would-deny and lets the submission proceed. The checkpoint does not enforce until the egress-guard epic installs the real policy pipeline; the seam docs and the venue-adapter docs say so.
The submission charge moves ahead of the guard verdict at the venue registry, so a denied egress spends one unit exactly as an accepted submit does: a module spamming denied egress exhausts its own budget instead of looping free once the guard turns enforcing. A regression test pins the rate limit on a repeated-deny loop.
Fixture files lead with a format version; a reader fails closed on an unknown tag and refuses an empty vector or golden set, which would conform vacuously. Published reference fixtures regenerated.
The registry's watch list grew without bound and was polled O(N) per cadence. Each watch now carries an eviction deadline and the set carries a cap, both operator-configurable via [limits.watch]; expired entries evict unpolled, at the cap a new watch is refused and logged rather than a live watch dropped, and every successful non-terminal poll pushes the deadline a full window out, so only a venue silent for the whole window expires.
The Extension seam becomes a trait contributing a namespace, a capability namespace, a linker hook, an optional HostService, and an optional ProviderKind. HostService is a sync, type-erased service held on a typed per-namespace HostState.services map built once at boot; ProviderKind carries a provider linker hook plus the one cold dyn async_trait install path. The worker boot path is unchanged.
…e known table The capability table and per-module world synthesis move into a new plain nexum-world library carrying only the core nexum:host rows. Per-namespace rows come from the composition root's extensions.toml registry, parsed and passed in by the macro layer, so no host crate carries a downstream name; synthesis rejects a row that shadows a core capability or another registration. WIT packages resolve crate-locally (wit/deps, then wit/) with an ancestor fallback for the transitional monorepo layout.
ComponentsBuilder grows a fourth logs slot, defaulting to the new LogPipelineBuilder (the in-memory pipeline sized from [limits.logs]); with_logs substitutes a custom builder. The Runtime preset names the slot as LogsBuilder and the launcher cars carry the parameter through.
… the privileged field The client face resolves the registry from the store's service map under the videre namespace; no service means every call resolves to unknown-venue. The boot path seeds the registry into the map until the videre extension takes it over, provider stores carry an empty map so the registry never cycles back into a store it owns, and the supervisor field is gone.
A trap in a routed adapter call marks a shared Liveness dead; the dispatch-time sweeps restart the provider after the module backoff policy and quarantine a crash-looper per the poison policy. A dead venue resolves to unavailable, distinct from unknown-venue, and adapter_alive_count reports live routability. The flaky-venue fixture drives the trap-to-recovery and poison paths end to end.
nexum-launch owns the shared CLI, config load, tracing init, and the preset-driven run; nexum-cli becomes the bare Ext=() engine binary over CoreRuntime with no cow edge; the cow composition root moves to a new shepherd binary wiring the cow-api extension as a Runtime preset. The venue-agnostic guard's crate-graph check now covers the launcher and the bare binary, and the cow deployment tooling builds shepherd.
…ough the seam Relocate the venue-adapter and client bindgens, the venue registry, the advisory egress guard, and the venue-adapter provider kind out of nexum-runtime into a new videre-host extension crate, registered whole via videre_host::platform(). The runtime grows two generic seams in exchange: extension-declared subscription kinds with attribute-filtered extension events through the event loop, and config-derived preset extensions. nexum-runtime now carries zero venue, intent, or cow symbols and the venue-agnostic check passes.
Scan the runtime sources for the charter set only (nexum:intent, value-flow, VenueAdapter, synthesize_venue, nexum:adapter, PoolRouter), dropping the loose word-shape sweep that false-flagged the opaque extension-map fixtures; keep the crate-graph and WIT-leaf checks, and state the invariant in the nexum-runtime crate charter. The CI job stays advisory until the physical repo cut.
Drop continue-on-error from the venue-agnostic job, add privileged-field and host-WIT namespace scans to the zero-leak script, and pin the invariant with two integration tests: the echo venue boots and routes a worker's submission purely through the generic extension seam, and the nexum-runtime crate graph names no venue-shaped crate. A missing wasm fixture hard-fails the boot oracle under CI so the gate cannot skip itself.
The derive reaches Vec and ToString through the venue SDK's __private alloc re-export instead of ::std, so a #![no_std] consumer needs no extern crate alloc. cow-venue flips to no_std (tests aside) and a compile-only no-std-probe crate keeps the contract under the workspace gate.
Replace the convert_fault / sdk_fault_into_wit / convert_level shim functions with From impls emitted by the macro (orphan-legal in the per-cdylib expansion), so module glue converts via ? and Into. No behaviour change.
Add HostTransport, an alloy Transport dispatching JSON-RPC packets through ChainHost::request, and host.provider(chain) minting a RootProvider over it, driven by a single-poll block_on. Methods outside the typed ChainMethod read surface (mirrored guest-side) fail as -32601 before reaching the host; node errors carry code, message and decoded revert bytes; host faults surface as typed transport errors. Chain and ChainId newtypes replace bare u64 ids at the SDK edge. The hoisted alloy-provider entry drops its native transport features so the wasm guest build stays transport-free; the engine and load-gen re-add theirs at the call site.
…d venue client Rename nexum-venue-sdk to videre-sdk across the workspace. Add the generic Keeper::sweep assembler (WatchSet to Gates to poll to Retrier to Journal) over a shared Sweep outcome resolving the dangling ConditionalSource::Outcome; the world-neutral primitives stay in nexum-sdk. Thread a VenueId newtype through the client seam, add the owned VenueFault mirror (retry-after-ms preserved) behind ClientError, mark the client-facing enums non-exhaustive, and seal IntentBody to its derive.
#[module] stays L1 in nexum-module-macros; #[venue] and derive(IntentBody) move to videre-macros with the venue-world synthesis. nexum-sdk re-exports module from nexum-module-macros; videre-sdk re-exports venue and IntentBody from videre-macros. No macro behaviour change.
Answer existence, value size and key cardinality without transferring the payload across the wasm boundary. The SDK trait ships default bodies over get/list-keys so backends opt into a cheaper path; the redb backend answers contains and len from the entry in place and count from a bounded range scan.
#[videre_sdk::venue] now takes the impl VenueAdapter block itself: it asserts the manifest kind, keeps the manifest-derived world narrowing, remaps the type interfaces onto the SDK bindings for type identity, and expands to the demoted internal export codegen. export_venue_adapter! is no longer public API; echo-venue and flaky-venue author through the trait, the golden bridges are gone, and the cow body codec is held to the kit's typed vectors.
The keeper author gets the mirror of #[venue]: #[videre_sdk::keeper] on a handler impl emits the per-cdylib module world (requiring the client capability), remaps the videre interfaces onto the SDK's shared shims, completes async handlers on the synchronous guest boundary, and folds ClientError into the wire fault so ? works in handlers. VenueClient<V> replaces the stringly byte-level trait: a Venue marker carries the VenueId and body schema, calls encode through IntentBody, and the byte seam under it (VenueTransport, native AFIT) dispatches statically with zero boxing. HostVenues binds the seam to the module's own videre:venue/client import; the SDK bindgen collapses to one import-only world so linking it never obliges a module to export the adapter face. CowClient becomes the VenueClient<CowVenue> alias, and the echo-keeper example drives echo-venue end to end (quote, submit, status, cancel), proven by a platform e2e test.
Add an Order typestate builder over the 12-field CoW OrderBody with SellToken and BuyToken newtypes, so a keeper cannot swap sides or skip a required field: sell/buy entry fixes the kind, the counter-side limit and expiry are compile-time required states, and the optionals default. Seal the extension traits: Host and HostFault (nexum-sdk), RuntimeTypes and Runtime (nexum-runtime), and VenueTransport (videre-sdk, the pool seam's successor) each gain a doc-hidden sealing marker an implementor opts into, so the surfaces can grow without silent downstream breakage. Apply #[non_exhaustive] uniformly across the public error and label enums, adding wildcard folds at the cross-crate match sites. Emit the mirrored vocabularies from single-source consts in nexum-world: the capability name consts and CORE_IFACES feed both the world-synthesis table and the runtime registry, and the fault-label consts feed the runtime's label projection with the SDK's strum labels pinned to them in test. Fix the operator logs that debug-formatted venue faults: the SDK Fault's rate-limited Display now carries the retry-after hint, the runtime's fault_message keeps it, and the venue registry renders wire errors through message projections instead of the bindgen's debug Display, so rate-limited retry-after-ms survives to the log line. The golden-bridge sweep found no residue.
The venue crate is the orderbook alone. ComposableBody and the structured poll seam (Verdict, LegacyRevertAdapter, IConditionalOrder) move to the new composable-cow keeper crate, the Composable variant drops from the venue body, the goldens shed the composable vectors, and a blocking CI gate keeps composable symbols out of crates/cow-venue.
Pin ConditionalOrderCreated and EthFlow OrderPlacement in a new shepherd:cow/cow-events package of record; keepers resolve topic-0s from the mirrored shepherd-sdk constants, parity-tested against the WIT, the sol! decoders and each module.toml. Mark the legacy cow-api extension surface retiring and gate the generic WIT packages against any shepherd:cow reference.
…Type enum Prune the phantom PriceExceedsMarketPrice row, record the table (not cowprotocol RetryHint) as the classification source of truth in the data header, and pin the reconciliation with parity tests: every row must name a real upstream errorType, and the divergence from retry_hint() must be exactly the ratified set.
The submitted: journal now keys on the deterministic intent-id (the generic sweep's venue-and-body submission key over the encoded CowIntentBody), derived pre-submit without assembling OrderCreation. SubmitOutcome's accepted receipt is fixed as the canonical 56-byte order UID (OrderUid), and the CowIntent schema gains the Signed kind a conditional-order keeper emits. A regression test covers resubmit after restart with a single orderbook POST.
…dleware cow-venue gains the adapter slice: CowAdapter under #[videre_sdk::venue] decodes CowIntentBody, derives the value-flow header, and speaks the orderbook REST API over scoped wasi:http. A signed order posts EIP-1271 and its receipt is the canonical 56-byte UID; an unsigned order posts pre-sign and returns requires-signing carrying the setPreSignature call. An already-held rejection is success with the client-derived UID, and errorType rejections project onto venue-error through the shipped classification table, so the retry hint survives the collapse. The chain-edge order assembly (gpv2_to_order_data, order_data_to_body, order_uid_hex, build_order_creation) moves into the new assembly slice the adapter owns; shepherd-sdk re-exports it for the legacy keeper run. videre-sdk gains TimedFetch, the per-request timeout middleware every adapter request rides. Codec vectors and header goldens are published under tests/vectors with a conformance suite, and the built component bundles into the shepherd distribution via the engine adapters stanza (example and docker configs, justfile, Dockerfile, CI wasm build).
The module flips from the shepherd:cow world onto #[videre_sdk::keeper]: the manifest declares the client capability and body version 1, the keeper run submits through the typed CowClient over the module's own videre:venue/client import, and the direct cow-api import and legacy cow client bridge drop out. Status transitions the registry polls back arrive on a cow intent-status subscription. Behaviour identity is proven at the VenueTransport seam: the dispatch tests script submit outcomes against the mock host and assert the same journal, gate, and retry effects as the legacy bridge, including the throttle hint surviving as an epoch backoff and the appData digest riding the body verbatim. The bundle boot proof moves to the videre platform suite (twap against the installed cow adapter); the cow-api boot-order invariant re-pins on ethflow-watcher. Engine configs that boot twap install the bundled adapter.
The adapter fixes its orderbook at init from its manifest chain, so a Sepolia run wired to the mainnet manifest submits to the wrong orderbook. Add per-chain manifest variants (sepolia, load-mock) and point every twap-wired engine config at the one matching the chain it indexes.
The videre client face gains observe: put an externally-obtained
receipt (an on-chain placement the pool never submitted) under the
registry's status watch. The registry resolves the venue and admits the
watch without touching the adapter; refusal at the watch cap is a typed
unavailable. HostVenues and the typed VenueClient carry the new verb;
the transport trait defaults it to unsupported so existing transports
opt in.
ethflow-watcher flips onto that seam: the module is now a
the legacy cow-api extension. on_chain_logs computes each placement's
orderbook UID and observes it at the cow venue; the registry polls the
adapter and fans intent-status transitions back, and the module
journals observed:{uid} on the first one. Observe-only is preserved: no
call path reaches quote, submit, status, or cancel.
The shepherd-backtest rlib replay drives the same strategy pair over a
recording pool transport and delivers the open transition the registry
would poll, keeping the Observed classification exact-UID strict. The
ethflow boot e2e moves from the cow-api extension suite to the venue
platform suite.
Delete shepherd-cow-host, the shepherd:cow legacy worlds (cow-api, cow-ext, shepherd; cow-events stays as the event-ABI package of record), the shepherd-sdk cow surface, and the shepherd-sdk-test mocks. The shepherd binary composes the core lattice with the videre platform alone. The keeper sweep moves to composable-cow behind the sweep slice; stop-loss migrates onto #[videre_sdk::keeper] and pool submit through the typed CowClient, keyed on the venue-and-body intent-id. twap gates topic-0 on the sol decoder pinned against cow-events; the WIT layering gate moves to cow-venue. ADR-0005/0006 marked superseded.
Subscribe the twap-monitor to the ComposableCoW v2 ConditionalOrderRemoved topic-0 and pin it in the shepherd:cow/cow-events package of record. The created and removed subscription streams merge in arrival order, not chain order, so each watch row carries the create log's (block, log-index) stamp and a removal drops the watch (with its gates) only when it postdates the stamp: a stale removal for a re-registered order is ignored, and an unprovable ordering keeps the watch for the poll path's self-healing drop.
…nature A first-time user's Safe wiring and ComposableCoW create() land in one block, so the orderbook rejects the first submission against its own head and the keeper dropped a valid watch. The classification table gains a drop-on-repeat action, the retry ledger records the refused block and gates one next-block retry, and a denied refusal re-enters the table by its errorType prefix so the grace survives the coarse venue-error collapse. A repeat on a later block still drops.
Doc 05 documents both personas as shipped: the videre-sdk surface, the venue and keeper macros, the typed client, and an author-a-venue walkthrough on the echo pair, with the CoW crates as the production instance. Doc 08 names venue adapters as the domain-extension mechanism, replaces the Layer-3 world-include pattern with the videre:venue contract and the registry route, marks shepherd:cow as the retired legacy read path (cow-events stays as the event-ABI package of record), and squares the WIT layout, server table, and SDK layering with the tree. One inbound sdk.md anchor follows the doc 05 heading.
Fork pull_request runs receive no repository secrets, so the R2 endpoint interpolated an empty account ID and sccache server startup failed before compiling anything. Gate RUSTC_WRAPPER and SCCACHE_BUCKET on the secret: without it the wrapper is disabled and sccache falls back to local disk, so fork PRs build uncached but green.
lgahdl
approved these changes
Jul 20, 2026
lgahdl
left a comment
lgahdl
left a comment
Collaborator
There was a problem hiding this comment.
Worth flagging: PR #356 (still open, mine) fixes the exact same issue (#337) with a near-identical RUSTC_WRAPPER guard, but this PR is actually more complete — it also empties SCCACHE_BUCKET for fork runs, which #356 left hardcoded. Since RUSTC_WRAPPER empty means cargo never invokes sccache at all, the malformed SCCACHE_ENDPOINT (empty account id) is harmless either way, but guarding SCCACHE_BUCKET too is the more defensively correct version. I'll close #356 in favor of this one. Logic checks out — approving.
This was referenced Jul 20, 2026
mfw78
force-pushed
the
feat/m4-source-of-truth
branch
from
July 23, 2026 12:03
d57e9b5 to
80b1ad0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Gate
RUSTC_WRAPPERandSCCACHE_BUCKETonsecrets.R2_ACCOUNT_IDbeing non-empty; both fall back to empty when absent.Why
Fork
pull_requestruns get no repo secrets, soR2_ACCOUNT_IDinterpolated empty and sccache tried to reach a nonexistent R2 host, hard-failing clippy and test before any compilation. EmptyRUSTC_WRAPPERis treated as unset by cargo and emptySCCACHE_BUCKETdrops sccache to its local-disk default, so fork PRs now build uncached but green.Closes #337
Testing
CI on this PR (same-repo head, secrets present) exercises the cached path unchanged; fork-PR uncached path is exercised by the existing red fork PRs once rebased.
AI Assistance
Drafted with Claude Code.