Draft
Conversation
lukevalenta
force-pushed
the
lvalenta/ietf-mtc-197
branch
from
819b314 to
d361d2b
Compare
lukevalenta
force-pushed
the
lvalenta/bootstrap-mtc-rename-196
branch
2 times, most recently
from
3628eb0 to
0f9e622
Compare
lukevalenta
force-pushed
the
lvalenta/ietf-mtc-197
branch
2 times, most recently
from
ad1c3e2 to
725b74e
Compare
Add golden-file tests for the two functions that reconstruct TBSCertificate DER with an extension removed: - extract_scts_from_cert (sct_validator): golden output is the TBS of cloudflare.pem with its SCT extension stripped - build_precert_tbs (static_ct_api): golden output is the TBS of the precertificate in preissuer-chain.pem with CT poison stripped The golden files capture output from the current implementation so that any future refactor of the reconstruction logic (e.g. the upcoming RustCrypto ecosystem upgrade, which rewrites these functions to work around x509-cert 0.3's private fields) is forced to produce byte-for-byte identical output. Regenerate with: UPDATE_GOLDEN=1 cargo test -p <crate> <test_name>
Upgrades all RustCrypto crates to their latest RC versions to unblock ml-dsa 0.1.0-rc.8, which fixes a WASM stack overflow with ML-DSA signatures in Cloudflare Workers. Workspace dependency changes: - der 0.7.10 (patched fork) → 0.8.0 (upstream; Tag::RelativeOid is now native, removing the need for the fork) - const-oid 0.9.6 → 0.10 - spki 0.7 → 0.8 - pkcs8 (new) 0.11.0-rc.11 - signature 2.2.0 → 3.0.0-rc.10 - sha2 0.10 → 0.11 - rand 0.8.5 → 0.10.0 (uses rand_core 0.10, unifying with ed25519-dalek) - rand_core 0.6.4 → 0.10.0 - getrandom 0.2 → 0.4 (unifies with RustCrypto RC crates; 0.3 remains only in build-script deps via ahash/jsonschema, never compiled for WASM) - ed25519-dalek 2.1.1 → 3.0.0-pre.6 (requires rand_core 0.10) - p256 0.13 → 0.14.0-rc.8 - p384 (new) 0.14.0-rc.4 - p521 (new) 0.14.0-rc.8 - rsa 0.9 → 0.10.0-rc.17 - x509-cert 0.2.5 → 0.3.0-rc.4 Remove x509-verify and inline signature verification into x509_util, supporting P-256, P-384, P-521, and RSA PKCS#1 v1.5 (SHA-256/384/512) for TLS certificate chain validation. Ed25519 is intentionally excluded: is_link_valid validates TLS/PKI chains only; Ed25519 is not permitted by the CA/Browser Forum Baseline Requirements, does not appear in any root pool, and was never reachable. DSA, SHA-1, k256, md2, and md5 are also intentionally not supported. Adapt to x509-cert 0.3 API changes: - All TbsCertificate and Certificate fields are now private; replaced direct field access with getter methods (.tbs_certificate(), .subject(), .validity(), .extensions(), etc.) - .get::<T>() renamed to .get_extension::<T>() - Validity, Name, RelativeDistinguishedName no longer constructible via struct literal syntax; use Validity::new(), TryFrom, etc. - CertificateBuilder::new() signature changed; Profile enum replaced by BuilderProfile trait; build_with_rng() now takes signer separately - AsExtension renamed to ToExtension; Criticality is now a separate trait - build_precert_tbs() and tbs_without_sct() re-encode TBS field-by-field using the public getter API since struct fields are private Adapt to rand 0.10 API changes: - OsRng renamed to SysRng (fallible; use rand::rng()/ThreadRng for CryptoRng-requiring APIs) - SmallRng no longer feature-gated; drop features = ["small_rng"] - RngCore no longer re-exported from rand; RngExt trait added for random_range/random/fill extension methods - rand_core::RngCore/CryptoRng ZeroRng doc example updated to TryRng/ TryCryptoRng
lukevalenta
force-pushed
the
lvalenta/ietf-mtc-197
branch
from
51833a8 to
74113e4
Compare
…d_roots The Workers runtime cancels any request that awaits a promise (OnceCell future) created by a different request context. Previously, a concurrent add-chain request arriving while another request was initializing ROOTS via get_or_try_init would be canceled with a 500. Fix: load_roots now checks ROOTS.get() first (fast path), then builds the pool itself if not yet initialized, then calls ROOTS.set(). If another concurrent request races and sets first, the losing request discards its result and returns the value already in the cell. All concurrent requests do the work independently rather than waiting on each other. Applied to both ct_worker and bootstrap_mtc_worker.
Adds a Metadata associated type to the LogEntry trait in tlog_tiles, allowing each tlog application to define its own sequence metadata type rather than sharing the fixed (LeafIndex, UnixTimestamp) tuple. tlog_tiles: - Adds LogEntry::Metadata: Serialize + DeserializeOwned + Copy + Default - Adds LogEntry::make_metadata(leaf_index, timestamp, old_tree_size, new_tree_size) to construct the metadata from raw sequencer values - TlogTilesLogEntry::Metadata = SequenceMetadata (unchanged behavior) - SequenceMetadata type alias is preserved for backward compatibility static_ct_api, bootstrap_mtc_api: - Metadata = SequenceMetadata; make_metadata ignores tree sizes generic_log_worker: - All cache infrastructure (DedupCache, MemoryCache, CacheRead, CacheWrite) is now generic over M: the metadata type - serialize_entries/deserialize_entries switched from fixed 32-byte binary format to serde_json (the binary format only worked for the fixed-size (u64, u64) tuple) - PoolState<P, M>, AddLeafResult<M>, add_leaf_to_pool, sequence_entries, GenericSequencer, GenericBatcher all generified over M - sequence_entries calls L::make_metadata(n, timestamp, old_size, new_size) - Assert n == new_size after the sequencing loop
Initial scaffolding for the IETF MTC implementation (draft-ietf-plants-merkle-tree-certs). Copied directly from bootstrap_mtc_api and bootstrap_mtc_worker as a starting point; subsequent commits will remove bootstrap-specific functionality and implement draft-02 behaviour.
…_worker Removes all bootstrap-experiment-specific code and replaces it with the IETF MTC draft-02 equivalent. ietf_mtc_api: - Removes validate_chain, validate_correspondence, tbs_cert_to_log_entry, filter_extensions and all X.509 bootstrap chain validation logic - Removes BootstrapMtcLogEntry/PendingLogEntry auxiliary tile; renames to IetfMtcLogEntry/IetfMtcPendingLogEntry with AUX_TILE_PATH = None - Removes x509_util and rand dependencies - Adds DraftVersion enum (Draft02, default) for compatibility with multiple versions - Replaces the add-entry request format with a PKCS#10 CSR in a 'csr' field, base64url-encoded without padding, matching the ACME finalize endpoint format (RFC 8555 §7.4); the server extracts subject, SPKI hash, and SubjectAltName extensions from the CSR - Document that ACME order notBefore/notAfter values are not currently supported (similar to Let's Encrypt Boulder) - Adds build_pending_entry() and extract_san_from_csr() helper - Adds base64 and serde_json (dev) dependencies ietf_mtc_worker: - Removes ccadb_roots_cron.rs and ct_logs_cron.rs - Removes load_roots(), ROOTS OnceCell, CCADB KV binding, and the dev-bootstrap-roots feature flag and dev-bootstrap-roots.pem - Removes SCT validation (sct_validator dep, enable_sct_validation config) - Removes csv dep and upload_issuers call - Removes the scheduled cron trigger from wrangler.jsonc - Rewrites add-entry to accept a CSR-based request; validity is set server-side as [now, now + max_certificate_lifetime_secs] - Replaces build_validity to derive not_before from now_millis() - Adds version field to config schema (enum: 'draft02') - Renames the production environment from 'bootstrap-mtca' to 'ietf-mtc-ca' and renames config.bootstrap-mtca.json to config.ietf-mtc-ca.json
…rmat Two wire format changes relative to the davidben-09 encoding used by bootstrap_mtc_api: 1. Remove the outer ASN.1 SEQUENCE wrapper from TBSCertificateLogEntry (dropped in davidben-10): the fields are now concatenated as raw DER without a SEQUENCE tag+length prefix. MerkleTreeCertEntry::encode/decode are updated accordingly; TbsCertificateLogEntry no longer derives Sequence and instead provides encode_fields()/decode_fields(). 2. Add subjectPublicKeyInfoAlgorithm field (new in plants-02): the AlgorithmIdentifier extracted from the submitted SPKI is stored immediately before the existing subjectPublicKeyInfoHash field. build_pending_entry() extracts it from the CSR's public_key field. Also renames the production deployment environment and config file from 'ietf-mtc-ca' to 'draft02' to reflect the versioned deployment model.
Adds IetfMtcClient to client.rs, make_ietf_mtc_csr() to fixtures.rs, a new tests/ietf_mtc_api.rs test file, a CI job, and AGENTS.md docs. Tests cover: - metadata_returns_valid_fields: GET /metadata shape and Ed25519 key - unknown_log_returns_400: 400 for unknown log - add_entry_returns_valid_response: CSR-based add-entry round-trip - add_entry_with_invalid_csr_returns_400: garbage bytes → 400 - add_entry_appears_in_checkpoint: leaf_index covered after sequencing - get_certificate_returns_valid_cert: signatureless DER cert after landmark Key differences from bootstrap_mtc tests: - No get-roots test (ietf_mtc_worker has no roots endpoint) - ensure_initialized() goes straight to add-entry (no CCADB OnceCell race) - Fixture is a PKCS#10 CSR (IetfMtcCsr) rather than an X.509 chain
…EADMEs Renames 'signatureless' to 'landmark-relative' (the term used in the IETF draft) in the IETF MTC crates only. The bootstrap crates retain the old 'signatureless' terminology since they are frozen on the older draft. Changes: - ietf_mtc_api: serialize_signatureless_cert → serialize_landmark_relative_cert; update all doc comments - ietf_mtc_worker: update all comments referencing signatureless certificates - integration_tests/tests/ietf_mtc_api.rs: update test doc comment Also updates all four MTC crate READMEs: - bootstrap_mtc_worker: clarifies this implements the older bootstrap experiment (~davidben-09), not the current IETF draft; uses 'signatureless' with a note that the IETF draft renamed it to 'landmark-relative' - bootstrap_mtc_api: new README; uses 'signatureless' with the same note - ietf_mtc_worker: replaces stale bootstrap copy with accurate description of the plants-02 implementation; uses 'landmark-relative'; lists known limitations (standalone certs, ML-DSA, subtree signing oracle) - ietf_mtc_api: new README describing the plants-02 wire format components
Replaces hardcoded Ed25519 signing with a flexible multi-algorithm design supporting Ed25519, ML-DSA-44, ML-DSA-65, and ML-DSA-87. ietf_mtc_api: - Adds MtcSigningKey and MtcVerifyingKey enums (both Clone) covering all four algorithms. - MtcSigningKey::try_sign() returns Result<Vec<u8>, signature::Error>; current variants are infallible but the Result allows for future fallible algorithms (e.g. randomized schemes requiring entropy). - MtcVerifyingKey::signature_type_bytes() returns &'static [u8]: Ed25519 uses the allocated single byte 0x01; ML-DSA variants use 0xff followed by the algorithm OID in dotted-decimal ASCII, following the c2sp.org/signed-note recommendation for unassigned types. TODO: replace with allocated bytes once c2sp.org/signed-note assigns them. - MtcVerifyingKey::to_public_key_der() returns the DER-encoded SubjectPublicKeyInfo, including the AlgorithmIdentifier, so clients can determine the signing algorithm without out-of-band information. - MtcNoteVerifier takes signature_type_bytes: &[u8] and feeds 0x0a || signature_type_bytes into the key ID SHA-256 hash. - MtcCosigner::new_checkpoint accepts MtcSigningKey + MtcVerifyingKey. - sign_subtree returns Result<Vec<u8>, signature::Error>. ietf_mtc_worker: - Algorithm is inferred from the PKCS#8 AlgorithmIdentifier OID embedded in the key file. - parse_key_pair() inspects PrivateKeyInfo.algorithm.oid and dispatches to the correct decoder (Ed25519 / ML-DSA-44/65/87). - CachedKeys is (MtcSigningKey, MtcVerifyingKey); both implement Clone so the keys can be cached and cloned directly on each request. - metadata endpoint: cosigner_public_key is now the DER-encoded SubjectPublicKeyInfo rather than raw key bytes. - .dev.vars updated with fresh ML-DSA-44 PKCS#8 keys; unused WITNESS_KEY entries removed. integration_tests: - metadata_returns_valid_fields updated to verify the cosigner_public_key as a valid ML-DSA-44 SubjectPublicKeyInfo via pkcs8::DecodePublicKey.
The /add-entry endpoint now returns a standalone MTC certificate
(draft-ietf-plants-merkle-tree-certs §6.2) directly in the response,
replacing the per-field (leaf_index, timestamp, not_before, not_after)
response. All relevant fields are encoded in the certificate itself:
the leaf_index is the serial number, validity is in the TBSCertificate,
and the inclusion proof and cosignature are in the signatureValue.
Changes:
generic_log_worker:
- CheckpointCallbacker now receives old_tree_size and new_tree_size in
addition to old_time and new_time. All existing callsites updated.
ietf_mtc_api:
- AddEntryResponse simplified to a single 'certificate' field.
- Replaces serialize_landmark_relative_cert with the unified
serialize_mtc_cert (empty cosignatures = landmark-relative §6.3,
non-empty = standalone §6.2).
- Adds SignedSubtree: JSON structure cached in R2 per signed subtree,
keyed as subtree-sig/{lo:020}-{hi:020}.
- Adds ParsedMtcProof with from_bytes() and verify_cosignature() for
decoding and verifying the signatureValue in MTC certificates.
- Adds from_ber_bytes() and derives Debug/PartialEq/Eq on RelativeOid.
- Uses signature::Error as unified error type for try_sign/sign_subtree.
- Gates ML-DSA signing key variants behind 'ml-dsa' feature flag
(disabled by default due to WASM stack overflow in Workers runtime).
ietf_mtc_worker:
- checkpoint_callback signs and caches batch subtrees on every checkpoint
and landmark subtrees on every landmark epoch, using the true subtree
root hash (from prove_subtree_consistency) rather than the full
checkpoint hash.
- Key pair loading uses get()-first pattern to avoid cross-request
OnceLock deadlocks while still caching on the fast path.
- build_standalone_cert retries briefly (6x 250ms) waiting for the
R2 write from the async checkpoint_callback to complete.
- clean_subtree_sigs() deletes subtree-sig/* keys where hi <= oldest_landmark.
integration_tests:
- Full signature + inclusion proof verification for both cert types:
standalone (verify_standalone_cert) and landmark-relative
(verify_landmark_relative_cert).
- assert_valid_mtc_cert helper verifies id-alg-mtcproof OID, non-empty
signatureValue, and non-empty subject.
- leaf_index extracted from certificate serial number.
- IetfMtcClient.get_signed_subtree() fetches cached subtree signatures.
…44 default
Adapt ietf_mtc_api and ietf_mtc_worker to the RustCrypto ecosystem upgrade:
- ml-dsa 0.0.4 → 0.1.0-rc.8 (removes rand_core feature, which no longer exists)
- ietf_mtc_worker: drop features = ["small_rng"] (removed in rand 0.10)
- TagNumber::N0/N1/N2/N3 → TagNumber(0/1/2/3) (der 0.8 API change)
- issuerUniqueID/subjectUniqueID: use ContextSpecificRef for IMPLICIT encoding
(der 0.7 allowed owned ContextSpecific<BitString> with clone; der 0.8
requires ContextSpecificRef for borrowed values)
- Certificate/TbsCertificate struct literals → field-by-field DER construction
via encode_tbs_certificate_der helper (x509-cert 0.3 made all fields private)
- reader.finish(value) → reader.finish()?; Ok(value) (der 0.8 API change)
- reader.peek_tag() → Tag::peek(&reader) (deprecated in der 0.8)
- Name vs RdnSequence: build_pending_entry now takes &RdnSequence and
converts via DER round-trip (x509-cert 0.3 separates the two types)
- PrivateKeyInfo::try_from → PrivateKeyInfoRef::try_from (type disambiguation)
- OsRng → rand::rng() in tests (OsRng renamed to SysRng in rand 0.10;
ThreadRng satisfies CryptoRng from rand_core 0.10)
- RelativeDistinguishedName(SetOfVec::from_iter(...)) → TryFrom
(x509-cert 0.3 made the inner field private)
- Validity { ... } → Validity::new(...) (x509-cert 0.3 made fields private)
ml-dsa 0.1.0-rc.8 restructures the key types: - SigningKey<P> (seed + ExpandedSigningKey) is no longer Clone — it holds the seed and is the output of key_gen() / from_pkcs8_pem() - ExpandedSigningKey<P> is Clone and implements Signer; obtain it via signing_key() on SigningKey<P> - verifying_key() is available via the signature::Keypair trait on both SigningKey<P> and ExpandedSigningKey<P> - ml_dsa::KeyPair type alias removed; key_gen() now returns SigningKey<P> directly (via KeyGen::KeyPair = SigningKey<P>) MtcSigningKey ML-DSA variants now store ExpandedSigningKey<P> (Clone) instead of SigningKey<P> (non-Clone). parse_key_pair uses SigningKey::<P>::from_pkcs8_pem and extracts the expanded key via signing_key().clone().
- fixtures.rs: SigningKey::random(&mut OsRng) → generate_from_rng(&mut rand::rng()); SubjectPublicKeyInfoOwned::from_key(*key) → from_key(key); RequestBuilder::new(subject, &signer) → new(subject); build::<Sig>() → build::<_, Sig>(&signer) (x509-cert 0.3 RequestBuilder API changes) - ietf_mtc_api.rs: all cert.field / cert.tbs_certificate.field → accessor methods (cert.signature_algorithm(), cert.tbs_certificate(), tbs.serial_number(), tbs.subject_public_key_info(), etc.); cert.signature.raw_bytes() → cert.signature().as_bytes().unwrap_or(); tbs.subject.0.is_empty() → tbs.subject().as_ref().is_empty(); TbsCertificateLogEntry construction updated to use getter return values (version(), issuer().clone(), *validity(), subject().clone(), etc.)
ml-dsa 0.1.0-rc.8 is included unconditionally. Note: the WASM stack overflow (RustCrypto/signatures#1024) is not yet fully resolved upstream — ML-DSA signing in Cloudflare Workers will fail until the fix lands. The gate is removed now so that ML-DSA is available on native targets and can be tested incrementally as upstream progresses. Regenerate .dev.vars.ml-dsa keys for the new PKCS#8 format introduced in ml-dsa 0.1.0-rc.x: seed is now stored as [0] IMPLICIT OCTET STRING instead of a plain OCTET STRING as in 0.0.4. - ietf_mtc_api: drop [features] section, make ml-dsa a required dep - ietf_mtc_worker: add ml-dsa as direct dep, remove unexpected_cfgs lint - cosigner.rs, lib.rs: remove all #[cfg(feature = "ml-dsa")] guards
MtcNoteVerifier key ID was computed using a custom b"mtc-checkpoint/v1" constant rather than the actual public key bytes. Fix to use the standard signed-note convention: SHA-256(name || 0x0A || sig_type_bytes || raw_pubkey_bytes)[:4] via compute_key_id. This makes the key ID consistent with what a client would compute from a published vkey. Add MtcVerifyingKey::to_raw_bytes() to extract the algorithm-agnostic public key bytes for the key ID computation. Update integration test fetch_verifying_key to decode ML-DSA-44 SPKI (via SubjectPublicKeyInfoRef + VerifyingKey::try_from) in addition to Ed25519, so the test works with either key type in .dev.vars.
…trap) Adopt LogEntry::Metadata associated type introduced by the Metadata infrastructure commit: - Add IetfSequenceMetadata struct - Implement LogEntry::Metadata = IetfSequenceMetadata and make_metadata - Add impl_json_cache_serialize!(IetfSequenceMetadata) - Add generic_log_worker dep to ietf_mtc_api for the macro - GenericBatcher<IetfSequenceMetadata> in batcher_do.rs - ietf_mtc_worker: deserialize IetfSequenceMetadata from sequencer response
lukevalenta
force-pushed
the
lvalenta/ietf-mtc-197
branch
from
74113e4 to
48b2fa8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.