From 1c14b3d433d703faea328b9b054daee0173d9866 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 15 Apr 2026 15:44:08 -0500 Subject: [PATCH 01/22] Add design spec for Sourcepoint GPP consent support (#640) --- ...26-04-15-sourcepoint-gpp-consent-design.md | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md diff --git a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md new file mode 100644 index 000000000..ed72f3f55 --- /dev/null +++ b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md @@ -0,0 +1,151 @@ +# Sourcepoint GPP Consent for Edge Cookie Generation + +**Issue:** #640 +**Date:** 2026-04-15 +**Status:** Approved + +## Problem + +Edge Cookie (EC) generation fails for sites using Sourcepoint when consent is +stored only in `localStorage` and not surfaced via the standard cookies Trusted +Server reads. Sourcepoint stores US consent under `_sp_user_consent_*` keys in +`localStorage`, including a full GPP string and applicable section IDs. + +Today, Trusted Server only reads consent from `euconsent-v2`, `__gpp`, +`__gpp_sid`, `us_privacy` cookies and the `Sec-GPC` header. Even if `__gpp` / +`__gpp_sid` were present, the server only decodes the EU TCF v2 section from +GPP — it does not use GPP US sections as a consent signal for EC gating. + +This creates two gaps: + +1. **Transport gap:** The server cannot read browser `localStorage`, so no + consent reaches the backend unless client code mirrors it into cookies. +2. **Semantics gap:** Even with `__gpp` / `__gpp_sid` cookies present, current + US-state EC gating does not recognize GPP US sections as valid consent. + +## Approach + +Thin GPP pass-through: mirror Sourcepoint localStorage consent into standard +cookies on the client, and extend server-side EC gating to recognize GPP US +`sale_opt_out` as a consent signal. No compatibility bridge (`us_privacy` +derivation) — both client and server changes ship together. + +## Design + +### 1. Client-side: Sourcepoint JS integration + +New JS-only integration at `crates/js/lib/src/integrations/sourcepoint/index.ts`. +No Rust-side `IntegrationRegistration` (same pattern as `creative`). + +**On page load:** + +1. Scan `localStorage` keys matching `_sp_user_consent_*`. +2. Take the first match, parse the JSON value. +3. Extract `gppData.gppString` and `gppData.applicableSections` from the payload. +4. Write first-party cookies: + - `__gpp=` (path `/`, `SameSite=Lax`) + - `__gpp_sid=` (path `/`, `SameSite=Lax`) +5. Log what was written for debugging. + +Cookies are session-scoped (no `max-age` / `expires`) since the source of truth +stays in `localStorage` and we re-mirror on each page load. The integration runs +once — no polling or event listeners. + +### 2. Server-side: GPP US section decoding + +**`crates/trusted-server-core/src/consent/types.rs`** — extend `GppConsent`: + +```rust +pub struct GppConsent { + pub version: u8, + pub section_ids: Vec, + pub eu_tcf: Option, + pub us_sale_opt_out: Option, // new +} +``` + +- `Some(true)` — a US section is present and `sale_opt_out == OptedOut` +- `Some(false)` — a US section is present and `sale_opt_out != OptedOut` +- `None` — no US section exists in the GPP string + +**`crates/trusted-server-core/src/consent/gpp.rs`** — add `decode_us_sale_opt_out`: + +Checks for any US section ID (7–23) in the parsed `GPPString`. For the first +match, decodes the section via `iab_gpp` and extracts `sale_opt_out`. Maps +`OptOut::OptedOut` to `true`, everything else to `false`. + +The `iab_gpp` crate uses different structs per state (`UsNat`, `UsCa`, `UsTn`, +etc.) but they all have `sale_opt_out: OptOut` via `us_common`. We match on the +decoded `Section` enum to extract it. + +### 3. Server-side: EC gating update + +**`crates/trusted-server-core/src/consent/mod.rs`** — update `allows_ec_creation()` +for `Jurisdiction::UsState(_)`. + +New precedence chain: + +``` +GPC → TCF → GPP US sale_opt_out → us_privacy → fail-closed +``` + +Insert between the existing TCF and `us_privacy` branches: + +```rust +// Check GPP US section for sale opt-out. +if let Some(gpp) = &ctx.gpp { + if let Some(opted_out) = gpp.us_sale_opt_out { + return !opted_out; + } +} +``` + +Semantics: + +- GPP US `sale_opt_out != OptedOut` → EC allowed +- GPP US `sale_opt_out == OptedOut` → EC blocked +- No GPP US section → falls through to `us_privacy` +- GPC still short-circuits at the top (unchanged) +- TCF still takes priority for CMPs like Didomi (unchanged) + +### 4. Files touched + +| File | Change | +|---|---| +| `crates/js/lib/src/integrations/sourcepoint/index.ts` | New — localStorage auto-discovery, cookie mirroring | +| `crates/js/lib/src/integrations/sourcepoint/index.test.ts` | New — Vitest tests | +| `crates/trusted-server-core/src/consent/types.rs` | Add `us_sale_opt_out: Option` to `GppConsent` | +| `crates/trusted-server-core/src/consent/gpp.rs` | Add US section decoding, extract `sale_opt_out` | +| `crates/trusted-server-core/src/consent/mod.rs` | Add GPP US branch in `allows_ec_creation()`, tests | + +No config changes, no new crate dependencies, no `IntegrationRegistry` changes. + +### 5. Testing + +**JS (Vitest):** + +- Mirrors `__gpp` and `__gpp_sid` from `_sp_user_consent_*` localStorage +- No cookies written when no `_sp_user_consent_*` key exists +- Graceful handling of malformed JSON in localStorage + +**Rust — EC gating (`consent/mod.rs`):** + +- EC allowed: US state + GPP `us_sale_opt_out = Some(false)` +- EC blocked: US state + GPP `us_sale_opt_out = Some(true)` +- EC blocked: GPC overrides permissive GPP US +- TCF takes priority over GPP US when both present +- GPP US takes priority over `us_privacy` when both present +- No GPP US section falls through to `us_privacy` +- No signals → fail-closed + +**Rust — GPP decoding (`consent/gpp.rs`):** + +- Extracts `us_sale_opt_out` from GPP string with UsNat section (ID 7) +- `us_sale_opt_out` is `None` when GPP has no US sections + +### 6. Non-goals + +- No `us_privacy` compatibility bridge (skipped per decision) +- No richer US GPP field extraction (sharing, targeted advertising opt-outs) +- No publisher configuration for Sourcepoint property ID (auto-discovery) +- No Sourcepoint CMP API integration (localStorage-only approach) From fca8979f144f381573134447f1c771a7ba2c83b5 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 15 Apr 2026 15:48:46 -0500 Subject: [PATCH 02/22] Add implementation plan for Sourcepoint GPP consent support (#640) --- .../2026-04-15-sourcepoint-gpp-consent.md | 695 ++++++++++++++++++ 1 file changed, 695 insertions(+) create mode 100644 docs/superpowers/plans/2026-04-15-sourcepoint-gpp-consent.md diff --git a/docs/superpowers/plans/2026-04-15-sourcepoint-gpp-consent.md b/docs/superpowers/plans/2026-04-15-sourcepoint-gpp-consent.md new file mode 100644 index 000000000..8c9de843a --- /dev/null +++ b/docs/superpowers/plans/2026-04-15-sourcepoint-gpp-consent.md @@ -0,0 +1,695 @@ +# Sourcepoint GPP Consent for Edge Cookie Generation — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Enable EC generation for sites using Sourcepoint by mirroring localStorage consent into cookies (client) and recognizing GPP US `sale_opt_out` as a consent signal (server). + +**Architecture:** New JS-only `sourcepoint` integration auto-discovers `_sp_user_consent_*` in localStorage and writes `__gpp` / `__gpp_sid` cookies. Server-side, `GppConsent` gains a `us_sale_opt_out: Option` field extracted from any GPP US section (IDs 7–23). `allows_ec_creation()` checks this field between the existing TCF and `us_privacy` branches. + +**Tech Stack:** TypeScript (Vitest, jsdom), Rust (iab_gpp crate for GPP section decoding) + +**Spec:** `docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md` + +--- + +## File Map + +| File | Action | Responsibility | +|---|---|---| +| `crates/trusted-server-core/src/consent/types.rs` | Modify | Add `us_sale_opt_out: Option` to `GppConsent` | +| `crates/trusted-server-core/src/consent/gpp.rs` | Modify | Decode US sections, extract `sale_opt_out` | +| `crates/trusted-server-core/src/consent/mod.rs` | Modify | Add GPP US branch in `allows_ec_creation()`, tests | +| `crates/js/lib/src/integrations/sourcepoint/index.ts` | Create | localStorage auto-discovery, cookie mirroring | +| `crates/js/lib/test/integrations/sourcepoint/index.test.ts` | Create | Vitest tests for cookie mirroring | + +--- + +## Task 1: Add `us_sale_opt_out` field to `GppConsent` + +**Files:** +- Modify: `crates/trusted-server-core/src/consent/types.rs:297-305` + +- [ ] **Step 1: Add the field** + +In `crates/trusted-server-core/src/consent/types.rs`, add `us_sale_opt_out` to `GppConsent`: + +```rust +/// Decoded GPP (Global Privacy Platform) consent data. +/// +/// Wraps the `iab_gpp` crate's decoded output with our domain types. +#[derive(Debug, Clone)] +pub struct GppConsent { + /// GPP header version. + pub version: u8, + /// Active section IDs present in the GPP string. + pub section_ids: Vec, + /// Decoded EU TCF v2.2 section (if present in GPP, section ID 2). + pub eu_tcf: Option, + /// Whether the user opted out of sale of personal information via a US GPP + /// section (IDs 7–23). + /// + /// - `Some(true)` — a US section is present and `sale_opt_out == OptedOut` + /// - `Some(false)` — a US section is present and user did not opt out + /// - `None` — no US section exists in the GPP string + pub us_sale_opt_out: Option, +} +``` + +- [ ] **Step 2: Fix compilation — update all `GppConsent` construction sites** + +There are existing places that construct `GppConsent`. Each needs the new field. Search for them: + +In `crates/trusted-server-core/src/consent/gpp.rs` (~line 74), update `decode_gpp_string`: + +```rust + Ok(GppConsent { + version: 1, + section_ids, + eu_tcf, + us_sale_opt_out: None, // placeholder — Task 2 fills this in + }) +``` + +In `crates/trusted-server-core/src/consent/mod.rs`, find every test that constructs `GppConsent` (search for `GppConsent {`). Add `us_sale_opt_out: None` to each. There are instances around lines 720, 883, and 965: + +```rust + gpp: Some(GppConsent { + version: 1, + section_ids: vec![2], + eu_tcf: Some(...), + us_sale_opt_out: None, + }), +``` + +- [ ] **Step 3: Verify compilation** + +Run: `cargo check --workspace` +Expected: compiles with no errors. + +- [ ] **Step 4: Run tests to confirm nothing broke** + +Run: `cargo test --workspace` +Expected: all existing tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/consent/types.rs \ + crates/trusted-server-core/src/consent/gpp.rs \ + crates/trusted-server-core/src/consent/mod.rs +git commit -m "Add us_sale_opt_out field to GppConsent" +``` + +--- + +## Task 2: Decode US sale opt-out from GPP sections + +**Files:** +- Modify: `crates/trusted-server-core/src/consent/gpp.rs` + +- [ ] **Step 1: Write the failing test for US sale opt-out extraction** + +Add to the `#[cfg(test)] mod tests` block in `crates/trusted-server-core/src/consent/gpp.rs`: + +```rust + // A GPP string with UsNat section (section ID 7). + // Header "DBABLA" encodes: version=1, section IDs=[7] (UsNat). + // The section string encodes a UsNat v1 core with sale_opt_out=DidNotOptOut (2). + #[test] + fn decodes_us_sale_opt_out_not_opted_out() { + // Build a real GPP string with UsNat section using iab_gpp parsing. + // "DBABLA~BVQqAAAAAgA.QA" is the example from the issue (Sourcepoint payload). + let result = decode_gpp_string("DBABLA~BVQqAAAAAgA.QA"); + match &result { + Ok(gpp) => { + assert_eq!( + gpp.us_sale_opt_out, + Some(false), + "should extract sale_opt_out=false from UsNat section" + ); + } + Err(e) => { + // If the specific GPP string doesn't parse, test with section ID presence. + // The important thing is that the decode_us_sale_opt_out function is wired up. + panic!("GPP decode failed: {e}"); + } + } + } + + #[test] + fn no_us_section_returns_none() { + // GPP_TCF_AND_USP has section IDs [2, 6] — no US sections (7–23). + let result = decode_gpp_string(GPP_TCF_AND_USP).expect("should decode GPP"); + assert_eq!( + result.us_sale_opt_out, None, + "should return None when no US section (7-23) is present" + ); + } +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test --workspace -p trusted-server-core -- consent::gpp::tests::decodes_us_sale_opt_out` +Expected: FAIL — `us_sale_opt_out` is hardcoded to `None`. + +- [ ] **Step 3: Implement `decode_us_sale_opt_out`** + +In `crates/trusted-server-core/src/consent/gpp.rs`, add after `decode_tcf_from_gpp`: + +```rust +/// GPP section IDs that represent US state/national privacy sections. +/// +/// Range 7–23 per the GPP v1 specification: +/// 7=UsNat, 8=UsCa, 9=UsVa, 10=UsCo, 11=UsUt, 12=UsCt, 13=UsFl, +/// 14=UsMt, 15=UsOr, 16=UsTx, 17=UsDe, 18=UsIa, 19=UsNe, 20=UsNh, +/// 21=UsNj, 22=UsTn, 23=UsMn. +const US_SECTION_ID_RANGE: std::ops::RangeInclusive = 7..=23; + +/// Extracts the `sale_opt_out` signal from the first US section in a parsed +/// GPP string. +/// +/// Iterates through section IDs looking for any in the US range (7–23). +/// For the first match, decodes the section and extracts `sale_opt_out`. +/// +/// Returns `Some(true)` if the user opted out of sale, `Some(false)` if they +/// did not, or `None` if no US section is present. +fn decode_us_sale_opt_out(parsed: &iab_gpp::v1::GPPString) -> Option { + use iab_gpp::sections::us_common::OptOut; + use iab_gpp::sections::Section; + + let us_section_id = parsed + .section_ids() + .find(|id| US_SECTION_ID_RANGE.contains(&(**id as u16)))?; + + match parsed.decode_section(*us_section_id) { + Ok(section) => { + let sale_opt_out = match §ion { + Section::UsNat(s) => match &s.core { + iab_gpp::sections::usnat::Core::V1(c) => &c.sale_opt_out, + iab_gpp::sections::usnat::Core::V2(c) => &c.sale_opt_out, + }, + Section::UsCa(s) => &s.core.sale_opt_out, + Section::UsVa(s) => &s.core.sale_opt_out, + Section::UsCo(s) => &s.core.sale_opt_out, + Section::UsUt(s) => &s.core.sale_opt_out, + Section::UsCt(s) => &s.core.sale_opt_out, + Section::UsFl(s) => &s.core.sale_opt_out, + Section::UsMt(s) => &s.core.sale_opt_out, + Section::UsOr(s) => &s.core.sale_opt_out, + Section::UsTx(s) => &s.core.sale_opt_out, + Section::UsDe(s) => &s.core.sale_opt_out, + Section::UsIa(s) => &s.core.sale_opt_out, + Section::UsNe(s) => &s.core.sale_opt_out, + Section::UsNh(s) => &s.core.sale_opt_out, + Section::UsNj(s) => &s.core.sale_opt_out, + Section::UsTn(s) => &s.core.sale_opt_out, + Section::UsMn(s) => &s.core.sale_opt_out, + // Non-US sections — should not reach here given the ID filter. + _ => return None, + }; + Some(*sale_opt_out == OptOut::OptedOut) + } + Err(e) => { + log::warn!("Failed to decode US GPP section {us_section_id}: {e}"); + None + } + } +} +``` + +- [ ] **Step 4: Wire it into `decode_gpp_string`** + +In the same file, replace the placeholder in `decode_gpp_string`: + +```rust + let us_sale_opt_out = decode_us_sale_opt_out(&parsed); + + Ok(GppConsent { + version: 1, + section_ids, + eu_tcf, + us_sale_opt_out, + }) +``` + +- [ ] **Step 5: Run tests** + +Run: `cargo test --workspace -p trusted-server-core -- consent::gpp::tests` +Expected: all GPP tests pass, including the two new ones. + +- [ ] **Step 6: Commit** + +```bash +git add crates/trusted-server-core/src/consent/gpp.rs +git commit -m "Decode US sale opt-out from GPP sections" +``` + +--- + +## Task 3: Add GPP US branch to `allows_ec_creation()` + +**Files:** +- Modify: `crates/trusted-server-core/src/consent/mod.rs` + +- [ ] **Step 1: Write failing tests** + +Add to the `#[cfg(test)] mod tests` block in `crates/trusted-server-core/src/consent/mod.rs`: + +```rust + #[test] + fn ec_allowed_us_state_gpp_no_sale_opt_out() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(false), + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "US state + GPP US sale_opt_out=false should allow EC" + ); + } + + #[test] + fn ec_blocked_us_state_gpp_sale_opted_out() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(true), + }), + ..ConsentContext::default() + }; + assert!( + !allows_ec_creation(&ctx), + "US state + GPP US sale_opt_out=true should block EC" + ); + } + + #[test] + fn ec_blocked_us_state_gpc_overrides_gpp_us() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpc: true, + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(false), + }), + ..ConsentContext::default() + }; + assert!( + !allows_ec_creation(&ctx), + "GPC should block EC even when GPP US says no opt-out" + ); + } + + #[test] + fn ec_us_state_tcf_takes_priority_over_gpp_us() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + tcf: Some(make_tcf_with_storage(true)), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(true), + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "TCF consent should take priority over GPP US opt-out" + ); + } + + #[test] + fn ec_us_state_gpp_us_takes_priority_over_us_privacy() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(false), + }), + us_privacy: Some(UsPrivacy { + version: 1, + notice_given: PrivacyFlag::Yes, + opt_out_sale: PrivacyFlag::Yes, + lspa_covered: PrivacyFlag::NotApplicable, + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "GPP US should take priority over us_privacy opt-out" + ); + } + + #[test] + fn ec_us_state_gpp_no_us_section_falls_through_to_us_privacy() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("CA".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![2], + eu_tcf: None, + us_sale_opt_out: None, + }), + us_privacy: Some(UsPrivacy { + version: 1, + notice_given: PrivacyFlag::Yes, + opt_out_sale: PrivacyFlag::No, + lspa_covered: PrivacyFlag::NotApplicable, + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "GPP without US section should fall through to us_privacy" + ); + } +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test --workspace -p trusted-server-core -- consent::tests::ec_allowed_us_state_gpp` +Expected: FAIL — the GPP US branch doesn't exist yet, so `ec_allowed_us_state_gpp_no_sale_opt_out` fails (falls through to fail-closed). + +- [ ] **Step 3: Add the GPP US branch to `allows_ec_creation()`** + +In `crates/trusted-server-core/src/consent/mod.rs`, update `allows_ec_creation()`. The `UsState` arm currently reads: + +```rust + jurisdiction::Jurisdiction::UsState(_) => { + if ctx.gpc { + return false; + } + if let Some(tcf) = effective_tcf(ctx) { + return tcf.has_storage_consent(); + } + if let Some(usp) = &ctx.us_privacy { + return usp.opt_out_sale != PrivacyFlag::Yes; + } + false + } +``` + +Insert the GPP US check between TCF and us_privacy: + +```rust + jurisdiction::Jurisdiction::UsState(_) => { + if ctx.gpc { + return false; + } + if let Some(tcf) = effective_tcf(ctx) { + return tcf.has_storage_consent(); + } + // Check GPP US section for sale opt-out. + if let Some(gpp) = &ctx.gpp { + if let Some(opted_out) = gpp.us_sale_opt_out { + return !opted_out; + } + } + if let Some(usp) = &ctx.us_privacy { + return usp.opt_out_sale != PrivacyFlag::Yes; + } + false + } +``` + +- [ ] **Step 4: Run all tests** + +Run: `cargo test --workspace` +Expected: all tests pass, including the six new EC gating tests. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/consent/mod.rs +git commit -m "Recognize GPP US sale opt-out in EC consent gating" +``` + +--- + +## Task 4: Create Sourcepoint JS integration + +**Files:** +- Create: `crates/js/lib/src/integrations/sourcepoint/index.ts` + +- [ ] **Step 1: Write the test file first** + +Create `crates/js/lib/test/integrations/sourcepoint/index.test.ts`: + +```typescript +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { mirrorSourcepointConsent } from '../../../src/integrations/sourcepoint'; + +describe('integrations/sourcepoint', () => { + beforeEach(() => { + // Clear cookies and localStorage before each test. + document.cookie.split(';').forEach((c) => { + const name = c.split('=')[0].trim(); + if (name) document.cookie = `${name}=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/`; + }); + localStorage.clear(); + }); + + afterEach(() => { + localStorage.clear(); + }); + + it('mirrors __gpp and __gpp_sid from _sp_user_consent_* localStorage', () => { + const payload = { + gppData: { + gppString: 'DBABLA~BVQqAAAAAgA.QA', + applicableSections: [7], + }, + }; + localStorage.setItem('_sp_user_consent_36026', JSON.stringify(payload)); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(true); + expect(document.cookie).toContain('__gpp=DBABLA~BVQqAAAAAgA.QA'); + expect(document.cookie).toContain('__gpp_sid=7'); + }); + + it('handles multiple applicable sections', () => { + const payload = { + gppData: { + gppString: 'DBABLA~BVQqAAAAAgA.QA', + applicableSections: [7, 8], + }, + }; + localStorage.setItem('_sp_user_consent_99999', JSON.stringify(payload)); + + mirrorSourcepointConsent(); + + expect(document.cookie).toContain('__gpp_sid=7,8'); + }); + + it('returns false when no _sp_user_consent_* key exists', () => { + localStorage.setItem('unrelated_key', 'value'); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + expect(document.cookie).not.toContain('__gpp_sid='); + }); + + it('returns false for malformed JSON in localStorage', () => { + localStorage.setItem('_sp_user_consent_12345', 'not-json!!!'); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + }); + + it('returns false when gppData is missing from payload', () => { + localStorage.setItem('_sp_user_consent_12345', JSON.stringify({ otherField: true })); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + }); + + it('returns false when gppString is empty', () => { + const payload = { + gppData: { + gppString: '', + applicableSections: [7], + }, + }; + localStorage.setItem('_sp_user_consent_12345', JSON.stringify(payload)); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd crates/js/lib && npx vitest run test/integrations/sourcepoint/index.test.ts` +Expected: FAIL — module `../../../src/integrations/sourcepoint` does not exist. + +- [ ] **Step 3: Implement the integration** + +Create `crates/js/lib/src/integrations/sourcepoint/index.ts`: + +```typescript +import { log } from '../../core/log'; + +const SP_CONSENT_PREFIX = '_sp_user_consent_'; + +interface SourcepointGppData { + gppString: string; + applicableSections: number[]; +} + +interface SourcepointConsentPayload { + gppData?: SourcepointGppData; +} + +function findSourcepointConsent(): SourcepointConsentPayload | null { + for (let i = 0; i < localStorage.length; i++) { + const key = localStorage.key(i); + if (!key?.startsWith(SP_CONSENT_PREFIX)) continue; + + const raw = localStorage.getItem(key); + if (!raw) continue; + + try { + return JSON.parse(raw) as SourcepointConsentPayload; + } catch { + log.debug('sourcepoint: failed to parse localStorage value', { key }); + return null; + } + } + return null; +} + +function writeCookie(name: string, value: string): void { + document.cookie = `${name}=${encodeURIComponent(value)}; path=/; SameSite=Lax`; +} + +/// Reads Sourcepoint consent from localStorage and mirrors it into +/// `__gpp` and `__gpp_sid` cookies for Trusted Server to read. +/// +/// Returns `true` if cookies were written, `false` otherwise. +export function mirrorSourcepointConsent(): boolean { + if (typeof localStorage === 'undefined' || typeof document === 'undefined') { + return false; + } + + const payload = findSourcepointConsent(); + if (!payload?.gppData) { + log.debug('sourcepoint: no GPP data found in localStorage'); + return false; + } + + const { gppString, applicableSections } = payload.gppData; + if (!gppString) { + log.debug('sourcepoint: gppString is empty'); + return false; + } + + writeCookie('__gpp', gppString); + + if (Array.isArray(applicableSections) && applicableSections.length > 0) { + writeCookie('__gpp_sid', applicableSections.join(',')); + } + + log.info('sourcepoint: mirrored GPP consent to cookies', { + gppLength: gppString.length, + sections: applicableSections, + }); + + return true; +} + +if (typeof window !== 'undefined') { + mirrorSourcepointConsent(); +} + +export default mirrorSourcepointConsent; +``` + +- [ ] **Step 4: Run tests** + +Run: `cd crates/js/lib && npx vitest run test/integrations/sourcepoint/index.test.ts` +Expected: all 6 tests pass. + +- [ ] **Step 5: Run the full JS test suite** + +Run: `cd crates/js/lib && npx vitest run` +Expected: all tests pass (existing + new). + +- [ ] **Step 6: Format** + +Run: `cd crates/js/lib && npm run format` +Expected: no formatting issues. + +- [ ] **Step 7: Commit** + +```bash +git add crates/js/lib/src/integrations/sourcepoint/index.ts \ + crates/js/lib/test/integrations/sourcepoint/index.test.ts +git commit -m "Add Sourcepoint JS integration for GPP consent cookie mirroring" +``` + +--- + +## Task 5: Final verification + +**Files:** None (verification only) + +- [ ] **Step 1: Build the JS bundles** + +Run: `cd crates/js/lib && node build-all.mjs` +Expected: builds successfully, `dist/tsjs-sourcepoint.js` appears in the output. + +- [ ] **Step 2: Full Rust build** + +Run: `cargo build --workspace` +Expected: compiles with no errors. + +- [ ] **Step 3: Full Rust test suite** + +Run: `cargo test --workspace` +Expected: all tests pass. + +- [ ] **Step 4: Clippy** + +Run: `cargo clippy --workspace --all-targets --all-features -- -D warnings` +Expected: no warnings. + +- [ ] **Step 5: Rust format check** + +Run: `cargo fmt --all -- --check` +Expected: no formatting issues. + +- [ ] **Step 6: Full JS test suite** + +Run: `cd crates/js/lib && npx vitest run` +Expected: all tests pass. + +- [ ] **Step 7: JS format check** + +Run: `cd crates/js/lib && npm run format` +Expected: no formatting issues. From a1bc65707748344a7906e717fac19056ba0fd3b9 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 15 Apr 2026 15:52:34 -0500 Subject: [PATCH 03/22] Add us_sale_opt_out field to GppConsent --- crates/trusted-server-core/src/consent/gpp.rs | 1 + crates/trusted-server-core/src/consent/mod.rs | 3 +++ crates/trusted-server-core/src/consent/types.rs | 7 +++++++ 3 files changed, 11 insertions(+) diff --git a/crates/trusted-server-core/src/consent/gpp.rs b/crates/trusted-server-core/src/consent/gpp.rs index 9d0e5c81f..76b69e087 100644 --- a/crates/trusted-server-core/src/consent/gpp.rs +++ b/crates/trusted-server-core/src/consent/gpp.rs @@ -76,6 +76,7 @@ pub fn decode_gpp_string(gpp_string: &str) -> Result, /// Decoded EU TCF v2.2 section (if present in GPP, section ID 2). pub eu_tcf: Option, + /// Whether the user opted out of sale of personal information via a US GPP + /// section (IDs 7–23). + /// + /// - `Some(true)` — a US section is present and `sale_opt_out == OptedOut` + /// - `Some(false)` — a US section is present and user did not opt out + /// - `None` — no US section exists in the GPP string + pub us_sale_opt_out: Option, } // --------------------------------------------------------------------------- From 196487292dfe1c394d8f5908205131fc8c9ab328 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 15 Apr 2026 15:54:47 -0500 Subject: [PATCH 04/22] Decode US sale opt-out from GPP sections --- crates/trusted-server-core/src/consent/gpp.rs | 90 ++++++++++++++++++- 1 file changed, 89 insertions(+), 1 deletion(-) diff --git a/crates/trusted-server-core/src/consent/gpp.rs b/crates/trusted-server-core/src/consent/gpp.rs index 76b69e087..d6e8902ed 100644 --- a/crates/trusted-server-core/src/consent/gpp.rs +++ b/crates/trusted-server-core/src/consent/gpp.rs @@ -72,11 +72,13 @@ pub fn decode_gpp_string(gpp_string: &str) -> Result Option { } } +/// GPP section IDs that represent US state/national privacy sections. +/// +/// Range 7–23 per the GPP v1 specification: +/// 7=UsNat, 8=UsCa, 9=UsVa, 10=UsCo, 11=UsUt, 12=UsCt, 13=UsFl, +/// 14=UsMt, 15=UsOr, 16=UsTx, 17=UsDe, 18=UsIa, 19=UsNe, 20=UsNh, +/// 21=UsNj, 22=UsTn, 23=UsMn. +const US_SECTION_ID_RANGE: std::ops::RangeInclusive = 7..=23; + +/// Extracts the `sale_opt_out` signal from the first US section in a parsed +/// GPP string. +/// +/// Iterates through section IDs looking for any in the US range (7–23). +/// For the first match, decodes the section and extracts `sale_opt_out`. +/// +/// Returns `Some(true)` if the user opted out of sale, `Some(false)` if they +/// did not, or `None` if no US section is present. +fn decode_us_sale_opt_out(parsed: &iab_gpp::v1::GPPString) -> Option { + use iab_gpp::sections::us_common::OptOut; + use iab_gpp::sections::Section; + + let us_section_id = parsed + .section_ids() + .find(|id| US_SECTION_ID_RANGE.contains(&(**id as u16)))?; + + match parsed.decode_section(*us_section_id) { + Ok(section) => { + let sale_opt_out = match §ion { + Section::UsNat(s) => match &s.core { + iab_gpp::sections::usnat::Core::V1(c) => &c.sale_opt_out, + iab_gpp::sections::usnat::Core::V2(c) => &c.sale_opt_out, + _ => return None, + }, + Section::UsCa(s) => &s.core.sale_opt_out, + Section::UsVa(s) => &s.core.sale_opt_out, + Section::UsCo(s) => &s.core.sale_opt_out, + Section::UsUt(s) => &s.core.sale_opt_out, + Section::UsCt(s) => &s.core.sale_opt_out, + Section::UsFl(s) => &s.core.sale_opt_out, + Section::UsMt(s) => &s.core.sale_opt_out, + Section::UsOr(s) => &s.core.sale_opt_out, + Section::UsTx(s) => &s.core.sale_opt_out, + Section::UsDe(s) => &s.core.sale_opt_out, + Section::UsIa(s) => &s.core.sale_opt_out, + Section::UsNe(s) => &s.core.sale_opt_out, + Section::UsNh(s) => &s.core.sale_opt_out, + Section::UsNj(s) => &s.core.sale_opt_out, + Section::UsTn(s) => &s.core.sale_opt_out, + Section::UsMn(s) => &s.core.sale_opt_out, + // Non-US sections — should not reach here given the ID filter. + _ => return None, + }; + Some(*sale_opt_out == OptOut::OptedOut) + } + Err(e) => { + log::warn!("Failed to decode US GPP section {us_section_id}: {e}"); + None + } + } +} + /// Parses a `__gpp_sid` cookie value into a vector of section IDs. /// /// The cookie is a comma-separated list of integer section IDs, e.g. `"2,6"`. @@ -240,4 +302,30 @@ mod tests { "all-invalid should be None" ); } + + #[test] + fn decodes_us_sale_opt_out_not_opted_out() { + let result = decode_gpp_string("DBABLA~BVQqAAAAAgA.QA"); + match &result { + Ok(gpp) => { + assert_eq!( + gpp.us_sale_opt_out, + Some(false), + "should extract sale_opt_out=false from UsNat section" + ); + } + Err(e) => { + panic!("GPP decode failed: {e}"); + } + } + } + + #[test] + fn no_us_section_returns_none() { + let result = decode_gpp_string(GPP_TCF_AND_USP).expect("should decode GPP"); + assert_eq!( + result.us_sale_opt_out, None, + "should return None when no US section (7-23) is present" + ); + } } From fbb2457e964af1c98c7ef3b6bd60b0d8cfca6127 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 15 Apr 2026 15:57:34 -0500 Subject: [PATCH 05/22] Recognize GPP US sale opt-out in EC consent gating --- crates/trusted-server-core/src/consent/mod.rs | 128 ++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/crates/trusted-server-core/src/consent/mod.rs b/crates/trusted-server-core/src/consent/mod.rs index b75f8b2f1..4a7b5ce22 100644 --- a/crates/trusted-server-core/src/consent/mod.rs +++ b/crates/trusted-server-core/src/consent/mod.rs @@ -492,6 +492,12 @@ pub fn allows_ec_creation(ctx: &ConsentContext) -> bool { if let Some(tcf) = effective_tcf(ctx) { return tcf.has_storage_consent(); } + // Check GPP US section for sale opt-out. + if let Some(gpp) = &ctx.gpp { + if let Some(opted_out) = gpp.us_sale_opt_out { + return !opted_out; + } + } // Check US Privacy string for explicit opt-out. if let Some(usp) = &ctx.us_privacy { return usp.opt_out_sale != PrivacyFlag::Yes; @@ -1104,4 +1110,126 @@ mod tests { "TCF consent should take priority over US Privacy opt-out when both present" ); } + + #[test] + fn ec_allowed_us_state_gpp_no_sale_opt_out() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(false), + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "US state + GPP US sale_opt_out=false should allow EC" + ); + } + + #[test] + fn ec_blocked_us_state_gpp_sale_opted_out() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(true), + }), + ..ConsentContext::default() + }; + assert!( + !allows_ec_creation(&ctx), + "US state + GPP US sale_opt_out=true should block EC" + ); + } + + #[test] + fn ec_blocked_us_state_gpc_overrides_gpp_us() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpc: true, + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(false), + }), + ..ConsentContext::default() + }; + assert!( + !allows_ec_creation(&ctx), + "GPC should block EC even when GPP US says no opt-out" + ); + } + + #[test] + fn ec_us_state_tcf_takes_priority_over_gpp_us() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + tcf: Some(make_tcf_with_storage(true)), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(true), + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "TCF consent should take priority over GPP US opt-out" + ); + } + + #[test] + fn ec_us_state_gpp_us_takes_priority_over_us_privacy() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("TN".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![7], + eu_tcf: None, + us_sale_opt_out: Some(false), + }), + us_privacy: Some(UsPrivacy { + version: 1, + notice_given: PrivacyFlag::Yes, + opt_out_sale: PrivacyFlag::Yes, + lspa_covered: PrivacyFlag::NotApplicable, + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "GPP US should take priority over us_privacy opt-out" + ); + } + + #[test] + fn ec_us_state_gpp_no_us_section_falls_through_to_us_privacy() { + let ctx = ConsentContext { + jurisdiction: Jurisdiction::UsState("CA".to_owned()), + gpp: Some(GppConsent { + version: 1, + section_ids: vec![2], + eu_tcf: None, + us_sale_opt_out: None, + }), + us_privacy: Some(UsPrivacy { + version: 1, + notice_given: PrivacyFlag::Yes, + opt_out_sale: PrivacyFlag::No, + lspa_covered: PrivacyFlag::NotApplicable, + }), + ..ConsentContext::default() + }; + assert!( + allows_ec_creation(&ctx), + "GPP without US section should fall through to us_privacy" + ); + } } From ad6e790427edcb19ba023f06a5be42ecb555f86a Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 15 Apr 2026 15:58:57 -0500 Subject: [PATCH 06/22] Add Sourcepoint JS integration for GPP consent cookie mirroring --- .../lib/src/integrations/sourcepoint/index.ts | 77 ++++++++++++++++ .../integrations/sourcepoint/index.test.ts | 91 +++++++++++++++++++ 2 files changed, 168 insertions(+) create mode 100644 crates/js/lib/src/integrations/sourcepoint/index.ts create mode 100644 crates/js/lib/test/integrations/sourcepoint/index.test.ts diff --git a/crates/js/lib/src/integrations/sourcepoint/index.ts b/crates/js/lib/src/integrations/sourcepoint/index.ts new file mode 100644 index 000000000..850659dcf --- /dev/null +++ b/crates/js/lib/src/integrations/sourcepoint/index.ts @@ -0,0 +1,77 @@ +import { log } from '../../core/log'; + +const SP_CONSENT_PREFIX = '_sp_user_consent_'; + +interface SourcepointGppData { + gppString: string; + applicableSections: number[]; +} + +interface SourcepointConsentPayload { + gppData?: SourcepointGppData; +} + +function findSourcepointConsent(): SourcepointConsentPayload | null { + for (let i = 0; i < localStorage.length; i++) { + const key = localStorage.key(i); + if (!key?.startsWith(SP_CONSENT_PREFIX)) continue; + + const raw = localStorage.getItem(key); + if (!raw) continue; + + try { + return JSON.parse(raw) as SourcepointConsentPayload; + } catch { + log.debug('sourcepoint: failed to parse localStorage value', { key }); + return null; + } + } + return null; +} + +function writeCookie(name: string, value: string): void { + document.cookie = `${name}=${value}; path=/; SameSite=Lax`; +} + +/** + * Reads Sourcepoint consent from localStorage and mirrors it into + * `__gpp` and `__gpp_sid` cookies for Trusted Server to read. + * + * Returns `true` if cookies were written, `false` otherwise. + */ +export function mirrorSourcepointConsent(): boolean { + if (typeof localStorage === 'undefined' || typeof document === 'undefined') { + return false; + } + + const payload = findSourcepointConsent(); + if (!payload?.gppData) { + log.debug('sourcepoint: no GPP data found in localStorage'); + return false; + } + + const { gppString, applicableSections } = payload.gppData; + if (!gppString) { + log.debug('sourcepoint: gppString is empty'); + return false; + } + + writeCookie('__gpp', gppString); + + if (Array.isArray(applicableSections) && applicableSections.length > 0) { + writeCookie('__gpp_sid', applicableSections.join(',')); + } + + log.info('sourcepoint: mirrored GPP consent to cookies', { + gppLength: gppString.length, + sections: applicableSections, + }); + + return true; +} + +if (typeof window !== 'undefined') { + mirrorSourcepointConsent(); +} + +export default mirrorSourcepointConsent; diff --git a/crates/js/lib/test/integrations/sourcepoint/index.test.ts b/crates/js/lib/test/integrations/sourcepoint/index.test.ts new file mode 100644 index 000000000..4eaf763f6 --- /dev/null +++ b/crates/js/lib/test/integrations/sourcepoint/index.test.ts @@ -0,0 +1,91 @@ +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { mirrorSourcepointConsent } from '../../../src/integrations/sourcepoint'; + +describe('integrations/sourcepoint', () => { + beforeEach(() => { + // Clear cookies and localStorage before each test. + document.cookie.split(';').forEach((c) => { + const name = c.split('=')[0].trim(); + if (name) document.cookie = `${name}=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/`; + }); + localStorage.clear(); + }); + + afterEach(() => { + localStorage.clear(); + }); + + it('mirrors __gpp and __gpp_sid from _sp_user_consent_* localStorage', () => { + const payload = { + gppData: { + gppString: 'DBABLA~BVQqAAAAAgA.QA', + applicableSections: [7], + }, + }; + localStorage.setItem('_sp_user_consent_36026', JSON.stringify(payload)); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(true); + expect(document.cookie).toContain('__gpp=DBABLA~BVQqAAAAAgA.QA'); + expect(document.cookie).toContain('__gpp_sid=7'); + }); + + it('handles multiple applicable sections', () => { + const payload = { + gppData: { + gppString: 'DBABLA~BVQqAAAAAgA.QA', + applicableSections: [7, 8], + }, + }; + localStorage.setItem('_sp_user_consent_99999', JSON.stringify(payload)); + + mirrorSourcepointConsent(); + + expect(document.cookie).toContain('__gpp_sid=7,8'); + }); + + it('returns false when no _sp_user_consent_* key exists', () => { + localStorage.setItem('unrelated_key', 'value'); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + expect(document.cookie).not.toContain('__gpp_sid='); + }); + + it('returns false for malformed JSON in localStorage', () => { + localStorage.setItem('_sp_user_consent_12345', 'not-json!!!'); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + }); + + it('returns false when gppData is missing from payload', () => { + localStorage.setItem('_sp_user_consent_12345', JSON.stringify({ otherField: true })); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + }); + + it('returns false when gppString is empty', () => { + const payload = { + gppData: { + gppString: '', + applicableSections: [7], + }, + }; + localStorage.setItem('_sp_user_consent_12345', JSON.stringify(payload)); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(document.cookie).not.toContain('__gpp='); + }); +}); From e0251981b6ab40054af47e707444039a784a0f82 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 11:55:16 -0500 Subject: [PATCH 07/22] Add design spec for Prebid User ID Module support --- ...2026-04-16-prebid-user-id-module-design.md | 198 ++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md diff --git a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md new file mode 100644 index 000000000..bc0f3f669 --- /dev/null +++ b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md @@ -0,0 +1,198 @@ +# Prebid User ID Module support + +**Date:** 2026-04-16 +**Status:** Design +**Scope:** JS bundle (`crates/js/lib/src/integrations/prebid/index.ts`) + +## Problem + +The Trusted Server Prebid integration strips each publisher's origin `prebid.js` +and replaces it with a server-bundled build. That bundle imports the consent +management modules but does **not** import Prebid's User ID core module or any +ID submodules. As a result `pbjs.getUserIdsAsEids` is `undefined` at runtime, +the `syncPrebidEidsCookie()` helper early-returns, and the `ts-eids` cookie is +never written — even when the publisher's origin-side code has a fully +configured `userSync.userIds` list. + +Downstream, `crates/trusted-server-core/src/ec/prebid_eids.rs` never receives a +cookie to ingest, so matched partner UIDs never land in the KV identity graph. + +## Goal + +Bundle Prebid's User ID core module and a broad, widely-deployed set of ID +submodules so publishers' existing `pbjs.setConfig({ userSync: { userIds: ... } })` +calls activate real ID resolution. After first auction completes, `ts-eids` +cookie is written and the backend ingestion path (already implemented) takes +over. + +## Non-goals + +- No Rust changes. No new `trusted-server.toml` fields. +- No runtime config injection from the server (`window.__tsjs_prebid.userIds`) + — deferred to a follow-up. +- No build-time env-var toggle for the bundled set (e.g. `TSJS_PREBID_USER_IDS` + mirroring `TSJS_PREBID_ADAPTERS`) — deferred to a follow-up. +- No automatic alignment between bundled ID submodules and configured + `[[ec.partners]]` — operators must keep those in sync themselves. + +## Design + +### Bundled modules + +Exactly one file changes: `crates/js/lib/src/integrations/prebid/index.ts`. +Add static imports near the existing `consentManagement*.js` imports. + +**Core (required):** + +- `prebid.js/modules/userId.js` + +**Zero-config / auto-populating submodules** (resolve without publisher params): + +- `prebid.js/modules/sharedIdSystem.js` +- `prebid.js/modules/criteoIdSystem.js` +- `prebid.js/modules/33acrossIdSystem.js` +- `prebid.js/modules/pubProvidedIdSystem.js` +- `prebid.js/modules/quantcastIdSystem.js` + +**Param-based submodules** (inert until the publisher's `setConfig` supplies +the relevant params): + +- `prebid.js/modules/id5IdSystem.js` +- `prebid.js/modules/identityLinkIdSystem.js` +- `prebid.js/modules/liveIntentIdSystem.js` +- `prebid.js/modules/uid2IdSystem.js` +- `prebid.js/modules/euidIdSystem.js` +- `prebid.js/modules/intentIqIdSystem.js` +- `prebid.js/modules/lotamePanoramaIdSystem.js` +- `prebid.js/modules/connectIdSystem.js` +- `prebid.js/modules/merkleIdSystem.js` + +**Legacy / compatibility:** + +- `prebid.js/modules/pubCommonIdSystem.js` — deprecated in favor of SharedID + but still present in some publisher configs. + +Total: 1 core + 15 submodules = 16 new imports. + +No changes to `installPrebidNpm`, no changes to the `bidsBackHandler` shim, no +changes to `syncPrebidEidsCookie`. The existing cookie-writing path is already +correct — it was only silent because `pbjs.getUserIdsAsEids` did not exist. + +### Runtime flow + +No new runtime logic. The sequence below is what will light up once the +submodules are present: + +1. Rust `IntegrationHeadInjector` emits the `window.pbjs` / `window.pbjs.que` + / `window.__tsjs_prebid` bootstrap before any publisher-origin script runs. +2. Publisher origin code queues its existing config: + `pbjs.que.push(() => pbjs.setConfig({ userSync: { userIds: [...] } }))`. +3. Our bundle loads. `installPrebidNpm()` registers the `trustedServer` + adapter, shims `requestBids` (already appends a chained `bidsBackHandler` + calling `syncPrebidEidsCookie`), then calls `pbjs.processQueue()` — the + publisher's queued `setConfig` runs at this point and activates the + configured submodules (each self-registered at import time). +4. User ID Module resolves IDs per its own rules (TCF/GPP/USP-gated, async). +5. First `requestBids` fires. Auction completes. Chained `bidsBackHandler` + calls `syncPrebidEidsCookie()`. +6. `syncPrebidEidsCookie` calls `pbjs.getUserIdsAsEids()` (now a real + function), flattens `[{source, id, atype}]`, base64-encodes JSON, writes + `document.cookie = "ts-eids=..."`. +7. Subsequent `/auction` requests carry `Cookie: ts-eids=...`. +8. Backend (`crates/trusted-server-core/src/ec/prebid_eids.rs`) parses the + cookie, matches `source` against `[[ec.partners]]`, syncs partner UIDs to + KV. + +The first `/auction` request after a cold page load still will not carry +`ts-eids`, because the cookie is written in the post-auction handler. This +matches preexisting behavior. + +### Error handling + +All failure modes are already covered by existing code. No new error paths. + +- **Publisher has no `userSync.userIds` configured** → + `pbjs.getUserIdsAsEids()` returns `[]` → early-return at `index.ts:380-382`. + No cookie written. Silent. Correct. +- **Submodule fails to resolve** (no consent, no third-party ID, network + error) → handled inside Prebid; `getUserIdsAsEids()` returns only the + resolved subset. Cookie reflects what resolved. +- **Cookie payload exceeds 3072 bytes** → existing trim-and-retry loop at + `index.ts:404-411` drops entries from the tail until it fits. If a single + entry alone exceeds the cap, no cookie is written. +- **Unexpected exception in sync path** → caught by the existing `try/catch` + at `index.ts:417-419`, logged via `log.warn`, does not break the auction. +- **Module import failure at build time** → esbuild fails the build. This + catches missing or renamed Prebid modules before they ship. + +### Known caveats + +- **Backend pairing** — an EID whose `source` has no matching `[[ec.partners]]` + entry is dropped at the backend (with a debug log). Bundling + `id5IdSystem.js` is inert for EC identity-graph purposes unless the + operator also adds an `[[ec.partners]]` entry with + `source_domain = "id5-sync.com"`. Operators must keep the two lists in + sync. Not a code change here; documented as an operator concern. +- **Bundle size** — adding 15 modules increases the shipped `tsjs-prebid.js` + by an estimated ~100-150kb gzipped. Not gated on a build-time toggle in + this change. + +## Testing + +### Automated (Vitest) + +Add tests under `crates/js/lib/src/integrations/prebid/`: + +- **Import smoke test** — import `./index.ts` and assert + `typeof pbjs.getUserIdsAsEids === 'function'`. Guards against the exact + regression that motivated this work. +- **`syncPrebidEidsCookie` unit tests** (new or expanded) — mock + `pbjs.getUserIdsAsEids` to return a fixed `[{source, uids: [{id, atype}]}]` + array and assert the cookie is written with base64-encoded + `[{source, id, atype}]`. Cover: + - empty array → no cookie written + - normal payload → cookie written with expected value + - oversize payload → trimmed to fit; partial entries persisted + - single oversize entry → no cookie written + +### Manual (after deploy to a dev publisher) + +- DevTools console: `typeof pbjs.getUserIdsAsEids === 'function'` returns + `true`. +- `pbjs.getUserIdsAsEids()` returns a non-empty array for a publisher with + configured `userIds`. +- After the first auction: `document.cookie` contains `ts-eids=...`. Decoded + payload (base64 → JSON) matches the raw EIDs. +- Network tab: second `/auction` request carries `Cookie: ts-eids=...`. + +### Explicitly out of scope + +- Each individual ID submodule's resolution behavior — that is Prebid's + responsibility and covered by Prebid's own test suite. +- Backend ingestion of `ts-eids` — already covered by `prebid_eids.rs` + tests; no new backend code. +- Bundle-size regression gating — noted as a caveat, not enforced. + +## Rollout + +This is a bundle change only. No migration, no feature flag, no staged +rollout beyond normal deploy. + +On first deploy, publishers with active origin-side `userSync.userIds` +configuration will begin emitting `ts-eids` cookies after their first +auction. Publishers without `userSync.userIds` configured see no change. + +## Follow-ups + +1. **Build-time configurability** — introduce `_user_ids.generated.ts` + driven by a `TSJS_PREBID_USER_IDS` env var, mirroring the existing + `TSJS_PREBID_ADAPTERS` / `_adapters.generated.ts` pattern. Allows + operators to slim the bundle per deployment. +2. **Server-injected `userSync.userIds`** — extend `trusted-server.toml` + with a `[[integrations.prebid.user_ids]]` array. Rust serializes into + `window.__tsjs_prebid.userIds`. JS applies via `pbjs.setConfig` before + `processQueue()`. Supports publishers who do not run their own Prebid + config on origin. +3. **Partner alignment tooling** — a startup-time check that warns when a + bundled ID submodule has no matching `[[ec.partners]]` entry, or vice + versa. From 6450637de783480a5656c4a814d01799962812df Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 12:00:21 -0500 Subject: [PATCH 08/22] Add implementation plan for Prebid User ID Module support --- .../plans/2026-04-16-prebid-user-id-module.md | 575 ++++++++++++++++++ 1 file changed, 575 insertions(+) create mode 100644 docs/superpowers/plans/2026-04-16-prebid-user-id-module.md diff --git a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md new file mode 100644 index 000000000..534422fa2 --- /dev/null +++ b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md @@ -0,0 +1,575 @@ +# Prebid User ID Module — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Bundle Prebid's User ID core module and a broad set of ID submodules so publisher-side `pbjs.setConfig({ userSync: { userIds: [...] } })` calls activate real ID resolution and the existing `syncPrebidEidsCookie()` helper begins writing `ts-eids`. + +**Architecture:** JS-only change. Add 16 static imports (1 core + 15 submodules) to `crates/js/lib/src/integrations/prebid/index.ts`. No Rust changes, no TOML changes, no new runtime logic. Existing `bidsBackHandler` shim and cookie-sync path already handle the rest; they were silent only because `pbjs.getUserIdsAsEids` did not exist. + +**Tech Stack:** TypeScript, Vitest, esbuild (via `build-all.mjs`), Prebid.js 9.x (via npm) + +**Spec:** `docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md` + +--- + +## File Map + +| File | Action | Responsibility | +|---|---|---| +| `crates/js/lib/src/integrations/prebid/index.ts` | Modify | Add 16 User ID Module imports alongside existing `consentManagement*` imports | +| `crates/js/lib/test/integrations/prebid/index.test.ts` | Modify | Add `vi.mock` stubs for the 16 new modules, add `mockPbjs.getUserIdsAsEids`, add new tests for cookie-sync behavior and import regression guard | + +No other files change. `build.rs` picks up the rebuilt `dist/tsjs-prebid.js` automatically via `include_str!`. + +--- + +## Task 1: Document existing `syncPrebidEidsCookie` behavior with tests + +The sync helper already exists but has no test coverage. Before changing anything, lock in its current contract so we can refactor or extend later without regressions. These tests exercise the `bidsBackHandler` shim path end-to-end using the existing mocks. + +**Files:** +- Modify: `crates/js/lib/test/integrations/prebid/index.test.ts` + +- [ ] **Step 1: Add `getUserIdsAsEids` to the hoisted pbjs mock** + +In `crates/js/lib/test/integrations/prebid/index.test.ts` inside the existing `vi.hoisted(() => { ... })` block, add a new mock function and include it on `mockPbjs`. Replace the current block with: + +```ts +const { + mockSetConfig, + mockProcessQueue, + mockRequestBids, + mockRegisterBidAdapter, + mockGetUserIdsAsEids, + mockPbjs, + mockGetBidAdapter, + mockAdapterManager, +} = vi.hoisted(() => { + const mockSetConfig = vi.fn(); + const mockProcessQueue = vi.fn(); + const mockRequestBids = vi.fn(); + const mockRegisterBidAdapter = vi.fn(); + const mockGetBidAdapter = vi.fn(); + const mockGetUserIdsAsEids = vi.fn(() => [] as Array<{ source: string; uids?: Array<{ id: string; atype?: number }> }>); + const mockPbjs = { + setConfig: mockSetConfig, + processQueue: mockProcessQueue, + requestBids: mockRequestBids, + registerBidAdapter: mockRegisterBidAdapter, + getUserIdsAsEids: mockGetUserIdsAsEids, + adUnits: [] as any[], + }; + const mockAdapterManager = { + getBidAdapter: mockGetBidAdapter, + }; + return { + mockSetConfig, + mockProcessQueue, + mockRequestBids, + mockRegisterBidAdapter, + mockGetUserIdsAsEids, + mockPbjs, + mockGetBidAdapter, + mockAdapterManager, + }; +}); +``` + +- [ ] **Step 2: Write the failing test — empty EID array writes no cookie** + +Append this new `describe` block at the end of `crates/js/lib/test/integrations/prebid/index.test.ts`: + +```ts +describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { + beforeEach(() => { + vi.clearAllMocks(); + mockPbjs.requestBids = mockRequestBids; + mockPbjs.adUnits = []; + mockGetUserIdsAsEids.mockReset(); + mockGetUserIdsAsEids.mockReturnValue([]); + // Restore the pbjs→mock wiring in case a prior test blanked it out. + (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; + delete (window as any).__tsjs_prebid; + // Wipe any leftover ts-eids cookie from previous tests. + document.cookie = 'ts-eids=; Path=/; Max-Age=0'; + }); + + afterEach(() => { + document.cookie = 'ts-eids=; Path=/; Max-Age=0'; + }); + + /** + * Helper: make mockRequestBids actually invoke the injected bidsBackHandler + * so the shim's post-auction sync path runs. + */ + function wireBidsBackHandler(): void { + mockRequestBids.mockImplementation((opts: any) => { + if (typeof opts?.bidsBackHandler === 'function') { + opts.bidsBackHandler(); + } + }); + } + + function getTsEidsCookie(): string | undefined { + const match = document.cookie.split('; ').find((c) => c.startsWith('ts-eids=')); + return match ? match.split('=').slice(1).join('=') : undefined; + } + + it('writes no cookie when getUserIdsAsEids returns empty array', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); +}); +``` + +- [ ] **Step 3: Run the test — expect PASS (documents existing behavior)** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "writes no cookie when getUserIdsAsEids returns empty array"` + +Expected: PASS. The shim already calls `syncPrebidEidsCookie` which early-returns on empty input. + +- [ ] **Step 4: Add test — writes base64 cookie for a normal payload** + +Append inside the same `describe` block: + +```ts + it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'sharedid.org', uids: [{ id: 'shared-abc', atype: 1 }] }, + { source: 'id5-sync.com', uids: [{ id: 'id5-xyz', atype: 3 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const encoded = getTsEidsCookie(); + expect(encoded).toBeDefined(); + const decoded = JSON.parse(atob(encoded!)); + expect(decoded).toEqual([ + { source: 'sharedid.org', id: 'shared-abc', atype: 1 }, + { source: 'id5-sync.com', id: 'id5-xyz', atype: 3 }, + ]); + }); +``` + +- [ ] **Step 5: Run and confirm PASS** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "writes ts-eids cookie with base64"` + +Expected: PASS. + +- [ ] **Step 6: Add test — defaults atype to 3 when missing** + +Append: + +```ts + it('defaults atype to 3 when the uid omits it', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'example.com', uids: [{ id: 'no-atype' }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const decoded = JSON.parse(atob(getTsEidsCookie()!)); + expect(decoded).toEqual([{ source: 'example.com', id: 'no-atype', atype: 3 }]); + }); +``` + +- [ ] **Step 7: Add test — skips entries without an id or source** + +Append: + +```ts + it('skips EID entries that are missing id or source', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'good.example', uids: [{ id: 'keep', atype: 1 }] }, + { source: 'empty-uids.example', uids: [] }, + { source: '', uids: [{ id: 'no-source', atype: 1 }] }, + { source: 'no-id.example', uids: [{ id: '', atype: 1 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const decoded = JSON.parse(atob(getTsEidsCookie()!)); + expect(decoded).toEqual([{ source: 'good.example', id: 'keep', atype: 1 }]); + }); +``` + +- [ ] **Step 8: Add test — takes first uid when multiple are present** + +Append: + +```ts + it('takes the first uid per source when multiple are present', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { + source: 'multi.example', + uids: [ + { id: 'first', atype: 1 }, + { id: 'second', atype: 2 }, + ], + }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const decoded = JSON.parse(atob(getTsEidsCookie()!)); + expect(decoded).toEqual([{ source: 'multi.example', id: 'first', atype: 1 }]); + }); +``` + +- [ ] **Step 9: Add test — trims tail when payload exceeds 3072 bytes** + +Append: + +```ts + it('trims EIDs from the tail when the cookie payload would exceed 3072 bytes', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + + // Build ~20 entries each ~200 bytes → definitely exceeds 3072-byte cap + // once base64-encoded. + const big = Array.from({ length: 20 }, (_, i) => ({ + source: `source-${i}.example`, + uids: [{ id: 'x'.repeat(200) + String(i), atype: 3 }], + })); + mockGetUserIdsAsEids.mockReturnValue(big); + + pbjs.requestBids({ adUnits: [] } as any); + + const encoded = getTsEidsCookie(); + expect(encoded).toBeDefined(); + expect(encoded!.length).toBeLessThanOrEqual(3072); + + const decoded = JSON.parse(atob(encoded!)); + // At least one entry kept, strictly fewer than original count. + expect(decoded.length).toBeGreaterThan(0); + expect(decoded.length).toBeLessThan(big.length); + // Head of the list is preserved (trimming happens from the tail). + expect(decoded[0].source).toBe('source-0.example'); + }); +``` + +- [ ] **Step 10: Add test — writes no cookie when a single entry alone exceeds the cap** + +Append: + +```ts + it('writes no cookie when a single entry alone exceeds the cap', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + + // Single entry large enough to blow past 3072 bytes after base64. + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); +``` + +- [ ] **Step 11: Add test — does not throw when getUserIdsAsEids is undefined** + +This mirrors the pre-fix production state and guards against regressions in the defensive check at `index.ts:375`. Append: + +```ts + it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + // Simulate a build that forgot the userId core module. + (mockPbjs as any).getUserIdsAsEids = undefined; + + expect(() => pbjs.requestBids({ adUnits: [] } as any)).not.toThrow(); + expect(getTsEidsCookie()).toBeUndefined(); + + // Restore for subsequent tests. + (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; + }); +``` + +- [ ] **Step 12: Add test — calls the original bidsBackHandler when one was supplied** + +Append: + +```ts + it('calls the original bidsBackHandler after syncing EIDs', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + const originalHandler = vi.fn(); + + pbjs.requestBids({ adUnits: [], bidsBackHandler: originalHandler } as any); + + expect(originalHandler).toHaveBeenCalledTimes(1); + }); +``` + +- [ ] **Step 13: Run the full new block and confirm all pass** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "syncPrebidEidsCookie"` + +Expected: all 9 new tests PASS. If any fail, investigate before proceeding — the rest of the plan assumes this behavior is locked in. + +- [ ] **Step 14: Run the entire prebid test file to confirm no regressions** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts` + +Expected: all tests PASS (new + existing). + +- [ ] **Step 15: Commit** + +```bash +git add crates/js/lib/test/integrations/prebid/index.test.ts +git commit -m "Add Vitest coverage for Prebid ts-eids cookie sync" +``` + +--- + +## Task 2: Add Prebid User ID core and submodule imports + +This is the substantive change. Add `vi.mock` stubs for the new modules first (so tests don't blow up when the imports are added), then add the imports. + +**Files:** +- Modify: `crates/js/lib/test/integrations/prebid/index.test.ts:44-47` +- Modify: `crates/js/lib/src/integrations/prebid/index.ts:16-18` + +- [ ] **Step 1: Add `vi.mock` stubs in the test file for all 16 new modules** + +In `crates/js/lib/test/integrations/prebid/index.test.ts`, locate the existing block (around line 44-47): + +```ts +// Side-effect imports are no-ops in tests +vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})); +vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); +vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); +``` + +Replace it with (consent management mocks stay; add 16 new ones): + +```ts +// Side-effect imports are no-ops in tests +vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})); +vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); +vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); + +// User ID Module core + submodules — no-op mocks so jsdom does not try to +// execute the real Prebid code paths. +vi.mock('prebid.js/modules/userId.js', () => ({})); +vi.mock('prebid.js/modules/sharedIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/criteoIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/33acrossIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})); +vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/liveIntentIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})); +vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/lotamePanoramaIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/connectIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/merkleIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/pubCommonIdSystem.js', () => ({})); +``` + +- [ ] **Step 2: Write the failing regression-guard test** + +Append this `describe` block to `crates/js/lib/test/integrations/prebid/index.test.ts`: + +```ts +import { readFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +describe('prebid/index.ts User ID Module imports (regression guard)', () => { + const REQUIRED_IMPORTS = [ + 'prebid.js/modules/userId.js', + 'prebid.js/modules/sharedIdSystem.js', + 'prebid.js/modules/criteoIdSystem.js', + 'prebid.js/modules/33acrossIdSystem.js', + 'prebid.js/modules/pubProvidedIdSystem.js', + 'prebid.js/modules/quantcastIdSystem.js', + 'prebid.js/modules/id5IdSystem.js', + 'prebid.js/modules/identityLinkIdSystem.js', + 'prebid.js/modules/liveIntentIdSystem.js', + 'prebid.js/modules/uid2IdSystem.js', + 'prebid.js/modules/euidIdSystem.js', + 'prebid.js/modules/intentIqIdSystem.js', + 'prebid.js/modules/lotamePanoramaIdSystem.js', + 'prebid.js/modules/connectIdSystem.js', + 'prebid.js/modules/merkleIdSystem.js', + 'prebid.js/modules/pubCommonIdSystem.js', + ]; + + // Source-text check: these mocks make the runtime pbjs mock a no-op for the + // User ID Module, so there is no way to assert `typeof getUserIdsAsEids === + // 'function'` at import time from within Vitest. Reading the source file + // directly is the most reliable way to catch accidental removal of an + // import, which is the exact regression that motivated this work. + const SOURCE_PATH = resolve(__dirname, '../../../src/integrations/prebid/index.ts'); + const source = readFileSync(SOURCE_PATH, 'utf8'); + + for (const module of REQUIRED_IMPORTS) { + it(`statically imports ${module}`, () => { + const pattern = new RegExp(`import\\s+['"]${module.replace(/\./g, '\\.')}['"]`); + expect(source).toMatch(pattern); + }); + } +}); +``` + +- [ ] **Step 3: Run the new block — expect 16 failures** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "regression guard"` + +Expected: FAIL — 16 failing assertions, one per expected import. This confirms the regression guard actually reads the source. + +- [ ] **Step 4: Add the 16 imports to `index.ts`** + +In `crates/js/lib/src/integrations/prebid/index.ts`, locate lines 16-18: + +```ts +import 'prebid.js/modules/consentManagementTcf.js'; +import 'prebid.js/modules/consentManagementGpp.js'; +import 'prebid.js/modules/consentManagementUsp.js'; +``` + +Insert the User ID imports immediately after them, before the existing `// Client-side bid adapters` comment block. The resulting section must read: + +```ts +import 'prebid.js/modules/consentManagementTcf.js'; +import 'prebid.js/modules/consentManagementGpp.js'; +import 'prebid.js/modules/consentManagementUsp.js'; + +// Prebid User ID Module — core + submodules. The core module exposes +// `pbjs.getUserIdsAsEids`; submodules self-register at import time and +// activate when the publisher's origin-side `pbjs.setConfig({ userSync: +// { userIds: [...] } })` call runs during `processQueue()`. +import 'prebid.js/modules/userId.js'; + +// Zero-config / auto-populating submodules (resolve without publisher params). +import 'prebid.js/modules/sharedIdSystem.js'; +import 'prebid.js/modules/criteoIdSystem.js'; +import 'prebid.js/modules/33acrossIdSystem.js'; +import 'prebid.js/modules/pubProvidedIdSystem.js'; +import 'prebid.js/modules/quantcastIdSystem.js'; + +// Param-based submodules — inert until publisher setConfig supplies params. +import 'prebid.js/modules/id5IdSystem.js'; +import 'prebid.js/modules/identityLinkIdSystem.js'; +import 'prebid.js/modules/liveIntentIdSystem.js'; +import 'prebid.js/modules/uid2IdSystem.js'; +import 'prebid.js/modules/euidIdSystem.js'; +import 'prebid.js/modules/intentIqIdSystem.js'; +import 'prebid.js/modules/lotamePanoramaIdSystem.js'; +import 'prebid.js/modules/connectIdSystem.js'; +import 'prebid.js/modules/merkleIdSystem.js'; + +// Legacy / deprecated but still present in some publisher configs. +import 'prebid.js/modules/pubCommonIdSystem.js'; +``` + +- [ ] **Step 5: Run the regression-guard block — expect PASS** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "regression guard"` + +Expected: all 16 tests PASS. + +- [ ] **Step 6: Run the full prebid test file — expect no regressions** + +Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts` + +Expected: all tests PASS (Task 1 tests + regression guards + all pre-existing tests). + +- [ ] **Step 7: Run the entire JS test suite** + +Run: `cd crates/js/lib && npx vitest run` + +Expected: all PASS. If any unrelated test fails because it imports `./index.ts` transitively and the real Prebid modules are not mocked there, add matching `vi.mock` stubs at the top of that test file (copy the same 16 lines). Common suspects: any test under `crates/js/lib/test/integrations/prebid/` you may have added, or tests that import core modules which in turn import the prebid integration. At the time of writing, no other test file imports the prebid integration. + +- [ ] **Step 8: Build the JS bundles** + +Run: `cd crates/js/lib && node build-all.mjs` + +Expected: build succeeds. `dist/tsjs-prebid.js` gets substantially larger (est. 100-150kb gzipped increase). No esbuild errors about missing modules — if there are, the module path in the new imports is wrong (check `crates/js/lib/node_modules/prebid.js/modules/` for the exact filename — note `33acrossIdSystem.js` really does start with a digit and is correct). + +- [ ] **Step 9: Format the JS** + +Run: `cd crates/js/lib && npm run format` + +Expected: prettier rewrites any formatting drift in the files you touched. No errors. + +- [ ] **Step 10: Verify the Rust build picks up the rebuilt bundle** + +Run: `cargo check --package trusted-server-core` + +Expected: PASS. `build.rs` re-runs because `dist/tsjs-prebid.js` changed; `include_str!` pulls in the new content. + +- [ ] **Step 11: Run full Rust test suite to confirm no downstream breakage** + +Run: `cargo test --workspace` + +Expected: PASS. The Rust side does not inspect bundle contents, only concatenates and hashes them, so tests should be unaffected. + +- [ ] **Step 12: Commit** + +```bash +git add crates/js/lib/src/integrations/prebid/index.ts crates/js/lib/test/integrations/prebid/index.test.ts +git commit -m "Bundle Prebid User ID core and submodules in Prebid integration" +``` + +--- + +## Task 3: Final verification + +- [ ] **Step 1: Full CI-equivalent check** + +Run the same sequence CI runs: + +```bash +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings +cargo test --workspace +cd crates/js/lib && npx vitest run && cd ../../.. +cd crates/js/lib && npm run format && cd ../../.. +``` + +Expected: everything PASS / clean. + +- [ ] **Step 2: Manual verification note** + +Manual browser verification (cannot be automated here; run against a dev publisher environment that has origin-side `pbjs.setConfig({ userSync: { userIds: [...] } })`): + +1. Load a publisher page. In DevTools console: `typeof pbjs.getUserIdsAsEids` should return `'function'`. +2. `pbjs.getUserIdsAsEids()` should return a non-empty array. +3. After the first ad-slot auction completes: `document.cookie.match(/ts-eids=/)` should match. +4. Decode the cookie: `JSON.parse(atob(document.cookie.match(/ts-eids=([^;]+)/)[1]))` should produce a `[{source, id, atype}]` array matching the raw EIDs. +5. Network tab: the second `/auction` request should carry `Cookie: ts-eids=...`. + +These are documented in the spec; they are not blockers for the PR, but they should be run before closing out the work. + +- [ ] **Step 3: No follow-up commits required** + +The work is complete when Tasks 1 and 2 are committed. Do not create a third "chore" commit unless format/clippy asks for one. + +--- + +## What this plan intentionally does NOT do + +- Does **not** add a build-time env-var toggle (`TSJS_PREBID_USER_IDS`) to mirror `TSJS_PREBID_ADAPTERS`. Deferred per spec. +- Does **not** add `window.__tsjs_prebid.userIds` server-side injection. Deferred per spec. +- Does **not** change `[[ec.partners]]` or `crates/trusted-server-core/src/ec/prebid_eids.rs`. Backend already handles received cookies correctly. +- Does **not** add a bundle-size regression gate. Noted as a known cost in the spec. +- Does **not** add tests for individual ID submodule resolution behavior. That is Prebid's own test surface, not ours. From a760c4cc8a35296ccc37a36d60e4d2b2c96561ce Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 12:01:49 -0500 Subject: [PATCH 09/22] Fix ESM path resolution in Prebid User ID plan regression guard --- docs/superpowers/plans/2026-04-16-prebid-user-id-module.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md index 534422fa2..9a1e33502 100644 --- a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md +++ b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md @@ -391,6 +391,7 @@ Append this `describe` block to `crates/js/lib/test/integrations/prebid/index.te ```ts import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; import { resolve } from 'node:path'; describe('prebid/index.ts User ID Module imports (regression guard)', () => { @@ -418,7 +419,10 @@ describe('prebid/index.ts User ID Module imports (regression guard)', () => { // 'function'` at import time from within Vitest. Reading the source file // directly is the most reliable way to catch accidental removal of an // import, which is the exact regression that motivated this work. - const SOURCE_PATH = resolve(__dirname, '../../../src/integrations/prebid/index.ts'); + // The package is ESM (`"type": "module"`), so `__dirname` is not defined — + // resolve relative to this file via `import.meta.url`. + const THIS_DIR = fileURLToPath(new URL('.', import.meta.url)); + const SOURCE_PATH = resolve(THIS_DIR, '../../../src/integrations/prebid/index.ts'); const source = readFileSync(SOURCE_PATH, 'utf8'); for (const module of REQUIRED_IMPORTS) { From c8abb71514daf175393013196da6c230744bb70b Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 12:03:12 -0500 Subject: [PATCH 10/22] Add Vitest coverage for Prebid ts-eids cookie sync --- .../test/integrations/prebid/index.test.ts | 184 +++++++++++++++++- 1 file changed, 181 insertions(+), 3 deletions(-) diff --git a/crates/js/lib/test/integrations/prebid/index.test.ts b/crates/js/lib/test/integrations/prebid/index.test.ts index e7cfd3a7a..2af4d6ec9 100644 --- a/crates/js/lib/test/integrations/prebid/index.test.ts +++ b/crates/js/lib/test/integrations/prebid/index.test.ts @@ -6,9 +6,9 @@ const { mockProcessQueue, mockRequestBids, mockRegisterBidAdapter, + mockGetUserIdsAsEids, mockPbjs, mockGetBidAdapter, - mockGetUserIdsAsEids, mockAdapterManager, } = vi.hoisted(() => { const mockSetConfig = vi.fn(); @@ -16,7 +16,9 @@ const { const mockRequestBids = vi.fn(); const mockRegisterBidAdapter = vi.fn(); const mockGetBidAdapter = vi.fn(); - const mockGetUserIdsAsEids = vi.fn(); + const mockGetUserIdsAsEids = vi.fn( + () => [] as Array<{ source: string; uids?: Array<{ id: string; atype?: number }> }> + ); const mockPbjs = { setConfig: mockSetConfig, processQueue: mockProcessQueue, @@ -33,9 +35,9 @@ const { mockProcessQueue, mockRequestBids, mockRegisterBidAdapter, + mockGetUserIdsAsEids, mockPbjs, mockGetBidAdapter, - mockGetUserIdsAsEids, mockAdapterManager, }; }); @@ -969,3 +971,179 @@ describe('prebid/client-side bidders', () => { errorSpy.mockRestore(); }); }); + +describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { + beforeEach(() => { + vi.clearAllMocks(); + mockPbjs.requestBids = mockRequestBids; + mockPbjs.adUnits = []; + mockGetUserIdsAsEids.mockReset(); + mockGetUserIdsAsEids.mockReturnValue([]); + // Restore the pbjs→mock wiring in case a prior test blanked it out. + (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; + delete (window as any).__tsjs_prebid; + // Wipe any leftover ts-eids cookie from previous tests. + document.cookie = 'ts-eids=; Path=/; Max-Age=0'; + }); + + afterEach(() => { + document.cookie = 'ts-eids=; Path=/; Max-Age=0'; + }); + + /** + * Helper: make mockRequestBids actually invoke the injected bidsBackHandler + * so the shim's post-auction sync path runs. + */ + function wireBidsBackHandler(): void { + mockRequestBids.mockImplementation((opts: any) => { + if (typeof opts?.bidsBackHandler === 'function') { + opts.bidsBackHandler(); + } + }); + } + + function getTsEidsCookie(): string | undefined { + const match = document.cookie.split('; ').find((c) => c.startsWith('ts-eids=')); + return match ? match.split('=').slice(1).join('=') : undefined; + } + + it('writes no cookie when getUserIdsAsEids returns empty array', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); + + it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'sharedid.org', uids: [{ id: 'shared-abc', atype: 1 }] }, + { source: 'id5-sync.com', uids: [{ id: 'id5-xyz', atype: 3 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const encoded = getTsEidsCookie(); + expect(encoded).toBeDefined(); + const decoded = JSON.parse(atob(encoded!)); + expect(decoded).toEqual([ + { source: 'sharedid.org', id: 'shared-abc', atype: 1 }, + { source: 'id5-sync.com', id: 'id5-xyz', atype: 3 }, + ]); + }); + + it('defaults atype to 3 when the uid omits it', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'example.com', uids: [{ id: 'no-atype' }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const decoded = JSON.parse(atob(getTsEidsCookie()!)); + expect(decoded).toEqual([{ source: 'example.com', id: 'no-atype', atype: 3 }]); + }); + + it('skips EID entries that are missing id or source', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'good.example', uids: [{ id: 'keep', atype: 1 }] }, + { source: 'empty-uids.example', uids: [] }, + { source: '', uids: [{ id: 'no-source', atype: 1 }] }, + { source: 'no-id.example', uids: [{ id: '', atype: 1 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const decoded = JSON.parse(atob(getTsEidsCookie()!)); + expect(decoded).toEqual([{ source: 'good.example', id: 'keep', atype: 1 }]); + }); + + it('takes the first uid per source when multiple are present', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + mockGetUserIdsAsEids.mockReturnValue([ + { + source: 'multi.example', + uids: [ + { id: 'first', atype: 1 }, + { id: 'second', atype: 2 }, + ], + }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + const decoded = JSON.parse(atob(getTsEidsCookie()!)); + expect(decoded).toEqual([{ source: 'multi.example', id: 'first', atype: 1 }]); + }); + + it('trims EIDs from the tail when the cookie payload would exceed 3072 bytes', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + + // Build ~20 entries each ~200 bytes → definitely exceeds 3072-byte cap + // once base64-encoded. + const big = Array.from({ length: 20 }, (_, i) => ({ + source: `source-${i}.example`, + uids: [{ id: 'x'.repeat(200) + String(i), atype: 3 }], + })); + mockGetUserIdsAsEids.mockReturnValue(big); + + pbjs.requestBids({ adUnits: [] } as any); + + const encoded = getTsEidsCookie(); + expect(encoded).toBeDefined(); + expect(encoded!.length).toBeLessThanOrEqual(3072); + + const decoded = JSON.parse(atob(encoded!)); + // At least one entry kept, strictly fewer than original count. + expect(decoded.length).toBeGreaterThan(0); + expect(decoded.length).toBeLessThan(big.length); + // Head of the list is preserved (trimming happens from the tail). + expect(decoded[0].source).toBe('source-0.example'); + }); + + it('writes no cookie when a single entry alone exceeds the cap', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + + // Single entry large enough to blow past 3072 bytes after base64. + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); + + it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + // Simulate a build that forgot the userId core module. + (mockPbjs as any).getUserIdsAsEids = undefined; + + expect(() => pbjs.requestBids({ adUnits: [] } as any)).not.toThrow(); + expect(getTsEidsCookie()).toBeUndefined(); + + // Restore for subsequent tests. + (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; + }); + + it('calls the original bidsBackHandler after syncing EIDs', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + const originalHandler = vi.fn(); + + pbjs.requestBids({ adUnits: [], bidsBackHandler: originalHandler } as any); + + expect(originalHandler).toHaveBeenCalledTimes(1); + }); +}); From eee99f063a3360ad8f65f1f5c8765e9d5d659b13 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 12:07:59 -0500 Subject: [PATCH 11/22] Bundle Prebid User ID core and submodules in Prebid integration --- .../js/lib/src/integrations/prebid/index.ts | 24 +++++++ .../test/integrations/prebid/index.test.ts | 62 ++++++++++++++++++- 2 files changed, 83 insertions(+), 3 deletions(-) diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index 6b03e820c..99ac3fb82 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -18,6 +18,30 @@ import 'prebid.js/modules/consentManagementGpp.js'; import 'prebid.js/modules/consentManagementUsp.js'; import 'prebid.js/modules/userId.js'; +// Prebid User ID Module — core + submodules. The core module exposes +// `pbjs.getUserIdsAsEids`; submodules self-register at import time and +// activate when the publisher's origin-side `pbjs.setConfig({ userSync: +// { userIds: [...] } })` call runs during `processQueue()`. +import 'prebid.js/modules/userId.js'; + +// Zero-config / auto-populating submodules (resolve without publisher params). +import 'prebid.js/modules/sharedIdSystem.js'; +import 'prebid.js/modules/criteoIdSystem.js'; +import 'prebid.js/modules/33acrossIdSystem.js'; +import 'prebid.js/modules/pubProvidedIdSystem.js'; +import 'prebid.js/modules/quantcastIdSystem.js'; + +// Param-based submodules — inert until publisher setConfig supplies params. +import 'prebid.js/modules/id5IdSystem.js'; +import 'prebid.js/modules/identityLinkIdSystem.js'; +import 'prebid.js/modules/liveIntentIdSystem.js'; +import 'prebid.js/modules/uid2IdSystem.js'; +import 'prebid.js/modules/euidIdSystem.js'; +import 'prebid.js/modules/intentIqIdSystem.js'; +import 'prebid.js/modules/lotamePanoramaIdSystem.js'; +import 'prebid.js/modules/connectIdSystem.js'; +import 'prebid.js/modules/merkleIdSystem.js'; + // Client-side bid adapters — self-register with prebid.js on import. // The set of adapters is controlled by the TSJS_PREBID_ADAPTERS env var at // build time. See _adapters.generated.ts (written by build-all.mjs). diff --git a/crates/js/lib/test/integrations/prebid/index.test.ts b/crates/js/lib/test/integrations/prebid/index.test.ts index 2af4d6ec9..30156d9b7 100644 --- a/crates/js/lib/test/integrations/prebid/index.test.ts +++ b/crates/js/lib/test/integrations/prebid/index.test.ts @@ -1,3 +1,6 @@ +import { readFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; // Define mocks using vi.hoisted so they're available inside vi.mock factories @@ -53,6 +56,24 @@ vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); vi.mock('prebid.js/modules/userId.js', () => ({})); +// User ID Module core + submodules — no-op mocks so jsdom does not try to +// execute the real Prebid code paths. +vi.mock('prebid.js/modules/userId.js', () => ({})); +vi.mock('prebid.js/modules/sharedIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/criteoIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/33acrossIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})); +vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/liveIntentIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})); +vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/lotamePanoramaIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/connectIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/merkleIdSystem.js', () => ({})); + // Mock the build-generated adapter imports (no-op in tests) vi.mock('../../../src/integrations/prebid/_adapters.generated', () => ({})); @@ -1039,9 +1060,7 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { it('defaults atype to 3 when the uid omits it', () => { wireBidsBackHandler(); const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'example.com', uids: [{ id: 'no-atype' }] }, - ]); + mockGetUserIdsAsEids.mockReturnValue([{ source: 'example.com', uids: [{ id: 'no-atype' }] }]); pbjs.requestBids({ adUnits: [] } as any); @@ -1147,3 +1166,40 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { expect(originalHandler).toHaveBeenCalledTimes(1); }); }); + +describe('prebid/index.ts User ID Module imports (regression guard)', () => { + const REQUIRED_IMPORTS = [ + 'prebid.js/modules/userId.js', + 'prebid.js/modules/sharedIdSystem.js', + 'prebid.js/modules/criteoIdSystem.js', + 'prebid.js/modules/33acrossIdSystem.js', + 'prebid.js/modules/pubProvidedIdSystem.js', + 'prebid.js/modules/quantcastIdSystem.js', + 'prebid.js/modules/id5IdSystem.js', + 'prebid.js/modules/identityLinkIdSystem.js', + 'prebid.js/modules/liveIntentIdSystem.js', + 'prebid.js/modules/uid2IdSystem.js', + 'prebid.js/modules/euidIdSystem.js', + 'prebid.js/modules/intentIqIdSystem.js', + 'prebid.js/modules/lotamePanoramaIdSystem.js', + 'prebid.js/modules/connectIdSystem.js', + 'prebid.js/modules/merkleIdSystem.js', + ]; + + // Source-text check: these mocks make the runtime pbjs mock a no-op for the + // User ID Module, so there is no way to assert `typeof getUserIdsAsEids === + // 'function'` at import time from within Vitest. Reading the source file + // directly is the most reliable way to catch accidental removal of an + // import, which is the exact regression that motivated this work. + // Vitest runs with `process.cwd()` set to the package root (`crates/js/lib`) + // where `package.json` lives. + const SOURCE_PATH = resolve(process.cwd(), 'src/integrations/prebid/index.ts'); + const source = readFileSync(SOURCE_PATH, 'utf8'); + + for (const module of REQUIRED_IMPORTS) { + it(`statically imports ${module}`, () => { + const pattern = new RegExp(`import\\s+['"]${module.replace(/\./g, '\\.')}['"]`); + expect(source).toMatch(pattern); + }); + } +}); From bd366ad0a9f0c21e48d1be9a7d6864362d2d500c Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 12:09:55 -0500 Subject: [PATCH 12/22] =?UTF-8?q?Correct=20Prebid=20User=20ID=20plan=20+?= =?UTF-8?q?=20spec=20=E2=80=94=20drop=20pubCommonIdSystem=20(removed=20in?= =?UTF-8?q?=20Prebid=2010)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../plans/2026-04-16-prebid-user-id-module.md | 452 +++++++++--------- ...2026-04-16-prebid-user-id-module-design.md | 10 +- 2 files changed, 241 insertions(+), 221 deletions(-) diff --git a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md index 9a1e33502..cbf3acaf2 100644 --- a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md +++ b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md @@ -4,7 +4,12 @@ **Goal:** Bundle Prebid's User ID core module and a broad set of ID submodules so publisher-side `pbjs.setConfig({ userSync: { userIds: [...] } })` calls activate real ID resolution and the existing `syncPrebidEidsCookie()` helper begins writing `ts-eids`. -**Architecture:** JS-only change. Add 16 static imports (1 core + 15 submodules) to `crates/js/lib/src/integrations/prebid/index.ts`. No Rust changes, no TOML changes, no new runtime logic. Existing `bidsBackHandler` shim and cookie-sync path already handle the rest; they were silent only because `pbjs.getUserIdsAsEids` did not exist. +**Architecture:** JS-only change. Add 15 static imports (1 core + 14 submodules) to `crates/js/lib/src/integrations/prebid/index.ts`. No Rust changes, no TOML changes, no new runtime logic. Existing `bidsBackHandler` shim and cookie-sync path already handle the rest; they were silent only because `pbjs.getUserIdsAsEids` did not exist. + +> **In-flight correction (2026-04-16):** `pubCommonIdSystem.js` was originally +> included in the plan as a legacy/compatibility submodule. It does not exist +> in Prebid 10.26.0 (consolidated into `sharedIdSystem`) and has been removed. +> All downstream counts reflect 14 submodules instead of 15. **Tech Stack:** TypeScript, Vitest, esbuild (via `build-all.mjs`), Prebid.js 9.x (via npm) @@ -14,10 +19,10 @@ ## File Map -| File | Action | Responsibility | -|---|---|---| -| `crates/js/lib/src/integrations/prebid/index.ts` | Modify | Add 16 User ID Module imports alongside existing `consentManagement*` imports | -| `crates/js/lib/test/integrations/prebid/index.test.ts` | Modify | Add `vi.mock` stubs for the 16 new modules, add `mockPbjs.getUserIdsAsEids`, add new tests for cookie-sync behavior and import regression guard | +| File | Action | Responsibility | +| ------------------------------------------------------ | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| `crates/js/lib/src/integrations/prebid/index.ts` | Modify | Add 15 User ID Module imports alongside existing `consentManagement*` imports | +| `crates/js/lib/test/integrations/prebid/index.test.ts` | Modify | Add `vi.mock` stubs for the 15 new modules, add `mockPbjs.getUserIdsAsEids`, add new tests for cookie-sync behavior and import regression guard | No other files change. `build.rs` picks up the rebuilt `dist/tsjs-prebid.js` automatically via `include_str!`. @@ -28,6 +33,7 @@ No other files change. `build.rs` picks up the rebuilt `dist/tsjs-prebid.js` aut The sync helper already exists but has no test coverage. Before changing anything, lock in its current contract so we can refactor or extend later without regressions. These tests exercise the `bidsBackHandler` shim path end-to-end using the existing mocks. **Files:** + - Modify: `crates/js/lib/test/integrations/prebid/index.test.ts` - [ ] **Step 1: Add `getUserIdsAsEids` to the hoisted pbjs mock** @@ -45,12 +51,18 @@ const { mockGetBidAdapter, mockAdapterManager, } = vi.hoisted(() => { - const mockSetConfig = vi.fn(); - const mockProcessQueue = vi.fn(); - const mockRequestBids = vi.fn(); - const mockRegisterBidAdapter = vi.fn(); - const mockGetBidAdapter = vi.fn(); - const mockGetUserIdsAsEids = vi.fn(() => [] as Array<{ source: string; uids?: Array<{ id: string; atype?: number }> }>); + const mockSetConfig = vi.fn() + const mockProcessQueue = vi.fn() + const mockRequestBids = vi.fn() + const mockRegisterBidAdapter = vi.fn() + const mockGetBidAdapter = vi.fn() + const mockGetUserIdsAsEids = vi.fn( + () => + [] as Array<{ + source: string + uids?: Array<{ id: string; atype?: number }> + }> + ) const mockPbjs = { setConfig: mockSetConfig, processQueue: mockProcessQueue, @@ -58,10 +70,10 @@ const { registerBidAdapter: mockRegisterBidAdapter, getUserIdsAsEids: mockGetUserIdsAsEids, adUnits: [] as any[], - }; + } const mockAdapterManager = { getBidAdapter: mockGetBidAdapter, - }; + } return { mockSetConfig, mockProcessQueue, @@ -71,8 +83,8 @@ const { mockPbjs, mockGetBidAdapter, mockAdapterManager, - }; -}); + } +}) ``` - [ ] **Step 2: Write the failing test — empty EID array writes no cookie** @@ -82,21 +94,21 @@ Append this new `describe` block at the end of `crates/js/lib/test/integrations/ ```ts describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { beforeEach(() => { - vi.clearAllMocks(); - mockPbjs.requestBids = mockRequestBids; - mockPbjs.adUnits = []; - mockGetUserIdsAsEids.mockReset(); - mockGetUserIdsAsEids.mockReturnValue([]); + vi.clearAllMocks() + mockPbjs.requestBids = mockRequestBids + mockPbjs.adUnits = [] + mockGetUserIdsAsEids.mockReset() + mockGetUserIdsAsEids.mockReturnValue([]) // Restore the pbjs→mock wiring in case a prior test blanked it out. - (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; - delete (window as any).__tsjs_prebid; + ;(mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids + delete (window as any).__tsjs_prebid // Wipe any leftover ts-eids cookie from previous tests. - document.cookie = 'ts-eids=; Path=/; Max-Age=0'; - }); + document.cookie = 'ts-eids=; Path=/; Max-Age=0' + }) afterEach(() => { - document.cookie = 'ts-eids=; Path=/; Max-Age=0'; - }); + document.cookie = 'ts-eids=; Path=/; Max-Age=0' + }) /** * Helper: make mockRequestBids actually invoke the injected bidsBackHandler @@ -105,26 +117,28 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { function wireBidsBackHandler(): void { mockRequestBids.mockImplementation((opts: any) => { if (typeof opts?.bidsBackHandler === 'function') { - opts.bidsBackHandler(); + opts.bidsBackHandler() } - }); + }) } function getTsEidsCookie(): string | undefined { - const match = document.cookie.split('; ').find((c) => c.startsWith('ts-eids=')); - return match ? match.split('=').slice(1).join('=') : undefined; + const match = document.cookie + .split('; ') + .find((c) => c.startsWith('ts-eids=')) + return match ? match.split('=').slice(1).join('=') : undefined } it('writes no cookie when getUserIdsAsEids returns empty array', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([]); + wireBidsBackHandler() + const pbjs = installPrebidNpm() + mockGetUserIdsAsEids.mockReturnValue([]) - pbjs.requestBids({ adUnits: [] } as any); + pbjs.requestBids({ adUnits: [] } as any) - expect(getTsEidsCookie()).toBeUndefined(); - }); -}); + expect(getTsEidsCookie()).toBeUndefined() + }) +}) ``` - [ ] **Step 3: Run the test — expect PASS (documents existing behavior)** @@ -138,24 +152,24 @@ Expected: PASS. The shim already calls `syncPrebidEidsCookie` which early-return Append inside the same `describe` block: ```ts - it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'sharedid.org', uids: [{ id: 'shared-abc', atype: 1 }] }, - { source: 'id5-sync.com', uids: [{ id: 'id5-xyz', atype: 3 }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const encoded = getTsEidsCookie(); - expect(encoded).toBeDefined(); - const decoded = JSON.parse(atob(encoded!)); - expect(decoded).toEqual([ - { source: 'sharedid.org', id: 'shared-abc', atype: 1 }, - { source: 'id5-sync.com', id: 'id5-xyz', atype: 3 }, - ]); - }); +it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'sharedid.org', uids: [{ id: 'shared-abc', atype: 1 }] }, + { source: 'id5-sync.com', uids: [{ id: 'id5-xyz', atype: 3 }] }, + ]) + + pbjs.requestBids({ adUnits: [] } as any) + + const encoded = getTsEidsCookie() + expect(encoded).toBeDefined() + const decoded = JSON.parse(atob(encoded!)) + expect(decoded).toEqual([ + { source: 'sharedid.org', id: 'shared-abc', atype: 1 }, + { source: 'id5-sync.com', id: 'id5-xyz', atype: 3 }, + ]) +}) ``` - [ ] **Step 5: Run and confirm PASS** @@ -169,18 +183,18 @@ Expected: PASS. Append: ```ts - it('defaults atype to 3 when the uid omits it', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'example.com', uids: [{ id: 'no-atype' }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const decoded = JSON.parse(atob(getTsEidsCookie()!)); - expect(decoded).toEqual([{ source: 'example.com', id: 'no-atype', atype: 3 }]); - }); +it('defaults atype to 3 when the uid omits it', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'example.com', uids: [{ id: 'no-atype' }] }, + ]) + + pbjs.requestBids({ adUnits: [] } as any) + + const decoded = JSON.parse(atob(getTsEidsCookie()!)) + expect(decoded).toEqual([{ source: 'example.com', id: 'no-atype', atype: 3 }]) +}) ``` - [ ] **Step 7: Add test — skips entries without an id or source** @@ -188,21 +202,21 @@ Append: Append: ```ts - it('skips EID entries that are missing id or source', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'good.example', uids: [{ id: 'keep', atype: 1 }] }, - { source: 'empty-uids.example', uids: [] }, - { source: '', uids: [{ id: 'no-source', atype: 1 }] }, - { source: 'no-id.example', uids: [{ id: '', atype: 1 }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const decoded = JSON.parse(atob(getTsEidsCookie()!)); - expect(decoded).toEqual([{ source: 'good.example', id: 'keep', atype: 1 }]); - }); +it('skips EID entries that are missing id or source', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'good.example', uids: [{ id: 'keep', atype: 1 }] }, + { source: 'empty-uids.example', uids: [] }, + { source: '', uids: [{ id: 'no-source', atype: 1 }] }, + { source: 'no-id.example', uids: [{ id: '', atype: 1 }] }, + ]) + + pbjs.requestBids({ adUnits: [] } as any) + + const decoded = JSON.parse(atob(getTsEidsCookie()!)) + expect(decoded).toEqual([{ source: 'good.example', id: 'keep', atype: 1 }]) +}) ``` - [ ] **Step 8: Add test — takes first uid when multiple are present** @@ -210,24 +224,24 @@ Append: Append: ```ts - it('takes the first uid per source when multiple are present', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { - source: 'multi.example', - uids: [ - { id: 'first', atype: 1 }, - { id: 'second', atype: 2 }, - ], - }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const decoded = JSON.parse(atob(getTsEidsCookie()!)); - expect(decoded).toEqual([{ source: 'multi.example', id: 'first', atype: 1 }]); - }); +it('takes the first uid per source when multiple are present', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + mockGetUserIdsAsEids.mockReturnValue([ + { + source: 'multi.example', + uids: [ + { id: 'first', atype: 1 }, + { id: 'second', atype: 2 }, + ], + }, + ]) + + pbjs.requestBids({ adUnits: [] } as any) + + const decoded = JSON.parse(atob(getTsEidsCookie()!)) + expect(decoded).toEqual([{ source: 'multi.example', id: 'first', atype: 1 }]) +}) ``` - [ ] **Step 9: Add test — trims tail when payload exceeds 3072 bytes** @@ -235,31 +249,31 @@ Append: Append: ```ts - it('trims EIDs from the tail when the cookie payload would exceed 3072 bytes', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - - // Build ~20 entries each ~200 bytes → definitely exceeds 3072-byte cap - // once base64-encoded. - const big = Array.from({ length: 20 }, (_, i) => ({ - source: `source-${i}.example`, - uids: [{ id: 'x'.repeat(200) + String(i), atype: 3 }], - })); - mockGetUserIdsAsEids.mockReturnValue(big); - - pbjs.requestBids({ adUnits: [] } as any); - - const encoded = getTsEidsCookie(); - expect(encoded).toBeDefined(); - expect(encoded!.length).toBeLessThanOrEqual(3072); - - const decoded = JSON.parse(atob(encoded!)); - // At least one entry kept, strictly fewer than original count. - expect(decoded.length).toBeGreaterThan(0); - expect(decoded.length).toBeLessThan(big.length); - // Head of the list is preserved (trimming happens from the tail). - expect(decoded[0].source).toBe('source-0.example'); - }); +it('trims EIDs from the tail when the cookie payload would exceed 3072 bytes', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + + // Build ~20 entries each ~200 bytes → definitely exceeds 3072-byte cap + // once base64-encoded. + const big = Array.from({ length: 20 }, (_, i) => ({ + source: `source-${i}.example`, + uids: [{ id: 'x'.repeat(200) + String(i), atype: 3 }], + })) + mockGetUserIdsAsEids.mockReturnValue(big) + + pbjs.requestBids({ adUnits: [] } as any) + + const encoded = getTsEidsCookie() + expect(encoded).toBeDefined() + expect(encoded!.length).toBeLessThanOrEqual(3072) + + const decoded = JSON.parse(atob(encoded!)) + // At least one entry kept, strictly fewer than original count. + expect(decoded.length).toBeGreaterThan(0) + expect(decoded.length).toBeLessThan(big.length) + // Head of the list is preserved (trimming happens from the tail). + expect(decoded[0].source).toBe('source-0.example') +}) ``` - [ ] **Step 10: Add test — writes no cookie when a single entry alone exceeds the cap** @@ -267,19 +281,19 @@ Append: Append: ```ts - it('writes no cookie when a single entry alone exceeds the cap', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); +it('writes no cookie when a single entry alone exceeds the cap', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() - // Single entry large enough to blow past 3072 bytes after base64. - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, - ]); + // Single entry large enough to blow past 3072 bytes after base64. + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, + ]) - pbjs.requestBids({ adUnits: [] } as any); + pbjs.requestBids({ adUnits: [] } as any) - expect(getTsEidsCookie()).toBeUndefined(); - }); + expect(getTsEidsCookie()).toBeUndefined() +}) ``` - [ ] **Step 11: Add test — does not throw when getUserIdsAsEids is undefined** @@ -287,18 +301,18 @@ Append: This mirrors the pre-fix production state and guards against regressions in the defensive check at `index.ts:375`. Append: ```ts - it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - // Simulate a build that forgot the userId core module. - (mockPbjs as any).getUserIdsAsEids = undefined; - - expect(() => pbjs.requestBids({ adUnits: [] } as any)).not.toThrow(); - expect(getTsEidsCookie()).toBeUndefined(); - - // Restore for subsequent tests. - (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; - }); +it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + // Simulate a build that forgot the userId core module. + ;(mockPbjs as any).getUserIdsAsEids = undefined + + expect(() => pbjs.requestBids({ adUnits: [] } as any)).not.toThrow() + expect(getTsEidsCookie()).toBeUndefined() + + // Restore for subsequent tests. + ;(mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids +}) ``` - [ ] **Step 12: Add test — calls the original bidsBackHandler when one was supplied** @@ -306,15 +320,15 @@ This mirrors the pre-fix production state and guards against regressions in the Append: ```ts - it('calls the original bidsBackHandler after syncing EIDs', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - const originalHandler = vi.fn(); +it('calls the original bidsBackHandler after syncing EIDs', () => { + wireBidsBackHandler() + const pbjs = installPrebidNpm() + const originalHandler = vi.fn() - pbjs.requestBids({ adUnits: [], bidsBackHandler: originalHandler } as any); + pbjs.requestBids({ adUnits: [], bidsBackHandler: originalHandler } as any) - expect(originalHandler).toHaveBeenCalledTimes(1); - }); + expect(originalHandler).toHaveBeenCalledTimes(1) +}) ``` - [ ] **Step 13: Run the full new block and confirm all pass** @@ -343,46 +357,47 @@ git commit -m "Add Vitest coverage for Prebid ts-eids cookie sync" This is the substantive change. Add `vi.mock` stubs for the new modules first (so tests don't blow up when the imports are added), then add the imports. **Files:** + - Modify: `crates/js/lib/test/integrations/prebid/index.test.ts:44-47` - Modify: `crates/js/lib/src/integrations/prebid/index.ts:16-18` -- [ ] **Step 1: Add `vi.mock` stubs in the test file for all 16 new modules** +- [ ] **Step 1: Add `vi.mock` stubs in the test file for all 15 new modules** In `crates/js/lib/test/integrations/prebid/index.test.ts`, locate the existing block (around line 44-47): ```ts // Side-effect imports are no-ops in tests -vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})); -vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); -vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); +vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})) +vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})) +vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})) ``` -Replace it with (consent management mocks stay; add 16 new ones): +Replace it with (consent management mocks stay; add 15 new ones): ```ts // Side-effect imports are no-ops in tests -vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})); -vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); -vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); +vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})) +vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})) +vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})) // User ID Module core + submodules — no-op mocks so jsdom does not try to // execute the real Prebid code paths. -vi.mock('prebid.js/modules/userId.js', () => ({})); -vi.mock('prebid.js/modules/sharedIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/criteoIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/33acrossIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})); -vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/liveIntentIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})); -vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/lotamePanoramaIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/connectIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/merkleIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/pubCommonIdSystem.js', () => ({})); +vi.mock('prebid.js/modules/userId.js', () => ({})) +vi.mock('prebid.js/modules/sharedIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/criteoIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/33acrossIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})) +vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/liveIntentIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})) +vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/lotamePanoramaIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/connectIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/merkleIdSystem.js', () => ({})) +vi.mock('prebid.js/modules/pubCommonIdSystem.js', () => ({})) ``` - [ ] **Step 2: Write the failing regression-guard test** @@ -390,9 +405,9 @@ vi.mock('prebid.js/modules/pubCommonIdSystem.js', () => ({})); Append this `describe` block to `crates/js/lib/test/integrations/prebid/index.test.ts`: ```ts -import { readFileSync } from 'node:fs'; -import { fileURLToPath } from 'node:url'; -import { resolve } from 'node:path'; +import { readFileSync } from 'node:fs' +import { fileURLToPath } from 'node:url' +import { resolve } from 'node:path' describe('prebid/index.ts User ID Module imports (regression guard)', () => { const REQUIRED_IMPORTS = [ @@ -412,7 +427,7 @@ describe('prebid/index.ts User ID Module imports (regression guard)', () => { 'prebid.js/modules/connectIdSystem.js', 'prebid.js/modules/merkleIdSystem.js', 'prebid.js/modules/pubCommonIdSystem.js', - ]; + ] // Source-text check: these mocks make the runtime pbjs mock a no-op for the // User ID Module, so there is no way to assert `typeof getUserIdsAsEids === @@ -421,75 +436,80 @@ describe('prebid/index.ts User ID Module imports (regression guard)', () => { // import, which is the exact regression that motivated this work. // The package is ESM (`"type": "module"`), so `__dirname` is not defined — // resolve relative to this file via `import.meta.url`. - const THIS_DIR = fileURLToPath(new URL('.', import.meta.url)); - const SOURCE_PATH = resolve(THIS_DIR, '../../../src/integrations/prebid/index.ts'); - const source = readFileSync(SOURCE_PATH, 'utf8'); + const THIS_DIR = fileURLToPath(new URL('.', import.meta.url)) + const SOURCE_PATH = resolve( + THIS_DIR, + '../../../src/integrations/prebid/index.ts' + ) + const source = readFileSync(SOURCE_PATH, 'utf8') for (const module of REQUIRED_IMPORTS) { it(`statically imports ${module}`, () => { - const pattern = new RegExp(`import\\s+['"]${module.replace(/\./g, '\\.')}['"]`); - expect(source).toMatch(pattern); - }); + const pattern = new RegExp( + `import\\s+['"]${module.replace(/\./g, '\\.')}['"]` + ) + expect(source).toMatch(pattern) + }) } -}); +}) ``` -- [ ] **Step 3: Run the new block — expect 16 failures** +- [ ] **Step 3: Run the new block — expect 15 failures** Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "regression guard"` -Expected: FAIL — 16 failing assertions, one per expected import. This confirms the regression guard actually reads the source. +Expected: FAIL — 15 failing assertions, one per expected import. This confirms the regression guard actually reads the source. -- [ ] **Step 4: Add the 16 imports to `index.ts`** +- [ ] **Step 4: Add the 15 imports to `index.ts`** In `crates/js/lib/src/integrations/prebid/index.ts`, locate lines 16-18: ```ts -import 'prebid.js/modules/consentManagementTcf.js'; -import 'prebid.js/modules/consentManagementGpp.js'; -import 'prebid.js/modules/consentManagementUsp.js'; +import 'prebid.js/modules/consentManagementTcf.js' +import 'prebid.js/modules/consentManagementGpp.js' +import 'prebid.js/modules/consentManagementUsp.js' ``` Insert the User ID imports immediately after them, before the existing `// Client-side bid adapters` comment block. The resulting section must read: ```ts -import 'prebid.js/modules/consentManagementTcf.js'; -import 'prebid.js/modules/consentManagementGpp.js'; -import 'prebid.js/modules/consentManagementUsp.js'; +import 'prebid.js/modules/consentManagementTcf.js' +import 'prebid.js/modules/consentManagementGpp.js' +import 'prebid.js/modules/consentManagementUsp.js' // Prebid User ID Module — core + submodules. The core module exposes // `pbjs.getUserIdsAsEids`; submodules self-register at import time and // activate when the publisher's origin-side `pbjs.setConfig({ userSync: // { userIds: [...] } })` call runs during `processQueue()`. -import 'prebid.js/modules/userId.js'; +import 'prebid.js/modules/userId.js' // Zero-config / auto-populating submodules (resolve without publisher params). -import 'prebid.js/modules/sharedIdSystem.js'; -import 'prebid.js/modules/criteoIdSystem.js'; -import 'prebid.js/modules/33acrossIdSystem.js'; -import 'prebid.js/modules/pubProvidedIdSystem.js'; -import 'prebid.js/modules/quantcastIdSystem.js'; +import 'prebid.js/modules/sharedIdSystem.js' +import 'prebid.js/modules/criteoIdSystem.js' +import 'prebid.js/modules/33acrossIdSystem.js' +import 'prebid.js/modules/pubProvidedIdSystem.js' +import 'prebid.js/modules/quantcastIdSystem.js' // Param-based submodules — inert until publisher setConfig supplies params. -import 'prebid.js/modules/id5IdSystem.js'; -import 'prebid.js/modules/identityLinkIdSystem.js'; -import 'prebid.js/modules/liveIntentIdSystem.js'; -import 'prebid.js/modules/uid2IdSystem.js'; -import 'prebid.js/modules/euidIdSystem.js'; -import 'prebid.js/modules/intentIqIdSystem.js'; -import 'prebid.js/modules/lotamePanoramaIdSystem.js'; -import 'prebid.js/modules/connectIdSystem.js'; -import 'prebid.js/modules/merkleIdSystem.js'; +import 'prebid.js/modules/id5IdSystem.js' +import 'prebid.js/modules/identityLinkIdSystem.js' +import 'prebid.js/modules/liveIntentIdSystem.js' +import 'prebid.js/modules/uid2IdSystem.js' +import 'prebid.js/modules/euidIdSystem.js' +import 'prebid.js/modules/intentIqIdSystem.js' +import 'prebid.js/modules/lotamePanoramaIdSystem.js' +import 'prebid.js/modules/connectIdSystem.js' +import 'prebid.js/modules/merkleIdSystem.js' // Legacy / deprecated but still present in some publisher configs. -import 'prebid.js/modules/pubCommonIdSystem.js'; +import 'prebid.js/modules/pubCommonIdSystem.js' ``` - [ ] **Step 5: Run the regression-guard block — expect PASS** Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "regression guard"` -Expected: all 16 tests PASS. +Expected: all 15 tests PASS. - [ ] **Step 6: Run the full prebid test file — expect no regressions** @@ -501,7 +521,7 @@ Expected: all tests PASS (Task 1 tests + regression guards + all pre-existing te Run: `cd crates/js/lib && npx vitest run` -Expected: all PASS. If any unrelated test fails because it imports `./index.ts` transitively and the real Prebid modules are not mocked there, add matching `vi.mock` stubs at the top of that test file (copy the same 16 lines). Common suspects: any test under `crates/js/lib/test/integrations/prebid/` you may have added, or tests that import core modules which in turn import the prebid integration. At the time of writing, no other test file imports the prebid integration. +Expected: all PASS. If any unrelated test fails because it imports `./index.ts` transitively and the real Prebid modules are not mocked there, add matching `vi.mock` stubs at the top of that test file (copy the same 15 lines). Common suspects: any test under `crates/js/lib/test/integrations/prebid/` you may have added, or tests that import core modules which in turn import the prebid integration. At the time of writing, no other test file imports the prebid integration. - [ ] **Step 8: Build the JS bundles** diff --git a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md index bc0f3f669..45341a62c 100644 --- a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md +++ b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md @@ -67,12 +67,12 @@ the relevant params): - `prebid.js/modules/connectIdSystem.js` - `prebid.js/modules/merkleIdSystem.js` -**Legacy / compatibility:** +Total: 1 core + 14 submodules = 15 new imports. -- `prebid.js/modules/pubCommonIdSystem.js` — deprecated in favor of SharedID - but still present in some publisher configs. - -Total: 1 core + 15 submodules = 16 new imports. +> **Note (2026-04-16, during implementation):** `pubCommonIdSystem.js`, which +> was originally planned as a legacy/compatibility submodule, was removed from +> Prebid.js in 10.x (consolidated into `sharedIdSystem`). It is not importable +> from our pinned Prebid 10.26.0 and has been dropped from this plan. No changes to `installPrebidNpm`, no changes to the `bidsBackHandler` shim, no changes to `syncPrebidEidsCookie`. The existing cookie-writing path is already From 05d1700846cbc23744864c47e215a4f203dad1f8 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 12:45:32 -0500 Subject: [PATCH 13/22] Drop liveIntentIdSystem from Prebid bundle MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Prebid's liveIntentIdSystem.js uses a dynamic require() inside a build-flag-guarded branch that their gulp pipeline dead-codes via constant folding. esbuild leaves the require() in the output, causing ReferenceError: require is not defined at browser runtime. Remove from the bundle until we add an esbuild resolver plugin (or switch to Prebid's own build pipeline) — tracked as a follow-up in the design spec. --- .../js/lib/src/integrations/prebid/index.ts | 7 +++++- .../test/integrations/prebid/index.test.ts | 2 -- ...2026-04-16-prebid-user-id-module-design.md | 23 ++++++++++++++----- 3 files changed, 23 insertions(+), 9 deletions(-) diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index 99ac3fb82..7c77aa579 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -34,7 +34,12 @@ import 'prebid.js/modules/quantcastIdSystem.js'; // Param-based submodules — inert until publisher setConfig supplies params. import 'prebid.js/modules/id5IdSystem.js'; import 'prebid.js/modules/identityLinkIdSystem.js'; -import 'prebid.js/modules/liveIntentIdSystem.js'; +// NOTE: `liveIntentIdSystem.js` is intentionally not imported. Its upstream +// module uses a dynamic `require()` inside a build-flag-guarded branch that +// Prebid's own gulp pipeline dead-codes via constant folding; esbuild leaves +// the `require()` call in the bundle, which throws at browser runtime. Re- +// enabling it requires an esbuild resolver plugin (or switching to Prebid's +// own build pipeline). Tracked as a follow-up in the design spec. import 'prebid.js/modules/uid2IdSystem.js'; import 'prebid.js/modules/euidIdSystem.js'; import 'prebid.js/modules/intentIqIdSystem.js'; diff --git a/crates/js/lib/test/integrations/prebid/index.test.ts b/crates/js/lib/test/integrations/prebid/index.test.ts index 30156d9b7..7546183ba 100644 --- a/crates/js/lib/test/integrations/prebid/index.test.ts +++ b/crates/js/lib/test/integrations/prebid/index.test.ts @@ -66,7 +66,6 @@ vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})); vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})); vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})); vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/liveIntentIdSystem.js', () => ({})); vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})); vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})); vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})); @@ -1177,7 +1176,6 @@ describe('prebid/index.ts User ID Module imports (regression guard)', () => { 'prebid.js/modules/quantcastIdSystem.js', 'prebid.js/modules/id5IdSystem.js', 'prebid.js/modules/identityLinkIdSystem.js', - 'prebid.js/modules/liveIntentIdSystem.js', 'prebid.js/modules/uid2IdSystem.js', 'prebid.js/modules/euidIdSystem.js', 'prebid.js/modules/intentIqIdSystem.js', diff --git a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md index 45341a62c..2c59005ee 100644 --- a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md +++ b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md @@ -59,7 +59,6 @@ the relevant params): - `prebid.js/modules/id5IdSystem.js` - `prebid.js/modules/identityLinkIdSystem.js` -- `prebid.js/modules/liveIntentIdSystem.js` - `prebid.js/modules/uid2IdSystem.js` - `prebid.js/modules/euidIdSystem.js` - `prebid.js/modules/intentIqIdSystem.js` @@ -67,12 +66,19 @@ the relevant params): - `prebid.js/modules/connectIdSystem.js` - `prebid.js/modules/merkleIdSystem.js` -Total: 1 core + 14 submodules = 15 new imports. +Total: 1 core + 13 submodules = 14 new imports. -> **Note (2026-04-16, during implementation):** `pubCommonIdSystem.js`, which -> was originally planned as a legacy/compatibility submodule, was removed from -> Prebid.js in 10.x (consolidated into `sharedIdSystem`). It is not importable -> from our pinned Prebid 10.26.0 and has been dropped from this plan. +> **Notes (2026-04-16, during implementation):** +> +> - `pubCommonIdSystem.js`, originally planned as a legacy/compatibility +> submodule, was removed from Prebid.js in 10.x (consolidated into +> `sharedIdSystem`). Not importable from our pinned Prebid 10.26.0; dropped. +> - `liveIntentIdSystem.js` was dropped from the day-1 bundle. Its upstream +> module uses a dynamic `require()` inside a build-flag-guarded branch that +> Prebid's own gulp pipeline dead-codes via constant folding; esbuild leaves +> the `require()` call in the bundle, which throws `ReferenceError: require +> is not defined` at browser runtime. Re-enabling requires an esbuild +> resolver plugin (or Prebid's own build pipeline). Tracked as a follow-up. No changes to `installPrebidNpm`, no changes to the `bidsBackHandler` shim, no changes to `syncPrebidEidsCookie`. The existing cookie-writing path is already @@ -196,3 +202,8 @@ auction. Publishers without `userSync.userIds` configured see no change. 3. **Partner alignment tooling** — a startup-time check that warns when a bundled ID submodule has no matching `[[ec.partners]]` entry, or vice versa. +4. **Re-enable `liveIntentIdSystem.js`** — requires either an esbuild + resolver plugin that rewrites the dynamic `require('../libraries/ + liveIntentId/idSystem.js')` inside `loadModule()` to a static import, or + adopting Prebid's own gulp build pipeline for the vendored bundle. + Out-of-scope for the initial ship. From 3d59d3df803c39da16442a8b4d48674b52054bc3 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 15:47:05 -0500 Subject: [PATCH 14/22] Make Prebid User ID submodule set configurable at build time Introduces TSJS_PREBID_USER_IDS env var (mirroring TSJS_PREBID_ADAPTERS) to control which Prebid User ID submodules are bundled. The hardcoded imports in index.ts are replaced with a generated file written by build-all.mjs at build time, defaulting to the same 13-submodule set. - build-all.mjs: generatePrebidUserIds() validates names, denylists liveIntentIdSystem, and writes _user_ids.generated.ts. Existence check also probes dist/src/public/ to handle modules shipped as .ts in sources (sharedIdSystem). - index.ts: replaces 13 hardcoded submodule imports with import './_user_ids.generated' - _user_ids.generated.ts: committed default with all 13 submodules - Tests: updated mocks and regression guard; added 9 syncPrebidEidsCookie behavior tests - Docs: new "User ID Modules" section in prebid.md with TSJS_PREBID_USER_IDS usage; spec follow-up #1 marked complete --- crates/js/lib/build-all.mjs | 132 ++++++++++++++++++ .../prebid/_user_ids.generated.ts | 22 +++ .../js/lib/src/integrations/prebid/index.ts | 35 ++--- .../test/integrations/prebid/index.test.ts | 93 ++++++------ docs/guide/integrations/prebid.md | 37 +++++ ...2026-04-16-prebid-user-id-module-design.md | 7 +- 6 files changed, 251 insertions(+), 75 deletions(-) create mode 100644 crates/js/lib/src/integrations/prebid/_user_ids.generated.ts diff --git a/crates/js/lib/build-all.mjs b/crates/js/lib/build-all.mjs index cc5690e0c..b1a650b80 100644 --- a/crates/js/lib/build-all.mjs +++ b/crates/js/lib/build-all.mjs @@ -13,6 +13,12 @@ * names to include in the bundle (e.g. "rubicon,appnexus,openx"). * Each name must have a corresponding {name}BidAdapter.js module in * the prebid.js package. Default: "rubicon". + * TSJS_PREBID_USER_IDS — Comma-separated list of Prebid.js User ID + * submodule filenames (no `.js` extension) to include in the bundle + * (e.g. "sharedIdSystem,id5IdSystem,criteoIdSystem"). The `userId.js` + * core module is always included and is not configurable here. + * Default: the full ship-set of 13 submodules (see + * DEFAULT_PREBID_USER_IDS below). */ import fs from 'node:fs'; @@ -107,6 +113,132 @@ function generatePrebidAdapters() { generatePrebidAdapters(); +// --------------------------------------------------------------------------- +// Prebid User ID submodule generation +// --------------------------------------------------------------------------- + +/** + * Default set of Prebid User ID submodules bundled when TSJS_PREBID_USER_IDS + * is unset. Matches the set originally hardcoded in index.ts when User ID + * support first shipped. `userId.js` (the core module) is imported + * unconditionally by index.ts and is not in this list. + */ +const DEFAULT_PREBID_USER_IDS = [ + 'sharedIdSystem', + 'criteoIdSystem', + '33acrossIdSystem', + 'pubProvidedIdSystem', + 'quantcastIdSystem', + 'id5IdSystem', + 'identityLinkIdSystem', + 'uid2IdSystem', + 'euidIdSystem', + 'intentIqIdSystem', + 'lotamePanoramaIdSystem', + 'connectIdSystem', + 'merkleIdSystem', +].join(','); + +/** + * Modules known to be incompatible with the current esbuild pipeline. + * + * `liveIntentIdSystem` uses a dynamic `require()` inside a build-flag-guarded + * branch that Prebid's own gulp pipeline dead-codes via constant folding. + * esbuild leaves the `require()` in the bundle, which throws + * `ReferenceError: require is not defined` at browser runtime. See spec + * follow-up #4 in `docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md`. + */ +const PREBID_USER_ID_DENYLIST = new Set(['liveIntentIdSystem']); + +const USER_IDS_FILE = path.join( + integrationsDir, + 'prebid', + '_user_ids.generated.ts', +); + +/** + * Generate `_user_ids.generated.ts` with import statements for each User ID + * submodule listed in the TSJS_PREBID_USER_IDS environment variable. + * + * Invalid submodule names (those without a matching module in prebid.js) or + * known-broken modules in the denylist are logged and skipped. + */ +function generatePrebidUserIds() { + const raw = process.env.TSJS_PREBID_USER_IDS || DEFAULT_PREBID_USER_IDS; + const names = raw + .split(',') + .map((s) => s.trim()) + .filter(Boolean); + + if (names.length === 0) { + console.warn( + '[build-all] TSJS_PREBID_USER_IDS is empty, falling back to default set', + ); + names.push(...DEFAULT_PREBID_USER_IDS.split(',')); + } + + const modulesDir = path.join( + __dirname, + 'node_modules', + 'prebid.js', + 'modules', + ); + + const imports = []; + const includedNames = []; + for (const name of names) { + if (PREBID_USER_ID_DENYLIST.has(name)) { + console.error( + `[build-all] WARNING: Prebid User ID submodule "${name}" is on the ` + + `esbuild-incompatibility denylist and will not be bundled. See ` + + `docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md ` + + `follow-up #4.`, + ); + continue; + } + const moduleFile = `${name}.js`; + const modulePath = path.join(modulesDir, moduleFile); + // Some modules ship as .ts in modules/ but resolve via the exports map + // to dist/src/public/*.js (e.g. sharedIdSystem). Accept either form. + const distPath = path.join(__dirname, 'node_modules', 'prebid.js', 'dist', 'src', 'public', moduleFile); + if (!fs.existsSync(modulePath) && !fs.existsSync(distPath)) { + console.error( + `[build-all] WARNING: Prebid User ID submodule "${name}" not found (expected ${moduleFile}), skipping`, + ); + continue; + } + imports.push(`import 'prebid.js/modules/${moduleFile}';`); + includedNames.push(name); + } + + if (imports.length === 0) { + console.error( + '[build-all] WARNING: No valid Prebid User ID submodules found, ' + + 'bundle will resolve no EIDs even if publisher configures userSync.userIds', + ); + } + + const content = [ + '// Auto-generated by build-all.mjs — manual edits will be overwritten at build time.', + '//', + '// Controls which Prebid.js User ID submodules are included in the bundle.', + '// Set the TSJS_PREBID_USER_IDS environment variable to a comma-separated', + '// list of submodule filenames without the `.js` extension', + '// (e.g. "sharedIdSystem,id5IdSystem,criteoIdSystem") before building.', + '// The userId.js core module is always included via a static import in', + '// index.ts and is not configurable here.', + '', + ...imports, + '', + ].join('\n'); + + fs.writeFileSync(USER_IDS_FILE, content); + + console.log('[build-all] Prebid User ID submodules:', includedNames); +} + +generatePrebidUserIds(); + // --------------------------------------------------------------------------- // Clean dist directory diff --git a/crates/js/lib/src/integrations/prebid/_user_ids.generated.ts b/crates/js/lib/src/integrations/prebid/_user_ids.generated.ts new file mode 100644 index 000000000..9eb586ddd --- /dev/null +++ b/crates/js/lib/src/integrations/prebid/_user_ids.generated.ts @@ -0,0 +1,22 @@ +// Auto-generated by build-all.mjs — manual edits will be overwritten at build time. +// +// Controls which Prebid.js User ID submodules are included in the bundle. +// Set the TSJS_PREBID_USER_IDS environment variable to a comma-separated +// list of submodule filenames without the `.js` extension +// (e.g. "sharedIdSystem,id5IdSystem,criteoIdSystem") before building. +// The userId.js core module is always included via a static import in +// index.ts and is not configurable here. + +import 'prebid.js/modules/sharedIdSystem.js'; +import 'prebid.js/modules/criteoIdSystem.js'; +import 'prebid.js/modules/33acrossIdSystem.js'; +import 'prebid.js/modules/pubProvidedIdSystem.js'; +import 'prebid.js/modules/quantcastIdSystem.js'; +import 'prebid.js/modules/id5IdSystem.js'; +import 'prebid.js/modules/identityLinkIdSystem.js'; +import 'prebid.js/modules/uid2IdSystem.js'; +import 'prebid.js/modules/euidIdSystem.js'; +import 'prebid.js/modules/intentIqIdSystem.js'; +import 'prebid.js/modules/lotamePanoramaIdSystem.js'; +import 'prebid.js/modules/connectIdSystem.js'; +import 'prebid.js/modules/merkleIdSystem.js'; diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index 7c77aa579..7d8118912 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -18,34 +18,17 @@ import 'prebid.js/modules/consentManagementGpp.js'; import 'prebid.js/modules/consentManagementUsp.js'; import 'prebid.js/modules/userId.js'; -// Prebid User ID Module — core + submodules. The core module exposes -// `pbjs.getUserIdsAsEids`; submodules self-register at import time and -// activate when the publisher's origin-side `pbjs.setConfig({ userSync: -// { userIds: [...] } })` call runs during `processQueue()`. +// Prebid User ID Module core — always bundled. Exposes +// `pbjs.getUserIdsAsEids` and registers the submodule machinery that each +// ID submodule in `_user_ids.generated.ts` hooks into. ID submodules +// activate only when the publisher's origin-side `pbjs.setConfig({ +// userSync: { userIds: [...] } })` call runs during `processQueue()`. import 'prebid.js/modules/userId.js'; -// Zero-config / auto-populating submodules (resolve without publisher params). -import 'prebid.js/modules/sharedIdSystem.js'; -import 'prebid.js/modules/criteoIdSystem.js'; -import 'prebid.js/modules/33acrossIdSystem.js'; -import 'prebid.js/modules/pubProvidedIdSystem.js'; -import 'prebid.js/modules/quantcastIdSystem.js'; - -// Param-based submodules — inert until publisher setConfig supplies params. -import 'prebid.js/modules/id5IdSystem.js'; -import 'prebid.js/modules/identityLinkIdSystem.js'; -// NOTE: `liveIntentIdSystem.js` is intentionally not imported. Its upstream -// module uses a dynamic `require()` inside a build-flag-guarded branch that -// Prebid's own gulp pipeline dead-codes via constant folding; esbuild leaves -// the `require()` call in the bundle, which throws at browser runtime. Re- -// enabling it requires an esbuild resolver plugin (or switching to Prebid's -// own build pipeline). Tracked as a follow-up in the design spec. -import 'prebid.js/modules/uid2IdSystem.js'; -import 'prebid.js/modules/euidIdSystem.js'; -import 'prebid.js/modules/intentIqIdSystem.js'; -import 'prebid.js/modules/lotamePanoramaIdSystem.js'; -import 'prebid.js/modules/connectIdSystem.js'; -import 'prebid.js/modules/merkleIdSystem.js'; +// Prebid User ID submodules — self-register with the core on import. +// The set of submodules is controlled by the TSJS_PREBID_USER_IDS env var +// at build time. See _user_ids.generated.ts (written by build-all.mjs). +import './_user_ids.generated'; // Client-side bid adapters — self-register with prebid.js on import. // The set of adapters is controlled by the TSJS_PREBID_ADAPTERS env var at diff --git a/crates/js/lib/test/integrations/prebid/index.test.ts b/crates/js/lib/test/integrations/prebid/index.test.ts index 7546183ba..c4c80b0f4 100644 --- a/crates/js/lib/test/integrations/prebid/index.test.ts +++ b/crates/js/lib/test/integrations/prebid/index.test.ts @@ -56,25 +56,13 @@ vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); vi.mock('prebid.js/modules/userId.js', () => ({})); -// User ID Module core + submodules — no-op mocks so jsdom does not try to -// execute the real Prebid code paths. +// User ID Module core — no-op mock so jsdom does not try to execute the +// real Prebid code paths. vi.mock('prebid.js/modules/userId.js', () => ({})); -vi.mock('prebid.js/modules/sharedIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/criteoIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/33acrossIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})); -vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})); -vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/lotamePanoramaIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/connectIdSystem.js', () => ({})); -vi.mock('prebid.js/modules/merkleIdSystem.js', () => ({})); - -// Mock the build-generated adapter imports (no-op in tests) + +// Mock the build-generated adapter and User ID submodule imports (no-op in tests) vi.mock('../../../src/integrations/prebid/_adapters.generated', () => ({})); +vi.mock('../../../src/integrations/prebid/_user_ids.generated', () => ({})); import { collectBidders, @@ -1166,38 +1154,49 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { }); }); -describe('prebid/index.ts User ID Module imports (regression guard)', () => { - const REQUIRED_IMPORTS = [ - 'prebid.js/modules/userId.js', - 'prebid.js/modules/sharedIdSystem.js', - 'prebid.js/modules/criteoIdSystem.js', - 'prebid.js/modules/33acrossIdSystem.js', - 'prebid.js/modules/pubProvidedIdSystem.js', - 'prebid.js/modules/quantcastIdSystem.js', - 'prebid.js/modules/id5IdSystem.js', - 'prebid.js/modules/identityLinkIdSystem.js', - 'prebid.js/modules/uid2IdSystem.js', - 'prebid.js/modules/euidIdSystem.js', - 'prebid.js/modules/intentIqIdSystem.js', - 'prebid.js/modules/lotamePanoramaIdSystem.js', - 'prebid.js/modules/connectIdSystem.js', - 'prebid.js/modules/merkleIdSystem.js', +describe('prebid User ID Module imports (regression guard)', () => { + // `userId.js` is the core module — bundled unconditionally via a static + // import in index.ts, never operator-configurable. Guard it there. + const INDEX_PATH = resolve(process.cwd(), 'src/integrations/prebid/index.ts'); + const indexSource = readFileSync(INDEX_PATH, 'utf8'); + + it('index.ts statically imports the User ID core module', () => { + expect(indexSource).toMatch(/import\s+['"]prebid\.js\/modules\/userId\.js['"]/); + }); + + it('index.ts statically imports the generated User ID submodule file', () => { + expect(indexSource).toMatch(/import\s+['"]\.\/_user_ids\.generated['"]/); + }); + + // The submodule list is operator-controlled via TSJS_PREBID_USER_IDS, but + // the default ship-set must keep resolving without env var action. Read + // the generated file produced by `node build-all.mjs` with no env override + // and assert every default submodule is imported. If this file is missing, + // the developer has not yet run the build — skip with a clear message. + const GENERATED_PATH = resolve(process.cwd(), 'src/integrations/prebid/_user_ids.generated.ts'); + const DEFAULT_SUBMODULES = [ + 'sharedIdSystem', + 'criteoIdSystem', + '33acrossIdSystem', + 'pubProvidedIdSystem', + 'quantcastIdSystem', + 'id5IdSystem', + 'identityLinkIdSystem', + 'uid2IdSystem', + 'euidIdSystem', + 'intentIqIdSystem', + 'lotamePanoramaIdSystem', + 'connectIdSystem', + 'merkleIdSystem', ]; - // Source-text check: these mocks make the runtime pbjs mock a no-op for the - // User ID Module, so there is no way to assert `typeof getUserIdsAsEids === - // 'function'` at import time from within Vitest. Reading the source file - // directly is the most reliable way to catch accidental removal of an - // import, which is the exact regression that motivated this work. - // Vitest runs with `process.cwd()` set to the package root (`crates/js/lib`) - // where `package.json` lives. - const SOURCE_PATH = resolve(process.cwd(), 'src/integrations/prebid/index.ts'); - const source = readFileSync(SOURCE_PATH, 'utf8'); - - for (const module of REQUIRED_IMPORTS) { - it(`statically imports ${module}`, () => { - const pattern = new RegExp(`import\\s+['"]${module.replace(/\./g, '\\.')}['"]`); - expect(source).toMatch(pattern); + for (const name of DEFAULT_SUBMODULES) { + it(`_user_ids.generated.ts imports ${name}.js by default`, () => { + const generated = readFileSync(GENERATED_PATH, 'utf8'); + const pattern = new RegExp( + `import\\s+['"]prebid\\.js/modules/${name.replace(/\./g, '\\.')}\\.js['"]` + ); + expect(generated).toMatch(pattern); }); } }); diff --git a/docs/guide/integrations/prebid.md b/docs/guide/integrations/prebid.md index f7b7a910e..2be659fdc 100644 --- a/docs/guide/integrations/prebid.md +++ b/docs/guide/integrations/prebid.md @@ -223,6 +223,43 @@ The build script (`build-all.mjs`) validates that each adapter exists in `prebid Adding a new client-side bidder requires both a config change (`client_side_bidders`) **and** a rebuild with the adapter included in `TSJS_PREBID_ADAPTERS`. Without the adapter in the bundle, the bidder is silently dropped from both server-side and client-side auctions. ::: +## User ID Modules + +Prebid's User ID Module resolves cross-publisher identifiers (SharedID, ID5, LiveRamp RampID, UID2, etc.) and exposes them via `pbjs.getUserIdsAsEids()`. The TSJS Prebid integration bundles the core `userId.js` module and a configurable set of ID submodules. When the publisher's origin-side `pbjs.setConfig({ userSync: { userIds: [...] } })` call runs during `processQueue()`, each listed submodule activates and begins resolving its ID asynchronously. After each auction the shim writes the resolved EIDs to a `ts-eids` cookie, which the Rust backend ingests into the Edge Cookie identity graph. + +### How it works + +1. `userId.js` is statically imported in `index.ts` — always bundled, not operator-configurable. +2. The set of ID submodules is controlled by `TSJS_PREBID_USER_IDS` at build time and emitted into `_user_ids.generated.ts`. +3. Publishers retain full control of which submodules actually run — activation is driven by their own `pbjs.setConfig({ userSync: { userIds: [...] } })` on origin. Bundling a submodule without a matching publisher config entry is inert (but costs bundle size). +4. The `bidsBackHandler` shim calls `pbjs.getUserIdsAsEids()` after each auction and writes the resolved entries to the `ts-eids` cookie (base64-encoded JSON, 3072-byte cap with tail-trim). + +### Build-time submodule selection + +```bash +# Default: the full ship-set of 13 submodules +# (sharedIdSystem, criteoIdSystem, 33acrossIdSystem, pubProvidedIdSystem, +# quantcastIdSystem, id5IdSystem, identityLinkIdSystem, uid2IdSystem, +# euidIdSystem, intentIqIdSystem, lotamePanoramaIdSystem, connectIdSystem, +# merkleIdSystem) + +# Slim build — only SharedID and ID5 +TSJS_PREBID_USER_IDS=sharedIdSystem,id5IdSystem + +# Single submodule +TSJS_PREBID_USER_IDS=sharedIdSystem +``` + +Values are Prebid module filenames without the `.js` extension. The build script (`build-all.mjs`) validates that each exists in `prebid.js/modules/{name}.js` and generates `_user_ids.generated.ts` with the appropriate imports. Unknown names log a warning and are skipped. + +::: warning +`liveIntentIdSystem` is on a build-time denylist — its upstream module uses a dynamic `require()` that esbuild cannot statically resolve, throwing `ReferenceError: require is not defined` at browser runtime. Listing it in `TSJS_PREBID_USER_IDS` logs a warning and skips the module. +::: + +::: tip +Each bundled submodule bloats `tsjs-prebid.js`. If a publisher deployment only needs SharedID and ID5, set `TSJS_PREBID_USER_IDS` accordingly — the other ~100kb of dormant module code won't ship. +::: + ## Identity Forwarding Trusted Server uses a **hybrid EID forwarding model** for Prebid-routed auctions: diff --git a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md index 2c59005ee..2ef80a3eb 100644 --- a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md +++ b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md @@ -190,10 +190,13 @@ auction. Publishers without `userSync.userIds` configured see no change. ## Follow-ups -1. **Build-time configurability** — introduce `_user_ids.generated.ts` +1. ~~**Build-time configurability** — introduce `_user_ids.generated.ts` driven by a `TSJS_PREBID_USER_IDS` env var, mirroring the existing `TSJS_PREBID_ADAPTERS` / `_adapters.generated.ts` pattern. Allows - operators to slim the bundle per deployment. + operators to slim the bundle per deployment.~~ **Implemented + 2026-04-16** — see `docs/guide/integrations/prebid.md` "Build-time + submodule selection" and the `generatePrebidUserIds()` function in + `crates/js/lib/build-all.mjs`. 2. **Server-injected `userSync.userIds`** — extend `trusted-server.toml` with a `[[integrations.prebid.user_ids]]` array. Rust serializes into `window.__tsjs_prebid.userIds`. JS applies via `pbjs.setConfig` before From 67e55d5995722f8cd2f3fb878e2d1bdc1eda6057 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 16:10:08 -0500 Subject: [PATCH 15/22] Clear stale consent cookies and aggregate US GPP opt-outs --- .../js/lib/src/integrations/prebid/index.ts | 1 + .../lib/src/integrations/sourcepoint/index.ts | 22 +- .../test/integrations/prebid/index.test.ts | 53 ++++ .../integrations/sourcepoint/index.test.ts | 64 ++++- crates/trusted-server-core/src/consent/gpp.rs | 227 ++++++++++++++---- 5 files changed, 318 insertions(+), 49 deletions(-) diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index 7d8118912..a9a561de9 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -463,6 +463,7 @@ function fitAuctionEidsToCookie(eids: AuctionEid[]): AuctionEid[] | undefined { function syncPrebidEidsCookie(): void { try { if (typeof pbjs.getUserIdsAsEids !== 'function') { + clearPrebidEidsCookie(); return; } diff --git a/crates/js/lib/src/integrations/sourcepoint/index.ts b/crates/js/lib/src/integrations/sourcepoint/index.ts index 850659dcf..84295f8a5 100644 --- a/crates/js/lib/src/integrations/sourcepoint/index.ts +++ b/crates/js/lib/src/integrations/sourcepoint/index.ts @@ -1,6 +1,8 @@ import { log } from '../../core/log'; const SP_CONSENT_PREFIX = '_sp_user_consent_'; +const GPP_COOKIE_NAME = '__gpp'; +const GPP_SID_COOKIE_NAME = '__gpp_sid'; interface SourcepointGppData { gppString: string; @@ -20,10 +22,12 @@ function findSourcepointConsent(): SourcepointConsentPayload | null { if (!raw) continue; try { - return JSON.parse(raw) as SourcepointConsentPayload; + const payload = JSON.parse(raw) as SourcepointConsentPayload; + if (payload.gppData?.gppString) { + return payload; + } } catch { log.debug('sourcepoint: failed to parse localStorage value', { key }); - return null; } } return null; @@ -33,6 +37,10 @@ function writeCookie(name: string, value: string): void { document.cookie = `${name}=${value}; path=/; SameSite=Lax`; } +function clearCookie(name: string): void { + document.cookie = `${name}=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/; SameSite=Lax`; +} + /** * Reads Sourcepoint consent from localStorage and mirrors it into * `__gpp` and `__gpp_sid` cookies for Trusted Server to read. @@ -46,20 +54,26 @@ export function mirrorSourcepointConsent(): boolean { const payload = findSourcepointConsent(); if (!payload?.gppData) { + clearCookie(GPP_COOKIE_NAME); + clearCookie(GPP_SID_COOKIE_NAME); log.debug('sourcepoint: no GPP data found in localStorage'); return false; } const { gppString, applicableSections } = payload.gppData; if (!gppString) { + clearCookie(GPP_COOKIE_NAME); + clearCookie(GPP_SID_COOKIE_NAME); log.debug('sourcepoint: gppString is empty'); return false; } - writeCookie('__gpp', gppString); + writeCookie(GPP_COOKIE_NAME, gppString); if (Array.isArray(applicableSections) && applicableSections.length > 0) { - writeCookie('__gpp_sid', applicableSections.join(',')); + writeCookie(GPP_SID_COOKIE_NAME, applicableSections.join(',')); + } else { + clearCookie(GPP_SID_COOKIE_NAME); } log.info('sourcepoint: mirrored GPP consent to cookies', { diff --git a/crates/js/lib/test/integrations/prebid/index.test.ts b/crates/js/lib/test/integrations/prebid/index.test.ts index c4c80b0f4..60df059cd 100644 --- a/crates/js/lib/test/integrations/prebid/index.test.ts +++ b/crates/js/lib/test/integrations/prebid/index.test.ts @@ -1025,6 +1025,17 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { expect(getTsEidsCookie()).toBeUndefined(); }); + it('clears an existing cookie when getUserIdsAsEids returns empty array', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + document.cookie = 'ts-eids=stale; Path=/'; + mockGetUserIdsAsEids.mockReturnValue([]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); + it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { wireBidsBackHandler(); const pbjs = installPrebidNpm(); @@ -1130,6 +1141,34 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { expect(getTsEidsCookie()).toBeUndefined(); }); + it('clears an existing cookie when flattening yields no valid EIDs', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + document.cookie = 'ts-eids=stale; Path=/'; + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'missing-id.example', uids: [{ id: '', atype: 3 }] }, + { source: '', uids: [{ id: 'missing-source', atype: 3 }] }, + { source: 'empty-uids.example', uids: [] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); + + it('clears an existing cookie when a single oversized entry cannot be written', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + document.cookie = 'ts-eids=stale; Path=/'; + mockGetUserIdsAsEids.mockReturnValue([ + { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, + ]); + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + }); + it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { wireBidsBackHandler(); const pbjs = installPrebidNpm(); @@ -1143,6 +1182,20 @@ describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; }); + it('clears an existing cookie when getUserIdsAsEids is undefined', () => { + wireBidsBackHandler(); + const pbjs = installPrebidNpm(); + document.cookie = 'ts-eids=stale; Path=/'; + (mockPbjs as any).getUserIdsAsEids = undefined; + + pbjs.requestBids({ adUnits: [] } as any); + + expect(getTsEidsCookie()).toBeUndefined(); + + // Restore for subsequent tests. + (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; + }); + it('calls the original bidsBackHandler after syncing EIDs', () => { wireBidsBackHandler(); const pbjs = installPrebidNpm(); diff --git a/crates/js/lib/test/integrations/sourcepoint/index.test.ts b/crates/js/lib/test/integrations/sourcepoint/index.test.ts index 4eaf763f6..872585c32 100644 --- a/crates/js/lib/test/integrations/sourcepoint/index.test.ts +++ b/crates/js/lib/test/integrations/sourcepoint/index.test.ts @@ -3,16 +3,26 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest'; import { mirrorSourcepointConsent } from '../../../src/integrations/sourcepoint'; describe('integrations/sourcepoint', () => { - beforeEach(() => { - // Clear cookies and localStorage before each test. + function clearAllCookies(): void { document.cookie.split(';').forEach((c) => { const name = c.split('=')[0].trim(); if (name) document.cookie = `${name}=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/`; }); + } + + function getCookie(name: string): string | undefined { + const match = document.cookie.split('; ').find((c) => c.startsWith(`${name}=`)); + return match ? match.split('=').slice(1).join('=') : undefined; + } + + beforeEach(() => { + // Clear cookies and localStorage before each test. + clearAllCookies(); localStorage.clear(); }); afterEach(() => { + clearAllCookies(); localStorage.clear(); }); @@ -56,6 +66,18 @@ describe('integrations/sourcepoint', () => { expect(document.cookie).not.toContain('__gpp_sid='); }); + it('clears stale mirrored cookies when no valid Sourcepoint payload exists', () => { + document.cookie = '__gpp=stale-gpp; path=/'; + document.cookie = '__gpp_sid=7,8; path=/'; + localStorage.setItem('unrelated_key', 'value'); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(getCookie('__gpp')).toBeUndefined(); + expect(getCookie('__gpp_sid')).toBeUndefined(); + }); + it('returns false for malformed JSON in localStorage', () => { localStorage.setItem('_sp_user_consent_12345', 'not-json!!!'); @@ -65,6 +87,25 @@ describe('integrations/sourcepoint', () => { expect(document.cookie).not.toContain('__gpp='); }); + it('skips malformed entries when a later Sourcepoint key is valid', () => { + localStorage.setItem('_sp_user_consent_12345', 'not-json!!!'); + localStorage.setItem( + '_sp_user_consent_67890', + JSON.stringify({ + gppData: { + gppString: 'DBABLA~BVQqAAAAAgA.QA', + applicableSections: [7], + }, + }) + ); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(true); + expect(getCookie('__gpp')).toBe('DBABLA~BVQqAAAAAgA.QA'); + expect(getCookie('__gpp_sid')).toBe('7'); + }); + it('returns false when gppData is missing from payload', () => { localStorage.setItem('_sp_user_consent_12345', JSON.stringify({ otherField: true })); @@ -88,4 +129,23 @@ describe('integrations/sourcepoint', () => { expect(result).toBe(false); expect(document.cookie).not.toContain('__gpp='); }); + + it('clears stale __gpp_sid when the payload has no applicable sections', () => { + document.cookie = '__gpp_sid=7,8; path=/'; + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify({ + gppData: { + gppString: 'DBABLA~BVQqAAAAAgA.QA', + applicableSections: [], + }, + }) + ); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(true); + expect(getCookie('__gpp')).toBe('DBABLA~BVQqAAAAAgA.QA'); + expect(getCookie('__gpp_sid')).toBeUndefined(); + }); }); diff --git a/crates/trusted-server-core/src/consent/gpp.rs b/crates/trusted-server-core/src/consent/gpp.rs index d6e8902ed..cb704b029 100644 --- a/crates/trusted-server-core/src/consent/gpp.rs +++ b/crates/trusted-server-core/src/consent/gpp.rs @@ -111,56 +111,73 @@ fn decode_tcf_from_gpp(parsed: &iab_gpp::v1::GPPString) -> Option { /// 21=UsNj, 22=UsTn, 23=UsMn. const US_SECTION_ID_RANGE: std::ops::RangeInclusive = 7..=23; -/// Extracts the `sale_opt_out` signal from the first US section in a parsed -/// GPP string. +/// Extracts the `sale_opt_out` signal across all US sections in a parsed GPP +/// string. /// -/// Iterates through section IDs looking for any in the US range (7–23). -/// For the first match, decodes the section and extracts `sale_opt_out`. +/// Iterates through section IDs looking for any in the US range (7–23), +/// decodes each US section, and aggregates the result conservatively: /// -/// Returns `Some(true)` if the user opted out of sale, `Some(false)` if they -/// did not, or `None` if no US section is present. +/// - `Some(true)` if any decodable US section says the user opted out of sale +/// - `Some(false)` if at least one decodable US section says they did not opt +/// out and none say they opted out +/// - `None` if no US section is present or no decodable US section yields a +/// usable `sale_opt_out` signal fn decode_us_sale_opt_out(parsed: &iab_gpp::v1::GPPString) -> Option { - use iab_gpp::sections::us_common::OptOut; - use iab_gpp::sections::Section; + let mut saw_not_opted_out = false; - let us_section_id = parsed + for us_section_id in parsed .section_ids() - .find(|id| US_SECTION_ID_RANGE.contains(&(**id as u16)))?; - - match parsed.decode_section(*us_section_id) { - Ok(section) => { - let sale_opt_out = match §ion { - Section::UsNat(s) => match &s.core { - iab_gpp::sections::usnat::Core::V1(c) => &c.sale_opt_out, - iab_gpp::sections::usnat::Core::V2(c) => &c.sale_opt_out, - _ => return None, - }, - Section::UsCa(s) => &s.core.sale_opt_out, - Section::UsVa(s) => &s.core.sale_opt_out, - Section::UsCo(s) => &s.core.sale_opt_out, - Section::UsUt(s) => &s.core.sale_opt_out, - Section::UsCt(s) => &s.core.sale_opt_out, - Section::UsFl(s) => &s.core.sale_opt_out, - Section::UsMt(s) => &s.core.sale_opt_out, - Section::UsOr(s) => &s.core.sale_opt_out, - Section::UsTx(s) => &s.core.sale_opt_out, - Section::UsDe(s) => &s.core.sale_opt_out, - Section::UsIa(s) => &s.core.sale_opt_out, - Section::UsNe(s) => &s.core.sale_opt_out, - Section::UsNh(s) => &s.core.sale_opt_out, - Section::UsNj(s) => &s.core.sale_opt_out, - Section::UsTn(s) => &s.core.sale_opt_out, - Section::UsMn(s) => &s.core.sale_opt_out, - // Non-US sections — should not reach here given the ID filter. - _ => return None, - }; - Some(*sale_opt_out == OptOut::OptedOut) - } - Err(e) => { - log::warn!("Failed to decode US GPP section {us_section_id}: {e}"); - None + .filter(|id| US_SECTION_ID_RANGE.contains(&(**id as u16))) + { + match parsed.decode_section(*us_section_id) { + Ok(section) => match us_sale_opt_out_from_section(§ion) { + Some(true) => return Some(true), + Some(false) => saw_not_opted_out = true, + None => {} + }, + Err(e) => { + log::warn!("Failed to decode US GPP section {us_section_id}: {e}"); + } } } + + if saw_not_opted_out { + Some(false) + } else { + None + } +} + +fn us_sale_opt_out_from_section(section: &iab_gpp::sections::Section) -> Option { + use iab_gpp::sections::us_common::OptOut; + use iab_gpp::sections::Section; + + let sale_opt_out = match section { + Section::UsNat(s) => match &s.core { + iab_gpp::sections::usnat::Core::V1(c) => &c.sale_opt_out, + iab_gpp::sections::usnat::Core::V2(c) => &c.sale_opt_out, + _ => return None, + }, + Section::UsCa(s) => &s.core.sale_opt_out, + Section::UsVa(s) => &s.core.sale_opt_out, + Section::UsCo(s) => &s.core.sale_opt_out, + Section::UsUt(s) => &s.core.sale_opt_out, + Section::UsCt(s) => &s.core.sale_opt_out, + Section::UsFl(s) => &s.core.sale_opt_out, + Section::UsMt(s) => &s.core.sale_opt_out, + Section::UsOr(s) => &s.core.sale_opt_out, + Section::UsTx(s) => &s.core.sale_opt_out, + Section::UsDe(s) => &s.core.sale_opt_out, + Section::UsIa(s) => &s.core.sale_opt_out, + Section::UsNe(s) => &s.core.sale_opt_out, + Section::UsNh(s) => &s.core.sale_opt_out, + Section::UsNj(s) => &s.core.sale_opt_out, + Section::UsTn(s) => &s.core.sale_opt_out, + Section::UsMn(s) => &s.core.sale_opt_out, + _ => return None, + }; + + Some(*sale_opt_out == OptOut::OptedOut) } /// Parses a `__gpp_sid` cookie value into a vector of section IDs. @@ -320,6 +337,70 @@ mod tests { } } + fn encode_fibonacci_integer(mut value: u16) -> String { + let mut fibs = vec![1_u16]; + let mut next = 2_u16; + while next <= value { + fibs.push(next); + next = if fibs.len() == 1 { + 2 + } else { + fibs[fibs.len() - 1] + fibs[fibs.len() - 2] + }; + } + + let mut bits = vec![false; fibs.len()]; + for (idx, fib) in fibs.iter().enumerate().rev() { + if *fib <= value { + value -= *fib; + bits[idx] = true; + } + } + bits.push(true); + + bits.into_iter() + .map(|bit| if bit { '1' } else { '0' }) + .collect() + } + + fn encode_header(section_ids: &[u16]) -> String { + const BASE64_URL: &[u8; 64] = + b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"; + + let mut bits = String::from("000011000001"); + bits.push_str(&format!("{:012b}", section_ids.len())); + + let mut previous = 0_u16; + for §ion_id in section_ids { + bits.push('0'); + bits.push_str(&encode_fibonacci_integer(section_id - previous)); + previous = section_id; + } + + while bits.len() % 6 != 0 { + bits.push('0'); + } + + bits.as_bytes() + .chunks(6) + .map(|chunk| { + let value = u8::from_str_radix( + core::str::from_utf8(chunk).expect("should encode header bits as utf8"), + 2, + ) + .expect("should parse 6-bit chunk"); + char::from(BASE64_URL[value as usize]) + }) + .collect() + } + + fn gpp_with_sections(sections: &[(u16, &str)]) -> String { + let ids = sections.iter().map(|(id, _)| *id).collect::>(); + let header = encode_header(&ids); + let section_payloads = sections.iter().map(|(_, raw)| *raw).collect::>(); + format!("{header}~{}", section_payloads.join("~")) + } + #[test] fn no_us_section_returns_none() { let result = decode_gpp_string(GPP_TCF_AND_USP).expect("should decode GPP"); @@ -328,4 +409,64 @@ mod tests { "should return None when no US section (7-23) is present" ); } + + #[test] + fn later_us_section_opt_out_overrides_earlier_non_opt_out() { + let gpp = gpp_with_sections(&[(7, "BVQqAAAAAgA.QA"), (9, "BVVVVVVVVWA.AA")]); + + let result = decode_gpp_string(&gpp).expect("should decode multi-section US GPP"); + + assert_eq!( + result.us_sale_opt_out, + Some(true), + "should treat any later decodable opt-out as authoritative" + ); + } + + #[test] + fn multiple_us_sections_without_opt_out_return_false() { + let gpp = gpp_with_sections(&[(7, "BVQqAAAAAgA.QA"), (9, "BVgVVVVVVWA.AA")]); + + let result = decode_gpp_string(&gpp).expect("should decode multi-section US GPP"); + + assert_eq!( + result.us_sale_opt_out, + Some(false), + "should return false when decodable US sections consistently do not opt out" + ); + } + + #[test] + fn valid_opt_out_wins_even_if_another_us_section_is_undecodable() { + let gpp = gpp_with_sections(&[(7, "BVQqAAAAAgA.QA"), (9, "not-a-valid-usva-section")]); + + let result = decode_gpp_string(&gpp).expect("should decode GPP header with raw sections"); + + assert_eq!( + result.us_sale_opt_out, + Some(false), + "should keep a valid non-opt-out signal even when another US section fails to decode" + ); + + let gpp = gpp_with_sections(&[(7, "not-a-valid-usnat-section"), (9, "BVVVVVVVVWA.AA")]); + let result = decode_gpp_string(&gpp).expect("should decode GPP header with raw sections"); + + assert_eq!( + result.us_sale_opt_out, + Some(true), + "should let a valid opt-out win even when another US section fails to decode" + ); + } + + #[test] + fn only_undecodable_us_sections_return_none() { + let gpp = gpp_with_sections(&[(7, "not-a-valid-usnat-section"), (9, "also-invalid")]); + + let result = decode_gpp_string(&gpp).expect("should decode GPP header with raw sections"); + + assert_eq!( + result.us_sale_opt_out, None, + "should return None when no decodable US section yields sale_opt_out" + ); + } } From f956e8ad7ea90510cb16fa02a2b560a62d41dcf6 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 16 Apr 2026 16:46:49 -0500 Subject: [PATCH 16/22] Add Secure flag and Max-Age to Sourcepoint GPP cookies __gpp and __gpp_sid are read by the Rust server over HTTPS; they must be Secure. Also sets Max-Age=86400 (matching ts-eids) so stale consent state doesn't outlast the session, and replaces the legacy expires= deletion pattern with Max-Age=0. --- crates/js/lib/src/integrations/sourcepoint/index.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/crates/js/lib/src/integrations/sourcepoint/index.ts b/crates/js/lib/src/integrations/sourcepoint/index.ts index 84295f8a5..e7024b625 100644 --- a/crates/js/lib/src/integrations/sourcepoint/index.ts +++ b/crates/js/lib/src/integrations/sourcepoint/index.ts @@ -33,12 +33,14 @@ function findSourcepointConsent(): SourcepointConsentPayload | null { return null; } +const GPP_COOKIE_MAX_AGE = 86400; + function writeCookie(name: string, value: string): void { - document.cookie = `${name}=${value}; path=/; SameSite=Lax`; + document.cookie = `${name}=${value}; path=/; Secure; SameSite=Lax; Max-Age=${GPP_COOKIE_MAX_AGE}`; } function clearCookie(name: string): void { - document.cookie = `${name}=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/; SameSite=Lax`; + document.cookie = `${name}=; path=/; Secure; SameSite=Lax; Max-Age=0`; } /** From 08440a647c741205c242df341e06803e9f73570c Mon Sep 17 00:00:00 2001 From: Christian Date: Fri, 17 Apr 2026 16:12:35 -0500 Subject: [PATCH 17/22] support ec partners map for env overrides --- crates/trusted-server-core/src/settings.rs | 212 ++++++++++++++++++--- 1 file changed, 188 insertions(+), 24 deletions(-) diff --git a/crates/trusted-server-core/src/settings.rs b/crates/trusted-server-core/src/settings.rs index a75eb2c84..ec9d9ef9d 100644 --- a/crates/trusted-server-core/src/settings.rs +++ b/crates/trusted-server-core/src/settings.rs @@ -98,6 +98,27 @@ pub trait IntegrationConfig: DeserializeOwned + Validate { fn is_enabled(&self) -> bool; } +fn normalize_env_value(value: JsonValue) -> JsonValue { + match value { + JsonValue::Object(map) => JsonValue::Object( + map.into_iter() + .map(|(key, val)| (key, normalize_env_value(val))) + .collect(), + ), + JsonValue::Array(items) => { + JsonValue::Array(items.into_iter().map(normalize_env_value).collect()) + } + JsonValue::String(raw) => { + if let Ok(parsed) = serde_json::from_str::(&raw) { + parsed + } else { + JsonValue::String(raw) + } + } + other => other, + } +} + impl IntegrationSettings { /// Inserts a configuration value for an integration. /// @@ -121,34 +142,13 @@ impl IntegrationSettings { Ok(()) } - fn normalize_env_value(value: JsonValue) -> JsonValue { - match value { - JsonValue::Object(map) => JsonValue::Object( - map.into_iter() - .map(|(key, val)| (key, Self::normalize_env_value(val))) - .collect(), - ), - JsonValue::Array(items) => { - JsonValue::Array(items.into_iter().map(Self::normalize_env_value).collect()) - } - JsonValue::String(raw) => { - if let Ok(parsed) = serde_json::from_str::(&raw) { - parsed - } else { - JsonValue::String(raw) - } - } - other => other, - } - } - /// Normalizes all entries in place, converting JSON-encoded strings from /// environment variables into their proper typed representations. /// Called eagerly after deserialization so that TOML serialization in /// build.rs preserves correct types. pub fn normalize(&mut self) { for value in self.entries.values_mut() { - *value = Self::normalize_env_value(value.clone()); + *value = normalize_env_value(value.clone()); } } @@ -959,7 +959,9 @@ where match v { JsonValue::Array(arr) => arr .into_iter() - .map(|item| serde_json::from_value(item).map_err(serde::de::Error::custom)) + .map(|item| { + serde_json::from_value(normalize_env_value(item)).map_err(serde::de::Error::custom) + }) .collect(), JsonValue::Object(map) => { let mut items: Vec<(usize, T)> = Vec::with_capacity(map.len()); @@ -967,7 +969,8 @@ where let idx = k.parse::().map_err(|_| { serde::de::Error::custom(format!("Invalid index '{}' in map for Vec field", k)) })?; - let parsed: T = serde_json::from_value(val).map_err(serde::de::Error::custom)?; + let parsed: T = serde_json::from_value(normalize_env_value(val)) + .map_err(serde::de::Error::custom)?; items.push((idx, parsed)); } items.sort_by_key(|(idx, _)| *idx); @@ -1669,6 +1672,167 @@ mod tests { ); } + #[test] + fn test_ec_partners_override_with_indexed_env() { + let toml_str = crate_test_settings_str(); + + let origin_key = format!( + "{}{}PUBLISHER{}ORIGIN_URL", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner0_id_key = format!( + "{}{}EC{}PARTNERS{}0{}ID", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner0_name_key = format!( + "{}{}EC{}PARTNERS{}0{}NAME", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner0_source_domain_key = format!( + "{}{}EC{}PARTNERS{}0{}SOURCE_DOMAIN", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner0_openrtb_atype_key = format!( + "{}{}EC{}PARTNERS{}0{}OPENRTB_ATYPE", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner0_bidstream_enabled_key = format!( + "{}{}EC{}PARTNERS{}0{}BIDSTREAM_ENABLED", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner0_api_token_key = format!( + "{}{}EC{}PARTNERS{}0{}API_TOKEN", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner1_id_key = format!( + "{}{}EC{}PARTNERS{}1{}ID", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner1_name_key = format!( + "{}{}EC{}PARTNERS{}1{}NAME", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner1_source_domain_key = format!( + "{}{}EC{}PARTNERS{}1{}SOURCE_DOMAIN", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner1_openrtb_atype_key = format!( + "{}{}EC{}PARTNERS{}1{}OPENRTB_ATYPE", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner1_bidstream_enabled_key = format!( + "{}{}EC{}PARTNERS{}1{}BIDSTREAM_ENABLED", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + let partner1_api_token_key = format!( + "{}{}EC{}PARTNERS{}1{}API_TOKEN", + ENVIRONMENT_VARIABLE_PREFIX, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR, + ENVIRONMENT_VARIABLE_SEPARATOR + ); + + temp_env::with_vars( + [ + (origin_key, Some("https://origin.test-publisher.com")), + (partner0_id_key, Some("criteo")), + (partner0_name_key, Some("Criteo")), + (partner0_source_domain_key, Some("criteo.com")), + (partner0_openrtb_atype_key, Some("1")), + (partner0_bidstream_enabled_key, Some("true")), + (partner0_api_token_key, Some("criteo-test-token")), + (partner1_id_key, Some("uidapi")), + (partner1_name_key, Some("UID2")), + (partner1_source_domain_key, Some("uidapi.com")), + (partner1_openrtb_atype_key, Some("3")), + (partner1_bidstream_enabled_key, Some("true")), + (partner1_api_token_key, Some("uidapi-test-token")), + ], + || { + let settings = Settings::from_toml_and_env(&toml_str) + .expect("should parse EC partners from indexed env override"); + + assert_eq!( + settings.ec.partners.len(), + 2, + "should override partner list" + ); + assert_eq!(settings.ec.partners[0].id, "criteo"); + assert_eq!(settings.ec.partners[0].name, "Criteo"); + assert_eq!(settings.ec.partners[0].source_domain, "criteo.com"); + assert_eq!(settings.ec.partners[0].openrtb_atype, 1); + assert!( + settings.ec.partners[0].bidstream_enabled, + "should enable bidstream for partner 0" + ); + assert_eq!( + settings.ec.partners[0].api_token.expose(), + "criteo-test-token" + ); + + assert_eq!(settings.ec.partners[1].id, "uidapi"); + assert_eq!(settings.ec.partners[1].name, "UID2"); + assert_eq!(settings.ec.partners[1].source_domain, "uidapi.com"); + assert_eq!(settings.ec.partners[1].openrtb_atype, 3); + assert!( + settings.ec.partners[1].bidstream_enabled, + "should enable bidstream for partner 1" + ); + assert_eq!( + settings.ec.partners[1].api_token.expose(), + "uidapi-test-token" + ); + }, + ); + } + #[test] fn test_settings_extra_fields() { let toml_str = crate_test_settings_str() + "\nhello = 1"; From 8e3ff806b862609b446a9f36fddf2d288e0fcf98 Mon Sep 17 00:00:00 2001 From: Christian Date: Tue, 21 Apr 2026 08:45:18 -0500 Subject: [PATCH 18/22] Scope Sourcepoint consent PR and address review feedback --- crates/js/lib/build-all.mjs | 132 ---- .../js/lib/src/integrations/prebid/index.ts | 7 - .../lib/src/integrations/sourcepoint/index.ts | 8 +- .../test/integrations/prebid/index.test.ts | 284 +-------- .../integrations/sourcepoint/index.test.ts | 4 +- crates/trusted-server-core/src/consent/gpp.rs | 1 + .../src/integrations/registry.rs | 14 +- .../src/platform/test_support.rs | 19 - crates/trusted-server-core/src/settings.rs | 212 +------ .../plans/2026-04-16-prebid-user-id-module.md | 599 ------------------ ...26-04-15-sourcepoint-gpp-consent-design.md | 8 +- ...2026-04-16-prebid-user-id-module-design.md | 212 ------- 12 files changed, 49 insertions(+), 1451 deletions(-) delete mode 100644 docs/superpowers/plans/2026-04-16-prebid-user-id-module.md delete mode 100644 docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md diff --git a/crates/js/lib/build-all.mjs b/crates/js/lib/build-all.mjs index b1a650b80..cc5690e0c 100644 --- a/crates/js/lib/build-all.mjs +++ b/crates/js/lib/build-all.mjs @@ -13,12 +13,6 @@ * names to include in the bundle (e.g. "rubicon,appnexus,openx"). * Each name must have a corresponding {name}BidAdapter.js module in * the prebid.js package. Default: "rubicon". - * TSJS_PREBID_USER_IDS — Comma-separated list of Prebid.js User ID - * submodule filenames (no `.js` extension) to include in the bundle - * (e.g. "sharedIdSystem,id5IdSystem,criteoIdSystem"). The `userId.js` - * core module is always included and is not configurable here. - * Default: the full ship-set of 13 submodules (see - * DEFAULT_PREBID_USER_IDS below). */ import fs from 'node:fs'; @@ -113,132 +107,6 @@ function generatePrebidAdapters() { generatePrebidAdapters(); -// --------------------------------------------------------------------------- -// Prebid User ID submodule generation -// --------------------------------------------------------------------------- - -/** - * Default set of Prebid User ID submodules bundled when TSJS_PREBID_USER_IDS - * is unset. Matches the set originally hardcoded in index.ts when User ID - * support first shipped. `userId.js` (the core module) is imported - * unconditionally by index.ts and is not in this list. - */ -const DEFAULT_PREBID_USER_IDS = [ - 'sharedIdSystem', - 'criteoIdSystem', - '33acrossIdSystem', - 'pubProvidedIdSystem', - 'quantcastIdSystem', - 'id5IdSystem', - 'identityLinkIdSystem', - 'uid2IdSystem', - 'euidIdSystem', - 'intentIqIdSystem', - 'lotamePanoramaIdSystem', - 'connectIdSystem', - 'merkleIdSystem', -].join(','); - -/** - * Modules known to be incompatible with the current esbuild pipeline. - * - * `liveIntentIdSystem` uses a dynamic `require()` inside a build-flag-guarded - * branch that Prebid's own gulp pipeline dead-codes via constant folding. - * esbuild leaves the `require()` in the bundle, which throws - * `ReferenceError: require is not defined` at browser runtime. See spec - * follow-up #4 in `docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md`. - */ -const PREBID_USER_ID_DENYLIST = new Set(['liveIntentIdSystem']); - -const USER_IDS_FILE = path.join( - integrationsDir, - 'prebid', - '_user_ids.generated.ts', -); - -/** - * Generate `_user_ids.generated.ts` with import statements for each User ID - * submodule listed in the TSJS_PREBID_USER_IDS environment variable. - * - * Invalid submodule names (those without a matching module in prebid.js) or - * known-broken modules in the denylist are logged and skipped. - */ -function generatePrebidUserIds() { - const raw = process.env.TSJS_PREBID_USER_IDS || DEFAULT_PREBID_USER_IDS; - const names = raw - .split(',') - .map((s) => s.trim()) - .filter(Boolean); - - if (names.length === 0) { - console.warn( - '[build-all] TSJS_PREBID_USER_IDS is empty, falling back to default set', - ); - names.push(...DEFAULT_PREBID_USER_IDS.split(',')); - } - - const modulesDir = path.join( - __dirname, - 'node_modules', - 'prebid.js', - 'modules', - ); - - const imports = []; - const includedNames = []; - for (const name of names) { - if (PREBID_USER_ID_DENYLIST.has(name)) { - console.error( - `[build-all] WARNING: Prebid User ID submodule "${name}" is on the ` + - `esbuild-incompatibility denylist and will not be bundled. See ` + - `docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md ` + - `follow-up #4.`, - ); - continue; - } - const moduleFile = `${name}.js`; - const modulePath = path.join(modulesDir, moduleFile); - // Some modules ship as .ts in modules/ but resolve via the exports map - // to dist/src/public/*.js (e.g. sharedIdSystem). Accept either form. - const distPath = path.join(__dirname, 'node_modules', 'prebid.js', 'dist', 'src', 'public', moduleFile); - if (!fs.existsSync(modulePath) && !fs.existsSync(distPath)) { - console.error( - `[build-all] WARNING: Prebid User ID submodule "${name}" not found (expected ${moduleFile}), skipping`, - ); - continue; - } - imports.push(`import 'prebid.js/modules/${moduleFile}';`); - includedNames.push(name); - } - - if (imports.length === 0) { - console.error( - '[build-all] WARNING: No valid Prebid User ID submodules found, ' + - 'bundle will resolve no EIDs even if publisher configures userSync.userIds', - ); - } - - const content = [ - '// Auto-generated by build-all.mjs — manual edits will be overwritten at build time.', - '//', - '// Controls which Prebid.js User ID submodules are included in the bundle.', - '// Set the TSJS_PREBID_USER_IDS environment variable to a comma-separated', - '// list of submodule filenames without the `.js` extension', - '// (e.g. "sharedIdSystem,id5IdSystem,criteoIdSystem") before building.', - '// The userId.js core module is always included via a static import in', - '// index.ts and is not configurable here.', - '', - ...imports, - '', - ].join('\n'); - - fs.writeFileSync(USER_IDS_FILE, content); - - console.log('[build-all] Prebid User ID submodules:', includedNames); -} - -generatePrebidUserIds(); - // --------------------------------------------------------------------------- // Clean dist directory diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index a9a561de9..53255d837 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -18,13 +18,6 @@ import 'prebid.js/modules/consentManagementGpp.js'; import 'prebid.js/modules/consentManagementUsp.js'; import 'prebid.js/modules/userId.js'; -// Prebid User ID Module core — always bundled. Exposes -// `pbjs.getUserIdsAsEids` and registers the submodule machinery that each -// ID submodule in `_user_ids.generated.ts` hooks into. ID submodules -// activate only when the publisher's origin-side `pbjs.setConfig({ -// userSync: { userIds: [...] } })` call runs during `processQueue()`. -import 'prebid.js/modules/userId.js'; - // Prebid User ID submodules — self-register with the core on import. // The set of submodules is controlled by the TSJS_PREBID_USER_IDS env var // at build time. See _user_ids.generated.ts (written by build-all.mjs). diff --git a/crates/js/lib/src/integrations/sourcepoint/index.ts b/crates/js/lib/src/integrations/sourcepoint/index.ts index e7024b625..1e3d7cea2 100644 --- a/crates/js/lib/src/integrations/sourcepoint/index.ts +++ b/crates/js/lib/src/integrations/sourcepoint/index.ts @@ -14,6 +14,8 @@ interface SourcepointConsentPayload { } function findSourcepointConsent(): SourcepointConsentPayload | null { + // Sourcepoint stores one consent payload per property under `_sp_user_consent_*`. + // We intentionally take the first valid match and mirror that origin-scoped payload. for (let i = 0; i < localStorage.length; i++) { const key = localStorage.key(i); if (!key?.startsWith(SP_CONSENT_PREFIX)) continue; @@ -33,13 +35,13 @@ function findSourcepointConsent(): SourcepointConsentPayload | null { return null; } -const GPP_COOKIE_MAX_AGE = 86400; - function writeCookie(name: string, value: string): void { - document.cookie = `${name}=${value}; path=/; Secure; SameSite=Lax; Max-Age=${GPP_COOKIE_MAX_AGE}`; + document.cookie = `${name}=${value}; path=/; Secure; SameSite=Lax`; } function clearCookie(name: string): void { + // Trusted Server is the only intended writer for these mirrored cookies, so + // clearing the origin-scoped cookie is sufficient for this integration. document.cookie = `${name}=; path=/; Secure; SameSite=Lax; Max-Age=0`; } diff --git a/crates/js/lib/test/integrations/prebid/index.test.ts b/crates/js/lib/test/integrations/prebid/index.test.ts index 60df059cd..470348613 100644 --- a/crates/js/lib/test/integrations/prebid/index.test.ts +++ b/crates/js/lib/test/integrations/prebid/index.test.ts @@ -1,6 +1,3 @@ -import { readFileSync } from 'node:fs'; -import { resolve } from 'node:path'; - import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; // Define mocks using vi.hoisted so they're available inside vi.mock factories @@ -56,13 +53,8 @@ vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})); vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})); vi.mock('prebid.js/modules/userId.js', () => ({})); -// User ID Module core — no-op mock so jsdom does not try to execute the -// real Prebid code paths. -vi.mock('prebid.js/modules/userId.js', () => ({})); - -// Mock the build-generated adapter and User ID submodule imports (no-op in tests) +// Mock the build-generated adapter imports (no-op in tests) vi.mock('../../../src/integrations/prebid/_adapters.generated', () => ({})); -vi.mock('../../../src/integrations/prebid/_user_ids.generated', () => ({})); import { collectBidders, @@ -979,277 +971,3 @@ describe('prebid/client-side bidders', () => { errorSpy.mockRestore(); }); }); - -describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { - beforeEach(() => { - vi.clearAllMocks(); - mockPbjs.requestBids = mockRequestBids; - mockPbjs.adUnits = []; - mockGetUserIdsAsEids.mockReset(); - mockGetUserIdsAsEids.mockReturnValue([]); - // Restore the pbjs→mock wiring in case a prior test blanked it out. - (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; - delete (window as any).__tsjs_prebid; - // Wipe any leftover ts-eids cookie from previous tests. - document.cookie = 'ts-eids=; Path=/; Max-Age=0'; - }); - - afterEach(() => { - document.cookie = 'ts-eids=; Path=/; Max-Age=0'; - }); - - /** - * Helper: make mockRequestBids actually invoke the injected bidsBackHandler - * so the shim's post-auction sync path runs. - */ - function wireBidsBackHandler(): void { - mockRequestBids.mockImplementation((opts: any) => { - if (typeof opts?.bidsBackHandler === 'function') { - opts.bidsBackHandler(); - } - }); - } - - function getTsEidsCookie(): string | undefined { - const match = document.cookie.split('; ').find((c) => c.startsWith('ts-eids=')); - return match ? match.split('=').slice(1).join('=') : undefined; - } - - it('writes no cookie when getUserIdsAsEids returns empty array', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([]); - - pbjs.requestBids({ adUnits: [] } as any); - - expect(getTsEidsCookie()).toBeUndefined(); - }); - - it('clears an existing cookie when getUserIdsAsEids returns empty array', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - document.cookie = 'ts-eids=stale; Path=/'; - mockGetUserIdsAsEids.mockReturnValue([]); - - pbjs.requestBids({ adUnits: [] } as any); - - expect(getTsEidsCookie()).toBeUndefined(); - }); - - it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'sharedid.org', uids: [{ id: 'shared-abc', atype: 1 }] }, - { source: 'id5-sync.com', uids: [{ id: 'id5-xyz', atype: 3 }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const encoded = getTsEidsCookie(); - expect(encoded).toBeDefined(); - const decoded = JSON.parse(atob(encoded!)); - expect(decoded).toEqual([ - { source: 'sharedid.org', id: 'shared-abc', atype: 1 }, - { source: 'id5-sync.com', id: 'id5-xyz', atype: 3 }, - ]); - }); - - it('defaults atype to 3 when the uid omits it', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([{ source: 'example.com', uids: [{ id: 'no-atype' }] }]); - - pbjs.requestBids({ adUnits: [] } as any); - - const decoded = JSON.parse(atob(getTsEidsCookie()!)); - expect(decoded).toEqual([{ source: 'example.com', id: 'no-atype', atype: 3 }]); - }); - - it('skips EID entries that are missing id or source', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'good.example', uids: [{ id: 'keep', atype: 1 }] }, - { source: 'empty-uids.example', uids: [] }, - { source: '', uids: [{ id: 'no-source', atype: 1 }] }, - { source: 'no-id.example', uids: [{ id: '', atype: 1 }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const decoded = JSON.parse(atob(getTsEidsCookie()!)); - expect(decoded).toEqual([{ source: 'good.example', id: 'keep', atype: 1 }]); - }); - - it('takes the first uid per source when multiple are present', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - mockGetUserIdsAsEids.mockReturnValue([ - { - source: 'multi.example', - uids: [ - { id: 'first', atype: 1 }, - { id: 'second', atype: 2 }, - ], - }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - const decoded = JSON.parse(atob(getTsEidsCookie()!)); - expect(decoded).toEqual([{ source: 'multi.example', id: 'first', atype: 1 }]); - }); - - it('trims EIDs from the tail when the cookie payload would exceed 3072 bytes', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - - // Build ~20 entries each ~200 bytes → definitely exceeds 3072-byte cap - // once base64-encoded. - const big = Array.from({ length: 20 }, (_, i) => ({ - source: `source-${i}.example`, - uids: [{ id: 'x'.repeat(200) + String(i), atype: 3 }], - })); - mockGetUserIdsAsEids.mockReturnValue(big); - - pbjs.requestBids({ adUnits: [] } as any); - - const encoded = getTsEidsCookie(); - expect(encoded).toBeDefined(); - expect(encoded!.length).toBeLessThanOrEqual(3072); - - const decoded = JSON.parse(atob(encoded!)); - // At least one entry kept, strictly fewer than original count. - expect(decoded.length).toBeGreaterThan(0); - expect(decoded.length).toBeLessThan(big.length); - // Head of the list is preserved (trimming happens from the tail). - expect(decoded[0].source).toBe('source-0.example'); - }); - - it('writes no cookie when a single entry alone exceeds the cap', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - - // Single entry large enough to blow past 3072 bytes after base64. - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - expect(getTsEidsCookie()).toBeUndefined(); - }); - - it('clears an existing cookie when flattening yields no valid EIDs', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - document.cookie = 'ts-eids=stale; Path=/'; - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'missing-id.example', uids: [{ id: '', atype: 3 }] }, - { source: '', uids: [{ id: 'missing-source', atype: 3 }] }, - { source: 'empty-uids.example', uids: [] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - expect(getTsEidsCookie()).toBeUndefined(); - }); - - it('clears an existing cookie when a single oversized entry cannot be written', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - document.cookie = 'ts-eids=stale; Path=/'; - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, - ]); - - pbjs.requestBids({ adUnits: [] } as any); - - expect(getTsEidsCookie()).toBeUndefined(); - }); - - it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - // Simulate a build that forgot the userId core module. - (mockPbjs as any).getUserIdsAsEids = undefined; - - expect(() => pbjs.requestBids({ adUnits: [] } as any)).not.toThrow(); - expect(getTsEidsCookie()).toBeUndefined(); - - // Restore for subsequent tests. - (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; - }); - - it('clears an existing cookie when getUserIdsAsEids is undefined', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - document.cookie = 'ts-eids=stale; Path=/'; - (mockPbjs as any).getUserIdsAsEids = undefined; - - pbjs.requestBids({ adUnits: [] } as any); - - expect(getTsEidsCookie()).toBeUndefined(); - - // Restore for subsequent tests. - (mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids; - }); - - it('calls the original bidsBackHandler after syncing EIDs', () => { - wireBidsBackHandler(); - const pbjs = installPrebidNpm(); - const originalHandler = vi.fn(); - - pbjs.requestBids({ adUnits: [], bidsBackHandler: originalHandler } as any); - - expect(originalHandler).toHaveBeenCalledTimes(1); - }); -}); - -describe('prebid User ID Module imports (regression guard)', () => { - // `userId.js` is the core module — bundled unconditionally via a static - // import in index.ts, never operator-configurable. Guard it there. - const INDEX_PATH = resolve(process.cwd(), 'src/integrations/prebid/index.ts'); - const indexSource = readFileSync(INDEX_PATH, 'utf8'); - - it('index.ts statically imports the User ID core module', () => { - expect(indexSource).toMatch(/import\s+['"]prebid\.js\/modules\/userId\.js['"]/); - }); - - it('index.ts statically imports the generated User ID submodule file', () => { - expect(indexSource).toMatch(/import\s+['"]\.\/_user_ids\.generated['"]/); - }); - - // The submodule list is operator-controlled via TSJS_PREBID_USER_IDS, but - // the default ship-set must keep resolving without env var action. Read - // the generated file produced by `node build-all.mjs` with no env override - // and assert every default submodule is imported. If this file is missing, - // the developer has not yet run the build — skip with a clear message. - const GENERATED_PATH = resolve(process.cwd(), 'src/integrations/prebid/_user_ids.generated.ts'); - const DEFAULT_SUBMODULES = [ - 'sharedIdSystem', - 'criteoIdSystem', - '33acrossIdSystem', - 'pubProvidedIdSystem', - 'quantcastIdSystem', - 'id5IdSystem', - 'identityLinkIdSystem', - 'uid2IdSystem', - 'euidIdSystem', - 'intentIqIdSystem', - 'lotamePanoramaIdSystem', - 'connectIdSystem', - 'merkleIdSystem', - ]; - - for (const name of DEFAULT_SUBMODULES) { - it(`_user_ids.generated.ts imports ${name}.js by default`, () => { - const generated = readFileSync(GENERATED_PATH, 'utf8'); - const pattern = new RegExp( - `import\\s+['"]prebid\\.js/modules/${name.replace(/\./g, '\\.')}\\.js['"]` - ); - expect(generated).toMatch(pattern); - }); - } -}); diff --git a/crates/js/lib/test/integrations/sourcepoint/index.test.ts b/crates/js/lib/test/integrations/sourcepoint/index.test.ts index 872585c32..f276f2414 100644 --- a/crates/js/lib/test/integrations/sourcepoint/index.test.ts +++ b/crates/js/lib/test/integrations/sourcepoint/index.test.ts @@ -6,7 +6,7 @@ describe('integrations/sourcepoint', () => { function clearAllCookies(): void { document.cookie.split(';').forEach((c) => { const name = c.split('=')[0].trim(); - if (name) document.cookie = `${name}=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/`; + if (name) document.cookie = `${name}=; path=/; Max-Age=0`; }); } @@ -26,7 +26,7 @@ describe('integrations/sourcepoint', () => { localStorage.clear(); }); - it('mirrors __gpp and __gpp_sid from _sp_user_consent_* localStorage', () => { + it('mirrors __gpp and __gpp_sid from _sp_user_consent_* localStorage as session cookies', () => { const payload = { gppData: { gppString: 'DBABLA~BVQqAAAAAgA.QA', diff --git a/crates/trusted-server-core/src/consent/gpp.rs b/crates/trusted-server-core/src/consent/gpp.rs index cb704b029..bd8c12f24 100644 --- a/crates/trusted-server-core/src/consent/gpp.rs +++ b/crates/trusted-server-core/src/consent/gpp.rs @@ -152,6 +152,7 @@ fn us_sale_opt_out_from_section(section: &iab_gpp::sections::Section) -> Option< use iab_gpp::sections::us_common::OptOut; use iab_gpp::sections::Section; + // Keep this match in sync with new US-state variants added by `iab_gpp`. let sale_opt_out = match section { Section::UsNat(s) => match &s.core { iab_gpp::sections::usnat::Core::V1(c) => &c.sale_opt_out, diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index 8d7a03abd..684910fd4 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -792,14 +792,14 @@ impl IntegrationRegistry { /// Return JS module IDs that should be included in the tsjs bundle. /// - /// Always includes "creative" (JS-only, no Rust-side registration). + /// Always includes JS-only modules with no Rust-side registration. /// Excludes integrations that have no JS module (e.g., "nextjs"). #[must_use] pub fn js_module_ids(&self) -> Vec<&'static str> { // Rust-only integrations with no corresponding JS module const JS_EXCLUDED: &[&str] = &["nextjs", "aps", "adserver_mock"]; // JS-only modules always included (no Rust-side registration) - const JS_ALWAYS: &[&str] = &["creative"]; + const JS_ALWAYS: &[&str] = &["creative", "sourcepoint"]; let mut ids: Vec<&'static str> = JS_ALWAYS.to_vec(); @@ -1377,7 +1377,7 @@ mod tests { } #[test] - fn js_module_ids_immediate_excludes_prebid() { + fn js_module_ids_immediate_excludes_prebid_and_includes_js_only_modules() { let settings = crate::test_support::tests::create_test_settings(); let mut settings_with_prebid = settings; settings_with_prebid @@ -1405,6 +1405,14 @@ mod tests { all.contains(&"prebid"), "should include prebid in full list" ); + assert!( + immediate.contains(&"creative"), + "should include creative in immediate IDs" + ); + assert!( + immediate.contains(&"sourcepoint"), + "should include sourcepoint in immediate IDs" + ); assert!( !immediate.contains(&"prebid"), "should not include prebid in immediate IDs" diff --git a/crates/trusted-server-core/src/platform/test_support.rs b/crates/trusted-server-core/src/platform/test_support.rs index 42e3539ea..1c167b99d 100644 --- a/crates/trusted-server-core/src/platform/test_support.rs +++ b/crates/trusted-server-core/src/platform/test_support.rs @@ -142,8 +142,6 @@ pub(crate) struct StubHttpClient { calls: Mutex>, // (status_code, body_bytes) — kept Send by avoiding Body::Stream responses: Mutex)>>, - // Headers captured per send call, stored as (name, value) string pairs. - request_headers: Mutex>>, } impl StubHttpClient { @@ -151,7 +149,6 @@ impl StubHttpClient { Self { calls: Mutex::new(Vec::new()), responses: Mutex::new(VecDeque::new()), - request_headers: Mutex::new(Vec::new()), } } @@ -181,22 +178,6 @@ impl PlatformHttpClient for StubHttpClient { .expect("should lock calls") .push(request.backend_name.clone()); - let headers: Vec<(String, String)> = request - .request - .headers() - .iter() - .filter_map(|(name, value)| { - value - .to_str() - .ok() - .map(|v| (name.as_str().to_string(), v.to_string())) - }) - .collect(); - self.request_headers - .lock() - .expect("should lock request_headers") - .push(headers); - let (status, body_bytes) = self .responses .lock() diff --git a/crates/trusted-server-core/src/settings.rs b/crates/trusted-server-core/src/settings.rs index ec9d9ef9d..a75eb2c84 100644 --- a/crates/trusted-server-core/src/settings.rs +++ b/crates/trusted-server-core/src/settings.rs @@ -98,27 +98,6 @@ pub trait IntegrationConfig: DeserializeOwned + Validate { fn is_enabled(&self) -> bool; } -fn normalize_env_value(value: JsonValue) -> JsonValue { - match value { - JsonValue::Object(map) => JsonValue::Object( - map.into_iter() - .map(|(key, val)| (key, normalize_env_value(val))) - .collect(), - ), - JsonValue::Array(items) => { - JsonValue::Array(items.into_iter().map(normalize_env_value).collect()) - } - JsonValue::String(raw) => { - if let Ok(parsed) = serde_json::from_str::(&raw) { - parsed - } else { - JsonValue::String(raw) - } - } - other => other, - } -} - impl IntegrationSettings { /// Inserts a configuration value for an integration. /// @@ -142,13 +121,34 @@ impl IntegrationSettings { Ok(()) } + fn normalize_env_value(value: JsonValue) -> JsonValue { + match value { + JsonValue::Object(map) => JsonValue::Object( + map.into_iter() + .map(|(key, val)| (key, Self::normalize_env_value(val))) + .collect(), + ), + JsonValue::Array(items) => { + JsonValue::Array(items.into_iter().map(Self::normalize_env_value).collect()) + } + JsonValue::String(raw) => { + if let Ok(parsed) = serde_json::from_str::(&raw) { + parsed + } else { + JsonValue::String(raw) + } + } + other => other, + } + } + /// Normalizes all entries in place, converting JSON-encoded strings from /// environment variables into their proper typed representations. /// Called eagerly after deserialization so that TOML serialization in /// build.rs preserves correct types. pub fn normalize(&mut self) { for value in self.entries.values_mut() { - *value = normalize_env_value(value.clone()); + *value = Self::normalize_env_value(value.clone()); } } @@ -959,9 +959,7 @@ where match v { JsonValue::Array(arr) => arr .into_iter() - .map(|item| { - serde_json::from_value(normalize_env_value(item)).map_err(serde::de::Error::custom) - }) + .map(|item| serde_json::from_value(item).map_err(serde::de::Error::custom)) .collect(), JsonValue::Object(map) => { let mut items: Vec<(usize, T)> = Vec::with_capacity(map.len()); @@ -969,8 +967,7 @@ where let idx = k.parse::().map_err(|_| { serde::de::Error::custom(format!("Invalid index '{}' in map for Vec field", k)) })?; - let parsed: T = serde_json::from_value(normalize_env_value(val)) - .map_err(serde::de::Error::custom)?; + let parsed: T = serde_json::from_value(val).map_err(serde::de::Error::custom)?; items.push((idx, parsed)); } items.sort_by_key(|(idx, _)| *idx); @@ -1672,167 +1669,6 @@ mod tests { ); } - #[test] - fn test_ec_partners_override_with_indexed_env() { - let toml_str = crate_test_settings_str(); - - let origin_key = format!( - "{}{}PUBLISHER{}ORIGIN_URL", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner0_id_key = format!( - "{}{}EC{}PARTNERS{}0{}ID", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner0_name_key = format!( - "{}{}EC{}PARTNERS{}0{}NAME", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner0_source_domain_key = format!( - "{}{}EC{}PARTNERS{}0{}SOURCE_DOMAIN", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner0_openrtb_atype_key = format!( - "{}{}EC{}PARTNERS{}0{}OPENRTB_ATYPE", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner0_bidstream_enabled_key = format!( - "{}{}EC{}PARTNERS{}0{}BIDSTREAM_ENABLED", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner0_api_token_key = format!( - "{}{}EC{}PARTNERS{}0{}API_TOKEN", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner1_id_key = format!( - "{}{}EC{}PARTNERS{}1{}ID", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner1_name_key = format!( - "{}{}EC{}PARTNERS{}1{}NAME", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner1_source_domain_key = format!( - "{}{}EC{}PARTNERS{}1{}SOURCE_DOMAIN", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner1_openrtb_atype_key = format!( - "{}{}EC{}PARTNERS{}1{}OPENRTB_ATYPE", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner1_bidstream_enabled_key = format!( - "{}{}EC{}PARTNERS{}1{}BIDSTREAM_ENABLED", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - let partner1_api_token_key = format!( - "{}{}EC{}PARTNERS{}1{}API_TOKEN", - ENVIRONMENT_VARIABLE_PREFIX, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR, - ENVIRONMENT_VARIABLE_SEPARATOR - ); - - temp_env::with_vars( - [ - (origin_key, Some("https://origin.test-publisher.com")), - (partner0_id_key, Some("criteo")), - (partner0_name_key, Some("Criteo")), - (partner0_source_domain_key, Some("criteo.com")), - (partner0_openrtb_atype_key, Some("1")), - (partner0_bidstream_enabled_key, Some("true")), - (partner0_api_token_key, Some("criteo-test-token")), - (partner1_id_key, Some("uidapi")), - (partner1_name_key, Some("UID2")), - (partner1_source_domain_key, Some("uidapi.com")), - (partner1_openrtb_atype_key, Some("3")), - (partner1_bidstream_enabled_key, Some("true")), - (partner1_api_token_key, Some("uidapi-test-token")), - ], - || { - let settings = Settings::from_toml_and_env(&toml_str) - .expect("should parse EC partners from indexed env override"); - - assert_eq!( - settings.ec.partners.len(), - 2, - "should override partner list" - ); - assert_eq!(settings.ec.partners[0].id, "criteo"); - assert_eq!(settings.ec.partners[0].name, "Criteo"); - assert_eq!(settings.ec.partners[0].source_domain, "criteo.com"); - assert_eq!(settings.ec.partners[0].openrtb_atype, 1); - assert!( - settings.ec.partners[0].bidstream_enabled, - "should enable bidstream for partner 0" - ); - assert_eq!( - settings.ec.partners[0].api_token.expose(), - "criteo-test-token" - ); - - assert_eq!(settings.ec.partners[1].id, "uidapi"); - assert_eq!(settings.ec.partners[1].name, "UID2"); - assert_eq!(settings.ec.partners[1].source_domain, "uidapi.com"); - assert_eq!(settings.ec.partners[1].openrtb_atype, 3); - assert!( - settings.ec.partners[1].bidstream_enabled, - "should enable bidstream for partner 1" - ); - assert_eq!( - settings.ec.partners[1].api_token.expose(), - "uidapi-test-token" - ); - }, - ); - } - #[test] fn test_settings_extra_fields() { let toml_str = crate_test_settings_str() + "\nhello = 1"; diff --git a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md b/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md deleted file mode 100644 index cbf3acaf2..000000000 --- a/docs/superpowers/plans/2026-04-16-prebid-user-id-module.md +++ /dev/null @@ -1,599 +0,0 @@ -# Prebid User ID Module — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Bundle Prebid's User ID core module and a broad set of ID submodules so publisher-side `pbjs.setConfig({ userSync: { userIds: [...] } })` calls activate real ID resolution and the existing `syncPrebidEidsCookie()` helper begins writing `ts-eids`. - -**Architecture:** JS-only change. Add 15 static imports (1 core + 14 submodules) to `crates/js/lib/src/integrations/prebid/index.ts`. No Rust changes, no TOML changes, no new runtime logic. Existing `bidsBackHandler` shim and cookie-sync path already handle the rest; they were silent only because `pbjs.getUserIdsAsEids` did not exist. - -> **In-flight correction (2026-04-16):** `pubCommonIdSystem.js` was originally -> included in the plan as a legacy/compatibility submodule. It does not exist -> in Prebid 10.26.0 (consolidated into `sharedIdSystem`) and has been removed. -> All downstream counts reflect 14 submodules instead of 15. - -**Tech Stack:** TypeScript, Vitest, esbuild (via `build-all.mjs`), Prebid.js 9.x (via npm) - -**Spec:** `docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md` - ---- - -## File Map - -| File | Action | Responsibility | -| ------------------------------------------------------ | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | -| `crates/js/lib/src/integrations/prebid/index.ts` | Modify | Add 15 User ID Module imports alongside existing `consentManagement*` imports | -| `crates/js/lib/test/integrations/prebid/index.test.ts` | Modify | Add `vi.mock` stubs for the 15 new modules, add `mockPbjs.getUserIdsAsEids`, add new tests for cookie-sync behavior and import regression guard | - -No other files change. `build.rs` picks up the rebuilt `dist/tsjs-prebid.js` automatically via `include_str!`. - ---- - -## Task 1: Document existing `syncPrebidEidsCookie` behavior with tests - -The sync helper already exists but has no test coverage. Before changing anything, lock in its current contract so we can refactor or extend later without regressions. These tests exercise the `bidsBackHandler` shim path end-to-end using the existing mocks. - -**Files:** - -- Modify: `crates/js/lib/test/integrations/prebid/index.test.ts` - -- [ ] **Step 1: Add `getUserIdsAsEids` to the hoisted pbjs mock** - -In `crates/js/lib/test/integrations/prebid/index.test.ts` inside the existing `vi.hoisted(() => { ... })` block, add a new mock function and include it on `mockPbjs`. Replace the current block with: - -```ts -const { - mockSetConfig, - mockProcessQueue, - mockRequestBids, - mockRegisterBidAdapter, - mockGetUserIdsAsEids, - mockPbjs, - mockGetBidAdapter, - mockAdapterManager, -} = vi.hoisted(() => { - const mockSetConfig = vi.fn() - const mockProcessQueue = vi.fn() - const mockRequestBids = vi.fn() - const mockRegisterBidAdapter = vi.fn() - const mockGetBidAdapter = vi.fn() - const mockGetUserIdsAsEids = vi.fn( - () => - [] as Array<{ - source: string - uids?: Array<{ id: string; atype?: number }> - }> - ) - const mockPbjs = { - setConfig: mockSetConfig, - processQueue: mockProcessQueue, - requestBids: mockRequestBids, - registerBidAdapter: mockRegisterBidAdapter, - getUserIdsAsEids: mockGetUserIdsAsEids, - adUnits: [] as any[], - } - const mockAdapterManager = { - getBidAdapter: mockGetBidAdapter, - } - return { - mockSetConfig, - mockProcessQueue, - mockRequestBids, - mockRegisterBidAdapter, - mockGetUserIdsAsEids, - mockPbjs, - mockGetBidAdapter, - mockAdapterManager, - } -}) -``` - -- [ ] **Step 2: Write the failing test — empty EID array writes no cookie** - -Append this new `describe` block at the end of `crates/js/lib/test/integrations/prebid/index.test.ts`: - -```ts -describe('prebid/syncPrebidEidsCookie (via bidsBackHandler)', () => { - beforeEach(() => { - vi.clearAllMocks() - mockPbjs.requestBids = mockRequestBids - mockPbjs.adUnits = [] - mockGetUserIdsAsEids.mockReset() - mockGetUserIdsAsEids.mockReturnValue([]) - // Restore the pbjs→mock wiring in case a prior test blanked it out. - ;(mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids - delete (window as any).__tsjs_prebid - // Wipe any leftover ts-eids cookie from previous tests. - document.cookie = 'ts-eids=; Path=/; Max-Age=0' - }) - - afterEach(() => { - document.cookie = 'ts-eids=; Path=/; Max-Age=0' - }) - - /** - * Helper: make mockRequestBids actually invoke the injected bidsBackHandler - * so the shim's post-auction sync path runs. - */ - function wireBidsBackHandler(): void { - mockRequestBids.mockImplementation((opts: any) => { - if (typeof opts?.bidsBackHandler === 'function') { - opts.bidsBackHandler() - } - }) - } - - function getTsEidsCookie(): string | undefined { - const match = document.cookie - .split('; ') - .find((c) => c.startsWith('ts-eids=')) - return match ? match.split('=').slice(1).join('=') : undefined - } - - it('writes no cookie when getUserIdsAsEids returns empty array', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - mockGetUserIdsAsEids.mockReturnValue([]) - - pbjs.requestBids({ adUnits: [] } as any) - - expect(getTsEidsCookie()).toBeUndefined() - }) -}) -``` - -- [ ] **Step 3: Run the test — expect PASS (documents existing behavior)** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "writes no cookie when getUserIdsAsEids returns empty array"` - -Expected: PASS. The shim already calls `syncPrebidEidsCookie` which early-returns on empty input. - -- [ ] **Step 4: Add test — writes base64 cookie for a normal payload** - -Append inside the same `describe` block: - -```ts -it('writes ts-eids cookie with base64-encoded flat JSON for normal payload', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'sharedid.org', uids: [{ id: 'shared-abc', atype: 1 }] }, - { source: 'id5-sync.com', uids: [{ id: 'id5-xyz', atype: 3 }] }, - ]) - - pbjs.requestBids({ adUnits: [] } as any) - - const encoded = getTsEidsCookie() - expect(encoded).toBeDefined() - const decoded = JSON.parse(atob(encoded!)) - expect(decoded).toEqual([ - { source: 'sharedid.org', id: 'shared-abc', atype: 1 }, - { source: 'id5-sync.com', id: 'id5-xyz', atype: 3 }, - ]) -}) -``` - -- [ ] **Step 5: Run and confirm PASS** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "writes ts-eids cookie with base64"` - -Expected: PASS. - -- [ ] **Step 6: Add test — defaults atype to 3 when missing** - -Append: - -```ts -it('defaults atype to 3 when the uid omits it', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'example.com', uids: [{ id: 'no-atype' }] }, - ]) - - pbjs.requestBids({ adUnits: [] } as any) - - const decoded = JSON.parse(atob(getTsEidsCookie()!)) - expect(decoded).toEqual([{ source: 'example.com', id: 'no-atype', atype: 3 }]) -}) -``` - -- [ ] **Step 7: Add test — skips entries without an id or source** - -Append: - -```ts -it('skips EID entries that are missing id or source', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'good.example', uids: [{ id: 'keep', atype: 1 }] }, - { source: 'empty-uids.example', uids: [] }, - { source: '', uids: [{ id: 'no-source', atype: 1 }] }, - { source: 'no-id.example', uids: [{ id: '', atype: 1 }] }, - ]) - - pbjs.requestBids({ adUnits: [] } as any) - - const decoded = JSON.parse(atob(getTsEidsCookie()!)) - expect(decoded).toEqual([{ source: 'good.example', id: 'keep', atype: 1 }]) -}) -``` - -- [ ] **Step 8: Add test — takes first uid when multiple are present** - -Append: - -```ts -it('takes the first uid per source when multiple are present', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - mockGetUserIdsAsEids.mockReturnValue([ - { - source: 'multi.example', - uids: [ - { id: 'first', atype: 1 }, - { id: 'second', atype: 2 }, - ], - }, - ]) - - pbjs.requestBids({ adUnits: [] } as any) - - const decoded = JSON.parse(atob(getTsEidsCookie()!)) - expect(decoded).toEqual([{ source: 'multi.example', id: 'first', atype: 1 }]) -}) -``` - -- [ ] **Step 9: Add test — trims tail when payload exceeds 3072 bytes** - -Append: - -```ts -it('trims EIDs from the tail when the cookie payload would exceed 3072 bytes', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - - // Build ~20 entries each ~200 bytes → definitely exceeds 3072-byte cap - // once base64-encoded. - const big = Array.from({ length: 20 }, (_, i) => ({ - source: `source-${i}.example`, - uids: [{ id: 'x'.repeat(200) + String(i), atype: 3 }], - })) - mockGetUserIdsAsEids.mockReturnValue(big) - - pbjs.requestBids({ adUnits: [] } as any) - - const encoded = getTsEidsCookie() - expect(encoded).toBeDefined() - expect(encoded!.length).toBeLessThanOrEqual(3072) - - const decoded = JSON.parse(atob(encoded!)) - // At least one entry kept, strictly fewer than original count. - expect(decoded.length).toBeGreaterThan(0) - expect(decoded.length).toBeLessThan(big.length) - // Head of the list is preserved (trimming happens from the tail). - expect(decoded[0].source).toBe('source-0.example') -}) -``` - -- [ ] **Step 10: Add test — writes no cookie when a single entry alone exceeds the cap** - -Append: - -```ts -it('writes no cookie when a single entry alone exceeds the cap', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - - // Single entry large enough to blow past 3072 bytes after base64. - mockGetUserIdsAsEids.mockReturnValue([ - { source: 'too-big.example', uids: [{ id: 'x'.repeat(4000), atype: 3 }] }, - ]) - - pbjs.requestBids({ adUnits: [] } as any) - - expect(getTsEidsCookie()).toBeUndefined() -}) -``` - -- [ ] **Step 11: Add test — does not throw when getUserIdsAsEids is undefined** - -This mirrors the pre-fix production state and guards against regressions in the defensive check at `index.ts:375`. Append: - -```ts -it('does not throw when getUserIdsAsEids is undefined (pre-fix production state)', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - // Simulate a build that forgot the userId core module. - ;(mockPbjs as any).getUserIdsAsEids = undefined - - expect(() => pbjs.requestBids({ adUnits: [] } as any)).not.toThrow() - expect(getTsEidsCookie()).toBeUndefined() - - // Restore for subsequent tests. - ;(mockPbjs as any).getUserIdsAsEids = mockGetUserIdsAsEids -}) -``` - -- [ ] **Step 12: Add test — calls the original bidsBackHandler when one was supplied** - -Append: - -```ts -it('calls the original bidsBackHandler after syncing EIDs', () => { - wireBidsBackHandler() - const pbjs = installPrebidNpm() - const originalHandler = vi.fn() - - pbjs.requestBids({ adUnits: [], bidsBackHandler: originalHandler } as any) - - expect(originalHandler).toHaveBeenCalledTimes(1) -}) -``` - -- [ ] **Step 13: Run the full new block and confirm all pass** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "syncPrebidEidsCookie"` - -Expected: all 9 new tests PASS. If any fail, investigate before proceeding — the rest of the plan assumes this behavior is locked in. - -- [ ] **Step 14: Run the entire prebid test file to confirm no regressions** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts` - -Expected: all tests PASS (new + existing). - -- [ ] **Step 15: Commit** - -```bash -git add crates/js/lib/test/integrations/prebid/index.test.ts -git commit -m "Add Vitest coverage for Prebid ts-eids cookie sync" -``` - ---- - -## Task 2: Add Prebid User ID core and submodule imports - -This is the substantive change. Add `vi.mock` stubs for the new modules first (so tests don't blow up when the imports are added), then add the imports. - -**Files:** - -- Modify: `crates/js/lib/test/integrations/prebid/index.test.ts:44-47` -- Modify: `crates/js/lib/src/integrations/prebid/index.ts:16-18` - -- [ ] **Step 1: Add `vi.mock` stubs in the test file for all 15 new modules** - -In `crates/js/lib/test/integrations/prebid/index.test.ts`, locate the existing block (around line 44-47): - -```ts -// Side-effect imports are no-ops in tests -vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})) -vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})) -vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})) -``` - -Replace it with (consent management mocks stay; add 15 new ones): - -```ts -// Side-effect imports are no-ops in tests -vi.mock('prebid.js/modules/consentManagementTcf.js', () => ({})) -vi.mock('prebid.js/modules/consentManagementGpp.js', () => ({})) -vi.mock('prebid.js/modules/consentManagementUsp.js', () => ({})) - -// User ID Module core + submodules — no-op mocks so jsdom does not try to -// execute the real Prebid code paths. -vi.mock('prebid.js/modules/userId.js', () => ({})) -vi.mock('prebid.js/modules/sharedIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/criteoIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/33acrossIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/pubProvidedIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/quantcastIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/id5IdSystem.js', () => ({})) -vi.mock('prebid.js/modules/identityLinkIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/liveIntentIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/uid2IdSystem.js', () => ({})) -vi.mock('prebid.js/modules/euidIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/intentIqIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/lotamePanoramaIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/connectIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/merkleIdSystem.js', () => ({})) -vi.mock('prebid.js/modules/pubCommonIdSystem.js', () => ({})) -``` - -- [ ] **Step 2: Write the failing regression-guard test** - -Append this `describe` block to `crates/js/lib/test/integrations/prebid/index.test.ts`: - -```ts -import { readFileSync } from 'node:fs' -import { fileURLToPath } from 'node:url' -import { resolve } from 'node:path' - -describe('prebid/index.ts User ID Module imports (regression guard)', () => { - const REQUIRED_IMPORTS = [ - 'prebid.js/modules/userId.js', - 'prebid.js/modules/sharedIdSystem.js', - 'prebid.js/modules/criteoIdSystem.js', - 'prebid.js/modules/33acrossIdSystem.js', - 'prebid.js/modules/pubProvidedIdSystem.js', - 'prebid.js/modules/quantcastIdSystem.js', - 'prebid.js/modules/id5IdSystem.js', - 'prebid.js/modules/identityLinkIdSystem.js', - 'prebid.js/modules/liveIntentIdSystem.js', - 'prebid.js/modules/uid2IdSystem.js', - 'prebid.js/modules/euidIdSystem.js', - 'prebid.js/modules/intentIqIdSystem.js', - 'prebid.js/modules/lotamePanoramaIdSystem.js', - 'prebid.js/modules/connectIdSystem.js', - 'prebid.js/modules/merkleIdSystem.js', - 'prebid.js/modules/pubCommonIdSystem.js', - ] - - // Source-text check: these mocks make the runtime pbjs mock a no-op for the - // User ID Module, so there is no way to assert `typeof getUserIdsAsEids === - // 'function'` at import time from within Vitest. Reading the source file - // directly is the most reliable way to catch accidental removal of an - // import, which is the exact regression that motivated this work. - // The package is ESM (`"type": "module"`), so `__dirname` is not defined — - // resolve relative to this file via `import.meta.url`. - const THIS_DIR = fileURLToPath(new URL('.', import.meta.url)) - const SOURCE_PATH = resolve( - THIS_DIR, - '../../../src/integrations/prebid/index.ts' - ) - const source = readFileSync(SOURCE_PATH, 'utf8') - - for (const module of REQUIRED_IMPORTS) { - it(`statically imports ${module}`, () => { - const pattern = new RegExp( - `import\\s+['"]${module.replace(/\./g, '\\.')}['"]` - ) - expect(source).toMatch(pattern) - }) - } -}) -``` - -- [ ] **Step 3: Run the new block — expect 15 failures** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "regression guard"` - -Expected: FAIL — 15 failing assertions, one per expected import. This confirms the regression guard actually reads the source. - -- [ ] **Step 4: Add the 15 imports to `index.ts`** - -In `crates/js/lib/src/integrations/prebid/index.ts`, locate lines 16-18: - -```ts -import 'prebid.js/modules/consentManagementTcf.js' -import 'prebid.js/modules/consentManagementGpp.js' -import 'prebid.js/modules/consentManagementUsp.js' -``` - -Insert the User ID imports immediately after them, before the existing `// Client-side bid adapters` comment block. The resulting section must read: - -```ts -import 'prebid.js/modules/consentManagementTcf.js' -import 'prebid.js/modules/consentManagementGpp.js' -import 'prebid.js/modules/consentManagementUsp.js' - -// Prebid User ID Module — core + submodules. The core module exposes -// `pbjs.getUserIdsAsEids`; submodules self-register at import time and -// activate when the publisher's origin-side `pbjs.setConfig({ userSync: -// { userIds: [...] } })` call runs during `processQueue()`. -import 'prebid.js/modules/userId.js' - -// Zero-config / auto-populating submodules (resolve without publisher params). -import 'prebid.js/modules/sharedIdSystem.js' -import 'prebid.js/modules/criteoIdSystem.js' -import 'prebid.js/modules/33acrossIdSystem.js' -import 'prebid.js/modules/pubProvidedIdSystem.js' -import 'prebid.js/modules/quantcastIdSystem.js' - -// Param-based submodules — inert until publisher setConfig supplies params. -import 'prebid.js/modules/id5IdSystem.js' -import 'prebid.js/modules/identityLinkIdSystem.js' -import 'prebid.js/modules/liveIntentIdSystem.js' -import 'prebid.js/modules/uid2IdSystem.js' -import 'prebid.js/modules/euidIdSystem.js' -import 'prebid.js/modules/intentIqIdSystem.js' -import 'prebid.js/modules/lotamePanoramaIdSystem.js' -import 'prebid.js/modules/connectIdSystem.js' -import 'prebid.js/modules/merkleIdSystem.js' - -// Legacy / deprecated but still present in some publisher configs. -import 'prebid.js/modules/pubCommonIdSystem.js' -``` - -- [ ] **Step 5: Run the regression-guard block — expect PASS** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts -t "regression guard"` - -Expected: all 15 tests PASS. - -- [ ] **Step 6: Run the full prebid test file — expect no regressions** - -Run: `cd crates/js/lib && npx vitest run test/integrations/prebid/index.test.ts` - -Expected: all tests PASS (Task 1 tests + regression guards + all pre-existing tests). - -- [ ] **Step 7: Run the entire JS test suite** - -Run: `cd crates/js/lib && npx vitest run` - -Expected: all PASS. If any unrelated test fails because it imports `./index.ts` transitively and the real Prebid modules are not mocked there, add matching `vi.mock` stubs at the top of that test file (copy the same 15 lines). Common suspects: any test under `crates/js/lib/test/integrations/prebid/` you may have added, or tests that import core modules which in turn import the prebid integration. At the time of writing, no other test file imports the prebid integration. - -- [ ] **Step 8: Build the JS bundles** - -Run: `cd crates/js/lib && node build-all.mjs` - -Expected: build succeeds. `dist/tsjs-prebid.js` gets substantially larger (est. 100-150kb gzipped increase). No esbuild errors about missing modules — if there are, the module path in the new imports is wrong (check `crates/js/lib/node_modules/prebid.js/modules/` for the exact filename — note `33acrossIdSystem.js` really does start with a digit and is correct). - -- [ ] **Step 9: Format the JS** - -Run: `cd crates/js/lib && npm run format` - -Expected: prettier rewrites any formatting drift in the files you touched. No errors. - -- [ ] **Step 10: Verify the Rust build picks up the rebuilt bundle** - -Run: `cargo check --package trusted-server-core` - -Expected: PASS. `build.rs` re-runs because `dist/tsjs-prebid.js` changed; `include_str!` pulls in the new content. - -- [ ] **Step 11: Run full Rust test suite to confirm no downstream breakage** - -Run: `cargo test --workspace` - -Expected: PASS. The Rust side does not inspect bundle contents, only concatenates and hashes them, so tests should be unaffected. - -- [ ] **Step 12: Commit** - -```bash -git add crates/js/lib/src/integrations/prebid/index.ts crates/js/lib/test/integrations/prebid/index.test.ts -git commit -m "Bundle Prebid User ID core and submodules in Prebid integration" -``` - ---- - -## Task 3: Final verification - -- [ ] **Step 1: Full CI-equivalent check** - -Run the same sequence CI runs: - -```bash -cargo fmt --all -- --check -cargo clippy --workspace --all-targets --all-features -- -D warnings -cargo test --workspace -cd crates/js/lib && npx vitest run && cd ../../.. -cd crates/js/lib && npm run format && cd ../../.. -``` - -Expected: everything PASS / clean. - -- [ ] **Step 2: Manual verification note** - -Manual browser verification (cannot be automated here; run against a dev publisher environment that has origin-side `pbjs.setConfig({ userSync: { userIds: [...] } })`): - -1. Load a publisher page. In DevTools console: `typeof pbjs.getUserIdsAsEids` should return `'function'`. -2. `pbjs.getUserIdsAsEids()` should return a non-empty array. -3. After the first ad-slot auction completes: `document.cookie.match(/ts-eids=/)` should match. -4. Decode the cookie: `JSON.parse(atob(document.cookie.match(/ts-eids=([^;]+)/)[1]))` should produce a `[{source, id, atype}]` array matching the raw EIDs. -5. Network tab: the second `/auction` request should carry `Cookie: ts-eids=...`. - -These are documented in the spec; they are not blockers for the PR, but they should be run before closing out the work. - -- [ ] **Step 3: No follow-up commits required** - -The work is complete when Tasks 1 and 2 are committed. Do not create a third "chore" commit unless format/clippy asks for one. - ---- - -## What this plan intentionally does NOT do - -- Does **not** add a build-time env-var toggle (`TSJS_PREBID_USER_IDS`) to mirror `TSJS_PREBID_ADAPTERS`. Deferred per spec. -- Does **not** add `window.__tsjs_prebid.userIds` server-side injection. Deferred per spec. -- Does **not** change `[[ec.partners]]` or `crates/trusted-server-core/src/ec/prebid_eids.rs`. Backend already handles received cookies correctly. -- Does **not** add a bundle-size regression gate. Noted as a known cost in the spec. -- Does **not** add tests for individual ID submodule resolution behavior. That is Prebid's own test surface, not ours. diff --git a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md index ed72f3f55..2455e12a4 100644 --- a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md +++ b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md @@ -40,7 +40,7 @@ No Rust-side `IntegrationRegistration` (same pattern as `creative`). **On page load:** 1. Scan `localStorage` keys matching `_sp_user_consent_*`. -2. Take the first match, parse the JSON value. +2. Take the first valid match, parse the JSON value. 3. Extract `gppData.gppString` and `gppData.applicableSections` from the payload. 4. Write first-party cookies: - `__gpp=` (path `/`, `SameSite=Lax`) @@ -48,8 +48,10 @@ No Rust-side `IntegrationRegistration` (same pattern as `creative`). 5. Log what was written for debugging. Cookies are session-scoped (no `max-age` / `expires`) since the source of truth -stays in `localStorage` and we re-mirror on each page load. The integration runs -once — no polling or event listeners. +stays in `localStorage` and we re-mirror on each page load. This design assumes +a single active Sourcepoint property per page; if multiple `_sp_user_consent_*` +entries coexist, the first valid one wins. The integration runs once — no +polling or event listeners. ### 2. Server-side: GPP US section decoding diff --git a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md b/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md deleted file mode 100644 index 2ef80a3eb..000000000 --- a/docs/superpowers/specs/2026-04-16-prebid-user-id-module-design.md +++ /dev/null @@ -1,212 +0,0 @@ -# Prebid User ID Module support - -**Date:** 2026-04-16 -**Status:** Design -**Scope:** JS bundle (`crates/js/lib/src/integrations/prebid/index.ts`) - -## Problem - -The Trusted Server Prebid integration strips each publisher's origin `prebid.js` -and replaces it with a server-bundled build. That bundle imports the consent -management modules but does **not** import Prebid's User ID core module or any -ID submodules. As a result `pbjs.getUserIdsAsEids` is `undefined` at runtime, -the `syncPrebidEidsCookie()` helper early-returns, and the `ts-eids` cookie is -never written — even when the publisher's origin-side code has a fully -configured `userSync.userIds` list. - -Downstream, `crates/trusted-server-core/src/ec/prebid_eids.rs` never receives a -cookie to ingest, so matched partner UIDs never land in the KV identity graph. - -## Goal - -Bundle Prebid's User ID core module and a broad, widely-deployed set of ID -submodules so publishers' existing `pbjs.setConfig({ userSync: { userIds: ... } })` -calls activate real ID resolution. After first auction completes, `ts-eids` -cookie is written and the backend ingestion path (already implemented) takes -over. - -## Non-goals - -- No Rust changes. No new `trusted-server.toml` fields. -- No runtime config injection from the server (`window.__tsjs_prebid.userIds`) - — deferred to a follow-up. -- No build-time env-var toggle for the bundled set (e.g. `TSJS_PREBID_USER_IDS` - mirroring `TSJS_PREBID_ADAPTERS`) — deferred to a follow-up. -- No automatic alignment between bundled ID submodules and configured - `[[ec.partners]]` — operators must keep those in sync themselves. - -## Design - -### Bundled modules - -Exactly one file changes: `crates/js/lib/src/integrations/prebid/index.ts`. -Add static imports near the existing `consentManagement*.js` imports. - -**Core (required):** - -- `prebid.js/modules/userId.js` - -**Zero-config / auto-populating submodules** (resolve without publisher params): - -- `prebid.js/modules/sharedIdSystem.js` -- `prebid.js/modules/criteoIdSystem.js` -- `prebid.js/modules/33acrossIdSystem.js` -- `prebid.js/modules/pubProvidedIdSystem.js` -- `prebid.js/modules/quantcastIdSystem.js` - -**Param-based submodules** (inert until the publisher's `setConfig` supplies -the relevant params): - -- `prebid.js/modules/id5IdSystem.js` -- `prebid.js/modules/identityLinkIdSystem.js` -- `prebid.js/modules/uid2IdSystem.js` -- `prebid.js/modules/euidIdSystem.js` -- `prebid.js/modules/intentIqIdSystem.js` -- `prebid.js/modules/lotamePanoramaIdSystem.js` -- `prebid.js/modules/connectIdSystem.js` -- `prebid.js/modules/merkleIdSystem.js` - -Total: 1 core + 13 submodules = 14 new imports. - -> **Notes (2026-04-16, during implementation):** -> -> - `pubCommonIdSystem.js`, originally planned as a legacy/compatibility -> submodule, was removed from Prebid.js in 10.x (consolidated into -> `sharedIdSystem`). Not importable from our pinned Prebid 10.26.0; dropped. -> - `liveIntentIdSystem.js` was dropped from the day-1 bundle. Its upstream -> module uses a dynamic `require()` inside a build-flag-guarded branch that -> Prebid's own gulp pipeline dead-codes via constant folding; esbuild leaves -> the `require()` call in the bundle, which throws `ReferenceError: require -> is not defined` at browser runtime. Re-enabling requires an esbuild -> resolver plugin (or Prebid's own build pipeline). Tracked as a follow-up. - -No changes to `installPrebidNpm`, no changes to the `bidsBackHandler` shim, no -changes to `syncPrebidEidsCookie`. The existing cookie-writing path is already -correct — it was only silent because `pbjs.getUserIdsAsEids` did not exist. - -### Runtime flow - -No new runtime logic. The sequence below is what will light up once the -submodules are present: - -1. Rust `IntegrationHeadInjector` emits the `window.pbjs` / `window.pbjs.que` - / `window.__tsjs_prebid` bootstrap before any publisher-origin script runs. -2. Publisher origin code queues its existing config: - `pbjs.que.push(() => pbjs.setConfig({ userSync: { userIds: [...] } }))`. -3. Our bundle loads. `installPrebidNpm()` registers the `trustedServer` - adapter, shims `requestBids` (already appends a chained `bidsBackHandler` - calling `syncPrebidEidsCookie`), then calls `pbjs.processQueue()` — the - publisher's queued `setConfig` runs at this point and activates the - configured submodules (each self-registered at import time). -4. User ID Module resolves IDs per its own rules (TCF/GPP/USP-gated, async). -5. First `requestBids` fires. Auction completes. Chained `bidsBackHandler` - calls `syncPrebidEidsCookie()`. -6. `syncPrebidEidsCookie` calls `pbjs.getUserIdsAsEids()` (now a real - function), flattens `[{source, id, atype}]`, base64-encodes JSON, writes - `document.cookie = "ts-eids=..."`. -7. Subsequent `/auction` requests carry `Cookie: ts-eids=...`. -8. Backend (`crates/trusted-server-core/src/ec/prebid_eids.rs`) parses the - cookie, matches `source` against `[[ec.partners]]`, syncs partner UIDs to - KV. - -The first `/auction` request after a cold page load still will not carry -`ts-eids`, because the cookie is written in the post-auction handler. This -matches preexisting behavior. - -### Error handling - -All failure modes are already covered by existing code. No new error paths. - -- **Publisher has no `userSync.userIds` configured** → - `pbjs.getUserIdsAsEids()` returns `[]` → early-return at `index.ts:380-382`. - No cookie written. Silent. Correct. -- **Submodule fails to resolve** (no consent, no third-party ID, network - error) → handled inside Prebid; `getUserIdsAsEids()` returns only the - resolved subset. Cookie reflects what resolved. -- **Cookie payload exceeds 3072 bytes** → existing trim-and-retry loop at - `index.ts:404-411` drops entries from the tail until it fits. If a single - entry alone exceeds the cap, no cookie is written. -- **Unexpected exception in sync path** → caught by the existing `try/catch` - at `index.ts:417-419`, logged via `log.warn`, does not break the auction. -- **Module import failure at build time** → esbuild fails the build. This - catches missing or renamed Prebid modules before they ship. - -### Known caveats - -- **Backend pairing** — an EID whose `source` has no matching `[[ec.partners]]` - entry is dropped at the backend (with a debug log). Bundling - `id5IdSystem.js` is inert for EC identity-graph purposes unless the - operator also adds an `[[ec.partners]]` entry with - `source_domain = "id5-sync.com"`. Operators must keep the two lists in - sync. Not a code change here; documented as an operator concern. -- **Bundle size** — adding 15 modules increases the shipped `tsjs-prebid.js` - by an estimated ~100-150kb gzipped. Not gated on a build-time toggle in - this change. - -## Testing - -### Automated (Vitest) - -Add tests under `crates/js/lib/src/integrations/prebid/`: - -- **Import smoke test** — import `./index.ts` and assert - `typeof pbjs.getUserIdsAsEids === 'function'`. Guards against the exact - regression that motivated this work. -- **`syncPrebidEidsCookie` unit tests** (new or expanded) — mock - `pbjs.getUserIdsAsEids` to return a fixed `[{source, uids: [{id, atype}]}]` - array and assert the cookie is written with base64-encoded - `[{source, id, atype}]`. Cover: - - empty array → no cookie written - - normal payload → cookie written with expected value - - oversize payload → trimmed to fit; partial entries persisted - - single oversize entry → no cookie written - -### Manual (after deploy to a dev publisher) - -- DevTools console: `typeof pbjs.getUserIdsAsEids === 'function'` returns - `true`. -- `pbjs.getUserIdsAsEids()` returns a non-empty array for a publisher with - configured `userIds`. -- After the first auction: `document.cookie` contains `ts-eids=...`. Decoded - payload (base64 → JSON) matches the raw EIDs. -- Network tab: second `/auction` request carries `Cookie: ts-eids=...`. - -### Explicitly out of scope - -- Each individual ID submodule's resolution behavior — that is Prebid's - responsibility and covered by Prebid's own test suite. -- Backend ingestion of `ts-eids` — already covered by `prebid_eids.rs` - tests; no new backend code. -- Bundle-size regression gating — noted as a caveat, not enforced. - -## Rollout - -This is a bundle change only. No migration, no feature flag, no staged -rollout beyond normal deploy. - -On first deploy, publishers with active origin-side `userSync.userIds` -configuration will begin emitting `ts-eids` cookies after their first -auction. Publishers without `userSync.userIds` configured see no change. - -## Follow-ups - -1. ~~**Build-time configurability** — introduce `_user_ids.generated.ts` - driven by a `TSJS_PREBID_USER_IDS` env var, mirroring the existing - `TSJS_PREBID_ADAPTERS` / `_adapters.generated.ts` pattern. Allows - operators to slim the bundle per deployment.~~ **Implemented - 2026-04-16** — see `docs/guide/integrations/prebid.md` "Build-time - submodule selection" and the `generatePrebidUserIds()` function in - `crates/js/lib/build-all.mjs`. -2. **Server-injected `userSync.userIds`** — extend `trusted-server.toml` - with a `[[integrations.prebid.user_ids]]` array. Rust serializes into - `window.__tsjs_prebid.userIds`. JS applies via `pbjs.setConfig` before - `processQueue()`. Supports publishers who do not run their own Prebid - config on origin. -3. **Partner alignment tooling** — a startup-time check that warns when a - bundled ID submodule has no matching `[[ec.partners]]` entry, or vice - versa. -4. **Re-enable `liveIntentIdSystem.js`** — requires either an esbuild - resolver plugin that rewrites the dynamic `require('../libraries/ - liveIntentId/idSystem.js')` inside `loadModule()` to a static import, or - adopting Prebid's own gulp build pipeline for the vendored bundle. - Out-of-scope for the initial ship. From fedcc347a26cf1b6439e0046c8910d05c5b8336f Mon Sep 17 00:00:00 2001 From: prk-Jr <49094961+prk-Jr@users.noreply.github.com> Date: Wed, 29 Apr 2026 13:23:56 +0530 Subject: [PATCH 19/22] Wire request signing to RuntimeServices store primitives (PR 9) (#609) and Move logging initialization into Fastly adapter (#610) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Rename crates to trusted-server-core and trusted-server-adapter-fastly Rename crates/common → crates/trusted-server-core and crates/fastly → crates/trusted-server-adapter-fastly following the EdgeZero naming convention. Add EdgeZero workspace dependencies pinned to rev 170b74b. Update all references across docs, CI workflows, scripts, agent files, and configuration. * Add platform abstraction layer with traits and RuntimeServices Introduces trusted-server-core::platform with PlatformConfigStore, PlatformSecretStore, PlatformKvStore, PlatformBackend, PlatformHttpClient, and PlatformGeo traits alongside ClientInfo, PlatformError, and RuntimeServices. Wires the Fastly adapter implementations and threads RuntimeServices into route_request. Moves GeoInfo to platform/types as platform-neutral data and adds geo_from_fastly for field mapping. * Address platform layer review feedback - Defer KV store opening: replace early error return with a local UnavailableKvStore fallback so routes that do not need synthetic ID access succeed when the KV store is missing or temporarily unavailable - Use ConfigStore::try_open + try_get and SecretStore::try_get throughout FastlyPlatformConfigStore and FastlyPlatformSecretStore to honour the Result contract instead of panicking on open/lookup failure - Encapsulate RuntimeServices service fields as pub(crate) with public getter methods (config_store, secret_store, backend, http_client, geo) and a pub new() constructor; adapter updated to use new() - Reference #487 in FastlyPlatformHttpClient stub (PR 6 implements it) - Remove unused KvPage re-export from platform/mod.rs - Use super::KvHandle shorthand in RuntimeServices::kv_handle() * Reject host strings containing control characters in BackendConfig * Fix clippy error * Validate scheme and host for control characters in BackendConfig * Address review findings on platform abstraction layer * Address review findings on platform abstraction layer * Add config store read path and storage module split - Split fastly_storage.rs into storage/{config_store,secret_store,api_client,mod}.rs - Add PlatformConfigStore read path via FastlyPlatformConfigStore::get using ConfigStore::try_open/try_get - Add PlatformError::NotImplemented variant; stub write methods on FastlyPlatformConfigStore and FastlyPlatformSecretStore - Add StoreName/StoreId newtypes with From, From<&str>, AsRef - Add UnavailableKvStore to core platform module - Add RuntimeServicesBuilder replacing 7-arg constructor - Migrate get_active_jwks and handle_trusted_server_discovery to use &RuntimeServices - Update call sites in signing.rs, rotation.rs, main.rs - Add success-path test for handle_trusted_server_discovery using StubJwksConfigStore - Fix test_parse_cookies_to_jar_empty typo (was emtpy) * Harden legacy config-store reads and align Fastly adapter stubs * Address storage review feedback * Resolved github-advanced-security bot problems * Address PR review feedback on platform abstraction layer - Make StoreName and StoreId inner fields private; From/AsRef provide all needed construction and access - Add #[deprecated] to GeoInfo::from_request with #[allow(deprecated)] at the three legacy call sites to track migration progress - Enumerate the six platform traits in the platform module doc comment - Extract backend_config_from_spec helper to remove duplicate BackendConfig construction in predict_name and ensure - Replace .into_iter().collect() with .to_vec() on secret plaintext bytes - Remove unused bytes dependency from trusted-server-adapter-fastly - Add comment on SecretStore::open clarifying it already returns Result (unlike ConfigStore::open which panics) * Add PR 4 design spec for secret store trait (read-only) * Clarify test scope and deferred branches in PR 4 spec * Add implementation plan for PR 4 secret store trait * Add test for get_secret_bytes open-failure path * Add NotImplemented tests for FastlyPlatformSecretStore write stubs * Inline StoreId binding and add section comment in write-stub tests * Remove plan * Add PR 6 design spec for backend and HTTP client traits * Address spec review findings on PR 6 design * Implement PlatformHttpClient and thread RuntimeServices through proxy layer - Add PlatformHttpClient trait with send(), send_async(), and select() methods - Add PlatformBackend trait with predict_name() and ensure() methods - Add PlatformResponse wrapper around EdgeZero HTTP responses - Add PlatformPendingRequest and PlatformSelectResult for auction fan-out - Thread RuntimeServices through IntegrationProxy::handle(), IntegrationRegistry::handle_proxy(), and all first-party proxy endpoints so handlers can reach the HTTP client - Add StubHttpClient and StubBackend test stubs with build_services_with_http_client helper - Add proxy_request_calls_platform_http_client_send integration test - Fix proxy_with_redirects to stay within 7-arg clippy limit via ProxyRequestHeaders struct - Document Body::Stream limitation in edge_request_to_fastly with warning log - Document intentional duplication of platform_response_to_fastly across proxy and orchestrator - Remove spec file (promoted to plan + implementation) * Address pr review findings * Resolve pr review findings * Add PR7 design spec for geo lookup + client info extract-once Documents the call site migration plan: five Fastly SDK extraction points in trusted-server-core replaced by RuntimeServices::client_info reads, following Phase 1 injection pattern from the EdgeZero migration design. * Fix spec review issues in PR7 design doc - Correct erroneous claim about generate_synthetic_id being called twice via DeviceInfo; it is called once (line 91 for fresh_id), DeviceInfo.ip is a separate req.get_client_ip_addr() call fixed independently - Add before/after snippet for handle_publisher_request call site in main.rs - Add noop_services import instruction for http_util.rs test module - Clarify _services rename (drop underscore, not add new param) in didomi.rs - Clarify nextjs #[allow(deprecated)] annotations are out of scope (different function) * Update PR7 spec to address all five agent review findings - Change RequestInfo::from_request signature to &ClientInfo (not &RuntimeServices) so prebid can call it with context.client_info - Scope SDK-call acceptance criteria to active non-deprecated code only - List all six AuctionContext construction sites including two production sites in orchestrator.rs and three test helpers in orchestrator/prebid - Add explicit warn-and-continue pattern for publisher.rs geo lookup - Correct testing table: formats.rs and endpoints.rs have no test modules; add orchestrator.rs and prebid.rs test helper update rows * Add PR7 implementation plan and address plan review findings Plan covers 6 tasks in compilation-safe order: AuctionContext struct change first, then from_request signature, then synthetic.rs cascade, then publisher geo, then didomi. Includes two new copy_headers unit tests (Some/None). Spec fixes: clarify injection pattern exceptions for &ClientInfo and Option; reword acceptance criterion to reflect that provider-layer reads flow through AuctionContext.client_info. * Fix three plan review findings and two open questions - Finding 1 (High): Add missing publisher.rs test call site at line ~695 for get_or_generate_synthetic_id — was omitted from Task 3 Step 6 - Finding 2 (Medium): Remove crate::geo::GeoInfo import from endpoints.rs rather than replacing it — type is not used by name after the change, keeping any import fails clippy -D warnings - Finding 3 (Low): Replace interactive git add -p in Task 6 with explicit file staging instruction - Open Q1: Add Task 2 step to update stale handle_publisher_request signature in auction/README.md - Open Q2: Add Task 2 step to update from_request doc comment to reflect ClientInfo-based TLS detection instead of Fastly SDK calls * Broaden two low-severity doc cleanup steps in PR7 plan - Step 7: cover all four stale Fastly-SDK-specific locations in http_util.rs (SPOOFABLE_FORWARDED_HEADERS doc, RequestInfo struct doc, from_request doc, detect_request_scheme doc) - Step 8: replace the whole routing snippet in auction/README.md, not just the one handle_publisher_request line — handle_auction and integration_registry.handle_proxy are also stale in that snippet * Fix two remaining low findings in PR7 plan - Add missing Location 2 (RequestInfo.scheme field doc, line ~67) to Step 7; renumber subsequent locations 3-5 - Replace &runtime_services with runtime_services in Step 5 and README snippet — runtime_services is already &RuntimeServices in route_request * Fix count drift in Step 7: four → five locations * Add client_info field to AuctionContext and fix all construction sites * Change RequestInfo::from_request to take &ClientInfo, thread services into handle_publisher_request * Add Task 2 follow-up coverage and README route fixes * Add services param to generate_synthetic_id, remove Fastly IP/geo calls in formats and endpoints * Revert premature publisher geo change from Task 3 * Replace deprecated GeoInfo::from_request in publisher.rs with services.geo().lookup() * Remove Fastly IP extraction from Didomi copy_headers, use ClientInfo instead * Move IpAddr import to test module level in didomi.rs * Apply rustfmt formatting to didomi.rs, publisher.rs, and synthetic.rs Fix multi-line function call style in didomi.rs, line-break wrapping in publisher.rs test, and import ordering in synthetic.rs test module. * Add test coverage for generate_synthetic_id with concrete client IP Adds noop_services_with_client_ip helper to test_support and a new test that verifies the client_ip path through generate_synthetic_id by asserting the HMAC differs when the IP changes. * Align geo lookup warn log format with codebase convention ({e} not {e:?}) * Apply Prettier formatting to PR7 plan and spec docs * Document content rewriting as platform-agnostic in platform module * Document html_processor as platform-agnostic * Document streaming_processor as platform-agnostic * Fix unresolved doc link: replace EdgeRequest with edgezero_core::http::Request * Add plan for content rewriting * Add plan for PR9: wire signing to store primitives * Add build_services_with_config_and_secret to test_support * Add FastlyManagementApiClient to adapter * Implement FastlyPlatformConfigStore and FastlyPlatformSecretStore write methods via management API Replace FastlyApiClient with FastlyManagementApiClient in the put/delete methods of FastlyPlatformConfigStore and the create/delete methods of FastlyPlatformSecretStore. Remove the now-unused FastlyApiClient import. * Add services to AuctionContext; remove deprecated from_config shim Thread RuntimeServices into AuctionContext so auction providers can access platform stores directly. Update PrebidAuctionProvider to use RequestSigner::from_services(context.services) instead of the now- removed from_config() shim. All construction sites and test helpers updated accordingly. This satisfies the final acceptance criterion of #490: no FastlyConfigStore/FastlySecretStore construction remains in the request_signing/ modules. * Fix rotate/delete atomicity, HTTP verb, idempotent deletes, and weak tests - Revert proxy.rs merge artifact: restore per-request allowed_domains at both redirect_is_permitted call sites; remove dead_code allow and stale comment — integration proxies defaulting to &[] get open mode again as documented - Drop unused trusted-server-js dep from adapter Cargo.toml - Fix check_response: gate body read behind error branch so 2xx paths do not buffer and discard the response body - Remove self-referential SECRET_UPSERT_METHOD test - Reorder write-cost doc so outbound HTTPS round-trip leads; handle-open caching noted as negligible - Refactor make_request to take fastly::http::Method; drop string match and unreachable arm; remove SECRET_UPSERT_METHOD const - Add SigningStoreIds named struct in endpoints.rs; update both call sites to destructure by name * Move logging initialization into Fastly adapter (PR 10) (#610) Move logging initialization into Fastly adapter (PR 10) (#610) - Reverted gratuitous _message rename and record.args() usage in logging.rs, returning to the idiomatic message parameter inside the fern format closure. - Refactored target_label to use .rsplit_once("::") rather than .split("::").last(). This provides a more explicit and robust way to extract the final module segment. - Expanded target_label test coverage to explicitly test edge cases such as inputs without :: separators, empty strings, and inputs with trailing ::. --- Cargo.lock | 4 + .../trusted-server-adapter-fastly/Cargo.toml | 6 +- .../src/logging.rs | 80 + .../trusted-server-adapter-fastly/src/main.rs | 46 +- .../src/management_api.rs | 417 ++++ .../src/platform.rs | 48 +- .../src/auction/endpoints.rs | 3 + crates/trusted-server-core/src/auction/mod.rs | 2 + .../src/auction/orchestrator.rs | 19 +- .../src/auction/test_support.rs | 26 + .../trusted-server-core/src/auction/types.rs | 3 + .../src/integrations/prebid.rs | 13 +- .../src/platform/test_support.rs | 189 +- .../src/request_signing/endpoints.rs | 558 +++-- .../src/request_signing/jwks.rs | 26 +- .../src/request_signing/mod.rs | 105 +- .../src/request_signing/rotation.rs | 854 ++++++-- .../src/request_signing/signing.rs | 252 +-- .../src/storage/api_client.rs | 291 --- crates/trusted-server-core/src/storage/mod.rs | 2 - ...31-pr9-wire-signing-to-store-primitives.md | 1835 +++++++++++++++++ .../2026-04-02-pr10-logging-initialization.md | 289 +++ ...4-02-pr10-logging-initialization-design.md | 69 + 23 files changed, 4337 insertions(+), 800 deletions(-) create mode 100644 crates/trusted-server-adapter-fastly/src/logging.rs create mode 100644 crates/trusted-server-adapter-fastly/src/management_api.rs create mode 100644 crates/trusted-server-core/src/auction/test_support.rs delete mode 100644 crates/trusted-server-core/src/storage/api_client.rs create mode 100644 docs/superpowers/plans/2026-03-31-pr9-wire-signing-to-store-primitives.md create mode 100644 docs/superpowers/plans/2026-04-02-pr10-logging-initialization.md create mode 100644 docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md diff --git a/Cargo.lock b/Cargo.lock index 3688be307..e06ac75e7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2695,6 +2695,7 @@ name = "trusted-server-adapter-fastly" version = "0.1.0" dependencies = [ "async-trait", + "base64", "chrono", "edgezero-adapter-fastly", "edgezero-core", @@ -2704,7 +2705,10 @@ dependencies = [ "futures", "log", "log-fastly", + "serde", + "serde_json", "trusted-server-core", + "urlencoding", ] [[package]] diff --git a/crates/trusted-server-adapter-fastly/Cargo.toml b/crates/trusted-server-adapter-fastly/Cargo.toml index 040c3620a..e483ea621 100644 --- a/crates/trusted-server-adapter-fastly/Cargo.toml +++ b/crates/trusted-server-adapter-fastly/Cargo.toml @@ -8,6 +8,7 @@ workspace = true [dependencies] async-trait = { workspace = true } +base64 = { workspace = true } chrono = { workspace = true } edgezero-adapter-fastly = { workspace = true, features = ["fastly"] } edgezero-core = { workspace = true } @@ -17,7 +18,10 @@ fern = { workspace = true } futures = { workspace = true } log = { workspace = true } log-fastly = { workspace = true } -trusted-server-core = {workspace = true} +serde = { workspace = true } +serde_json = { workspace = true } +trusted-server-core = { workspace = true } +urlencoding = { workspace = true } [dev-dependencies] edgezero-core = { workspace = true, features = ["test-utils"] } diff --git a/crates/trusted-server-adapter-fastly/src/logging.rs b/crates/trusted-server-adapter-fastly/src/logging.rs new file mode 100644 index 000000000..f110c63bd --- /dev/null +++ b/crates/trusted-server-adapter-fastly/src/logging.rs @@ -0,0 +1,80 @@ +//! Fastly-specific logger wiring for the trusted-server adapter. + +use chrono::{SecondsFormat, Utc}; +use log_fastly::Logger; + +/// Extracts the final `::` segment from a Rust module path for use as a log label. +/// +/// When the input has no `::` separator, returns the full target. When the +/// separator is at the trailing position (e.g. `"foo::"`), returns the head +/// segment (`"foo"`) to avoid emitting an empty label. +fn target_label(target: &str) -> &str { + match target.rsplit_once("::") { + Some((head, "")) => head, + Some((_, last)) => last, + None => target, + } +} + +/// Initialises the Fastly-backed `fern` logger and installs it as the global logger. +/// +/// Log records are forwarded to the `tslog` Fastly endpoint and echoed to stdout. +/// Each line is prefixed with an RFC 3339 timestamp, level, and the final segment +/// of the record's target module path. +/// +/// # Panics +/// +/// Panics if the Fastly logger cannot be built or if the global logger has already +/// been set. +pub(crate) fn init_logger() { + let logger = Logger::builder() + .default_endpoint("tslog") + .echo_stdout(true) + .max_level(log::LevelFilter::Info) + .build() + .expect("should build Logger"); + + fern::Dispatch::new() + .format(|out, message, record| { + out.finish(format_args!( + "{} {} [{}] {}", + Utc::now().to_rfc3339_opts(SecondsFormat::Millis, true), + record.level(), + target_label(record.target()), + message + )); + }) + .chain(Box::new(logger) as Box) + .apply() + .expect("should initialize logger"); +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn target_label_extracts_correct_segment() { + assert_eq!( + target_label("trusted_server_adapter_fastly::proxy"), + "proxy", + "should handle standard single-separator case" + ); + assert_eq!( + target_label("foo::bar::baz"), + "baz", + "should handle multiple separators" + ); + assert_eq!( + target_label("no_separators_here"), + "no_separators_here", + "should handle inputs without ::" + ); + assert_eq!(target_label(""), "", "should handle empty strings"); + assert_eq!( + target_label("trailing::"), + "trailing", + "should strip separator when trailing segment is empty" + ); + } +} diff --git a/crates/trusted-server-adapter-fastly/src/main.rs b/crates/trusted-server-adapter-fastly/src/main.rs index b0b24bf4c..30c6e3917 100644 --- a/crates/trusted-server-adapter-fastly/src/main.rs +++ b/crates/trusted-server-adapter-fastly/src/main.rs @@ -1,7 +1,6 @@ use error_stack::Report; use fastly::http::Method; use fastly::{Error, Request, Response}; -use log_fastly::Logger; use trusted_server_core::auction::endpoints::handle_auction; use trusted_server_core::auction::{build_orchestrator, AuctionOrchestrator}; @@ -41,11 +40,14 @@ use trusted_server_core::settings::Settings; use trusted_server_core::settings_data::get_settings; mod error; +mod logging; +mod management_api; mod platform; #[cfg(test)] mod route_tests; use crate::error::to_error_response; +use crate::logging::init_logger; use crate::platform::{build_runtime_services, UnavailableKvStore}; fn main() -> Result<(), Error> { @@ -274,14 +276,20 @@ async fn route_request( ), // Signature verification endpoint - (Method::POST, "/verify-signature") => (handle_verify_signature(settings, req), false), + (Method::POST, "/verify-signature") => ( + handle_verify_signature(settings, runtime_services, req), + false, + ), // Admin endpoints // Keep in sync with Settings::ADMIN_ENDPOINTS in crates/trusted-server-core/src/settings.rs - (Method::POST, "/_ts/admin/keys/rotate") => (handle_rotate_key(settings, req), false), - (Method::POST, "/_ts/admin/keys/deactivate") => { - (handle_deactivate_key(settings, req), false) + (Method::POST, "/_ts/admin/keys/rotate") => { + (handle_rotate_key(settings, runtime_services, req), false) } + (Method::POST, "/_ts/admin/keys/deactivate") => ( + handle_deactivate_key(settings, runtime_services, req), + false, + ), (Method::GET, "/_ts/api/v1/identify") => ( // Bot gate is intentionally write-only in this PR. `/identify` reads // remain gated by bearer auth + consent, even when request-classification @@ -308,6 +316,7 @@ async fn route_request( kv_graph.as_ref(), registry_ref, &ec_context, + runtime_services, req, ) .await, @@ -500,33 +509,6 @@ fn finalize_response(settings: &Settings, geo_info: Option<&GeoInfo>, response: } } -fn init_logger() { - let logger = Logger::builder() - .default_endpoint("tslog") - .echo_stdout(true) - .max_level(log::LevelFilter::Info) - .build() - .expect("should build Logger"); - - fern::Dispatch::new() - .format(|out, message, record| { - out.finish(format_args!( - "{} {} [{}] {}", - chrono::Local::now().to_rfc3339_opts(chrono::SecondsFormat::Millis, true), - record.level(), - record - .target() - .split("::") - .last() - .unwrap_or(record.target()), - message - )) - }) - .chain(Box::new(logger) as Box) - .apply() - .expect("should initialize logger"); -} - /// Constructs a `KvIdentityGraph` from settings, or returns 503 if the /// `ec_store` config is not set. fn require_identity_graph( diff --git a/crates/trusted-server-adapter-fastly/src/management_api.rs b/crates/trusted-server-adapter-fastly/src/management_api.rs new file mode 100644 index 000000000..92ae8e6c0 --- /dev/null +++ b/crates/trusted-server-adapter-fastly/src/management_api.rs @@ -0,0 +1,417 @@ +//! Fastly management API transport for store write operations. +//! +//! Provides [`FastlyManagementApiClient`], which wraps the Fastly REST +//! management API for write operations on config and secret stores. +//! Used by [`super::platform::FastlyPlatformConfigStore`] and +//! [`super::platform::FastlyPlatformSecretStore`] to back store write methods. +//! +//! # Credentials +//! +//! The Fastly API token is read from the `api-keys` secret store under the +//! `api_key` entry. The token must have config-store write and secret-store +//! write permissions only — no service-level admin or purge permissions. +//! +//! # Security +//! +//! Credential values are never logged. Log messages include store IDs and +//! operation names only. + +use std::io::Read; + +use base64::{engine::general_purpose, Engine as _}; +use error_stack::{Report, ResultExt}; +use fastly::http::{Method, StatusCode}; +use fastly::{Request, Response}; +use trusted_server_core::platform::{PlatformError, PlatformSecretStore, StoreName}; + +use crate::platform::FastlyPlatformSecretStore; + +const FASTLY_API_HOST: &str = "https://api.fastly.com"; +const API_KEYS_STORE: &str = "api-keys"; +const API_KEY_ENTRY: &str = "api_key"; +const ERROR_BODY_LIMIT: usize = 200; + +fn encode_path_segment(value: &str) -> String { + urlencoding::encode(value).into_owned() +} + +pub(crate) fn build_config_item_payload(value: &str) -> String { + format!("item_value={}", urlencoding::encode(value)) +} + +pub(crate) fn build_config_item_path(store_id: &str, key: &str) -> String { + format!( + "/resources/stores/config/{}/item/{}", + encode_path_segment(store_id), + encode_path_segment(key) + ) +} + +fn build_secret_collection_path(store_id: &str) -> String { + format!( + "/resources/stores/secret/{}/secrets", + encode_path_segment(store_id) + ) +} + +fn build_secret_path(store_id: &str, secret_name: &str) -> String { + format!( + "/resources/stores/secret/{}/secrets/{}", + encode_path_segment(store_id), + encode_path_segment(secret_name) + ) +} + +fn build_secret_payload(secret_name: &str, secret_value: &str) -> String { + serde_json::json!({ + "name": secret_name, + "secret": general_purpose::STANDARD.encode(secret_value.as_bytes()), + "method": "create_or_recreate", + }) + .to_string() +} + +fn truncate_error_body(body: &str) -> String { + body.trim().chars().take(ERROR_BODY_LIMIT).collect() +} + +fn check_response( + response: &mut Response, + error_kind: fn() -> PlatformError, + operation: &str, + entity_description: &str, + store_id: &str, +) -> Result<(), Report> { + if response.get_status().is_success() { + return Ok(()); + } + + let mut body = String::new(); + response + .get_body_mut() + .read_to_string(&mut body) + .change_context(error_kind())?; + + Err(Report::new(error_kind()).attach(format!( + "{} failed with HTTP {} - {} for {} in store '{}'", + operation, + response.get_status(), + truncate_error_body(&body), + entity_description, + store_id + ))) +} + +/// HTTP client for Fastly management API write operations. +/// +/// Backs the `put`/`delete` methods of +/// [`super::platform::FastlyPlatformConfigStore`] and the `create`/`delete` +/// methods of [`super::platform::FastlyPlatformSecretStore`]. +pub(crate) struct FastlyManagementApiClient { + api_key: String, + base_url: &'static str, + backend_name: String, +} + +impl FastlyManagementApiClient { + /// Initialize the client by reading the API token from the `api-keys` secret store. + /// + /// # Errors + /// + /// Returns [`PlatformError::Backend`] if the management API backend cannot + /// be registered, or [`PlatformError::SecretStore`] if the API key cannot + /// be read. + pub(crate) fn new() -> Result> { + use trusted_server_core::backend::BackendConfig; + + let backend_name = BackendConfig::from_url(FASTLY_API_HOST, true) + .change_context(PlatformError::Backend) + .attach("failed to register Fastly management API backend")?; + + let api_key = FastlyPlatformSecretStore + .get_string(&StoreName::from(API_KEYS_STORE), API_KEY_ENTRY) + .change_context(PlatformError::SecretStore) + .attach("failed to read Fastly API key from secret store")?; + + log::debug!("FastlyManagementApiClient: initialized for management API operations"); + + Ok(Self { + api_key, + base_url: FASTLY_API_HOST, + backend_name, + }) + } + + /// Construct a client from explicit components for tests. + #[cfg(test)] + pub(crate) fn with_components( + api_key: String, + base_url: &'static str, + backend_name: String, + ) -> Self { + Self { + api_key, + base_url, + backend_name, + } + } + + fn make_request( + &self, + method: Method, + path: &str, + body: Option, + content_type: &str, + error_kind: fn() -> PlatformError, + ) -> Result> { + let url = format!("{}{}", self.base_url, path); + + let mut request = Request::new(method, &url); + + request = request + .with_header("Fastly-Key", &self.api_key) + .with_header("Accept", "application/json"); + + if let Some(body_content) = body { + request = request + .with_header("Content-Type", content_type) + .with_body(body_content); + } + + request.send(&self.backend_name).map_err(|e| { + Report::new(error_kind()).attach(format!("management API request failed: {}", e)) + }) + } + + /// Update or create a config store item. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns a non-OK status. + pub(crate) fn update_config_item( + &self, + store_id: &str, + key: &str, + value: &str, + ) -> Result<(), Report> { + let path = build_config_item_path(store_id, key); + let payload = build_config_item_payload(value); + + let mut response = self.make_request( + Method::PUT, + &path, + Some(payload), + "application/x-www-form-urlencoded", + || PlatformError::ConfigStore, + )?; + + let entity_description = format!("key '{}'", key); + check_response( + &mut response, + || PlatformError::ConfigStore, + "config item update", + &entity_description, + store_id, + )?; + + log::debug!( + "FastlyManagementApiClient: updated config key '{}' in store '{}'", + key, + store_id + ); + Ok(()) + } + + /// Delete a config store item. + /// + /// Returns `Ok(())` if the item does not exist (404), so retries after + /// partial failures converge without error. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns an unexpected non-2xx status. + pub(crate) fn delete_config_item( + &self, + store_id: &str, + key: &str, + ) -> Result<(), Report> { + let path = build_config_item_path(store_id, key); + + let mut response = + self.make_request(Method::DELETE, &path, None, "application/json", || { + PlatformError::ConfigStore + })?; + + if response.get_status() == StatusCode::NOT_FOUND { + return Ok(()); + } + + let entity_description = format!("key '{}'", key); + check_response( + &mut response, + || PlatformError::ConfigStore, + "config item delete", + &entity_description, + store_id, + )?; + + log::debug!( + "FastlyManagementApiClient: deleted config key '{}' from store '{}'", + key, + store_id + ); + Ok(()) + } + + /// Create or overwrite a secret store entry. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns a non-OK status. + pub(crate) fn create_secret( + &self, + store_id: &str, + secret_name: &str, + secret_value: &str, + ) -> Result<(), Report> { + let path = build_secret_collection_path(store_id); + let payload = build_secret_payload(secret_name, secret_value); + + let mut response = self.make_request( + Method::POST, + &path, + Some(payload), + "application/json", + || PlatformError::SecretStore, + )?; + + let entity_description = format!("name '{}'", secret_name); + check_response( + &mut response, + || PlatformError::SecretStore, + "secret upsert", + &entity_description, + store_id, + )?; + + log::debug!( + "FastlyManagementApiClient: upserted secret '{}' in store '{}'", + secret_name, + store_id + ); + Ok(()) + } + + /// Delete a secret store entry. + /// + /// Returns `Ok(())` if the secret does not exist (404), so retries after + /// partial failures converge without error. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns an unexpected non-2xx status. + pub(crate) fn delete_secret( + &self, + store_id: &str, + secret_name: &str, + ) -> Result<(), Report> { + let path = build_secret_path(store_id, secret_name); + + let mut response = + self.make_request(Method::DELETE, &path, None, "application/json", || { + PlatformError::SecretStore + })?; + + if response.get_status() == StatusCode::NOT_FOUND { + return Ok(()); + } + + let entity_description = format!("name '{}'", secret_name); + check_response( + &mut response, + || PlatformError::SecretStore, + "secret delete", + &entity_description, + store_id, + )?; + + log::debug!( + "FastlyManagementApiClient: deleted secret '{}' from store '{}'", + secret_name, + store_id + ); + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn build_config_item_payload_url_encodes_reserved_characters() { + let payload = build_config_item_payload(r#"value with spaces + symbols &= {"kid":"a+b"}"#); + + assert_eq!( + payload, + "item_value=value%20with%20spaces%20%2B%20symbols%20%26%3D%20%7B%22kid%22%3A%22a%2Bb%22%7D", + "should URL-encode config item values in form payloads" + ); + } + + #[test] + fn build_config_item_path_url_encodes_store_id_and_key() { + let path = build_config_item_path("store/id", "current?kid+#1"); + + assert_eq!( + path, "/resources/stores/config/store%2Fid/item/current%3Fkid%2B%231", + "should percent-encode reserved path characters" + ); + } + + #[test] + fn build_secret_payload_base64_encodes_raw_secret_value() { + let payload = build_secret_payload("signing-key", "raw-secret-value"); + let json: serde_json::Value = + serde_json::from_str(&payload).expect("should serialize secret payload as JSON"); + + assert_eq!(json["name"], "signing-key"); + assert_eq!( + json["secret"], + base64::engine::general_purpose::STANDARD.encode("raw-secret-value"), + "should base64-encode the secret payload for the Fastly API" + ); + assert_eq!( + json["method"], "create_or_recreate", + "should request upsert semantics so re-rotation of the same kid succeeds" + ); + } + + #[test] + fn truncate_error_body_limits_length_after_trimming() { + let body = format!(" {} ", "a".repeat(250)); + + let truncated = truncate_error_body(&body); + + assert_eq!(truncated.len(), 200, "should cap error bodies at 200 chars"); + assert_eq!(truncated, "a".repeat(200), "should trim before truncating"); + } + + #[test] + fn create_secret_uses_secret_store_error_for_transport_failures() { + let client = FastlyManagementApiClient::with_components( + "test-api-key".to_string(), + FASTLY_API_HOST, + "missing-management-backend".to_string(), + ); + + let err = client + .create_secret("store-id", "secret-name", "secret-value") + .expect_err("should fail when the management API backend is unavailable"); + + assert!( + matches!(err.current_context(), &PlatformError::SecretStore), + "should classify secret transport failures as secret-store errors" + ); + } +} diff --git a/crates/trusted-server-adapter-fastly/src/platform.rs b/crates/trusted-server-adapter-fastly/src/platform.rs index 947a18a30..64640dbdc 100644 --- a/crates/trusted-server-adapter-fastly/src/platform.rs +++ b/crates/trusted-server-adapter-fastly/src/platform.rs @@ -16,15 +16,13 @@ use fastly::{ConfigStore, Request, SecretStore}; use trusted_server_core::backend::BackendConfig; use trusted_server_core::geo::geo_from_fastly; +pub(crate) use trusted_server_core::platform::UnavailableKvStore; use trusted_server_core::platform::{ ClientInfo, GeoInfo, PlatformBackend, PlatformBackendSpec, PlatformConfigStore, PlatformError, PlatformGeo, PlatformHttpClient, PlatformHttpRequest, PlatformKvStore, PlatformPendingRequest, PlatformResponse, PlatformSecretStore, PlatformSelectResult, RuntimeServices, StoreId, StoreName, }; -use trusted_server_core::storage::FastlyApiClient; - -pub(crate) use trusted_server_core::platform::UnavailableKvStore; // --------------------------------------------------------------------------- // FastlyPlatformConfigStore @@ -38,12 +36,11 @@ pub(crate) use trusted_server_core::platform::UnavailableKvStore; /// /// # Write cost /// -/// `put` and `delete` construct a [`FastlyApiClient`] on every call, which -/// opens the `"api-keys"` secret store to read the management API key. On -/// Fastly Compute, the SDK caches the open handle so repeated opens within a -/// single request are cheap. Callers that issue many writes in one request -/// should be aware that each call performs a synchronous outbound API -/// request to the Fastly management API. +/// `put` and `delete` each perform a synchronous outbound HTTPS request to the +/// Fastly management API (`api.fastly.com`). Callers that issue many writes in +/// one request pay one round-trip per call. The `"api-keys"` secret store is +/// opened per call to read the management token; the Fastly Compute SDK caches +/// the open handle so that cost is negligible. pub struct FastlyPlatformConfigStore; impl PlatformConfigStore for FastlyPlatformConfigStore { @@ -67,19 +64,13 @@ impl PlatformConfigStore for FastlyPlatformConfigStore { } fn put(&self, store_id: &StoreId, key: &str, value: &str) -> Result<(), Report> { - FastlyApiClient::new() - .change_context(PlatformError::ConfigStore) - .attach("failed to initialize Fastly API client for config store write")? - .update_config_item(store_id.as_ref(), key, value) - .change_context(PlatformError::ConfigStore) + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.update_config_item(store_id.as_ref(), key, value) } fn delete(&self, store_id: &StoreId, key: &str) -> Result<(), Report> { - FastlyApiClient::new() - .change_context(PlatformError::ConfigStore) - .attach("failed to initialize Fastly API client for config store delete")? - .delete_config_item(store_id.as_ref(), key) - .change_context(PlatformError::ConfigStore) + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.delete_config_item(store_id.as_ref(), key) } } @@ -95,8 +86,9 @@ impl PlatformConfigStore for FastlyPlatformConfigStore { /// /// # Write cost /// -/// `create` and `delete` have the same per-call [`FastlyApiClient`] cost -/// described on [`FastlyPlatformConfigStore`]. +/// `create` and `delete` have the same per-call +/// [`crate::management_api::FastlyManagementApiClient`] cost described on +/// [`FastlyPlatformConfigStore`]. pub struct FastlyPlatformSecretStore; impl PlatformSecretStore for FastlyPlatformSecretStore { @@ -138,19 +130,13 @@ impl PlatformSecretStore for FastlyPlatformSecretStore { name: &str, value: &str, ) -> Result<(), Report> { - FastlyApiClient::new() - .change_context(PlatformError::SecretStore) - .attach("failed to initialize Fastly API client for secret store create")? - .create_secret(store_id.as_ref(), name, value) - .change_context(PlatformError::SecretStore) + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.create_secret(store_id.as_ref(), name, value) } fn delete(&self, store_id: &StoreId, name: &str) -> Result<(), Report> { - FastlyApiClient::new() - .change_context(PlatformError::SecretStore) - .attach("failed to initialize Fastly API client for secret store delete")? - .delete_secret(store_id.as_ref(), name) - .change_context(PlatformError::SecretStore) + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.delete_secret(store_id.as_ref(), name) } } diff --git a/crates/trusted-server-core/src/auction/endpoints.rs b/crates/trusted-server-core/src/auction/endpoints.rs index 15076f65f..c8784ff28 100644 --- a/crates/trusted-server-core/src/auction/endpoints.rs +++ b/crates/trusted-server-core/src/auction/endpoints.rs @@ -15,6 +15,7 @@ use crate::ec::registry::PartnerRegistry; use crate::ec::EcContext; use crate::error::TrustedServerError; use crate::openrtb::{Eid, Uid}; +use crate::platform::RuntimeServices; use crate::settings::Settings; use super::formats::{convert_to_openrtb_response, convert_tsjs_to_auction_request}; @@ -40,6 +41,7 @@ pub async fn handle_auction( kv: Option<&KvIdentityGraph>, registry: Option<&PartnerRegistry>, ec_context: &EcContext, + services: &RuntimeServices, mut req: Request, ) -> Result> { // Parse request body @@ -104,6 +106,7 @@ pub async fn handle_auction( request: &req, timeout_ms: settings.auction.timeout_ms, provider_responses: None, + services, }; // Run the auction diff --git a/crates/trusted-server-core/src/auction/mod.rs b/crates/trusted-server-core/src/auction/mod.rs index 6fa9cc2d8..acf6d520f 100644 --- a/crates/trusted-server-core/src/auction/mod.rs +++ b/crates/trusted-server-core/src/auction/mod.rs @@ -19,6 +19,8 @@ pub mod endpoints; pub mod formats; pub mod orchestrator; pub mod provider; +#[cfg(test)] +pub(crate) mod test_support; pub mod types; pub use config::AuctionConfig; diff --git a/crates/trusted-server-core/src/auction/orchestrator.rs b/crates/trusted-server-core/src/auction/orchestrator.rs index d290534e7..aac37e4af 100644 --- a/crates/trusted-server-core/src/auction/orchestrator.rs +++ b/crates/trusted-server-core/src/auction/orchestrator.rs @@ -143,6 +143,7 @@ impl AuctionOrchestrator { request: context.request, timeout_ms: remaining_ms, provider_responses: Some(&provider_responses), + services: context.services, }; let start_time = Instant::now(); @@ -306,6 +307,7 @@ impl AuctionOrchestrator { request: context.request, timeout_ms: effective_timeout, provider_responses: context.provider_responses, + services: context.services, }; log::info!( @@ -594,8 +596,9 @@ impl OrchestrationResult { #[cfg(test)] mod tests { use crate::auction::config::AuctionConfig; + use crate::auction::test_support::create_test_auction_context; use crate::auction::types::{ - AdFormat, AdSlot, AuctionContext, AuctionRequest, Bid, MediaType, PublisherInfo, UserInfo, + AdFormat, AdSlot, AuctionRequest, Bid, MediaType, PublisherInfo, UserInfo, }; use crate::test_support::tests::crate_test_settings_str; use fastly::Request; @@ -650,18 +653,6 @@ mod tests { crate::settings::Settings::from_toml(&settings_str).expect("should parse test settings") } - fn create_test_context<'a>( - settings: &'a crate::settings::Settings, - req: &'a Request, - ) -> AuctionContext<'a> { - AuctionContext { - settings, - request: req, - timeout_ms: 2000, - provider_responses: None, - } - } - #[test] fn filters_winning_bids_below_floor() { let orchestrator = AuctionOrchestrator::new(AuctionConfig::default()); @@ -749,7 +740,7 @@ mod tests { let request = create_test_auction_request(); let settings = create_test_settings(); let req = Request::get("https://test.com/test"); - let context = create_test_context(&settings, &req); + let context = create_test_auction_context(&settings, &req, 2000); let result = orchestrator.run_auction(&request, &context).await; diff --git a/crates/trusted-server-core/src/auction/test_support.rs b/crates/trusted-server-core/src/auction/test_support.rs new file mode 100644 index 000000000..2c5b54383 --- /dev/null +++ b/crates/trusted-server-core/src/auction/test_support.rs @@ -0,0 +1,26 @@ +use std::sync::LazyLock; + +use fastly::Request; + +use super::AuctionContext; +use crate::platform::{test_support::noop_services, ClientInfo, RuntimeServices}; +use crate::settings::Settings; + +static TEST_SERVICES: LazyLock = LazyLock::new(noop_services); + +pub(crate) fn create_test_auction_context<'a>( + settings: &'a Settings, + request: &'a Request, + client_info: &'a ClientInfo, + timeout_ms: u32, +) -> AuctionContext<'a> { + let services: &'static RuntimeServices = &TEST_SERVICES; + AuctionContext { + settings, + request, + client_info, + timeout_ms, + provider_responses: None, + services, + } +} diff --git a/crates/trusted-server-core/src/auction/types.rs b/crates/trusted-server-core/src/auction/types.rs index 7934a4514..9b74d89ec 100644 --- a/crates/trusted-server-core/src/auction/types.rs +++ b/crates/trusted-server-core/src/auction/types.rs @@ -6,6 +6,7 @@ use std::collections::HashMap; use crate::auction::context::ContextValue; use crate::geo::GeoInfo; +use crate::platform::RuntimeServices; use crate::settings::Settings; /// Represents a unified auction request across all providers. @@ -113,6 +114,8 @@ pub struct AuctionContext<'a> { /// Provider responses from the bidding phase, used by mediators. /// This is `None` for regular bidders and `Some` when calling a mediator. pub provider_responses: Option<&'a [AuctionResponse]>, + /// Platform services (config store, secret store, etc.) for use by providers. + pub services: &'a RuntimeServices, } /// Response from a single auction provider. diff --git a/crates/trusted-server-core/src/integrations/prebid.rs b/crates/trusted-server-core/src/integrations/prebid.rs index d520b835d..d38c43a47 100644 --- a/crates/trusted-server-core/src/integrations/prebid.rs +++ b/crates/trusted-server-core/src/integrations/prebid.rs @@ -1089,7 +1089,7 @@ impl AuctionProvider for PrebidAuctionProvider { { if request_signing_config.enabled { let request_info = RequestInfo::from_request(context.request); - let signer = RequestSigner::from_config()?; + let signer = RequestSigner::from_services(context.services)?; let params = SigningParams::new(request.id.clone(), request_info.host, request_info.scheme); let signature = signer.sign_request(¶ms)?; @@ -1313,6 +1313,7 @@ pub fn register_auction_provider( #[cfg(test)] mod tests { use super::*; + use crate::auction::test_support::create_test_auction_context as shared_test_auction_context; use crate::auction::types::{ AdFormat, AdSlot, AuctionContext, AuctionRequest, DeviceInfo, PublisherInfo, UserInfo, }; @@ -1385,12 +1386,7 @@ mod tests { settings: &'a Settings, request: &'a Request, ) -> AuctionContext<'a> { - AuctionContext { - settings, - request, - timeout_ms: 1000, - provider_responses: None, - } + shared_test_auction_context(settings, request, 1000) } fn config_from_settings( @@ -2884,14 +2880,17 @@ server_url = "https://prebid.example" config: PrebidIntegrationConfig, request: &AuctionRequest, ) -> OpenRtbRequest { + use crate::platform::test_support::noop_services; let provider = PrebidAuctionProvider::new(config); let settings = make_settings(); let fastly_req = Request::new(Method::POST, "https://example.com/auction"); + let services = noop_services(); let context = AuctionContext { settings: &settings, request: &fastly_req, timeout_ms: 1000, provider_responses: None, + services: &services, }; provider.to_openrtb(request, &context, None) } diff --git a/crates/trusted-server-core/src/platform/test_support.rs b/crates/trusted-server-core/src/platform/test_support.rs index 1c167b99d..55a095a90 100644 --- a/crates/trusted-server-core/src/platform/test_support.rs +++ b/crates/trusted-server-core/src/platform/test_support.rs @@ -1,14 +1,18 @@ -use std::collections::VecDeque; +use std::collections::{HashMap, VecDeque}; use std::net::IpAddr; use std::sync::{Arc, Mutex}; +use base64::{engine::general_purpose, Engine as _}; +use ed25519_dalek::SigningKey; use error_stack::{Report, ResultExt}; +use rand::rngs::OsRng; use super::{ ClientInfo, GeoInfo, PlatformBackend, PlatformBackendSpec, PlatformConfigStore, PlatformError, PlatformGeo, PlatformHttpClient, PlatformHttpRequest, PlatformPendingRequest, PlatformResponse, PlatformSecretStore, PlatformSelectResult, RuntimeServices, StoreId, StoreName, }; +use crate::request_signing::{JWKS_STORE_NAME, SIGNING_STORE_NAME}; pub(crate) struct NoopConfigStore; @@ -56,6 +60,74 @@ impl PlatformSecretStore for NoopSecretStore { } } +pub(crate) struct HashMapConfigStore { + data: HashMap, +} + +impl HashMapConfigStore { + pub(crate) fn new(data: HashMap) -> Self { + Self { data } + } +} + +impl PlatformConfigStore for HashMapConfigStore { + fn get(&self, _store_name: &StoreName, key: &str) -> Result> { + self.data + .get(key) + .cloned() + .ok_or_else(|| Report::new(PlatformError::ConfigStore)) + } + + fn put( + &self, + _store_id: &StoreId, + _key: &str, + _value: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + + fn delete(&self, _store_id: &StoreId, _key: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } +} + +pub(crate) struct HashMapSecretStore { + data: HashMap>, +} + +impl HashMapSecretStore { + pub(crate) fn new(data: HashMap>) -> Self { + Self { data } + } +} + +impl PlatformSecretStore for HashMapSecretStore { + fn get_bytes( + &self, + _store_name: &StoreName, + key: &str, + ) -> Result, Report> { + self.data + .get(key) + .cloned() + .ok_or_else(|| Report::new(PlatformError::SecretStore)) + } + + fn create( + &self, + _store_id: &StoreId, + _name: &str, + _value: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + + fn delete(&self, _store_id: &StoreId, _name: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } +} + pub(crate) struct NoopBackend; impl PlatformBackend for NoopBackend { @@ -263,6 +335,51 @@ impl PlatformGeo for NoopGeo { } } +/// Build a [`RuntimeServices`] instance with a custom config store and a custom secret store. +/// +/// Use this when a test exercises code that reads from config AND secret stores, +/// such as `request_signing::signing` and `request_signing::rotation`. +pub(crate) fn build_services_with_config_and_secret( + config_store: impl PlatformConfigStore + 'static, + secret_store: impl PlatformSecretStore + 'static, +) -> RuntimeServices { + RuntimeServices::builder() + .config_store(Arc::new(config_store)) + .secret_store(Arc::new(secret_store)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(NoopBackend)) + .http_client(Arc::new(NoopHttpClient)) + .geo(Arc::new(NoopGeo)) + .client_info(ClientInfo { + client_ip: None, + tls_protocol: None, + tls_cipher: None, + }) + .build() +} + +pub(crate) fn build_request_signing_services() -> RuntimeServices { + let signing_key = SigningKey::generate(&mut OsRng); + let key_b64 = general_purpose::STANDARD.encode(signing_key.as_bytes()); + let x_b64 = general_purpose::URL_SAFE_NO_PAD.encode(signing_key.verifying_key().as_bytes()); + let jwk_json = format!( + r#"{{"kty":"OKP","crv":"Ed25519","x":"{}","kid":"test-kid","alg":"EdDSA"}}"#, + x_b64 + ); + + let mut config_data = HashMap::new(); + config_data.insert("current-kid".to_string(), "test-kid".to_string()); + config_data.insert("test-kid".to_string(), jwk_json); + + let mut secret_data = HashMap::new(); + secret_data.insert("test-kid".to_string(), key_b64.into_bytes()); + + build_services_with_config_and_secret( + HashMapConfigStore::new(config_data), + HashMapSecretStore::new(secret_data), + ) +} + pub(crate) fn build_services_with_config( config_store: impl PlatformConfigStore + 'static, ) -> RuntimeServices { @@ -442,4 +559,74 @@ mod tests { let name = stub.ensure(&spec).expect("should return a backend name"); assert_eq!(name, "stub-backend", "should return fixed name"); } + + #[test] + fn build_services_with_config_and_secret_uses_provided_stores() { + // Arrange: noop stores + let services = build_services_with_config_and_secret(NoopConfigStore, NoopSecretStore); + + // Act: both stores return Unsupported (confirming the injected impls are active) + let config_result = services.config_store().get(&StoreName::from("s"), "k"); + let secret_result = services + .secret_store() + .get_bytes(&StoreName::from("s"), "k"); + + assert!( + config_result.is_err(), + "should delegate to injected config store" + ); + assert!( + secret_result.is_err(), + "should delegate to injected secret store" + ); + } + + #[test] + fn hash_map_stores_return_preset_values() { + let mut config = HashMap::new(); + config.insert("current-kid".to_string(), "test-kid".to_string()); + + let mut secrets = HashMap::new(); + secrets.insert("test-kid".to_string(), b"secret-material".to_vec()); + + let services = build_services_with_config_and_secret( + HashMapConfigStore::new(config), + HashMapSecretStore::new(secrets), + ); + + assert_eq!( + services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .expect("should read current-kid from config test store"), + "test-kid" + ); + assert_eq!( + services + .secret_store() + .get_bytes(&SIGNING_STORE_NAME, "test-kid") + .expect("should read signing key bytes from secret test store"), + b"secret-material".to_vec() + ); + } + + #[test] + fn build_request_signing_services_provides_current_kid_and_signing_key() { + let services = build_request_signing_services(); + + let kid = services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .expect("should expose current-kid in config store"); + let key_bytes = services + .secret_store() + .get_bytes(&SIGNING_STORE_NAME, &kid) + .expect("should expose signing key bytes in secret store"); + + assert_eq!(kid, "test-kid", "should use the standard signing test kid"); + assert!( + !key_bytes.is_empty(), + "should provide key material for the current signing key" + ); + } } diff --git a/crates/trusted-server-core/src/request_signing/endpoints.rs b/crates/trusted-server-core/src/request_signing/endpoints.rs index 9b4b957e8..04638db04 100644 --- a/crates/trusted-server-core/src/request_signing/endpoints.rs +++ b/crates/trusted-server-core/src/request_signing/endpoints.rs @@ -7,7 +7,7 @@ use error_stack::{Report, ResultExt}; use fastly::{Request, Response}; use serde::{Deserialize, Serialize}; -use crate::error::TrustedServerError; +use crate::error::{IntoHttpResponse, TrustedServerError}; use crate::platform::RuntimeServices; use crate::request_signing::discovery::TrustedServerDiscovery; use crate::request_signing::rotation::KeyRotationManager; @@ -52,18 +52,27 @@ pub fn handle_trusted_server_discovery( .with_body(json)) } +/// JSON request body for the signature verification endpoint. #[derive(Debug, Deserialize, Serialize)] pub struct VerifySignatureRequest { + /// Canonical payload that was signed. pub payload: String, + /// Base64-encoded Ed25519 signature to verify. pub signature: String, + /// Key identifier used to look up the public JWK. pub kid: String, } +/// JSON response body for the signature verification endpoint. #[derive(Debug, Deserialize, Serialize)] pub struct VerifySignatureResponse { + /// Whether signature verification succeeded. pub verified: bool, + /// Key identifier that was used during verification. pub kid: String, + /// Human-readable verification result summary. pub message: String, + /// Error detail when verification fails unexpectedly. #[serde(skip_serializing_if = "Option::is_none")] pub error: Option, } @@ -73,9 +82,11 @@ pub struct VerifySignatureResponse { /// /// # Errors /// -/// Returns an error if the request body cannot be parsed as JSON or if verification fails. +/// Returns an error if the request body cannot be parsed as JSON or if the +/// response body cannot be serialized. pub fn handle_verify_signature( _settings: &Settings, + services: &RuntimeServices, mut req: Request, ) -> Result> { let body = req.take_body_str(); @@ -88,6 +99,7 @@ pub fn handle_verify_signature( verify_req.payload.as_bytes(), &verify_req.signature, &verify_req.kid, + services, ); let response = match verification_result { @@ -103,12 +115,15 @@ pub fn handle_verify_signature( message: "Signature verification failed".into(), error: Some("Invalid signature".into()), }, - Err(e) => VerifySignatureResponse { - verified: false, - kid: verify_req.kid, - message: "Verification error".into(), - error: Some(format!("{}", e)), - }, + Err(e) => { + log::warn!("signature verification failed: {e}"); + VerifySignatureResponse { + verified: false, + kid: verify_req.kid, + message: "Verification error".into(), + error: Some("internal verification error".into()), + } + } }; let response_json = serde_json::to_string(&response).map_err(|e| { @@ -122,42 +137,101 @@ pub fn handle_verify_signature( .with_body(response_json)) } +/// JSON request body for the key-rotation endpoint. #[derive(Debug, Deserialize, Serialize)] pub struct RotateKeyRequest { + /// Optional explicit key identifier for the new signing key. #[serde(skip_serializing_if = "Option::is_none")] pub kid: Option, } +/// JSON response body for the key-rotation endpoint. #[derive(Debug, Deserialize, Serialize)] pub struct RotateKeyResponse { + /// Whether the rotation operation succeeded. pub success: bool, + /// Human-readable summary of the rotation result. pub message: String, + /// Newly generated or supplied key identifier. pub new_kid: String, + /// Previously active key identifier, if one existed. pub previous_kid: Option, + /// Active key identifiers after the rotation completes. pub active_kids: Vec, + /// Public JWK associated with the newly active key. pub jwk: serde_json::Value, + /// Error detail when rotation fails. #[serde(skip_serializing_if = "Option::is_none")] pub error: Option, } -/// Rotates the current active kid by generating and saving a new one +struct SigningStoreIds<'a> { + config_store_id: &'a str, + secret_store_id: &'a str, +} + +const MAX_KID_LENGTH: usize = 128; + +fn signing_store_ids( + settings: &Settings, +) -> Result, Report> { + settings + .request_signing + .as_ref() + .map(|setting| SigningStoreIds { + config_store_id: setting.config_store_id.as_str(), + secret_store_id: setting.secret_store_id.as_str(), + }) + .ok_or_else(|| { + TrustedServerError::Configuration { + message: "missing signing storage configuration".to_string(), + } + .into() + }) +} + +fn validate_kid(kid: &str) -> Result<(), Report> { + if kid.is_empty() || kid.len() > MAX_KID_LENGTH { + return Err(Report::new(TrustedServerError::BadRequest { + message: format!("kid must be 1..={MAX_KID_LENGTH} characters"), + })); + } + + if !kid + .chars() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | ':')) + { + return Err(Report::new(TrustedServerError::BadRequest { + message: "kid must contain only ASCII alphanumerics, '-', '_', '.', ':'".into(), + })); + } + + Ok(()) +} + +/// Rotates the current active kid by generating and saving a new one. +/// +/// # Response contract +/// +/// Returns `200 OK` with `success: true` on success, `400 Bad Request` for an +/// invalid operator-supplied `kid`, or `500 Internal Server Error` when rotation +/// fails. Failure responses include `success: false` and a populated `error` +/// field. Unlike [`handle_verify_signature`], the error field contains internal +/// detail — this is intentional because this endpoint is auth-gated and +/// operator-facing only. /// /// # Errors /// -/// Returns an error if the request signing settings are missing, JSON parsing fails, or key rotation fails. +/// Returns an error if the request signing settings are missing or JSON parsing fails. pub fn handle_rotate_key( settings: &Settings, + services: &RuntimeServices, mut req: Request, ) -> Result> { - let (config_store_id, secret_store_id) = match &settings.request_signing { - Some(setting) => (&setting.config_store_id, &setting.secret_store_id), - None => { - return Err(TrustedServerError::Configuration { - message: "missing signing storage configuration".to_string(), - } - .into()); - } - }; + let SigningStoreIds { + config_store_id, + secret_store_id, + } = signing_store_ids(settings)?; let body = req.take_body_str(); let rotate_req: RotateKeyRequest = if body.is_empty() { @@ -168,13 +242,15 @@ pub fn handle_rotate_key( })? }; - let manager = KeyRotationManager::new(config_store_id, secret_store_id).change_context( - TrustedServerError::Configuration { - message: "failed to create KeyRotationManager".into(), - }, - )?; + let manager = KeyRotationManager::new(config_store_id, secret_store_id); + let validation_result = if let Some(kid) = rotate_req.kid.as_deref() { + validate_kid(kid) + } else { + Ok(()) + }; + let result = validation_result.and_then(|()| manager.rotate_key(services, rotate_req.kid)); - match manager.rotate_key(rotate_req.kid) { + match result { Ok(result) => { let jwk_value = serde_json::to_value(&result.jwk).map_err(|e| { Report::new(TrustedServerError::Configuration { @@ -203,6 +279,7 @@ pub fn handle_rotate_key( .with_body(response_json)) } Err(e) => { + let status = e.current_context().status_code(); let response = RotateKeyResponse { success: false, message: "Key rotation failed".to_string(), @@ -219,49 +296,64 @@ pub fn handle_rotate_key( }) })?; - Ok(Response::from_status(500) + Ok(Response::from_status(status) .with_content_type(fastly::mime::APPLICATION_JSON) .with_body(response_json)) } } } +/// JSON request body for the key-deactivation endpoint. #[derive(Debug, Deserialize, Serialize)] pub struct DeactivateKeyRequest { + /// Key identifier to deactivate or delete. pub kid: String, + /// Whether the key should be deleted from storage after deactivation. #[serde(default)] pub delete: bool, } +/// JSON response body for the key-deactivation endpoint. #[derive(Debug, Deserialize, Serialize)] pub struct DeactivateKeyResponse { + /// Whether the deactivation or deletion succeeded. pub success: bool, + /// Human-readable summary of the operation result. pub message: String, + /// Key identifier that was deactivated or deleted. pub deactivated_kid: String, + /// Whether the key was deleted from storage. pub deleted: bool, + /// Active key identifiers remaining after the operation. pub remaining_active_kids: Vec, + /// Error detail when the operation fails. #[serde(skip_serializing_if = "Option::is_none")] pub error: Option, } -/// Deactivates an active key +/// Deactivates or deletes an active signing key. +/// +/// # Response contract +/// +/// Returns `200 OK` with `success: true` on success, `400 Bad Request` for an +/// invalid operator-supplied `kid`, or `500 Internal Server Error` when +/// deactivation fails. Failure responses include `success: false` and a populated +/// `error` field. Like [`handle_rotate_key`] and unlike +/// [`handle_verify_signature`], the error field contains internal detail — this +/// is intentional because this endpoint is auth-gated and operator-facing only. /// /// # Errors /// -/// Returns an error if the request signing settings are missing, JSON parsing fails, or key deactivation fails. +/// Returns an error if the request signing settings are missing or JSON parsing fails. pub fn handle_deactivate_key( settings: &Settings, + services: &RuntimeServices, mut req: Request, ) -> Result> { - let (config_store_id, secret_store_id) = match &settings.request_signing { - Some(setting) => (&setting.config_store_id, &setting.secret_store_id), - None => { - return Err(TrustedServerError::Configuration { - message: "missing signing storage configuration".to_string(), - } - .into()); - } - }; + let SigningStoreIds { + config_store_id, + secret_store_id, + } = signing_store_ids(settings)?; let body = req.take_body_str(); let deactivate_req: DeactivateKeyRequest = @@ -269,21 +361,19 @@ pub fn handle_deactivate_key( message: "invalid JSON request body".into(), })?; - let manager = KeyRotationManager::new(config_store_id, secret_store_id).change_context( - TrustedServerError::Configuration { - message: "failed to create KeyRotationManager".into(), - }, - )?; + let manager = KeyRotationManager::new(config_store_id, secret_store_id); - let result = if deactivate_req.delete { - manager.delete_key(&deactivate_req.kid) - } else { - manager.deactivate_key(&deactivate_req.kid) - }; + let result = validate_kid(&deactivate_req.kid).and_then(|()| { + if deactivate_req.delete { + manager.delete_key(services, &deactivate_req.kid) + } else { + manager.deactivate_key(services, &deactivate_req.kid) + } + }); match result { Ok(()) => { - let remaining_keys = manager.list_active_keys().unwrap_or_else(|e| { + let remaining_keys = manager.list_active_keys(services).unwrap_or_else(|e| { log::warn!("failed to list active keys after deactivation: {}", e); vec![] }); @@ -312,6 +402,7 @@ pub fn handle_deactivate_key( .with_body(response_json)) } Err(e) => { + let status = e.current_context().status_code(); let response = DeactivateKeyResponse { success: false, message: if deactivate_req.delete { @@ -331,7 +422,7 @@ pub fn handle_deactivate_key( }) })?; - Ok(Response::from_status(500) + Ok(Response::from_status(status) .with_content_type(fastly::mime::APPLICATION_JSON) .with_body(response_json)) } @@ -340,10 +431,8 @@ pub fn handle_deactivate_key( #[cfg(test)] mod tests { - use error_stack::Report; - use crate::platform::{ - test_support::{build_services_with_config, noop_services}, + test_support::{build_request_signing_services, build_services_with_config, noop_services}, PlatformConfigStore, PlatformError, StoreId, StoreName, }; @@ -373,19 +462,19 @@ mod tests { Err(Report::new(PlatformError::Unsupported)) } } + #[test] fn test_handle_verify_signature_valid() { let settings = crate::test_support::tests::create_test_settings(); + let services = build_request_signing_services(); - // First, create a valid signature let payload = "test message"; - let signer = crate::request_signing::RequestSigner::from_config() - .expect("should create signer from config"); + let signer = crate::request_signing::RequestSigner::from_services(&services) + .expect("should create signer from services"); let signature = signer .sign(payload.as_bytes()) .expect("should sign payload"); - // Create verification request let verify_req = VerifySignatureRequest { payload: payload.to_string(), signature, @@ -396,9 +485,8 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/verify-signature"); req.set_body(body); - // Handle the request - let mut resp = - handle_verify_signature(&settings, req).expect("should handle verification request"); + let mut resp = handle_verify_signature(&settings, &services, req) + .expect("should handle verification request"); assert_eq!(resp.get_status(), StatusCode::OK); assert_eq!( resp.get_content_type(), @@ -406,12 +494,11 @@ mod tests { "should return application/json content type" ); - // Parse response let resp_body = resp.take_body_str(); let verify_resp: VerifySignatureResponse = serde_json::from_str(&resp_body).expect("should deserialize verify response"); - assert!(verify_resp.verified, "Signature should be verified"); + assert!(verify_resp.verified, "should verify a valid signature"); assert_eq!(verify_resp.kid, signer.kid); assert!(verify_resp.error.is_none()); } @@ -419,15 +506,15 @@ mod tests { #[test] fn test_handle_verify_signature_invalid() { let settings = crate::test_support::tests::create_test_settings(); - let signer = crate::request_signing::RequestSigner::from_config() - .expect("should create signer from config"); + let services = build_request_signing_services(); + + let signer = crate::request_signing::RequestSigner::from_services(&services) + .expect("should create signer from services"); - // Create a signature for a different payload let wrong_signature = signer .sign(b"different payload") .expect("should sign different payload"); - // Create request with signature that does not match the payload let verify_req = VerifySignatureRequest { payload: "test message".to_string(), signature: wrong_signature, @@ -438,9 +525,8 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/verify-signature"); req.set_body(body); - // Handle the request - let mut resp = - handle_verify_signature(&settings, req).expect("should handle verification request"); + let mut resp = handle_verify_signature(&settings, &services, req) + .expect("should handle verification request"); assert_eq!(resp.get_status(), StatusCode::OK); assert_eq!( resp.get_content_type(), @@ -448,16 +534,59 @@ mod tests { "should return application/json content type" ); - // Parse response let resp_body = resp.take_body_str(); let verify_resp: VerifySignatureResponse = serde_json::from_str(&resp_body).expect("should deserialize verify response"); - assert!(!verify_resp.verified, "Invalid signature should not verify"); + assert!( + !verify_resp.verified, + "should not verify an invalid signature" + ); assert_eq!(verify_resp.kid, signer.kid); assert!(verify_resp.error.is_some()); } + #[test] + fn test_handle_verify_signature_hides_internal_error_details() { + let settings = crate::test_support::tests::create_test_settings(); + + let verify_req = VerifySignatureRequest { + payload: "test message".to_string(), + signature: "any-signature".to_string(), + kid: "missing-kid".to_string(), + }; + + let body = serde_json::to_string(&verify_req).expect("should serialize verify request"); + let mut req = Request::new(Method::POST, "https://test.com/verify-signature"); + req.set_body(body); + + let services = noop_services(); + let mut resp = handle_verify_signature(&settings, &services, req) + .expect("should return a verification response for internal errors"); + + assert_eq!(resp.get_status(), StatusCode::OK, "should return 200 OK"); + + let resp_body = resp.take_body_str(); + let verify_resp: VerifySignatureResponse = + serde_json::from_str(&resp_body).expect("should deserialize verify response"); + + assert!( + !verify_resp.verified, + "should mark internal verification errors as unverified" + ); + assert_eq!(verify_resp.kid, "missing-kid"); + assert_eq!(verify_resp.message, "Verification error"); + assert_eq!( + verify_resp.error.as_deref(), + Some("internal verification error"), + "should return a generic error to unauthenticated callers" + ); + assert!( + !resp_body.contains("failed"), + "should not leak internal error details in the response body" + ); + } + #[test] fn test_handle_verify_signature_malformed_request() { let settings = crate::test_support::tests::create_test_settings(); @@ -465,8 +594,7 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/verify-signature"); req.set_body("not valid json"); - // Should return an error response - let result = handle_verify_signature(&settings, req); + let result = handle_verify_signature(&settings, &noop_services(), req); assert!(result.is_err(), "Malformed JSON should error"); } @@ -475,20 +603,27 @@ mod tests { let settings = crate::test_support::tests::create_test_settings(); let req = Request::new(Method::POST, "https://test.com/_ts/admin/keys/rotate"); - let result = handle_rotate_key(&settings, req); - match result { - Ok(mut resp) => { - let body = resp.take_body_str(); - let response: RotateKeyResponse = - serde_json::from_str(&body).expect("should deserialize rotate response"); - log::debug!( - "Rotation response: success={}, message={}", - response.success, - response.message - ); - } - Err(e) => log::debug!("Expected error in test environment: {}", e), - } + let mut resp = handle_rotate_key(&settings, &noop_services(), req) + .expect("should return a response even when stores are unavailable"); + + assert_eq!( + resp.get_status(), + StatusCode::INTERNAL_SERVER_ERROR, + "should return 500 when store writes fail" + ); + + let body = resp.take_body_str(); + let response: RotateKeyResponse = + serde_json::from_str(&body).expect("should deserialize rotate response"); + + assert!( + !response.success, + "should report failure when store writes fail" + ); + assert!( + response.error.is_some(), + "should include error detail in failure response" + ); } #[test] @@ -503,20 +638,27 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/_ts/admin/keys/rotate"); req.set_body(body_json); - let result = handle_rotate_key(&settings, req); - match result { - Ok(mut resp) => { - let body = resp.take_body_str(); - let response: RotateKeyResponse = - serde_json::from_str(&body).expect("should deserialize rotate response"); - log::debug!( - "Custom KID rotation: success={}, new_kid={}", - response.success, - response.new_kid - ); - } - Err(e) => log::debug!("Expected error in test environment: {}", e), - } + let mut resp = handle_rotate_key(&settings, &noop_services(), req) + .expect("should return a response even when stores are unavailable"); + + assert_eq!( + resp.get_status(), + StatusCode::INTERNAL_SERVER_ERROR, + "should return 500 when store writes fail" + ); + + let body = resp.take_body_str(); + let response: RotateKeyResponse = + serde_json::from_str(&body).expect("should deserialize rotate response"); + + assert!( + !response.success, + "should report failure when store writes fail" + ); + assert!( + response.error.is_some(), + "should include error detail in failure response" + ); } #[test] @@ -525,10 +667,48 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/_ts/admin/keys/rotate"); req.set_body("invalid json"); - let result = handle_rotate_key(&settings, req); + let result = handle_rotate_key(&settings, &noop_services(), req); assert!(result.is_err(), "Invalid JSON should return error"); } + #[test] + fn test_handle_rotate_key_rejects_invalid_kid() { + let settings = crate::test_support::tests::create_test_settings(); + + let req_body = RotateKeyRequest { + kid: Some("bad,kid".to_string()), + }; + + let body_json = serde_json::to_string(&req_body).expect("should serialize rotate request"); + let mut req = Request::new(Method::POST, "https://test.com/admin/keys/rotate"); + req.set_body(body_json); + + let mut resp = handle_rotate_key(&settings, &noop_services(), req) + .expect("should return a response for invalid kid"); + + assert_eq!( + resp.get_status(), + StatusCode::BAD_REQUEST, + "should reject malformed kid as a bad request" + ); + + let body = resp.take_body_str(); + let response: RotateKeyResponse = + serde_json::from_str(&body).expect("should deserialize rotate response"); + + assert!( + !response.success, + "should report failure when supplied kid is invalid" + ); + assert!( + response + .error + .as_deref() + .is_some_and(|error| error.contains("kid must contain only")), + "should explain the kid character restrictions" + ); + } + #[test] fn test_handle_deactivate_key_request() { let settings = crate::test_support::tests::create_test_settings(); @@ -543,20 +723,27 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/_ts/admin/keys/deactivate"); req.set_body(body_json); - let result = handle_deactivate_key(&settings, req); - match result { - Ok(mut resp) => { - let body = resp.take_body_str(); - let response: DeactivateKeyResponse = - serde_json::from_str(&body).expect("should deserialize deactivate response"); - log::debug!( - "Deactivate response: success={}, message={}", - response.success, - response.message - ); - } - Err(e) => log::debug!("Expected error in test environment: {}", e), - } + let mut resp = handle_deactivate_key(&settings, &noop_services(), req) + .expect("should return a response even when stores are unavailable"); + + assert_eq!( + resp.get_status(), + StatusCode::INTERNAL_SERVER_ERROR, + "should return 500 when active-kids cannot be read" + ); + + let body = resp.take_body_str(); + let response: DeactivateKeyResponse = + serde_json::from_str(&body).expect("should deserialize deactivate response"); + + assert!( + !response.success, + "should report failure when store reads fail" + ); + assert!( + response.error.is_some(), + "should include error detail in failure response" + ); } #[test] @@ -573,20 +760,31 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/_ts/admin/keys/deactivate"); req.set_body(body_json); - let result = handle_deactivate_key(&settings, req); - match result { - Ok(mut resp) => { - let body = resp.take_body_str(); - let response: DeactivateKeyResponse = - serde_json::from_str(&body).expect("should deserialize deactivate response"); - log::debug!( - "Delete response: success={}, deleted={}", - response.success, - response.deleted - ); - } - Err(e) => log::debug!("Expected error in test environment: {}", e), - } + let mut resp = handle_deactivate_key(&settings, &noop_services(), req) + .expect("should return a response even when stores are unavailable"); + + assert_eq!( + resp.get_status(), + StatusCode::INTERNAL_SERVER_ERROR, + "should return 500 when active-kids cannot be read" + ); + + let body = resp.take_body_str(); + let response: DeactivateKeyResponse = + serde_json::from_str(&body).expect("should deserialize deactivate response"); + + assert!( + !response.success, + "should report failure when store reads fail" + ); + assert!( + !response.deleted, + "should not report deletion when the operation failed" + ); + assert!( + response.error.is_some(), + "should include error detail in failure response" + ); } #[test] @@ -595,10 +793,76 @@ mod tests { let mut req = Request::new(Method::POST, "https://test.com/_ts/admin/keys/deactivate"); req.set_body("invalid json"); - let result = handle_deactivate_key(&settings, req); + let result = handle_deactivate_key(&settings, &noop_services(), req); assert!(result.is_err(), "Invalid JSON should return error"); } + #[test] + fn test_handle_deactivate_key_rejects_invalid_kid() { + let settings = crate::test_support::tests::create_test_settings(); + + let req_body = DeactivateKeyRequest { + kid: "bad kid".to_string(), + delete: false, + }; + + let body_json = + serde_json::to_string(&req_body).expect("should serialize deactivate request"); + let mut req = Request::new(Method::POST, "https://test.com/admin/keys/deactivate"); + req.set_body(body_json); + + let mut resp = handle_deactivate_key(&settings, &noop_services(), req) + .expect("should return a response for invalid kid"); + + assert_eq!( + resp.get_status(), + StatusCode::BAD_REQUEST, + "should reject malformed kid as a bad request" + ); + + let body = resp.take_body_str(); + let response: DeactivateKeyResponse = + serde_json::from_str(&body).expect("should deserialize deactivate response"); + + assert!( + !response.success, + "should report failure when supplied kid is invalid" + ); + assert!( + response + .error + .as_deref() + .is_some_and(|error| error.contains("kid must contain only")), + "should explain the kid character restrictions" + ); + } + + #[test] + fn validate_kid_accepts_allowed_operator_supplied_ids() { + validate_kid("azAZ09-_.:").expect("should accept allowed kid characters"); + } + + #[test] + fn validate_kid_rejects_empty_ids() { + let result = validate_kid(""); + + assert!(result.is_err(), "should reject empty kid values"); + } + + #[test] + fn validate_kid_rejects_overlong_ids() { + let result = validate_kid(&"a".repeat(129)); + + assert!(result.is_err(), "should reject kids longer than 128 chars"); + } + + #[test] + fn validate_kid_rejects_csv_separator() { + let result = validate_kid("kid-a,kid-b"); + + assert!(result.is_err(), "should reject commas in kid values"); + } + #[test] fn test_rotate_key_request_deserialization() { let json = r#"{"kid":"custom-key"}"#; @@ -624,32 +888,14 @@ mod tests { "https://test.com/.well-known/trusted-server.json", ); - let services = noop_services(); - let result = handle_trusted_server_discovery(&settings, &services, req); - match result { - Ok(mut resp) => { - assert_eq!(resp.get_status(), StatusCode::OK); - assert_eq!( - resp.get_content_type(), - Some(fastly::mime::APPLICATION_JSON), - "should return application/json content type" - ); - let body = resp.take_body_str(); - - // Parse the discovery document - let discovery: serde_json::Value = - serde_json::from_str(&body).expect("should parse discovery document"); - - // Verify structure - only version and jwks - assert_eq!(discovery["version"], "1.0"); - assert!(discovery["jwks"].is_object()); - - // Verify no extra fields - assert!(discovery.get("endpoints").is_none()); - assert!(discovery.get("capabilities").is_none()); - } - Err(e) => log::debug!("Expected error in test environment: {}", e), - } + // noop_services() config store always returns Err, so the discovery + // handler propagates the error rather than absorbing it into a 500. + let result = handle_trusted_server_discovery(&settings, &noop_services(), req); + + assert!( + result.is_err(), + "should propagate store errors when JWKS cannot be retrieved" + ); } #[test] diff --git a/crates/trusted-server-core/src/request_signing/jwks.rs b/crates/trusted-server-core/src/request_signing/jwks.rs index 5c4dda94e..8d2066390 100644 --- a/crates/trusted-server-core/src/request_signing/jwks.rs +++ b/crates/trusted-server-core/src/request_signing/jwks.rs @@ -3,8 +3,6 @@ //! This module provides functionality for generating, storing, and retrieving //! Ed25519 keypairs in JWK format for request signing. -use std::sync::LazyLock; - use ed25519_dalek::{SigningKey, VerifyingKey}; use error_stack::{Report, ResultExt}; use jose_jwk::{ @@ -14,11 +12,8 @@ use jose_jwk::{ use rand::rngs::OsRng; use crate::error::TrustedServerError; -use crate::platform::{RuntimeServices, StoreName}; -use crate::request_signing::JWKS_CONFIG_STORE_NAME; - -static JWKS_STORE_NAME: LazyLock = - LazyLock::new(|| StoreName::from(JWKS_CONFIG_STORE_NAME)); +use crate::platform::RuntimeServices; +use crate::request_signing::{read_active_kids, JWKS_STORE_NAME}; /// An Ed25519 keypair used for request signing. pub struct Keypair { @@ -75,25 +70,12 @@ impl Keypair { /// cannot be read. The underlying [`crate::platform::PlatformError`] is /// preserved as context in the error chain. pub fn get_active_jwks(services: &RuntimeServices) -> Result> { - let active_kids_str = services - .config_store() - .get(&JWKS_STORE_NAME, "active-kids") - .change_context(TrustedServerError::Configuration { - message: "failed to read active-kids from config store".into(), - }) - .attach("while fetching active kids list")?; - - let active_kids: Vec<&str> = active_kids_str - .split(',') - .map(str::trim) - .filter(|s| !s.is_empty()) - .collect(); - + let active_kids = read_active_kids(services)?; let mut jwks = Vec::new(); for kid in active_kids { let jwk = services .config_store() - .get(&JWKS_STORE_NAME, kid) + .get(&JWKS_STORE_NAME, &kid) .change_context(TrustedServerError::Configuration { message: format!("failed to get JWK for kid: {}", kid), })?; diff --git a/crates/trusted-server-core/src/request_signing/mod.rs b/crates/trusted-server-core/src/request_signing/mod.rs index 415079404..d5d0e79ce 100644 --- a/crates/trusted-server-core/src/request_signing/mod.rs +++ b/crates/trusted-server-core/src/request_signing/mod.rs @@ -5,15 +5,27 @@ //! //! # Store names vs store IDs //! -//! Fastly stores have two identifiers: +//! Platform stores have two identifiers: //! //! - **Store name** ([`JWKS_CONFIG_STORE_NAME`], [`SIGNING_SECRET_STORE_NAME`]): -//! used at the edge for reads via `ConfigStore::open` / `SecretStore::open`. -//! These are configured in `fastly.toml`. +//! used for runtime reads via [`crate::platform::PlatformConfigStore::get`] +//! and [`crate::platform::PlatformSecretStore::get_bytes`] through +//! [`crate::platform::RuntimeServices`]. These names are configured in +//! `fastly.toml` for the Fastly adapter. //! -//! - **Store ID** (`RequestSigning::config_store_id`, `RequestSigning::secret_store_id`): -//! used by the Fastly management API for writes (creating, updating, and -//! deleting items). These are set in `trusted-server.toml`. +//! - **Store ID**: used for write operations via +//! [`crate::platform::PlatformConfigStore::put`] / +//! [`crate::platform::PlatformConfigStore::delete`] and +//! [`crate::platform::PlatformSecretStore::create`] / +//! [`crate::platform::PlatformSecretStore::delete`]. These identifiers come +//! from the request-signing settings in `trusted-server.toml`. + +use std::sync::LazyLock; + +use error_stack::{Report, ResultExt}; + +use crate::error::TrustedServerError; +use crate::platform::{RuntimeServices, StoreName}; pub mod discovery; pub mod endpoints; @@ -21,20 +33,97 @@ pub mod jwks; pub mod rotation; pub mod signing; -/// Config store name for JWKS public keys (edge reads via `ConfigStore::open`). +/// Config store name for JWKS public keys used by runtime read operations. /// /// This must match the store name declared in `fastly.toml` under /// `[local_server.config_stores]`. pub const JWKS_CONFIG_STORE_NAME: &str = "jwks_store"; -/// Secret store name for Ed25519 signing keys (edge reads via `SecretStore::open`). +/// Secret store name for Ed25519 signing keys used by runtime read operations. /// /// This must match the store name declared in `fastly.toml` under /// `[local_server.secret_stores]`. pub const SIGNING_SECRET_STORE_NAME: &str = "signing_keys"; +/// Lazily constructed [`StoreName`] for JWKS config-store reads. +pub(crate) static JWKS_STORE_NAME: LazyLock = + LazyLock::new(|| StoreName::from(JWKS_CONFIG_STORE_NAME)); + +/// Lazily constructed [`StoreName`] for signing-key secret-store reads. +pub(crate) static SIGNING_STORE_NAME: LazyLock = + LazyLock::new(|| StoreName::from(SIGNING_SECRET_STORE_NAME)); + +fn parse_active_kids(active_kids: &str) -> Vec { + active_kids + .split(',') + .map(|kid| kid.trim().to_string()) + .filter(|kid| !kid.is_empty()) + .collect() +} + +fn read_active_kids(services: &RuntimeServices) -> Result, Report> { + services + .config_store() + .get(&JWKS_STORE_NAME, "active-kids") + .change_context(TrustedServerError::Configuration { + message: "failed to read active-kids from config store".into(), + }) + .attach("while fetching active kids list") + .map(|active_kids| parse_active_kids(&active_kids)) +} + pub use discovery::*; pub use endpoints::*; pub use jwks::*; pub use rotation::*; pub use signing::*; + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parse_active_kids_splits_comma_separated_kids() { + let result = parse_active_kids("kid-a,kid-b,kid-c"); + assert_eq!(result, vec!["kid-a", "kid-b", "kid-c"]); + } + + #[test] + fn parse_active_kids_trims_whitespace_around_each_kid() { + let result = parse_active_kids(" kid-a , kid-b "); + assert_eq!(result, vec!["kid-a", "kid-b"]); + } + + #[test] + fn parse_active_kids_skips_empty_segments() { + let result = parse_active_kids("kid-a,,kid-b"); + assert_eq!(result, vec!["kid-a", "kid-b"]); + } + + #[test] + fn parse_active_kids_skips_whitespace_only_segments() { + let result = parse_active_kids(" kid-a , , kid-b "); + assert_eq!(result, vec!["kid-a", "kid-b"]); + } + + #[test] + fn parse_active_kids_returns_empty_for_empty_string() { + let result = parse_active_kids(""); + assert!(result.is_empty(), "should return no kids for empty input"); + } + + #[test] + fn parse_active_kids_returns_empty_for_only_commas() { + let result = parse_active_kids(",,,"); + assert!( + result.is_empty(), + "should return no kids when input is only commas" + ); + } + + #[test] + fn parse_active_kids_handles_single_kid() { + let result = parse_active_kids("only-kid"); + assert_eq!(result, vec!["only-kid"]); + } +} diff --git a/crates/trusted-server-core/src/request_signing/rotation.rs b/crates/trusted-server-core/src/request_signing/rotation.rs index 252ccd590..f78ac882f 100644 --- a/crates/trusted-server-core/src/request_signing/rotation.rs +++ b/crates/trusted-server-core/src/request_signing/rotation.rs @@ -1,66 +1,58 @@ //! Key rotation management for request signing. //! -//! This module provides functionality for rotating signing keys, managing key lifecycle, -//! and storing keys in Fastly Config and Secret stores. +//! This module provides functionality for rotating signing keys, managing key +//! lifecycle, and storing keys via platform store primitives through +//! [`RuntimeServices`]. use base64::{engine::general_purpose, Engine}; +use chrono::Utc; use ed25519_dalek::SigningKey; use error_stack::{Report, ResultExt}; use jose_jwk::Jwk; +use uuid::Uuid; +use super::{read_active_kids, Keypair}; use crate::error::TrustedServerError; -use crate::request_signing::JWKS_CONFIG_STORE_NAME; -use crate::storage::{FastlyApiClient, FastlyConfigStore}; - -use super::Keypair; +use crate::platform::{RuntimeServices, StoreId}; +use crate::request_signing::JWKS_STORE_NAME; +/// Result of a key rotation operation. #[derive(Debug, Clone)] pub struct KeyRotationResult { + /// Newly generated or supplied key identifier. pub new_kid: String, + /// Previously active key identifier, if one existed. pub previous_kid: Option, + /// Active key identifiers after rotation completes. pub active_kids: Vec, + /// Public JWK associated with the newly active key. pub jwk: Jwk, } -#[allow(deprecated)] +/// Manages signing key lifecycle using platform store primitives. +/// +/// Reads use the edge-visible store name ([`super::JWKS_CONFIG_STORE_NAME`]). +/// Writes use the management API store identifiers supplied at construction. pub struct KeyRotationManager { - /// Edge-side config store for reading JWKS (uses store name). - config_store: FastlyConfigStore, - /// Management API client for writing to stores (uses store IDs). - api_client: FastlyApiClient, - /// Fastly API store ID for config store writes. - config_store_id: String, - /// Fastly API store ID for secret store writes. - secret_store_id: String, + /// Management API store ID for config store writes. + config_store_id: StoreId, + /// Management API store ID for secret store writes. + secret_store_id: StoreId, } -#[allow(deprecated)] impl KeyRotationManager { /// Creates a new key rotation manager. /// - /// The `config_store_id` and `secret_store_id` are Fastly management API + /// The `config_store_id` and `secret_store_id` are platform management API /// identifiers used for write operations. Edge reads use the store names - /// defined in [`JWKS_CONFIG_STORE_NAME`] and [`crate::request_signing::SIGNING_SECRET_STORE_NAME`]. - /// - /// # Errors - /// - /// Returns an error if the API client cannot be initialized. - pub fn new( - config_store_id: impl Into, - secret_store_id: impl Into, - ) -> Result> { - let config_store_id = config_store_id.into(); - let secret_store_id = secret_store_id.into(); - - let config_store = FastlyConfigStore::new(JWKS_CONFIG_STORE_NAME); - let api_client = FastlyApiClient::new()?; - - Ok(Self { - config_store, - api_client, - config_store_id, - secret_store_id, - }) + /// defined in [`super::JWKS_CONFIG_STORE_NAME`] and + /// [`crate::request_signing::SIGNING_SECRET_STORE_NAME`]. + #[must_use] + pub fn new(config_store_id: &str, secret_store_id: &str) -> Self { + Self { + config_store_id: StoreId::from(config_store_id), + secret_store_id: StoreId::from(secret_store_id), + } } /// Rotates the signing key by generating a new keypair and storing it. @@ -70,24 +62,83 @@ impl KeyRotationManager { /// Returns an error if key storage or update operations fail. pub fn rotate_key( &self, + services: &RuntimeServices, kid: Option, ) -> Result> { - let new_kid = kid.unwrap_or_else(generate_date_based_kid); + let previous_kid = services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .ok(); + let active_kids = read_active_kids(services).unwrap_or_default(); + let new_kid = match kid { + Some(kid) => { + if self.key_exists(services, &kid, &active_kids) { + return Err(Report::new(TrustedServerError::Configuration { + message: format!("kid '{}' already exists; choose a unique kid", kid), + })); + } + kid + } + None => self.generate_unique_date_based_kid(services, &active_kids), + }; let keypair = Keypair::generate(); let jwk = keypair.get_jwk(new_kid.clone()); - let previous_kid = self.config_store.get("current-kid").ok(); - self.store_private_key(&new_kid, &keypair.signing_key)?; - self.store_public_jwk(&new_kid, &jwk)?; + // Step 1: write private key. Nothing to roll back on failure. + self.store_private_key(services, &new_kid, &keypair.signing_key)?; + + // Step 2: write public JWK. Roll back the private key on failure so no + // orphaned key material is left in the secret store. + if let Err(err) = self.store_public_jwk(services, &new_kid, &jwk) { + if let Err(rollback_err) = services + .secret_store() + .delete(&self.secret_store_id, &new_kid) + { + log::warn!( + "rotate_key: rollback of private key '{}' failed after JWK write error: {}", + new_kid, + rollback_err + ); + } + return Err(err); + } - let active_kids = match &previous_kid { - Some(prev) if prev != &new_kid => vec![prev.clone(), new_kid.clone()], - _ => vec![new_kid.clone()], - }; + let mut active_kids = active_kids; + if !active_kids.iter().any(|kid| kid == &new_kid) { + active_kids.push(new_kid.clone()); + } - self.update_current_kid(&new_kid)?; - self.update_active_kids(&active_kids)?; + // Step 3: publish the new kid in active-kids BEFORE flipping current-kid. + // Roll back both artifacts on failure so the new kid never appears in JWKS + // without a reachable private key. + if let Err(err) = self.update_active_kids(services, &active_kids) { + if let Err(rollback_err) = services + .config_store() + .delete(&self.config_store_id, &new_kid) + { + log::warn!( + "rotate_key: rollback of JWK '{}' failed after active-kids write error: {}", + new_kid, + rollback_err + ); + } + if let Err(rollback_err) = services + .secret_store() + .delete(&self.secret_store_id, &new_kid) + { + log::warn!( + "rotate_key: rollback of private key '{}' failed after active-kids write error: {}", + new_kid, + rollback_err + ); + } + return Err(err); + } + + // Step 4: flip current-kid last. A failure here leaves the old kid still + // active and the new kid visible in JWKS but unused — a recoverable state. + self.update_current_kid(services, &new_kid)?; Ok(KeyRotationResult { new_kid, @@ -97,50 +148,88 @@ impl KeyRotationManager { }) } + fn key_exists(&self, services: &RuntimeServices, kid: &str, active_kids: &[String]) -> bool { + active_kids.iter().any(|active_kid| active_kid == kid) + || services.config_store().get(&JWKS_STORE_NAME, kid).is_ok() + } + + fn generate_unique_date_based_kid( + &self, + services: &RuntimeServices, + active_kids: &[String], + ) -> String { + let base_kid = generate_date_based_kid(); + if !self.key_exists(services, &base_kid, active_kids) { + return base_kid; + } + + format!("{base_kid}-{}", Uuid::new_v4().simple()) + } + fn store_private_key( &self, + services: &RuntimeServices, kid: &str, signing_key: &SigningKey, ) -> Result<(), Report> { - let key_bytes = signing_key.as_bytes(); - let key_b64 = general_purpose::STANDARD.encode(key_bytes); - - self.api_client - .create_secret(&self.secret_store_id, kid, &key_b64) + // The platform secret-store write interface is string-based, so signing + // keys are persisted as base64 text. The Fastly adapter applies its own + // transport-level base64 encoding when calling the management API. + let key_b64 = general_purpose::STANDARD.encode(signing_key.as_bytes()); + + services + .secret_store() + .create(&self.secret_store_id, kid, &key_b64) .change_context(TrustedServerError::Configuration { - message: format!("Failed to store private key '{}'", kid), + message: format!("failed to store private key '{}'", kid), }) } - fn store_public_jwk(&self, kid: &str, jwk: &Jwk) -> Result<(), Report> { + fn store_public_jwk( + &self, + services: &RuntimeServices, + kid: &str, + jwk: &Jwk, + ) -> Result<(), Report> { let jwk_json = serde_json::to_string(jwk).map_err(|e| { Report::new(TrustedServerError::Configuration { - message: format!("Failed to serialize JWK: {}", e), + message: format!("failed to serialize JWK: {}", e), }) })?; - self.api_client - .update_config_item(&self.config_store_id, kid, &jwk_json) + services + .config_store() + .put(&self.config_store_id, kid, &jwk_json) .change_context(TrustedServerError::Configuration { - message: format!("Failed to store public JWK '{}'", kid), + message: format!("failed to store public JWK '{}'", kid), }) } - fn update_current_kid(&self, kid: &str) -> Result<(), Report> { - self.api_client - .update_config_item(&self.config_store_id, "current-kid", kid) + fn update_current_kid( + &self, + services: &RuntimeServices, + kid: &str, + ) -> Result<(), Report> { + services + .config_store() + .put(&self.config_store_id, "current-kid", kid) .change_context(TrustedServerError::Configuration { - message: "Failed to update current-kid".into(), + message: "failed to update current-kid".into(), }) } - fn update_active_kids(&self, active_kids: &[String]) -> Result<(), Report> { + fn update_active_kids( + &self, + services: &RuntimeServices, + active_kids: &[String], + ) -> Result<(), Report> { let active_kids_str = active_kids.join(","); - self.api_client - .update_config_item(&self.config_store_id, "active-kids", &active_kids_str) + services + .config_store() + .put(&self.config_store_id, "active-kids", &active_kids_str) .change_context(TrustedServerError::Configuration { - message: "Failed to update active-kids".into(), + message: "failed to update active-kids".into(), }) } @@ -149,16 +238,11 @@ impl KeyRotationManager { /// # Errors /// /// Returns an error if the active keys cannot be retrieved from the config store. - pub fn list_active_keys(&self) -> Result, Report> { - let active_kids_str = self.config_store.get("active-kids")?; - - let active_kids: Vec = active_kids_str - .split(',') - .map(|s| s.trim().to_string()) - .filter(|s| !s.is_empty()) - .collect(); - - Ok(active_kids) + pub fn list_active_keys( + &self, + services: &RuntimeServices, + ) -> Result, Report> { + read_active_kids(services) } /// Deactivates a key by removing it from the active keys list. @@ -166,18 +250,23 @@ impl KeyRotationManager { /// # Errors /// /// Returns an error if this would deactivate the last active key, or if the update fails. - pub fn deactivate_key(&self, kid: &str) -> Result<(), Report> { - let mut active_kids = self.list_active_keys()?; + pub fn deactivate_key( + &self, + services: &RuntimeServices, + kid: &str, + ) -> Result<(), Report> { + self.ensure_not_current_key(services, kid, "deactivate")?; + let mut active_kids = self.list_active_keys(services)?; active_kids.retain(|k| k != kid); if active_kids.is_empty() { return Err(Report::new(TrustedServerError::Configuration { - message: "Cannot deactivate the last active key".into(), + message: "cannot deactivate the last active key".into(), })); } - self.update_active_kids(&active_kids) + self.update_active_kids(services, &active_kids) } /// Deletes a key by deactivating it and removing it from storage. @@ -185,79 +274,466 @@ impl KeyRotationManager { /// # Errors /// /// Returns an error if deactivation fails or if the key cannot be deleted from storage. - pub fn delete_key(&self, kid: &str) -> Result<(), Report> { - self.deactivate_key(kid)?; - - self.api_client - .delete_config_item(&self.config_store_id, kid) + pub fn delete_key( + &self, + services: &RuntimeServices, + kid: &str, + ) -> Result<(), Report> { + self.ensure_not_current_key(services, kid, "delete")?; + self.deactivate_key(services, kid)?; + + // Delete the private key first. A failure here leaves the JWK in the + // config store but no private key — the key is verifiable but cannot + // sign, which is safer than orphaned key material with no JWK. Both + // deletes treat 404 as success so retries converge after partial failures. + services + .secret_store() + .delete(&self.secret_store_id, kid) .change_context(TrustedServerError::Configuration { - message: "Failed to delete JWK from ConfigStore".into(), + message: "failed to delete signing key from secret store".into(), })?; - self.api_client - .delete_secret(&self.secret_store_id, kid) + services + .config_store() + .delete(&self.config_store_id, kid) .change_context(TrustedServerError::Configuration { - message: "Failed to delete secret from SecretStore".into(), + message: "failed to delete JWK from config store".into(), })?; Ok(()) } + + fn ensure_not_current_key( + &self, + services: &RuntimeServices, + kid: &str, + operation: &str, + ) -> Result<(), Report> { + if services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .is_ok_and(|current| current == kid) + { + return Err(Report::new(TrustedServerError::Configuration { + message: format!( + "cannot {operation} '{kid}' because it is the current signing key; rotate first" + ), + })); + } + + Ok(()) + } } +/// Generates a date-based key ID in the format `ts-YYYY-MM-DD`. #[must_use] pub fn generate_date_based_kid() -> String { - use chrono::Utc; format!("ts-{}", Utc::now().format("%Y-%m-%d")) } #[cfg(test)] mod tests { + use std::collections::HashMap; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::sync::{Arc, Mutex}; + + use error_stack::Report; + + use crate::platform::test_support::build_services_with_config_and_secret; + use crate::platform::{ + PlatformConfigStore, PlatformError, PlatformSecretStore, StoreId, StoreName, + }; use crate::request_signing::Keypair; use super::*; - #[test] - fn test_generate_date_based_kid() { - let kid = generate_date_based_kid(); - // Verify format: ts-YYYY-MM-DD - assert!(kid.starts_with("ts-")); - assert!(kid.len() >= 13); - let parts: Vec<&str> = kid.split('-').collect(); - assert_eq!(parts.len(), 4); - assert_eq!(parts[0], "ts"); + // --------------------------------------------------------------------------- + // Spy stores: record put/create/delete calls, serve preset get values + // --------------------------------------------------------------------------- + + #[derive(Clone)] + struct SpyConfigStore { + inner: Arc, } - #[test] - fn test_key_rotation_manager_creation() { - let result = KeyRotationManager::new("test-config-store-id", "test-secret-store-id"); - match result { - Ok(manager) => { - assert_eq!(manager.config_store_id, "test-config-store-id"); - assert_eq!(manager.secret_store_id, "test-secret-store-id"); + struct SpyConfigStoreInner { + data: Mutex>, + puts: Mutex>, + deletes: Mutex>, + /// Fail `put` after this many successful calls. `usize::MAX` means never fail. + fail_after_n_puts: AtomicUsize, + } + + impl SpyConfigStore { + fn new(initial: HashMap) -> Self { + Self { + inner: Arc::new(SpyConfigStoreInner { + data: Mutex::new(initial), + puts: Mutex::new(vec![]), + deletes: Mutex::new(vec![]), + fail_after_n_puts: AtomicUsize::new(usize::MAX), + }), } - Err(e) => { - println!("Expected error in test environment: {}", e); + } + + /// Returns a store whose `put` succeeds for the first `n` calls, then + /// returns an error. Use `n = 0` to fail immediately. + fn with_put_failure_after(n: usize) -> Self { + Self { + inner: Arc::new(SpyConfigStoreInner { + data: Mutex::new(HashMap::new()), + puts: Mutex::new(vec![]), + deletes: Mutex::new(vec![]), + fail_after_n_puts: AtomicUsize::new(n), + }), } } + + fn puts(&self) -> Vec<(String, String, String)> { + self.inner.puts.lock().expect("should lock puts").clone() + } + + fn deletes(&self) -> Vec<(String, String)> { + self.inner + .deletes + .lock() + .expect("should lock deletes") + .clone() + } } - #[test] - fn test_list_active_keys() { - let result = KeyRotationManager::new("test-config-store-id", "test-secret-store-id"); - if let Ok(manager) = result { - match manager.list_active_keys() { - Ok(keys) => { - assert!(!keys.is_empty(), "Should have at least one active key"); - } - Err(e) => println!("Expected error in test environment: {}", e), + impl PlatformConfigStore for SpyConfigStore { + fn get(&self, _: &StoreName, key: &str) -> Result> { + self.inner + .data + .lock() + .expect("should lock data") + .get(key) + .cloned() + .ok_or_else(|| Report::new(PlatformError::ConfigStore)) + } + + fn put( + &self, + store_id: &StoreId, + key: &str, + value: &str, + ) -> Result<(), Report> { + let remaining = self.inner.fail_after_n_puts.load(Ordering::SeqCst); + if remaining == 0 { + return Err(Report::new(PlatformError::ConfigStore)); + } + if remaining != usize::MAX { + self.inner.fail_after_n_puts.fetch_sub(1, Ordering::SeqCst); } + self.inner.puts.lock().expect("should lock puts").push(( + store_id.to_string(), + key.to_string(), + value.to_string(), + )); + self.inner + .data + .lock() + .expect("should lock data") + .insert(key.to_string(), value.to_string()); + Ok(()) } + + fn delete(&self, store_id: &StoreId, key: &str) -> Result<(), Report> { + self.inner + .deletes + .lock() + .expect("should lock deletes") + .push((store_id.to_string(), key.to_string())); + self.inner + .data + .lock() + .expect("should lock data") + .remove(key); + Ok(()) + } + } + + #[derive(Clone)] + struct SpySecretStore { + inner: Arc, + } + + struct SpySecretStoreInner { + creates: Mutex>, + deletes: Mutex>, + /// Fail `create` after this many successful calls. `usize::MAX` means never fail. + fail_after_n_creates: AtomicUsize, } + impl SpySecretStore { + fn new() -> Self { + Self { + inner: Arc::new(SpySecretStoreInner { + creates: Mutex::new(vec![]), + deletes: Mutex::new(vec![]), + fail_after_n_creates: AtomicUsize::new(usize::MAX), + }), + } + } + + /// Returns a store whose `create` succeeds for the first `n` calls, then + /// returns an error. Use `n = 0` to fail immediately. + fn with_create_failure_after(n: usize) -> Self { + Self { + inner: Arc::new(SpySecretStoreInner { + creates: Mutex::new(vec![]), + deletes: Mutex::new(vec![]), + fail_after_n_creates: AtomicUsize::new(n), + }), + } + } + + fn creates(&self) -> Vec<(String, String, String)> { + self.inner + .creates + .lock() + .expect("should lock creates") + .clone() + } + + fn deletes(&self) -> Vec<(String, String)> { + self.inner + .deletes + .lock() + .expect("should lock deletes") + .clone() + } + } + + impl PlatformSecretStore for SpySecretStore { + fn get_bytes(&self, _: &StoreName, _: &str) -> Result, Report> { + Err(Report::new(PlatformError::SecretStore)) + } + + fn create( + &self, + store_id: &StoreId, + name: &str, + value: &str, + ) -> Result<(), Report> { + let remaining = self.inner.fail_after_n_creates.load(Ordering::SeqCst); + if remaining == 0 { + return Err(Report::new(PlatformError::SecretStore)); + } + if remaining != usize::MAX { + self.inner + .fail_after_n_creates + .fetch_sub(1, Ordering::SeqCst); + } + self.inner + .creates + .lock() + .expect("should lock creates") + .push((store_id.to_string(), name.to_string(), value.to_string())); + Ok(()) + } + + fn delete(&self, store_id: &StoreId, name: &str) -> Result<(), Report> { + self.inner + .deletes + .lock() + .expect("should lock deletes") + .push((store_id.to_string(), name.to_string())); + Ok(()) + } + } + + // --------------------------------------------------------------------------- + // Tests + // --------------------------------------------------------------------------- + #[test] - fn test_key_rotation_result_structure() { - let jwk = Keypair::generate().get_jwk("test-key".to_string()); + fn generate_date_based_kid_has_correct_format() { + let kid = generate_date_based_kid(); + assert!(kid.starts_with("ts-"), "should start with 'ts-'"); + assert!(kid.len() >= 13, "should be at least 13 characters"); + let parts: Vec<&str> = kid.split('-').collect(); + assert_eq!(parts.len(), 4, "should have 4 dash-separated parts"); + assert_eq!(parts[0], "ts", "first part should be 'ts'"); + } + + #[test] + fn new_is_infallible_and_stores_ids() { + let manager = KeyRotationManager::new("cfg-store-123", "sec-store-456"); + assert_eq!( + manager.config_store_id.as_ref(), + "cfg-store-123", + "should store config_store_id" + ); + assert_eq!( + manager.secret_store_id.as_ref(), + "sec-store-456", + "should store secret_store_id" + ); + } + + #[test] + fn rotate_key_stores_private_key_via_secret_store_create() { + let config_store = SpyConfigStore::new(HashMap::new()); + let secret_store = SpySecretStore::new(); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.rotate_key(&services, Some("new-kid".to_string())); + + assert!(result.is_ok(), "should succeed when stores accept writes"); + let rotation = result.expect("should produce rotation result"); + assert_eq!(rotation.new_kid, "new-kid", "should use the provided kid"); + assert!( + rotation.active_kids.contains(&"new-kid".to_string()), + "should include new kid in active kids" + ); + } + #[test] + fn rotate_key_preserves_existing_active_kids() { + let mut data = HashMap::new(); + data.insert("current-kid".to_string(), "kid-b".to_string()); + data.insert("active-kids".to_string(), "kid-a, kid-b".to_string()); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let rotation = manager + .rotate_key(&services, Some("kid-c".to_string())) + .expect("should rotate key successfully"); + + assert_eq!( + rotation.active_kids, + vec![ + "kid-a".to_string(), + "kid-b".to_string(), + "kid-c".to_string() + ], + "should preserve previously active keys and append the new kid" + ); + + let active_kids = manager + .list_active_keys(&services) + .expect("should read back updated active kids"); + assert_eq!( + active_kids, + vec![ + "kid-a".to_string(), + "kid-b".to_string(), + "kid-c".to_string() + ], + "should store the full active kid list after rotation" + ); + } + + #[test] + fn rotate_key_does_not_reactivate_deactivated_previous_kid() { + let mut data = HashMap::new(); + data.insert("current-kid".to_string(), "kid-a".to_string()); + data.insert("active-kids".to_string(), "kid-b".to_string()); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let rotation = manager + .rotate_key(&services, Some("kid-c".to_string())) + .expect("should rotate key successfully"); + + assert_eq!( + rotation.active_kids, + vec!["kid-b".to_string(), "kid-c".to_string()], + "should not resurrect a previous kid that is no longer active" + ); + } + + #[test] + fn rotate_key_rejects_explicit_kid_that_is_already_active() { + let mut data = HashMap::new(); + data.insert("current-kid".to_string(), "kid-b".to_string()); + data.insert("active-kids".to_string(), "kid-a,kid-b".to_string()); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store.clone(), secret_store.clone()); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.rotate_key(&services, Some("kid-a".to_string())); + + assert!( + result.is_err(), + "should reject explicit rotation to an existing kid" + ); + assert!( + secret_store.creates().is_empty(), + "should reject duplicate kids before writing private key material" + ); + assert!( + config_store.puts().is_empty(), + "should reject duplicate kids before writing config store entries" + ); + } + + #[test] + fn rotate_key_uniquifies_generated_kid_when_date_based_kid_is_active() { + let base_kid = generate_date_based_kid(); + let mut data = HashMap::new(); + data.insert("current-kid".to_string(), base_kid.clone()); + data.insert("active-kids".to_string(), base_kid.clone()); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let rotation = manager + .rotate_key(&services, None) + .expect("should rotate with a uniquified generated kid"); + + assert_ne!( + rotation.new_kid, base_kid, + "should not reuse an active date-based kid" + ); + assert!( + rotation.new_kid.starts_with(&format!("{base_kid}-")), + "should preserve the date-based kid prefix for generated collisions" + ); + assert!( + rotation.active_kids.contains(&base_kid), + "should keep the existing kid active" + ); + assert!( + rotation.active_kids.contains(&rotation.new_kid), + "should add the uniquified generated kid" + ); + } + + #[test] + fn deactivate_key_fails_when_only_one_key_remains() { + let mut data = HashMap::new(); + data.insert("active-kids".to_string(), "only-key".to_string()); + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.deactivate_key(&services, "only-key"); + + assert!( + result.is_err(), + "should fail to deactivate the last active key" + ); + } + + #[test] + fn key_rotation_result_structure_is_valid() { + let jwk = Keypair::generate().get_jwk("test-key".to_string()); let result = KeyRotationResult { new_kid: "ts-2024-01-01".to_string(), previous_kid: Some("ts-2023-12-31".to_string()), @@ -270,4 +746,142 @@ mod tests { assert_eq!(result.active_kids.len(), 2); assert_eq!(result.jwk.prm.kid, Some("test-key".to_string())); } + + #[test] + fn rotate_key_fails_when_private_key_store_write_fails() { + let config_store = SpyConfigStore::new(HashMap::new()); + let secret_store = SpySecretStore::with_create_failure_after(0); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.rotate_key(&services, Some("new-kid".to_string())); + + assert!( + result.is_err(), + "should fail when the secret store rejects the private key write" + ); + } + + #[test] + fn rotate_key_rolls_back_secret_when_jwk_write_fails() { + let config_store = SpyConfigStore::with_put_failure_after(0); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store.clone(), secret_store.clone()); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.rotate_key(&services, Some("rollback-kid".to_string())); + + assert!(result.is_err(), "should fail when JWK write fails"); + assert_eq!( + secret_store.deletes(), + vec![("sec-id".to_string(), "rollback-kid".to_string())], + "should roll back private key material after JWK write failure" + ); + assert!( + config_store.deletes().is_empty(), + "should not roll back a JWK that was never stored" + ); + } + + #[test] + fn rotate_key_rolls_back_secret_and_jwk_when_active_kids_write_fails() { + let config_store = SpyConfigStore::with_put_failure_after(1); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store.clone(), secret_store.clone()); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.rotate_key(&services, Some("rollback-kid".to_string())); + + assert!(result.is_err(), "should fail when active-kids write fails"); + assert_eq!( + config_store.deletes(), + vec![("cfg-id".to_string(), "rollback-kid".to_string())], + "should roll back the stored JWK after active-kids write failure" + ); + assert_eq!( + secret_store.deletes(), + vec![("sec-id".to_string(), "rollback-kid".to_string())], + "should roll back private key material after active-kids write failure" + ); + } + + #[test] + fn deactivate_key_rejects_current_kid() { + let mut data = HashMap::new(); + data.insert("current-kid".to_string(), "kid-a".to_string()); + data.insert("active-kids".to_string(), "kid-a,kid-b".to_string()); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store.clone(), secret_store.clone()); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.deactivate_key(&services, "kid-a"); + + assert!(result.is_err(), "should reject deactivating current-kid"); + assert!( + config_store.puts().is_empty(), + "should reject current-kid deactivation before updating active-kids" + ); + assert!( + secret_store.deletes().is_empty(), + "should not touch secret store during failed deactivation" + ); + } + + #[test] + fn delete_key_rejects_current_kid_before_deleting_storage() { + let mut data = HashMap::new(); + data.insert("current-kid".to_string(), "kid-a".to_string()); + data.insert("active-kids".to_string(), "kid-a,kid-b".to_string()); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store.clone(), secret_store.clone()); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.delete_key(&services, "kid-a"); + + assert!(result.is_err(), "should reject deleting current-kid"); + assert!( + secret_store.deletes().is_empty(), + "should reject current-kid deletion before deleting private key material" + ); + assert!( + config_store.deletes().is_empty(), + "should reject current-kid deletion before deleting JWK storage" + ); + } + + #[test] + fn delete_key_removes_secret_before_jwk() { + let mut data = HashMap::new(); + data.insert("active-kids".to_string(), "kid-a, kid-b".to_string()); + data.insert( + "kid-a".to_string(), + r#"{"kty":"OKP","crv":"Ed25519"}"#.to_string(), + ); + + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + manager + .delete_key(&services, "kid-a") + .expect("should delete key successfully"); + + // After deletion, the JWK entry should be gone from the config store. + let jwk_gone = services + .config_store() + .get(&crate::request_signing::JWKS_STORE_NAME, "kid-a"); + assert!( + jwk_gone.is_err(), + "should remove JWK from the config store after deletion" + ); + } } diff --git a/crates/trusted-server-core/src/request_signing/signing.rs b/crates/trusted-server-core/src/request_signing/signing.rs index a30226bba..176f75e11 100644 --- a/crates/trusted-server-core/src/request_signing/signing.rs +++ b/crates/trusted-server-core/src/request_signing/signing.rs @@ -1,7 +1,7 @@ //! Request signing and verification utilities. //! //! This module provides Ed25519-based signing and verification of HTTP requests -//! using keys stored in Fastly Config and Secret stores. +//! using keys stored via platform store primitives. use base64::{engine::general_purpose, Engine}; use ed25519_dalek::{Signature, Signer as Ed25519Signer, SigningKey, Verifier, VerifyingKey}; @@ -9,42 +9,52 @@ use error_stack::{Report, ResultExt}; use serde::Serialize; use crate::error::TrustedServerError; -use crate::request_signing::{JWKS_CONFIG_STORE_NAME, SIGNING_SECRET_STORE_NAME}; -use crate::storage::{FastlyConfigStore, FastlySecretStore}; +use crate::platform::RuntimeServices; +use crate::request_signing::{JWKS_STORE_NAME, SIGNING_STORE_NAME}; /// Retrieves the current active key ID from the config store. /// /// # Errors /// /// Returns an error if the config store cannot be accessed or the current-kid key is not found. -#[allow(deprecated)] -pub fn get_current_key_id() -> Result> { - let store = FastlyConfigStore::new(JWKS_CONFIG_STORE_NAME); - store.get("current-kid") +pub fn get_current_key_id( + services: &RuntimeServices, +) -> Result> { + services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .change_context(TrustedServerError::Configuration { + message: "failed to read current-kid from config store".into(), + }) } -fn parse_ed25519_signing_key(key_bytes: Vec) -> Result> { - let bytes = if key_bytes.len() > 32 { - general_purpose::STANDARD.decode(&key_bytes).map_err(|_| { - Report::new(TrustedServerError::Configuration { - message: "Failed to decode base64 key".into(), - }) - })? - } else { - key_bytes - }; +/// Parses an Ed25519 signing key from secret-store bytes. +/// +/// Request-signing rotation always stores private keys as standard base64 text +/// via [`crate::request_signing::rotation::KeyRotationManager`]. A non-base64 +/// value in the secret store indicates data corruption and is surfaced as an +/// explicit error rather than silently falling back to a length heuristic. +fn parse_ed25519_signing_key(key_bytes: &[u8]) -> Result> { + let bytes = general_purpose::STANDARD.decode(key_bytes).map_err(|_| { + Report::new(TrustedServerError::Configuration { + message: "signing key is not valid base64 — corrupt key material in secret store" + .into(), + }) + })?; let key_array: [u8; 32] = bytes.try_into().map_err(|_| { Report::new(TrustedServerError::Configuration { - message: "Invalid key length (expected 32 bytes for Ed25519)".into(), + message: "signing key must be 32 bytes after base64 decoding".into(), }) })?; Ok(SigningKey::from_bytes(&key_array)) } +/// Signs request payloads using the current Ed25519 private key. pub struct RequestSigner { key: SigningKey, + /// Key identifier associated with the loaded private key. pub kid: String, } @@ -68,9 +78,13 @@ struct SigningPayload<'a> { /// Parameters for enhanced request signing #[derive(Debug, Clone)] pub struct SigningParams { + /// Request identifier to bind into the signature payload. pub request_id: String, + /// Host header value expected by the receiving service. pub request_host: String, + /// Request scheme bound into the signature payload. pub request_scheme: String, + /// Signature timestamp in Unix milliseconds. pub timestamp: u64, } @@ -91,8 +105,8 @@ impl SigningParams { /// Builds the canonical payload string for signing. /// - /// The payload is a JSON-serialized [`SigningPayload`] to prevent signature - /// confusion attacks that could exploit delimiter-based formats. + /// The payload is JSON-serialized to prevent signature confusion attacks + /// that could exploit delimiter-based formats. /// /// # Errors /// @@ -115,26 +129,25 @@ impl SigningParams { } impl RequestSigner { - /// Creates a `RequestSigner` from the current key ID stored in config. + /// Creates a `RequestSigner` from the current key ID stored in platform stores. /// /// # Errors /// /// Returns an error if the key ID cannot be retrieved or the key cannot be parsed. - #[allow(deprecated)] - pub fn from_config() -> Result> { - let config_store = FastlyConfigStore::new(JWKS_CONFIG_STORE_NAME); + pub fn from_services(services: &RuntimeServices) -> Result> { let key_id = - config_store - .get("current-kid") - .change_context(TrustedServerError::Configuration { - message: "Failed to get current-kid".into(), - })?; - - let secret_store = FastlySecretStore::new(SIGNING_SECRET_STORE_NAME); - let key_bytes = secret_store - .get(&key_id) - .attach(format!("Failed to get signing key for kid: {}", key_id))?; - let signing_key = parse_ed25519_signing_key(key_bytes)?; + get_current_key_id(services).change_context(TrustedServerError::Configuration { + message: "failed to get current-kid".into(), + })?; + + let key_bytes = services + .secret_store() + .get_bytes(&SIGNING_STORE_NAME, &key_id) + .change_context(TrustedServerError::Configuration { + message: format!("failed to get signing key for kid: {}", key_id), + })?; + + let signing_key = parse_ed25519_signing_key(&key_bytes)?; Ok(Self { key: signing_key, @@ -175,17 +188,17 @@ impl RequestSigner { /// # Errors /// /// Returns an error if the JWK cannot be retrieved, parsed, or if signature verification fails. -#[allow(deprecated)] pub fn verify_signature( payload: &[u8], signature_b64: &str, kid: &str, + services: &RuntimeServices, ) -> Result> { - let store = FastlyConfigStore::new(JWKS_CONFIG_STORE_NAME); - let jwk_json = store - .get(kid) + let jwk_json = services + .config_store() + .get(&JWKS_STORE_NAME, kid) .change_context(TrustedServerError::Configuration { - message: format!("Failed to get JWK for kid: {}", kid), + message: format!("failed to get JWK for kid: {}", kid), })?; let jwk: serde_json::Value = serde_json::from_str(&jwk_json).map_err(|e| { @@ -242,88 +255,88 @@ pub fn verify_signature( #[cfg(test)] mod tests { + use crate::platform::test_support::build_request_signing_services; + use super::*; #[test] - fn test_request_signer_sign() { - // Report unwraps print full error chain on test failure - // Note: unwrapping a Report prints it nicely if test fails. - let signer = RequestSigner::from_config().expect("should create signer from config"); + fn from_services_loads_kid_from_config_store() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); + + assert_eq!(signer.kid, "test-kid", "should load kid from config store"); + } + + #[test] + fn sign_produces_non_empty_url_safe_base64_signature() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); + let signature = signer .sign(b"these pretzels are making me thirsty") .expect("should sign payload"); - assert!(!signature.is_empty()); - assert!(signature.len() > 32); - } - #[test] - fn test_request_signer_from_config() { - let signer = RequestSigner::from_config().expect("should create signer from config"); - assert!(!signer.kid.is_empty()); + assert!(!signature.is_empty(), "should produce non-empty signature"); + assert!( + signature.len() > 32, + "should produce a full-length signature" + ); } #[test] - fn test_sign_and_verify() { + fn sign_and_verify_roundtrip_succeeds() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); let payload = b"test payload for verification"; - let signer = RequestSigner::from_config().expect("should create signer from config"); + let signature = signer.sign(payload).expect("should sign payload"); + let verified = verify_signature(payload, &signature, &signer.kid, &services) + .expect("should attempt verification"); - let result = - verify_signature(payload, &signature, &signer.kid).expect("should verify signature"); - assert!(result, "Signature should be valid"); + assert!(verified, "should verify a valid signature"); } #[test] - fn test_verify_invalid_signature() { - let payload = b"test payload"; - let signer = RequestSigner::from_config().expect("should create signer from config"); - - let wrong_signature = signer - .sign(b"different payload") - .expect("should sign different payload"); + fn verify_returns_false_for_wrong_payload() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); + let signature = signer.sign(b"original").expect("should sign"); - let result = verify_signature(payload, &wrong_signature, &signer.kid) + let verified = verify_signature(b"wrong payload", &signature, &signer.kid, &services) .expect("should attempt verification"); - assert!(!result, "Invalid signature should not verify"); - } - - #[test] - fn test_verify_wrong_payload() { - let original_payload = b"original payload"; - let signer = RequestSigner::from_config().expect("should create signer from config"); - let signature = signer - .sign(original_payload) - .expect("should sign original payload"); - let wrong_payload = b"wrong payload"; - let result = verify_signature(wrong_payload, &signature, &signer.kid) - .expect("should attempt verification"); - assert!(!result, "Signature should not verify with wrong payload"); + assert!(!verified, "should not verify signature for wrong payload"); } #[test] - fn test_verify_missing_key() { - let payload = b"test payload"; - let signer = RequestSigner::from_config().expect("should create signer from config"); - let signature = signer.sign(payload).expect("should sign payload"); - let nonexistent_kid = "nonexistent-key-id"; + fn verify_errors_for_unknown_kid() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); + let signature = signer.sign(b"payload").expect("should sign"); + + let result = verify_signature(b"payload", &signature, "nonexistent-kid", &services); - let result = verify_signature(payload, &signature, nonexistent_kid); - assert!(result.is_err(), "Should error for missing key"); + assert!(result.is_err(), "should error for unknown kid"); } #[test] - fn test_verify_malformed_signature() { - let payload = b"test payload"; - let signer = RequestSigner::from_config().expect("should create signer from config"); - let malformed_signature = "not-valid-base64!!!"; + fn verify_errors_for_malformed_signature() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); - let result = verify_signature(payload, malformed_signature, &signer.kid); - assert!(result.is_err(), "Should error for malformed signature"); + let result = verify_signature(b"payload", "not-valid-base64!!!", &signer.kid, &services); + + assert!(result.is_err(), "should error for malformed signature"); } #[test] - fn test_signing_params_build_payload() { + fn signing_params_build_payload_serializes_all_fields() { let params = SigningParams { request_id: "req-123".to_string(), request_host: "example.com".to_string(), @@ -336,6 +349,7 @@ mod tests { .expect("should build payload"); let parsed: serde_json::Value = serde_json::from_str(&payload).expect("should be valid JSON"); + assert_eq!(parsed["version"], SIGNING_VERSION); assert_eq!(parsed["kid"], "kid-abc"); assert_eq!(parsed["host"], "example.com"); @@ -345,46 +359,55 @@ mod tests { } #[test] - fn test_signing_params_new_creates_timestamp() { + fn signing_params_new_creates_recent_timestamp() { let params = SigningParams::new( "req-123".to_string(), "example.com".to_string(), "https".to_string(), ); - assert_eq!(params.request_id, "req-123"); - assert_eq!(params.request_host, "example.com"); - assert_eq!(params.request_scheme, "https"); - // Timestamp should be recent (within last minute), in milliseconds let now_ms = std::time::SystemTime::now() .duration_since(std::time::UNIX_EPOCH) - .unwrap() + .expect("should get system time") .as_millis() as u64; - assert!(params.timestamp <= now_ms); - assert!(params.timestamp >= now_ms - 60_000); + + assert!( + params.timestamp <= now_ms, + "timestamp should not be in the future" + ); + assert!( + params.timestamp >= now_ms - 60_000, + "timestamp should be within the last minute" + ); } #[test] - fn test_sign_request_enhanced() { - let signer = RequestSigner::from_config().unwrap(); + fn sign_request_enhanced_produces_verifiable_signature() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); let params = SigningParams::new( "auction-123".to_string(), "publisher.com".to_string(), "https".to_string(), ); - let signature = signer.sign_request(¶ms).unwrap(); - assert!(!signature.is_empty()); + let signature = signer.sign_request(¶ms).expect("should sign request"); + let payload = params + .build_payload(&signer.kid) + .expect("should build payload"); - // Verify the signature is valid by reconstructing the payload - let payload = params.build_payload(&signer.kid).unwrap(); - let result = verify_signature(payload.as_bytes(), &signature, &signer.kid).unwrap(); - assert!(result, "Enhanced signature should be valid"); + let verified = verify_signature(payload.as_bytes(), &signature, &signer.kid, &services) + .expect("should verify"); + + assert!(verified, "enhanced request signature should be verifiable"); } #[test] - fn test_sign_request_different_params_different_signature() { - let signer = RequestSigner::from_config().unwrap(); + fn sign_request_different_hosts_produce_different_signatures() { + let services = build_request_signing_services(); + let signer = + RequestSigner::from_services(&services).expect("should create signer from services"); let params1 = SigningParams { request_id: "req-1".to_string(), @@ -392,20 +415,19 @@ mod tests { request_scheme: "https".to_string(), timestamp: 1706900000, }; - let params2 = SigningParams { request_id: "req-1".to_string(), - request_host: "host2.com".to_string(), // Different host + request_host: "host2.com".to_string(), request_scheme: "https".to_string(), timestamp: 1706900000, }; - let sig1 = signer.sign_request(¶ms1).unwrap(); - let sig2 = signer.sign_request(¶ms2).unwrap(); + let sig1 = signer.sign_request(¶ms1).expect("should sign params1"); + let sig2 = signer.sign_request(¶ms2).expect("should sign params2"); assert_ne!( sig1, sig2, - "Different hosts should produce different signatures" + "different hosts should produce different signatures" ); } } diff --git a/crates/trusted-server-core/src/storage/api_client.rs b/crates/trusted-server-core/src/storage/api_client.rs deleted file mode 100644 index 81a2d57b8..000000000 --- a/crates/trusted-server-core/src/storage/api_client.rs +++ /dev/null @@ -1,291 +0,0 @@ -//! Fastly management API client (legacy). -//! -//! This module holds [`FastlyApiClient`], which wraps the Fastly management -//! REST API for write operations on config and secret stores. -//! New code should use [`crate::platform::PlatformConfigStore`] and -//! [`crate::platform::PlatformSecretStore`] write methods instead. -//! This type will be removed once all call sites have migrated. - -use std::io::Read; - -use error_stack::{Report, ResultExt}; -use fastly::{Request, Response}; -use http::StatusCode; - -use crate::backend::BackendConfig; -use crate::error::TrustedServerError; -use crate::storage::secret_store::FastlySecretStore; - -const FASTLY_API_HOST: &str = "https://api.fastly.com"; - -fn build_config_item_payload(value: &str) -> String { - format!("item_value={}", urlencoding::encode(value)) -} - -/// HTTP client for the Fastly management API. -/// -/// Used to perform write operations on config and secret stores via the -/// Fastly REST API. Reads are performed directly through the edge-side SDK. -/// -/// # Migration note -/// -/// This type predates the `platform` abstraction. New code should use -/// [`crate::platform::PlatformConfigStore`] and -/// [`crate::platform::PlatformSecretStore`] write methods instead. -pub struct FastlyApiClient { - api_key: Vec, - base_url: &'static str, - backend_name: String, -} - -impl FastlyApiClient { - /// Creates a new Fastly API client using the default secret store. - /// - /// # Errors - /// - /// Returns an error if the secret store cannot be opened or the API key - /// cannot be retrieved. - pub fn new() -> Result> { - Self::from_secret_store("api-keys", "api_key") - } - - /// Creates a new Fastly API client reading credentials from a specified - /// secret store entry. - /// - /// # Errors - /// - /// Returns an error if the API backend cannot be ensured or the API key - /// cannot be retrieved. - pub fn from_secret_store( - store_name: &str, - key_name: &str, - ) -> Result> { - let backend_name = BackendConfig::from_url("https://api.fastly.com", true)?; - let api_key = FastlySecretStore::new(store_name).get(key_name)?; - - log::debug!("FastlyApiClient initialized"); - - Ok(Self { - api_key, - base_url: FASTLY_API_HOST, - backend_name, - }) - } - - fn make_request( - &self, - method: &str, - path: &str, - body: Option, - content_type: &str, - ) -> Result> { - let url = format!("{}{}", self.base_url, path); - let api_key_str = String::from_utf8_lossy(&self.api_key).to_string(); - - let mut request = match method { - "GET" => Request::get(&url), - "POST" => Request::post(&url), - "PUT" => Request::put(&url), - "DELETE" => Request::delete(&url), - _ => { - return Err(Report::new(TrustedServerError::Configuration { - message: format!("unsupported HTTP method: {}", method), - })) - } - }; - - request = request - .with_header("Fastly-Key", api_key_str) - .with_header("Accept", "application/json"); - - if let Some(body_content) = body { - request = request - .with_header("Content-Type", content_type) - .with_body(body_content); - } - - request.send(&self.backend_name).map_err(|e| { - Report::new(TrustedServerError::Configuration { - message: format!("failed to send API request: {}", e), - }) - }) - } - - /// Updates a configuration item in a Fastly config store. - /// - /// # Errors - /// - /// Returns an error if the API request fails or returns a non-OK status. - pub fn update_config_item( - &self, - store_id: &str, - key: &str, - value: &str, - ) -> Result<(), Report> { - let path = format!("/resources/stores/config/{}/item/{}", store_id, key); - let payload = build_config_item_payload(value); - - let mut response = self.make_request( - "PUT", - &path, - Some(payload), - "application/x-www-form-urlencoded", - )?; - - let mut buf = String::new(); - response - .get_body_mut() - .read_to_string(&mut buf) - .change_context(TrustedServerError::Configuration { - message: "failed to read config store API response".into(), - })?; - - if response.get_status() == StatusCode::OK { - Ok(()) - } else { - Err(Report::new(TrustedServerError::Configuration { - message: format!( - "failed to update config item: HTTP {} - {}", - response.get_status(), - buf - ), - })) - } - } - - /// Creates a secret in a Fastly secret store. - /// - /// # Errors - /// - /// Returns an error if the API request fails or returns a non-OK status. - pub fn create_secret( - &self, - store_id: &str, - secret_name: &str, - secret_value: &str, - ) -> Result<(), Report> { - let path = format!("/resources/stores/secret/{}/secrets", store_id); - let payload = serde_json::json!({ - "name": secret_name, - "secret": secret_value - }); - - let mut response = - self.make_request("POST", &path, Some(payload.to_string()), "application/json")?; - - let mut buf = String::new(); - response - .get_body_mut() - .read_to_string(&mut buf) - .change_context(TrustedServerError::Configuration { - message: "failed to read secret store API response".into(), - })?; - - if response.get_status() == StatusCode::OK { - Ok(()) - } else { - Err(Report::new(TrustedServerError::Configuration { - message: format!( - "failed to create secret: HTTP {} - {}", - response.get_status(), - buf - ), - })) - } - } - - /// Deletes a configuration item from a Fastly config store. - /// - /// # Errors - /// - /// Returns an error if the API request fails or returns a non-OK or - /// non-NO_CONTENT status. - pub fn delete_config_item( - &self, - store_id: &str, - key: &str, - ) -> Result<(), Report> { - let path = format!("/resources/stores/config/{}/item/{}", store_id, key); - - let mut response = self.make_request("DELETE", &path, None, "application/json")?; - - let mut buf = String::new(); - response - .get_body_mut() - .read_to_string(&mut buf) - .change_context(TrustedServerError::Configuration { - message: "failed to read config store delete API response".into(), - })?; - - if response.get_status() == StatusCode::OK - || response.get_status() == StatusCode::NO_CONTENT - { - Ok(()) - } else { - Err(Report::new(TrustedServerError::Configuration { - message: format!( - "failed to delete config item: HTTP {} - {}", - response.get_status(), - buf - ), - })) - } - } - - /// Deletes a secret from a Fastly secret store. - /// - /// # Errors - /// - /// Returns an error if the API request fails or returns a non-OK or - /// non-NO_CONTENT status. - pub fn delete_secret( - &self, - store_id: &str, - secret_name: &str, - ) -> Result<(), Report> { - let path = format!( - "/resources/stores/secret/{}/secrets/{}", - store_id, secret_name - ); - - let mut response = self.make_request("DELETE", &path, None, "application/json")?; - - let mut buf = String::new(); - response - .get_body_mut() - .read_to_string(&mut buf) - .change_context(TrustedServerError::Configuration { - message: "failed to read secret store delete API response".into(), - })?; - - if response.get_status() == StatusCode::OK - || response.get_status() == StatusCode::NO_CONTENT - { - Ok(()) - } else { - Err(Report::new(TrustedServerError::Configuration { - message: format!( - "failed to delete secret: HTTP {} - {}", - response.get_status(), - buf - ), - })) - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn config_item_payload_url_encodes_reserved_characters() { - let payload = build_config_item_payload(r#"value with spaces + symbols &= {"kid":"a+b"}"#); - - assert_eq!( - payload, - "item_value=value%20with%20spaces%20%2B%20symbols%20%26%3D%20%7B%22kid%22%3A%22a%2Bb%22%7D", - "should URL-encode config item values in form payloads" - ); - } -} diff --git a/crates/trusted-server-core/src/storage/mod.rs b/crates/trusted-server-core/src/storage/mod.rs index 0d8ed1452..62c42ba14 100644 --- a/crates/trusted-server-core/src/storage/mod.rs +++ b/crates/trusted-server-core/src/storage/mod.rs @@ -7,10 +7,8 @@ //! [`crate::platform::PlatformSecretStore`], and the management write methods //! via [`crate::platform::RuntimeServices`]. -pub(crate) mod api_client; pub(crate) mod config_store; pub(crate) mod secret_store; -pub use api_client::FastlyApiClient; pub use config_store::FastlyConfigStore; pub use secret_store::FastlySecretStore; diff --git a/docs/superpowers/plans/2026-03-31-pr9-wire-signing-to-store-primitives.md b/docs/superpowers/plans/2026-03-31-pr9-wire-signing-to-store-primitives.md new file mode 100644 index 000000000..c03de5fa9 --- /dev/null +++ b/docs/superpowers/plans/2026-03-31-pr9-wire-signing-to-store-primitives.md @@ -0,0 +1,1835 @@ +# PR 9: Wire Request-Signing to Platform Store Primitives + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Remove `api_client.rs` from `trusted-server-core`, move Fastly management API transport to the adapter as `management_api.rs`, and replace all direct `Fastly*Store` / `FastlyApiClient` usage in `request_signing/` with `RuntimeServices` store primitives. + +**Architecture:** Core `request_signing/` code calls platform-agnostic `services.config_store()` and `services.secret_store()` for all reads and writes. The Fastly adapter's `management_api.rs` absorbs the HTTP transport (calls to `api.fastly.com`) and backs the `put`/`delete`/`create` write methods in `FastlyPlatformConfigStore` and `FastlyPlatformSecretStore`. No signing-specific trait is introduced — adapters only implement store CRUD, and core owns all signing business logic. + +**Tech Stack:** Rust 2024 edition, `error-stack`, `derive_more::Display`, `fastly`, `ed25519-dalek`, `serde_json`, `urlencoding` + +--- + +## Background: What the Current Code Does + +Before touching anything, read these files to understand the current state: + +| File | Status | Notes | +| ------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------ | +| `crates/trusted-server-core/src/storage/api_client.rs` | **Delete** | Contains `FastlyApiClient` — HTTP calls to `api.fastly.com`. Used only by `rotation.rs`. | +| `crates/trusted-server-core/src/request_signing/rotation.rs` | **Migrate** | Uses `FastlyConfigStore` (reads) + `FastlyApiClient` (writes). Main migration target. | +| `crates/trusted-server-core/src/request_signing/signing.rs` | **Migrate** | Uses `FastlyConfigStore` + `FastlySecretStore` in 3 places. | +| `crates/trusted-server-core/src/request_signing/endpoints.rs` | **Update** | `handle_verify_signature`, `handle_rotate_key`, `handle_deactivate_key` don't receive `&RuntimeServices`. | +| `crates/trusted-server-core/src/request_signing/jwks.rs` | Already migrated ✓ | Uses `RuntimeServices`. No changes needed. | +| `crates/trusted-server-adapter-fastly/src/platform.rs` | **Update** | `FastlyPlatformConfigStore::put/delete` and `FastlyPlatformSecretStore::create/delete` return `PlatformError::NotImplemented`. | +| `crates/trusted-server-adapter-fastly/src/main.rs` | **Update** | Three call sites pass handlers without `runtime_services`. | + +## File Map + +### Delete + +- `crates/trusted-server-core/src/storage/api_client.rs` + +### Modify (core) + +- `crates/trusted-server-core/src/storage/mod.rs` — remove `api_client` submodule + re-export +- `crates/trusted-server-core/src/platform/test_support.rs` — add `build_services_with_config_and_secret` +- `crates/trusted-server-core/src/request_signing/rotation.rs` — replace `FastlyConfigStore`/`FastlyApiClient` with `RuntimeServices` +- `crates/trusted-server-core/src/request_signing/signing.rs` — replace `FastlyConfigStore`/`FastlySecretStore` with `RuntimeServices` +- `crates/trusted-server-core/src/request_signing/endpoints.rs` — add `&RuntimeServices` to three handlers + +### Create (adapter) + +- `crates/trusted-server-adapter-fastly/src/management_api.rs` — Fastly management API transport (absorbs `api_client.rs` logic, returns `PlatformError`) + +### Modify (adapter) + +- `crates/trusted-server-adapter-fastly/src/platform.rs` — implement `put`/`delete` for config, `create`/`delete` for secrets +- `crates/trusted-server-adapter-fastly/src/main.rs` — pass `runtime_services` to three handlers + +--- + +## Tasks + +### Task 1: Add `build_services_with_config_and_secret` to `test_support.rs` + +**Why:** Tasks 4 and 5 need a `RuntimeServices` with both a custom config store AND a custom secret store. The current `build_services_with_config` only customises the config store. + +**Files:** + +- Modify: `crates/trusted-server-core/src/platform/test_support.rs` + +- [ ] **Step 1: Write a failing test that calls `build_services_with_config_and_secret`** + +Add to the `#[cfg(test)]` block at the bottom of `test_support.rs`: + +```rust +#[test] +fn build_services_with_config_and_secret_uses_provided_stores() { + // Arrange: noop stores + let services = build_services_with_config_and_secret(NoopConfigStore, NoopSecretStore); + + // Act: both stores return Unsupported (confirming the injected impls are active) + let config_result = services.config_store().get(&StoreName::from("s"), "k"); + let secret_result = services.secret_store().get_bytes(&StoreName::from("s"), "k"); + + assert!(config_result.is_err(), "should delegate to injected config store"); + assert!(secret_result.is_err(), "should delegate to injected secret store"); +} +``` + +- [ ] **Step 2: Run to confirm it fails to compile** + +```bash +cargo test --package trusted-server-core platform::test_support 2>&1 | head -20 +``` + +Expected: compile error — `build_services_with_config_and_secret` not found. + +- [ ] **Step 3: Add the function above the existing `build_services_with_config`** + +```rust +/// Build a [`RuntimeServices`] instance with a custom config store and a custom secret store. +/// +/// Use this when a test exercises code that reads from config AND secret stores, +/// such as `request_signing::signing` and `request_signing::rotation`. +pub(crate) fn build_services_with_config_and_secret( + config_store: impl PlatformConfigStore + 'static, + secret_store: impl PlatformSecretStore + 'static, +) -> RuntimeServices { + RuntimeServices::builder() + .config_store(Arc::new(config_store)) + .secret_store(Arc::new(secret_store)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(NoopBackend)) + .http_client(Arc::new(NoopHttpClient)) + .geo(Arc::new(NoopGeo)) + .client_info(ClientInfo { + client_ip: None, + tls_protocol: None, + tls_cipher: None, + }) + .build() +} +``` + +- [ ] **Step 4: Run the test to confirm it passes** + +```bash +cargo test --package trusted-server-core platform::test_support::tests::build_services_with_config_and_secret +``` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/platform/test_support.rs +git commit -m "Add build_services_with_config_and_secret to test_support" +``` + +--- + +### Task 2: Create `management_api.rs` in the adapter + +**Why:** Move the Fastly management API transport (currently in `api_client.rs` in core) to the adapter, where Fastly SDK usage is appropriate. Returns `PlatformError` instead of `TrustedServerError`. + +**Credential security note (from spec):** The Fastly API token is read from the `api-keys` secret store, key `api_key`. Log store IDs and operation names only — never the token or secret value. + +**Files:** + +- Create: `crates/trusted-server-adapter-fastly/src/management_api.rs` +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` — add `mod management_api;` + +- [ ] **Step 1: Write the new module** + +Create `crates/trusted-server-adapter-fastly/src/management_api.rs`: + +```rust +//! Fastly management API transport for store write operations. +//! +//! Provides [`FastlyManagementApiClient`], which wraps the Fastly REST +//! management API for write operations on config and secret stores. +//! Used by [`super::platform::FastlyPlatformConfigStore`] and +//! [`super::platform::FastlyPlatformSecretStore`] to back store write methods. +//! +//! # Credentials +//! +//! The Fastly API token is read from the `api-keys` secret store under the +//! `api_key` entry. The token must have config-store write and secret-store +//! write permissions only — no service-level admin or purge permissions. +//! +//! # Security +//! +//! Credential values are never logged. Log messages include store IDs and +//! operation names only. + +use std::io::Read; + +use error_stack::{Report, ResultExt}; +use fastly::{Request, Response}; +use http::StatusCode; +use trusted_server_core::backend::BackendConfig; +use trusted_server_core::platform::{PlatformError, PlatformSecretStore, StoreName}; + +use crate::platform::FastlyPlatformSecretStore; + +const FASTLY_API_HOST: &str = "https://api.fastly.com"; +const API_KEYS_STORE: &str = "api-keys"; +const API_KEY_ENTRY: &str = "api_key"; + +pub(crate) fn build_config_item_payload(value: &str) -> String { + format!("item_value={}", urlencoding::encode(value)) +} + +/// HTTP client for Fastly management API write operations. +/// +/// Backs the `put`/`delete` methods of [`FastlyPlatformConfigStore`] and +/// the `create`/`delete` methods of [`FastlyPlatformSecretStore`]. +pub(crate) struct FastlyManagementApiClient { + api_key: Vec, + base_url: &'static str, + backend_name: String, +} + +impl FastlyManagementApiClient { + /// Initialize the client by reading the API token from the `api-keys` secret store. + /// + /// # Errors + /// + /// Returns [`PlatformError::Backend`] if the management API backend cannot + /// be registered, or [`PlatformError::SecretStore`] if the API key cannot + /// be read. + pub(crate) fn new() -> Result> { + let backend_name = BackendConfig::from_url(FASTLY_API_HOST, true) + .change_context(PlatformError::Backend) + .attach("failed to register Fastly management API backend")?; + + let api_key = FastlyPlatformSecretStore + .get_bytes(&StoreName::from(API_KEYS_STORE), API_KEY_ENTRY) + .change_context(PlatformError::SecretStore) + .attach("failed to read Fastly API key from secret store")?; + + log::debug!("FastlyManagementApiClient: initialized for management API operations"); + + Ok(Self { + api_key, + base_url: FASTLY_API_HOST, + backend_name, + }) + } + + fn make_request( + &self, + method: &str, + path: &str, + body: Option, + content_type: &str, + ) -> Result> { + let url = format!("{}{}", self.base_url, path); + let api_key_str = String::from_utf8_lossy(&self.api_key).to_string(); + + let mut request = match method { + "GET" => Request::get(&url), + "POST" => Request::post(&url), + "PUT" => Request::put(&url), + "DELETE" => Request::delete(&url), + _ => { + return Err(Report::new(PlatformError::ConfigStore) + .attach(format!("unsupported HTTP method: {}", method))) + } + }; + + request = request + .with_header("Fastly-Key", api_key_str) + .with_header("Accept", "application/json"); + + if let Some(body_content) = body { + request = request + .with_header("Content-Type", content_type) + .with_body(body_content); + } + + request.send(&self.backend_name).map_err(|e| { + Report::new(PlatformError::ConfigStore) + .attach(format!("management API request failed: {}", e)) + }) + } + + /// Update or create a config store item. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns a non-OK status. + pub(crate) fn update_config_item( + &self, + store_id: &str, + key: &str, + value: &str, + ) -> Result<(), Report> { + let path = format!("/resources/stores/config/{}/item/{}", store_id, key); + let payload = build_config_item_payload(value); + + let mut response = self.make_request( + "PUT", + &path, + Some(payload), + "application/x-www-form-urlencoded", + )?; + + let mut buf = String::new(); + response + .get_body_mut() + .read_to_string(&mut buf) + .change_context(PlatformError::ConfigStore)?; + + if response.get_status() == StatusCode::OK { + log::debug!( + "FastlyManagementApiClient: updated config key '{}' in store '{}'", + key, + store_id + ); + Ok(()) + } else { + Err(Report::new(PlatformError::ConfigStore).attach(format!( + "config item update failed with HTTP {} for key '{}' in store '{}'", + response.get_status(), + key, + store_id + ))) + } + } + + /// Delete a config store item. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns an unexpected status. + pub(crate) fn delete_config_item( + &self, + store_id: &str, + key: &str, + ) -> Result<(), Report> { + let path = format!("/resources/stores/config/{}/item/{}", store_id, key); + + let mut response = self.make_request("DELETE", &path, None, "application/json")?; + + let mut buf = String::new(); + response + .get_body_mut() + .read_to_string(&mut buf) + .change_context(PlatformError::ConfigStore)?; + + if response.get_status() == StatusCode::OK + || response.get_status() == StatusCode::NO_CONTENT + { + log::debug!( + "FastlyManagementApiClient: deleted config key '{}' from store '{}'", + key, + store_id + ); + Ok(()) + } else { + Err(Report::new(PlatformError::ConfigStore).attach(format!( + "config item delete failed with HTTP {} for key '{}' in store '{}'", + response.get_status(), + key, + store_id + ))) + } + } + + /// Create or overwrite a secret store entry. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns a non-OK status. + pub(crate) fn create_secret( + &self, + store_id: &str, + secret_name: &str, + secret_value: &str, + ) -> Result<(), Report> { + let path = format!("/resources/stores/secret/{}/secrets", store_id); + let payload = serde_json::json!({ + "name": secret_name, + "secret": secret_value + }); + + let mut response = + self.make_request("POST", &path, Some(payload.to_string()), "application/json")?; + + let mut buf = String::new(); + response + .get_body_mut() + .read_to_string(&mut buf) + .change_context(PlatformError::SecretStore)?; + + if response.get_status() == StatusCode::OK { + log::debug!( + "FastlyManagementApiClient: created secret '{}' in store '{}'", + secret_name, + store_id + ); + Ok(()) + } else { + Err(Report::new(PlatformError::SecretStore).attach(format!( + "secret create failed with HTTP {} for name '{}' in store '{}'", + response.get_status(), + secret_name, + store_id + ))) + } + } + + /// Delete a secret store entry. + /// + /// # Errors + /// + /// Returns an error if the API request fails or returns an unexpected status. + pub(crate) fn delete_secret( + &self, + store_id: &str, + secret_name: &str, + ) -> Result<(), Report> { + let path = format!( + "/resources/stores/secret/{}/secrets/{}", + store_id, secret_name + ); + + let mut response = self.make_request("DELETE", &path, None, "application/json")?; + + let mut buf = String::new(); + response + .get_body_mut() + .read_to_string(&mut buf) + .change_context(PlatformError::SecretStore)?; + + if response.get_status() == StatusCode::OK + || response.get_status() == StatusCode::NO_CONTENT + { + log::debug!( + "FastlyManagementApiClient: deleted secret '{}' from store '{}'", + secret_name, + store_id + ); + Ok(()) + } else { + Err(Report::new(PlatformError::SecretStore).attach(format!( + "secret delete failed with HTTP {} for name '{}' in store '{}'", + response.get_status(), + secret_name, + store_id + ))) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn build_config_item_payload_url_encodes_reserved_characters() { + let payload = + build_config_item_payload(r#"value with spaces + symbols &= {"kid":"a+b"}"#); + + assert_eq!( + payload, + "item_value=value%20with%20spaces%20%2B%20symbols%20%26%3D%20%7B%22kid%22%3A%22a%2Bb%22%7D", + "should URL-encode config item values in form payloads" + ); + } +} +``` + +- [ ] **Step 2: Add `mod management_api;` to `main.rs`** + +In `crates/trusted-server-adapter-fastly/src/main.rs`, add near the top (alongside the other `mod` declarations): + +```rust +mod management_api; +``` + +- [ ] **Step 3: Run the payload test** + +```bash +cargo test --package trusted-server-adapter-fastly management_api -- --nocapture +``` + +Expected: `build_config_item_payload_url_encodes_reserved_characters` passes. + +- [ ] **Step 4: Commit** + +```bash +git add crates/trusted-server-adapter-fastly/src/management_api.rs crates/trusted-server-adapter-fastly/src/main.rs +git commit -m "Add FastlyManagementApiClient to adapter" +``` + +--- + +### Task 3: Implement `FastlyPlatformConfigStore` write methods + +**Why:** Replace the `NotImplemented` stubs in `platform.rs` with real calls to `FastlyManagementApiClient`. The existing `NotImplemented` test for secret store (`fastly_platform_secret_store_create_returns_not_implemented`, `fastly_platform_secret_store_delete_returns_not_implemented`) must be deleted now that the real implementation lands. Check if there are equivalent config store tests to delete too. + +**Files:** + +- Modify: `crates/trusted-server-adapter-fastly/src/platform.rs` + +- [ ] **Step 1: Delete `NotImplemented` tests for secret store writes** + +In `platform.rs` tests, find and delete these two tests (they assert the old stub behavior that no longer holds): + +- `fastly_platform_secret_store_create_returns_not_implemented` +- `fastly_platform_secret_store_delete_returns_not_implemented` + +There are no analogous `NotImplemented` tests for `FastlyPlatformConfigStore::put/delete` — only the secret store stubs have them. No config-store equivalent to search for. + +- [ ] **Step 2: Update `FastlyPlatformConfigStore::put` and `delete`** + +In `platform.rs`, replace: + +```rust +fn put( + &self, + _store_id: &StoreId, + _key: &str, + _value: &str, +) -> Result<(), Report> { + Err(Report::new(PlatformError::NotImplemented)) +} + +fn delete(&self, _store_id: &StoreId, _key: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::NotImplemented)) +} +``` + +With: + +```rust +fn put( + &self, + store_id: &StoreId, + key: &str, + value: &str, +) -> Result<(), Report> { + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.update_config_item(store_id.as_ref(), key, value) +} + +fn delete(&self, store_id: &StoreId, key: &str) -> Result<(), Report> { + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.delete_config_item(store_id.as_ref(), key) +} +``` + +- [ ] **Step 3: Update `FastlyPlatformSecretStore::create` and `delete`** + +Replace: + +```rust +fn create( + &self, + _store_id: &StoreId, + _name: &str, + _value: &str, +) -> Result<(), Report> { + Err(Report::new(PlatformError::NotImplemented)) +} + +fn delete(&self, _store_id: &StoreId, _name: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::NotImplemented)) +} +``` + +With: + +```rust +fn create( + &self, + store_id: &StoreId, + name: &str, + value: &str, +) -> Result<(), Report> { + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.create_secret(store_id.as_ref(), name, value) +} + +fn delete(&self, store_id: &StoreId, name: &str) -> Result<(), Report> { + let client = crate::management_api::FastlyManagementApiClient::new()?; + client.delete_secret(store_id.as_ref(), name) +} +``` + +- [ ] **Step 4: Verify adapter compiles and remaining tests pass** + +```bash +cargo test --package trusted-server-adapter-fastly -- --nocapture +``` + +Expected: all tests pass (the `NotImplemented` tests were deleted; remaining tests still pass). + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-adapter-fastly/src/platform.rs +git commit -m "Implement FastlyPlatformConfigStore and FastlyPlatformSecretStore write methods via management API" +``` + +--- + +### Task 4: Migrate `rotation.rs` to `RuntimeServices` + +**Why:** `KeyRotationManager` currently holds `FastlyConfigStore` (reads) and `FastlyApiClient` (writes) as fields. Replace both with `&RuntimeServices` passed to each method. + +**New design:** + +- Drop `config_store: FastlyConfigStore` and `api_client: FastlyApiClient` fields +- Keep `config_store_id: StoreId` and `secret_store_id: StoreId` (passed to write methods) +- `new()` is now infallible (no API key fetch at construction time) +- All `rotate_key`, `list_active_keys`, `deactivate_key`, `delete_key` accept `services: &RuntimeServices` +- Reads use `JWKS_STORE_NAME` (edge-visible name); writes use the stored `StoreId` values + +**Files:** + +- Modify: `crates/trusted-server-core/src/request_signing/rotation.rs` + +- [ ] **Step 1: Write failing tests that define the new API** + +Replace the `#[cfg(test)]` module in `rotation.rs` with: + +```rust +#[cfg(test)] +mod tests { + use std::collections::HashMap; + use std::sync::Mutex; + + use error_stack::Report; + + use crate::platform::test_support::build_services_with_config_and_secret; + use crate::platform::{ + PlatformConfigStore, PlatformError, PlatformSecretStore, StoreId, StoreName, + }; + use crate::request_signing::Keypair; + + use super::*; + + // --------------------------------------------------------------------------- + // Spy stores: record put/create/delete calls, serve preset get values + // --------------------------------------------------------------------------- + + struct SpyConfigStore { + data: Mutex>, + puts: Mutex>, + deletes: Mutex>, + } + + impl SpyConfigStore { + fn new(initial: HashMap) -> Self { + Self { + data: Mutex::new(initial), + puts: Mutex::new(vec![]), + deletes: Mutex::new(vec![]), + } + } + } + + impl PlatformConfigStore for SpyConfigStore { + fn get(&self, _: &StoreName, key: &str) -> Result> { + self.data + .lock() + .expect("should lock data") + .get(key) + .cloned() + .ok_or_else(|| Report::new(PlatformError::ConfigStore)) + } + + fn put( + &self, + store_id: &StoreId, + key: &str, + value: &str, + ) -> Result<(), Report> { + self.puts.lock().expect("should lock puts").push(( + store_id.to_string(), + key.to_string(), + value.to_string(), + )); + self.data + .lock() + .expect("should lock data") + .insert(key.to_string(), value.to_string()); + Ok(()) + } + + fn delete(&self, store_id: &StoreId, key: &str) -> Result<(), Report> { + self.deletes + .lock() + .expect("should lock deletes") + .push((store_id.to_string(), key.to_string())); + self.data + .lock() + .expect("should lock data") + .remove(key); + Ok(()) + } + } + + struct SpySecretStore { + creates: Mutex>, + deletes: Mutex>, + } + + impl SpySecretStore { + fn new() -> Self { + Self { + creates: Mutex::new(vec![]), + deletes: Mutex::new(vec![]), + } + } + } + + impl PlatformSecretStore for SpySecretStore { + fn get_bytes(&self, _: &StoreName, _: &str) -> Result, Report> { + Err(Report::new(PlatformError::SecretStore)) + } + + fn create( + &self, + store_id: &StoreId, + name: &str, + value: &str, + ) -> Result<(), Report> { + self.creates.lock().expect("should lock creates").push(( + store_id.to_string(), + name.to_string(), + value.to_string(), + )); + Ok(()) + } + + fn delete(&self, store_id: &StoreId, name: &str) -> Result<(), Report> { + self.deletes + .lock() + .expect("should lock deletes") + .push((store_id.to_string(), name.to_string())); + Ok(()) + } + } + + // --------------------------------------------------------------------------- + // Tests + // --------------------------------------------------------------------------- + + #[test] + fn generate_date_based_kid_has_correct_format() { + let kid = generate_date_based_kid(); + assert!(kid.starts_with("ts-"), "should start with 'ts-'"); + assert!(kid.len() >= 13, "should be at least 13 characters"); + let parts: Vec<&str> = kid.split('-').collect(); + assert_eq!(parts.len(), 4, "should have 4 dash-separated parts"); + assert_eq!(parts[0], "ts", "first part should be 'ts'"); + } + + #[test] + fn new_is_infallible_and_stores_ids() { + let manager = KeyRotationManager::new("cfg-store-123", "sec-store-456"); + assert_eq!( + manager.config_store_id.as_ref(), + "cfg-store-123", + "should store config_store_id" + ); + assert_eq!( + manager.secret_store_id.as_ref(), + "sec-store-456", + "should store secret_store_id" + ); + } + + #[test] + fn rotate_key_stores_private_key_via_secret_store_create() { + let config_store = SpyConfigStore::new(HashMap::new()); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.rotate_key(&services, Some("new-kid".to_string())); + + assert!(result.is_ok(), "should succeed when stores accept writes"); + let rotation = result.expect("should produce rotation result"); + assert_eq!(rotation.new_kid, "new-kid", "should use the provided kid"); + assert!( + rotation.active_kids.contains(&"new-kid".to_string()), + "should include new kid in active kids" + ); + } + + #[test] + fn deactivate_key_fails_when_only_one_key_remains() { + let mut data = HashMap::new(); + data.insert("active-kids".to_string(), "only-key".to_string()); + let config_store = SpyConfigStore::new(data); + let secret_store = SpySecretStore::new(); + let services = + build_services_with_config_and_secret(config_store, secret_store); + + let manager = KeyRotationManager::new("cfg-id", "sec-id"); + let result = manager.deactivate_key(&services, "only-key"); + + assert!( + result.is_err(), + "should fail to deactivate the last active key" + ); + } + + #[test] + fn key_rotation_result_structure_is_valid() { + let jwk = Keypair::generate().get_jwk("test-key".to_string()); + let result = KeyRotationResult { + new_kid: "ts-2024-01-01".to_string(), + previous_kid: Some("ts-2023-12-31".to_string()), + active_kids: vec![ + "ts-2023-12-31".to_string(), + "ts-2024-01-01".to_string(), + ], + jwk: jwk.clone(), + }; + + assert_eq!(result.new_kid, "ts-2024-01-01"); + assert_eq!(result.previous_kid, Some("ts-2023-12-31".to_string())); + assert_eq!(result.active_kids.len(), 2); + assert_eq!(result.jwk.prm.kid, Some("test-key".to_string())); + } +} +``` + +- [ ] **Step 2: Run tests to confirm they fail (expected compile error)** + +```bash +cargo test --package trusted-server-core request_signing::rotation 2>&1 | head -30 +``` + +Expected: compile error — `KeyRotationManager::new` still returns `Result`, and `rotate_key` doesn't take `services`. + +- [ ] **Step 3: Rewrite `rotation.rs`** + +Replace the entire file with the following (preserving `generate_date_based_kid` and `KeyRotationResult`): + +```rust +//! Key rotation management for request signing. +//! +//! This module provides functionality for rotating signing keys, managing key +//! lifecycle, and storing keys via platform store primitives through +//! [`RuntimeServices`]. + +use std::sync::LazyLock; + +use base64::{engine::general_purpose, Engine}; +use ed25519_dalek::SigningKey; +use error_stack::{Report, ResultExt}; +use jose_jwk::Jwk; + +use crate::error::TrustedServerError; +use crate::platform::{RuntimeServices, StoreId, StoreName}; +use crate::request_signing::JWKS_CONFIG_STORE_NAME; + +use super::Keypair; + +static JWKS_STORE_NAME: LazyLock = + LazyLock::new(|| StoreName::from(JWKS_CONFIG_STORE_NAME)); + +#[derive(Debug, Clone)] +pub struct KeyRotationResult { + pub new_kid: String, + pub previous_kid: Option, + pub active_kids: Vec, + pub jwk: Jwk, +} + +/// Manages signing key lifecycle using platform store primitives. +/// +/// Reads use the edge-visible store name ([`JWKS_CONFIG_STORE_NAME`]). +/// Writes use the management API store identifiers supplied at construction. +pub struct KeyRotationManager { + /// Management API store ID for config store writes. + config_store_id: StoreId, + /// Management API store ID for secret store writes. + secret_store_id: StoreId, +} + +impl KeyRotationManager { + /// Creates a new key rotation manager. + /// + /// The `config_store_id` and `secret_store_id` are platform management API + /// identifiers used for write operations. Edge reads use the store names + /// defined in [`JWKS_CONFIG_STORE_NAME`] and + /// [`crate::request_signing::SIGNING_SECRET_STORE_NAME`]. + #[must_use] + pub fn new( + config_store_id: impl Into, + secret_store_id: impl Into, + ) -> Self { + Self { + config_store_id: StoreId::from(config_store_id.into()), + secret_store_id: StoreId::from(secret_store_id.into()), + } + } + + /// Rotates the signing key by generating a new keypair and storing it. + /// + /// # Errors + /// + /// Returns an error if key storage or update operations fail. + pub fn rotate_key( + &self, + services: &RuntimeServices, + kid: Option, + ) -> Result> { + let new_kid = kid.unwrap_or_else(generate_date_based_kid); + + let keypair = Keypair::generate(); + let jwk = keypair.get_jwk(new_kid.clone()); + let previous_kid = services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .ok(); + + self.store_private_key(services, &new_kid, &keypair.signing_key)?; + self.store_public_jwk(services, &new_kid, &jwk)?; + + let active_kids = match &previous_kid { + Some(prev) if prev != &new_kid => vec![prev.clone(), new_kid.clone()], + _ => vec![new_kid.clone()], + }; + + self.update_current_kid(services, &new_kid)?; + self.update_active_kids(services, &active_kids)?; + + Ok(KeyRotationResult { + new_kid, + previous_kid, + active_kids, + jwk, + }) + } + + fn store_private_key( + &self, + services: &RuntimeServices, + kid: &str, + signing_key: &SigningKey, + ) -> Result<(), Report> { + let key_b64 = general_purpose::STANDARD.encode(signing_key.as_bytes()); + + services + .secret_store() + .create(&self.secret_store_id, kid, &key_b64) + .change_context(TrustedServerError::Configuration { + message: format!("failed to store private key '{}'", kid), + }) + } + + fn store_public_jwk( + &self, + services: &RuntimeServices, + kid: &str, + jwk: &Jwk, + ) -> Result<(), Report> { + let jwk_json = serde_json::to_string(jwk).map_err(|e| { + Report::new(TrustedServerError::Configuration { + message: format!("failed to serialize JWK: {}", e), + }) + })?; + + services + .config_store() + .put(&self.config_store_id, kid, &jwk_json) + .change_context(TrustedServerError::Configuration { + message: format!("failed to store public JWK '{}'", kid), + }) + } + + fn update_current_kid( + &self, + services: &RuntimeServices, + kid: &str, + ) -> Result<(), Report> { + services + .config_store() + .put(&self.config_store_id, "current-kid", kid) + .change_context(TrustedServerError::Configuration { + message: "failed to update current-kid".into(), + }) + } + + fn update_active_kids( + &self, + services: &RuntimeServices, + active_kids: &[String], + ) -> Result<(), Report> { + let active_kids_str = active_kids.join(","); + + services + .config_store() + .put(&self.config_store_id, "active-kids", &active_kids_str) + .change_context(TrustedServerError::Configuration { + message: "failed to update active-kids".into(), + }) + } + + /// Lists all active key IDs. + /// + /// # Errors + /// + /// Returns an error if the active keys cannot be retrieved from the config store. + pub fn list_active_keys( + &self, + services: &RuntimeServices, + ) -> Result, Report> { + let active_kids_str = services + .config_store() + .get(&JWKS_STORE_NAME, "active-kids") + .change_context(TrustedServerError::Configuration { + message: "failed to read active-kids from config store".into(), + })?; + + Ok(active_kids_str + .split(',') + .map(|s| s.trim().to_string()) + .filter(|s| !s.is_empty()) + .collect()) + } + + /// Deactivates a key by removing it from the active keys list. + /// + /// # Errors + /// + /// Returns an error if this would deactivate the last active key, or if the update fails. + pub fn deactivate_key( + &self, + services: &RuntimeServices, + kid: &str, + ) -> Result<(), Report> { + let mut active_kids = self.list_active_keys(services)?; + active_kids.retain(|k| k != kid); + + if active_kids.is_empty() { + return Err(Report::new(TrustedServerError::Configuration { + message: "cannot deactivate the last active key".into(), + })); + } + + self.update_active_kids(services, &active_kids) + } + + /// Deletes a key by deactivating it and removing it from storage. + /// + /// # Errors + /// + /// Returns an error if deactivation fails or if the key cannot be deleted from storage. + pub fn delete_key( + &self, + services: &RuntimeServices, + kid: &str, + ) -> Result<(), Report> { + self.deactivate_key(services, kid)?; + + services + .config_store() + .delete(&self.config_store_id, kid) + .change_context(TrustedServerError::Configuration { + message: "failed to delete JWK from config store".into(), + })?; + + services + .secret_store() + .delete(&self.secret_store_id, kid) + .change_context(TrustedServerError::Configuration { + message: "failed to delete signing key from secret store".into(), + })?; + + Ok(()) + } +} + +#[must_use] +pub fn generate_date_based_kid() -> String { + use chrono::Utc; + format!("ts-{}", Utc::now().format("%Y-%m-%d")) +} +``` + +(Append the test module from Step 1 at the bottom.) + +- [ ] **Step 4: Run rotation tests** + +```bash +cargo test --package trusted-server-core request_signing::rotation -- --nocapture +``` + +Expected: all tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/request_signing/rotation.rs +git commit -m "Migrate KeyRotationManager from FastlyApiClient to RuntimeServices store primitives" +``` + +--- + +### Task 5: Migrate `signing.rs` to `RuntimeServices` + +**Why:** Three items in `signing.rs` still construct `FastlyConfigStore`/`FastlySecretStore` directly. Replace all three with `RuntimeServices`. The existing viceroy-dependent tests are replaced with proper unit tests using stub stores. + +**Changed signatures:** + +- `get_current_key_id()` → `get_current_key_id(services: &RuntimeServices)` +- `RequestSigner::from_config()` → `RequestSigner::from_services(services: &RuntimeServices)` (rename to make the break explicit) +- `verify_signature(payload, sig, kid)` → `verify_signature(payload, sig, kid, services: &RuntimeServices)` + +**Files:** + +- Modify: `crates/trusted-server-core/src/request_signing/signing.rs` + +- [ ] **Step 1: Write failing tests for the new API** + +Replace the entire `#[cfg(test)]` module in `signing.rs` with the following (before updating the production code, so the tests fail to compile): + +```rust +#[cfg(test)] +mod tests { + use std::collections::HashMap; + + use error_stack::Report; + + use crate::platform::test_support::build_services_with_config_and_secret; + use crate::platform::{PlatformConfigStore, PlatformError, PlatformSecretStore, StoreId, StoreName}; + + use super::*; + + // --------------------------------------------------------------------------- + // Stub stores with preset data + // --------------------------------------------------------------------------- + + struct StubConfigStore(HashMap); + + impl PlatformConfigStore for StubConfigStore { + fn get(&self, _: &StoreName, key: &str) -> Result> { + self.0 + .get(key) + .cloned() + .ok_or_else(|| Report::new(PlatformError::ConfigStore)) + } + + fn put(&self, _: &StoreId, _: &str, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + + fn delete(&self, _: &StoreId, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + } + + struct StubSecretStore(HashMap>); + + impl PlatformSecretStore for StubSecretStore { + fn get_bytes(&self, _: &StoreName, key: &str) -> Result, Report> { + self.0 + .get(key) + .cloned() + .ok_or_else(|| Report::new(PlatformError::SecretStore)) + } + + fn create(&self, _: &StoreId, _: &str, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + + fn delete(&self, _: &StoreId, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + } + + /// Build `RuntimeServices` with a real Ed25519 keypair pre-loaded in the + /// stub stores. Returns the `kid` used so callers can reference it. + fn build_signing_services() -> crate::platform::RuntimeServices { + use base64::{engine::general_purpose, Engine}; + use ed25519_dalek::SigningKey; + use rand::rngs::OsRng; + + let signing_key = SigningKey::generate(&mut OsRng); + let key_b64 = general_purpose::STANDARD.encode(signing_key.as_bytes()); + let verifying_key = signing_key.verifying_key(); + let x_b64 = general_purpose::URL_SAFE_NO_PAD.encode(verifying_key.as_bytes()); + let jwk_json = format!( + r#"{{"kty":"OKP","crv":"Ed25519","x":"{}","kid":"test-kid","alg":"EdDSA"}}"#, + x_b64 + ); + + let mut config_data = HashMap::new(); + config_data.insert("current-kid".to_string(), "test-kid".to_string()); + config_data.insert("test-kid".to_string(), jwk_json); + + let mut secret_data = HashMap::new(); + secret_data.insert("test-kid".to_string(), key_b64.into_bytes()); + + build_services_with_config_and_secret( + StubConfigStore(config_data), + StubSecretStore(secret_data), + ) + } + + // --------------------------------------------------------------------------- + // Tests + // --------------------------------------------------------------------------- + + #[test] + fn from_services_loads_kid_from_config_store() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + + assert_eq!(signer.kid, "test-kid", "should load kid from config store"); + } + + #[test] + fn sign_produces_non_empty_url_safe_base64_signature() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + + let signature = signer + .sign(b"these pretzels are making me thirsty") + .expect("should sign payload"); + + assert!(!signature.is_empty(), "should produce non-empty signature"); + assert!(signature.len() > 32, "should produce a full-length signature"); + } + + #[test] + fn sign_and_verify_roundtrip_succeeds() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + let payload = b"test payload for verification"; + + let signature = signer.sign(payload).expect("should sign payload"); + let verified = verify_signature(payload, &signature, &signer.kid, &services) + .expect("should attempt verification"); + + assert!(verified, "should verify a valid signature"); + } + + #[test] + fn verify_returns_false_for_wrong_payload() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + let signature = signer.sign(b"original").expect("should sign"); + + let verified = verify_signature(b"wrong payload", &signature, &signer.kid, &services) + .expect("should attempt verification"); + + assert!(!verified, "should not verify signature for wrong payload"); + } + + #[test] + fn verify_errors_for_unknown_kid() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + let signature = signer.sign(b"payload").expect("should sign"); + + let result = verify_signature(b"payload", &signature, "nonexistent-kid", &services); + + assert!(result.is_err(), "should error for unknown kid"); + } + + #[test] + fn verify_errors_for_malformed_signature() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + + let result = verify_signature(b"payload", "not-valid-base64!!!", &signer.kid, &services); + + assert!(result.is_err(), "should error for malformed signature"); + } + + #[test] + fn signing_params_build_payload_serializes_all_fields() { + let params = SigningParams { + request_id: "req-123".to_string(), + request_host: "example.com".to_string(), + request_scheme: "https".to_string(), + timestamp: 1706900000, + }; + + let payload = params.build_payload("kid-abc").expect("should build payload"); + let parsed: serde_json::Value = + serde_json::from_str(&payload).expect("should be valid JSON"); + + assert_eq!(parsed["version"], SIGNING_VERSION); + assert_eq!(parsed["kid"], "kid-abc"); + assert_eq!(parsed["host"], "example.com"); + assert_eq!(parsed["scheme"], "https"); + assert_eq!(parsed["id"], "req-123"); + assert_eq!(parsed["ts"], 1706900000); + } + + #[test] + fn signing_params_new_creates_recent_timestamp() { + let params = SigningParams::new( + "req-123".to_string(), + "example.com".to_string(), + "https".to_string(), + ); + + let now_ms = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("should get system time") + .as_millis() as u64; + + assert!( + params.timestamp <= now_ms, + "timestamp should not be in the future" + ); + assert!( + params.timestamp >= now_ms - 60_000, + "timestamp should be within the last minute" + ); + } + + #[test] + fn sign_request_enhanced_produces_verifiable_signature() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + let params = SigningParams::new( + "auction-123".to_string(), + "publisher.com".to_string(), + "https".to_string(), + ); + + let signature = signer.sign_request(¶ms).expect("should sign request"); + let payload = params.build_payload(&signer.kid).expect("should build payload"); + + let verified = + verify_signature(payload.as_bytes(), &signature, &signer.kid, &services) + .expect("should verify"); + + assert!(verified, "enhanced request signature should be verifiable"); + } + + #[test] + fn sign_request_different_hosts_produce_different_signatures() { + let services = build_signing_services(); + let signer = RequestSigner::from_services(&services) + .expect("should create signer from services"); + + let params1 = SigningParams { + request_id: "req-1".to_string(), + request_host: "host1.com".to_string(), + request_scheme: "https".to_string(), + timestamp: 1706900000, + }; + let params2 = SigningParams { + request_id: "req-1".to_string(), + request_host: "host2.com".to_string(), + request_scheme: "https".to_string(), + timestamp: 1706900000, + }; + + let sig1 = signer.sign_request(¶ms1).expect("should sign params1"); + let sig2 = signer.sign_request(¶ms2).expect("should sign params2"); + + assert_ne!( + sig1, sig2, + "different hosts should produce different signatures" + ); + } +} +``` + +- [ ] **Step 2: Run to confirm compile failure** + +```bash +cargo test --package trusted-server-core request_signing::signing 2>&1 | head -20 +``` + +Expected: compile error — `from_services`, `verify_signature` with 4 args not found. + +- [ ] **Step 3: Rewrite `signing.rs` production code** + +Replace the imports, `LazyLock` statics, and function bodies. Key changes: + +**Imports — replace:** + +```rust +use crate::storage::{FastlyConfigStore, FastlySecretStore}; +``` + +**With:** + +```rust +use std::sync::LazyLock; + +use crate::platform::{RuntimeServices, StoreName}; +``` + +**Add after imports:** + +```rust +static JWKS_STORE_NAME: LazyLock = + LazyLock::new(|| StoreName::from(JWKS_CONFIG_STORE_NAME)); + +static SIGNING_STORE_NAME: LazyLock = + LazyLock::new(|| StoreName::from(SIGNING_SECRET_STORE_NAME)); +``` + +**Replace `get_current_key_id`:** + +```rust +pub fn get_current_key_id( + services: &RuntimeServices, +) -> Result> { + services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .change_context(TrustedServerError::Configuration { + message: "failed to read current-kid from config store".into(), + }) +} +``` + +**Replace `RequestSigner::from_config` with `from_services`:** + +```rust +pub fn from_services( + services: &RuntimeServices, +) -> Result> { + let key_id = services + .config_store() + .get(&JWKS_STORE_NAME, "current-kid") + .change_context(TrustedServerError::Configuration { + message: "failed to get current-kid".into(), + })?; + + let key_bytes = services + .secret_store() + .get_bytes(&SIGNING_STORE_NAME, &key_id) + .change_context(TrustedServerError::Configuration { + message: format!("failed to get signing key for kid: {}", key_id), + })?; + + let signing_key = parse_ed25519_signing_key(key_bytes)?; + + Ok(Self { + key: signing_key, + kid: key_id, + }) +} +``` + +**Replace `verify_signature` — add `services: &RuntimeServices` parameter and replace `FastlyConfigStore::new(...)` with `services.config_store().get(&JWKS_STORE_NAME, kid)`.** + +Full new signature: + +```rust +pub fn verify_signature( + payload: &[u8], + signature_b64: &str, + kid: &str, + services: &RuntimeServices, +) -> Result> { + let jwk_json = services + .config_store() + .get(&JWKS_STORE_NAME, kid) + .change_context(TrustedServerError::Configuration { + message: format!("failed to get JWK for kid: {}", kid), + })?; + // ... rest of verification logic unchanged ... +} +``` + +- [ ] **Step 4: Run signing tests** + +```bash +cargo test --package trusted-server-core request_signing::signing -- --nocapture +``` + +Expected: all tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/request_signing/signing.rs +git commit -m "Migrate signing.rs from FastlyConfigStore/FastlySecretStore to RuntimeServices" +``` + +--- + +### Task 6: Update `endpoints.rs` to accept `&RuntimeServices` + +**Why:** Three handlers don't receive `&RuntimeServices`: `handle_verify_signature`, `handle_rotate_key`, `handle_deactivate_key`. They need it to pass to `verify_signature`, `KeyRotationManager` methods, and (for verify) `RequestSigner::from_services`. + +**Note:** `fastly::{Request, Response}` and `fastly::mime` remain — type migration is Phase 2 (PR 12). + +**Files:** + +- Modify: `crates/trusted-server-core/src/request_signing/endpoints.rs` + +- [ ] **Step 1: Update `handle_verify_signature` signature and body** + +Change: + +```rust +pub fn handle_verify_signature( + _settings: &Settings, + mut req: Request, +) -> Result> { +``` + +To: + +```rust +pub fn handle_verify_signature( + _settings: &Settings, + services: &RuntimeServices, + mut req: Request, +) -> Result> { +``` + +Update the `verify_signature` call: + +```rust +let verification_result = signing::verify_signature( + verify_req.payload.as_bytes(), + &verify_req.signature, + &verify_req.kid, + services, +); +``` + +- [ ] **Step 2: Update `handle_rotate_key` signature and body** + +Change signature to add `services: &RuntimeServices` as second parameter. Update the `KeyRotationManager` usage: + +```rust +// Before: +let manager = KeyRotationManager::new(config_store_id, secret_store_id).change_context(...)?; +match manager.rotate_key(rotate_req.kid) { ... } +manager.list_active_keys().unwrap_or_else(...) + +// After: +let manager = KeyRotationManager::new(config_store_id, secret_store_id); +match manager.rotate_key(services, rotate_req.kid) { ... } +manager.list_active_keys(services).unwrap_or_else(...) +``` + +Remove the `.change_context(...)` on `KeyRotationManager::new(...)` — it's now infallible. + +- [ ] **Step 3: Update `handle_deactivate_key` signature and body** + +Same pattern: add `services: &RuntimeServices`, update all `manager.*` calls to pass `services`: + +- `manager.delete_key(&deactivate_req.kid)` → `manager.delete_key(services, &deactivate_req.kid)` +- `manager.deactivate_key(&deactivate_req.kid)` → `manager.deactivate_key(services, &deactivate_req.kid)` +- `manager.list_active_keys()` → `manager.list_active_keys(services)` + +Remove the `.change_context(...)` on `KeyRotationManager::new(...)`. + +- [ ] **Step 4: Update `endpoints.rs` tests** + +The tests in `endpoints.rs` that call `handle_verify_signature`, `handle_rotate_key`, `handle_deactivate_key` must be updated to pass a `&RuntimeServices`. Use `noop_services()` (from `test_support`) for rotation/deactivation tests (they test error paths that don't reach the stores). For `test_handle_verify_signature_valid` and `test_handle_verify_signature_invalid`, build a `RuntimeServices` with actual key material using `build_signing_services` (inline the helper or import logic). + +Also update `RequestSigner::from_config()` calls in test helpers to `RequestSigner::from_services(&services)`. + +**Add this helper to the `#[cfg(test)]` block at the top of `endpoints.rs` tests** (it cannot be imported from `signing.rs` because that function lives inside a `#[cfg(test)]` private module): + +```rust +/// Build `RuntimeServices` pre-loaded with a real Ed25519 keypair for +/// testing signature creation and verification in endpoint handlers. +fn build_signing_services_for_test() -> crate::platform::RuntimeServices { + use std::collections::HashMap; + + use base64::{engine::general_purpose, Engine}; + use ed25519_dalek::SigningKey; + use error_stack::Report; + use rand::rngs::OsRng; + + use crate::platform::test_support::build_services_with_config_and_secret; + use crate::platform::{ + PlatformConfigStore, PlatformError, PlatformSecretStore, StoreId, StoreName, + }; + + struct MapConfigStore(HashMap); + impl PlatformConfigStore for MapConfigStore { + fn get(&self, _: &StoreName, key: &str) -> Result> { + self.0.get(key).cloned().ok_or_else(|| Report::new(PlatformError::ConfigStore)) + } + fn put(&self, _: &StoreId, _: &str, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + fn delete(&self, _: &StoreId, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + } + + struct MapSecretStore(HashMap>); + impl PlatformSecretStore for MapSecretStore { + fn get_bytes(&self, _: &StoreName, key: &str) -> Result, Report> { + self.0.get(key).cloned().ok_or_else(|| Report::new(PlatformError::SecretStore)) + } + fn create(&self, _: &StoreId, _: &str, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + fn delete(&self, _: &StoreId, _: &str) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } + } + + let signing_key = SigningKey::generate(&mut OsRng); + let key_b64 = general_purpose::STANDARD.encode(signing_key.as_bytes()); + let x_b64 = general_purpose::URL_SAFE_NO_PAD.encode(signing_key.verifying_key().as_bytes()); + let jwk_json = format!( + r#"{{"kty":"OKP","crv":"Ed25519","x":"{}","kid":"test-kid","alg":"EdDSA"}}"#, + x_b64 + ); + + let mut cfg = HashMap::new(); + cfg.insert("current-kid".to_string(), "test-kid".to_string()); + cfg.insert("test-kid".to_string(), jwk_json); + + let mut sec = HashMap::new(); + sec.insert("test-kid".to_string(), key_b64.into_bytes()); + + build_services_with_config_and_secret(MapConfigStore(cfg), MapSecretStore(sec)) +} +``` + +Pattern for verify tests: + +```rust +// In test_handle_verify_signature_valid: +let services = build_signing_services_for_test(); +let signer = crate::request_signing::RequestSigner::from_services(&services) + .expect("should create signer from services"); +// ... build req as before ... +let mut resp = handle_verify_signature(&settings, &services, req) + .expect("should handle verification request"); +``` + +For rotation/deactivation tests, `noop_services()` is fine — these tests use the `match result { Ok => log, Err => log }` pattern and do not assert against store state. The `noop_services()` causes `KeyRotationManager` methods to fail at the store read/write level, which is the expected behavior in a test environment without real stores: + +```rust +let services = noop_services(); +let result = handle_rotate_key(&settings, &services, req); +// existing match-and-log pattern works unchanged +``` + +- [ ] **Step 5: Run endpoints tests** + +```bash +cargo test --package trusted-server-core request_signing::endpoints -- --nocapture +``` + +Expected: all tests pass. + +- [ ] **Step 6: Commit** + +```bash +git add crates/trusted-server-core/src/request_signing/endpoints.rs +git commit -m "Add RuntimeServices parameter to handle_verify_signature, handle_rotate_key, handle_deactivate_key" +``` + +--- + +### Task 7: Update `main.rs` to pass `runtime_services` to updated handlers + +**Why:** The adapter `main.rs` calls the three handlers without `runtime_services`. Add it. `runtime_services` is already in scope in all call sites. + +**Files:** + +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` + +- [ ] **Step 1: Update the three handler call sites** + +Find (approximate lines, verify exact lines before editing): + +```rust +(Method::POST, "/verify-signature") => handle_verify_signature(settings, req), +(Method::POST, "/admin/keys/rotate") => handle_rotate_key(settings, req), +(Method::POST, "/admin/keys/deactivate") => handle_deactivate_key(settings, req), +``` + +Replace with: + +```rust +(Method::POST, "/verify-signature") => { + handle_verify_signature(settings, runtime_services, req) +} +(Method::POST, "/admin/keys/rotate") => { + handle_rotate_key(settings, runtime_services, req) +} +(Method::POST, "/admin/keys/deactivate") => { + handle_deactivate_key(settings, runtime_services, req) +} +``` + +- [ ] **Step 2: Verify the full workspace compiles** + +```bash +cargo check --workspace +``` + +Expected: no errors. + +- [ ] **Step 3: Commit** + +```bash +git add crates/trusted-server-adapter-fastly/src/main.rs +git commit -m "Pass runtime_services to signing endpoint handlers in main.rs" +``` + +--- + +### Task 8: Delete `api_client.rs` and clean up `storage/mod.rs` + +**Why:** `api_client.rs` is now fully superseded by `management_api.rs` in the adapter. No core code references `FastlyApiClient` anymore (verified by rotation.rs migration). + +**Files:** + +- Delete: `crates/trusted-server-core/src/storage/api_client.rs` +- Modify: `crates/trusted-server-core/src/storage/mod.rs` + +- [ ] **Step 1: Verify zero legacy storage imports remain in `request_signing/`** + +```bash +grep -r "FastlyApiClient\|FastlyConfigStore\|FastlySecretStore" crates/trusted-server-core/src/request_signing/ +``` + +Expected: no output. If any matches appear, fix those call sites before continuing. + +Also verify no remaining references to `FastlyApiClient` anywhere in core: + +```bash +grep -r "FastlyApiClient" crates/trusted-server-core/src/ +``` + +Expected: no output. + +- [ ] **Step 2: Delete `api_client.rs`** + +```bash +rm crates/trusted-server-core/src/storage/api_client.rs +``` + +- [ ] **Step 3: Update `storage/mod.rs`** + +Remove the `api_client` module declaration and re-export. Change from: + +```rust +//! Legacy Fastly-backed store types. +//! +//! These types predate the [`crate::platform`] abstraction and will be removed +//! once all call sites have migrated to the platform traits. New code should +//! use [`crate::platform::PlatformConfigStore`], +//! [`crate::platform::PlatformSecretStore`], and the management write methods +//! via [`crate::platform::RuntimeServices`]. + +pub(crate) mod api_client; +pub(crate) mod config_store; +pub(crate) mod secret_store; + +pub use api_client::FastlyApiClient; +pub use config_store::FastlyConfigStore; +pub use secret_store::FastlySecretStore; +``` + +To: + +```rust +//! Legacy Fastly-backed store types. +//! +//! These types predate the [`crate::platform`] abstraction and will be removed +//! once all call sites have migrated to the platform traits. New code should +//! use [`crate::platform::PlatformConfigStore`] and +//! [`crate::platform::PlatformSecretStore`] via [`crate::platform::RuntimeServices`]. + +pub(crate) mod config_store; +pub(crate) mod secret_store; + +pub use config_store::FastlyConfigStore; +pub use secret_store::FastlySecretStore; +``` + +- [ ] **Step 4: Run the full workspace test suite** + +```bash +cargo test --workspace +``` + +Expected: all tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/storage/mod.rs +git rm crates/trusted-server-core/src/storage/api_client.rs +git commit -m "Delete storage/api_client.rs from core; remove FastlyApiClient" +``` + +--- + +### Task 9: Run CI gates + +- [ ] **Step 1: Format check** + +```bash +cargo fmt --all -- --check +``` + +If it fails, fix with `cargo fmt --all` and re-run. + +- [ ] **Step 2: Clippy** + +```bash +cargo clippy --workspace --all-targets --all-features -- -D warnings +``` + +Fix any lints before proceeding. + +- [ ] **Step 3: Full test suite** + +```bash +cargo test --workspace +``` + +Expected: all tests pass. + +- [ ] **Step 4: Commit any lint/format fixes** + +```bash +git add -A +git commit -m "Fix clippy lints and formatting" +``` + +Only create this commit if there are actual changes. + +--- + +## Acceptance Checklist + +Verify all of the following before considering PR 9 complete: + +- [ ] `crates/trusted-server-core/src/storage/api_client.rs` no longer exists +- [ ] `crates/trusted-server-adapter-fastly/src/management_api.rs` exists +- [ ] `grep -r "FastlyApiClient\|from crate::storage::api" crates/trusted-server-core/src/request_signing/` returns no matches +- [ ] `grep -r "FastlyConfigStore\|FastlySecretStore" crates/trusted-server-core/src/request_signing/` returns no matches +- [ ] `cargo test --workspace` passes +- [ ] `cargo clippy --workspace --all-targets --all-features -- -D warnings` passes +- [ ] `cargo fmt --all -- --check` passes +- [ ] `handle_verify_signature`, `handle_rotate_key`, `handle_deactivate_key` in `endpoints.rs` all accept `&RuntimeServices` +- [ ] `FastlyPlatformConfigStore::put/delete` and `FastlyPlatformSecretStore::create/delete` in `platform.rs` no longer return `PlatformError::NotImplemented` diff --git a/docs/superpowers/plans/2026-04-02-pr10-logging-initialization.md b/docs/superpowers/plans/2026-04-02-pr10-logging-initialization.md new file mode 100644 index 000000000..88a39b55d --- /dev/null +++ b/docs/superpowers/plans/2026-04-02-pr10-logging-initialization.md @@ -0,0 +1,289 @@ +# PR 10 Logging Initialization Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Keep logging backend initialization adapter-owned by extracting Fastly logging setup into an adapter-local module and removing `log-fastly` from `trusted-server-core`. + +**Architecture:** `trusted-server-core` continues to emit logs only through `log` macros and has no platform logging backend dependency. `trusted-server-adapter-fastly` owns Fastly-specific logger initialization behind a local `logging.rs` module, and `main.rs` just calls into that adapter-local entrypoint. + +**Tech Stack:** Rust 2024 edition conventions, `log`, `log-fastly`, `fern`, `chrono` + +--- + +## File Structure + +- Create: `crates/trusted-server-adapter-fastly/src/logging.rs` + - Own Fastly-specific logger setup and any small formatting helpers needed for unit testing. +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` + - Stop carrying logger implementation details directly; import the adapter-local module and call `logging::init_logger()`. +- Modify: `crates/trusted-server-core/Cargo.toml` + - Remove `log-fastly` from core dependencies. +- Modify: `Cargo.lock` + - Lockfile update after dependency removal. + +The plan intentionally avoids any core logging trait or shared abstraction. Future adapters can mirror the same adapter-local module shape without forcing a premature common interface. + +## Tasks + +### Task 1: Extract Fastly logger helper and initializer into an adapter-local module + +**Files:** + +- Create: `crates/trusted-server-adapter-fastly/src/logging.rs` + +- [ ] **Step 1: Write a failing unit test for a non-allocating formatting helper** + +Create `crates/trusted-server-adapter-fastly/src/logging.rs` with a test-first skeleton. Add a helper test for the target-label extraction logic without trying to install a global logger: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn target_label_uses_last_target_segment() { + assert_eq!( + target_label("trusted_server_adapter_fastly::proxy"), + "proxy", + "should use the final target segment" + ); + } +} +``` + +Also add a production skeleton so the file compiles but the test fails: + +```rust +fn target_label(target: &str) -> &str { + target +} +``` + +- [ ] **Step 2: Run the adapter test to verify it fails** + +Run: + +```bash +cargo test --package trusted-server-adapter-fastly logging -- --nocapture +``` + +Expected: FAIL because `target_label()` returns the full target instead of the final segment. + +- [ ] **Step 3: Implement the minimal helper and adapter logger initializer** + +Replace the skeleton with the real adapter-local module: + +```rust +use chrono::{SecondsFormat, Utc}; +use log_fastly::Logger; + +fn target_label(target: &str) -> &str { + match target.rsplit_once("::") { + Some((head, "")) => head, + Some((_, last)) => last, + None => target, + } +} + +pub(crate) fn init_logger() { + let logger = Logger::builder() + .default_endpoint("tslog") + .echo_stdout(true) + .max_level(log::LevelFilter::Info) + .build() + .expect("should build Logger"); + + fern::Dispatch::new() + .format(|out, message, record| { + out.finish(format_args!( + "{} {} [{}] {}", + Utc::now().to_rfc3339_opts(SecondsFormat::Millis, true), + record.level(), + target_label(record.target()), + message + )); + }) + .chain(Box::new(logger) as Box) + .apply() + .expect("should initialize logger"); +} +``` + +Keep the logic semantically equivalent to the current `main.rs` formatting and avoid introducing a new heap allocation on each log call. + +- [ ] **Step 4: Run the adapter test to verify it passes** + +Run: + +```bash +cargo test --package trusted-server-adapter-fastly logging -- --nocapture +``` + +Expected: PASS. + +- [ ] **Step 5: Commit the extracted logging module** + +```bash +git add crates/trusted-server-adapter-fastly/src/logging.rs +git commit -m "Extract Fastly logging initialization into adapter module" +``` + +--- + +### Task 2: Wire `main.rs` to the adapter-local logging module + +**Files:** + +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` + +- [ ] **Step 1: Write a failing compile-time integration step for the new module wiring** + +Update `main.rs` to reference `logging::init_logger()` before the module exists in the file: + +```rust +mod logging; + +#[fastly::main] +fn main(req: Request) -> Result { + logging::init_logger(); + // ... +} +``` + +Delete the old inline `init_logger()` function and remove imports that only it used: + +- `use log_fastly::Logger;` +- any `chrono`/`fern` imports that are no longer needed in `main.rs` + +- [ ] **Step 2: Run the adapter package tests to verify the extraction is wired correctly** + +Run: + +```bash +cargo test --package trusted-server-adapter-fastly -- --nocapture +``` + +Expected: PASS. If compilation fails, fix the module imports and remaining references in `main.rs`. + +- [ ] **Step 3: Commit the adapter wiring cleanup** + +```bash +git add crates/trusted-server-adapter-fastly/src/main.rs +git commit -m "Wire Fastly main.rs to adapter-local logging module" +``` + +--- + +### Task 3: Remove `log-fastly` from core + +**Files:** + +- Modify: `crates/trusted-server-core/Cargo.toml` +- Modify: `Cargo.lock` + +- [ ] **Step 1: Verify core does not reference `log-fastly` directly** + +Run: + +```bash +rg -n "log_fastly|Logger::builder|Logger::from_env" crates/trusted-server-core +``` + +Expected: no matches. + +- [ ] **Step 2: Remove `log-fastly` from core dependencies** + +In `crates/trusted-server-core/Cargo.toml`, delete: + +```toml +log-fastly = { workspace = true } +``` + +Do not remove `log = { workspace = true }`. + +- [ ] **Step 3: Update the lockfile** + +Run: + +```bash +cargo test --package trusted-server-core --lib --no-run +``` + +Expected: `Cargo.lock` updates only as needed for the dependency graph while core still compiles. + +- [ ] **Step 4: Confirm `log-fastly` remains adapter-only** + +Run: + +```bash +rg -n "log-fastly" crates +``` + +Expected: match only in `crates/trusted-server-adapter-fastly/Cargo.toml`. + +- [ ] **Step 5: Commit the dependency cleanup** + +```bash +git add crates/trusted-server-core/Cargo.toml Cargo.lock +git commit -m "Remove log-fastly from trusted-server-core" +``` + +--- + +### Task 4: Run project verification gates + +**Files:** + +- Verify the whole workspace after the logging extraction and dependency cleanup + +- [ ] **Step 1: Format check** + +Run: + +```bash +cargo fmt --all -- --check +``` + +Expected: PASS. If it fails, run `cargo fmt --all` and re-run the check. + +- [ ] **Step 2: Clippy** + +Run: + +```bash +cargo clippy --workspace --all-targets --all-features -- -D warnings +``` + +Expected: PASS. + +- [ ] **Step 3: Full workspace tests** + +Run: + +```bash +cargo test --workspace +``` + +Expected: PASS. + +- [ ] **Step 4: Commit any formatting fallout** + +Only if `cargo fmt --all` changed files: + +```bash +git add -A +git commit -m "Fix formatting after logging extraction" +``` + +--- + +## Acceptance Checklist + +- [ ] `crates/trusted-server-adapter-fastly/src/logging.rs` exists +- [ ] `crates/trusted-server-adapter-fastly/src/main.rs` no longer contains the inline Fastly logger implementation +- [ ] `crates/trusted-server-core/Cargo.toml` no longer depends on `log-fastly` +- [ ] `rg -n "log-fastly" crates` reports only the Fastly adapter crate +- [ ] `trusted-server-core` still uses `log` macros and compiles without a Fastly-specific logging backend dependency +- [ ] `cargo fmt --all -- --check` passes +- [ ] `cargo clippy --workspace --all-targets --all-features -- -D warnings` passes +- [ ] `cargo test --workspace` passes diff --git a/docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md b/docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md new file mode 100644 index 000000000..4bf5af218 --- /dev/null +++ b/docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md @@ -0,0 +1,69 @@ +# PR 10 Logging Initialization Design + +**Goal:** Keep logging backend initialization adapter-owned so `trusted-server-core` remains platform-agnostic while Fastly continues to initialize its own `log-fastly` backend. + +## Problem + +`trusted-server-core` still declares a `log-fastly` dependency even though log +backend setup already happens in the Fastly adapter entrypoint. That keeps a +Fastly-specific crate in the core dependency graph and weakens the migration +boundary needed for future EdgeZero adapters such as Cloudflare, Spin, and +Axum. + +## Design + +### Responsibility split + +- `trusted-server-core` emits logs only through `log` macros. +- Each adapter crate owns logger initialization and backend wiring. +- Fastly-specific logger setup moves behind an adapter-local module boundary. + +This keeps core free of platform logging backends while establishing a clean +pattern future adapters can mirror without forcing a shared abstraction too +early. + +### Fastly adapter shape + +Create an adapter-local logging module in +`crates/trusted-server-adapter-fastly/src/logging.rs` that exposes a small +`init_logger()` function. The implementation stays Fastly-specific and can keep +using `log-fastly`, `fern`, and the existing formatting choices. + +`crates/trusted-server-adapter-fastly/src/main.rs` should only import that +module and call `logging::init_logger()` during startup. + +### Core crate shape + +Remove `log-fastly` from +`crates/trusted-server-core/Cargo.toml`. No production code in core should +change unless compilation reveals an unexpected dependency. The intended end +state is that core depends on `log` only. + +## File impact + +- Create: `crates/trusted-server-adapter-fastly/src/logging.rs` +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` +- Modify: `crates/trusted-server-core/Cargo.toml` +- Modify: `Cargo.lock` + +## Testing and verification + +- Add or update small adapter-local tests only if needed for the extracted + logging module. +- Run the standard project gates: + - `cargo fmt --all -- --check` + - `cargo clippy --workspace --all-targets --all-features -- -D warnings` + - `cargo test --workspace` + +## Out of scope + +- Introducing a cross-adapter logging trait in core +- Changing log formatting semantics beyond what is needed to extract the module +- Adding logging implementations for non-Fastly adapters + +## Acceptance + +- `log-fastly` exists only in the Fastly adapter dependency graph +- Core uses `log` macros without any Fastly-specific logging backend dependency +- Fastly adapter still initializes logging at startup +- Workspace verification gates pass From 24d451568863759b3adb439f96193b333e3fb442 Mon Sep 17 00:00:00 2001 From: Christian Date: Fri, 1 May 2026 09:45:20 -0500 Subject: [PATCH 20/22] Remove generated Prebid user ID shim --- .../prebid/_user_ids.generated.ts | 22 ------------------- .../js/lib/src/integrations/prebid/index.ts | 5 ----- .../src/auction/test_support.rs | 4 +--- .../src/platform/test_support.rs | 19 ++++++++++++++++ 4 files changed, 20 insertions(+), 30 deletions(-) delete mode 100644 crates/js/lib/src/integrations/prebid/_user_ids.generated.ts diff --git a/crates/js/lib/src/integrations/prebid/_user_ids.generated.ts b/crates/js/lib/src/integrations/prebid/_user_ids.generated.ts deleted file mode 100644 index 9eb586ddd..000000000 --- a/crates/js/lib/src/integrations/prebid/_user_ids.generated.ts +++ /dev/null @@ -1,22 +0,0 @@ -// Auto-generated by build-all.mjs — manual edits will be overwritten at build time. -// -// Controls which Prebid.js User ID submodules are included in the bundle. -// Set the TSJS_PREBID_USER_IDS environment variable to a comma-separated -// list of submodule filenames without the `.js` extension -// (e.g. "sharedIdSystem,id5IdSystem,criteoIdSystem") before building. -// The userId.js core module is always included via a static import in -// index.ts and is not configurable here. - -import 'prebid.js/modules/sharedIdSystem.js'; -import 'prebid.js/modules/criteoIdSystem.js'; -import 'prebid.js/modules/33acrossIdSystem.js'; -import 'prebid.js/modules/pubProvidedIdSystem.js'; -import 'prebid.js/modules/quantcastIdSystem.js'; -import 'prebid.js/modules/id5IdSystem.js'; -import 'prebid.js/modules/identityLinkIdSystem.js'; -import 'prebid.js/modules/uid2IdSystem.js'; -import 'prebid.js/modules/euidIdSystem.js'; -import 'prebid.js/modules/intentIqIdSystem.js'; -import 'prebid.js/modules/lotamePanoramaIdSystem.js'; -import 'prebid.js/modules/connectIdSystem.js'; -import 'prebid.js/modules/merkleIdSystem.js'; diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index 53255d837..b8d6d73b6 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -18,11 +18,6 @@ import 'prebid.js/modules/consentManagementGpp.js'; import 'prebid.js/modules/consentManagementUsp.js'; import 'prebid.js/modules/userId.js'; -// Prebid User ID submodules — self-register with the core on import. -// The set of submodules is controlled by the TSJS_PREBID_USER_IDS env var -// at build time. See _user_ids.generated.ts (written by build-all.mjs). -import './_user_ids.generated'; - // Client-side bid adapters — self-register with prebid.js on import. // The set of adapters is controlled by the TSJS_PREBID_ADAPTERS env var at // build time. See _adapters.generated.ts (written by build-all.mjs). diff --git a/crates/trusted-server-core/src/auction/test_support.rs b/crates/trusted-server-core/src/auction/test_support.rs index 2c5b54383..bfcdd10de 100644 --- a/crates/trusted-server-core/src/auction/test_support.rs +++ b/crates/trusted-server-core/src/auction/test_support.rs @@ -3,7 +3,7 @@ use std::sync::LazyLock; use fastly::Request; use super::AuctionContext; -use crate::platform::{test_support::noop_services, ClientInfo, RuntimeServices}; +use crate::platform::{test_support::noop_services, RuntimeServices}; use crate::settings::Settings; static TEST_SERVICES: LazyLock = LazyLock::new(noop_services); @@ -11,14 +11,12 @@ static TEST_SERVICES: LazyLock = LazyLock::new(noop_services); pub(crate) fn create_test_auction_context<'a>( settings: &'a Settings, request: &'a Request, - client_info: &'a ClientInfo, timeout_ms: u32, ) -> AuctionContext<'a> { let services: &'static RuntimeServices = &TEST_SERVICES; AuctionContext { settings, request, - client_info, timeout_ms, provider_responses: None, services, diff --git a/crates/trusted-server-core/src/platform/test_support.rs b/crates/trusted-server-core/src/platform/test_support.rs index 55a095a90..818819f52 100644 --- a/crates/trusted-server-core/src/platform/test_support.rs +++ b/crates/trusted-server-core/src/platform/test_support.rs @@ -214,6 +214,8 @@ pub(crate) struct StubHttpClient { calls: Mutex>, // (status_code, body_bytes) — kept Send by avoiding Body::Stream responses: Mutex)>>, + // Headers captured per send call, stored as (name, value) string pairs. + request_headers: Mutex>>, } impl StubHttpClient { @@ -221,6 +223,7 @@ impl StubHttpClient { Self { calls: Mutex::new(Vec::new()), responses: Mutex::new(VecDeque::new()), + request_headers: Mutex::new(Vec::new()), } } @@ -250,6 +253,22 @@ impl PlatformHttpClient for StubHttpClient { .expect("should lock calls") .push(request.backend_name.clone()); + let headers: Vec<(String, String)> = request + .request + .headers() + .iter() + .filter_map(|(name, value)| { + value + .to_str() + .ok() + .map(|v| (name.as_str().to_string(), v.to_string())) + }) + .collect(); + self.request_headers + .lock() + .expect("should lock request_headers") + .push(headers); + let (status, body_bytes) = self .responses .lock() From a2c6d831c909dd1e836df1d69870acc5b5138231 Mon Sep 17 00:00:00 2001 From: Christian Date: Tue, 5 May 2026 07:48:17 -0500 Subject: [PATCH 21/22] Address Sourcepoint consent review feedback --- .../lib/src/integrations/sourcepoint/index.ts | 76 +++++++++-- .../integrations/sourcepoint/index.test.ts | 127 ++++++++++++------ crates/trusted-server-core/src/consent/gpp.rs | 2 +- crates/trusted-server-core/src/consent/mod.rs | 1 + .../src/integrations/registry.rs | 3 +- docs/guide/integrations/prebid.md | 39 +----- ...26-04-15-sourcepoint-gpp-consent-design.md | 37 +++-- 7 files changed, 192 insertions(+), 93 deletions(-) diff --git a/crates/js/lib/src/integrations/sourcepoint/index.ts b/crates/js/lib/src/integrations/sourcepoint/index.ts index 1e3d7cea2..0e31b0869 100644 --- a/crates/js/lib/src/integrations/sourcepoint/index.ts +++ b/crates/js/lib/src/integrations/sourcepoint/index.ts @@ -3,6 +3,9 @@ import { log } from '../../core/log'; const SP_CONSENT_PREFIX = '_sp_user_consent_'; const GPP_COOKIE_NAME = '__gpp'; const GPP_SID_COOKIE_NAME = '__gpp_sid'; +const GPP_SOURCE_COOKIE_NAME = '_ts_gpp_src'; +const GPP_SOURCE_SOURCEPOINT = 'sp'; +const INITIAL_RETRY_DELAY_MS = 500; interface SourcepointGppData { gppString: string; @@ -13,6 +16,8 @@ interface SourcepointConsentPayload { gppData?: SourcepointGppData; } +let initialized = false; + function findSourcepointConsent(): SourcepointConsentPayload | null { // Sourcepoint stores one consent payload per property under `_sp_user_consent_*`. // We intentionally take the first valid match and mirror that origin-scoped payload. @@ -35,16 +40,52 @@ function findSourcepointConsent(): SourcepointConsentPayload | null { return null; } +function readCookie(name: string): string | undefined { + const prefix = `${name}=`; + const cookie = document.cookie.split('; ').find((entry) => entry.startsWith(prefix)); + return cookie?.slice(prefix.length); +} + +function hasSourcepointMarker(): boolean { + return readCookie(GPP_SOURCE_COOKIE_NAME) === GPP_SOURCE_SOURCEPOINT; +} + function writeCookie(name: string, value: string): void { document.cookie = `${name}=${value}; path=/; Secure; SameSite=Lax`; } function clearCookie(name: string): void { - // Trusted Server is the only intended writer for these mirrored cookies, so - // clearing the origin-scoped cookie is sufficient for this integration. document.cookie = `${name}=; path=/; Secure; SameSite=Lax; Max-Age=0`; } +function clearSourcepointCookies(): void { + if (!hasSourcepointMarker()) { + return; + } + + clearCookie(GPP_COOKIE_NAME); + clearCookie(GPP_SID_COOKIE_NAME); + clearCookie(GPP_SOURCE_COOKIE_NAME); +} + +function mirrorOnVisible(): void { + if (document.visibilityState === 'visible') { + mirrorSourcepointConsent(); + } +} + +function scheduleInitialRetry(): void { + const retry = (): void => { + mirrorSourcepointConsent(); + }; + + if (document.readyState === 'loading') { + document.addEventListener('DOMContentLoaded', retry, { once: true }); + } + + window.setTimeout(retry, INITIAL_RETRY_DELAY_MS); +} + /** * Reads Sourcepoint consent from localStorage and mirrors it into * `__gpp` and `__gpp_sid` cookies for Trusted Server to read. @@ -58,25 +99,24 @@ export function mirrorSourcepointConsent(): boolean { const payload = findSourcepointConsent(); if (!payload?.gppData) { - clearCookie(GPP_COOKIE_NAME); - clearCookie(GPP_SID_COOKIE_NAME); + clearSourcepointCookies(); log.debug('sourcepoint: no GPP data found in localStorage'); return false; } const { gppString, applicableSections } = payload.gppData; if (!gppString) { - clearCookie(GPP_COOKIE_NAME); - clearCookie(GPP_SID_COOKIE_NAME); + clearSourcepointCookies(); log.debug('sourcepoint: gppString is empty'); return false; } + writeCookie(GPP_SOURCE_COOKIE_NAME, GPP_SOURCE_SOURCEPOINT); writeCookie(GPP_COOKIE_NAME, gppString); if (Array.isArray(applicableSections) && applicableSections.length > 0) { writeCookie(GPP_SID_COOKIE_NAME, applicableSections.join(',')); - } else { + } else if (hasSourcepointMarker()) { clearCookie(GPP_SID_COOKIE_NAME); } @@ -88,8 +128,24 @@ export function mirrorSourcepointConsent(): boolean { return true; } -if (typeof window !== 'undefined') { - mirrorSourcepointConsent(); +/** + * Initializes Sourcepoint consent mirroring and bounded refresh hooks. + */ +export function initializeSourcepointConsentMirror(): void { + if (initialized || typeof window === 'undefined' || typeof document === 'undefined') { + return; + } + + initialized = true; + + if (!mirrorSourcepointConsent()) { + scheduleInitialRetry(); + } + + // Sourcepoint persists consent changes to localStorage. Re-mirror when a + // user returns to the page so session cookies do not remain stale. + document.addEventListener('visibilitychange', mirrorOnVisible); + window.addEventListener('focus', mirrorSourcepointConsent); } -export default mirrorSourcepointConsent; +initializeSourcepointConsentMirror(); diff --git a/crates/js/lib/test/integrations/sourcepoint/index.test.ts b/crates/js/lib/test/integrations/sourcepoint/index.test.ts index f276f2414..3914c2ef9 100644 --- a/crates/js/lib/test/integrations/sourcepoint/index.test.ts +++ b/crates/js/lib/test/integrations/sourcepoint/index.test.ts @@ -1,7 +1,18 @@ -import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { mirrorSourcepointConsent } from '../../../src/integrations/sourcepoint'; +const SOURCEPOINT_MARKER_COOKIE = '_ts_gpp_src'; + +function sourcepointPayload(gppString = 'DBABLA~BVQqAAAAAgA.QA', applicableSections = [7]) { + return { + gppData: { + gppString, + applicableSections, + }, + }; +} + describe('integrations/sourcepoint', () => { function clearAllCookies(): void { document.cookie.split(';').forEach((c) => { @@ -22,34 +33,27 @@ describe('integrations/sourcepoint', () => { }); afterEach(() => { + vi.useRealTimers(); clearAllCookies(); localStorage.clear(); }); it('mirrors __gpp and __gpp_sid from _sp_user_consent_* localStorage as session cookies', () => { - const payload = { - gppData: { - gppString: 'DBABLA~BVQqAAAAAgA.QA', - applicableSections: [7], - }, - }; - localStorage.setItem('_sp_user_consent_36026', JSON.stringify(payload)); + localStorage.setItem('_sp_user_consent_36026', JSON.stringify(sourcepointPayload())); const result = mirrorSourcepointConsent(); expect(result).toBe(true); expect(document.cookie).toContain('__gpp=DBABLA~BVQqAAAAAgA.QA'); expect(document.cookie).toContain('__gpp_sid=7'); + expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBe('sp'); }); it('handles multiple applicable sections', () => { - const payload = { - gppData: { - gppString: 'DBABLA~BVQqAAAAAgA.QA', - applicableSections: [7, 8], - }, - }; - localStorage.setItem('_sp_user_consent_99999', JSON.stringify(payload)); + localStorage.setItem( + '_sp_user_consent_99999', + JSON.stringify(sourcepointPayload('DBABLA~BVQqAAAAAgA.QA', [7, 8])) + ); mirrorSourcepointConsent(); @@ -66,9 +70,22 @@ describe('integrations/sourcepoint', () => { expect(document.cookie).not.toContain('__gpp_sid='); }); - it('clears stale mirrored cookies when no valid Sourcepoint payload exists', () => { + it('does not clear non-Sourcepoint GPP cookies when no valid Sourcepoint payload exists', () => { + document.cookie = '__gpp=other-cmp-gpp; path=/'; + document.cookie = '__gpp_sid=7,8; path=/'; + localStorage.setItem('unrelated_key', 'value'); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(getCookie('__gpp')).toBe('other-cmp-gpp'); + expect(getCookie('__gpp_sid')).toBe('7,8'); + }); + + it('clears stale Sourcepoint-owned mirrored cookies when no valid Sourcepoint payload exists', () => { document.cookie = '__gpp=stale-gpp; path=/'; document.cookie = '__gpp_sid=7,8; path=/'; + document.cookie = `${SOURCEPOINT_MARKER_COOKIE}=sp; path=/`; localStorage.setItem('unrelated_key', 'value'); const result = mirrorSourcepointConsent(); @@ -76,6 +93,7 @@ describe('integrations/sourcepoint', () => { expect(result).toBe(false); expect(getCookie('__gpp')).toBeUndefined(); expect(getCookie('__gpp_sid')).toBeUndefined(); + expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBeUndefined(); }); it('returns false for malformed JSON in localStorage', () => { @@ -89,15 +107,7 @@ describe('integrations/sourcepoint', () => { it('skips malformed entries when a later Sourcepoint key is valid', () => { localStorage.setItem('_sp_user_consent_12345', 'not-json!!!'); - localStorage.setItem( - '_sp_user_consent_67890', - JSON.stringify({ - gppData: { - gppString: 'DBABLA~BVQqAAAAAgA.QA', - applicableSections: [7], - }, - }) - ); + localStorage.setItem('_sp_user_consent_67890', JSON.stringify(sourcepointPayload())); const result = mirrorSourcepointConsent(); @@ -116,13 +126,7 @@ describe('integrations/sourcepoint', () => { }); it('returns false when gppString is empty', () => { - const payload = { - gppData: { - gppString: '', - applicableSections: [7], - }, - }; - localStorage.setItem('_sp_user_consent_12345', JSON.stringify(payload)); + localStorage.setItem('_sp_user_consent_12345', JSON.stringify(sourcepointPayload('', [7]))); const result = mirrorSourcepointConsent(); @@ -132,14 +136,10 @@ describe('integrations/sourcepoint', () => { it('clears stale __gpp_sid when the payload has no applicable sections', () => { document.cookie = '__gpp_sid=7,8; path=/'; + document.cookie = `${SOURCEPOINT_MARKER_COOKIE}=sp; path=/`; localStorage.setItem( '_sp_user_consent_12345', - JSON.stringify({ - gppData: { - gppString: 'DBABLA~BVQqAAAAAgA.QA', - applicableSections: [], - }, - }) + JSON.stringify(sourcepointPayload('DBABLA~BVQqAAAAAgA.QA', [])) ); const result = mirrorSourcepointConsent(); @@ -147,5 +147,56 @@ describe('integrations/sourcepoint', () => { expect(result).toBe(true); expect(getCookie('__gpp')).toBe('DBABLA~BVQqAAAAAgA.QA'); expect(getCookie('__gpp_sid')).toBeUndefined(); + expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBe('sp'); + }); + + it('refreshes mirrored cookies when the window regains focus', () => { + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('initial-gpp', [7])) + ); + + mirrorSourcepointConsent(); + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('updated-gpp', [8])) + ); + window.dispatchEvent(new Event('focus')); + + expect(getCookie('__gpp')).toBe('updated-gpp'); + expect(getCookie('__gpp_sid')).toBe('8'); + }); + + it('clears Sourcepoint-owned cookies when consent is retracted before focus', () => { + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('initial-gpp', [7])) + ); + + mirrorSourcepointConsent(); + localStorage.removeItem('_sp_user_consent_12345'); + window.dispatchEvent(new Event('focus')); + + expect(getCookie('__gpp')).toBeUndefined(); + expect(getCookie('__gpp_sid')).toBeUndefined(); + expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBeUndefined(); + }); + + it('retries once after module initialization when Sourcepoint data appears shortly after load', async () => { + vi.useFakeTimers(); + vi.resetModules(); + localStorage.clear(); + clearAllCookies(); + + await import('../../../src/integrations/sourcepoint'); + + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('retry-gpp', [7])) + ); + vi.advanceTimersByTime(500); + + expect(getCookie('__gpp')).toBe('retry-gpp'); + expect(getCookie('__gpp_sid')).toBe('7'); }); }); diff --git a/crates/trusted-server-core/src/consent/gpp.rs b/crates/trusted-server-core/src/consent/gpp.rs index bd8c12f24..a6a9766d6 100644 --- a/crates/trusted-server-core/src/consent/gpp.rs +++ b/crates/trusted-server-core/src/consent/gpp.rs @@ -71,10 +71,10 @@ pub fn decode_gpp_string(gpp_string: &str) -> Result bool { } // When a CMP uses TCF in the US (e.g. Didomi), respect the // TCF Purpose 1 decision — this is an explicit opt-in signal. + // The Sourcepoint GPP design documents this precedence decision. if let Some(tcf) = effective_tcf(ctx) { return tcf.has_storage_consent(); } diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index 684910fd4..968696a9f 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -798,7 +798,8 @@ impl IntegrationRegistry { pub fn js_module_ids(&self) -> Vec<&'static str> { // Rust-only integrations with no corresponding JS module const JS_EXCLUDED: &[&str] = &["nextjs", "aps", "adserver_mock"]; - // JS-only modules always included (no Rust-side registration) + // JS-only modules always included (no Rust-side registration). + // Sourcepoint's JS guards cookie clearing with a Sourcepoint-owned marker. const JS_ALWAYS: &[&str] = &["creative", "sourcepoint"]; let mut ids: Vec<&'static str> = JS_ALWAYS.to_vec(); diff --git a/docs/guide/integrations/prebid.md b/docs/guide/integrations/prebid.md index 2be659fdc..9a9c24e70 100644 --- a/docs/guide/integrations/prebid.md +++ b/docs/guide/integrations/prebid.md @@ -225,40 +225,13 @@ Adding a new client-side bidder requires both a config change (`client_side_bidd ## User ID Modules -Prebid's User ID Module resolves cross-publisher identifiers (SharedID, ID5, LiveRamp RampID, UID2, etc.) and exposes them via `pbjs.getUserIdsAsEids()`. The TSJS Prebid integration bundles the core `userId.js` module and a configurable set of ID submodules. When the publisher's origin-side `pbjs.setConfig({ userSync: { userIds: [...] } })` call runs during `processQueue()`, each listed submodule activates and begins resolving its ID asynchronously. After each auction the shim writes the resolved EIDs to a `ts-eids` cookie, which the Rust backend ingests into the Edge Cookie identity graph. +Prebid.js can expose publisher-configured User ID Module output via +`pbjs.getUserIdsAsEids()`. The TSJS Prebid shim reads those current-request +EIDs after auctions and forwards them to Trusted Server when they are available. -### How it works - -1. `userId.js` is statically imported in `index.ts` — always bundled, not operator-configurable. -2. The set of ID submodules is controlled by `TSJS_PREBID_USER_IDS` at build time and emitted into `_user_ids.generated.ts`. -3. Publishers retain full control of which submodules actually run — activation is driven by their own `pbjs.setConfig({ userSync: { userIds: [...] } })` on origin. Bundling a submodule without a matching publisher config entry is inert (but costs bundle size). -4. The `bidsBackHandler` shim calls `pbjs.getUserIdsAsEids()` after each auction and writes the resolved entries to the `ts-eids` cookie (base64-encoded JSON, 3072-byte cap with tail-trim). - -### Build-time submodule selection - -```bash -# Default: the full ship-set of 13 submodules -# (sharedIdSystem, criteoIdSystem, 33acrossIdSystem, pubProvidedIdSystem, -# quantcastIdSystem, id5IdSystem, identityLinkIdSystem, uid2IdSystem, -# euidIdSystem, intentIqIdSystem, lotamePanoramaIdSystem, connectIdSystem, -# merkleIdSystem) - -# Slim build — only SharedID and ID5 -TSJS_PREBID_USER_IDS=sharedIdSystem,id5IdSystem - -# Single submodule -TSJS_PREBID_USER_IDS=sharedIdSystem -``` - -Values are Prebid module filenames without the `.js` extension. The build script (`build-all.mjs`) validates that each exists in `prebid.js/modules/{name}.js` and generates `_user_ids.generated.ts` with the appropriate imports. Unknown names log a warning and are skipped. - -::: warning -`liveIntentIdSystem` is on a build-time denylist — its upstream module uses a dynamic `require()` that esbuild cannot statically resolve, throwing `ReferenceError: require is not defined` at browser runtime. Listing it in `TSJS_PREBID_USER_IDS` logs a warning and skips the module. -::: - -::: tip -Each bundled submodule bloats `tsjs-prebid.js`. If a publisher deployment only needs SharedID and ID5, set `TSJS_PREBID_USER_IDS` accordingly — the other ~100kb of dormant module code won't ship. -::: +Build-time configurable User ID submodule selection is not currently part of the +TSJS build pipeline. Do not rely on a `TSJS_PREBID_USER_IDS` environment +variable or generated `_user_ids.generated.ts` file for slim User ID builds. ## Identity Forwarding diff --git a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md index 2455e12a4..6c23502ea 100644 --- a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md +++ b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md @@ -45,13 +45,18 @@ No Rust-side `IntegrationRegistration` (same pattern as `creative`). 4. Write first-party cookies: - `__gpp=` (path `/`, `SameSite=Lax`) - `__gpp_sid=` (path `/`, `SameSite=Lax`) + - `_ts_gpp_src=sp` marker (path `/`, `SameSite=Lax`) 5. Log what was written for debugging. Cookies are session-scoped (no `max-age` / `expires`) since the source of truth -stays in `localStorage` and we re-mirror on each page load. This design assumes -a single active Sourcepoint property per page; if multiple `_sp_user_consent_*` -entries coexist, the first valid one wins. The integration runs once — no -polling or event listeners. +stays in `localStorage` and we re-mirror on each page load. The marker cookie +tracks Trusted Server's Sourcepoint-owned writes so the integration only clears +`__gpp` / `__gpp_sid` values that it previously mirrored; this avoids clobbering +cookies written by other CMPs. This design assumes a single active Sourcepoint +property per page; if multiple `_sp_user_consent_*` entries coexist, the first +valid one wins. The integration runs immediately, performs bounded first-load +retries, and re-mirrors on page focus/visibility refresh so session cookies do +not remain stale after mid-session consent updates. ### 2. Server-side: GPP US section decoding @@ -104,11 +109,20 @@ if let Some(gpp) = &ctx.gpp { Semantics: -- GPP US `sale_opt_out != OptedOut` → EC allowed -- GPP US `sale_opt_out == OptedOut` → EC blocked -- No GPP US section → falls through to `us_privacy` -- GPC still short-circuits at the top (unchanged) -- TCF still takes priority for CMPs like Didomi (unchanged) +- GPC still short-circuits at the top and blocks EC creation. +- TCF still takes priority for CMPs like Didomi. In US-state jurisdictions, an + effective TCF Purpose 1 signal is treated as the authoritative EC storage + consent decision and is evaluated before GPP US sale opt-out. +- GPP US `sale_opt_out != OptedOut` → EC allowed when no effective TCF signal is + present. +- GPP US `sale_opt_out == OptedOut` → EC blocked when no effective TCF signal is + present. +- No GPP US section → falls through to `us_privacy`. + +The TCF-before-GPP precedence is intentional rather than accidental: it preserves +existing CMP behavior where TCF Purpose 1 is the explicit storage/access signal +for the EC cookie itself. Publishers that need US-section-wins behavior should +raise that as a separate consent-policy configuration change. ### 4. Files touched @@ -120,7 +134,9 @@ Semantics: | `crates/trusted-server-core/src/consent/gpp.rs` | Add US section decoding, extract `sale_opt_out` | | `crates/trusted-server-core/src/consent/mod.rs` | Add GPP US branch in `allows_ec_creation()`, tests | -No config changes, no new crate dependencies, no `IntegrationRegistry` changes. +No config changes and no new crate dependencies. `IntegrationRegistry` includes +`sourcepoint` in the JS-only always-shipped module list; the client-side marker +cookie prevents the always-shipped module from clearing other CMPs' GPP cookies. ### 5. Testing @@ -151,3 +167,4 @@ No config changes, no new crate dependencies, no `IntegrationRegistry` changes. - No richer US GPP field extraction (sharing, targeted advertising opt-outs) - No publisher configuration for Sourcepoint property ID (auto-discovery) - No Sourcepoint CMP API integration (localStorage-only approach) +- No consent-policy knob for making GPP US sale opt-out override TCF Purpose 1 From e7b561a2922c33ae86beec06f8633375b492cf45 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 6 May 2026 15:47:34 -0500 Subject: [PATCH 22/22] Address Sourcepoint review feedback --- .../setup-integration-test-env/action.yml | 2 +- .github/workflows/test.yml | 2 +- .../js/lib/src/integrations/prebid/index.ts | 1 + .../lib/src/integrations/sourcepoint/index.ts | 34 +++++++- .../integrations/sourcepoint/index.test.ts | 85 +++++++++++++++++++ crates/trusted-server-core/src/consent/gpp.rs | 10 +-- ...26-04-15-sourcepoint-gpp-consent-design.md | 5 +- scripts/integration-tests-browser.sh | 2 +- scripts/integration-tests.sh | 2 +- 9 files changed, 128 insertions(+), 15 deletions(-) diff --git a/.github/actions/setup-integration-test-env/action.yml b/.github/actions/setup-integration-test-env/action.yml index a491eb6e1..9c95f8bb5 100644 --- a/.github/actions/setup-integration-test-env/action.yml +++ b/.github/actions/setup-integration-test-env/action.yml @@ -80,7 +80,7 @@ runs: env: TRUSTED_SERVER__PUBLISHER__ORIGIN_URL: http://127.0.0.1:${{ inputs.origin-port }} TRUSTED_SERVER__PUBLISHER__PROXY_SECRET: integration-test-proxy-secret - TRUSTED_SERVER__EC__PASSPHRASE: integration-test-ec-secret + TRUSTED_SERVER__EC__PASSPHRASE: integration-test-ec-secret-padded-32 TRUSTED_SERVER__PROXY__CERTIFICATE_CHECK: "false" run: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1 diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1dd8f0323..2da273aa0 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -55,7 +55,7 @@ jobs: env: TRUSTED_SERVER__PUBLISHER__ORIGIN_URL: http://127.0.0.1:8080 TRUSTED_SERVER__PUBLISHER__PROXY_SECRET: integration-test-proxy-secret - TRUSTED_SERVER__EC__PASSPHRASE: integration-test-ec-secret + TRUSTED_SERVER__EC__PASSPHRASE: integration-test-ec-secret-padded-32 TRUSTED_SERVER__PROXY__CERTIFICATE_CHECK: "false" run: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1 diff --git a/crates/js/lib/src/integrations/prebid/index.ts b/crates/js/lib/src/integrations/prebid/index.ts index b8d6d73b6..fbc798c20 100644 --- a/crates/js/lib/src/integrations/prebid/index.ts +++ b/crates/js/lib/src/integrations/prebid/index.ts @@ -451,6 +451,7 @@ function fitAuctionEidsToCookie(eids: AuctionEid[]): AuctionEid[] | undefined { function syncPrebidEidsCookie(): void { try { if (typeof pbjs.getUserIdsAsEids !== 'function') { + // Without Prebid EIDs to forward, stale auction fallback IDs must not persist. clearPrebidEidsCookie(); return; } diff --git a/crates/js/lib/src/integrations/sourcepoint/index.ts b/crates/js/lib/src/integrations/sourcepoint/index.ts index 0e31b0869..f84131349 100644 --- a/crates/js/lib/src/integrations/sourcepoint/index.ts +++ b/crates/js/lib/src/integrations/sourcepoint/index.ts @@ -17,6 +17,8 @@ interface SourcepointConsentPayload { } let initialized = false; +let initialRetryDone = false; +let retryTimer: ReturnType | undefined; function findSourcepointConsent(): SourcepointConsentPayload | null { // Sourcepoint stores one consent payload per property under `_sp_user_consent_*`. @@ -74,8 +76,27 @@ function mirrorOnVisible(): void { } } +function clearInitialRetryTimer(): void { + if (retryTimer === undefined) { + return; + } + + window.clearTimeout(retryTimer); + retryTimer = undefined; +} + function scheduleInitialRetry(): void { + if (initialRetryDone || retryTimer !== undefined) { + return; + } + const retry = (): void => { + if (initialRetryDone) { + return; + } + + initialRetryDone = true; + clearInitialRetryTimer(); mirrorSourcepointConsent(); }; @@ -83,7 +104,7 @@ function scheduleInitialRetry(): void { document.addEventListener('DOMContentLoaded', retry, { once: true }); } - window.setTimeout(retry, INITIAL_RETRY_DELAY_MS); + retryTimer = window.setTimeout(retry, INITIAL_RETRY_DELAY_MS); } /** @@ -111,15 +132,24 @@ export function mirrorSourcepointConsent(): boolean { return false; } + const existingGppCookie = readCookie(GPP_COOKIE_NAME); + if (existingGppCookie && existingGppCookie !== gppString && !hasSourcepointMarker()) { + log.debug('sourcepoint: preserving existing __gpp cookie from another writer'); + return false; + } + writeCookie(GPP_SOURCE_COOKIE_NAME, GPP_SOURCE_SOURCEPOINT); writeCookie(GPP_COOKIE_NAME, gppString); if (Array.isArray(applicableSections) && applicableSections.length > 0) { writeCookie(GPP_SID_COOKIE_NAME, applicableSections.join(',')); - } else if (hasSourcepointMarker()) { + } else { clearCookie(GPP_SID_COOKIE_NAME); } + initialRetryDone = true; + clearInitialRetryTimer(); + log.info('sourcepoint: mirrored GPP consent to cookies', { gppLength: gppString.length, sections: applicableSections, diff --git a/crates/js/lib/test/integrations/sourcepoint/index.test.ts b/crates/js/lib/test/integrations/sourcepoint/index.test.ts index 3914c2ef9..21dbc1785 100644 --- a/crates/js/lib/test/integrations/sourcepoint/index.test.ts +++ b/crates/js/lib/test/integrations/sourcepoint/index.test.ts @@ -34,6 +34,7 @@ describe('integrations/sourcepoint', () => { afterEach(() => { vi.useRealTimers(); + Object.defineProperty(document, 'readyState', { value: 'complete', configurable: true }); clearAllCookies(); localStorage.clear(); }); @@ -82,6 +83,22 @@ describe('integrations/sourcepoint', () => { expect(getCookie('__gpp_sid')).toBe('7,8'); }); + it('does not overwrite GPP cookies owned by another CMP', () => { + document.cookie = '__gpp=other-cmp-gpp; path=/'; + document.cookie = '__gpp_sid=2; path=/'; + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('sourcepoint-gpp', [7])) + ); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(false); + expect(getCookie('__gpp')).toBe('other-cmp-gpp'); + expect(getCookie('__gpp_sid')).toBe('2'); + expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBeUndefined(); + }); + it('clears stale Sourcepoint-owned mirrored cookies when no valid Sourcepoint payload exists', () => { document.cookie = '__gpp=stale-gpp; path=/'; document.cookie = '__gpp_sid=7,8; path=/'; @@ -150,6 +167,23 @@ describe('integrations/sourcepoint', () => { expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBe('sp'); }); + it('updates GPP cookies when Sourcepoint owns the marker', () => { + document.cookie = '__gpp=stale-sourcepoint-gpp; path=/'; + document.cookie = '__gpp_sid=7; path=/'; + document.cookie = `${SOURCEPOINT_MARKER_COOKIE}=sp; path=/`; + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('updated-sourcepoint-gpp', [8])) + ); + + const result = mirrorSourcepointConsent(); + + expect(result).toBe(true); + expect(getCookie('__gpp')).toBe('updated-sourcepoint-gpp'); + expect(getCookie('__gpp_sid')).toBe('8'); + expect(getCookie(SOURCEPOINT_MARKER_COOKIE)).toBe('sp'); + }); + it('refreshes mirrored cookies when the window regains focus', () => { localStorage.setItem( '_sp_user_consent_12345', @@ -199,4 +233,55 @@ describe('integrations/sourcepoint', () => { expect(getCookie('__gpp')).toBe('retry-gpp'); expect(getCookie('__gpp_sid')).toBe('7'); }); + + it('clears a pending initial retry after a successful manual mirror', async () => { + vi.useFakeTimers(); + vi.resetModules(); + localStorage.clear(); + clearAllCookies(); + Object.defineProperty(document, 'readyState', { value: 'loading', configurable: true }); + + const sourcepoint = await import('../../../src/integrations/sourcepoint'); + + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('manual-gpp', [7])) + ); + expect(sourcepoint.mirrorSourcepointConsent()).toBe(true); + + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('timer-gpp', [8])) + ); + document.dispatchEvent(new Event('DOMContentLoaded')); + vi.advanceTimersByTime(500); + + expect(getCookie('__gpp')).toBe('manual-gpp'); + expect(getCookie('__gpp_sid')).toBe('7'); + }); + + it('does not run both DOMContentLoaded and timer retries', async () => { + vi.useFakeTimers(); + vi.resetModules(); + localStorage.clear(); + clearAllCookies(); + Object.defineProperty(document, 'readyState', { value: 'loading', configurable: true }); + + await import('../../../src/integrations/sourcepoint'); + + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('domcontentloaded-gpp', [7])) + ); + document.dispatchEvent(new Event('DOMContentLoaded')); + + localStorage.setItem( + '_sp_user_consent_12345', + JSON.stringify(sourcepointPayload('timer-gpp', [8])) + ); + vi.advanceTimersByTime(500); + + expect(getCookie('__gpp')).toBe('domcontentloaded-gpp'); + expect(getCookie('__gpp_sid')).toBe('7'); + }); }); diff --git a/crates/trusted-server-core/src/consent/gpp.rs b/crates/trusted-server-core/src/consent/gpp.rs index a6a9766d6..ffb770c2f 100644 --- a/crates/trusted-server-core/src/consent/gpp.rs +++ b/crates/trusted-server-core/src/consent/gpp.rs @@ -123,7 +123,7 @@ const US_SECTION_ID_RANGE: std::ops::RangeInclusive = 7..=23; /// - `None` if no US section is present or no decodable US section yields a /// usable `sale_opt_out` signal fn decode_us_sale_opt_out(parsed: &iab_gpp::v1::GPPString) -> Option { - let mut saw_not_opted_out = false; + let mut result = None; for us_section_id in parsed .section_ids() @@ -132,7 +132,7 @@ fn decode_us_sale_opt_out(parsed: &iab_gpp::v1::GPPString) -> Option { match parsed.decode_section(*us_section_id) { Ok(section) => match us_sale_opt_out_from_section(§ion) { Some(true) => return Some(true), - Some(false) => saw_not_opted_out = true, + Some(false) => result = Some(false), None => {} }, Err(e) => { @@ -141,11 +141,7 @@ fn decode_us_sale_opt_out(parsed: &iab_gpp::v1::GPPString) -> Option { } } - if saw_not_opted_out { - Some(false) - } else { - None - } + result } fn us_sale_opt_out_from_section(section: &iab_gpp::sections::Section) -> Option { diff --git a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md index 6c23502ea..725857db5 100644 --- a/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md +++ b/docs/superpowers/specs/2026-04-15-sourcepoint-gpp-consent-design.md @@ -129,14 +129,15 @@ raise that as a separate consent-policy configuration change. | File | Change | |---|---| | `crates/js/lib/src/integrations/sourcepoint/index.ts` | New — localStorage auto-discovery, cookie mirroring | -| `crates/js/lib/src/integrations/sourcepoint/index.test.ts` | New — Vitest tests | +| `crates/js/lib/test/integrations/sourcepoint/index.test.ts` | New — Vitest tests | | `crates/trusted-server-core/src/consent/types.rs` | Add `us_sale_opt_out: Option` to `GppConsent` | | `crates/trusted-server-core/src/consent/gpp.rs` | Add US section decoding, extract `sale_opt_out` | | `crates/trusted-server-core/src/consent/mod.rs` | Add GPP US branch in `allows_ec_creation()`, tests | No config changes and no new crate dependencies. `IntegrationRegistry` includes `sourcepoint` in the JS-only always-shipped module list; the client-side marker -cookie prevents the always-shipped module from clearing other CMPs' GPP cookies. +cookie prevents the always-shipped module from clearing or overwriting other +CMPs' GPP cookies. ### 5. Testing diff --git a/scripts/integration-tests-browser.sh b/scripts/integration-tests-browser.sh index fb1289d3e..46555fcf9 100755 --- a/scripts/integration-tests-browser.sh +++ b/scripts/integration-tests-browser.sh @@ -32,7 +32,7 @@ echo "==> Validating shared integration-test dependency versions..." echo "==> Building WASM binary (origin=http://127.0.0.1:$ORIGIN_PORT)..." TRUSTED_SERVER__PUBLISHER__ORIGIN_URL="http://127.0.0.1:$ORIGIN_PORT" \ TRUSTED_SERVER__PUBLISHER__PROXY_SECRET="integration-test-proxy-secret" \ -TRUSTED_SERVER__EC__PASSPHRASE="integration-test-ec-secret" \ +TRUSTED_SERVER__EC__PASSPHRASE="integration-test-ec-secret-padded-32" \ TRUSTED_SERVER__PROXY__CERTIFICATE_CHECK=false \ cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1 diff --git a/scripts/integration-tests.sh b/scripts/integration-tests.sh index 318b9323c..6f40f62be 100755 --- a/scripts/integration-tests.sh +++ b/scripts/integration-tests.sh @@ -53,7 +53,7 @@ fi echo "==> Building WASM binary (origin=http://127.0.0.1:$ORIGIN_PORT)..." TRUSTED_SERVER__PUBLISHER__ORIGIN_URL="http://127.0.0.1:$ORIGIN_PORT" \ TRUSTED_SERVER__PUBLISHER__PROXY_SECRET="integration-test-proxy-secret" \ -TRUSTED_SERVER__EC__PASSPHRASE="integration-test-ec-secret" \ +TRUSTED_SERVER__EC__PASSPHRASE="integration-test-ec-secret-padded-32" \ TRUSTED_SERVER__PROXY__CERTIFICATE_CHECK=false \ cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1