From 6bad97838c4c5126679c21853cf675c83c501039 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 09:10:26 +0000 Subject: [PATCH 01/48] docs: design for PXE store selection by identity Replaces wipe-on-mismatch with store selection keyed on (l1ChainId, rollupAddress, schemaVersion) for both PXE backends. --- ...7-09-store-selection-by-identity-design.md | 155 ++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100644 docs/plans/2026-07-09-store-selection-by-identity-design.md diff --git a/docs/plans/2026-07-09-store-selection-by-identity-design.md b/docs/plans/2026-07-09-store-selection-by-identity-design.md new file mode 100644 index 000000000000..da3bcc24dd76 --- /dev/null +++ b/docs/plans/2026-07-09-store-selection-by-identity-design.md @@ -0,0 +1,155 @@ +# PXE store selection by identity + +**Date:** 2026-07-09 +**Branch:** `martin/change-store-version-behavior` +**Status:** Approved design, pending implementation + +## Problem + +The PXE-backing stores are wiped whenever the rollup address or schema version recorded in the +store differs from what the node reports at startup: + +- Browser (sqlite-opfs): `initStoreForRollupAndSchemaVersion` (`yarn-project/kv-store/src/utils.ts`) + unconditionally calls `store.clear()` on any mismatch, then rewrites the `dbVersion` marker. +- Node.js (lmdb-v2): `DatabaseVersionManager` (`yarn-project/stdlib/src/database-version/version_manager.ts`) + supports a `'reset' | 'throw'` policy for schema mismatches, but the rollup-address branch resets + unconditionally regardless of policy. + +The store handle is shared by `KeyStore` (`yarn-project/pxe/src/storage/open_pxe_stores.ts`), so the +wipe destroys the account master secret keys (`ivsk_m/ovsk_m/tsk_m/nhk_m`) along with notes, +contracts, and tagging state. The trigger is unauthenticated node-reported data +(`getL1ContractAddresses()` / `getNodeInfo()`), and no attacker is required: a testnet rollup +redeploy or switching between nodes on different rollups is enough. The wipe is also sticky — +after clearing, the marker is rewritten to the new identity, so reconnecting to the original node +wipes again. + +The embedded wallet already *tries* to partition per rollup by passing +`dataDirectory: wallet_data_${rollupAddress}` / `pxe_data_${rollupAddress}` +(`yarn-project/wallets/src/embedded/entrypoints/browser.ts`), but sqlite-opfs keys stores on `name` +only and ignores `dataDirectory`, so the partitioning is a silent no-op in the browser. + +History: version detection was introduced in PR #20007, which acknowledged wipe-on-mismatch as an +aggressive stopgap. An alternative proposal (add a `'throw'` refusal policy) handles the +round-trip case badly: every switch between two rollups raises an "erase your data?" decision, and +consenting destroys the other rollup's data. + +## Decision + +A store's identity is **`(l1ChainId, rollupAddress, schemaVersion)`**. Opening a store *selects* +the physical store matching that identity: if it exists, reuse it; if not, create it empty. +Nothing is ever cleared. "Mismatch" stops being expressible — a different identity is a different +store. + +Scope decisions: + +- **Both PXE paths**: browser sqlite-opfs and Node.js lmdb-v2 (server PXE, CLI wallets, sandbox). +- **Node infra untouched**: archiver/world-state/p2p keep `DatabaseVersionManager` reset semantics. + Their data is public and resyncable; keeping per-rollup copies of 100GB+ stores is an operator + cost decision out of scope here. +- **KeyStore split deferred**: keys remain inside the per-identity store. They are no longer + destroyed on rollup change, only stranded per store (accounts appear empty on a new rollup until + re-imported). Giving key material its own chain-agnostic store is a follow-up; this design keeps + it possible (a future `keystore_data` store simply omits chain identity from its slug). + +## Design + +### Identity slug helper (new, in `kv-store`) + +`storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion })` → e.g. +`31337-0x-v12`. Full address, no truncation. Shared by both backends and +exported so callers that hand-build store names (e.g. the encrypted embedded-store path, which +takes caller-provided names) can compose correctly. Missing values default as today: +`rollupAddress` → zero address, `l1ChainId` → 0. + +### `DataStoreConfig` + +Gains optional `l1ChainId` (`yarn-project/stdlib/src/kv-store/config.ts`). + +### sqlite-opfs `createStore` + +- Composes the effective DB name as `${name}_${slug}`. +- Stops calling `initStoreForRollupAndSchemaVersion` entirely. +- Still writes the `dbVersion` singleton on first open; a mismatch on reopen **throws** a typed + `StoreIdentityMismatchError` — it can only mean a naming bug — and must never wipe. +- No opt-in flag: the only callers are PXE and the embedded wallet, both of which want the new + behavior. + +### lmdb-v2 `createStore` + +- Gains an opt-in `CreateStoreOptions.partitionByIdentity`. +- When set: data dir becomes `join(dataDirectory, name, slug)` and `DatabaseVersionManager` runs + with `schemaVersionMismatchPolicy: 'throw'` and `versionFileReadFailurePolicy: 'throw'` — the + version file becomes a pure invariant check plus migration metadata. +- Default off: every node-infra caller is untouched. + +### Callers + +- Browser `createPXE` (bundle + lazy): switch `getL1ContractAddresses()` → `getNodeInfo()` to + obtain `l1ChainId`, thread it into the store config. +- Server `createPXE`: already has `l1ChainId`; pass `partitionByIdentity: true`. +- Embedded wallet (browser + node entrypoints): drop the hand-rolled + `wallet_data_${rollupAddress}` / `pxe_data_${rollupAddress}` dataDirectory suffixes; the + partitioning now lives where it is enforced. The node entrypoint's `wallet_data` store passes + `partitionByIdentity: true`; the browser entrypoint gets the behavior from sqlite-opfs + `createStore` directly. + +### Store management utilities (sqlite-opfs) + +`listStores()` and `deleteStore(name)` exposed from the kv-store package. The worker already +implements `deleteDb`; listing comes from the SAH pool's file names. No auto-pruning — retention +policy belongs to the wallet UI, which can now show "you have data for N networks" and offer +cleanup. + +### OPFS pool verification (implementation-time task, with a test) + +Two facts must be established empirically before the sqlite-opfs work can be called done: + +1. How `pxe_data` + `wallet_data` coexist today in the default pool directory + (`.aztec-kv`), given the documented exclusive per-directory lock of the SAH pool. +2. Whether the pool grows past `initialCapacity: 8` — orphaned per-identity stores consume pool + slots, so without growth, accumulation could make *new* store creation fail. + +If capacity does not auto-grow, either handle `addCapacity` on open-failure or move to a +pool-directory-per-store layout. The design adapts here if forced; everything else is unaffected. + +## Error handling + +No path in the new code ever calls `store.clear()` or resets a directory. Failures are typed +throws: identity mismatch (naming bug), version-file unreadable (lmdb-v2 `'throw'` policy), +decryption failure (existing `SqliteEncryptionError`). The stickiness bug is structurally gone — +there is no marker to rewrite on any failure path. + +## Migration & compatibility + +- Existing `pxe_data` / `wallet_data` stores are orphaned in place, not adopted: a one-time + "fresh store" experience identical to a schema bump, with old data left recoverable on disk. +- Changelog entry required (behavior change: stores are no longer wiped on rollup/schema change; + first start after upgrade begins with an empty store). +- Future schema migrations become read-old-store → write-new-store — crash-safe by construction — + but implementing migrations is explicitly out of scope. + +## Security notes + +- A malicious or misconfigured node can no longer destroy local data; at worst it directs the + wallet at an empty namespace, and the real store is recovered by reconnecting to the right node. +- A malicious node can still mint unbounded empty stores (it controls the reported identity). + Mitigated by list/delete utilities and the pool-capacity handling above; full rate-limiting is + out of scope. + +## Testing + +Red/green: + +1. Red: kv-store test reproducing today's wipe — open store, write, reopen with a different + `rollupAddress`, observe data gone. +2. Green after the change: both stores coexist; original data intact under the original identity. + +Plus: + +- Slug composition unit tests (defaults, formatting stability). +- sqlite-opfs isolation-by-identity (browser test suite): write under identity A, open identity B + (empty), reopen A (intact). +- lmdb-v2 flag off = today's reset behavior (regression guard for node infra); flag on = + partitioned directories, no reset. +- `listStores` / `deleteStore` behavior. +- OPFS pool capacity/coexistence findings encoded as tests. From 7c3c29d8615841a2ae5506539b570e6355243680 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 09:23:39 +0000 Subject: [PATCH 02/48] docs: implementation plan for PXE store selection by identity --- ...-07-09-store-selection-by-identity-plan.md | 1033 +++++++++++++++++ 1 file changed, 1033 insertions(+) create mode 100644 docs/plans/2026-07-09-store-selection-by-identity-plan.md diff --git a/docs/plans/2026-07-09-store-selection-by-identity-plan.md b/docs/plans/2026-07-09-store-selection-by-identity-plan.md new file mode 100644 index 000000000000..5997b19e5afa --- /dev/null +++ b/docs/plans/2026-07-09-store-selection-by-identity-plan.md @@ -0,0 +1,1033 @@ +# PXE Store Selection by Identity — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace wipe-on-mismatch with store selection keyed on `(l1ChainId, rollupAddress, schemaVersion)` for both PXE storage backends, per the approved design in `docs/plans/2026-07-09-store-selection-by-identity-design.md`. + +**Architecture:** A new identity-slug helper in `@aztec/kv-store` composes a discriminator string that both backends embed in the physical store location (sqlite-opfs: DB name + per-store OPFS pool directory; lmdb-v2: subdirectory, behind an opt-in flag). Opening a store selects the matching physical store — reuse if it exists, create empty if not. No code path ever clears a populated store; residual version markers become throw-on-mismatch invariant checks. + +**Tech Stack:** TypeScript monorepo (`yarn-project`), vitest (kv-store: node + playwright-browser projects), jest (stdlib), sqlite-wasm OPFS SAH pool, LMDB. + +## Global Constraints + +- Working directory for all commands is `yarn-project` (the Bash tool already runs there — never `cd`). +- Base branch: `merge-train/spartan`; working branch: `martin/change-store-version-behavior` (already checked out). +- Conventional commits; PR squashes on merge; no `Co-Authored-By: Claude` trailers; `git add` must name specific files (never `-u`, `-A`, or `.`). +- Line width 120 chars everywhere including comments. A post-edit hook runs the formatter. +- **Never wipe:** no new or modified code path may call `store.clear()` or delete/reset a directory that could contain user data. First-boot reset of a freshly created empty directory is the only allowed reset. +- New identity slug format (fixed by design, tasks must agree): `--v`, defaults `0`, `EthAddress.ZERO`, `0`. Effective store name: `_`. OPFS pool directory: `.aztec-kv-`. +- Test commands: + - kv-store node project: `yarn workspace @aztec/kv-store test:node ` + - kv-store browser project: `yarn workspace @aztec/kv-store test:browser` (full suite; per-file filtering is not supported by the wrapper script) + - stdlib (jest): `yarn workspace @aztec/stdlib test src/database-version/version_manager.test.ts` +- kv-store vitest browser tests stub `@aztec/foundation/eth-address` and `buffer` (see `kv-store/vitest.config.ts` aliases) — the stubs already support `toString()`, `random()`, `ZERO`, and `schema`, which is all the new code uses. +- Long test output: redirect to a file under `/tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/` and inspect with Read/Grep. + +--- + +### Task 1: Identity slug helper + `DataStoreConfig.l1ChainId` + +**Files:** +- Create: `yarn-project/kv-store/src/store_identity.ts` +- Create: `yarn-project/kv-store/src/store_identity.test.ts` +- Modify: `yarn-project/stdlib/src/kv-store/config.ts` +- Modify: `yarn-project/kv-store/vitest.config.ts` (node-project include list) +- Modify: `yarn-project/kv-store/src/sqlite-opfs/index.ts`, `yarn-project/kv-store/src/lmdb-v2/index.ts` (re-exports) + +**Interfaces:** +- Consumes: `EthAddress` from `@aztec/foundation/eth-address`. +- Produces (used by Tasks 3–7): + - `type StoreIdentity = { l1ChainId?: number; rollupAddress?: EthAddress; schemaVersion?: number }` + - `storeIdentitySlug(identity: StoreIdentity): string` + - `effectiveStoreName(name: string, identity: StoreIdentity): string` + - `class StoreIdentityMismatchError extends Error { storeName: string; expected: string; actual: string }` + - `DataStoreConfig` gains optional `l1ChainId?: number`. + +- [ ] **Step 1: Write the failing test** + +Create `yarn-project/kv-store/src/store_identity.test.ts` (vitest globals are enabled — no test-fn imports): + +```ts +import { EthAddress } from '@aztec/foundation/eth-address'; + +import { effectiveStoreName, storeIdentitySlug } from './store_identity.js'; + +describe('storeIdentitySlug', () => { + it('composes chain id, rollup address and schema version', () => { + const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); + expect(storeIdentitySlug({ l1ChainId: 31337, rollupAddress, schemaVersion: 12 })).toEqual( + '31337-0x1234567890abcdef1234567890abcdef12345678-v12', + ); + }); + + it('defaults missing values to chain 0, zero address, schema 0', () => { + expect(storeIdentitySlug({})).toEqual(`0-${EthAddress.ZERO.toString()}-v0`); + }); + + it('normalizes the rollup address to lowercase hex', () => { + const rollupAddress = EthAddress.fromString('0x1234567890ABCDEF1234567890ABCDEF12345678'); + expect(storeIdentitySlug({ rollupAddress, schemaVersion: 1 })).toEqual( + `0-0x1234567890abcdef1234567890abcdef12345678-v1`, + ); + }); +}); + +describe('effectiveStoreName', () => { + it('joins the logical name and the slug with an underscore', () => { + const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); + expect(effectiveStoreName('pxe_data', { l1ChainId: 1, rollupAddress, schemaVersion: 2 })).toEqual( + 'pxe_data_1-0x1234567890abcdef1234567890abcdef12345678-v2', + ); + }); +}); +``` + +Add the file to the node project's include list in `yarn-project/kv-store/vitest.config.ts`: + +```ts + include: [ + './src/*.test.ts', + './src/lmdb/**/*.test.ts', + './src/lmdb-v2/**/*.test.ts', + './src/stores/**/*.test.ts', + './src/interfaces/**/*.test.ts', + ], +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `yarn workspace @aztec/kv-store test:node src/store_identity.test.ts` +Expected: FAIL — cannot resolve `./store_identity.js`. + +- [ ] **Step 3: Write the implementation** + +Create `yarn-project/kv-store/src/store_identity.ts`: + +```ts +import { EthAddress } from '@aztec/foundation/eth-address'; + +/** The coordinates that determine which physical store a logical store name maps to. */ +export type StoreIdentity = { + /** Chain ID of the L1 the rollup is deployed to. */ + l1ChainId?: number; + /** Address of the rollup contract the store's data pertains to. */ + rollupAddress?: EthAddress; + /** Schema version of the data held in the store. */ + schemaVersion?: number; +}; + +/** + * Composes the store-name discriminator for a store identity. Two identities map to the same physical store iff + * their slugs are equal, so the format must stay stable: `--v`. + */ +export function storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion }: StoreIdentity): string { + return `${l1ChainId ?? 0}-${(rollupAddress ?? EthAddress.ZERO).toString()}-v${schemaVersion ?? 0}`; +} + +/** Composes the physical store name for a logical store name and identity. */ +export function effectiveStoreName(name: string, identity: StoreIdentity): string { + return `${name}_${storeIdentitySlug(identity)}`; +} + +/** + * Thrown when a store's recorded identity does not match the identity it was opened under. Since the identity is + * part of the physical store name, this can only indicate a store-naming bug; the store is left untouched. + */ +export class StoreIdentityMismatchError extends Error { + constructor( + public readonly storeName: string, + public readonly expected: string, + public readonly actual: string, + ) { + super( + `Store '${storeName}' records identity ${actual} but was opened as ${expected}. ` + + `Refusing to open; data was NOT modified.`, + ); + this.name = 'StoreIdentityMismatchError'; + } +} +``` + +In `yarn-project/stdlib/src/kv-store/config.ts`, extend the type and mappings (mirrors `ChainConfig.l1ChainId`, same env var and default, so the duplicate key in `pxeConfigMappings` spreads is harmless): + +```ts +export type DataStoreConfig = { + dataDirectory?: string; + dataStoreMapSizeKb: number; + l1ChainId?: number; +} & Partial>; +``` + +and inside `dataConfigMappings`: + +```ts + l1ChainId: { + env: 'L1_CHAIN_ID', + ...numberConfigHelper(31337), + description: 'The chain ID of the ethereum host.', + }, +``` + +Re-export from both live backends. In `yarn-project/kv-store/src/sqlite-opfs/index.ts` and `yarn-project/kv-store/src/lmdb-v2/index.ts` add: + +```ts +export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; +export type { StoreIdentity } from '../store_identity.js'; +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `yarn workspace @aztec/kv-store test:node src/store_identity.test.ts` +Expected: PASS (4 tests). + +- [ ] **Step 5: Build and commit** + +```bash +yarn build +git add kv-store/src/store_identity.ts kv-store/src/store_identity.test.ts kv-store/vitest.config.ts \ + kv-store/src/sqlite-opfs/index.ts kv-store/src/lmdb-v2/index.ts stdlib/src/kv-store/config.ts +git commit -m "feat(kv-store): store identity slug helper and DataStoreConfig.l1ChainId" +``` + +(Paths in `git add` are relative to the CWD, which is `yarn-project`.) + +--- + +### Task 2: `DatabaseVersionManager` — rollup mismatch honors `'throw'` + +**Files:** +- Modify: `yarn-project/stdlib/src/database-version/version_manager.ts:199-208` +- Test: `yarn-project/stdlib/src/database-version/version_manager.test.ts` + +**Interfaces:** +- Consumes: existing `DatabaseVersionManager`, `SchemaVersionMismatchPolicy`. +- Produces: under `schemaVersionMismatchPolicy: 'throw'`, a rollup-address mismatch against a **successfully parsed** version file throws instead of resetting. First boot (ENOENT) and read-failure paths are unchanged (read failure keeps its own `versionFileReadFailurePolicy`). + +- [ ] **Step 1: Write the failing tests** + +In `yarn-project/stdlib/src/database-version/version_manager.test.ts`, inside `describe('resets the database', ...)`, after the `'when the rollup address changes'` test (line ~142), add (uses the file's existing `fs`, `createManager`, `currentVersion`, `openSpy`, `expectVersionFileWritten` fixtures): + +```ts + it('unless rollup mismatches are configured to throw', async () => { + fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); + versionManager = createManager({ schemaVersionMismatchPolicy: 'throw' }); + expectVersionFileWritten = false; + + await expect(versionManager.open()).rejects.toThrow(/stored rollup address/); + expect(fs.rm).not.toHaveBeenCalled(); + expect(openSpy).not.toHaveBeenCalled(); + }); + + it("still opens fresh on first boot when policy is 'throw' (no version file, different rollup)", async () => { + fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); + versionManager = createManager({ schemaVersionMismatchPolicy: 'throw', onUpgrade: undefined }); + const [_, wasReset] = await versionManager.open(); + expect(wasReset).toEqual(true); + expect(openSpy).toHaveBeenCalled(); + }); +``` + +- [ ] **Step 2: Run tests to verify the first fails** + +Run: `yarn workspace @aztec/stdlib test src/database-version/version_manager.test.ts` +Expected: `unless rollup mismatches are configured to throw` FAILS (open resolves, resets instead of throwing). The first-boot test may already pass — that is fine; it pins behavior we must not break. + +- [ ] **Step 3: Implement** + +In `yarn-project/stdlib/src/database-version/version_manager.ts`, track whether a version file was successfully parsed. In `open()`, next to the existing `shouldLogDataReset` declaration (line ~140), add: + +```ts + // Distinguishes "a version file existed and parsed" from first boot / unreadable file: only a parsed + // version file can prove a genuine rollup mismatch, which is what the 'throw' policy protects against. + let versionFileParsed = false; +``` + +Set it right after the successful parse in the `try` block (line ~144): + +```ts + const versionBuf = await this.fileSystem.readFile(this.versionFile); + storedVersion = DatabaseVersion.fromBuffer(versionBuf); + versionFileParsed = true; +``` + +Replace the rollup-mismatch `else` branch (lines ~199-208) with: + +```ts + } else { + if (versionFileParsed && this.schemaVersionMismatchPolicy === 'throw') { + throw new Error( + `Cannot open database at ${this.dataDirectory}: stored rollup address ` + + `${storedVersion.rollupAddress} does not match expected ${this.currentVersion.rollupAddress}`, + ); + } + if (shouldLogDataReset) { + this.log.warn('Rollup address has changed, resetting data directory', { + versionFile: this.versionFile, + storedVersion, + currentVersion: this.currentVersion, + }); + } + needsReset = true; + } +``` + +Also update the `SchemaVersionMismatchPolicy` JSDoc in the same file (the `@param schemaVersionMismatchPolicy` line in the constructor doc, line ~64) to: + +```ts + * @param schemaVersionMismatchPolicy - Whether schema or rollup-address mismatches against an existing version + * file should reset data or throw +``` + +- [ ] **Step 4: Run tests to verify all pass** + +Run: `yarn workspace @aztec/stdlib test src/database-version/version_manager.test.ts` +Expected: PASS, including all pre-existing tests (the default-'reset' rollup test at line ~137 must still pass). + +- [ ] **Step 5: Commit** + +```bash +git add stdlib/src/database-version/version_manager.ts stdlib/src/database-version/version_manager.test.ts +git commit -m "fix(stdlib): honor 'throw' mismatch policy on rollup address change" +``` + +--- + +### Task 3: sqlite-opfs `createStore` selects by identity (core red/green) + +**Files:** +- Modify: `yarn-project/kv-store/src/sqlite-opfs/index.ts` +- Create: `yarn-project/kv-store/src/sqlite-opfs/manage.ts` (pool-directory naming only in this task; list/delete come in Task 4) +- Test: `yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts` + +**Interfaces:** +- Consumes: `storeIdentitySlug`, `effectiveStoreName`, `StoreIdentityMismatchError` (Task 1); `DatabaseVersion` from `@aztec/stdlib/database-version/version`; `AztecSQLiteOPFSStore.open(log, name?, ephemeral?, poolDirectory?, encryptionKey?)`. +- Produces: `createStore(name, config, schemaVersion?, log?)` with unchanged signature but select-by-identity semantics; `storePoolDirectory(effectiveName: string): string` and `OPFS_POOL_DIR_PREFIX = '.aztec-kv-'` from `manage.ts`. + +**Why per-store pool directories:** sqlite-wasm documents that only one SAH-pool VFS instance may use a directory concurrently, and pool capacity (`initialCapacity: 8`) does not auto-grow. One pool directory per store gives every store its own worker-owned pool (no cross-store lock contention, no slot exhaustion from orphaned stores) and makes delete = remove one OPFS directory. + +- [ ] **Step 1: Write the failing test (data survives a rollup switch)** + +Create `yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts`: + +```ts +import { EthAddress } from '@aztec/foundation/eth-address'; +import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; + +import { mockLogger } from '../interfaces/utils.js'; +import { createStore } from './index.js'; + +const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ + dataDirectory: 'test', + dataStoreMapSizeKb: 1024, + rollupAddress, + l1ChainId, +}); + +describe('sqlite-opfs createStore', () => { + it('keeps data intact when switching rollup addresses back and forth', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + + const storeA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.openSingleton('payload').set('data-for-B'); + await storeB.close(); + + const reopenedA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); + expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); + await reopenedA.close(); + + const reopenedB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); + expect(await reopenedB.openSingleton('payload').getAsync()).toEqual('data-for-B'); + await reopenedB.close(); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-red.log 2>&1`, then Grep the log for `roundtrip_test|create_store`. +Expected: the new test FAILS — today both opens hit the same physical store and `initStoreForRollupAndSchemaVersion` wipes it on the address change, so `reopenedA` reads `undefined`. All pre-existing browser tests must still pass. + +- [ ] **Step 3: Implement** + +Create `yarn-project/kv-store/src/sqlite-opfs/manage.ts`: + +```ts +/** Prefix for the per-store OPFS SAH pool directories owned by this package. */ +export const OPFS_POOL_DIR_PREFIX = '.aztec-kv-'; + +/** + * OPFS directory holding a store's SAH pool. One directory per store: the SAH-pool VFS allows only one + * concurrent instance per directory and its capacity does not grow automatically, so sharing a pool across + * stores would make concurrently opened stores contend for locks and orphaned stores exhaust pool slots. + */ +export function storePoolDirectory(effectiveName: string): string { + return `${OPFS_POOL_DIR_PREFIX}${effectiveName}`; +} +``` + +Rewrite `createStore` in `yarn-project/kv-store/src/sqlite-opfs/index.ts` (replacing the `initStoreForRollupAndSchemaVersion` import and call): + +```ts +import { EthAddress } from '@aztec/foundation/eth-address'; +import { type Logger, createLogger } from '@aztec/foundation/log'; +import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; +import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; + +import { StoreIdentityMismatchError, effectiveStoreName } from '../store_identity.js'; +import { storePoolDirectory } from './manage.js'; +import { AztecSQLiteOPFSStore } from './store.js'; + +export { AztecSQLiteOPFSStore } from './store.js'; +export { SqliteEncryptionError } from './errors.js'; +export type { SqliteEncryptionErrorCode } from './errors.js'; +export { OPFS_POOL_DIR_PREFIX, storePoolDirectory } from './manage.js'; +export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; +export type { StoreIdentity } from '../store_identity.js'; + +/** + * Opens the persistent store selected by `name` and the identity `(config.l1ChainId, config.rollupAddress, + * schemaVersion)`. A store exists per identity: reopening with the same identity returns the same data, a + * different identity selects a different (possibly fresh) store. Nothing is ever cleared. + */ +export async function createStore( + name: string, + config: DataStoreConfig, + schemaVersion: number | undefined = undefined, + log: Logger = createLogger('kv-store'), +) { + const storeName = effectiveStoreName(name, { + l1ChainId: config.l1ChainId, + rollupAddress: config.rollupAddress, + schemaVersion, + }); + log.info(`Creating ${storeName} SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB`); + const store = await AztecSQLiteOPFSStore.open( + createLogger('kv-store:sqlite-opfs'), + storeName, + false, + storePoolDirectory(storeName), + ); + try { + await assertStoreIdentity(store, storeName, schemaVersion, config.rollupAddress); + } catch (err) { + // The store handle owns a worker and OPFS locks; release them before surfacing the refusal. + await store.close().catch(() => {}); + throw err; + } + return store; +} + +/** + * Belt-and-braces invariant check: the identity is part of the physical store name, so the recorded version + * can only disagree if there is a store-naming bug. Refuses to open on mismatch; never clears. + */ +async function assertStoreIdentity( + store: AztecSQLiteOPFSStore, + storeName: string, + schemaVersion: number | undefined, + rollupAddress: EthAddress | undefined, +): Promise { + const expected = new DatabaseVersion(schemaVersion ?? 0, rollupAddress ?? EthAddress.ZERO); + const singleton = store.openSingleton('dbVersion'); + const stored = await singleton.getAsync(); + if (stored === undefined) { + await singleton.set(expected.toBuffer().toString('utf-8')); + return; + } + let storedVersion: DatabaseVersion; + try { + storedVersion = DatabaseVersion.fromBuffer(Buffer.from(stored, 'utf-8')); + } catch { + throw new StoreIdentityMismatchError(storeName, expected.toString(), stored); + } + if (!storedVersion.equals(expected)) { + throw new StoreIdentityMismatchError(storeName, expected.toString(), storedVersion.toString()); + } +} +``` + +Keep `openTmpStore` and `openEncryptedStore` exactly as they are. + +- [ ] **Step 4: Add the isolation, concurrency, and mismatch tests** + +Append to `create_store.test.ts` inside the `describe` block: + +```ts + it('opens two different stores concurrently in the same tab', async () => { + const addr = EthAddress.random(); + const pxeStore = await createStore('pxe_data', configFor(addr), 1, mockLogger); + const walletStore = await createStore('wallet_data', configFor(addr), 1, mockLogger); + + await pxeStore.openSingleton('k').set('pxe'); + await walletStore.openSingleton('k').set('wallet'); + expect(await pxeStore.openSingleton('k').getAsync()).toEqual('pxe'); + expect(await walletStore.openSingleton('k').getAsync()).toEqual('wallet'); + + await pxeStore.close(); + await walletStore.close(); + }); + + it('separates stores by schema version', async () => { + const addr = EthAddress.random(); + const v1 = await createStore('schema_test', configFor(addr), 1, mockLogger); + await v1.openSingleton('k').set('v1-data'); + await v1.close(); + + const v2 = await createStore('schema_test', configFor(addr), 2, mockLogger); + expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); + await v2.close(); + + const v1Again = await createStore('schema_test', configFor(addr), 1, mockLogger); + expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); + await v1Again.close(); + }); + + it('refuses to open on a recorded-identity mismatch and leaves data untouched', async () => { + const addr = EthAddress.random(); + const store = await createStore('mismatch_test', configFor(addr), 1, mockLogger); + await store.openSingleton('payload').set('precious'); + // Simulate a naming bug by corrupting the recorded identity. + await store.openSingleton('dbVersion').set('garbage'); + await store.close(); + + await expect(createStore('mismatch_test', configFor(addr), 1, mockLogger)).rejects.toThrow( + StoreIdentityMismatchError, + ); + + // The refusal must not have modified the store: read it raw, bypassing the identity check. + const storeName = effectiveStoreName('mismatch_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); + const raw = await AztecSQLiteOPFSStore.open(mockLogger, storeName, false, storePoolDirectory(storeName)); + expect(await raw.openSingleton('payload').getAsync()).toEqual('precious'); + await raw.close(); + }); +``` + +and extend the imports at the top of the test file: + +```ts +import { AztecSQLiteOPFSStore, StoreIdentityMismatchError, createStore, effectiveStoreName } from './index.js'; +import { storePoolDirectory } from './manage.js'; +``` + +(drop the now-redundant `import { createStore } from './index.js';` line). + +- [ ] **Step 5: Run the browser suite to verify green** + +Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-green.log 2>&1`, then Grep for failures. +Expected: ALL browser tests pass, including the four new ones and every pre-existing sqlite-opfs test (`encrypted_store.test.ts` in particular — it uses `AztecSQLiteOPFSStore.open` directly and must be unaffected). + +- [ ] **Step 6: Commit** + +```bash +git add kv-store/src/sqlite-opfs/index.ts kv-store/src/sqlite-opfs/manage.ts \ + kv-store/src/sqlite-opfs/create_store.test.ts +git commit -m "feat(kv-store): sqlite-opfs stores selected by (chain, rollup, schema) identity instead of wiped" +``` + +--- + +### Task 4: sqlite-opfs `listStores` / `deleteStore` + +**Files:** +- Modify: `yarn-project/kv-store/src/sqlite-opfs/manage.ts` +- Modify: `yarn-project/kv-store/src/sqlite-opfs/index.ts` (re-export) +- Test: `yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts` (append) + +**Interfaces:** +- Consumes: `OPFS_POOL_DIR_PREFIX`, `storePoolDirectory` (Task 3), OPFS `navigator.storage.getDirectory()`. +- Produces: `listStores(): Promise` (effective store names, slug included) and `deleteStore(effectiveName: string): Promise`. + +- [ ] **Step 1: Write the failing tests** + +Append to `create_store.test.ts`: + +```ts + it('lists created stores and deletes them', async () => { + const addr = EthAddress.random(); + const store = await createStore('managed_test', configFor(addr), 1, mockLogger); + await store.openSingleton('k').set('v'); + await store.close(); + + const storeName = effectiveStoreName('managed_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); + expect(await listStores()).toContain(storeName); + + await deleteStore(storeName); + expect(await listStores()).not.toContain(storeName); + + // Recreating after deletion starts empty. + const fresh = await createStore('managed_test', configFor(addr), 1, mockLogger); + expect(await fresh.openSingleton('k').getAsync()).toBeUndefined(); + await fresh.close(); + }); + + it('refuses to delete a store that is currently open', async () => { + const addr = EthAddress.random(); + const store = await createStore('locked_test', configFor(addr), 1, mockLogger); + const storeName = effectiveStoreName('locked_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); + + await expect(deleteStore(storeName)).rejects.toThrow(); + + await store.close(); + await deleteStore(storeName); + }); +``` + +and add `listStores`, `deleteStore` to the `./manage.js` import in the test file. + +- [ ] **Step 2: Run to verify the new tests fail** + +Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-manage-red.log 2>&1` +Expected: FAIL — `listStores` / `deleteStore` do not exist. + +- [ ] **Step 3: Implement** + +Append to `yarn-project/kv-store/src/sqlite-opfs/manage.ts`: + +```ts +/** + * Lists the effective names (logical name + identity slug) of every persistent sqlite-opfs store in this + * origin, by enumerating the per-store pool directories. Includes stores created under identities other than + * the current one — that is the point: wallets can surface and clean up data for networks no longer in use. + */ +export async function listStores(): Promise { + const root = await navigator.storage.getDirectory(); + const names: string[] = []; + for await (const [entryName, handle] of root.entries()) { + if (handle.kind === 'directory' && entryName.startsWith(OPFS_POOL_DIR_PREFIX)) { + names.push(entryName.slice(OPFS_POOL_DIR_PREFIX.length)); + } + } + return names; +} + +/** + * Permanently deletes a store by effective name (as returned by {@link listStores}). The store must be closed: + * an open store's SAH pool holds locks on the directory and the removal will reject. + */ +export async function deleteStore(effectiveName: string): Promise { + const root = await navigator.storage.getDirectory(); + await root.removeEntry(storePoolDirectory(effectiveName), { recursive: true }); +} +``` + +Re-export from `yarn-project/kv-store/src/sqlite-opfs/index.ts`: + +```ts +export { OPFS_POOL_DIR_PREFIX, deleteStore, listStores, storePoolDirectory } from './manage.js'; +``` + +If `tsc` reports that `FileSystemDirectoryHandle.entries()` does not exist, add `"dom.asynciterable"` to the `lib` array in `yarn-project/tsconfig.json` (currently `["dom", "esnext", "es2017.object"]`) rather than casting. + +- [ ] **Step 4: Run to verify green** + +Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-manage-green.log 2>&1` +Expected: ALL pass. If `refuses to delete a store that is currently open` fails because Chromium allows the removal, delete that test case and the "must be closed" sentence stays in the JSDoc as documentation only — do not weaken `deleteStore` itself. + +- [ ] **Step 5: Commit** + +```bash +git add kv-store/src/sqlite-opfs/manage.ts kv-store/src/sqlite-opfs/index.ts \ + kv-store/src/sqlite-opfs/create_store.test.ts +git commit -m "feat(kv-store): list and delete sqlite-opfs stores" +``` + +--- + +### Task 5: lmdb-v2 `partitionByIdentity` + +**Files:** +- Modify: `yarn-project/kv-store/src/lmdb-v2/factory.ts` +- Test: `yarn-project/kv-store/src/lmdb-v2/factory.test.ts` (new) + +**Interfaces:** +- Consumes: `storeIdentitySlug` (Task 1), `DatabaseVersionManager` with the Task 2 fix. +- Produces: `CreateStoreOptions.partitionByIdentity?: boolean`. When true, the store directory is `join(dataDirectory, name, storeIdentitySlug(identity))` and both version-manager policies are forced to `'throw'`. When absent/false, behavior is byte-for-byte today's. + +- [ ] **Step 1: Write the failing tests** + +Create `yarn-project/kv-store/src/lmdb-v2/factory.test.ts` (node project — already covered by the `./src/lmdb-v2/**/*.test.ts` include): + +```ts +import { EthAddress } from '@aztec/foundation/eth-address'; +import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; + +import { mkdtemp, rm } from 'fs/promises'; +import { tmpdir } from 'os'; +import { join } from 'path'; + +import { createStore } from './factory.js'; + +describe('lmdb-v2 createStore', () => { + let dataDirectory: string; + + beforeEach(async () => { + dataDirectory = await mkdtemp(join(tmpdir(), 'factory-test-')); + }); + + afterEach(async () => { + await rm(dataDirectory, { recursive: true, force: true }); + }); + + const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ + dataDirectory, + dataStoreMapSizeKb: 10 * 1024, + rollupAddress, + l1ChainId, + }); + + it('with partitionByIdentity, keeps data intact when switching rollup addresses back and forth', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + const options = { partitionByIdentity: true }; + + const storeA = await createStore('test_store', 1, configFor(addrA), undefined, options); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await createStore('test_store', 1, configFor(addrB), undefined, options); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.close(); + + const reopenedA = await createStore('test_store', 1, configFor(addrA), undefined, options); + expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); + await reopenedA.close(); + }); + + it('with partitionByIdentity, separates stores by schema version', async () => { + const addr = EthAddress.random(); + const options = { partitionByIdentity: true }; + + const v1 = await createStore('test_store', 1, configFor(addr), undefined, options); + await v1.openSingleton('k').set('v1-data'); + await v1.close(); + + const v2 = await createStore('test_store', 2, configFor(addr), undefined, options); + expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); + await v2.close(); + + const v1Again = await createStore('test_store', 1, configFor(addr), undefined, options); + expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); + await v1Again.close(); + }); + + it('without the flag, keeps the historical reset-on-rollup-change behavior', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + + const storeA = await createStore('test_store', 1, configFor(addrA)); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await createStore('test_store', 1, configFor(addrB)); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.close(); + + // Historical behavior is destructive: the data does not come back. + const reopenedA = await createStore('test_store', 1, configFor(addrA)); + expect(await reopenedA.openSingleton('payload').getAsync()).toBeUndefined(); + await reopenedA.close(); + }); +}); +``` + +- [ ] **Step 2: Run to verify the identity tests fail** + +Run: `yarn workspace @aztec/kv-store test:node src/lmdb-v2/factory.test.ts` +Expected: the two `partitionByIdentity` tests FAIL (the option is unknown/ignored, so the reset wipes `data-for-A`); the historical-behavior test PASSES. + +- [ ] **Step 3: Implement** + +In `yarn-project/kv-store/src/lmdb-v2/factory.ts`, extend the options type: + +```ts +/** Optional versioning hooks for persistent LMDB stores. */ +export type CreateStoreOptions = { + onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; + schemaVersionMismatchPolicy?: SchemaVersionMismatchPolicy; + versionFileReadFailurePolicy?: VersionFileReadFailurePolicy; + /** + * When true, the store directory is keyed by (l1ChainId, rollupAddress, schemaVersion): a different identity + * selects a different directory instead of resetting this one, so no identity change is ever destructive. + * The version file becomes a pure invariant check (any mismatch throws). + */ + partitionByIdentity?: boolean; +}; +``` + +and change the persistent branch of `createStore` (currently `const subDir = join(dataDirectory, name);` and the `DatabaseVersionManager` construction): + +```ts + const rollupAddress = rollupFromConfig ?? EthAddress.ZERO; + const subDir = options.partitionByIdentity + ? join(dataDirectory, name, storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress, schemaVersion })) + : join(dataDirectory, name); + await mkdir(subDir, { recursive: true }); + + const versionManager = new DatabaseVersionManager({ + schemaVersion, + rollupAddress, + dataDirectory: subDir, + onOpen: dbDirectory => + AztecLMDBStoreV2.new(dbDirectory, config.dataStoreMapSizeKb, MAX_READERS, () => Promise.resolve(), bindings), + onUpgrade: options.onUpgrade, + schemaVersionMismatchPolicy: options.partitionByIdentity ? 'throw' : options.schemaVersionMismatchPolicy, + versionFileReadFailurePolicy: options.partitionByIdentity ? 'throw' : options.versionFileReadFailurePolicy, + }); +``` + +with the import added at the top: + +```ts +import { storeIdentitySlug } from '../store_identity.js'; +``` + +(Note the original code computed `rollupAddress` after `mkdir`; it moves above `subDir` because the slug needs it.) + +- [ ] **Step 4: Run to verify green** + +Run: `yarn workspace @aztec/kv-store test:node src/lmdb-v2/factory.test.ts` +Expected: all 3 PASS. Then run the whole node project to guard against regressions: `yarn workspace @aztec/kv-store test:node`. + +- [ ] **Step 5: Commit** + +```bash +git add kv-store/src/lmdb-v2/factory.ts kv-store/src/lmdb-v2/factory.test.ts +git commit -m "feat(kv-store): opt-in identity-partitioned lmdb-v2 stores" +``` + +--- + +### Task 6: Wire up the Node.js PXE and node embedded wallet + +**Files:** +- Modify: `yarn-project/pxe/src/entrypoints/server/utils.ts:49-54` +- Modify: `yarn-project/wallets/src/embedded/entrypoints/node.ts:31,40,71-87` + +**Interfaces:** +- Consumes: `createStore(name, schemaVersion, config, bindings, { partitionByIdentity: true })` from `@aztec/kv-store/lmdb-v2` (Task 5). +- Produces: no new interfaces; behavior change only. + +- [ ] **Step 1: Server PXE passes the flag** + +In `yarn-project/pxe/src/entrypoints/server/utils.ts`, the store creation becomes: + +```ts + options.store = await createStore( + 'pxe_data', + PXE_DATA_SCHEMA_VERSION, + configWithContracts, + storeLogger.getBindings(), + { partitionByIdentity: true }, + ); +``` + +(`configWithContracts` already carries `l1ChainId` and `rollupAddress` from `getNodeInfo()`.) + +- [ ] **Step 2: Node embedded wallet drops hand-rolled suffixes** + +In `yarn-project/wallets/src/embedded/entrypoints/node.ts`, keep the existing `aztecNode` variable and replace the +`getL1ContractAddresses` call (line ~31) so the two lines read: + +```ts + const aztecNode = typeof nodeOrUrl === 'string' ? createAztecNodeClient(nodeOrUrl) : nodeOrUrl; + const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); +``` + +Change the PXE config default directory (line ~40) from `` dataDirectory: `pxe_data_${l1Contracts.rollupAddress}` `` to: + +```ts + dataDirectory: 'aztec-wallet-data', +``` + +Change the walletDB store creation (lines ~68-87) to: + +```ts + const walletDBStore = + options.walletDb?.store ?? + (options.ephemeral + ? await openTmpStore( + 'wallet_data', + true, + undefined, + undefined, + rootLogger.createChild('wallet:data').getBindings(), + ) + : await createStore( + 'wallet_data', + 1, + { + dataDirectory: pxeConfig.dataDirectory ?? 'aztec-wallet-data', + dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, + rollupAddress: l1ContractAddresses.rollupAddress, + l1ChainId, + }, + rootLogger.createChild('wallet:data').getBindings(), + { partitionByIdentity: true }, + )); +``` + +There are no other `l1Contracts` references in the file after these edits (verify with Grep; the pre-change references are lines 31, 40, 72, 82, 84). + +- [ ] **Step 3: Build and run affected tests** + +```bash +yarn build +yarn workspace @aztec/pxe test +``` + +Expected: build clean; pxe unit tests pass (they mock `getNodeInfo` already). If any wallets tests exist that construct `NodeEmbeddedWallet`, run `yarn workspace @aztec/wallets test` as well and fix fallout — the likely failure mode is a missing `getNodeInfo` mock where only `getL1ContractAddresses` was mocked; extend the mock with `getNodeInfo` returning `{ l1ChainId: 31337, l1ContractAddresses: { rollupAddress }, rollupVersion: 1 }` shaped like `stdlib`'s `NodeInfo`. + +- [ ] **Step 4: Commit** + +```bash +git add pxe/src/entrypoints/server/utils.ts wallets/src/embedded/entrypoints/node.ts +git commit -m "feat(pxe): node PXE and embedded wallet stores partitioned by identity" +``` + +--- + +### Task 7: Wire up the browser PXE and browser embedded wallet + +**Files:** +- Modify: `yarn-project/pxe/src/entrypoints/client/bundle/utils.ts:34-43` +- Modify: `yarn-project/pxe/src/entrypoints/client/lazy/utils.ts` (same edit; the files differ only in import specifiers) +- Modify: `yarn-project/wallets/src/embedded/entrypoints/browser.ts:30,39,71-80` + +**Interfaces:** +- Consumes: sqlite-opfs `createStore` (Task 3) — no signature change, it now reads `config.l1ChainId`. +- Produces: no new interfaces; behavior change only. + +- [ ] **Step 1: Browser createPXE fetches chain identity from the node** + +In both `bundle/utils.ts` and `lazy/utils.ts`, replace: + +```ts + const l1ContractAddresses = await aztecNode.getL1ContractAddresses(); + const configWithContracts = { + ...config, + ...l1ContractAddresses, + } as PXEConfig; +``` + +with: + +```ts + const { l1ChainId, l1ContractAddresses, rollupVersion } = await aztecNode.getNodeInfo(); + const configWithContracts = { + ...config, + ...l1ContractAddresses, + l1ChainId, + rollupVersion, + } as PXEConfig; +``` + +- [ ] **Step 2: Browser embedded wallet drops hand-rolled suffixes** + +In `yarn-project/wallets/src/embedded/entrypoints/browser.ts`: + +Replace line ~30: + +```ts + const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); +``` + +Replace the PXE config default directory (line ~39) with a plain marker (sqlite-opfs uses `dataDirectory` only to word its log line): + +```ts + dataDirectory: 'pxe_data', +``` + +Replace the walletDB store creation (lines ~67-80): + +```ts + const walletDBStore = + options.walletDb?.store ?? + (options.ephemeral + ? await openTmpStore(true) + : await createStore( + 'wallet_data', + { + dataDirectory: 'wallet_data', + dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, + rollupAddress: l1ContractAddresses.rollupAddress, + l1ChainId, + }, + 1, + rootLogger.createChild('wallet:data'), + )); +``` + +Update any remaining `l1Contracts` references in the file (pre-change: lines 30, 39, 74, 76) to `l1ContractAddresses`. + +- [ ] **Step 3: Build, lint, and sweep for stragglers** + +```bash +yarn build +``` + +Then Grep the repo for `wallet_data_${` and `pxe_data_${` under `yarn-project` (excluding `dest/` and `node_modules/`) — expected: zero hits. Also Grep for remaining production imports of `initStoreForRollupAndSchemaVersion`: expected hits only in `kv-store/src/utils.ts` itself, `kv-store/src/lmdb/index.ts`, and `kv-store/src/deprecated/indexeddb/` (test-only backends, deliberately untouched). + +- [ ] **Step 4: Commit** + +```bash +git add pxe/src/entrypoints/client/bundle/utils.ts pxe/src/entrypoints/client/lazy/utils.ts \ + wallets/src/embedded/entrypoints/browser.ts +git commit -m "feat(pxe): browser PXE and embedded wallet stores partitioned by identity" +``` + +--- + +### Task 8: Full verification, changelog, docs + +**Files:** +- Modify: changelog files as directed by the `updating-changelog` skill +- No other code changes expected + +- [ ] **Step 1: Full build + format + lint** + +```bash +yarn build +yarn format +yarn lint kv-store stdlib pxe wallets +``` + +Expected: all clean. Commit any formatter-only diffs with `chore: format`. + +- [ ] **Step 2: Full test pass for touched packages** + +```bash +yarn workspace @aztec/kv-store test:node +yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-final.log 2>&1 +yarn workspace @aztec/stdlib test src/database-version +yarn workspace @aztec/pxe test +yarn workspace @aztec/wallets test +``` + +Expected: all pass. Check `yarn-project/pxe/src/storage/backwards_compatibility_tests/` specifically — those tests open stores directly via tmp-store helpers and should be unaffected; if one constructs a store through `createStore`, it now gets an identity-slugged location, which is fine as long as the test is self-contained. + +- [ ] **Step 3: Changelog** + +Invoke the `updating-changelog` skill to document the behavior change: +- PXE/wallet stores are no longer wiped when the rollup address or schema version changes; a store now exists per `(l1ChainId, rollupAddress, schemaVersion)` and switching selects the matching store. +- First start after upgrading begins with a fresh (empty) store; previous data remains on disk under the old name and is not deleted. +- New `listStores()` / `deleteStore()` utilities in `@aztec/kv-store/sqlite-opfs` for wallet-driven cleanup. + +- [ ] **Step 4: Final review and commit** + +```bash +git log --oneline origin/merge-train/spartan..HEAD +git diff origin/merge-train/spartan...HEAD --stat +``` + +Confirm every commit is conventional and scoped. Commit the changelog updates: + +```bash +git add +git commit -m "docs: changelog for identity-partitioned PXE stores" +``` From 071b0af196991cd044a4599ff311d7428393c72d Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 09:32:52 +0000 Subject: [PATCH 03/48] feat(kv-store): store identity slug helper and DataStoreConfig.l1ChainId --- yarn-project/kv-store/src/lmdb-v2/index.ts | 2 + .../kv-store/src/sqlite-opfs/index.ts | 2 + .../kv-store/src/store_identity.test.ts | 32 ++++++++++++++ yarn-project/kv-store/src/store_identity.ts | 42 +++++++++++++++++++ yarn-project/kv-store/vitest.config.ts | 1 + .../stdlib/src/ha-signing/local_config.ts | 1 + yarn-project/stdlib/src/kv-store/config.ts | 6 +++ yarn-project/validator-client/src/config.ts | 6 --- 8 files changed, 86 insertions(+), 6 deletions(-) create mode 100644 yarn-project/kv-store/src/store_identity.test.ts create mode 100644 yarn-project/kv-store/src/store_identity.ts diff --git a/yarn-project/kv-store/src/lmdb-v2/index.ts b/yarn-project/kv-store/src/lmdb-v2/index.ts index af723281fd47..50713eff1d81 100644 --- a/yarn-project/kv-store/src/lmdb-v2/index.ts +++ b/yarn-project/kv-store/src/lmdb-v2/index.ts @@ -1,2 +1,4 @@ export * from './store.js'; export * from './factory.js'; +export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; +export type { StoreIdentity } from '../store_identity.js'; diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index 9599a1532144..845e2c4509f6 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -7,6 +7,8 @@ import { AztecSQLiteOPFSStore } from './store.js'; export { AztecSQLiteOPFSStore } from './store.js'; export { SqliteEncryptionError } from './errors.js'; export type { SqliteEncryptionErrorCode } from './errors.js'; +export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; +export type { StoreIdentity } from '../store_identity.js'; export async function createStore( name: string, diff --git a/yarn-project/kv-store/src/store_identity.test.ts b/yarn-project/kv-store/src/store_identity.test.ts new file mode 100644 index 000000000000..738879e747fd --- /dev/null +++ b/yarn-project/kv-store/src/store_identity.test.ts @@ -0,0 +1,32 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; + +import { effectiveStoreName, storeIdentitySlug } from './store_identity.js'; + +describe('storeIdentitySlug', () => { + it('composes chain id, rollup address and schema version', () => { + const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); + expect(storeIdentitySlug({ l1ChainId: 31337, rollupAddress, schemaVersion: 12 })).toEqual( + '31337-0x1234567890abcdef1234567890abcdef12345678-v12', + ); + }); + + it('defaults missing values to chain 0, zero address, schema 0', () => { + expect(storeIdentitySlug({})).toEqual(`0-${EthAddress.ZERO.toString()}-v0`); + }); + + it('normalizes the rollup address to lowercase hex', () => { + const rollupAddress = EthAddress.fromString('0x1234567890ABCDEF1234567890ABCDEF12345678'); + expect(storeIdentitySlug({ rollupAddress, schemaVersion: 1 })).toEqual( + `0-0x1234567890abcdef1234567890abcdef12345678-v1`, + ); + }); +}); + +describe('effectiveStoreName', () => { + it('joins the logical name and the slug with an underscore', () => { + const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); + expect(effectiveStoreName('pxe_data', { l1ChainId: 1, rollupAddress, schemaVersion: 2 })).toEqual( + 'pxe_data_1-0x1234567890abcdef1234567890abcdef12345678-v2', + ); + }); +}); diff --git a/yarn-project/kv-store/src/store_identity.ts b/yarn-project/kv-store/src/store_identity.ts new file mode 100644 index 000000000000..bb62312d9e4a --- /dev/null +++ b/yarn-project/kv-store/src/store_identity.ts @@ -0,0 +1,42 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; + +/** The coordinates that determine which physical store a logical store name maps to. */ +export type StoreIdentity = { + /** Chain ID of the L1 the rollup is deployed to. */ + l1ChainId?: number; + /** Address of the rollup contract the store's data pertains to. */ + rollupAddress?: EthAddress; + /** Schema version of the data held in the store. */ + schemaVersion?: number; +}; + +/** + * Composes the store-name discriminator for a store identity. Two identities map to the same physical store iff + * their slugs are equal, so the format must stay stable: `--v`. + */ +export function storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion }: StoreIdentity): string { + return `${l1ChainId ?? 0}-${(rollupAddress ?? EthAddress.ZERO).toString()}-v${schemaVersion ?? 0}`; +} + +/** Composes the physical store name for a logical store name and identity. */ +export function effectiveStoreName(name: string, identity: StoreIdentity): string { + return `${name}_${storeIdentitySlug(identity)}`; +} + +/** + * Thrown when a store's recorded identity does not match the identity it was opened under. Since the identity is + * part of the physical store name, this can only indicate a store-naming bug; the store is left untouched. + */ +export class StoreIdentityMismatchError extends Error { + constructor( + public readonly storeName: string, + public readonly expected: string, + public readonly actual: string, + ) { + super( + `Store '${storeName}' records identity ${actual} but was opened as ${expected}. ` + + `Refusing to open; data was NOT modified.`, + ); + this.name = 'StoreIdentityMismatchError'; + } +} diff --git a/yarn-project/kv-store/vitest.config.ts b/yarn-project/kv-store/vitest.config.ts index 7fd105321a53..3e51261da9d5 100644 --- a/yarn-project/kv-store/vitest.config.ts +++ b/yarn-project/kv-store/vitest.config.ts @@ -53,6 +53,7 @@ export default defineConfig({ name: 'node', environment: 'node', include: [ + './src/*.test.ts', './src/lmdb/**/*.test.ts', './src/lmdb-v2/**/*.test.ts', './src/stores/**/*.test.ts', diff --git a/yarn-project/stdlib/src/ha-signing/local_config.ts b/yarn-project/stdlib/src/ha-signing/local_config.ts index c16f3e0c4b99..896c1a3c9791 100644 --- a/yarn-project/stdlib/src/ha-signing/local_config.ts +++ b/yarn-project/stdlib/src/ha-signing/local_config.ts @@ -52,6 +52,7 @@ export const LocalSignerConfigSchema = zodFor()( BaseSignerConfigSchema.extend({ dataDirectory: z.string().optional(), dataStoreMapSizeKb: z.number(), + l1ChainId: z.number().optional(), signingProtectionMapSizeKb: z.number().optional(), allowEphemeralSigningProtection: z.boolean().optional(), }), diff --git a/yarn-project/stdlib/src/kv-store/config.ts b/yarn-project/stdlib/src/kv-store/config.ts index c0892f96020e..e29a9779b14d 100644 --- a/yarn-project/stdlib/src/kv-store/config.ts +++ b/yarn-project/stdlib/src/kv-store/config.ts @@ -4,6 +4,7 @@ import { type ConfigMappingsType, getConfigFromMappings, numberConfigHelper } fr export type DataStoreConfig = { dataDirectory?: string; dataStoreMapSizeKb: number; + l1ChainId?: number; } & Partial>; export const dataConfigMappings: ConfigMappingsType = { @@ -16,6 +17,11 @@ export const dataConfigMappings: ConfigMappingsType = { description: 'The maximum possible size of a data store DB in KB. Can be overridden by component-specific options.', ...numberConfigHelper(128 * 1_024 * 1_024), // Defaulted to 128 GB }, + l1ChainId: { + env: 'L1_CHAIN_ID', + ...numberConfigHelper(31337), + description: 'The chain ID of the ethereum host.', + }, ...pickL1ContractAddressMappings('rollupAddress'), }; diff --git a/yarn-project/validator-client/src/config.ts b/yarn-project/validator-client/src/config.ts index 38d914a6d6af..36d433fbb5f5 100644 --- a/yarn-project/validator-client/src/config.ts +++ b/yarn-project/validator-client/src/config.ts @@ -42,12 +42,6 @@ export const validatorClientConfigMappings: ConfigMappingsType< .map(address => EthAddress.fromString(address.trim())), defaultValue: [], }, - l1ChainId: { - env: 'L1_CHAIN_ID', - description: 'The chain ID of the ethereum host.', - parseEnv: (val: string) => +val, - defaultValue: 31337, - }, disableValidator: { env: 'VALIDATOR_DISABLED', description: 'Do not run the validator', From f8076f4c86c1bd6cc740f32527b6cc7f5e5b1b10 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 09:40:18 +0000 Subject: [PATCH 04/48] fix(stdlib): honor 'throw' mismatch policy on rollup address change --- .../database-version/version_manager.test.ts | 18 ++++++++++++++++++ .../src/database-version/version_manager.ts | 13 ++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/yarn-project/stdlib/src/database-version/version_manager.test.ts b/yarn-project/stdlib/src/database-version/version_manager.test.ts index 94e561afb238..676dfa3d1e9c 100644 --- a/yarn-project/stdlib/src/database-version/version_manager.test.ts +++ b/yarn-project/stdlib/src/database-version/version_manager.test.ts @@ -140,6 +140,24 @@ describe('VersionManager', () => { expect(wasReset).toEqual(true); expect(upgradeSpy).not.toHaveBeenCalled(); }); + + it('unless rollup mismatches are configured to throw', async () => { + fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); + versionManager = createManager({ schemaVersionMismatchPolicy: 'throw' }); + expectVersionFileWritten = false; + + await expect(versionManager.open()).rejects.toThrow(/stored rollup address/); + expect(fs.rm).not.toHaveBeenCalled(); + expect(openSpy).not.toHaveBeenCalled(); + }); + + it("still opens fresh on first boot when policy is 'throw' (no version file, different rollup)", async () => { + fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); + versionManager = createManager({ schemaVersionMismatchPolicy: 'throw', onUpgrade: undefined }); + const [_, wasReset] = await versionManager.open(); + expect(wasReset).toEqual(true); + expect(openSpy).toHaveBeenCalled(); + }); }); describe('version file read-failure policy', () => { diff --git a/yarn-project/stdlib/src/database-version/version_manager.ts b/yarn-project/stdlib/src/database-version/version_manager.ts index 1a2d0c519018..bc9f7de2dff6 100644 --- a/yarn-project/stdlib/src/database-version/version_manager.ts +++ b/yarn-project/stdlib/src/database-version/version_manager.ts @@ -61,7 +61,8 @@ export class DatabaseVersionManager { * @param onUpgrade - An optional callback to upgrade the database before opening. If not provided it will reset the * database. Must be idempotent: since the version marker is written only after a successful open, a crash after * onUpgrade but before the marker is written re-runs onUpgrade on the next start. - * @param schemaVersionMismatchPolicy - Whether schema mismatches should reset data or throw + * @param schemaVersionMismatchPolicy - Whether schema or rollup-address mismatches against an existing version + * file should reset data or throw * @param versionFileReadFailurePolicy - Whether an unreadable (non-missing) version file should reset data or throw * @param fileSystem - An interface to access the filesystem * @param log - Optional custom logger @@ -138,10 +139,14 @@ export class DatabaseVersionManager { let storedVersion: DatabaseVersion; // a flag to suppress logs about 'resetting the data dir' when starting from an empty state let shouldLogDataReset = true; + // Distinguishes "a version file existed and parsed" from first boot / unreadable file: only a parsed + // version file can prove a genuine rollup mismatch, which is what the 'throw' policy protects against. + let versionFileParsed = false; try { const versionBuf = await this.fileSystem.readFile(this.versionFile); storedVersion = DatabaseVersion.fromBuffer(versionBuf); + versionFileParsed = true; } catch (err) { if (err && (err as Error & { code: string }).code === 'ENOENT') { storedVersion = DatabaseVersion.empty(); @@ -197,6 +202,12 @@ export class DatabaseVersionManager { needsReset = true; } } else { + if (versionFileParsed && this.schemaVersionMismatchPolicy === 'throw') { + throw new Error( + `Cannot open database at ${this.dataDirectory}: stored rollup address ` + + `${storedVersion.rollupAddress} does not match expected ${this.currentVersion.rollupAddress}`, + ); + } if (shouldLogDataReset) { this.log.warn('Rollup address has changed, resetting data directory', { versionFile: this.versionFile, From 46349617083313fbf37f6ea16c6bf51bc6b2284e Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 09:50:05 +0000 Subject: [PATCH 05/48] feat(kv-store): sqlite-opfs stores selected by (chain, rollup, schema) identity instead of wiped --- .../src/sqlite-opfs/create_store.test.ts | 85 +++++++++++++++++++ .../kv-store/src/sqlite-opfs/index.ts | 65 ++++++++++++-- .../kv-store/src/sqlite-opfs/manage.ts | 11 +++ 3 files changed, 153 insertions(+), 8 deletions(-) create mode 100644 yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts create mode 100644 yarn-project/kv-store/src/sqlite-opfs/manage.ts diff --git a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts new file mode 100644 index 000000000000..81f9aa77ff5d --- /dev/null +++ b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts @@ -0,0 +1,85 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; +import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; + +import { mockLogger } from '../interfaces/utils.js'; +import { AztecSQLiteOPFSStore, StoreIdentityMismatchError, createStore, effectiveStoreName } from './index.js'; +import { storePoolDirectory } from './manage.js'; + +const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ + dataDirectory: 'test', + dataStoreMapSizeKb: 1024, + rollupAddress, + l1ChainId, +}); + +describe('sqlite-opfs createStore', () => { + it('keeps data intact when switching rollup addresses back and forth', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + + const storeA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.openSingleton('payload').set('data-for-B'); + await storeB.close(); + + const reopenedA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); + expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); + await reopenedA.close(); + + const reopenedB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); + expect(await reopenedB.openSingleton('payload').getAsync()).toEqual('data-for-B'); + await reopenedB.close(); + }); + + it('opens two different stores concurrently in the same tab', async () => { + const addr = EthAddress.random(); + const pxeStore = await createStore('pxe_data', configFor(addr), 1, mockLogger); + const walletStore = await createStore('wallet_data', configFor(addr), 1, mockLogger); + + await pxeStore.openSingleton('k').set('pxe'); + await walletStore.openSingleton('k').set('wallet'); + expect(await pxeStore.openSingleton('k').getAsync()).toEqual('pxe'); + expect(await walletStore.openSingleton('k').getAsync()).toEqual('wallet'); + + await pxeStore.close(); + await walletStore.close(); + }); + + it('separates stores by schema version', async () => { + const addr = EthAddress.random(); + const v1 = await createStore('schema_test', configFor(addr), 1, mockLogger); + await v1.openSingleton('k').set('v1-data'); + await v1.close(); + + const v2 = await createStore('schema_test', configFor(addr), 2, mockLogger); + expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); + await v2.close(); + + const v1Again = await createStore('schema_test', configFor(addr), 1, mockLogger); + expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); + await v1Again.close(); + }); + + it('refuses to open on a recorded-identity mismatch and leaves data untouched', async () => { + const addr = EthAddress.random(); + const store = await createStore('mismatch_test', configFor(addr), 1, mockLogger); + await store.openSingleton('payload').set('precious'); + // Simulate a naming bug by corrupting the recorded identity. + await store.openSingleton('dbVersion').set('garbage'); + await store.close(); + + await expect(createStore('mismatch_test', configFor(addr), 1, mockLogger)).rejects.toThrow( + StoreIdentityMismatchError, + ); + + // The refusal must not have modified the store: read it raw, bypassing the identity check. + const storeName = effectiveStoreName('mismatch_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); + const raw = await AztecSQLiteOPFSStore.open(mockLogger, storeName, false, storePoolDirectory(storeName)); + expect(await raw.openSingleton('payload').getAsync()).toEqual('precious'); + await raw.close(); + }); +}); diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index 845e2c4509f6..1de37cfd384f 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -1,29 +1,78 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; import { type Logger, createLogger } from '@aztec/foundation/log'; +import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; -import { initStoreForRollupAndSchemaVersion } from '../utils.js'; +import { StoreIdentityMismatchError, effectiveStoreName } from '../store_identity.js'; +import { storePoolDirectory } from './manage.js'; import { AztecSQLiteOPFSStore } from './store.js'; export { AztecSQLiteOPFSStore } from './store.js'; export { SqliteEncryptionError } from './errors.js'; export type { SqliteEncryptionErrorCode } from './errors.js'; +export { OPFS_POOL_DIR_PREFIX, storePoolDirectory } from './manage.js'; export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; export type { StoreIdentity } from '../store_identity.js'; +/** + * Opens the persistent store selected by `name` and the identity `(config.l1ChainId, config.rollupAddress, + * schemaVersion)`. A store exists per identity: reopening with the same identity returns the same data, a + * different identity selects a different (possibly fresh) store. Nothing is ever cleared. + */ export async function createStore( name: string, config: DataStoreConfig, schemaVersion: number | undefined = undefined, log: Logger = createLogger('kv-store'), ) { - const { dataDirectory } = config; - log.info( - dataDirectory - ? `Creating ${name} SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB` - : `Creating ${name} ephemeral SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB`, + const storeName = effectiveStoreName(name, { + l1ChainId: config.l1ChainId, + rollupAddress: config.rollupAddress, + schemaVersion, + }); + log.info(`Creating ${storeName} SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB`); + const store = await AztecSQLiteOPFSStore.open( + createLogger('kv-store:sqlite-opfs'), + storeName, + false, + storePoolDirectory(storeName), ); - const store = await AztecSQLiteOPFSStore.open(createLogger('kv-store:sqlite-opfs'), name, false); - return initStoreForRollupAndSchemaVersion(store, schemaVersion, config.rollupAddress, log); + try { + await assertStoreIdentity(store, storeName, schemaVersion, config.rollupAddress); + } catch (err) { + // The store handle owns a worker and OPFS locks; release them before surfacing the refusal. + await store.close().catch(() => {}); + throw err; + } + return store; +} + +/** + * Belt-and-braces invariant check: the identity is part of the physical store name, so the recorded version + * can only disagree if there is a store-naming bug. Refuses to open on mismatch; never clears. + */ +async function assertStoreIdentity( + store: AztecSQLiteOPFSStore, + storeName: string, + schemaVersion: number | undefined, + rollupAddress: EthAddress | undefined, +): Promise { + const expected = new DatabaseVersion(schemaVersion ?? 0, rollupAddress ?? EthAddress.ZERO); + const singleton = store.openSingleton('dbVersion'); + const stored = await singleton.getAsync(); + if (stored === undefined) { + await singleton.set(expected.toBuffer().toString('utf-8')); + return; + } + let storedVersion: DatabaseVersion; + try { + storedVersion = DatabaseVersion.fromBuffer(Buffer.from(stored, 'utf-8')); + } catch { + throw new StoreIdentityMismatchError(storeName, expected.toString(), stored); + } + if (!storedVersion.equals(expected)) { + throw new StoreIdentityMismatchError(storeName, expected.toString(), storedVersion.toString()); + } } export function openTmpStore(ephemeral: boolean = false): Promise { diff --git a/yarn-project/kv-store/src/sqlite-opfs/manage.ts b/yarn-project/kv-store/src/sqlite-opfs/manage.ts new file mode 100644 index 000000000000..ce80d0d7041d --- /dev/null +++ b/yarn-project/kv-store/src/sqlite-opfs/manage.ts @@ -0,0 +1,11 @@ +/** Prefix for the per-store OPFS SAH pool directories owned by this package. */ +export const OPFS_POOL_DIR_PREFIX = '.aztec-kv-'; + +/** + * OPFS directory holding a store's SAH pool. One directory per store: the SAH-pool VFS allows only one + * concurrent instance per directory and its capacity does not grow automatically, so sharing a pool across + * stores would make concurrently opened stores contend for locks and orphaned stores exhaust pool slots. + */ +export function storePoolDirectory(effectiveName: string): string { + return `${OPFS_POOL_DIR_PREFIX}${effectiveName}`; +} From 38e63824523ef29f7bc3d8b8a698f61fb9f89916 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 09:59:47 +0000 Subject: [PATCH 06/48] feat(kv-store): list and delete sqlite-opfs stores --- .../src/sqlite-opfs/create_store.test.ts | 31 ++++++++++++++++++- .../kv-store/src/sqlite-opfs/index.ts | 2 +- .../kv-store/src/sqlite-opfs/manage.ts | 25 +++++++++++++++ yarn-project/tsconfig.json | 2 +- 4 files changed, 57 insertions(+), 3 deletions(-) diff --git a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts index 81f9aa77ff5d..eb6ae0982d27 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts @@ -3,7 +3,7 @@ import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; import { mockLogger } from '../interfaces/utils.js'; import { AztecSQLiteOPFSStore, StoreIdentityMismatchError, createStore, effectiveStoreName } from './index.js'; -import { storePoolDirectory } from './manage.js'; +import { deleteStore, listStores, storePoolDirectory } from './manage.js'; const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ dataDirectory: 'test', @@ -82,4 +82,33 @@ describe('sqlite-opfs createStore', () => { expect(await raw.openSingleton('payload').getAsync()).toEqual('precious'); await raw.close(); }); + + it('lists created stores and deletes them', async () => { + const addr = EthAddress.random(); + const store = await createStore('managed_test', configFor(addr), 1, mockLogger); + await store.openSingleton('k').set('v'); + await store.close(); + + const storeName = effectiveStoreName('managed_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); + expect(await listStores()).toContain(storeName); + + await deleteStore(storeName); + expect(await listStores()).not.toContain(storeName); + + // Recreating after deletion starts empty. + const fresh = await createStore('managed_test', configFor(addr), 1, mockLogger); + expect(await fresh.openSingleton('k').getAsync()).toBeUndefined(); + await fresh.close(); + }); + + it('refuses to delete a store that is currently open', async () => { + const addr = EthAddress.random(); + const store = await createStore('locked_test', configFor(addr), 1, mockLogger); + const storeName = effectiveStoreName('locked_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); + + await expect(deleteStore(storeName)).rejects.toThrow(); + + await store.close(); + await deleteStore(storeName); + }); }); diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index 1de37cfd384f..a30a936fe645 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -10,7 +10,7 @@ import { AztecSQLiteOPFSStore } from './store.js'; export { AztecSQLiteOPFSStore } from './store.js'; export { SqliteEncryptionError } from './errors.js'; export type { SqliteEncryptionErrorCode } from './errors.js'; -export { OPFS_POOL_DIR_PREFIX, storePoolDirectory } from './manage.js'; +export { OPFS_POOL_DIR_PREFIX, deleteStore, listStores, storePoolDirectory } from './manage.js'; export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; export type { StoreIdentity } from '../store_identity.js'; diff --git a/yarn-project/kv-store/src/sqlite-opfs/manage.ts b/yarn-project/kv-store/src/sqlite-opfs/manage.ts index ce80d0d7041d..35e3f760a765 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/manage.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/manage.ts @@ -9,3 +9,28 @@ export const OPFS_POOL_DIR_PREFIX = '.aztec-kv-'; export function storePoolDirectory(effectiveName: string): string { return `${OPFS_POOL_DIR_PREFIX}${effectiveName}`; } + +/** + * Lists the effective names (logical name + identity slug) of every persistent sqlite-opfs store in this + * origin, by enumerating the per-store pool directories. Includes stores created under identities other than + * the current one — that is the point: wallets can surface and clean up data for networks no longer in use. + */ +export async function listStores(): Promise { + const root = await navigator.storage.getDirectory(); + const names: string[] = []; + for await (const [entryName, handle] of root.entries()) { + if (handle.kind === 'directory' && entryName.startsWith(OPFS_POOL_DIR_PREFIX)) { + names.push(entryName.slice(OPFS_POOL_DIR_PREFIX.length)); + } + } + return names; +} + +/** + * Permanently deletes a store by effective name (as returned by {@link listStores}). The store must be closed: + * an open store's SAH pool holds locks on the directory and the removal will reject. + */ +export async function deleteStore(effectiveName: string): Promise { + const root = await navigator.storage.getDirectory(); + await root.removeEntry(storePoolDirectory(effectiveName), { recursive: true }); +} diff --git a/yarn-project/tsconfig.json b/yarn-project/tsconfig.json index 1da5508133cb..367b736ec023 100644 --- a/yarn-project/tsconfig.json +++ b/yarn-project/tsconfig.json @@ -1,7 +1,7 @@ { "compilerOptions": { "target": "es2022", - "lib": ["dom", "esnext", "es2017.object"], + "lib": ["dom", "dom.asynciterable", "esnext", "es2017.object"], "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, From d3b25355d1abd0daf343b14567994ae97b0d4c41 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 10:06:55 +0000 Subject: [PATCH 07/48] feat(kv-store): opt-in identity-partitioned lmdb-v2 stores --- .../kv-store/src/lmdb-v2/factory.test.ts | 80 +++++++++++++++++++ yarn-project/kv-store/src/lmdb-v2/factory.ts | 18 +++-- 2 files changed, 93 insertions(+), 5 deletions(-) create mode 100644 yarn-project/kv-store/src/lmdb-v2/factory.test.ts diff --git a/yarn-project/kv-store/src/lmdb-v2/factory.test.ts b/yarn-project/kv-store/src/lmdb-v2/factory.test.ts new file mode 100644 index 000000000000..4630d8573560 --- /dev/null +++ b/yarn-project/kv-store/src/lmdb-v2/factory.test.ts @@ -0,0 +1,80 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; +import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; + +import { mkdtemp, rm } from 'fs/promises'; +import { tmpdir } from 'os'; +import { join } from 'path'; + +import { createStore } from './factory.js'; + +describe('lmdb-v2 createStore', () => { + let dataDirectory: string; + + beforeEach(async () => { + dataDirectory = await mkdtemp(join(tmpdir(), 'factory-test-')); + }); + + afterEach(async () => { + await rm(dataDirectory, { recursive: true, force: true }); + }); + + const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ + dataDirectory, + dataStoreMapSizeKb: 10 * 1024, + rollupAddress, + l1ChainId, + }); + + it('with partitionByIdentity, keeps data intact when switching rollup addresses back and forth', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + const options = { partitionByIdentity: true }; + + const storeA = await createStore('test_store', 1, configFor(addrA), undefined, options); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await createStore('test_store', 1, configFor(addrB), undefined, options); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.close(); + + const reopenedA = await createStore('test_store', 1, configFor(addrA), undefined, options); + expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); + await reopenedA.close(); + }); + + it('with partitionByIdentity, separates stores by schema version', async () => { + const addr = EthAddress.random(); + const options = { partitionByIdentity: true }; + + const v1 = await createStore('test_store', 1, configFor(addr), undefined, options); + await v1.openSingleton('k').set('v1-data'); + await v1.close(); + + const v2 = await createStore('test_store', 2, configFor(addr), undefined, options); + expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); + await v2.close(); + + const v1Again = await createStore('test_store', 1, configFor(addr), undefined, options); + expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); + await v1Again.close(); + }); + + it('without the flag, keeps the historical reset-on-rollup-change behavior', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + + const storeA = await createStore('test_store', 1, configFor(addrA)); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await createStore('test_store', 1, configFor(addrB)); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.close(); + + // Historical behavior is destructive: the data does not come back. + const reopenedA = await createStore('test_store', 1, configFor(addrA)); + expect(await reopenedA.openSingleton('payload').getAsync()).toBeUndefined(); + await reopenedA.close(); + }); +}); diff --git a/yarn-project/kv-store/src/lmdb-v2/factory.ts b/yarn-project/kv-store/src/lmdb-v2/factory.ts index d4396ef7dd48..6fd286f46090 100644 --- a/yarn-project/kv-store/src/lmdb-v2/factory.ts +++ b/yarn-project/kv-store/src/lmdb-v2/factory.ts @@ -11,6 +11,7 @@ import { copyFile, mkdir, mkdtemp, rm } from 'fs/promises'; import { tmpdir } from 'os'; import { join } from 'path'; +import { storeIdentitySlug } from '../store_identity.js'; import { AztecLMDBStoreV2 } from './store.js'; const MAX_READERS = 16; @@ -20,6 +21,12 @@ export type CreateStoreOptions = { onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; schemaVersionMismatchPolicy?: SchemaVersionMismatchPolicy; versionFileReadFailurePolicy?: VersionFileReadFailurePolicy; + /** + * When true, the store directory is keyed by (l1ChainId, rollupAddress, schemaVersion): a different identity + * selects a different directory instead of resetting this one, so no identity change is ever destructive. + * The version file becomes a pure invariant check (any mismatch throws). + */ + partitionByIdentity?: boolean; }; export async function createStore( @@ -35,10 +42,11 @@ export async function createStore( let store: AztecLMDBStoreV2; if (typeof dataDirectory !== 'undefined') { // Get rollup address from contracts config, or use zero address - const subDir = join(dataDirectory, name); - await mkdir(subDir, { recursive: true }); - const rollupAddress = rollupFromConfig ?? EthAddress.ZERO; + const subDir = options.partitionByIdentity + ? join(dataDirectory, name, storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress, schemaVersion })) + : join(dataDirectory, name); + await mkdir(subDir, { recursive: true }); // Create a version manager const versionManager = new DatabaseVersionManager({ @@ -48,8 +56,8 @@ export async function createStore( onOpen: dbDirectory => AztecLMDBStoreV2.new(dbDirectory, config.dataStoreMapSizeKb, MAX_READERS, () => Promise.resolve(), bindings), onUpgrade: options.onUpgrade, - schemaVersionMismatchPolicy: options.schemaVersionMismatchPolicy, - versionFileReadFailurePolicy: options.versionFileReadFailurePolicy, + schemaVersionMismatchPolicy: options.partitionByIdentity ? 'throw' : options.schemaVersionMismatchPolicy, + versionFileReadFailurePolicy: options.partitionByIdentity ? 'throw' : options.versionFileReadFailurePolicy, }); log.info( From 6509dd1d5cf6d0c0718342cfced16ba90f72f483 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 10:15:19 +0000 Subject: [PATCH 08/48] feat(pxe): node PXE and embedded wallet stores partitioned by identity --- yarn-project/pxe/src/entrypoints/server/utils.ts | 1 + .../wallets/src/embedded/entrypoints/node.ts | 12 +++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/server/utils.ts b/yarn-project/pxe/src/entrypoints/server/utils.ts index 815c30e01071..0138f2101ddf 100644 --- a/yarn-project/pxe/src/entrypoints/server/utils.ts +++ b/yarn-project/pxe/src/entrypoints/server/utils.ts @@ -51,6 +51,7 @@ export async function createPXE( PXE_DATA_SCHEMA_VERSION, configWithContracts, storeLogger.getBindings(), + { partitionByIdentity: true }, ); } const proverLogger = loggers.prover ?? createLogger('pxe:bb:native', { actor }); diff --git a/yarn-project/wallets/src/embedded/entrypoints/node.ts b/yarn-project/wallets/src/embedded/entrypoints/node.ts index f7772599c5a8..583567d1f214 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/node.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/node.ts @@ -28,7 +28,7 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { const rootLogger = options.logger ?? createLogger('embedded-wallet'); const aztecNode = typeof nodeOrUrl === 'string' ? createAztecNodeClient(nodeOrUrl) : nodeOrUrl; - const l1Contracts = await aztecNode.getL1ContractAddresses(); + const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); // Support both the new unified `pxe` option and the deprecated `pxeConfig`/`pxeOptions`. const { config: pxeConfigFromPxe, creation: pxeCreationFromPxe } = splitPxeOptions(options.pxe); @@ -37,7 +37,7 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { const pxeConfig: PXEConfig = Object.assign(getPXEConfig(), { proverEnabled: mergedConfigOverrides.proverEnabled, - dataDirectory: `pxe_data_${l1Contracts.rollupAddress}`, + dataDirectory: 'aztec-wallet-data', autoSync: false, ...mergedConfigOverrides, }); @@ -69,7 +69,7 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { options.walletDb?.store ?? (options.ephemeral ? await openTmpStore( - `wallet_data_${l1Contracts.rollupAddress}`, + 'wallet_data', true, undefined, undefined, @@ -79,11 +79,13 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { 'wallet_data', 1, { - dataDirectory: `wallet_data_${l1Contracts.rollupAddress}`, + dataDirectory: pxeConfig.dataDirectory ?? 'aztec-wallet-data', dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, - rollupAddress: l1Contracts.rollupAddress, + rollupAddress: l1ContractAddresses.rollupAddress, + l1ChainId, }, rootLogger.createChild('wallet:data').getBindings(), + { partitionByIdentity: true }, )); const walletDB = new WalletDB(walletDBStore, rootLogger.createChild('wallet:db').info); From 4e0036d8b28f9831e8c8e1a987db7c9223f13fe5 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 10:21:13 +0000 Subject: [PATCH 09/48] feat(pxe): browser PXE and embedded wallet stores partitioned by identity --- yarn-project/pxe/src/entrypoints/client/bundle/utils.ts | 4 +++- yarn-project/pxe/src/entrypoints/client/lazy/utils.ts | 4 +++- yarn-project/wallets/src/embedded/entrypoints/browser.ts | 9 +++++---- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts index 3c37cd2d3490..0f64428d9545 100644 --- a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts @@ -31,10 +31,12 @@ export async function createPXE( const actor = options.loggerActorLabel; const loggers = options.loggers ?? {}; - const l1ContractAddresses = await aztecNode.getL1ContractAddresses(); + const { l1ChainId, l1ContractAddresses, rollupVersion } = await aztecNode.getNodeInfo(); const configWithContracts = { ...config, ...l1ContractAddresses, + l1ChainId, + rollupVersion, } as PXEConfig; const storeLogger = loggers.store ?? createLogger('pxe:data', { actor }); diff --git a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts index a036c9b7bc7a..b4dd06ab3312 100644 --- a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts @@ -29,10 +29,12 @@ export async function createPXE( ) { const actor = options.loggerActorLabel; - const l1ContractAddresses = await aztecNode.getL1ContractAddresses(); + const { l1ChainId, l1ContractAddresses, rollupVersion } = await aztecNode.getNodeInfo(); const configWithContracts = { ...config, ...l1ContractAddresses, + l1ChainId, + rollupVersion, } as PXEConfig; const loggers = options.loggers ?? {}; diff --git a/yarn-project/wallets/src/embedded/entrypoints/browser.ts b/yarn-project/wallets/src/embedded/entrypoints/browser.ts index 8a5a7bdaf1ca..d72f0df46d70 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/browser.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/browser.ts @@ -27,7 +27,7 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { const rootLogger = options.logger ?? createLogger('embedded-wallet'); const aztecNode = typeof nodeOrUrl === 'string' ? createAztecNodeClient(nodeOrUrl) : nodeOrUrl; - const l1Contracts = await aztecNode.getL1ContractAddresses(); + const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); // Support both the new unified `pxe` option and the deprecated `pxeConfig`/`pxeOptions`. const { config: pxeConfigFromPxe, creation: pxeCreationFromPxe } = splitPxeOptions(options.pxe); @@ -36,7 +36,7 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { const pxeConfig: PXEConfig = Object.assign(getPXEConfig(), { proverEnabled: mergedConfigOverrides.proverEnabled, - dataDirectory: `pxe_data_${l1Contracts.rollupAddress}`, + dataDirectory: 'pxe_data', autoSync: false, ...mergedConfigOverrides, }); @@ -71,9 +71,10 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { : await createStore( 'wallet_data', { - dataDirectory: `wallet_data_${l1Contracts.rollupAddress}`, + dataDirectory: 'wallet_data', dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, - rollupAddress: l1Contracts.rollupAddress, + rollupAddress: l1ContractAddresses.rollupAddress, + l1ChainId, }, 1, rootLogger.createChild('wallet:data'), From 31093a9116515060a1c13568363368739c4a819e Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 10:34:20 +0000 Subject: [PATCH 10/48] docs: changelog for identity-partitioned PXE stores --- .../docs/resources/migration_notes.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 821acf763150..0badccf65fa2 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -9,6 +9,19 @@ Aztec is in active development. Each version may introduce breaking changes that ## TBD +### [PXE] Stores are now selected by `(l1ChainId, rollupAddress, schemaVersion)` instead of being wiped on mismatch + +Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. A store now exists per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. + +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; data from the previous store is not deleted and remains on disk under its old identity. Browser apps using `@aztec/kv-store/sqlite-opfs` can enumerate and clean up stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: + +```typescript +import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; + +const names = await listStores(); +await deleteStore(names[0]); // the store must be closed first +``` + ### [Aztec.nr] `TestEnvironmentOptions::with_tagging_secret_strategy` replaced `TestEnvironmentOptions::with_tagging_secret_strategy` is now `with_default_tag_secret_strategy_all_modes` for tests From 0bdd2e4bba355a7448714161c90f579982c142e1 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 11:10:11 +0000 Subject: [PATCH 11/48] fix(stdlib): separate rollup-address mismatch policy and fix zero-address first boot Gate the schema-mismatch throw on a successfully parsed version file so a first boot with a zero rollup address (storedVersion (0, ZERO) vs (N, ZERO)) resets and opens instead of throwing forever. Split rollup-address mismatch handling into its own RollupAddressMismatchPolicy (default 'reset') so the validator HA signer keeps silent reset-on-rollup-change while schema mismatches stay fail-closed. For partitioned lmdb-v2 stores, use a sibling '-stores' directory instead of nesting inside the legacy env dir, so an old binary's rollup-mismatch rm -rf of the legacy dir cannot destroy per-identity stores; force all three mismatch policies to 'throw' when partitionByIdentity is set. --- .../kv-store/src/lmdb-v2/factory.test.ts | 25 +++++++++++++++- yarn-project/kv-store/src/lmdb-v2/factory.ts | 11 +++++-- .../database-version/version_manager.test.ts | 24 ++++++++++++++- .../src/database-version/version_manager.ts | 30 ++++++++++++++++--- 4 files changed, 82 insertions(+), 8 deletions(-) diff --git a/yarn-project/kv-store/src/lmdb-v2/factory.test.ts b/yarn-project/kv-store/src/lmdb-v2/factory.test.ts index 4630d8573560..1f727f96951a 100644 --- a/yarn-project/kv-store/src/lmdb-v2/factory.test.ts +++ b/yarn-project/kv-store/src/lmdb-v2/factory.test.ts @@ -1,7 +1,7 @@ import { EthAddress } from '@aztec/foundation/eth-address'; import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; -import { mkdtemp, rm } from 'fs/promises'; +import { mkdtemp, rm, stat } from 'fs/promises'; import { tmpdir } from 'os'; import { join } from 'path'; @@ -60,6 +60,29 @@ describe('lmdb-v2 createStore', () => { await v1Again.close(); }); + it('with partitionByIdentity, opens and persists on first boot with a zero identity (no rollup, no chain id)', async () => { + const config: DataStoreConfig = { dataDirectory, dataStoreMapSizeKb: 10 * 1024 }; + const options = { partitionByIdentity: true }; + + const store = await createStore('test_store', 1, config, undefined, options); + await store.openSingleton('payload').set('zero-identity-data'); + await store.close(); + + const reopened = await createStore('test_store', 1, config, undefined, options); + expect(await reopened.openSingleton('payload').getAsync()).toEqual('zero-identity-data'); + await reopened.close(); + }); + + it('with partitionByIdentity, places stores under a sibling -stores directory, not nested in ', async () => { + const store = await createStore('test_store', 1, configFor(EthAddress.random()), undefined, { + partitionByIdentity: true, + }); + await store.close(); + + await expect(stat(join(dataDirectory, 'test_store-stores'))).resolves.toBeDefined(); + await expect(stat(join(dataDirectory, 'test_store'))).rejects.toMatchObject({ code: 'ENOENT' }); + }); + it('without the flag, keeps the historical reset-on-rollup-change behavior', async () => { const addrA = EthAddress.random(); const addrB = EthAddress.random(); diff --git a/yarn-project/kv-store/src/lmdb-v2/factory.ts b/yarn-project/kv-store/src/lmdb-v2/factory.ts index 6fd286f46090..9727d5807860 100644 --- a/yarn-project/kv-store/src/lmdb-v2/factory.ts +++ b/yarn-project/kv-store/src/lmdb-v2/factory.ts @@ -24,7 +24,9 @@ export type CreateStoreOptions = { /** * When true, the store directory is keyed by (l1ChainId, rollupAddress, schemaVersion): a different identity * selects a different directory instead of resetting this one, so no identity change is ever destructive. - * The version file becomes a pure invariant check (any mismatch throws). + * Per-identity stores live under a sibling `-stores` directory (not nested inside the legacy `` + * env dir), so an old binary's reset of the legacy dir cannot wipe them. The version file becomes a pure + * invariant check (any mismatch throws). */ partitionByIdentity?: boolean; }; @@ -44,7 +46,11 @@ export async function createStore( // Get rollup address from contracts config, or use zero address const rollupAddress = rollupFromConfig ?? EthAddress.ZERO; const subDir = options.partitionByIdentity - ? join(dataDirectory, name, storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress, schemaVersion })) + ? join( + dataDirectory, + `${name}-stores`, + storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress, schemaVersion }), + ) : join(dataDirectory, name); await mkdir(subDir, { recursive: true }); @@ -57,6 +63,7 @@ export async function createStore( AztecLMDBStoreV2.new(dbDirectory, config.dataStoreMapSizeKb, MAX_READERS, () => Promise.resolve(), bindings), onUpgrade: options.onUpgrade, schemaVersionMismatchPolicy: options.partitionByIdentity ? 'throw' : options.schemaVersionMismatchPolicy, + rollupAddressMismatchPolicy: options.partitionByIdentity ? 'throw' : undefined, versionFileReadFailurePolicy: options.partitionByIdentity ? 'throw' : options.versionFileReadFailurePolicy, }); diff --git a/yarn-project/stdlib/src/database-version/version_manager.test.ts b/yarn-project/stdlib/src/database-version/version_manager.test.ts index 676dfa3d1e9c..4870c81308d3 100644 --- a/yarn-project/stdlib/src/database-version/version_manager.test.ts +++ b/yarn-project/stdlib/src/database-version/version_manager.test.ts @@ -143,7 +143,7 @@ describe('VersionManager', () => { it('unless rollup mismatches are configured to throw', async () => { fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); - versionManager = createManager({ schemaVersionMismatchPolicy: 'throw' }); + versionManager = createManager({ rollupAddressMismatchPolicy: 'throw' }); expectVersionFileWritten = false; await expect(versionManager.open()).rejects.toThrow(/stored rollup address/); @@ -151,6 +151,15 @@ describe('VersionManager', () => { expect(openSpy).not.toHaveBeenCalled(); }); + it('resets on a rollup mismatch when only the schema policy is throw, preserving HA-signer behavior', async () => { + fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); + versionManager = createManager({ schemaVersionMismatchPolicy: 'throw' }); + const [_, wasReset] = await versionManager.open(); + expect(wasReset).toEqual(true); + expect(openSpy).toHaveBeenCalled(); + expect(upgradeSpy).not.toHaveBeenCalled(); + }); + it("still opens fresh on first boot when policy is 'throw' (no version file, different rollup)", async () => { fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); versionManager = createManager({ schemaVersionMismatchPolicy: 'throw', onUpgrade: undefined }); @@ -158,6 +167,19 @@ describe('VersionManager', () => { expect(wasReset).toEqual(true); expect(openSpy).toHaveBeenCalled(); }); + + it("opens fresh on first boot with a zero rollup address even when schema policy is 'throw'", async () => { + fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); + rollupAddress = EthAddress.ZERO; + versionManager = createManager({ + rollupAddress: EthAddress.ZERO, + schemaVersionMismatchPolicy: 'throw', + onUpgrade: undefined, + }); + const [_, wasReset] = await versionManager.open(); + expect(wasReset).toEqual(true); + expect(openSpy).toHaveBeenCalled(); + }); }); describe('version file read-failure policy', () => { diff --git a/yarn-project/stdlib/src/database-version/version_manager.ts b/yarn-project/stdlib/src/database-version/version_manager.ts index bc9f7de2dff6..2f40f66b5119 100644 --- a/yarn-project/stdlib/src/database-version/version_manager.ts +++ b/yarn-project/stdlib/src/database-version/version_manager.ts @@ -9,8 +9,24 @@ import { DatabaseVersion } from './database_version.js'; export type DatabaseVersionManagerFs = Pick; export const DATABASE_VERSION_FILE_NAME = 'db_version'; + +/** + * How to react when a parsed version file records a schema version incompatible with the current one and no + * upgrade path applies. `'reset'` (default) wipes and recreates the data directory. `'throw'` refuses to open, + * leaving data untouched — for stores that must fail closed rather than be silently reset. Only a successfully + * parsed version file can trigger the throw; first boot (no version file) always proceeds. + */ export type SchemaVersionMismatchPolicy = 'reset' | 'throw'; +/** + * How to react when a parsed version file records a different rollup address than the current one. `'reset'` + * (default) keeps the historical behavior of resetting the data directory when the stored rollup address differs. + * `'throw'` refuses to open instead, for stores whose location already encodes the rollup identity (a mismatch + * then indicates a bug, and data must not be destroyed). Only a successfully parsed version file can trigger the + * throw; first boot (no version file) always proceeds. + */ +export type RollupAddressMismatchPolicy = 'reset' | 'throw'; + /** * How to react when the version file exists but cannot be read (permissions, IO error, truncation). * `'reset'` (default) treats the store as unversioned and lets the reset path run — safe for stores @@ -27,6 +43,7 @@ export type DatabaseVersionManagerOptions = { onOpen: (dataDir: string) => Promise; onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; schemaVersionMismatchPolicy?: SchemaVersionMismatchPolicy; + rollupAddressMismatchPolicy?: RollupAddressMismatchPolicy; versionFileReadFailurePolicy?: VersionFileReadFailurePolicy; fileSystem?: DatabaseVersionManagerFs; log?: Logger; @@ -47,6 +64,7 @@ export class DatabaseVersionManager { private onOpen: (dataDir: string) => Promise; private onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; private schemaVersionMismatchPolicy: SchemaVersionMismatchPolicy; + private rollupAddressMismatchPolicy: RollupAddressMismatchPolicy; private versionFileReadFailurePolicy: VersionFileReadFailurePolicy; private fileSystem: DatabaseVersionManagerFs; private log: Logger; @@ -61,8 +79,10 @@ export class DatabaseVersionManager { * @param onUpgrade - An optional callback to upgrade the database before opening. If not provided it will reset the * database. Must be idempotent: since the version marker is written only after a successful open, a crash after * onUpgrade but before the marker is written re-runs onUpgrade on the next start. - * @param schemaVersionMismatchPolicy - Whether schema or rollup-address mismatches against an existing version - * file should reset data or throw + * @param schemaVersionMismatchPolicy - Whether an incompatible schema version in a parsed version file should + * reset data or throw + * @param rollupAddressMismatchPolicy - Whether a different rollup address in a parsed version file should reset + * data or throw * @param versionFileReadFailurePolicy - Whether an unreadable (non-missing) version file should reset data or throw * @param fileSystem - An interface to access the filesystem * @param log - Optional custom logger @@ -75,6 +95,7 @@ export class DatabaseVersionManager { onOpen, onUpgrade, schemaVersionMismatchPolicy = 'reset', + rollupAddressMismatchPolicy = 'reset', versionFileReadFailurePolicy = 'reset', fileSystem = fs, log = createLogger(`foundation:version-manager`), @@ -90,6 +111,7 @@ export class DatabaseVersionManager { this.onOpen = onOpen; this.onUpgrade = onUpgrade; this.schemaVersionMismatchPolicy = schemaVersionMismatchPolicy; + this.rollupAddressMismatchPolicy = rollupAddressMismatchPolicy; this.versionFileReadFailurePolicy = versionFileReadFailurePolicy; this.fileSystem = fileSystem; this.log = log; @@ -189,7 +211,7 @@ export class DatabaseVersionManager { needsReset = true; } } else if (cmp !== 0) { - if (this.schemaVersionMismatchPolicy === 'throw') { + if (versionFileParsed && this.schemaVersionMismatchPolicy === 'throw') { throw new Error( `Cannot open database at ${this.dataDirectory}: stored schema version ${storedVersion.schemaVersion} is incompatible with expected schema version ${this.currentVersion.schemaVersion}`, ); @@ -202,7 +224,7 @@ export class DatabaseVersionManager { needsReset = true; } } else { - if (versionFileParsed && this.schemaVersionMismatchPolicy === 'throw') { + if (versionFileParsed && this.rollupAddressMismatchPolicy === 'throw') { throw new Error( `Cannot open database at ${this.dataDirectory}: stored rollup address ` + `${storedVersion.rollupAddress} does not match expected ${this.currentVersion.rollupAddress}`, From d4b69f110d52874e1e9ee3805eeef1c554dda257 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 11:10:25 +0000 Subject: [PATCH 12/48] docs: align store-selection docs with rollup-policy split and sibling layout Update the database-version README reset conditions, the store-selection design doc (sibling -stores layout, three throw policies, HA-signer reset default), and the migration note (ts fence, per-backend on-disk data location). --- .../docs/resources/migration_notes.md | 4 ++-- ...-07-09-store-selection-by-identity-design.md | 12 +++++++++--- .../stdlib/src/database-version/README.md | 17 +++++++++++------ 3 files changed, 22 insertions(+), 11 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 0badccf65fa2..2726597fe2b2 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -13,9 +13,9 @@ Aztec is in active development. Each version may introduce breaking changes that Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. A store now exists per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; data from the previous store is not deleted and remains on disk under its old identity. Browser apps using `@aztec/kv-store/sqlite-opfs` can enumerate and clean up stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is no longer returned by `listStores()`; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity stores live under `/-stores/`. Browser apps can enumerate and clean up stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: -```typescript +```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; const names = await listStores(); diff --git a/docs/plans/2026-07-09-store-selection-by-identity-design.md b/docs/plans/2026-07-09-store-selection-by-identity-design.md index da3bcc24dd76..9a7d1e92f4d0 100644 --- a/docs/plans/2026-07-09-store-selection-by-identity-design.md +++ b/docs/plans/2026-07-09-store-selection-by-identity-design.md @@ -46,6 +46,9 @@ Scope decisions: - **Node infra untouched**: archiver/world-state/p2p keep `DatabaseVersionManager` reset semantics. Their data is public and resyncable; keeping per-rollup copies of 100GB+ stores is an operator cost decision out of scope here. +- **Validator HA signer untouched**: `rollupAddressMismatchPolicy` defaults to `'reset'`, so the signer + store keeps its historical reset-on-rollup-change behavior. Only stores that opt into a rollup-encoding + location (via `partitionByIdentity`) upgrade the rollup mismatch to a fail-closed throw. - **KeyStore split deferred**: keys remain inside the per-identity store. They are no longer destroyed on rollup change, only stranded per store (accounts appear empty on a new rollup until re-imported). Giving key material its own chain-agnostic store is a follow-up; this design keeps @@ -77,9 +80,12 @@ Gains optional `l1ChainId` (`yarn-project/stdlib/src/kv-store/config.ts`). ### lmdb-v2 `createStore` - Gains an opt-in `CreateStoreOptions.partitionByIdentity`. -- When set: data dir becomes `join(dataDirectory, name, slug)` and `DatabaseVersionManager` runs - with `schemaVersionMismatchPolicy: 'throw'` and `versionFileReadFailurePolicy: 'throw'` — the - version file becomes a pure invariant check plus migration metadata. +- When set: data dir becomes `join(dataDirectory, `${name}-stores`, slug)` — a sibling of the legacy + `join(dataDirectory, name)` env dir, not a child of it. An old binary reaching a rollup mismatch + `rm -rf`s the legacy dir; nesting the per-identity stores inside it would let that wipe them, so the + partitioned stores must live outside. `DatabaseVersionManager` runs with `schemaVersionMismatchPolicy`, + `rollupAddressMismatchPolicy`, and `versionFileReadFailurePolicy` all set to `'throw'` — the version + file becomes a pure invariant check plus migration metadata. - Default off: every node-infra caller is untouched. ### Callers diff --git a/yarn-project/stdlib/src/database-version/README.md b/yarn-project/stdlib/src/database-version/README.md index ea2b739377c7..a10a4230eb87 100644 --- a/yarn-project/stdlib/src/database-version/README.md +++ b/yarn-project/stdlib/src/database-version/README.md @@ -53,11 +53,16 @@ const dataDir = versionManager.getDataDirectory(); ## Automatic Reset Conditions -The database will be reset in the following conditions: +The database is reset in the following conditions: -1. No version information exists (first run) -2. Rollup address has changed -3. Version has changed and no upgrade callback is provided -4. Upgrade callback throws an error +1. First boot: no version file exists yet. +2. The version file exists but cannot be read or parsed (permissions, IO error, truncation), unless + `versionFileReadFailurePolicy: 'throw'` is set — then it refuses to open and leaves data untouched. +3. The stored schema version is incompatible with the current one and no upgrade path applies (no upgrade callback, + or the callback throws), unless `schemaVersionMismatchPolicy: 'throw'` is set — then it refuses to open. +4. The stored rollup address differs from the current one, unless `rollupAddressMismatchPolicy: 'throw'` is set — + then it refuses to open. -When a reset occurs, the data directory is deleted and recreated, and the reset callback is called to initialize a fresh database. +The `'throw'` policies only fire when a version file was successfully parsed: first boot always proceeds and never +throws. When a reset occurs, the data directory is deleted and recreated, and the reset callback is called to +initialize a fresh database. From 1551290bd3be07fea54a3734521b7dd90cd0e260 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 11:16:35 +0000 Subject: [PATCH 13/48] docs: mark implementation plan as historical after final-review amendments --- docs/plans/2026-07-09-store-selection-by-identity-plan.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/plans/2026-07-09-store-selection-by-identity-plan.md b/docs/plans/2026-07-09-store-selection-by-identity-plan.md index 5997b19e5afa..e61ffdd9de64 100644 --- a/docs/plans/2026-07-09-store-selection-by-identity-plan.md +++ b/docs/plans/2026-07-09-store-selection-by-identity-plan.md @@ -1,5 +1,11 @@ # PXE Store Selection by Identity — Implementation Plan +> **Historical artifact.** This plan was executed and then amended during final review: the lmdb-v2 layout +> became a sibling directory (`-stores/`) and the rollup-address mismatch policy was split into its +> own `rollupAddressMismatchPolicy` option (default `'reset'`). The design doc +> (`2026-07-09-store-selection-by-identity-design.md`) reflects the final semantics; code snippets below predate +> the amendments. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Replace wipe-on-mismatch with store selection keyed on `(l1ChainId, rollupAddress, schemaVersion)` for both PXE storage backends, per the approved design in `docs/plans/2026-07-09-store-selection-by-identity-design.md`. From 0452dbe0c6e63f49112bac29448c56484ba4cf12 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 11:29:27 +0000 Subject: [PATCH 14/48] chore: remove local planning docs from version control --- ...7-09-store-selection-by-identity-design.md | 161 --- ...-07-09-store-selection-by-identity-plan.md | 1039 ----------------- 2 files changed, 1200 deletions(-) delete mode 100644 docs/plans/2026-07-09-store-selection-by-identity-design.md delete mode 100644 docs/plans/2026-07-09-store-selection-by-identity-plan.md diff --git a/docs/plans/2026-07-09-store-selection-by-identity-design.md b/docs/plans/2026-07-09-store-selection-by-identity-design.md deleted file mode 100644 index 9a7d1e92f4d0..000000000000 --- a/docs/plans/2026-07-09-store-selection-by-identity-design.md +++ /dev/null @@ -1,161 +0,0 @@ -# PXE store selection by identity - -**Date:** 2026-07-09 -**Branch:** `martin/change-store-version-behavior` -**Status:** Approved design, pending implementation - -## Problem - -The PXE-backing stores are wiped whenever the rollup address or schema version recorded in the -store differs from what the node reports at startup: - -- Browser (sqlite-opfs): `initStoreForRollupAndSchemaVersion` (`yarn-project/kv-store/src/utils.ts`) - unconditionally calls `store.clear()` on any mismatch, then rewrites the `dbVersion` marker. -- Node.js (lmdb-v2): `DatabaseVersionManager` (`yarn-project/stdlib/src/database-version/version_manager.ts`) - supports a `'reset' | 'throw'` policy for schema mismatches, but the rollup-address branch resets - unconditionally regardless of policy. - -The store handle is shared by `KeyStore` (`yarn-project/pxe/src/storage/open_pxe_stores.ts`), so the -wipe destroys the account master secret keys (`ivsk_m/ovsk_m/tsk_m/nhk_m`) along with notes, -contracts, and tagging state. The trigger is unauthenticated node-reported data -(`getL1ContractAddresses()` / `getNodeInfo()`), and no attacker is required: a testnet rollup -redeploy or switching between nodes on different rollups is enough. The wipe is also sticky — -after clearing, the marker is rewritten to the new identity, so reconnecting to the original node -wipes again. - -The embedded wallet already *tries* to partition per rollup by passing -`dataDirectory: wallet_data_${rollupAddress}` / `pxe_data_${rollupAddress}` -(`yarn-project/wallets/src/embedded/entrypoints/browser.ts`), but sqlite-opfs keys stores on `name` -only and ignores `dataDirectory`, so the partitioning is a silent no-op in the browser. - -History: version detection was introduced in PR #20007, which acknowledged wipe-on-mismatch as an -aggressive stopgap. An alternative proposal (add a `'throw'` refusal policy) handles the -round-trip case badly: every switch between two rollups raises an "erase your data?" decision, and -consenting destroys the other rollup's data. - -## Decision - -A store's identity is **`(l1ChainId, rollupAddress, schemaVersion)`**. Opening a store *selects* -the physical store matching that identity: if it exists, reuse it; if not, create it empty. -Nothing is ever cleared. "Mismatch" stops being expressible — a different identity is a different -store. - -Scope decisions: - -- **Both PXE paths**: browser sqlite-opfs and Node.js lmdb-v2 (server PXE, CLI wallets, sandbox). -- **Node infra untouched**: archiver/world-state/p2p keep `DatabaseVersionManager` reset semantics. - Their data is public and resyncable; keeping per-rollup copies of 100GB+ stores is an operator - cost decision out of scope here. -- **Validator HA signer untouched**: `rollupAddressMismatchPolicy` defaults to `'reset'`, so the signer - store keeps its historical reset-on-rollup-change behavior. Only stores that opt into a rollup-encoding - location (via `partitionByIdentity`) upgrade the rollup mismatch to a fail-closed throw. -- **KeyStore split deferred**: keys remain inside the per-identity store. They are no longer - destroyed on rollup change, only stranded per store (accounts appear empty on a new rollup until - re-imported). Giving key material its own chain-agnostic store is a follow-up; this design keeps - it possible (a future `keystore_data` store simply omits chain identity from its slug). - -## Design - -### Identity slug helper (new, in `kv-store`) - -`storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion })` → e.g. -`31337-0x-v12`. Full address, no truncation. Shared by both backends and -exported so callers that hand-build store names (e.g. the encrypted embedded-store path, which -takes caller-provided names) can compose correctly. Missing values default as today: -`rollupAddress` → zero address, `l1ChainId` → 0. - -### `DataStoreConfig` - -Gains optional `l1ChainId` (`yarn-project/stdlib/src/kv-store/config.ts`). - -### sqlite-opfs `createStore` - -- Composes the effective DB name as `${name}_${slug}`. -- Stops calling `initStoreForRollupAndSchemaVersion` entirely. -- Still writes the `dbVersion` singleton on first open; a mismatch on reopen **throws** a typed - `StoreIdentityMismatchError` — it can only mean a naming bug — and must never wipe. -- No opt-in flag: the only callers are PXE and the embedded wallet, both of which want the new - behavior. - -### lmdb-v2 `createStore` - -- Gains an opt-in `CreateStoreOptions.partitionByIdentity`. -- When set: data dir becomes `join(dataDirectory, `${name}-stores`, slug)` — a sibling of the legacy - `join(dataDirectory, name)` env dir, not a child of it. An old binary reaching a rollup mismatch - `rm -rf`s the legacy dir; nesting the per-identity stores inside it would let that wipe them, so the - partitioned stores must live outside. `DatabaseVersionManager` runs with `schemaVersionMismatchPolicy`, - `rollupAddressMismatchPolicy`, and `versionFileReadFailurePolicy` all set to `'throw'` — the version - file becomes a pure invariant check plus migration metadata. -- Default off: every node-infra caller is untouched. - -### Callers - -- Browser `createPXE` (bundle + lazy): switch `getL1ContractAddresses()` → `getNodeInfo()` to - obtain `l1ChainId`, thread it into the store config. -- Server `createPXE`: already has `l1ChainId`; pass `partitionByIdentity: true`. -- Embedded wallet (browser + node entrypoints): drop the hand-rolled - `wallet_data_${rollupAddress}` / `pxe_data_${rollupAddress}` dataDirectory suffixes; the - partitioning now lives where it is enforced. The node entrypoint's `wallet_data` store passes - `partitionByIdentity: true`; the browser entrypoint gets the behavior from sqlite-opfs - `createStore` directly. - -### Store management utilities (sqlite-opfs) - -`listStores()` and `deleteStore(name)` exposed from the kv-store package. The worker already -implements `deleteDb`; listing comes from the SAH pool's file names. No auto-pruning — retention -policy belongs to the wallet UI, which can now show "you have data for N networks" and offer -cleanup. - -### OPFS pool verification (implementation-time task, with a test) - -Two facts must be established empirically before the sqlite-opfs work can be called done: - -1. How `pxe_data` + `wallet_data` coexist today in the default pool directory - (`.aztec-kv`), given the documented exclusive per-directory lock of the SAH pool. -2. Whether the pool grows past `initialCapacity: 8` — orphaned per-identity stores consume pool - slots, so without growth, accumulation could make *new* store creation fail. - -If capacity does not auto-grow, either handle `addCapacity` on open-failure or move to a -pool-directory-per-store layout. The design adapts here if forced; everything else is unaffected. - -## Error handling - -No path in the new code ever calls `store.clear()` or resets a directory. Failures are typed -throws: identity mismatch (naming bug), version-file unreadable (lmdb-v2 `'throw'` policy), -decryption failure (existing `SqliteEncryptionError`). The stickiness bug is structurally gone — -there is no marker to rewrite on any failure path. - -## Migration & compatibility - -- Existing `pxe_data` / `wallet_data` stores are orphaned in place, not adopted: a one-time - "fresh store" experience identical to a schema bump, with old data left recoverable on disk. -- Changelog entry required (behavior change: stores are no longer wiped on rollup/schema change; - first start after upgrade begins with an empty store). -- Future schema migrations become read-old-store → write-new-store — crash-safe by construction — - but implementing migrations is explicitly out of scope. - -## Security notes - -- A malicious or misconfigured node can no longer destroy local data; at worst it directs the - wallet at an empty namespace, and the real store is recovered by reconnecting to the right node. -- A malicious node can still mint unbounded empty stores (it controls the reported identity). - Mitigated by list/delete utilities and the pool-capacity handling above; full rate-limiting is - out of scope. - -## Testing - -Red/green: - -1. Red: kv-store test reproducing today's wipe — open store, write, reopen with a different - `rollupAddress`, observe data gone. -2. Green after the change: both stores coexist; original data intact under the original identity. - -Plus: - -- Slug composition unit tests (defaults, formatting stability). -- sqlite-opfs isolation-by-identity (browser test suite): write under identity A, open identity B - (empty), reopen A (intact). -- lmdb-v2 flag off = today's reset behavior (regression guard for node infra); flag on = - partitioned directories, no reset. -- `listStores` / `deleteStore` behavior. -- OPFS pool capacity/coexistence findings encoded as tests. diff --git a/docs/plans/2026-07-09-store-selection-by-identity-plan.md b/docs/plans/2026-07-09-store-selection-by-identity-plan.md deleted file mode 100644 index e61ffdd9de64..000000000000 --- a/docs/plans/2026-07-09-store-selection-by-identity-plan.md +++ /dev/null @@ -1,1039 +0,0 @@ -# PXE Store Selection by Identity — Implementation Plan - -> **Historical artifact.** This plan was executed and then amended during final review: the lmdb-v2 layout -> became a sibling directory (`-stores/`) and the rollup-address mismatch policy was split into its -> own `rollupAddressMismatchPolicy` option (default `'reset'`). The design doc -> (`2026-07-09-store-selection-by-identity-design.md`) reflects the final semantics; code snippets below predate -> the amendments. - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Replace wipe-on-mismatch with store selection keyed on `(l1ChainId, rollupAddress, schemaVersion)` for both PXE storage backends, per the approved design in `docs/plans/2026-07-09-store-selection-by-identity-design.md`. - -**Architecture:** A new identity-slug helper in `@aztec/kv-store` composes a discriminator string that both backends embed in the physical store location (sqlite-opfs: DB name + per-store OPFS pool directory; lmdb-v2: subdirectory, behind an opt-in flag). Opening a store selects the matching physical store — reuse if it exists, create empty if not. No code path ever clears a populated store; residual version markers become throw-on-mismatch invariant checks. - -**Tech Stack:** TypeScript monorepo (`yarn-project`), vitest (kv-store: node + playwright-browser projects), jest (stdlib), sqlite-wasm OPFS SAH pool, LMDB. - -## Global Constraints - -- Working directory for all commands is `yarn-project` (the Bash tool already runs there — never `cd`). -- Base branch: `merge-train/spartan`; working branch: `martin/change-store-version-behavior` (already checked out). -- Conventional commits; PR squashes on merge; no `Co-Authored-By: Claude` trailers; `git add` must name specific files (never `-u`, `-A`, or `.`). -- Line width 120 chars everywhere including comments. A post-edit hook runs the formatter. -- **Never wipe:** no new or modified code path may call `store.clear()` or delete/reset a directory that could contain user data. First-boot reset of a freshly created empty directory is the only allowed reset. -- New identity slug format (fixed by design, tasks must agree): `--v`, defaults `0`, `EthAddress.ZERO`, `0`. Effective store name: `_`. OPFS pool directory: `.aztec-kv-`. -- Test commands: - - kv-store node project: `yarn workspace @aztec/kv-store test:node ` - - kv-store browser project: `yarn workspace @aztec/kv-store test:browser` (full suite; per-file filtering is not supported by the wrapper script) - - stdlib (jest): `yarn workspace @aztec/stdlib test src/database-version/version_manager.test.ts` -- kv-store vitest browser tests stub `@aztec/foundation/eth-address` and `buffer` (see `kv-store/vitest.config.ts` aliases) — the stubs already support `toString()`, `random()`, `ZERO`, and `schema`, which is all the new code uses. -- Long test output: redirect to a file under `/tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/` and inspect with Read/Grep. - ---- - -### Task 1: Identity slug helper + `DataStoreConfig.l1ChainId` - -**Files:** -- Create: `yarn-project/kv-store/src/store_identity.ts` -- Create: `yarn-project/kv-store/src/store_identity.test.ts` -- Modify: `yarn-project/stdlib/src/kv-store/config.ts` -- Modify: `yarn-project/kv-store/vitest.config.ts` (node-project include list) -- Modify: `yarn-project/kv-store/src/sqlite-opfs/index.ts`, `yarn-project/kv-store/src/lmdb-v2/index.ts` (re-exports) - -**Interfaces:** -- Consumes: `EthAddress` from `@aztec/foundation/eth-address`. -- Produces (used by Tasks 3–7): - - `type StoreIdentity = { l1ChainId?: number; rollupAddress?: EthAddress; schemaVersion?: number }` - - `storeIdentitySlug(identity: StoreIdentity): string` - - `effectiveStoreName(name: string, identity: StoreIdentity): string` - - `class StoreIdentityMismatchError extends Error { storeName: string; expected: string; actual: string }` - - `DataStoreConfig` gains optional `l1ChainId?: number`. - -- [ ] **Step 1: Write the failing test** - -Create `yarn-project/kv-store/src/store_identity.test.ts` (vitest globals are enabled — no test-fn imports): - -```ts -import { EthAddress } from '@aztec/foundation/eth-address'; - -import { effectiveStoreName, storeIdentitySlug } from './store_identity.js'; - -describe('storeIdentitySlug', () => { - it('composes chain id, rollup address and schema version', () => { - const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); - expect(storeIdentitySlug({ l1ChainId: 31337, rollupAddress, schemaVersion: 12 })).toEqual( - '31337-0x1234567890abcdef1234567890abcdef12345678-v12', - ); - }); - - it('defaults missing values to chain 0, zero address, schema 0', () => { - expect(storeIdentitySlug({})).toEqual(`0-${EthAddress.ZERO.toString()}-v0`); - }); - - it('normalizes the rollup address to lowercase hex', () => { - const rollupAddress = EthAddress.fromString('0x1234567890ABCDEF1234567890ABCDEF12345678'); - expect(storeIdentitySlug({ rollupAddress, schemaVersion: 1 })).toEqual( - `0-0x1234567890abcdef1234567890abcdef12345678-v1`, - ); - }); -}); - -describe('effectiveStoreName', () => { - it('joins the logical name and the slug with an underscore', () => { - const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); - expect(effectiveStoreName('pxe_data', { l1ChainId: 1, rollupAddress, schemaVersion: 2 })).toEqual( - 'pxe_data_1-0x1234567890abcdef1234567890abcdef12345678-v2', - ); - }); -}); -``` - -Add the file to the node project's include list in `yarn-project/kv-store/vitest.config.ts`: - -```ts - include: [ - './src/*.test.ts', - './src/lmdb/**/*.test.ts', - './src/lmdb-v2/**/*.test.ts', - './src/stores/**/*.test.ts', - './src/interfaces/**/*.test.ts', - ], -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `yarn workspace @aztec/kv-store test:node src/store_identity.test.ts` -Expected: FAIL — cannot resolve `./store_identity.js`. - -- [ ] **Step 3: Write the implementation** - -Create `yarn-project/kv-store/src/store_identity.ts`: - -```ts -import { EthAddress } from '@aztec/foundation/eth-address'; - -/** The coordinates that determine which physical store a logical store name maps to. */ -export type StoreIdentity = { - /** Chain ID of the L1 the rollup is deployed to. */ - l1ChainId?: number; - /** Address of the rollup contract the store's data pertains to. */ - rollupAddress?: EthAddress; - /** Schema version of the data held in the store. */ - schemaVersion?: number; -}; - -/** - * Composes the store-name discriminator for a store identity. Two identities map to the same physical store iff - * their slugs are equal, so the format must stay stable: `--v`. - */ -export function storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion }: StoreIdentity): string { - return `${l1ChainId ?? 0}-${(rollupAddress ?? EthAddress.ZERO).toString()}-v${schemaVersion ?? 0}`; -} - -/** Composes the physical store name for a logical store name and identity. */ -export function effectiveStoreName(name: string, identity: StoreIdentity): string { - return `${name}_${storeIdentitySlug(identity)}`; -} - -/** - * Thrown when a store's recorded identity does not match the identity it was opened under. Since the identity is - * part of the physical store name, this can only indicate a store-naming bug; the store is left untouched. - */ -export class StoreIdentityMismatchError extends Error { - constructor( - public readonly storeName: string, - public readonly expected: string, - public readonly actual: string, - ) { - super( - `Store '${storeName}' records identity ${actual} but was opened as ${expected}. ` + - `Refusing to open; data was NOT modified.`, - ); - this.name = 'StoreIdentityMismatchError'; - } -} -``` - -In `yarn-project/stdlib/src/kv-store/config.ts`, extend the type and mappings (mirrors `ChainConfig.l1ChainId`, same env var and default, so the duplicate key in `pxeConfigMappings` spreads is harmless): - -```ts -export type DataStoreConfig = { - dataDirectory?: string; - dataStoreMapSizeKb: number; - l1ChainId?: number; -} & Partial>; -``` - -and inside `dataConfigMappings`: - -```ts - l1ChainId: { - env: 'L1_CHAIN_ID', - ...numberConfigHelper(31337), - description: 'The chain ID of the ethereum host.', - }, -``` - -Re-export from both live backends. In `yarn-project/kv-store/src/sqlite-opfs/index.ts` and `yarn-project/kv-store/src/lmdb-v2/index.ts` add: - -```ts -export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; -export type { StoreIdentity } from '../store_identity.js'; -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `yarn workspace @aztec/kv-store test:node src/store_identity.test.ts` -Expected: PASS (4 tests). - -- [ ] **Step 5: Build and commit** - -```bash -yarn build -git add kv-store/src/store_identity.ts kv-store/src/store_identity.test.ts kv-store/vitest.config.ts \ - kv-store/src/sqlite-opfs/index.ts kv-store/src/lmdb-v2/index.ts stdlib/src/kv-store/config.ts -git commit -m "feat(kv-store): store identity slug helper and DataStoreConfig.l1ChainId" -``` - -(Paths in `git add` are relative to the CWD, which is `yarn-project`.) - ---- - -### Task 2: `DatabaseVersionManager` — rollup mismatch honors `'throw'` - -**Files:** -- Modify: `yarn-project/stdlib/src/database-version/version_manager.ts:199-208` -- Test: `yarn-project/stdlib/src/database-version/version_manager.test.ts` - -**Interfaces:** -- Consumes: existing `DatabaseVersionManager`, `SchemaVersionMismatchPolicy`. -- Produces: under `schemaVersionMismatchPolicy: 'throw'`, a rollup-address mismatch against a **successfully parsed** version file throws instead of resetting. First boot (ENOENT) and read-failure paths are unchanged (read failure keeps its own `versionFileReadFailurePolicy`). - -- [ ] **Step 1: Write the failing tests** - -In `yarn-project/stdlib/src/database-version/version_manager.test.ts`, inside `describe('resets the database', ...)`, after the `'when the rollup address changes'` test (line ~142), add (uses the file's existing `fs`, `createManager`, `currentVersion`, `openSpy`, `expectVersionFileWritten` fixtures): - -```ts - it('unless rollup mismatches are configured to throw', async () => { - fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); - versionManager = createManager({ schemaVersionMismatchPolicy: 'throw' }); - expectVersionFileWritten = false; - - await expect(versionManager.open()).rejects.toThrow(/stored rollup address/); - expect(fs.rm).not.toHaveBeenCalled(); - expect(openSpy).not.toHaveBeenCalled(); - }); - - it("still opens fresh on first boot when policy is 'throw' (no version file, different rollup)", async () => { - fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); - versionManager = createManager({ schemaVersionMismatchPolicy: 'throw', onUpgrade: undefined }); - const [_, wasReset] = await versionManager.open(); - expect(wasReset).toEqual(true); - expect(openSpy).toHaveBeenCalled(); - }); -``` - -- [ ] **Step 2: Run tests to verify the first fails** - -Run: `yarn workspace @aztec/stdlib test src/database-version/version_manager.test.ts` -Expected: `unless rollup mismatches are configured to throw` FAILS (open resolves, resets instead of throwing). The first-boot test may already pass — that is fine; it pins behavior we must not break. - -- [ ] **Step 3: Implement** - -In `yarn-project/stdlib/src/database-version/version_manager.ts`, track whether a version file was successfully parsed. In `open()`, next to the existing `shouldLogDataReset` declaration (line ~140), add: - -```ts - // Distinguishes "a version file existed and parsed" from first boot / unreadable file: only a parsed - // version file can prove a genuine rollup mismatch, which is what the 'throw' policy protects against. - let versionFileParsed = false; -``` - -Set it right after the successful parse in the `try` block (line ~144): - -```ts - const versionBuf = await this.fileSystem.readFile(this.versionFile); - storedVersion = DatabaseVersion.fromBuffer(versionBuf); - versionFileParsed = true; -``` - -Replace the rollup-mismatch `else` branch (lines ~199-208) with: - -```ts - } else { - if (versionFileParsed && this.schemaVersionMismatchPolicy === 'throw') { - throw new Error( - `Cannot open database at ${this.dataDirectory}: stored rollup address ` + - `${storedVersion.rollupAddress} does not match expected ${this.currentVersion.rollupAddress}`, - ); - } - if (shouldLogDataReset) { - this.log.warn('Rollup address has changed, resetting data directory', { - versionFile: this.versionFile, - storedVersion, - currentVersion: this.currentVersion, - }); - } - needsReset = true; - } -``` - -Also update the `SchemaVersionMismatchPolicy` JSDoc in the same file (the `@param schemaVersionMismatchPolicy` line in the constructor doc, line ~64) to: - -```ts - * @param schemaVersionMismatchPolicy - Whether schema or rollup-address mismatches against an existing version - * file should reset data or throw -``` - -- [ ] **Step 4: Run tests to verify all pass** - -Run: `yarn workspace @aztec/stdlib test src/database-version/version_manager.test.ts` -Expected: PASS, including all pre-existing tests (the default-'reset' rollup test at line ~137 must still pass). - -- [ ] **Step 5: Commit** - -```bash -git add stdlib/src/database-version/version_manager.ts stdlib/src/database-version/version_manager.test.ts -git commit -m "fix(stdlib): honor 'throw' mismatch policy on rollup address change" -``` - ---- - -### Task 3: sqlite-opfs `createStore` selects by identity (core red/green) - -**Files:** -- Modify: `yarn-project/kv-store/src/sqlite-opfs/index.ts` -- Create: `yarn-project/kv-store/src/sqlite-opfs/manage.ts` (pool-directory naming only in this task; list/delete come in Task 4) -- Test: `yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts` - -**Interfaces:** -- Consumes: `storeIdentitySlug`, `effectiveStoreName`, `StoreIdentityMismatchError` (Task 1); `DatabaseVersion` from `@aztec/stdlib/database-version/version`; `AztecSQLiteOPFSStore.open(log, name?, ephemeral?, poolDirectory?, encryptionKey?)`. -- Produces: `createStore(name, config, schemaVersion?, log?)` with unchanged signature but select-by-identity semantics; `storePoolDirectory(effectiveName: string): string` and `OPFS_POOL_DIR_PREFIX = '.aztec-kv-'` from `manage.ts`. - -**Why per-store pool directories:** sqlite-wasm documents that only one SAH-pool VFS instance may use a directory concurrently, and pool capacity (`initialCapacity: 8`) does not auto-grow. One pool directory per store gives every store its own worker-owned pool (no cross-store lock contention, no slot exhaustion from orphaned stores) and makes delete = remove one OPFS directory. - -- [ ] **Step 1: Write the failing test (data survives a rollup switch)** - -Create `yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts`: - -```ts -import { EthAddress } from '@aztec/foundation/eth-address'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; - -import { mockLogger } from '../interfaces/utils.js'; -import { createStore } from './index.js'; - -const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ - dataDirectory: 'test', - dataStoreMapSizeKb: 1024, - rollupAddress, - l1ChainId, -}); - -describe('sqlite-opfs createStore', () => { - it('keeps data intact when switching rollup addresses back and forth', async () => { - const addrA = EthAddress.random(); - const addrB = EthAddress.random(); - - const storeA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); - await storeA.openSingleton('payload').set('data-for-A'); - await storeA.close(); - - const storeB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); - expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); - await storeB.openSingleton('payload').set('data-for-B'); - await storeB.close(); - - const reopenedA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); - expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); - await reopenedA.close(); - - const reopenedB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); - expect(await reopenedB.openSingleton('payload').getAsync()).toEqual('data-for-B'); - await reopenedB.close(); - }); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-red.log 2>&1`, then Grep the log for `roundtrip_test|create_store`. -Expected: the new test FAILS — today both opens hit the same physical store and `initStoreForRollupAndSchemaVersion` wipes it on the address change, so `reopenedA` reads `undefined`. All pre-existing browser tests must still pass. - -- [ ] **Step 3: Implement** - -Create `yarn-project/kv-store/src/sqlite-opfs/manage.ts`: - -```ts -/** Prefix for the per-store OPFS SAH pool directories owned by this package. */ -export const OPFS_POOL_DIR_PREFIX = '.aztec-kv-'; - -/** - * OPFS directory holding a store's SAH pool. One directory per store: the SAH-pool VFS allows only one - * concurrent instance per directory and its capacity does not grow automatically, so sharing a pool across - * stores would make concurrently opened stores contend for locks and orphaned stores exhaust pool slots. - */ -export function storePoolDirectory(effectiveName: string): string { - return `${OPFS_POOL_DIR_PREFIX}${effectiveName}`; -} -``` - -Rewrite `createStore` in `yarn-project/kv-store/src/sqlite-opfs/index.ts` (replacing the `initStoreForRollupAndSchemaVersion` import and call): - -```ts -import { EthAddress } from '@aztec/foundation/eth-address'; -import { type Logger, createLogger } from '@aztec/foundation/log'; -import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; - -import { StoreIdentityMismatchError, effectiveStoreName } from '../store_identity.js'; -import { storePoolDirectory } from './manage.js'; -import { AztecSQLiteOPFSStore } from './store.js'; - -export { AztecSQLiteOPFSStore } from './store.js'; -export { SqliteEncryptionError } from './errors.js'; -export type { SqliteEncryptionErrorCode } from './errors.js'; -export { OPFS_POOL_DIR_PREFIX, storePoolDirectory } from './manage.js'; -export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; -export type { StoreIdentity } from '../store_identity.js'; - -/** - * Opens the persistent store selected by `name` and the identity `(config.l1ChainId, config.rollupAddress, - * schemaVersion)`. A store exists per identity: reopening with the same identity returns the same data, a - * different identity selects a different (possibly fresh) store. Nothing is ever cleared. - */ -export async function createStore( - name: string, - config: DataStoreConfig, - schemaVersion: number | undefined = undefined, - log: Logger = createLogger('kv-store'), -) { - const storeName = effectiveStoreName(name, { - l1ChainId: config.l1ChainId, - rollupAddress: config.rollupAddress, - schemaVersion, - }); - log.info(`Creating ${storeName} SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB`); - const store = await AztecSQLiteOPFSStore.open( - createLogger('kv-store:sqlite-opfs'), - storeName, - false, - storePoolDirectory(storeName), - ); - try { - await assertStoreIdentity(store, storeName, schemaVersion, config.rollupAddress); - } catch (err) { - // The store handle owns a worker and OPFS locks; release them before surfacing the refusal. - await store.close().catch(() => {}); - throw err; - } - return store; -} - -/** - * Belt-and-braces invariant check: the identity is part of the physical store name, so the recorded version - * can only disagree if there is a store-naming bug. Refuses to open on mismatch; never clears. - */ -async function assertStoreIdentity( - store: AztecSQLiteOPFSStore, - storeName: string, - schemaVersion: number | undefined, - rollupAddress: EthAddress | undefined, -): Promise { - const expected = new DatabaseVersion(schemaVersion ?? 0, rollupAddress ?? EthAddress.ZERO); - const singleton = store.openSingleton('dbVersion'); - const stored = await singleton.getAsync(); - if (stored === undefined) { - await singleton.set(expected.toBuffer().toString('utf-8')); - return; - } - let storedVersion: DatabaseVersion; - try { - storedVersion = DatabaseVersion.fromBuffer(Buffer.from(stored, 'utf-8')); - } catch { - throw new StoreIdentityMismatchError(storeName, expected.toString(), stored); - } - if (!storedVersion.equals(expected)) { - throw new StoreIdentityMismatchError(storeName, expected.toString(), storedVersion.toString()); - } -} -``` - -Keep `openTmpStore` and `openEncryptedStore` exactly as they are. - -- [ ] **Step 4: Add the isolation, concurrency, and mismatch tests** - -Append to `create_store.test.ts` inside the `describe` block: - -```ts - it('opens two different stores concurrently in the same tab', async () => { - const addr = EthAddress.random(); - const pxeStore = await createStore('pxe_data', configFor(addr), 1, mockLogger); - const walletStore = await createStore('wallet_data', configFor(addr), 1, mockLogger); - - await pxeStore.openSingleton('k').set('pxe'); - await walletStore.openSingleton('k').set('wallet'); - expect(await pxeStore.openSingleton('k').getAsync()).toEqual('pxe'); - expect(await walletStore.openSingleton('k').getAsync()).toEqual('wallet'); - - await pxeStore.close(); - await walletStore.close(); - }); - - it('separates stores by schema version', async () => { - const addr = EthAddress.random(); - const v1 = await createStore('schema_test', configFor(addr), 1, mockLogger); - await v1.openSingleton('k').set('v1-data'); - await v1.close(); - - const v2 = await createStore('schema_test', configFor(addr), 2, mockLogger); - expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); - await v2.close(); - - const v1Again = await createStore('schema_test', configFor(addr), 1, mockLogger); - expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); - await v1Again.close(); - }); - - it('refuses to open on a recorded-identity mismatch and leaves data untouched', async () => { - const addr = EthAddress.random(); - const store = await createStore('mismatch_test', configFor(addr), 1, mockLogger); - await store.openSingleton('payload').set('precious'); - // Simulate a naming bug by corrupting the recorded identity. - await store.openSingleton('dbVersion').set('garbage'); - await store.close(); - - await expect(createStore('mismatch_test', configFor(addr), 1, mockLogger)).rejects.toThrow( - StoreIdentityMismatchError, - ); - - // The refusal must not have modified the store: read it raw, bypassing the identity check. - const storeName = effectiveStoreName('mismatch_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); - const raw = await AztecSQLiteOPFSStore.open(mockLogger, storeName, false, storePoolDirectory(storeName)); - expect(await raw.openSingleton('payload').getAsync()).toEqual('precious'); - await raw.close(); - }); -``` - -and extend the imports at the top of the test file: - -```ts -import { AztecSQLiteOPFSStore, StoreIdentityMismatchError, createStore, effectiveStoreName } from './index.js'; -import { storePoolDirectory } from './manage.js'; -``` - -(drop the now-redundant `import { createStore } from './index.js';` line). - -- [ ] **Step 5: Run the browser suite to verify green** - -Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-green.log 2>&1`, then Grep for failures. -Expected: ALL browser tests pass, including the four new ones and every pre-existing sqlite-opfs test (`encrypted_store.test.ts` in particular — it uses `AztecSQLiteOPFSStore.open` directly and must be unaffected). - -- [ ] **Step 6: Commit** - -```bash -git add kv-store/src/sqlite-opfs/index.ts kv-store/src/sqlite-opfs/manage.ts \ - kv-store/src/sqlite-opfs/create_store.test.ts -git commit -m "feat(kv-store): sqlite-opfs stores selected by (chain, rollup, schema) identity instead of wiped" -``` - ---- - -### Task 4: sqlite-opfs `listStores` / `deleteStore` - -**Files:** -- Modify: `yarn-project/kv-store/src/sqlite-opfs/manage.ts` -- Modify: `yarn-project/kv-store/src/sqlite-opfs/index.ts` (re-export) -- Test: `yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts` (append) - -**Interfaces:** -- Consumes: `OPFS_POOL_DIR_PREFIX`, `storePoolDirectory` (Task 3), OPFS `navigator.storage.getDirectory()`. -- Produces: `listStores(): Promise` (effective store names, slug included) and `deleteStore(effectiveName: string): Promise`. - -- [ ] **Step 1: Write the failing tests** - -Append to `create_store.test.ts`: - -```ts - it('lists created stores and deletes them', async () => { - const addr = EthAddress.random(); - const store = await createStore('managed_test', configFor(addr), 1, mockLogger); - await store.openSingleton('k').set('v'); - await store.close(); - - const storeName = effectiveStoreName('managed_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); - expect(await listStores()).toContain(storeName); - - await deleteStore(storeName); - expect(await listStores()).not.toContain(storeName); - - // Recreating after deletion starts empty. - const fresh = await createStore('managed_test', configFor(addr), 1, mockLogger); - expect(await fresh.openSingleton('k').getAsync()).toBeUndefined(); - await fresh.close(); - }); - - it('refuses to delete a store that is currently open', async () => { - const addr = EthAddress.random(); - const store = await createStore('locked_test', configFor(addr), 1, mockLogger); - const storeName = effectiveStoreName('locked_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); - - await expect(deleteStore(storeName)).rejects.toThrow(); - - await store.close(); - await deleteStore(storeName); - }); -``` - -and add `listStores`, `deleteStore` to the `./manage.js` import in the test file. - -- [ ] **Step 2: Run to verify the new tests fail** - -Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-manage-red.log 2>&1` -Expected: FAIL — `listStores` / `deleteStore` do not exist. - -- [ ] **Step 3: Implement** - -Append to `yarn-project/kv-store/src/sqlite-opfs/manage.ts`: - -```ts -/** - * Lists the effective names (logical name + identity slug) of every persistent sqlite-opfs store in this - * origin, by enumerating the per-store pool directories. Includes stores created under identities other than - * the current one — that is the point: wallets can surface and clean up data for networks no longer in use. - */ -export async function listStores(): Promise { - const root = await navigator.storage.getDirectory(); - const names: string[] = []; - for await (const [entryName, handle] of root.entries()) { - if (handle.kind === 'directory' && entryName.startsWith(OPFS_POOL_DIR_PREFIX)) { - names.push(entryName.slice(OPFS_POOL_DIR_PREFIX.length)); - } - } - return names; -} - -/** - * Permanently deletes a store by effective name (as returned by {@link listStores}). The store must be closed: - * an open store's SAH pool holds locks on the directory and the removal will reject. - */ -export async function deleteStore(effectiveName: string): Promise { - const root = await navigator.storage.getDirectory(); - await root.removeEntry(storePoolDirectory(effectiveName), { recursive: true }); -} -``` - -Re-export from `yarn-project/kv-store/src/sqlite-opfs/index.ts`: - -```ts -export { OPFS_POOL_DIR_PREFIX, deleteStore, listStores, storePoolDirectory } from './manage.js'; -``` - -If `tsc` reports that `FileSystemDirectoryHandle.entries()` does not exist, add `"dom.asynciterable"` to the `lib` array in `yarn-project/tsconfig.json` (currently `["dom", "esnext", "es2017.object"]`) rather than casting. - -- [ ] **Step 4: Run to verify green** - -Run: `yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-manage-green.log 2>&1` -Expected: ALL pass. If `refuses to delete a store that is currently open` fails because Chromium allows the removal, delete that test case and the "must be closed" sentence stays in the JSDoc as documentation only — do not weaken `deleteStore` itself. - -- [ ] **Step 5: Commit** - -```bash -git add kv-store/src/sqlite-opfs/manage.ts kv-store/src/sqlite-opfs/index.ts \ - kv-store/src/sqlite-opfs/create_store.test.ts -git commit -m "feat(kv-store): list and delete sqlite-opfs stores" -``` - ---- - -### Task 5: lmdb-v2 `partitionByIdentity` - -**Files:** -- Modify: `yarn-project/kv-store/src/lmdb-v2/factory.ts` -- Test: `yarn-project/kv-store/src/lmdb-v2/factory.test.ts` (new) - -**Interfaces:** -- Consumes: `storeIdentitySlug` (Task 1), `DatabaseVersionManager` with the Task 2 fix. -- Produces: `CreateStoreOptions.partitionByIdentity?: boolean`. When true, the store directory is `join(dataDirectory, name, storeIdentitySlug(identity))` and both version-manager policies are forced to `'throw'`. When absent/false, behavior is byte-for-byte today's. - -- [ ] **Step 1: Write the failing tests** - -Create `yarn-project/kv-store/src/lmdb-v2/factory.test.ts` (node project — already covered by the `./src/lmdb-v2/**/*.test.ts` include): - -```ts -import { EthAddress } from '@aztec/foundation/eth-address'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; - -import { mkdtemp, rm } from 'fs/promises'; -import { tmpdir } from 'os'; -import { join } from 'path'; - -import { createStore } from './factory.js'; - -describe('lmdb-v2 createStore', () => { - let dataDirectory: string; - - beforeEach(async () => { - dataDirectory = await mkdtemp(join(tmpdir(), 'factory-test-')); - }); - - afterEach(async () => { - await rm(dataDirectory, { recursive: true, force: true }); - }); - - const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ - dataDirectory, - dataStoreMapSizeKb: 10 * 1024, - rollupAddress, - l1ChainId, - }); - - it('with partitionByIdentity, keeps data intact when switching rollup addresses back and forth', async () => { - const addrA = EthAddress.random(); - const addrB = EthAddress.random(); - const options = { partitionByIdentity: true }; - - const storeA = await createStore('test_store', 1, configFor(addrA), undefined, options); - await storeA.openSingleton('payload').set('data-for-A'); - await storeA.close(); - - const storeB = await createStore('test_store', 1, configFor(addrB), undefined, options); - expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); - await storeB.close(); - - const reopenedA = await createStore('test_store', 1, configFor(addrA), undefined, options); - expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); - await reopenedA.close(); - }); - - it('with partitionByIdentity, separates stores by schema version', async () => { - const addr = EthAddress.random(); - const options = { partitionByIdentity: true }; - - const v1 = await createStore('test_store', 1, configFor(addr), undefined, options); - await v1.openSingleton('k').set('v1-data'); - await v1.close(); - - const v2 = await createStore('test_store', 2, configFor(addr), undefined, options); - expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); - await v2.close(); - - const v1Again = await createStore('test_store', 1, configFor(addr), undefined, options); - expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); - await v1Again.close(); - }); - - it('without the flag, keeps the historical reset-on-rollup-change behavior', async () => { - const addrA = EthAddress.random(); - const addrB = EthAddress.random(); - - const storeA = await createStore('test_store', 1, configFor(addrA)); - await storeA.openSingleton('payload').set('data-for-A'); - await storeA.close(); - - const storeB = await createStore('test_store', 1, configFor(addrB)); - expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); - await storeB.close(); - - // Historical behavior is destructive: the data does not come back. - const reopenedA = await createStore('test_store', 1, configFor(addrA)); - expect(await reopenedA.openSingleton('payload').getAsync()).toBeUndefined(); - await reopenedA.close(); - }); -}); -``` - -- [ ] **Step 2: Run to verify the identity tests fail** - -Run: `yarn workspace @aztec/kv-store test:node src/lmdb-v2/factory.test.ts` -Expected: the two `partitionByIdentity` tests FAIL (the option is unknown/ignored, so the reset wipes `data-for-A`); the historical-behavior test PASSES. - -- [ ] **Step 3: Implement** - -In `yarn-project/kv-store/src/lmdb-v2/factory.ts`, extend the options type: - -```ts -/** Optional versioning hooks for persistent LMDB stores. */ -export type CreateStoreOptions = { - onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; - schemaVersionMismatchPolicy?: SchemaVersionMismatchPolicy; - versionFileReadFailurePolicy?: VersionFileReadFailurePolicy; - /** - * When true, the store directory is keyed by (l1ChainId, rollupAddress, schemaVersion): a different identity - * selects a different directory instead of resetting this one, so no identity change is ever destructive. - * The version file becomes a pure invariant check (any mismatch throws). - */ - partitionByIdentity?: boolean; -}; -``` - -and change the persistent branch of `createStore` (currently `const subDir = join(dataDirectory, name);` and the `DatabaseVersionManager` construction): - -```ts - const rollupAddress = rollupFromConfig ?? EthAddress.ZERO; - const subDir = options.partitionByIdentity - ? join(dataDirectory, name, storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress, schemaVersion })) - : join(dataDirectory, name); - await mkdir(subDir, { recursive: true }); - - const versionManager = new DatabaseVersionManager({ - schemaVersion, - rollupAddress, - dataDirectory: subDir, - onOpen: dbDirectory => - AztecLMDBStoreV2.new(dbDirectory, config.dataStoreMapSizeKb, MAX_READERS, () => Promise.resolve(), bindings), - onUpgrade: options.onUpgrade, - schemaVersionMismatchPolicy: options.partitionByIdentity ? 'throw' : options.schemaVersionMismatchPolicy, - versionFileReadFailurePolicy: options.partitionByIdentity ? 'throw' : options.versionFileReadFailurePolicy, - }); -``` - -with the import added at the top: - -```ts -import { storeIdentitySlug } from '../store_identity.js'; -``` - -(Note the original code computed `rollupAddress` after `mkdir`; it moves above `subDir` because the slug needs it.) - -- [ ] **Step 4: Run to verify green** - -Run: `yarn workspace @aztec/kv-store test:node src/lmdb-v2/factory.test.ts` -Expected: all 3 PASS. Then run the whole node project to guard against regressions: `yarn workspace @aztec/kv-store test:node`. - -- [ ] **Step 5: Commit** - -```bash -git add kv-store/src/lmdb-v2/factory.ts kv-store/src/lmdb-v2/factory.test.ts -git commit -m "feat(kv-store): opt-in identity-partitioned lmdb-v2 stores" -``` - ---- - -### Task 6: Wire up the Node.js PXE and node embedded wallet - -**Files:** -- Modify: `yarn-project/pxe/src/entrypoints/server/utils.ts:49-54` -- Modify: `yarn-project/wallets/src/embedded/entrypoints/node.ts:31,40,71-87` - -**Interfaces:** -- Consumes: `createStore(name, schemaVersion, config, bindings, { partitionByIdentity: true })` from `@aztec/kv-store/lmdb-v2` (Task 5). -- Produces: no new interfaces; behavior change only. - -- [ ] **Step 1: Server PXE passes the flag** - -In `yarn-project/pxe/src/entrypoints/server/utils.ts`, the store creation becomes: - -```ts - options.store = await createStore( - 'pxe_data', - PXE_DATA_SCHEMA_VERSION, - configWithContracts, - storeLogger.getBindings(), - { partitionByIdentity: true }, - ); -``` - -(`configWithContracts` already carries `l1ChainId` and `rollupAddress` from `getNodeInfo()`.) - -- [ ] **Step 2: Node embedded wallet drops hand-rolled suffixes** - -In `yarn-project/wallets/src/embedded/entrypoints/node.ts`, keep the existing `aztecNode` variable and replace the -`getL1ContractAddresses` call (line ~31) so the two lines read: - -```ts - const aztecNode = typeof nodeOrUrl === 'string' ? createAztecNodeClient(nodeOrUrl) : nodeOrUrl; - const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); -``` - -Change the PXE config default directory (line ~40) from `` dataDirectory: `pxe_data_${l1Contracts.rollupAddress}` `` to: - -```ts - dataDirectory: 'aztec-wallet-data', -``` - -Change the walletDB store creation (lines ~68-87) to: - -```ts - const walletDBStore = - options.walletDb?.store ?? - (options.ephemeral - ? await openTmpStore( - 'wallet_data', - true, - undefined, - undefined, - rootLogger.createChild('wallet:data').getBindings(), - ) - : await createStore( - 'wallet_data', - 1, - { - dataDirectory: pxeConfig.dataDirectory ?? 'aztec-wallet-data', - dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, - rollupAddress: l1ContractAddresses.rollupAddress, - l1ChainId, - }, - rootLogger.createChild('wallet:data').getBindings(), - { partitionByIdentity: true }, - )); -``` - -There are no other `l1Contracts` references in the file after these edits (verify with Grep; the pre-change references are lines 31, 40, 72, 82, 84). - -- [ ] **Step 3: Build and run affected tests** - -```bash -yarn build -yarn workspace @aztec/pxe test -``` - -Expected: build clean; pxe unit tests pass (they mock `getNodeInfo` already). If any wallets tests exist that construct `NodeEmbeddedWallet`, run `yarn workspace @aztec/wallets test` as well and fix fallout — the likely failure mode is a missing `getNodeInfo` mock where only `getL1ContractAddresses` was mocked; extend the mock with `getNodeInfo` returning `{ l1ChainId: 31337, l1ContractAddresses: { rollupAddress }, rollupVersion: 1 }` shaped like `stdlib`'s `NodeInfo`. - -- [ ] **Step 4: Commit** - -```bash -git add pxe/src/entrypoints/server/utils.ts wallets/src/embedded/entrypoints/node.ts -git commit -m "feat(pxe): node PXE and embedded wallet stores partitioned by identity" -``` - ---- - -### Task 7: Wire up the browser PXE and browser embedded wallet - -**Files:** -- Modify: `yarn-project/pxe/src/entrypoints/client/bundle/utils.ts:34-43` -- Modify: `yarn-project/pxe/src/entrypoints/client/lazy/utils.ts` (same edit; the files differ only in import specifiers) -- Modify: `yarn-project/wallets/src/embedded/entrypoints/browser.ts:30,39,71-80` - -**Interfaces:** -- Consumes: sqlite-opfs `createStore` (Task 3) — no signature change, it now reads `config.l1ChainId`. -- Produces: no new interfaces; behavior change only. - -- [ ] **Step 1: Browser createPXE fetches chain identity from the node** - -In both `bundle/utils.ts` and `lazy/utils.ts`, replace: - -```ts - const l1ContractAddresses = await aztecNode.getL1ContractAddresses(); - const configWithContracts = { - ...config, - ...l1ContractAddresses, - } as PXEConfig; -``` - -with: - -```ts - const { l1ChainId, l1ContractAddresses, rollupVersion } = await aztecNode.getNodeInfo(); - const configWithContracts = { - ...config, - ...l1ContractAddresses, - l1ChainId, - rollupVersion, - } as PXEConfig; -``` - -- [ ] **Step 2: Browser embedded wallet drops hand-rolled suffixes** - -In `yarn-project/wallets/src/embedded/entrypoints/browser.ts`: - -Replace line ~30: - -```ts - const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); -``` - -Replace the PXE config default directory (line ~39) with a plain marker (sqlite-opfs uses `dataDirectory` only to word its log line): - -```ts - dataDirectory: 'pxe_data', -``` - -Replace the walletDB store creation (lines ~67-80): - -```ts - const walletDBStore = - options.walletDb?.store ?? - (options.ephemeral - ? await openTmpStore(true) - : await createStore( - 'wallet_data', - { - dataDirectory: 'wallet_data', - dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, - rollupAddress: l1ContractAddresses.rollupAddress, - l1ChainId, - }, - 1, - rootLogger.createChild('wallet:data'), - )); -``` - -Update any remaining `l1Contracts` references in the file (pre-change: lines 30, 39, 74, 76) to `l1ContractAddresses`. - -- [ ] **Step 3: Build, lint, and sweep for stragglers** - -```bash -yarn build -``` - -Then Grep the repo for `wallet_data_${` and `pxe_data_${` under `yarn-project` (excluding `dest/` and `node_modules/`) — expected: zero hits. Also Grep for remaining production imports of `initStoreForRollupAndSchemaVersion`: expected hits only in `kv-store/src/utils.ts` itself, `kv-store/src/lmdb/index.ts`, and `kv-store/src/deprecated/indexeddb/` (test-only backends, deliberately untouched). - -- [ ] **Step 4: Commit** - -```bash -git add pxe/src/entrypoints/client/bundle/utils.ts pxe/src/entrypoints/client/lazy/utils.ts \ - wallets/src/embedded/entrypoints/browser.ts -git commit -m "feat(pxe): browser PXE and embedded wallet stores partitioned by identity" -``` - ---- - -### Task 8: Full verification, changelog, docs - -**Files:** -- Modify: changelog files as directed by the `updating-changelog` skill -- No other code changes expected - -- [ ] **Step 1: Full build + format + lint** - -```bash -yarn build -yarn format -yarn lint kv-store stdlib pxe wallets -``` - -Expected: all clean. Commit any formatter-only diffs with `chore: format`. - -- [ ] **Step 2: Full test pass for touched packages** - -```bash -yarn workspace @aztec/kv-store test:node -yarn workspace @aztec/kv-store test:browser > /tmp/claude-30077/-mnt-user-data-martin-aztec-packages-2/f4cd57d5-2cdb-4c77-8e2b-143b4559fd6e/scratchpad/browser-final.log 2>&1 -yarn workspace @aztec/stdlib test src/database-version -yarn workspace @aztec/pxe test -yarn workspace @aztec/wallets test -``` - -Expected: all pass. Check `yarn-project/pxe/src/storage/backwards_compatibility_tests/` specifically — those tests open stores directly via tmp-store helpers and should be unaffected; if one constructs a store through `createStore`, it now gets an identity-slugged location, which is fine as long as the test is self-contained. - -- [ ] **Step 3: Changelog** - -Invoke the `updating-changelog` skill to document the behavior change: -- PXE/wallet stores are no longer wiped when the rollup address or schema version changes; a store now exists per `(l1ChainId, rollupAddress, schemaVersion)` and switching selects the matching store. -- First start after upgrading begins with a fresh (empty) store; previous data remains on disk under the old name and is not deleted. -- New `listStores()` / `deleteStore()` utilities in `@aztec/kv-store/sqlite-opfs` for wallet-driven cleanup. - -- [ ] **Step 4: Final review and commit** - -```bash -git log --oneline origin/merge-train/spartan..HEAD -git diff origin/merge-train/spartan...HEAD --stat -``` - -Confirm every commit is conventional and scoped. Commit the changelog updates: - -```bash -git add -git commit -m "docs: changelog for identity-partitioned PXE stores" -``` From f7dae326efc631f72e917e2154ae224579e0f4fd Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 14:37:39 +0000 Subject: [PATCH 15/48] feat(pxe): identity-partitioned store helper for node-side stores --- .../pxe/src/entrypoints/server/index.ts | 1 + .../pxe/src/entrypoints/server/store.test.ts | 88 +++++++++++++++++++ .../pxe/src/entrypoints/server/store.ts | 45 ++++++++++ 3 files changed, 134 insertions(+) create mode 100644 yarn-project/pxe/src/entrypoints/server/store.test.ts create mode 100644 yarn-project/pxe/src/entrypoints/server/store.ts diff --git a/yarn-project/pxe/src/entrypoints/server/index.ts b/yarn-project/pxe/src/entrypoints/server/index.ts index c4a96d12cd9f..aaefc18a5a8b 100644 --- a/yarn-project/pxe/src/entrypoints/server/index.ts +++ b/yarn-project/pxe/src/entrypoints/server/index.ts @@ -6,6 +6,7 @@ export * from '../../config/index.js'; export * from '../../error_enriching.js'; export * from '../../storage/index.js'; export * from './utils.js'; +export * from './store.js'; export { NoteService } from '../../notes/note_service.js'; export { ORACLE_VERSION_MAJOR, ORACLE_VERSION_MINOR } from '../../oracle_version.js'; export { type PXECreationOptions } from '../pxe_creation_options.js'; diff --git a/yarn-project/pxe/src/entrypoints/server/store.test.ts b/yarn-project/pxe/src/entrypoints/server/store.test.ts new file mode 100644 index 000000000000..4c475c33f824 --- /dev/null +++ b/yarn-project/pxe/src/entrypoints/server/store.test.ts @@ -0,0 +1,88 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; + +import { mkdtemp, rm, stat } from 'fs/promises'; +import { tmpdir } from 'os'; +import { join } from 'path'; + +import { openStoreForIdentity } from './store.js'; + +describe('openStoreForIdentity', () => { + let dataDirectory: string; + + beforeEach(async () => { + dataDirectory = await mkdtemp(join(tmpdir(), 'store-identity-test-')); + }); + + afterEach(async () => { + await rm(dataDirectory, { recursive: true, force: true }); + }); + + const configFor = (rollupAddress?: EthAddress, l1ChainId = 31337) => ({ + dataDirectory, + dataStoreMapSizeKb: 10 * 1024, + rollupAddress, + l1ChainId, + }); + + it('keeps data intact when switching rollup addresses back and forth', async () => { + const addrA = EthAddress.random(); + const addrB = EthAddress.random(); + + const storeA = await openStoreForIdentity('test_store', 1, configFor(addrA)); + await storeA.openSingleton('payload').set('data-for-A'); + await storeA.close(); + + const storeB = await openStoreForIdentity('test_store', 1, configFor(addrB)); + expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); + await storeB.close(); + + const reopenedA = await openStoreForIdentity('test_store', 1, configFor(addrA)); + expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); + await reopenedA.close(); + }); + + it('separates stores by schema version', async () => { + const addr = EthAddress.random(); + + const v1 = await openStoreForIdentity('test_store', 1, configFor(addr)); + await v1.openSingleton('k').set('v1-data'); + await v1.close(); + + const v2 = await openStoreForIdentity('test_store', 2, configFor(addr)); + expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); + await v2.close(); + + const v1Again = await openStoreForIdentity('test_store', 1, configFor(addr)); + expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); + await v1Again.close(); + }); + + it('opens and persists with a zero identity (no rollup, no chain id)', async () => { + const config = { dataDirectory, dataStoreMapSizeKb: 10 * 1024 }; + + const store = await openStoreForIdentity('test_store', 1, config); + await store.openSingleton('payload').set('zero-identity-data'); + await store.close(); + + const reopened = await openStoreForIdentity('test_store', 1, config); + expect(await reopened.openSingleton('payload').getAsync()).toEqual('zero-identity-data'); + await reopened.close(); + }); + + it('places stores under a sibling -stores directory, not nested in ', async () => { + const store = await openStoreForIdentity('test_store', 1, configFor(EthAddress.random())); + await store.close(); + + await expect(stat(join(dataDirectory, 'test_store-stores'))).resolves.toBeDefined(); + await expect(stat(join(dataDirectory, 'test_store'))).rejects.toMatchObject({ code: 'ENOENT' }); + }); + + it('falls back to an ephemeral tmp store when no data directory is configured', async () => { + const store = await openStoreForIdentity('test_store', 1, { dataStoreMapSizeKb: 10 * 1024 }); + await store.openSingleton('payload').set('tmp-data'); + expect(await store.openSingleton('payload').getAsync()).toEqual('tmp-data'); + await store.close(); + + await expect(stat(join(dataDirectory, 'test_store-stores'))).rejects.toMatchObject({ code: 'ENOENT' }); + }); +}); diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts new file mode 100644 index 000000000000..2e87fcf733a5 --- /dev/null +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -0,0 +1,45 @@ +import type { EthAddress } from '@aztec/foundation/eth-address'; +import { type LoggerBindings, createLogger } from '@aztec/foundation/log'; +import { type AztecLMDBStoreV2, openStoreAt, openTmpStore, storeIdentitySlug } from '@aztec/kv-store/lmdb-v2'; + +import { mkdir } from 'fs/promises'; +import { join } from 'path'; + +/** Location and identity inputs for opening an identity-partitioned PXE-side store. */ +export type IdentityStoreConfig = { + dataDirectory?: string; + dataStoreMapSizeKb?: number; + l1ChainId?: number; + rollupAddress?: EthAddress; +}; + +/** + * Opens the persistent LMDB store selected by `name` and the identity `(l1ChainId, rollupAddress, schemaVersion)`. + * A store exists per identity: reopening with the same identity returns the same data, a different identity selects + * a different (possibly fresh) store. Nothing is ever cleared, and no version marker is kept — the directory name is + * the identity. Falls back to an ephemeral tmp store when no data directory is configured. + * + * Stores live under `/-stores/`, a sibling of the legacy `/` + * directory: older binaries reset the legacy directory on a rollup mismatch, so per-identity stores must not nest + * inside it. + */ +export async function openStoreForIdentity( + name: string, + schemaVersion: number, + config: IdentityStoreConfig, + bindings?: LoggerBindings, +): Promise { + if (!config.dataDirectory) { + return openTmpStore(name, true, config.dataStoreMapSizeKb, undefined, bindings); + } + const subDir = join( + config.dataDirectory, + `${name}-stores`, + storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress: config.rollupAddress, schemaVersion }), + ); + await mkdir(subDir, { recursive: true }); + createLogger(`pxe:data:${name}`, bindings).info( + `Opening ${name} data store at directory ${subDir} with map size ${config.dataStoreMapSizeKb} KB (LMDB v2)`, + ); + return openStoreAt(subDir, config.dataStoreMapSizeKb, undefined, bindings); +} From 4935622b5a214594c1194b89bbd76480dedbdfc7 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 14:45:13 +0000 Subject: [PATCH 16/48] refactor(pxe): open pxe_data and wallet_data via the identity store helper --- yarn-project/pxe/src/entrypoints/server/utils.ts | 5 ++--- yarn-project/wallets/src/embedded/entrypoints/node.ts | 7 +++---- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/server/utils.ts b/yarn-project/pxe/src/entrypoints/server/utils.ts index 0138f2101ddf..84df52ccde35 100644 --- a/yarn-project/pxe/src/entrypoints/server/utils.ts +++ b/yarn-project/pxe/src/entrypoints/server/utils.ts @@ -1,7 +1,6 @@ import { BBBundlePrivateKernelProver } from '@aztec/bb-prover/client/bundle'; import type { L1ContractAddresses } from '@aztec/ethereum/l1-contract-addresses'; import { createLogger } from '@aztec/foundation/log'; -import { createStore } from '@aztec/kv-store/lmdb-v2'; import { BundledProtocolContractsProvider } from '@aztec/protocol-contracts/providers/bundle'; import { WASMSimulator } from '@aztec/simulator/client'; import { MemoryCircuitRecorder, SimulatorRecorderWrapper } from '@aztec/simulator/server'; @@ -15,6 +14,7 @@ import type { PXEConfig } from '../../config/index.js'; import { PXE } from '../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../storage/index.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../pxe_creation_options.js'; +import { openStoreForIdentity } from './store.js'; type PXEConfigWithoutDefaults = Omit< PXEConfig, @@ -46,12 +46,11 @@ export async function createPXE( if (!options.store) { const storeLogger = loggers.store ?? createLogger('pxe:data:lmdb', { actor }); - options.store = await createStore( + options.store = await openStoreForIdentity( 'pxe_data', PXE_DATA_SCHEMA_VERSION, configWithContracts, storeLogger.getBindings(), - { partitionByIdentity: true }, ); } const proverLogger = loggers.prover ?? createLogger('pxe:bb:native', { actor }); diff --git a/yarn-project/wallets/src/embedded/entrypoints/node.ts b/yarn-project/wallets/src/embedded/entrypoints/node.ts index 583567d1f214..af1f98e0ebec 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/node.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/node.ts @@ -1,8 +1,8 @@ import { createAztecNodeClient } from '@aztec/aztec.js/node'; import { type Logger, createLogger } from '@aztec/foundation/log'; -import { createStore, openTmpStore } from '@aztec/kv-store/lmdb-v2'; +import { openTmpStore } from '@aztec/kv-store/lmdb-v2'; import { type PXEConfig, getPXEConfig } from '@aztec/pxe/config'; -import { type PXE, type PXECreationOptions, createPXE } from '@aztec/pxe/server'; +import { type PXE, type PXECreationOptions, createPXE, openStoreForIdentity } from '@aztec/pxe/server'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry'; import { getStandardHandshakeRegistry } from '@aztec/standard-contracts/handshake-registry'; import { getStandardMultiCallEntrypoint } from '@aztec/standard-contracts/multi-call-entrypoint'; @@ -75,7 +75,7 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { undefined, rootLogger.createChild('wallet:data').getBindings(), ) - : await createStore( + : await openStoreForIdentity( 'wallet_data', 1, { @@ -85,7 +85,6 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { l1ChainId, }, rootLogger.createChild('wallet:data').getBindings(), - { partitionByIdentity: true }, )); const walletDB = new WalletDB(walletDBStore, rootLogger.createChild('wallet:db').info); From d39c9851e3fd915ae846ed69ef497d06d190f25a Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 14:52:53 +0000 Subject: [PATCH 17/48] refactor(kv-store): contain store-identity logic to sqlite-opfs and pxe callers --- .../kv-store/src/lmdb-v2/factory.test.ts | 103 ------------------ yarn-project/kv-store/src/lmdb-v2/factory.ts | 25 +---- .../src/sqlite-opfs/create_store.test.ts | 3 +- .../kv-store/src/sqlite-opfs/index.ts | 2 +- yarn-project/kv-store/tsconfig.json | 3 +- .../stdlib/src/database-version/README.md | 17 +-- .../database-version/version_manager.test.ts | 40 ------- .../src/database-version/version_manager.ts | 37 +------ .../stdlib/src/ha-signing/local_config.ts | 1 - yarn-project/stdlib/src/kv-store/config.ts | 6 - yarn-project/tsconfig.json | 2 +- yarn-project/validator-client/src/config.ts | 6 + 12 files changed, 24 insertions(+), 221 deletions(-) delete mode 100644 yarn-project/kv-store/src/lmdb-v2/factory.test.ts diff --git a/yarn-project/kv-store/src/lmdb-v2/factory.test.ts b/yarn-project/kv-store/src/lmdb-v2/factory.test.ts deleted file mode 100644 index 1f727f96951a..000000000000 --- a/yarn-project/kv-store/src/lmdb-v2/factory.test.ts +++ /dev/null @@ -1,103 +0,0 @@ -import { EthAddress } from '@aztec/foundation/eth-address'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; - -import { mkdtemp, rm, stat } from 'fs/promises'; -import { tmpdir } from 'os'; -import { join } from 'path'; - -import { createStore } from './factory.js'; - -describe('lmdb-v2 createStore', () => { - let dataDirectory: string; - - beforeEach(async () => { - dataDirectory = await mkdtemp(join(tmpdir(), 'factory-test-')); - }); - - afterEach(async () => { - await rm(dataDirectory, { recursive: true, force: true }); - }); - - const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ - dataDirectory, - dataStoreMapSizeKb: 10 * 1024, - rollupAddress, - l1ChainId, - }); - - it('with partitionByIdentity, keeps data intact when switching rollup addresses back and forth', async () => { - const addrA = EthAddress.random(); - const addrB = EthAddress.random(); - const options = { partitionByIdentity: true }; - - const storeA = await createStore('test_store', 1, configFor(addrA), undefined, options); - await storeA.openSingleton('payload').set('data-for-A'); - await storeA.close(); - - const storeB = await createStore('test_store', 1, configFor(addrB), undefined, options); - expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); - await storeB.close(); - - const reopenedA = await createStore('test_store', 1, configFor(addrA), undefined, options); - expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); - await reopenedA.close(); - }); - - it('with partitionByIdentity, separates stores by schema version', async () => { - const addr = EthAddress.random(); - const options = { partitionByIdentity: true }; - - const v1 = await createStore('test_store', 1, configFor(addr), undefined, options); - await v1.openSingleton('k').set('v1-data'); - await v1.close(); - - const v2 = await createStore('test_store', 2, configFor(addr), undefined, options); - expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); - await v2.close(); - - const v1Again = await createStore('test_store', 1, configFor(addr), undefined, options); - expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); - await v1Again.close(); - }); - - it('with partitionByIdentity, opens and persists on first boot with a zero identity (no rollup, no chain id)', async () => { - const config: DataStoreConfig = { dataDirectory, dataStoreMapSizeKb: 10 * 1024 }; - const options = { partitionByIdentity: true }; - - const store = await createStore('test_store', 1, config, undefined, options); - await store.openSingleton('payload').set('zero-identity-data'); - await store.close(); - - const reopened = await createStore('test_store', 1, config, undefined, options); - expect(await reopened.openSingleton('payload').getAsync()).toEqual('zero-identity-data'); - await reopened.close(); - }); - - it('with partitionByIdentity, places stores under a sibling -stores directory, not nested in ', async () => { - const store = await createStore('test_store', 1, configFor(EthAddress.random()), undefined, { - partitionByIdentity: true, - }); - await store.close(); - - await expect(stat(join(dataDirectory, 'test_store-stores'))).resolves.toBeDefined(); - await expect(stat(join(dataDirectory, 'test_store'))).rejects.toMatchObject({ code: 'ENOENT' }); - }); - - it('without the flag, keeps the historical reset-on-rollup-change behavior', async () => { - const addrA = EthAddress.random(); - const addrB = EthAddress.random(); - - const storeA = await createStore('test_store', 1, configFor(addrA)); - await storeA.openSingleton('payload').set('data-for-A'); - await storeA.close(); - - const storeB = await createStore('test_store', 1, configFor(addrB)); - expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); - await storeB.close(); - - // Historical behavior is destructive: the data does not come back. - const reopenedA = await createStore('test_store', 1, configFor(addrA)); - expect(await reopenedA.openSingleton('payload').getAsync()).toBeUndefined(); - await reopenedA.close(); - }); -}); diff --git a/yarn-project/kv-store/src/lmdb-v2/factory.ts b/yarn-project/kv-store/src/lmdb-v2/factory.ts index 9727d5807860..d4396ef7dd48 100644 --- a/yarn-project/kv-store/src/lmdb-v2/factory.ts +++ b/yarn-project/kv-store/src/lmdb-v2/factory.ts @@ -11,7 +11,6 @@ import { copyFile, mkdir, mkdtemp, rm } from 'fs/promises'; import { tmpdir } from 'os'; import { join } from 'path'; -import { storeIdentitySlug } from '../store_identity.js'; import { AztecLMDBStoreV2 } from './store.js'; const MAX_READERS = 16; @@ -21,14 +20,6 @@ export type CreateStoreOptions = { onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; schemaVersionMismatchPolicy?: SchemaVersionMismatchPolicy; versionFileReadFailurePolicy?: VersionFileReadFailurePolicy; - /** - * When true, the store directory is keyed by (l1ChainId, rollupAddress, schemaVersion): a different identity - * selects a different directory instead of resetting this one, so no identity change is ever destructive. - * Per-identity stores live under a sibling `-stores` directory (not nested inside the legacy `` - * env dir), so an old binary's reset of the legacy dir cannot wipe them. The version file becomes a pure - * invariant check (any mismatch throws). - */ - partitionByIdentity?: boolean; }; export async function createStore( @@ -44,16 +35,11 @@ export async function createStore( let store: AztecLMDBStoreV2; if (typeof dataDirectory !== 'undefined') { // Get rollup address from contracts config, or use zero address - const rollupAddress = rollupFromConfig ?? EthAddress.ZERO; - const subDir = options.partitionByIdentity - ? join( - dataDirectory, - `${name}-stores`, - storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress, schemaVersion }), - ) - : join(dataDirectory, name); + const subDir = join(dataDirectory, name); await mkdir(subDir, { recursive: true }); + const rollupAddress = rollupFromConfig ?? EthAddress.ZERO; + // Create a version manager const versionManager = new DatabaseVersionManager({ schemaVersion, @@ -62,9 +48,8 @@ export async function createStore( onOpen: dbDirectory => AztecLMDBStoreV2.new(dbDirectory, config.dataStoreMapSizeKb, MAX_READERS, () => Promise.resolve(), bindings), onUpgrade: options.onUpgrade, - schemaVersionMismatchPolicy: options.partitionByIdentity ? 'throw' : options.schemaVersionMismatchPolicy, - rollupAddressMismatchPolicy: options.partitionByIdentity ? 'throw' : undefined, - versionFileReadFailurePolicy: options.partitionByIdentity ? 'throw' : options.versionFileReadFailurePolicy, + schemaVersionMismatchPolicy: options.schemaVersionMismatchPolicy, + versionFileReadFailurePolicy: options.versionFileReadFailurePolicy, }); log.info( diff --git a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts index eb6ae0982d27..45a3085df8cf 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts @@ -1,11 +1,10 @@ import { EthAddress } from '@aztec/foundation/eth-address'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; import { mockLogger } from '../interfaces/utils.js'; import { AztecSQLiteOPFSStore, StoreIdentityMismatchError, createStore, effectiveStoreName } from './index.js'; import { deleteStore, listStores, storePoolDirectory } from './manage.js'; -const configFor = (rollupAddress: EthAddress, l1ChainId = 31337): DataStoreConfig => ({ +const configFor = (rollupAddress: EthAddress, l1ChainId = 31337) => ({ dataDirectory: 'test', dataStoreMapSizeKb: 1024, rollupAddress, diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index a30a936fe645..c744c72f7fd0 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -21,7 +21,7 @@ export type { StoreIdentity } from '../store_identity.js'; */ export async function createStore( name: string, - config: DataStoreConfig, + config: DataStoreConfig & { l1ChainId?: number }, schemaVersion: number | undefined = undefined, log: Logger = createLogger('kv-store'), ) { diff --git a/yarn-project/kv-store/tsconfig.json b/yarn-project/kv-store/tsconfig.json index b9f4050a2816..673519ee091f 100644 --- a/yarn-project/kv-store/tsconfig.json +++ b/yarn-project/kv-store/tsconfig.json @@ -5,7 +5,8 @@ "rootDir": "src", "tsBuildInfoFile": ".tsbuildinfo", "allowJs": true, - "checkJs": false + "checkJs": false, + "lib": ["dom", "dom.asynciterable", "esnext", "es2017.object"] }, "references": [ { diff --git a/yarn-project/stdlib/src/database-version/README.md b/yarn-project/stdlib/src/database-version/README.md index a10a4230eb87..ea2b739377c7 100644 --- a/yarn-project/stdlib/src/database-version/README.md +++ b/yarn-project/stdlib/src/database-version/README.md @@ -53,16 +53,11 @@ const dataDir = versionManager.getDataDirectory(); ## Automatic Reset Conditions -The database is reset in the following conditions: +The database will be reset in the following conditions: -1. First boot: no version file exists yet. -2. The version file exists but cannot be read or parsed (permissions, IO error, truncation), unless - `versionFileReadFailurePolicy: 'throw'` is set — then it refuses to open and leaves data untouched. -3. The stored schema version is incompatible with the current one and no upgrade path applies (no upgrade callback, - or the callback throws), unless `schemaVersionMismatchPolicy: 'throw'` is set — then it refuses to open. -4. The stored rollup address differs from the current one, unless `rollupAddressMismatchPolicy: 'throw'` is set — - then it refuses to open. +1. No version information exists (first run) +2. Rollup address has changed +3. Version has changed and no upgrade callback is provided +4. Upgrade callback throws an error -The `'throw'` policies only fire when a version file was successfully parsed: first boot always proceeds and never -throws. When a reset occurs, the data directory is deleted and recreated, and the reset callback is called to -initialize a fresh database. +When a reset occurs, the data directory is deleted and recreated, and the reset callback is called to initialize a fresh database. diff --git a/yarn-project/stdlib/src/database-version/version_manager.test.ts b/yarn-project/stdlib/src/database-version/version_manager.test.ts index 4870c81308d3..94e561afb238 100644 --- a/yarn-project/stdlib/src/database-version/version_manager.test.ts +++ b/yarn-project/stdlib/src/database-version/version_manager.test.ts @@ -140,46 +140,6 @@ describe('VersionManager', () => { expect(wasReset).toEqual(true); expect(upgradeSpy).not.toHaveBeenCalled(); }); - - it('unless rollup mismatches are configured to throw', async () => { - fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); - versionManager = createManager({ rollupAddressMismatchPolicy: 'throw' }); - expectVersionFileWritten = false; - - await expect(versionManager.open()).rejects.toThrow(/stored rollup address/); - expect(fs.rm).not.toHaveBeenCalled(); - expect(openSpy).not.toHaveBeenCalled(); - }); - - it('resets on a rollup mismatch when only the schema policy is throw, preserving HA-signer behavior', async () => { - fs.readFile.mockResolvedValueOnce(new DatabaseVersion(currentVersion, EthAddress.random()).toBuffer()); - versionManager = createManager({ schemaVersionMismatchPolicy: 'throw' }); - const [_, wasReset] = await versionManager.open(); - expect(wasReset).toEqual(true); - expect(openSpy).toHaveBeenCalled(); - expect(upgradeSpy).not.toHaveBeenCalled(); - }); - - it("still opens fresh on first boot when policy is 'throw' (no version file, different rollup)", async () => { - fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); - versionManager = createManager({ schemaVersionMismatchPolicy: 'throw', onUpgrade: undefined }); - const [_, wasReset] = await versionManager.open(); - expect(wasReset).toEqual(true); - expect(openSpy).toHaveBeenCalled(); - }); - - it("opens fresh on first boot with a zero rollup address even when schema policy is 'throw'", async () => { - fs.readFile.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' })); - rollupAddress = EthAddress.ZERO; - versionManager = createManager({ - rollupAddress: EthAddress.ZERO, - schemaVersionMismatchPolicy: 'throw', - onUpgrade: undefined, - }); - const [_, wasReset] = await versionManager.open(); - expect(wasReset).toEqual(true); - expect(openSpy).toHaveBeenCalled(); - }); }); describe('version file read-failure policy', () => { diff --git a/yarn-project/stdlib/src/database-version/version_manager.ts b/yarn-project/stdlib/src/database-version/version_manager.ts index 2f40f66b5119..1a2d0c519018 100644 --- a/yarn-project/stdlib/src/database-version/version_manager.ts +++ b/yarn-project/stdlib/src/database-version/version_manager.ts @@ -9,24 +9,8 @@ import { DatabaseVersion } from './database_version.js'; export type DatabaseVersionManagerFs = Pick; export const DATABASE_VERSION_FILE_NAME = 'db_version'; - -/** - * How to react when a parsed version file records a schema version incompatible with the current one and no - * upgrade path applies. `'reset'` (default) wipes and recreates the data directory. `'throw'` refuses to open, - * leaving data untouched — for stores that must fail closed rather than be silently reset. Only a successfully - * parsed version file can trigger the throw; first boot (no version file) always proceeds. - */ export type SchemaVersionMismatchPolicy = 'reset' | 'throw'; -/** - * How to react when a parsed version file records a different rollup address than the current one. `'reset'` - * (default) keeps the historical behavior of resetting the data directory when the stored rollup address differs. - * `'throw'` refuses to open instead, for stores whose location already encodes the rollup identity (a mismatch - * then indicates a bug, and data must not be destroyed). Only a successfully parsed version file can trigger the - * throw; first boot (no version file) always proceeds. - */ -export type RollupAddressMismatchPolicy = 'reset' | 'throw'; - /** * How to react when the version file exists but cannot be read (permissions, IO error, truncation). * `'reset'` (default) treats the store as unversioned and lets the reset path run — safe for stores @@ -43,7 +27,6 @@ export type DatabaseVersionManagerOptions = { onOpen: (dataDir: string) => Promise; onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; schemaVersionMismatchPolicy?: SchemaVersionMismatchPolicy; - rollupAddressMismatchPolicy?: RollupAddressMismatchPolicy; versionFileReadFailurePolicy?: VersionFileReadFailurePolicy; fileSystem?: DatabaseVersionManagerFs; log?: Logger; @@ -64,7 +47,6 @@ export class DatabaseVersionManager { private onOpen: (dataDir: string) => Promise; private onUpgrade?: (dataDir: string, currentVersion: number, latestVersion: number) => Promise; private schemaVersionMismatchPolicy: SchemaVersionMismatchPolicy; - private rollupAddressMismatchPolicy: RollupAddressMismatchPolicy; private versionFileReadFailurePolicy: VersionFileReadFailurePolicy; private fileSystem: DatabaseVersionManagerFs; private log: Logger; @@ -79,10 +61,7 @@ export class DatabaseVersionManager { * @param onUpgrade - An optional callback to upgrade the database before opening. If not provided it will reset the * database. Must be idempotent: since the version marker is written only after a successful open, a crash after * onUpgrade but before the marker is written re-runs onUpgrade on the next start. - * @param schemaVersionMismatchPolicy - Whether an incompatible schema version in a parsed version file should - * reset data or throw - * @param rollupAddressMismatchPolicy - Whether a different rollup address in a parsed version file should reset - * data or throw + * @param schemaVersionMismatchPolicy - Whether schema mismatches should reset data or throw * @param versionFileReadFailurePolicy - Whether an unreadable (non-missing) version file should reset data or throw * @param fileSystem - An interface to access the filesystem * @param log - Optional custom logger @@ -95,7 +74,6 @@ export class DatabaseVersionManager { onOpen, onUpgrade, schemaVersionMismatchPolicy = 'reset', - rollupAddressMismatchPolicy = 'reset', versionFileReadFailurePolicy = 'reset', fileSystem = fs, log = createLogger(`foundation:version-manager`), @@ -111,7 +89,6 @@ export class DatabaseVersionManager { this.onOpen = onOpen; this.onUpgrade = onUpgrade; this.schemaVersionMismatchPolicy = schemaVersionMismatchPolicy; - this.rollupAddressMismatchPolicy = rollupAddressMismatchPolicy; this.versionFileReadFailurePolicy = versionFileReadFailurePolicy; this.fileSystem = fileSystem; this.log = log; @@ -161,14 +138,10 @@ export class DatabaseVersionManager { let storedVersion: DatabaseVersion; // a flag to suppress logs about 'resetting the data dir' when starting from an empty state let shouldLogDataReset = true; - // Distinguishes "a version file existed and parsed" from first boot / unreadable file: only a parsed - // version file can prove a genuine rollup mismatch, which is what the 'throw' policy protects against. - let versionFileParsed = false; try { const versionBuf = await this.fileSystem.readFile(this.versionFile); storedVersion = DatabaseVersion.fromBuffer(versionBuf); - versionFileParsed = true; } catch (err) { if (err && (err as Error & { code: string }).code === 'ENOENT') { storedVersion = DatabaseVersion.empty(); @@ -211,7 +184,7 @@ export class DatabaseVersionManager { needsReset = true; } } else if (cmp !== 0) { - if (versionFileParsed && this.schemaVersionMismatchPolicy === 'throw') { + if (this.schemaVersionMismatchPolicy === 'throw') { throw new Error( `Cannot open database at ${this.dataDirectory}: stored schema version ${storedVersion.schemaVersion} is incompatible with expected schema version ${this.currentVersion.schemaVersion}`, ); @@ -224,12 +197,6 @@ export class DatabaseVersionManager { needsReset = true; } } else { - if (versionFileParsed && this.rollupAddressMismatchPolicy === 'throw') { - throw new Error( - `Cannot open database at ${this.dataDirectory}: stored rollup address ` + - `${storedVersion.rollupAddress} does not match expected ${this.currentVersion.rollupAddress}`, - ); - } if (shouldLogDataReset) { this.log.warn('Rollup address has changed, resetting data directory', { versionFile: this.versionFile, diff --git a/yarn-project/stdlib/src/ha-signing/local_config.ts b/yarn-project/stdlib/src/ha-signing/local_config.ts index 896c1a3c9791..c16f3e0c4b99 100644 --- a/yarn-project/stdlib/src/ha-signing/local_config.ts +++ b/yarn-project/stdlib/src/ha-signing/local_config.ts @@ -52,7 +52,6 @@ export const LocalSignerConfigSchema = zodFor()( BaseSignerConfigSchema.extend({ dataDirectory: z.string().optional(), dataStoreMapSizeKb: z.number(), - l1ChainId: z.number().optional(), signingProtectionMapSizeKb: z.number().optional(), allowEphemeralSigningProtection: z.boolean().optional(), }), diff --git a/yarn-project/stdlib/src/kv-store/config.ts b/yarn-project/stdlib/src/kv-store/config.ts index e29a9779b14d..c0892f96020e 100644 --- a/yarn-project/stdlib/src/kv-store/config.ts +++ b/yarn-project/stdlib/src/kv-store/config.ts @@ -4,7 +4,6 @@ import { type ConfigMappingsType, getConfigFromMappings, numberConfigHelper } fr export type DataStoreConfig = { dataDirectory?: string; dataStoreMapSizeKb: number; - l1ChainId?: number; } & Partial>; export const dataConfigMappings: ConfigMappingsType = { @@ -17,11 +16,6 @@ export const dataConfigMappings: ConfigMappingsType = { description: 'The maximum possible size of a data store DB in KB. Can be overridden by component-specific options.', ...numberConfigHelper(128 * 1_024 * 1_024), // Defaulted to 128 GB }, - l1ChainId: { - env: 'L1_CHAIN_ID', - ...numberConfigHelper(31337), - description: 'The chain ID of the ethereum host.', - }, ...pickL1ContractAddressMappings('rollupAddress'), }; diff --git a/yarn-project/tsconfig.json b/yarn-project/tsconfig.json index 367b736ec023..1da5508133cb 100644 --- a/yarn-project/tsconfig.json +++ b/yarn-project/tsconfig.json @@ -1,7 +1,7 @@ { "compilerOptions": { "target": "es2022", - "lib": ["dom", "dom.asynciterable", "esnext", "es2017.object"], + "lib": ["dom", "esnext", "es2017.object"], "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, diff --git a/yarn-project/validator-client/src/config.ts b/yarn-project/validator-client/src/config.ts index 36d433fbb5f5..38d914a6d6af 100644 --- a/yarn-project/validator-client/src/config.ts +++ b/yarn-project/validator-client/src/config.ts @@ -42,6 +42,12 @@ export const validatorClientConfigMappings: ConfigMappingsType< .map(address => EthAddress.fromString(address.trim())), defaultValue: [], }, + l1ChainId: { + env: 'L1_CHAIN_ID', + description: 'The chain ID of the ethereum host.', + parseEnv: (val: string) => +val, + defaultValue: 31337, + }, disableValidator: { env: 'VALIDATOR_DISABLED', description: 'Do not run the validator', From d5c96d9af7de0452a67329570f9c12ef3df7d4ac Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 15:19:44 +0000 Subject: [PATCH 18/48] docs: cover embedded-wallet data locations in store-selection migration note --- docs/docs-developers/docs/resources/migration_notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 2726597fe2b2..b14f9991c764 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -13,7 +13,7 @@ Aztec is in active development. Each version may introduce breaking changes that Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. A store now exists per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is no longer returned by `listStores()`; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity stores live under `/-stores/`. Browser apps can enumerate and clean up stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is not visible to the new `listStores()` utility; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity stores live under `/-stores/`. That covers the server PXE and the CLI wallet; the embedded node wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`. Browser apps can enumerate and clean up stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: ```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; From d4f8bdb945b0ec3cdf23ea29d953efb54a923a80 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 15:19:55 +0000 Subject: [PATCH 19/48] refactor(pxe): polish identity-store helper logging, exports, and wallet data-root constant --- yarn-project/kv-store/src/lmdb-v2/index.ts | 2 +- yarn-project/pxe/src/entrypoints/server/store.ts | 9 ++++++--- yarn-project/wallets/src/embedded/entrypoints/browser.ts | 3 +++ yarn-project/wallets/src/embedded/entrypoints/node.ts | 6 ++++-- 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/yarn-project/kv-store/src/lmdb-v2/index.ts b/yarn-project/kv-store/src/lmdb-v2/index.ts index 50713eff1d81..510d5d752254 100644 --- a/yarn-project/kv-store/src/lmdb-v2/index.ts +++ b/yarn-project/kv-store/src/lmdb-v2/index.ts @@ -1,4 +1,4 @@ export * from './store.js'; export * from './factory.js'; -export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; +export { storeIdentitySlug } from '../store_identity.js'; export type { StoreIdentity } from '../store_identity.js'; diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 2e87fcf733a5..9905ea55a743 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -8,6 +8,7 @@ import { join } from 'path'; /** Location and identity inputs for opening an identity-partitioned PXE-side store. */ export type IdentityStoreConfig = { dataDirectory?: string; + /** Maximum LMDB map size in KB. When omitted, the kv-store default map size applies. */ dataStoreMapSizeKb?: number; l1ChainId?: number; rollupAddress?: EthAddress; @@ -38,8 +39,10 @@ export async function openStoreForIdentity( storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress: config.rollupAddress, schemaVersion }), ); await mkdir(subDir, { recursive: true }); - createLogger(`pxe:data:${name}`, bindings).info( - `Opening ${name} data store at directory ${subDir} with map size ${config.dataStoreMapSizeKb} KB (LMDB v2)`, - ); + createLogger(`pxe:data:${name}`, bindings).info(`Opening ${name} data store (LMDB v2)`, { + storeName: name, + subDir, + dataStoreMapSizeKb: config.dataStoreMapSizeKb, + }); return openStoreAt(subDir, config.dataStoreMapSizeKb, undefined, bindings); } diff --git a/yarn-project/wallets/src/embedded/entrypoints/browser.ts b/yarn-project/wallets/src/embedded/entrypoints/browser.ts index d72f0df46d70..54074f5cd166 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/browser.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/browser.ts @@ -36,6 +36,7 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { const pxeConfig: PXEConfig = Object.assign(getPXEConfig(), { proverEnabled: mergedConfigOverrides.proverEnabled, + // Unused in the browser: sqlite-opfs keys stores by name, not directory. dataDirectory: 'pxe_data', autoSync: false, ...mergedConfigOverrides, @@ -71,6 +72,8 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { : await createStore( 'wallet_data', { + // Unused in the browser: sqlite-opfs keys stores by name, not directory. Present only to + // satisfy the DataStoreConfig type. dataDirectory: 'wallet_data', dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, rollupAddress: l1ContractAddresses.rollupAddress, diff --git a/yarn-project/wallets/src/embedded/entrypoints/node.ts b/yarn-project/wallets/src/embedded/entrypoints/node.ts index af1f98e0ebec..063d177a1882 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/node.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/node.ts @@ -13,6 +13,8 @@ import type { AccountContractsProvider } from '../account-contract-providers/typ import { EmbeddedWallet, type EmbeddedWalletOptions, splitPxeOptions } from '../embedded_wallet.js'; import { WalletDB } from '../wallet_db.js'; +const DEFAULT_WALLET_DATA_DIRECTORY = 'aztec-wallet-data'; + export class NodeEmbeddedWallet extends EmbeddedWallet { static async create( this: new ( @@ -37,7 +39,7 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { const pxeConfig: PXEConfig = Object.assign(getPXEConfig(), { proverEnabled: mergedConfigOverrides.proverEnabled, - dataDirectory: 'aztec-wallet-data', + dataDirectory: DEFAULT_WALLET_DATA_DIRECTORY, autoSync: false, ...mergedConfigOverrides, }); @@ -79,7 +81,7 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { 'wallet_data', 1, { - dataDirectory: pxeConfig.dataDirectory ?? 'aztec-wallet-data', + dataDirectory: pxeConfig.dataDirectory ?? DEFAULT_WALLET_DATA_DIRECTORY, dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, rollupAddress: l1ContractAddresses.rollupAddress, l1ChainId, From 6864ad0da39e40e6d152b7ac0d03c4217b47b33c Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 16:06:27 +0000 Subject: [PATCH 20/48] fix(kv-store): throw on incomplete store identity instead of defaulting to zero Opening a persistent sqlite-opfs store (or composing its identity slug) with a missing l1ChainId, rollupAddress, or schemaVersion silently fell back to the zero identity, which could route two logically different callers to the same physical store. Require the full identity and throw instead. Files: yarn-project/kv-store/src/store_identity.ts, yarn-project/kv-store/src/store_identity.test.ts, yarn-project/kv-store/src/sqlite-opfs/index.ts, yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts --- .../src/sqlite-opfs/create_store.test.ts | 19 ++++++++++++ .../kv-store/src/sqlite-opfs/index.ts | 29 ++++++++++++------- .../kv-store/src/store_identity.test.ts | 8 +++-- yarn-project/kv-store/src/store_identity.ts | 10 +++---- 4 files changed, 48 insertions(+), 18 deletions(-) diff --git a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts index 45a3085df8cf..52de9f16a6ed 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts @@ -12,6 +12,25 @@ const configFor = (rollupAddress: EthAddress, l1ChainId = 31337) => ({ }); describe('sqlite-opfs createStore', () => { + it('rejects when a required identity component is missing', async () => { + const addr = EthAddress.random(); + const { l1ChainId: _l1ChainId, ...configWithoutChainId } = configFor(addr); + + await expect(createStore('incomplete_test', configWithoutChainId, 1, mockLogger)).rejects.toThrow( + /without a complete identity/, + ); + await expect(createStore('incomplete_test', configFor(addr), undefined, mockLogger)).rejects.toThrow( + /without a complete identity/, + ); + + const storeName = effectiveStoreName('incomplete_test', { + l1ChainId: 31337, + rollupAddress: addr, + schemaVersion: 1, + }); + expect(await listStores()).not.toContain(storeName); + }); + it('keeps data intact when switching rollup addresses back and forth', async () => { const addrA = EthAddress.random(); const addrB = EthAddress.random(); diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index c744c72f7fd0..411f2fb9b04d 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -1,4 +1,4 @@ -import { EthAddress } from '@aztec/foundation/eth-address'; +import type { EthAddress } from '@aztec/foundation/eth-address'; import { type Logger, createLogger } from '@aztec/foundation/log'; import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; @@ -18,6 +18,9 @@ export type { StoreIdentity } from '../store_identity.js'; * Opens the persistent store selected by `name` and the identity `(config.l1ChainId, config.rollupAddress, * schemaVersion)`. A store exists per identity: reopening with the same identity returns the same data, a * different identity selects a different (possibly fresh) store. Nothing is ever cleared. + * + * @throws If `schemaVersion`, `config.rollupAddress`, or `config.l1ChainId` is missing — an incomplete identity + * must never silently default to the zero identity. */ export async function createStore( name: string, @@ -25,11 +28,17 @@ export async function createStore( schemaVersion: number | undefined = undefined, log: Logger = createLogger('kv-store'), ) { - const storeName = effectiveStoreName(name, { - l1ChainId: config.l1ChainId, - rollupAddress: config.rollupAddress, - schemaVersion, - }); + const { rollupAddress, l1ChainId } = config; + if (schemaVersion === undefined || rollupAddress === undefined || l1ChainId === undefined) { + const missing = [ + l1ChainId === undefined && 'l1ChainId', + rollupAddress === undefined && 'rollupAddress', + schemaVersion === undefined && 'schemaVersion', + ].filter((component): component is string => component !== false); + throw new Error(`Cannot open store '${name}' without a complete identity: missing ${missing.join(', ')}`); + } + + const storeName = effectiveStoreName(name, { l1ChainId, rollupAddress, schemaVersion }); log.info(`Creating ${storeName} SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB`); const store = await AztecSQLiteOPFSStore.open( createLogger('kv-store:sqlite-opfs'), @@ -38,7 +47,7 @@ export async function createStore( storePoolDirectory(storeName), ); try { - await assertStoreIdentity(store, storeName, schemaVersion, config.rollupAddress); + await assertStoreIdentity(store, storeName, schemaVersion, rollupAddress); } catch (err) { // The store handle owns a worker and OPFS locks; release them before surfacing the refusal. await store.close().catch(() => {}); @@ -54,10 +63,10 @@ export async function createStore( async function assertStoreIdentity( store: AztecSQLiteOPFSStore, storeName: string, - schemaVersion: number | undefined, - rollupAddress: EthAddress | undefined, + schemaVersion: number, + rollupAddress: EthAddress, ): Promise { - const expected = new DatabaseVersion(schemaVersion ?? 0, rollupAddress ?? EthAddress.ZERO); + const expected = new DatabaseVersion(schemaVersion, rollupAddress); const singleton = store.openSingleton('dbVersion'); const stored = await singleton.getAsync(); if (stored === undefined) { diff --git a/yarn-project/kv-store/src/store_identity.test.ts b/yarn-project/kv-store/src/store_identity.test.ts index 738879e747fd..6da2aa10b94f 100644 --- a/yarn-project/kv-store/src/store_identity.test.ts +++ b/yarn-project/kv-store/src/store_identity.test.ts @@ -10,13 +10,15 @@ describe('storeIdentitySlug', () => { ); }); - it('defaults missing values to chain 0, zero address, schema 0', () => { - expect(storeIdentitySlug({})).toEqual(`0-${EthAddress.ZERO.toString()}-v0`); + it('composes the exact slug format for the zero identity', () => { + expect(storeIdentitySlug({ l1ChainId: 0, rollupAddress: EthAddress.ZERO, schemaVersion: 0 })).toEqual( + '0-0x0000000000000000000000000000000000000000-v0', + ); }); it('normalizes the rollup address to lowercase hex', () => { const rollupAddress = EthAddress.fromString('0x1234567890ABCDEF1234567890ABCDEF12345678'); - expect(storeIdentitySlug({ rollupAddress, schemaVersion: 1 })).toEqual( + expect(storeIdentitySlug({ l1ChainId: 0, rollupAddress, schemaVersion: 1 })).toEqual( `0-0x1234567890abcdef1234567890abcdef12345678-v1`, ); }); diff --git a/yarn-project/kv-store/src/store_identity.ts b/yarn-project/kv-store/src/store_identity.ts index bb62312d9e4a..4a988236c0be 100644 --- a/yarn-project/kv-store/src/store_identity.ts +++ b/yarn-project/kv-store/src/store_identity.ts @@ -1,13 +1,13 @@ -import { EthAddress } from '@aztec/foundation/eth-address'; +import type { EthAddress } from '@aztec/foundation/eth-address'; /** The coordinates that determine which physical store a logical store name maps to. */ export type StoreIdentity = { /** Chain ID of the L1 the rollup is deployed to. */ - l1ChainId?: number; + l1ChainId: number; /** Address of the rollup contract the store's data pertains to. */ - rollupAddress?: EthAddress; + rollupAddress: EthAddress; /** Schema version of the data held in the store. */ - schemaVersion?: number; + schemaVersion: number; }; /** @@ -15,7 +15,7 @@ export type StoreIdentity = { * their slugs are equal, so the format must stay stable: `--v`. */ export function storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion }: StoreIdentity): string { - return `${l1ChainId ?? 0}-${(rollupAddress ?? EthAddress.ZERO).toString()}-v${schemaVersion ?? 0}`; + return `${l1ChainId}-${rollupAddress.toString()}-v${schemaVersion}`; } /** Composes the physical store name for a logical store name and identity. */ From 3b3e56bacffb25745ec485c7bb0c79d2a2fb9c37 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 16:06:47 +0000 Subject: [PATCH 21/48] fix(pxe): require a full identity in openStoreForIdentity IdentityStoreConfig previously allowed l1ChainId and rollupAddress to be omitted, in which case the identity-partitioned store fell back to the zero identity. Make both fields required so callers can no longer open a PXE data store without a complete identity. Files: yarn-project/pxe/src/entrypoints/server/store.ts, yarn-project/pxe/src/entrypoints/server/utils.ts, yarn-project/pxe/src/entrypoints/server/store.test.ts --- .../pxe/src/entrypoints/server/store.test.ts | 20 ++++++------------- .../pxe/src/entrypoints/server/store.ts | 4 ++-- .../pxe/src/entrypoints/server/utils.ts | 7 ++++++- 3 files changed, 14 insertions(+), 17 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/server/store.test.ts b/yarn-project/pxe/src/entrypoints/server/store.test.ts index 4c475c33f824..552fb15d9baa 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.test.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.test.ts @@ -17,7 +17,7 @@ describe('openStoreForIdentity', () => { await rm(dataDirectory, { recursive: true, force: true }); }); - const configFor = (rollupAddress?: EthAddress, l1ChainId = 31337) => ({ + const configFor = (rollupAddress: EthAddress, l1ChainId = 31337) => ({ dataDirectory, dataStoreMapSizeKb: 10 * 1024, rollupAddress, @@ -57,18 +57,6 @@ describe('openStoreForIdentity', () => { await v1Again.close(); }); - it('opens and persists with a zero identity (no rollup, no chain id)', async () => { - const config = { dataDirectory, dataStoreMapSizeKb: 10 * 1024 }; - - const store = await openStoreForIdentity('test_store', 1, config); - await store.openSingleton('payload').set('zero-identity-data'); - await store.close(); - - const reopened = await openStoreForIdentity('test_store', 1, config); - expect(await reopened.openSingleton('payload').getAsync()).toEqual('zero-identity-data'); - await reopened.close(); - }); - it('places stores under a sibling -stores directory, not nested in ', async () => { const store = await openStoreForIdentity('test_store', 1, configFor(EthAddress.random())); await store.close(); @@ -78,7 +66,11 @@ describe('openStoreForIdentity', () => { }); it('falls back to an ephemeral tmp store when no data directory is configured', async () => { - const store = await openStoreForIdentity('test_store', 1, { dataStoreMapSizeKb: 10 * 1024 }); + const store = await openStoreForIdentity('test_store', 1, { + dataStoreMapSizeKb: 10 * 1024, + l1ChainId: 31337, + rollupAddress: EthAddress.random(), + }); await store.openSingleton('payload').set('tmp-data'); expect(await store.openSingleton('payload').getAsync()).toEqual('tmp-data'); await store.close(); diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 9905ea55a743..203d1c7a18fd 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -10,8 +10,8 @@ export type IdentityStoreConfig = { dataDirectory?: string; /** Maximum LMDB map size in KB. When omitted, the kv-store default map size applies. */ dataStoreMapSizeKb?: number; - l1ChainId?: number; - rollupAddress?: EthAddress; + l1ChainId: number; + rollupAddress: EthAddress; }; /** diff --git a/yarn-project/pxe/src/entrypoints/server/utils.ts b/yarn-project/pxe/src/entrypoints/server/utils.ts index 84df52ccde35..8fd1c18cd3b4 100644 --- a/yarn-project/pxe/src/entrypoints/server/utils.ts +++ b/yarn-project/pxe/src/entrypoints/server/utils.ts @@ -49,7 +49,12 @@ export async function createPXE( options.store = await openStoreForIdentity( 'pxe_data', PXE_DATA_SCHEMA_VERSION, - configWithContracts, + { + dataDirectory: configWithContracts.dataDirectory, + dataStoreMapSizeKb: configWithContracts.dataStoreMapSizeKb, + l1ChainId, + rollupAddress: l1ContractAddresses.rollupAddress, + }, storeLogger.getBindings(), ); } From 5a637cef3401a126cdcb9bebe5fb95fcd8efc0d0 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 16:07:00 +0000 Subject: [PATCH 22/48] fix(wallets): make wallet_data chain-agnostic with a schema-only identity The embedded wallet's wallet_data store held account secrets and aliases, which are not tied to any particular network, yet it was keyed by (l1ChainId, rollupAddress, schemaVersion) like the PXE data store. That meant switching networks silently swapped in a different (empty) wallet store. Key wallet_data by schema version alone (wallet_data_v1) so it stays the same store across networks. Files: yarn-project/wallets/src/embedded/wallet_db.ts, yarn-project/wallets/src/embedded/entrypoints/node.ts, yarn-project/wallets/src/embedded/entrypoints/browser.ts --- .../src/embedded/entrypoints/browser.ts | 20 ++++----- .../wallets/src/embedded/entrypoints/node.ts | 42 ++++++++----------- .../wallets/src/embedded/wallet_db.ts | 9 ++++ 3 files changed, 33 insertions(+), 38 deletions(-) diff --git a/yarn-project/wallets/src/embedded/entrypoints/browser.ts b/yarn-project/wallets/src/embedded/entrypoints/browser.ts index 54074f5cd166..1aeb1c067327 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/browser.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/browser.ts @@ -1,6 +1,6 @@ import { type AztecNode, createAztecNodeClient } from '@aztec/aztec.js/node'; import { type Logger, createLogger } from '@aztec/foundation/log'; -import { createStore, openTmpStore } from '@aztec/kv-store/sqlite-opfs'; +import { AztecSQLiteOPFSStore, openTmpStore, storePoolDirectory } from '@aztec/kv-store/sqlite-opfs'; import { type PXE, type PXECreationOptions, createPXE } from '@aztec/pxe/client/lazy'; import { type PXEConfig, getPXEConfig } from '@aztec/pxe/config'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry/lazy'; @@ -10,7 +10,7 @@ import { getStandardMultiCallEntrypoint } from '@aztec/standard-contracts/multi- import { LazyAccountContractsProvider } from '../account-contract-providers/lazy.js'; import type { AccountContractsProvider } from '../account-contract-providers/types.js'; import { EmbeddedWallet, type EmbeddedWalletOptions, splitPxeOptions } from '../embedded_wallet.js'; -import { WalletDB } from '../wallet_db.js'; +import { WALLET_DATA_STORE_NAME, WalletDB } from '../wallet_db.js'; export class BrowserEmbeddedWallet extends EmbeddedWallet { static async create( @@ -69,18 +69,12 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { options.walletDb?.store ?? (options.ephemeral ? await openTmpStore(true) - : await createStore( - 'wallet_data', - { - // Unused in the browser: sqlite-opfs keys stores by name, not directory. Present only to - // satisfy the DataStoreConfig type. - dataDirectory: 'wallet_data', - dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, - rollupAddress: l1ContractAddresses.rollupAddress, - l1ChainId, - }, - 1, + : await AztecSQLiteOPFSStore.open( + // Chain-agnostic store: keyed by a fixed schema-versioned name, not by network identity. rootLogger.createChild('wallet:data'), + WALLET_DATA_STORE_NAME, + false, + storePoolDirectory(WALLET_DATA_STORE_NAME), )); const walletDB = new WalletDB(walletDBStore, rootLogger.createChild('wallet:db').info); diff --git a/yarn-project/wallets/src/embedded/entrypoints/node.ts b/yarn-project/wallets/src/embedded/entrypoints/node.ts index 063d177a1882..45ad80bd1017 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/node.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/node.ts @@ -1,17 +1,20 @@ import { createAztecNodeClient } from '@aztec/aztec.js/node'; import { type Logger, createLogger } from '@aztec/foundation/log'; -import { openTmpStore } from '@aztec/kv-store/lmdb-v2'; +import { openStoreAt, openTmpStore } from '@aztec/kv-store/lmdb-v2'; import { type PXEConfig, getPXEConfig } from '@aztec/pxe/config'; -import { type PXE, type PXECreationOptions, createPXE, openStoreForIdentity } from '@aztec/pxe/server'; +import { type PXE, type PXECreationOptions, createPXE } from '@aztec/pxe/server'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry'; import { getStandardHandshakeRegistry } from '@aztec/standard-contracts/handshake-registry'; import { getStandardMultiCallEntrypoint } from '@aztec/standard-contracts/multi-call-entrypoint'; import type { AztecNode } from '@aztec/stdlib/interfaces/client'; +import { mkdir } from 'fs/promises'; +import { join } from 'path'; + import { BundleAccountContractsProvider } from '../account-contract-providers/bundle.js'; import type { AccountContractsProvider } from '../account-contract-providers/types.js'; import { EmbeddedWallet, type EmbeddedWalletOptions, splitPxeOptions } from '../embedded_wallet.js'; -import { WalletDB } from '../wallet_db.js'; +import { WALLET_DATA_STORE_NAME, WalletDB } from '../wallet_db.js'; const DEFAULT_WALLET_DATA_DIRECTORY = 'aztec-wallet-data'; @@ -30,7 +33,6 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { const rootLogger = options.logger ?? createLogger('embedded-wallet'); const aztecNode = typeof nodeOrUrl === 'string' ? createAztecNodeClient(nodeOrUrl) : nodeOrUrl; - const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); // Support both the new unified `pxe` option and the deprecated `pxeConfig`/`pxeOptions`. const { config: pxeConfigFromPxe, creation: pxeCreationFromPxe } = splitPxeOptions(options.pxe); @@ -67,27 +69,17 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { const pxe = await createPXE(aztecNode, pxeConfig, pxeOptions); - const walletDBStore = - options.walletDb?.store ?? - (options.ephemeral - ? await openTmpStore( - 'wallet_data', - true, - undefined, - undefined, - rootLogger.createChild('wallet:data').getBindings(), - ) - : await openStoreForIdentity( - 'wallet_data', - 1, - { - dataDirectory: pxeConfig.dataDirectory ?? DEFAULT_WALLET_DATA_DIRECTORY, - dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, - rollupAddress: l1ContractAddresses.rollupAddress, - l1ChainId, - }, - rootLogger.createChild('wallet:data').getBindings(), - )); + let walletDBStore = options.walletDb?.store; + if (!walletDBStore) { + const bindings = rootLogger.createChild('wallet:data').getBindings(); + if (options.ephemeral) { + walletDBStore = await openTmpStore('wallet_data', true, undefined, undefined, bindings); + } else { + const walletDataDir = join(pxeConfig.dataDirectory ?? DEFAULT_WALLET_DATA_DIRECTORY, WALLET_DATA_STORE_NAME); + await mkdir(walletDataDir, { recursive: true }); + walletDBStore = await openStoreAt(walletDataDir, pxeConfig.dataStoreMapSizeKb, undefined, bindings); + } + } const walletDB = new WalletDB(walletDBStore, rootLogger.createChild('wallet:db').info); const wallet = new this(pxe, aztecNode, walletDB, new BundleAccountContractsProvider(), rootLogger) as T; diff --git a/yarn-project/wallets/src/embedded/wallet_db.ts b/yarn-project/wallets/src/embedded/wallet_db.ts index 6671cdee0171..24de2b2238e0 100644 --- a/yarn-project/wallets/src/embedded/wallet_db.ts +++ b/yarn-project/wallets/src/embedded/wallet_db.ts @@ -11,6 +11,15 @@ function accountKey(field: string, address: AztecAddress | string): string { return `${field}:${address.toString()}`; } +/** Bump when the WalletDB layout changes; a new version selects a fresh store, leaving the old one intact. */ +export const WALLET_DATA_SCHEMA_VERSION = 1; + +/** + * Name of the wallet DB store. Chain-agnostic: accounts and aliases apply across networks, so the name carries + * only the schema version. + */ +export const WALLET_DATA_STORE_NAME = `wallet_data_v${WALLET_DATA_SCHEMA_VERSION}`; + export class WalletDB { private accounts: AztecAsyncMap; private aliases: AztecAsyncMap; From 17f6a9dc26e23dec015f9cf71be296b4b2f37fbc Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 16:07:11 +0000 Subject: [PATCH 23/48] docs: wallet_data is chain-agnostic in the store-selection migration note Files: docs/docs-developers/docs/resources/migration_notes.md --- docs/docs-developers/docs/resources/migration_notes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index b14f9991c764..9ac0197ebe89 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -11,9 +11,9 @@ Aztec is in active development. Each version may introduce breaking changes that ### [PXE] Stores are now selected by `(l1ChainId, rollupAddress, schemaVersion)` instead of being wiped on mismatch -Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. A store now exists per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. +Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores (`pxe_data`) now exist per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. The embedded wallet's `wallet_data` store takes a different approach: since its contents (account secrets and aliases) are not network-specific, it is now chain-agnostic: a single schema-versioned store (`wallet_data_v1`) shared across networks, so stored accounts and aliases survive switching networks. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is not visible to the new `listStores()` utility; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity stores live under `/-stores/`. That covers the server PXE and the CLI wallet; the embedded node wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`. Browser apps can enumerate and clean up stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is not visible to the new `listStores()` utility; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. That covers the server PXE and the CLI wallet; the embedded node wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: ```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; From 7a5d23679b779725aaa5358b11562852c5586426 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 16:09:40 +0000 Subject: [PATCH 24/48] refactor(wallets): drop unused node-info destructure in browser entrypoint --- yarn-project/wallets/src/embedded/entrypoints/browser.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/yarn-project/wallets/src/embedded/entrypoints/browser.ts b/yarn-project/wallets/src/embedded/entrypoints/browser.ts index 1aeb1c067327..8f2a503ff879 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/browser.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/browser.ts @@ -27,7 +27,6 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { const rootLogger = options.logger ?? createLogger('embedded-wallet'); const aztecNode = typeof nodeOrUrl === 'string' ? createAztecNodeClient(nodeOrUrl) : nodeOrUrl; - const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); // Support both the new unified `pxe` option and the deprecated `pxeConfig`/`pxeOptions`. const { config: pxeConfigFromPxe, creation: pxeCreationFromPxe } = splitPxeOptions(options.pxe); From b4fe53c6a1beb54a36fd8bb9e214acc8cf48ae9c Mon Sep 17 00:00:00 2001 From: mverzilli Date: Thu, 9 Jul 2026 16:16:02 +0000 Subject: [PATCH 25/48] test(kv-store): cover missing rollup address in identity throw; note wallet_data_v1 location --- docs/docs-developers/docs/resources/migration_notes.md | 2 +- yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 9ac0197ebe89..dd9a1725257c 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -13,7 +13,7 @@ Aztec is in active development. Each version may introduce breaking changes that Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores (`pxe_data`) now exist per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. The embedded wallet's `wallet_data` store takes a different approach: since its contents (account secrets and aliases) are not network-specific, it is now chain-agnostic: a single schema-versioned store (`wallet_data_v1`) shared across networks, so stored accounts and aliases survive switching networks. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is not visible to the new `listStores()` utility; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. That covers the server PXE and the CLI wallet; the embedded node wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is not visible to the new `listStores()` utility; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. That covers the server PXE and the CLI wallet; the embedded node wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with the chain-agnostic wallet store at `/wallet_data_v1` on node and under the fixed name `wallet_data_v1` in browser OPFS. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: ```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; diff --git a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts index 52de9f16a6ed..f73546e421d4 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts @@ -15,10 +15,14 @@ describe('sqlite-opfs createStore', () => { it('rejects when a required identity component is missing', async () => { const addr = EthAddress.random(); const { l1ChainId: _l1ChainId, ...configWithoutChainId } = configFor(addr); + const { rollupAddress: _rollupAddress, ...configWithoutRollup } = configFor(addr); await expect(createStore('incomplete_test', configWithoutChainId, 1, mockLogger)).rejects.toThrow( /without a complete identity/, ); + await expect(createStore('incomplete_test', configWithoutRollup, 1, mockLogger)).rejects.toThrow( + /without a complete identity/, + ); await expect(createStore('incomplete_test', configFor(addr), undefined, mockLogger)).rejects.toThrow( /without a complete identity/, ); From 0f37e8d1e71bffbc6d04323cee61c7655515a014 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 07:42:40 +0000 Subject: [PATCH 26/48] refactor(kv-store): move store identity into sqlite-opfs and drop the lmdb-v2 re-export The identity slug only has one consumer outside the browser backend: the pxe node-side helper, whose lmdb directory names never interoperate with browser DB names. Let the helper compose its own directory-name format inline and keep the identity module private to sqlite-opfs, returning the kv-store node entrypoint to its base state. --- yarn-project/kv-store/src/lmdb-v2/index.ts | 2 -- yarn-project/kv-store/src/sqlite-opfs/index.ts | 6 +++--- .../src/{ => sqlite-opfs}/store_identity.test.ts | 0 .../kv-store/src/{ => sqlite-opfs}/store_identity.ts | 0 yarn-project/pxe/src/entrypoints/server/store.ts | 11 ++++++----- 5 files changed, 9 insertions(+), 10 deletions(-) rename yarn-project/kv-store/src/{ => sqlite-opfs}/store_identity.test.ts (100%) rename yarn-project/kv-store/src/{ => sqlite-opfs}/store_identity.ts (100%) diff --git a/yarn-project/kv-store/src/lmdb-v2/index.ts b/yarn-project/kv-store/src/lmdb-v2/index.ts index 510d5d752254..af723281fd47 100644 --- a/yarn-project/kv-store/src/lmdb-v2/index.ts +++ b/yarn-project/kv-store/src/lmdb-v2/index.ts @@ -1,4 +1,2 @@ export * from './store.js'; export * from './factory.js'; -export { storeIdentitySlug } from '../store_identity.js'; -export type { StoreIdentity } from '../store_identity.js'; diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index 411f2fb9b04d..7b6b3fd23913 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -3,16 +3,16 @@ import { type Logger, createLogger } from '@aztec/foundation/log'; import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; -import { StoreIdentityMismatchError, effectiveStoreName } from '../store_identity.js'; import { storePoolDirectory } from './manage.js'; import { AztecSQLiteOPFSStore } from './store.js'; +import { StoreIdentityMismatchError, effectiveStoreName } from './store_identity.js'; export { AztecSQLiteOPFSStore } from './store.js'; export { SqliteEncryptionError } from './errors.js'; export type { SqliteEncryptionErrorCode } from './errors.js'; export { OPFS_POOL_DIR_PREFIX, deleteStore, listStores, storePoolDirectory } from './manage.js'; -export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from '../store_identity.js'; -export type { StoreIdentity } from '../store_identity.js'; +export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from './store_identity.js'; +export type { StoreIdentity } from './store_identity.js'; /** * Opens the persistent store selected by `name` and the identity `(config.l1ChainId, config.rollupAddress, diff --git a/yarn-project/kv-store/src/store_identity.test.ts b/yarn-project/kv-store/src/sqlite-opfs/store_identity.test.ts similarity index 100% rename from yarn-project/kv-store/src/store_identity.test.ts rename to yarn-project/kv-store/src/sqlite-opfs/store_identity.test.ts diff --git a/yarn-project/kv-store/src/store_identity.ts b/yarn-project/kv-store/src/sqlite-opfs/store_identity.ts similarity index 100% rename from yarn-project/kv-store/src/store_identity.ts rename to yarn-project/kv-store/src/sqlite-opfs/store_identity.ts diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 203d1c7a18fd..92e8afe3be9d 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -1,6 +1,6 @@ import type { EthAddress } from '@aztec/foundation/eth-address'; import { type LoggerBindings, createLogger } from '@aztec/foundation/log'; -import { type AztecLMDBStoreV2, openStoreAt, openTmpStore, storeIdentitySlug } from '@aztec/kv-store/lmdb-v2'; +import { type AztecLMDBStoreV2, openStoreAt, openTmpStore } from '@aztec/kv-store/lmdb-v2'; import { mkdir } from 'fs/promises'; import { join } from 'path'; @@ -20,9 +20,10 @@ export type IdentityStoreConfig = { * a different (possibly fresh) store. Nothing is ever cleared, and no version marker is kept — the directory name is * the identity. Falls back to an ephemeral tmp store when no data directory is configured. * - * Stores live under `/-stores/`, a sibling of the legacy `/` - * directory: older binaries reset the legacy directory on a rollup mismatch, so per-identity stores must not nest - * inside it. + * Stores live under `/-stores/--v`, a sibling of the + * legacy `/` directory: older binaries reset the legacy directory on a rollup mismatch, so + * per-identity stores must not nest inside it. Two identities select the same store iff their directory names are + * equal, so the name format must stay stable — changing it orphans every existing store. */ export async function openStoreForIdentity( name: string, @@ -36,7 +37,7 @@ export async function openStoreForIdentity( const subDir = join( config.dataDirectory, `${name}-stores`, - storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress: config.rollupAddress, schemaVersion }), + `${config.l1ChainId}-${config.rollupAddress.toString()}-v${schemaVersion}`, ); await mkdir(subDir, { recursive: true }); createLogger(`pxe:data:${name}`, bindings).info(`Opening ${name} data store (LMDB v2)`, { From c1673038d16f6f00e597cdda9115bc1a8500de18 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 08:06:56 +0000 Subject: [PATCH 27/48] chore(kv-store): drop the src-root test glob from the node vitest project It only existed to pick up store_identity.test.ts, which now lives in sqlite-opfs and runs in the browser project; the config is back to its base state. --- yarn-project/kv-store/vitest.config.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/yarn-project/kv-store/vitest.config.ts b/yarn-project/kv-store/vitest.config.ts index 3e51261da9d5..7fd105321a53 100644 --- a/yarn-project/kv-store/vitest.config.ts +++ b/yarn-project/kv-store/vitest.config.ts @@ -53,7 +53,6 @@ export default defineConfig({ name: 'node', environment: 'node', include: [ - './src/*.test.ts', './src/lmdb/**/*.test.ts', './src/lmdb-v2/**/*.test.ts', './src/stores/**/*.test.ts', From 81c021e22763259f8fe1ef1a848f31b42b32ac33 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 08:18:58 +0000 Subject: [PATCH 28/48] refactor(pxe): rename openStoreForIdentity to openPXEStore --- .../pxe/src/entrypoints/server/store.test.ts | 20 +++++++++---------- .../pxe/src/entrypoints/server/store.ts | 2 +- .../pxe/src/entrypoints/server/utils.ts | 4 ++-- 3 files changed, 13 insertions(+), 13 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/server/store.test.ts b/yarn-project/pxe/src/entrypoints/server/store.test.ts index 552fb15d9baa..70df459d1227 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.test.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.test.ts @@ -4,9 +4,9 @@ import { mkdtemp, rm, stat } from 'fs/promises'; import { tmpdir } from 'os'; import { join } from 'path'; -import { openStoreForIdentity } from './store.js'; +import { openPXEStore } from './store.js'; -describe('openStoreForIdentity', () => { +describe('openPXEStore', () => { let dataDirectory: string; beforeEach(async () => { @@ -28,15 +28,15 @@ describe('openStoreForIdentity', () => { const addrA = EthAddress.random(); const addrB = EthAddress.random(); - const storeA = await openStoreForIdentity('test_store', 1, configFor(addrA)); + const storeA = await openPXEStore('test_store', 1, configFor(addrA)); await storeA.openSingleton('payload').set('data-for-A'); await storeA.close(); - const storeB = await openStoreForIdentity('test_store', 1, configFor(addrB)); + const storeB = await openPXEStore('test_store', 1, configFor(addrB)); expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); await storeB.close(); - const reopenedA = await openStoreForIdentity('test_store', 1, configFor(addrA)); + const reopenedA = await openPXEStore('test_store', 1, configFor(addrA)); expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); await reopenedA.close(); }); @@ -44,21 +44,21 @@ describe('openStoreForIdentity', () => { it('separates stores by schema version', async () => { const addr = EthAddress.random(); - const v1 = await openStoreForIdentity('test_store', 1, configFor(addr)); + const v1 = await openPXEStore('test_store', 1, configFor(addr)); await v1.openSingleton('k').set('v1-data'); await v1.close(); - const v2 = await openStoreForIdentity('test_store', 2, configFor(addr)); + const v2 = await openPXEStore('test_store', 2, configFor(addr)); expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); await v2.close(); - const v1Again = await openStoreForIdentity('test_store', 1, configFor(addr)); + const v1Again = await openPXEStore('test_store', 1, configFor(addr)); expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); await v1Again.close(); }); it('places stores under a sibling -stores directory, not nested in ', async () => { - const store = await openStoreForIdentity('test_store', 1, configFor(EthAddress.random())); + const store = await openPXEStore('test_store', 1, configFor(EthAddress.random())); await store.close(); await expect(stat(join(dataDirectory, 'test_store-stores'))).resolves.toBeDefined(); @@ -66,7 +66,7 @@ describe('openStoreForIdentity', () => { }); it('falls back to an ephemeral tmp store when no data directory is configured', async () => { - const store = await openStoreForIdentity('test_store', 1, { + const store = await openPXEStore('test_store', 1, { dataStoreMapSizeKb: 10 * 1024, l1ChainId: 31337, rollupAddress: EthAddress.random(), diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 92e8afe3be9d..95327737a79e 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -25,7 +25,7 @@ export type IdentityStoreConfig = { * per-identity stores must not nest inside it. Two identities select the same store iff their directory names are * equal, so the name format must stay stable — changing it orphans every existing store. */ -export async function openStoreForIdentity( +export async function openPXEStore( name: string, schemaVersion: number, config: IdentityStoreConfig, diff --git a/yarn-project/pxe/src/entrypoints/server/utils.ts b/yarn-project/pxe/src/entrypoints/server/utils.ts index 8fd1c18cd3b4..01ae45ddd283 100644 --- a/yarn-project/pxe/src/entrypoints/server/utils.ts +++ b/yarn-project/pxe/src/entrypoints/server/utils.ts @@ -14,7 +14,7 @@ import type { PXEConfig } from '../../config/index.js'; import { PXE } from '../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../storage/index.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../pxe_creation_options.js'; -import { openStoreForIdentity } from './store.js'; +import { openPXEStore } from './store.js'; type PXEConfigWithoutDefaults = Omit< PXEConfig, @@ -46,7 +46,7 @@ export async function createPXE( if (!options.store) { const storeLogger = loggers.store ?? createLogger('pxe:data:lmdb', { actor }); - options.store = await openStoreForIdentity( + options.store = await openPXEStore( 'pxe_data', PXE_DATA_SCHEMA_VERSION, { From 7cc23e3727d07e168285893ec8d325887b9cb76d Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 08:36:31 +0000 Subject: [PATCH 29/48] docs: tighten the store-selection impact note --- docs/docs-developers/docs/resources/migration_notes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index dd9a1725257c..a452a9a4b4b3 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -13,13 +13,13 @@ Aztec is in active development. Each version may introduce breaking changes that Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores (`pxe_data`) now exist per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. The embedded wallet's `wallet_data` store takes a different approach: since its contents (account secrets and aliases) are not network-specific, it is now chain-agnostic: a single schema-versioned store (`wallet_data_v1`) shared across networks, so stored accounts and aliases survive switching networks. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. In the browser (sqlite-opfs) the old store stays in OPFS but is not visible to the new `listStores()` utility; on node (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. That covers the server PXE and the CLI wallet; the embedded node wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with the chain-agnostic wallet store at `/wallet_data_v1` on node and under the fixed name `wallet_data_v1` in browser OPFS. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. On Node.js environments (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. The embedded Node.js wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with the chain-agnostic wallet store at `/wallet_data_v1` on Node.js and under the fixed name `wallet_data_v1` in browser OPFS. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: ```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; const names = await listStores(); -await deleteStore(names[0]); // the store must be closed first +await deleteStore(names[0]); ``` ### [Aztec.nr] `TestEnvironmentOptions::with_tagging_secret_strategy` replaced From 3a02d7890803bacacb13edf95adb0d663dbb8cfd Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 08:36:32 +0000 Subject: [PATCH 30/48] chore(kv-store): simplify the listStores jsdoc --- yarn-project/kv-store/src/sqlite-opfs/manage.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/yarn-project/kv-store/src/sqlite-opfs/manage.ts b/yarn-project/kv-store/src/sqlite-opfs/manage.ts index 35e3f760a765..ff524172b684 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/manage.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/manage.ts @@ -13,7 +13,7 @@ export function storePoolDirectory(effectiveName: string): string { /** * Lists the effective names (logical name + identity slug) of every persistent sqlite-opfs store in this * origin, by enumerating the per-store pool directories. Includes stores created under identities other than - * the current one — that is the point: wallets can surface and clean up data for networks no longer in use. + * the current one, so wallets can surface and clean up data for networks no longer in use. */ export async function listStores(): Promise { const root = await navigator.storage.getDirectory(); From 6032ae7c2db84ac174fddf96b2c88a19d9f45890 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 09:34:05 +0000 Subject: [PATCH 31/48] feat(pxe): add a pxe-owned store identity module --- .../pxe/src/storage/store_identity.test.ts | 96 +++++++++++++++++++ .../pxe/src/storage/store_identity.ts | 96 +++++++++++++++++++ 2 files changed, 192 insertions(+) create mode 100644 yarn-project/pxe/src/storage/store_identity.test.ts create mode 100644 yarn-project/pxe/src/storage/store_identity.ts diff --git a/yarn-project/pxe/src/storage/store_identity.test.ts b/yarn-project/pxe/src/storage/store_identity.test.ts new file mode 100644 index 000000000000..2bc9984358ae --- /dev/null +++ b/yarn-project/pxe/src/storage/store_identity.test.ts @@ -0,0 +1,96 @@ +import { EthAddress } from '@aztec/foundation/eth-address'; +import { openTmpStore } from '@aztec/kv-store/lmdb-v2'; + +import { + StoreIdentityMismatchError, + assertStoreIdentity, + effectiveStoreName, + requireCompleteIdentity, + storeIdentitySlug, +} from './store_identity.js'; + +describe('storeIdentitySlug', () => { + it('composes chain id, rollup address and schema version', () => { + const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); + expect(storeIdentitySlug({ l1ChainId: 31337, rollupAddress, schemaVersion: 12 })).toEqual( + '31337-0x1234567890abcdef1234567890abcdef12345678-v12', + ); + }); + + it('normalizes the rollup address to lowercase hex', () => { + const rollupAddress = EthAddress.fromString('0x1234567890ABCDEF1234567890ABCDEF12345678'); + expect(storeIdentitySlug({ l1ChainId: 0, rollupAddress, schemaVersion: 1 })).toEqual( + '0-0x1234567890abcdef1234567890abcdef12345678-v1', + ); + }); +}); + +describe('effectiveStoreName', () => { + it('joins the logical name and the slug with an underscore', () => { + const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); + expect(effectiveStoreName('pxe_data', { l1ChainId: 1, rollupAddress, schemaVersion: 2 })).toEqual( + 'pxe_data_1-0x1234567890abcdef1234567890abcdef12345678-v2', + ); + }); +}); + +describe('requireCompleteIdentity', () => { + it('returns the identity when all components are present', () => { + const rollupAddress = EthAddress.random(); + expect(requireCompleteIdentity('s', { l1ChainId: 31337, rollupAddress }, 1)).toEqual({ + l1ChainId: 31337, + rollupAddress, + schemaVersion: 1, + }); + }); + + it('throws naming each missing component', () => { + const rollupAddress = EthAddress.random(); + expect(() => requireCompleteIdentity('s', { rollupAddress }, 1)).toThrow( + "Cannot open store 's' without a complete identity: missing l1ChainId", + ); + expect(() => requireCompleteIdentity('s', { l1ChainId: 31337 }, 1)).toThrow( + "Cannot open store 's' without a complete identity: missing rollupAddress", + ); + expect(() => requireCompleteIdentity('s', { l1ChainId: 31337, rollupAddress }, undefined)).toThrow( + "Cannot open store 's' without a complete identity: missing schemaVersion", + ); + expect(() => requireCompleteIdentity('s', {}, undefined)).toThrow( + "Cannot open store 's' without a complete identity: missing l1ChainId, rollupAddress, schemaVersion", + ); + }); +}); + +describe('assertStoreIdentity', () => { + const identityFor = (rollupAddress: EthAddress) => ({ l1ChainId: 31337, rollupAddress, schemaVersion: 1 }); + + it('writes the marker on first open and accepts a matching reopen', async () => { + const store = await openTmpStore('identity-test'); + const identity = identityFor(EthAddress.random()); + await assertStoreIdentity(store, 'test_store', identity); + await expect(assertStoreIdentity(store, 'test_store', identity)).resolves.toBeUndefined(); + await store.close(); + }); + + it('refuses a mismatching identity and leaves data untouched', async () => { + const store = await openTmpStore('identity-test'); + const identity = identityFor(EthAddress.random()); + await assertStoreIdentity(store, 'test_store', identity); + await store.openSingleton('payload').set('precious'); + + const bumped = { ...identity, schemaVersion: identity.schemaVersion + 1 }; + await expect(assertStoreIdentity(store, 'test_store', bumped)).rejects.toThrow(StoreIdentityMismatchError); + + expect(await store.openSingleton('payload').getAsync()).toEqual('precious'); + await store.close(); + }); + + it('refuses a corrupted marker', async () => { + const store = await openTmpStore('identity-test'); + await store.openSingleton('dbVersion').set('garbage'); + await expect(assertStoreIdentity(store, 'test_store', identityFor(EthAddress.random()))).rejects.toThrow( + StoreIdentityMismatchError, + ); + await store.close(); + }); +}); diff --git a/yarn-project/pxe/src/storage/store_identity.ts b/yarn-project/pxe/src/storage/store_identity.ts new file mode 100644 index 000000000000..ef1c35fa6b59 --- /dev/null +++ b/yarn-project/pxe/src/storage/store_identity.ts @@ -0,0 +1,96 @@ +import type { EthAddress } from '@aztec/foundation/eth-address'; +import type { AztecAsyncKVStore } from '@aztec/kv-store'; +import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; + +/** The coordinates that determine which physical store a logical store name maps to. */ +export type StoreIdentity = { + /** Chain ID of the L1 the rollup is deployed to. */ + l1ChainId: number; + /** Address of the rollup contract the store's data pertains to. */ + rollupAddress: EthAddress; + /** Schema version of the data held in the store. */ + schemaVersion: number; +}; + +/** + * Composes the store-name discriminator for a store identity. Two identities map to the same physical store iff + * their slugs are equal, so the format must stay stable: `--v` — changing + * it orphans every existing store. + */ +export function storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion }: StoreIdentity): string { + return `${l1ChainId}-${rollupAddress.toString()}-v${schemaVersion}`; +} + +/** Composes the physical store name for a logical store name and identity. */ +export function effectiveStoreName(name: string, identity: StoreIdentity): string { + return `${name}_${storeIdentitySlug(identity)}`; +} + +/** + * Validates that every identity component is present, so an incomplete identity never silently defaults to the + * zero identity. + * @throws If `config.l1ChainId`, `config.rollupAddress`, or `schemaVersion` is missing. + */ +export function requireCompleteIdentity( + name: string, + config: { l1ChainId?: number; rollupAddress?: EthAddress }, + schemaVersion?: number, +): StoreIdentity { + const { l1ChainId, rollupAddress } = config; + if (l1ChainId === undefined || rollupAddress === undefined || schemaVersion === undefined) { + const missing = [ + l1ChainId === undefined && 'l1ChainId', + rollupAddress === undefined && 'rollupAddress', + schemaVersion === undefined && 'schemaVersion', + ].filter((component): component is string => component !== false); + throw new Error(`Cannot open store '${name}' without a complete identity: missing ${missing.join(', ')}`); + } + return { l1ChainId, rollupAddress, schemaVersion }; +} + +/** + * Belt-and-braces invariant check for stores whose physical name carries their identity: the recorded version can + * only disagree with the expected one if there is a store-naming bug. Writes the marker on first open; refuses to + * proceed on mismatch; never clears. The marker records `(schemaVersion, rollupAddress)` only — the store name + * carries the full identity. + */ +export async function assertStoreIdentity( + store: AztecAsyncKVStore, + storeName: string, + identity: StoreIdentity, +): Promise { + const expected = new DatabaseVersion(identity.schemaVersion, identity.rollupAddress); + const singleton = store.openSingleton('dbVersion'); + const stored = await singleton.getAsync(); + if (stored === undefined) { + await singleton.set(expected.toBuffer().toString('utf-8')); + return; + } + let storedVersion: DatabaseVersion; + try { + storedVersion = DatabaseVersion.fromBuffer(Buffer.from(stored, 'utf-8')); + } catch { + throw new StoreIdentityMismatchError(storeName, expected.toString(), stored); + } + if (!storedVersion.equals(expected)) { + throw new StoreIdentityMismatchError(storeName, expected.toString(), storedVersion.toString()); + } +} + +/** + * Thrown when a store's recorded identity does not match the identity it was opened under. Since the identity is + * part of the physical store name, this can only indicate a store-naming bug; the store is left untouched. + */ +export class StoreIdentityMismatchError extends Error { + constructor( + public readonly storeName: string, + public readonly expected: string, + public readonly actual: string, + ) { + super( + `Store '${storeName}' records identity ${actual} but was opened as ${expected}. ` + + `Refusing to open; data was NOT modified.`, + ); + this.name = 'StoreIdentityMismatchError'; + } +} From 3cbd2c2ddc55ebf6e2efeffe389ef49d8dbd1780 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 09:39:35 +0000 Subject: [PATCH 32/48] refactor(pxe): compose store identity in pxe for browser and node stores --- .../src/entrypoints/client/bundle/utils.ts | 4 +- .../pxe/src/entrypoints/client/lazy/utils.ts | 4 +- .../pxe/src/entrypoints/client/store.ts | 41 +++++++++++++++++++ .../pxe/src/entrypoints/server/store.ts | 4 +- 4 files changed, 48 insertions(+), 5 deletions(-) create mode 100644 yarn-project/pxe/src/entrypoints/client/store.ts diff --git a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts index 0f64428d9545..eb668d1d6eca 100644 --- a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts @@ -1,6 +1,5 @@ import { BBBundlePrivateKernelProver } from '@aztec/bb-prover/client/bundle'; import { createLogger } from '@aztec/foundation/log'; -import { createStore } from '@aztec/kv-store/sqlite-opfs'; import { BundledProtocolContractsProvider } from '@aztec/protocol-contracts/providers/bundle'; import { WASMSimulator } from '@aztec/simulator/client'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry'; @@ -12,6 +11,7 @@ import type { PXEConfig } from '../../../config/index.js'; import { PXE } from '../../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../../storage/metadata.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../../pxe_creation_options.js'; +import { openPXEBrowserStore } from '../store.js'; /** * Create and start an PXE instance with the given AztecNode. @@ -42,7 +42,7 @@ export async function createPXE( const storeLogger = loggers.store ?? createLogger('pxe:data', { actor }); const store = - options.store ?? (await createStore('pxe_data', configWithContracts, PXE_DATA_SCHEMA_VERSION, storeLogger)); + options.store ?? (await openPXEBrowserStore('pxe_data', PXE_DATA_SCHEMA_VERSION, configWithContracts, storeLogger)); const simulator = options.simulator ?? new WASMSimulator(); const proverLogger = loggers.prover ?? createLogger('pxe:bb:wasm:bundle', { actor }); diff --git a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts index b4dd06ab3312..0b7cb1430af6 100644 --- a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts @@ -1,6 +1,5 @@ import { BBLazyPrivateKernelProver } from '@aztec/bb-prover/client/lazy'; import { createLogger } from '@aztec/foundation/log'; -import { createStore } from '@aztec/kv-store/sqlite-opfs'; import { LazyProtocolContractsProvider } from '@aztec/protocol-contracts/providers/lazy'; import { WASMSimulator } from '@aztec/simulator/client'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry/lazy'; @@ -12,6 +11,7 @@ import type { PXEConfig } from '../../../config/index.js'; import { PXE } from '../../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../../storage/metadata.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../../pxe_creation_options.js'; +import { openPXEBrowserStore } from '../store.js'; /** * Create and start an PXE instance with the given AztecNode. @@ -42,7 +42,7 @@ export async function createPXE( const storeLogger = loggers.store ?? createLogger('pxe:data', { actor }); const store = - options.store ?? (await createStore('pxe_data', configWithContracts, PXE_DATA_SCHEMA_VERSION, storeLogger)); + options.store ?? (await openPXEBrowserStore('pxe_data', PXE_DATA_SCHEMA_VERSION, configWithContracts, storeLogger)); const simulator = options.simulator ?? new WASMSimulator(); const proverLogger = loggers.prover ?? createLogger('pxe:bb:wasm:bundle', { actor }); diff --git a/yarn-project/pxe/src/entrypoints/client/store.ts b/yarn-project/pxe/src/entrypoints/client/store.ts new file mode 100644 index 000000000000..e6447377fcdd --- /dev/null +++ b/yarn-project/pxe/src/entrypoints/client/store.ts @@ -0,0 +1,41 @@ +import type { EthAddress } from '@aztec/foundation/eth-address'; +import { type Logger, createLogger } from '@aztec/foundation/log'; +import { AztecSQLiteOPFSStore, storePoolDirectory } from '@aztec/kv-store/sqlite-opfs'; + +import { assertStoreIdentity, effectiveStoreName, requireCompleteIdentity } from '../../storage/store_identity.js'; + +/** + * Opens the persistent browser (sqlite-opfs) store selected by `name` and the identity `(config.l1ChainId, + * config.rollupAddress, schemaVersion)`. A store exists per identity: reopening with the same identity returns the + * same data, a different identity selects a different (possibly fresh) store. Nothing is ever cleared. + * + * @throws If `config.rollupAddress` or `config.l1ChainId` is missing — an incomplete identity must never silently + * default to the zero identity. + */ +export async function openPXEBrowserStore( + name: string, + schemaVersion: number, + config: { l1ChainId?: number; rollupAddress?: EthAddress; dataStoreMapSizeKb?: number }, + log: Logger = createLogger('pxe:data'), +): Promise { + const identity = requireCompleteIdentity(name, config, schemaVersion); + const storeName = effectiveStoreName(name, identity); + log.info(`Creating ${storeName} SQLite-OPFS data store`, { + storeName, + dataStoreMapSizeKb: config.dataStoreMapSizeKb, + }); + const store = await AztecSQLiteOPFSStore.open( + createLogger('kv-store:sqlite-opfs'), + storeName, + false, + storePoolDirectory(storeName), + ); + try { + await assertStoreIdentity(store, storeName, identity); + } catch (err) { + // The store handle owns a worker and OPFS locks; release them before surfacing the refusal. + await store.close().catch(() => {}); + throw err; + } + return store; +} diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 95327737a79e..1a5edea65081 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -5,6 +5,8 @@ import { type AztecLMDBStoreV2, openStoreAt, openTmpStore } from '@aztec/kv-stor import { mkdir } from 'fs/promises'; import { join } from 'path'; +import { storeIdentitySlug } from '../../storage/store_identity.js'; + /** Location and identity inputs for opening an identity-partitioned PXE-side store. */ export type IdentityStoreConfig = { dataDirectory?: string; @@ -37,7 +39,7 @@ export async function openPXEStore( const subDir = join( config.dataDirectory, `${name}-stores`, - `${config.l1ChainId}-${config.rollupAddress.toString()}-v${schemaVersion}`, + storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress: config.rollupAddress, schemaVersion }), ); await mkdir(subDir, { recursive: true }); createLogger(`pxe:data:${name}`, bindings).info(`Opening ${name} data store (LMDB v2)`, { From 57cb4b522586a23a230280656f10c8e4e2138fd1 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 09:54:05 +0000 Subject: [PATCH 33/48] refactor(kv-store): drop identity-aware createStore from the browser backends --- .../src/deprecated/indexeddb/index.ts | 27 +--- .../src/sqlite-opfs/create_store.test.ts | 136 ------------------ .../kv-store/src/sqlite-opfs/index.ts | 79 +--------- .../src/sqlite-opfs/store_identity.test.ts | 34 ----- .../src/sqlite-opfs/store_identity.ts | 42 ------ .../src/sqlite-opfs/store_management.test.ts | 56 ++++++++ 6 files changed, 58 insertions(+), 316 deletions(-) delete mode 100644 yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts delete mode 100644 yarn-project/kv-store/src/sqlite-opfs/store_identity.test.ts delete mode 100644 yarn-project/kv-store/src/sqlite-opfs/store_identity.ts create mode 100644 yarn-project/kv-store/src/sqlite-opfs/store_management.test.ts diff --git a/yarn-project/kv-store/src/deprecated/indexeddb/index.ts b/yarn-project/kv-store/src/deprecated/indexeddb/index.ts index 7ce637b8ab0c..e2ab6b6d5ad3 100644 --- a/yarn-project/kv-store/src/deprecated/indexeddb/index.ts +++ b/yarn-project/kv-store/src/deprecated/indexeddb/index.ts @@ -1,34 +1,9 @@ -import { type Logger, createLogger } from '@aztec/foundation/log'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; +import { createLogger } from '@aztec/foundation/log'; -import { initStoreForRollupAndSchemaVersion } from '../../utils.js'; import { AztecIndexedDBStore } from './store.js'; export { AztecIndexedDBStore } from './store.js'; -/** - * @deprecated The IndexedDB backend is being retired. Use `@aztec/kv-store/sqlite-opfs` instead. - */ -export async function createStore( - name: string, - config: DataStoreConfig, - schemaVersion: number | undefined = undefined, - log: Logger = createLogger('kv-store'), -) { - let { dataDirectory } = config; - if (typeof dataDirectory !== 'undefined') { - dataDirectory = `${dataDirectory}/${name}`; - } - - log.info( - dataDirectory - ? `Creating ${name} data store at directory ${dataDirectory} with map size ${config.dataStoreMapSizeKb} KB` - : `Creating ${name} ephemeral data store with map size ${config.dataStoreMapSizeKb} KB`, - ); - const store = await AztecIndexedDBStore.open(createLogger('kv-store:indexeddb'), dataDirectory ?? '', false); - return initStoreForRollupAndSchemaVersion(store, schemaVersion, config.rollupAddress, log); -} - /** * @deprecated The IndexedDB backend is being retired. Use `@aztec/kv-store/sqlite-opfs` instead. */ diff --git a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts b/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts deleted file mode 100644 index f73546e421d4..000000000000 --- a/yarn-project/kv-store/src/sqlite-opfs/create_store.test.ts +++ /dev/null @@ -1,136 +0,0 @@ -import { EthAddress } from '@aztec/foundation/eth-address'; - -import { mockLogger } from '../interfaces/utils.js'; -import { AztecSQLiteOPFSStore, StoreIdentityMismatchError, createStore, effectiveStoreName } from './index.js'; -import { deleteStore, listStores, storePoolDirectory } from './manage.js'; - -const configFor = (rollupAddress: EthAddress, l1ChainId = 31337) => ({ - dataDirectory: 'test', - dataStoreMapSizeKb: 1024, - rollupAddress, - l1ChainId, -}); - -describe('sqlite-opfs createStore', () => { - it('rejects when a required identity component is missing', async () => { - const addr = EthAddress.random(); - const { l1ChainId: _l1ChainId, ...configWithoutChainId } = configFor(addr); - const { rollupAddress: _rollupAddress, ...configWithoutRollup } = configFor(addr); - - await expect(createStore('incomplete_test', configWithoutChainId, 1, mockLogger)).rejects.toThrow( - /without a complete identity/, - ); - await expect(createStore('incomplete_test', configWithoutRollup, 1, mockLogger)).rejects.toThrow( - /without a complete identity/, - ); - await expect(createStore('incomplete_test', configFor(addr), undefined, mockLogger)).rejects.toThrow( - /without a complete identity/, - ); - - const storeName = effectiveStoreName('incomplete_test', { - l1ChainId: 31337, - rollupAddress: addr, - schemaVersion: 1, - }); - expect(await listStores()).not.toContain(storeName); - }); - - it('keeps data intact when switching rollup addresses back and forth', async () => { - const addrA = EthAddress.random(); - const addrB = EthAddress.random(); - - const storeA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); - await storeA.openSingleton('payload').set('data-for-A'); - await storeA.close(); - - const storeB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); - expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); - await storeB.openSingleton('payload').set('data-for-B'); - await storeB.close(); - - const reopenedA = await createStore('roundtrip_test', configFor(addrA), 1, mockLogger); - expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); - await reopenedA.close(); - - const reopenedB = await createStore('roundtrip_test', configFor(addrB), 1, mockLogger); - expect(await reopenedB.openSingleton('payload').getAsync()).toEqual('data-for-B'); - await reopenedB.close(); - }); - - it('opens two different stores concurrently in the same tab', async () => { - const addr = EthAddress.random(); - const pxeStore = await createStore('pxe_data', configFor(addr), 1, mockLogger); - const walletStore = await createStore('wallet_data', configFor(addr), 1, mockLogger); - - await pxeStore.openSingleton('k').set('pxe'); - await walletStore.openSingleton('k').set('wallet'); - expect(await pxeStore.openSingleton('k').getAsync()).toEqual('pxe'); - expect(await walletStore.openSingleton('k').getAsync()).toEqual('wallet'); - - await pxeStore.close(); - await walletStore.close(); - }); - - it('separates stores by schema version', async () => { - const addr = EthAddress.random(); - const v1 = await createStore('schema_test', configFor(addr), 1, mockLogger); - await v1.openSingleton('k').set('v1-data'); - await v1.close(); - - const v2 = await createStore('schema_test', configFor(addr), 2, mockLogger); - expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); - await v2.close(); - - const v1Again = await createStore('schema_test', configFor(addr), 1, mockLogger); - expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); - await v1Again.close(); - }); - - it('refuses to open on a recorded-identity mismatch and leaves data untouched', async () => { - const addr = EthAddress.random(); - const store = await createStore('mismatch_test', configFor(addr), 1, mockLogger); - await store.openSingleton('payload').set('precious'); - // Simulate a naming bug by corrupting the recorded identity. - await store.openSingleton('dbVersion').set('garbage'); - await store.close(); - - await expect(createStore('mismatch_test', configFor(addr), 1, mockLogger)).rejects.toThrow( - StoreIdentityMismatchError, - ); - - // The refusal must not have modified the store: read it raw, bypassing the identity check. - const storeName = effectiveStoreName('mismatch_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); - const raw = await AztecSQLiteOPFSStore.open(mockLogger, storeName, false, storePoolDirectory(storeName)); - expect(await raw.openSingleton('payload').getAsync()).toEqual('precious'); - await raw.close(); - }); - - it('lists created stores and deletes them', async () => { - const addr = EthAddress.random(); - const store = await createStore('managed_test', configFor(addr), 1, mockLogger); - await store.openSingleton('k').set('v'); - await store.close(); - - const storeName = effectiveStoreName('managed_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); - expect(await listStores()).toContain(storeName); - - await deleteStore(storeName); - expect(await listStores()).not.toContain(storeName); - - // Recreating after deletion starts empty. - const fresh = await createStore('managed_test', configFor(addr), 1, mockLogger); - expect(await fresh.openSingleton('k').getAsync()).toBeUndefined(); - await fresh.close(); - }); - - it('refuses to delete a store that is currently open', async () => { - const addr = EthAddress.random(); - const store = await createStore('locked_test', configFor(addr), 1, mockLogger); - const storeName = effectiveStoreName('locked_test', { l1ChainId: 31337, rollupAddress: addr, schemaVersion: 1 }); - - await expect(deleteStore(storeName)).rejects.toThrow(); - - await store.close(); - await deleteStore(storeName); - }); -}); diff --git a/yarn-project/kv-store/src/sqlite-opfs/index.ts b/yarn-project/kv-store/src/sqlite-opfs/index.ts index 7b6b3fd23913..65074aa2f865 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/index.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/index.ts @@ -1,88 +1,11 @@ -import type { EthAddress } from '@aztec/foundation/eth-address'; -import { type Logger, createLogger } from '@aztec/foundation/log'; -import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; -import type { DataStoreConfig } from '@aztec/stdlib/kv-store'; +import { createLogger } from '@aztec/foundation/log'; -import { storePoolDirectory } from './manage.js'; import { AztecSQLiteOPFSStore } from './store.js'; -import { StoreIdentityMismatchError, effectiveStoreName } from './store_identity.js'; export { AztecSQLiteOPFSStore } from './store.js'; export { SqliteEncryptionError } from './errors.js'; export type { SqliteEncryptionErrorCode } from './errors.js'; export { OPFS_POOL_DIR_PREFIX, deleteStore, listStores, storePoolDirectory } from './manage.js'; -export { StoreIdentityMismatchError, effectiveStoreName, storeIdentitySlug } from './store_identity.js'; -export type { StoreIdentity } from './store_identity.js'; - -/** - * Opens the persistent store selected by `name` and the identity `(config.l1ChainId, config.rollupAddress, - * schemaVersion)`. A store exists per identity: reopening with the same identity returns the same data, a - * different identity selects a different (possibly fresh) store. Nothing is ever cleared. - * - * @throws If `schemaVersion`, `config.rollupAddress`, or `config.l1ChainId` is missing — an incomplete identity - * must never silently default to the zero identity. - */ -export async function createStore( - name: string, - config: DataStoreConfig & { l1ChainId?: number }, - schemaVersion: number | undefined = undefined, - log: Logger = createLogger('kv-store'), -) { - const { rollupAddress, l1ChainId } = config; - if (schemaVersion === undefined || rollupAddress === undefined || l1ChainId === undefined) { - const missing = [ - l1ChainId === undefined && 'l1ChainId', - rollupAddress === undefined && 'rollupAddress', - schemaVersion === undefined && 'schemaVersion', - ].filter((component): component is string => component !== false); - throw new Error(`Cannot open store '${name}' without a complete identity: missing ${missing.join(', ')}`); - } - - const storeName = effectiveStoreName(name, { l1ChainId, rollupAddress, schemaVersion }); - log.info(`Creating ${storeName} SQLite-OPFS data store with map size ${config.dataStoreMapSizeKb} KB`); - const store = await AztecSQLiteOPFSStore.open( - createLogger('kv-store:sqlite-opfs'), - storeName, - false, - storePoolDirectory(storeName), - ); - try { - await assertStoreIdentity(store, storeName, schemaVersion, rollupAddress); - } catch (err) { - // The store handle owns a worker and OPFS locks; release them before surfacing the refusal. - await store.close().catch(() => {}); - throw err; - } - return store; -} - -/** - * Belt-and-braces invariant check: the identity is part of the physical store name, so the recorded version - * can only disagree if there is a store-naming bug. Refuses to open on mismatch; never clears. - */ -async function assertStoreIdentity( - store: AztecSQLiteOPFSStore, - storeName: string, - schemaVersion: number, - rollupAddress: EthAddress, -): Promise { - const expected = new DatabaseVersion(schemaVersion, rollupAddress); - const singleton = store.openSingleton('dbVersion'); - const stored = await singleton.getAsync(); - if (stored === undefined) { - await singleton.set(expected.toBuffer().toString('utf-8')); - return; - } - let storedVersion: DatabaseVersion; - try { - storedVersion = DatabaseVersion.fromBuffer(Buffer.from(stored, 'utf-8')); - } catch { - throw new StoreIdentityMismatchError(storeName, expected.toString(), stored); - } - if (!storedVersion.equals(expected)) { - throw new StoreIdentityMismatchError(storeName, expected.toString(), storedVersion.toString()); - } -} export function openTmpStore(ephemeral: boolean = false): Promise { return AztecSQLiteOPFSStore.open(createLogger('kv-store:sqlite-opfs'), undefined, ephemeral); diff --git a/yarn-project/kv-store/src/sqlite-opfs/store_identity.test.ts b/yarn-project/kv-store/src/sqlite-opfs/store_identity.test.ts deleted file mode 100644 index 6da2aa10b94f..000000000000 --- a/yarn-project/kv-store/src/sqlite-opfs/store_identity.test.ts +++ /dev/null @@ -1,34 +0,0 @@ -import { EthAddress } from '@aztec/foundation/eth-address'; - -import { effectiveStoreName, storeIdentitySlug } from './store_identity.js'; - -describe('storeIdentitySlug', () => { - it('composes chain id, rollup address and schema version', () => { - const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); - expect(storeIdentitySlug({ l1ChainId: 31337, rollupAddress, schemaVersion: 12 })).toEqual( - '31337-0x1234567890abcdef1234567890abcdef12345678-v12', - ); - }); - - it('composes the exact slug format for the zero identity', () => { - expect(storeIdentitySlug({ l1ChainId: 0, rollupAddress: EthAddress.ZERO, schemaVersion: 0 })).toEqual( - '0-0x0000000000000000000000000000000000000000-v0', - ); - }); - - it('normalizes the rollup address to lowercase hex', () => { - const rollupAddress = EthAddress.fromString('0x1234567890ABCDEF1234567890ABCDEF12345678'); - expect(storeIdentitySlug({ l1ChainId: 0, rollupAddress, schemaVersion: 1 })).toEqual( - `0-0x1234567890abcdef1234567890abcdef12345678-v1`, - ); - }); -}); - -describe('effectiveStoreName', () => { - it('joins the logical name and the slug with an underscore', () => { - const rollupAddress = EthAddress.fromString('0x1234567890abcdef1234567890abcdef12345678'); - expect(effectiveStoreName('pxe_data', { l1ChainId: 1, rollupAddress, schemaVersion: 2 })).toEqual( - 'pxe_data_1-0x1234567890abcdef1234567890abcdef12345678-v2', - ); - }); -}); diff --git a/yarn-project/kv-store/src/sqlite-opfs/store_identity.ts b/yarn-project/kv-store/src/sqlite-opfs/store_identity.ts deleted file mode 100644 index 4a988236c0be..000000000000 --- a/yarn-project/kv-store/src/sqlite-opfs/store_identity.ts +++ /dev/null @@ -1,42 +0,0 @@ -import type { EthAddress } from '@aztec/foundation/eth-address'; - -/** The coordinates that determine which physical store a logical store name maps to. */ -export type StoreIdentity = { - /** Chain ID of the L1 the rollup is deployed to. */ - l1ChainId: number; - /** Address of the rollup contract the store's data pertains to. */ - rollupAddress: EthAddress; - /** Schema version of the data held in the store. */ - schemaVersion: number; -}; - -/** - * Composes the store-name discriminator for a store identity. Two identities map to the same physical store iff - * their slugs are equal, so the format must stay stable: `--v`. - */ -export function storeIdentitySlug({ l1ChainId, rollupAddress, schemaVersion }: StoreIdentity): string { - return `${l1ChainId}-${rollupAddress.toString()}-v${schemaVersion}`; -} - -/** Composes the physical store name for a logical store name and identity. */ -export function effectiveStoreName(name: string, identity: StoreIdentity): string { - return `${name}_${storeIdentitySlug(identity)}`; -} - -/** - * Thrown when a store's recorded identity does not match the identity it was opened under. Since the identity is - * part of the physical store name, this can only indicate a store-naming bug; the store is left untouched. - */ -export class StoreIdentityMismatchError extends Error { - constructor( - public readonly storeName: string, - public readonly expected: string, - public readonly actual: string, - ) { - super( - `Store '${storeName}' records identity ${actual} but was opened as ${expected}. ` + - `Refusing to open; data was NOT modified.`, - ); - this.name = 'StoreIdentityMismatchError'; - } -} diff --git a/yarn-project/kv-store/src/sqlite-opfs/store_management.test.ts b/yarn-project/kv-store/src/sqlite-opfs/store_management.test.ts new file mode 100644 index 000000000000..4a99674e9b38 --- /dev/null +++ b/yarn-project/kv-store/src/sqlite-opfs/store_management.test.ts @@ -0,0 +1,56 @@ +import { mockLogger } from '../interfaces/utils.js'; +import { AztecSQLiteOPFSStore } from './index.js'; +import { deleteStore, listStores, storePoolDirectory } from './manage.js'; + +const openByName = (name: string) => AztecSQLiteOPFSStore.open(mockLogger, name, false, storePoolDirectory(name)); + +describe('sqlite-opfs store management', () => { + it('round-trips data for a store reopened by name', async () => { + const store = await openByName('mech_roundtrip'); + await store.openSingleton('payload').set('data'); + await store.close(); + + const reopened = await openByName('mech_roundtrip'); + expect(await reopened.openSingleton('payload').getAsync()).toEqual('data'); + await reopened.close(); + await deleteStore('mech_roundtrip'); + }); + + it('opens two different stores concurrently in the same tab', async () => { + const a = await openByName('mech_concurrent_a'); + const b = await openByName('mech_concurrent_b'); + + await a.openSingleton('k').set('a'); + await b.openSingleton('k').set('b'); + expect(await a.openSingleton('k').getAsync()).toEqual('a'); + expect(await b.openSingleton('k').getAsync()).toEqual('b'); + + await a.close(); + await b.close(); + await deleteStore('mech_concurrent_a'); + await deleteStore('mech_concurrent_b'); + }); + + it('lists created stores and deletes them', async () => { + const store = await openByName('mech_managed'); + await store.openSingleton('k').set('v'); + await store.close(); + + expect(await listStores()).toContain('mech_managed'); + await deleteStore('mech_managed'); + expect(await listStores()).not.toContain('mech_managed'); + + // Recreating after deletion starts empty. + const fresh = await openByName('mech_managed'); + expect(await fresh.openSingleton('k').getAsync()).toBeUndefined(); + await fresh.close(); + await deleteStore('mech_managed'); + }); + + it('refuses to delete a store that is currently open', async () => { + const store = await openByName('mech_locked'); + await expect(deleteStore('mech_locked')).rejects.toThrow(); + await store.close(); + await deleteStore('mech_locked'); + }); +}); From 278818c792a10ede9cb372872f57b9fbc1fe1971 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 10:15:31 +0000 Subject: [PATCH 34/48] fix(playground): open the playground store by name without identity createStore was deleted from @aztec/kv-store/sqlite-opfs when store selection moved into @aztec/pxe, breaking the playground build. The playground_data store is chain-agnostic, so open it directly with AztecSQLiteOPFSStore.open by a fixed name, the same pattern the embedded wallet uses for its chain-agnostic wallet_data store. --- .../components/navbar/components/NetworkSelector.tsx | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/playground/src/components/navbar/components/NetworkSelector.tsx b/playground/src/components/navbar/components/NetworkSelector.tsx index 05854ca06611..0d707ef3d4ba 100644 --- a/playground/src/components/navbar/components/NetworkSelector.tsx +++ b/playground/src/components/navbar/components/NetworkSelector.tsx @@ -7,7 +7,7 @@ import AddIcon from '@mui/icons-material/Add'; import { AddNetworksDialog } from './AddNetworkDialog'; import CircularProgress from '@mui/material/CircularProgress'; import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown'; -import { createStore } from '@aztec/kv-store/sqlite-opfs'; +import { AztecSQLiteOPFSStore, storePoolDirectory } from '@aztec/kv-store/sqlite-opfs'; import { AztecContext } from '../../../aztecContext'; import { navbarButtonStyle, navbarSelect } from '../../../styles/common'; import { NETWORKS } from '../../../utils/networks'; @@ -50,10 +50,12 @@ export function NetworkSelector() { } setIsContextInitialized(true); WebLogger.create(setLogs, setTotalLogCount); - const store = await createStore('playground_data', { - dataDirectory: 'playground', - dataStoreMapSizeKb: 1e6, - }); + const store = await AztecSQLiteOPFSStore.open( + WebLogger.getInstance().createLogger('playground_data'), + 'playground_data', + false, + storePoolDirectory('playground_data'), + ); const playgroundDB = PlaygroundDB.getInstance(); playgroundDB.init(store, WebLogger.getInstance().createLogger('playground_db').info); setPlaygroundDB(PlaygroundDB.getInstance()); From a20070f45e542d109234784b875944dc9bd146e2 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 10:15:36 +0000 Subject: [PATCH 35/48] docs: point the indexeddb migration note at the store open APIs createStore no longer exists in @aztec/kv-store/sqlite-opfs or @aztec/kv-store/deprecated/indexeddb. Update the TBD migration entry to show the current AztecSQLiteOPFSStore.open / AztecIndexedDBStore.open APIs instead. --- docs/docs-developers/docs/resources/migration_notes.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index a452a9a4b4b3..de51e11772ba 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -216,14 +216,20 @@ The browser PXE data store and the embedded wallet (`@aztec/wallets`) now persis ```diff - import { createStore } from '@aztec/kv-store/indexeddb'; -+ import { createStore } from '@aztec/kv-store/sqlite-opfs'; +- const store = await createStore(name); ++ import { AztecSQLiteOPFSStore } from '@aztec/kv-store/sqlite-opfs'; ++ const store = await AztecSQLiteOPFSStore.open(logger, name); ``` +Use `openTmpStore()` from the same entrypoint for an ephemeral store. + If you must stay on IndexedDB for now, import from the deprecated entrypoint instead: ```diff - import { createStore } from '@aztec/kv-store/indexeddb'; -+ import { createStore } from '@aztec/kv-store/deprecated/indexeddb'; +- const store = await createStore(name); ++ import { AztecIndexedDBStore } from '@aztec/kv-store/deprecated/indexeddb'; ++ const store = await AztecIndexedDBStore.open(logger, name); ``` **Impact**: Existing IndexedDB-backed data is not migrated, so browser PXE and wallet state starts fresh on SQLite-OPFS (the v5 protocol upgrade wipes local state regardless). SQLite-OPFS also holds an exclusive, origin-wide lock on its store directory, so a second browser tab opening the same store will fail. Consequently, we recommend to explicitly manage this case in your app if it uses `EmbeddedWallet`. From 097e11187526977b748eff8390c616ef1c5621bb Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 10:15:42 +0000 Subject: [PATCH 36/48] chore(kv-store): drop identity wording from the listStores jsdoc kv-store no longer knows about caller identity; that logic moved to @aztec/pxe. Reword the first sentence of listStores' jsdoc to describe the store names it lists without referencing identity. --- yarn-project/kv-store/src/sqlite-opfs/manage.ts | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/yarn-project/kv-store/src/sqlite-opfs/manage.ts b/yarn-project/kv-store/src/sqlite-opfs/manage.ts index ff524172b684..af99e9062dd3 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/manage.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/manage.ts @@ -11,9 +11,9 @@ export function storePoolDirectory(effectiveName: string): string { } /** - * Lists the effective names (logical name + identity slug) of every persistent sqlite-opfs store in this - * origin, by enumerating the per-store pool directories. Includes stores created under identities other than - * the current one, so wallets can surface and clean up data for networks no longer in use. + * Lists the names of every persistent sqlite-opfs store in this origin, by enumerating the per-store pool + * directories. Includes stores other than the current one, so wallets can surface and clean up data for + * networks no longer in use. */ export async function listStores(): Promise { const root = await navigator.storage.getDirectory(); From f9885e9a0824d02cba5b61e07e43f31d67a6f158 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 10:45:38 +0000 Subject: [PATCH 37/48] polish migration notes --- docs/docs-developers/docs/resources/migration_notes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index de51e11772ba..40092ac24532 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -11,9 +11,9 @@ Aztec is in active development. Each version may introduce breaking changes that ### [PXE] Stores are now selected by `(l1ChainId, rollupAddress, schemaVersion)` instead of being wiped on mismatch -Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores (`pxe_data`) now exist per `(l1ChainId, rollupAddress, schemaVersion)` identity, and switching networks (or upgrading) selects the matching store instead of overwriting the previous one. The embedded wallet's `wallet_data` store takes a different approach: since its contents (account secrets and aliases) are not network-specific, it is now chain-agnostic: a single schema-versioned store (`wallet_data_v1`) shared across networks, so stored accounts and aliases survive switching networks. +Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores now exist per `(l1ChainId, rollupAddress, schemaVersion)` triple, and switching networks (or upgrading) selects or creates the matching store instead of overwriting previous ones. The embedded wallet's `wallet_data` store takes a different approach: since its contents (account secrets and aliases) are not network-specific, it is now chain-agnostic: a single schema-versioned store (`wallet_data_v1`) shared across networks, so stored accounts and aliases survive switching networks. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. On Node.js environments (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. The embedded Node.js wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where your old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with the chain-agnostic wallet store at `/wallet_data_v1` on Node.js and under the fixed name `wallet_data_v1` in browser OPFS. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. On Node.js environments (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. The embedded Node.js wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where the old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with the chain-agnostic wallet store at `/wallet_data_v1` on Node.js and under the fixed name `wallet_data_v1` in browser OPFS. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: ```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; From de95d5bdc66b75b10992463e496e1dcd572ae2bb Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 11:00:02 +0000 Subject: [PATCH 38/48] polish --- yarn-project/kv-store/src/sqlite-opfs/manage.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/yarn-project/kv-store/src/sqlite-opfs/manage.ts b/yarn-project/kv-store/src/sqlite-opfs/manage.ts index af99e9062dd3..2f2f24fe02e3 100644 --- a/yarn-project/kv-store/src/sqlite-opfs/manage.ts +++ b/yarn-project/kv-store/src/sqlite-opfs/manage.ts @@ -13,7 +13,7 @@ export function storePoolDirectory(effectiveName: string): string { /** * Lists the names of every persistent sqlite-opfs store in this origin, by enumerating the per-store pool * directories. Includes stores other than the current one, so wallets can surface and clean up data for - * networks no longer in use. + * networks/versions no longer in use. */ export async function listStores(): Promise { const root = await navigator.storage.getDirectory(); From 9db18525601a0ae6a3597c703daed73c2bf168f5 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 11:14:54 +0000 Subject: [PATCH 39/48] polish --- yarn-project/pxe/src/entrypoints/client/store.ts | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/client/store.ts b/yarn-project/pxe/src/entrypoints/client/store.ts index e6447377fcdd..75fdbc662883 100644 --- a/yarn-project/pxe/src/entrypoints/client/store.ts +++ b/yarn-project/pxe/src/entrypoints/client/store.ts @@ -5,12 +5,11 @@ import { AztecSQLiteOPFSStore, storePoolDirectory } from '@aztec/kv-store/sqlite import { assertStoreIdentity, effectiveStoreName, requireCompleteIdentity } from '../../storage/store_identity.js'; /** - * Opens the persistent browser (sqlite-opfs) store selected by `name` and the identity `(config.l1ChainId, - * config.rollupAddress, schemaVersion)`. A store exists per identity: reopening with the same identity returns the - * same data, a different identity selects a different (possibly fresh) store. Nothing is ever cleared. + * Opens the persistent browser (sqlite-opfs) store selected by `name` and identity `(config.l1ChainId, + * config.rollupAddress, schemaVersion)` triple. A store exists per identity: reopening with the same identity returns + * the same data, a different identity selects a different (possibly fresh) store. * - * @throws If `config.rollupAddress` or `config.l1ChainId` is missing — an incomplete identity must never silently - * default to the zero identity. + * @throws If `config.rollupAddress` or `config.l1ChainId` are missing. */ export async function openPXEBrowserStore( name: string, From 52f947c1b15fe96659d7a2246a037fa776e8192b Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 11:19:10 +0000 Subject: [PATCH 40/48] polish --- yarn-project/pxe/src/entrypoints/server/store.ts | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 1a5edea65081..51c0e64ce7b4 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -17,15 +17,9 @@ export type IdentityStoreConfig = { }; /** - * Opens the persistent LMDB store selected by `name` and the identity `(l1ChainId, rollupAddress, schemaVersion)`. + * Opens the persistent LMDB store selected by `name` and identity triple `(l1ChainId, rollupAddress, schemaVersion)`. * A store exists per identity: reopening with the same identity returns the same data, a different identity selects - * a different (possibly fresh) store. Nothing is ever cleared, and no version marker is kept — the directory name is - * the identity. Falls back to an ephemeral tmp store when no data directory is configured. - * - * Stores live under `/-stores/--v`, a sibling of the - * legacy `/` directory: older binaries reset the legacy directory on a rollup mismatch, so - * per-identity stores must not nest inside it. Two identities select the same store iff their directory names are - * equal, so the name format must stay stable — changing it orphans every existing store. + * a different (possibly fresh) store. Falls back to an ephemeral tmp store when no data directory is configured. */ export async function openPXEStore( name: string, @@ -36,16 +30,22 @@ export async function openPXEStore( if (!config.dataDirectory) { return openTmpStore(name, true, config.dataStoreMapSizeKb, undefined, bindings); } + + // Stores live under `/-stores/--v`, a sibling of the + // legacy `/` directory: older binaries reset the legacy directory on a rollup mismatch, so + // per-identity stores must not nest inside it. const subDir = join( config.dataDirectory, `${name}-stores`, storeIdentitySlug({ l1ChainId: config.l1ChainId, rollupAddress: config.rollupAddress, schemaVersion }), ); await mkdir(subDir, { recursive: true }); + createLogger(`pxe:data:${name}`, bindings).info(`Opening ${name} data store (LMDB v2)`, { storeName: name, subDir, dataStoreMapSizeKb: config.dataStoreMapSizeKb, }); + return openStoreAt(subDir, config.dataStoreMapSizeKb, undefined, bindings); } From d18ad828548634b469346380bd5f9f9d7c2ac8ef Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 11:28:08 +0000 Subject: [PATCH 41/48] refactor(pxe): enforce complete store identity at compile time requireCompleteIdentity existed to guard kv-store createStore's optional public signature, which is now deleted. openPXEBrowserStore takes required identity fields like the node-side IdentityStoreConfig already does, and the browser call sites pass explicit values from getNodeInfo(), which the RPC layer validates. --- .../src/entrypoints/client/bundle/utils.ts | 12 +++++++- .../pxe/src/entrypoints/client/lazy/utils.ts | 12 +++++++- .../pxe/src/entrypoints/client/store.ts | 8 ++---- .../pxe/src/storage/store_identity.test.ts | 28 ------------------- .../pxe/src/storage/store_identity.ts | 24 +--------------- 5 files changed, 26 insertions(+), 58 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts index eb668d1d6eca..4b3ec173acc4 100644 --- a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts @@ -42,7 +42,17 @@ export async function createPXE( const storeLogger = loggers.store ?? createLogger('pxe:data', { actor }); const store = - options.store ?? (await openPXEBrowserStore('pxe_data', PXE_DATA_SCHEMA_VERSION, configWithContracts, storeLogger)); + options.store ?? + (await openPXEBrowserStore( + 'pxe_data', + PXE_DATA_SCHEMA_VERSION, + { + l1ChainId, + rollupAddress: l1ContractAddresses.rollupAddress, + dataStoreMapSizeKb: configWithContracts.dataStoreMapSizeKb, + }, + storeLogger, + )); const simulator = options.simulator ?? new WASMSimulator(); const proverLogger = loggers.prover ?? createLogger('pxe:bb:wasm:bundle', { actor }); diff --git a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts index 0b7cb1430af6..24447c51b55e 100644 --- a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts @@ -42,7 +42,17 @@ export async function createPXE( const storeLogger = loggers.store ?? createLogger('pxe:data', { actor }); const store = - options.store ?? (await openPXEBrowserStore('pxe_data', PXE_DATA_SCHEMA_VERSION, configWithContracts, storeLogger)); + options.store ?? + (await openPXEBrowserStore( + 'pxe_data', + PXE_DATA_SCHEMA_VERSION, + { + l1ChainId, + rollupAddress: l1ContractAddresses.rollupAddress, + dataStoreMapSizeKb: configWithContracts.dataStoreMapSizeKb, + }, + storeLogger, + )); const simulator = options.simulator ?? new WASMSimulator(); const proverLogger = loggers.prover ?? createLogger('pxe:bb:wasm:bundle', { actor }); diff --git a/yarn-project/pxe/src/entrypoints/client/store.ts b/yarn-project/pxe/src/entrypoints/client/store.ts index 75fdbc662883..14387f8c940a 100644 --- a/yarn-project/pxe/src/entrypoints/client/store.ts +++ b/yarn-project/pxe/src/entrypoints/client/store.ts @@ -2,22 +2,20 @@ import type { EthAddress } from '@aztec/foundation/eth-address'; import { type Logger, createLogger } from '@aztec/foundation/log'; import { AztecSQLiteOPFSStore, storePoolDirectory } from '@aztec/kv-store/sqlite-opfs'; -import { assertStoreIdentity, effectiveStoreName, requireCompleteIdentity } from '../../storage/store_identity.js'; +import { assertStoreIdentity, effectiveStoreName } from '../../storage/store_identity.js'; /** * Opens the persistent browser (sqlite-opfs) store selected by `name` and identity `(config.l1ChainId, * config.rollupAddress, schemaVersion)` triple. A store exists per identity: reopening with the same identity returns * the same data, a different identity selects a different (possibly fresh) store. - * - * @throws If `config.rollupAddress` or `config.l1ChainId` are missing. */ export async function openPXEBrowserStore( name: string, schemaVersion: number, - config: { l1ChainId?: number; rollupAddress?: EthAddress; dataStoreMapSizeKb?: number }, + config: { l1ChainId: number; rollupAddress: EthAddress; dataStoreMapSizeKb?: number }, log: Logger = createLogger('pxe:data'), ): Promise { - const identity = requireCompleteIdentity(name, config, schemaVersion); + const identity = { l1ChainId: config.l1ChainId, rollupAddress: config.rollupAddress, schemaVersion }; const storeName = effectiveStoreName(name, identity); log.info(`Creating ${storeName} SQLite-OPFS data store`, { storeName, diff --git a/yarn-project/pxe/src/storage/store_identity.test.ts b/yarn-project/pxe/src/storage/store_identity.test.ts index 2bc9984358ae..b8288bbf2232 100644 --- a/yarn-project/pxe/src/storage/store_identity.test.ts +++ b/yarn-project/pxe/src/storage/store_identity.test.ts @@ -5,7 +5,6 @@ import { StoreIdentityMismatchError, assertStoreIdentity, effectiveStoreName, - requireCompleteIdentity, storeIdentitySlug, } from './store_identity.js'; @@ -34,33 +33,6 @@ describe('effectiveStoreName', () => { }); }); -describe('requireCompleteIdentity', () => { - it('returns the identity when all components are present', () => { - const rollupAddress = EthAddress.random(); - expect(requireCompleteIdentity('s', { l1ChainId: 31337, rollupAddress }, 1)).toEqual({ - l1ChainId: 31337, - rollupAddress, - schemaVersion: 1, - }); - }); - - it('throws naming each missing component', () => { - const rollupAddress = EthAddress.random(); - expect(() => requireCompleteIdentity('s', { rollupAddress }, 1)).toThrow( - "Cannot open store 's' without a complete identity: missing l1ChainId", - ); - expect(() => requireCompleteIdentity('s', { l1ChainId: 31337 }, 1)).toThrow( - "Cannot open store 's' without a complete identity: missing rollupAddress", - ); - expect(() => requireCompleteIdentity('s', { l1ChainId: 31337, rollupAddress }, undefined)).toThrow( - "Cannot open store 's' without a complete identity: missing schemaVersion", - ); - expect(() => requireCompleteIdentity('s', {}, undefined)).toThrow( - "Cannot open store 's' without a complete identity: missing l1ChainId, rollupAddress, schemaVersion", - ); - }); -}); - describe('assertStoreIdentity', () => { const identityFor = (rollupAddress: EthAddress) => ({ l1ChainId: 31337, rollupAddress, schemaVersion: 1 }); diff --git a/yarn-project/pxe/src/storage/store_identity.ts b/yarn-project/pxe/src/storage/store_identity.ts index ef1c35fa6b59..9d359c4c4ed9 100644 --- a/yarn-project/pxe/src/storage/store_identity.ts +++ b/yarn-project/pxe/src/storage/store_identity.ts @@ -2,7 +2,7 @@ import type { EthAddress } from '@aztec/foundation/eth-address'; import type { AztecAsyncKVStore } from '@aztec/kv-store'; import { DatabaseVersion } from '@aztec/stdlib/database-version/version'; -/** The coordinates that determine which physical store a logical store name maps to. */ +/** The triple that determine which physical store a logical store name maps to. */ export type StoreIdentity = { /** Chain ID of the L1 the rollup is deployed to. */ l1ChainId: number; @@ -26,28 +26,6 @@ export function effectiveStoreName(name: string, identity: StoreIdentity): strin return `${name}_${storeIdentitySlug(identity)}`; } -/** - * Validates that every identity component is present, so an incomplete identity never silently defaults to the - * zero identity. - * @throws If `config.l1ChainId`, `config.rollupAddress`, or `schemaVersion` is missing. - */ -export function requireCompleteIdentity( - name: string, - config: { l1ChainId?: number; rollupAddress?: EthAddress }, - schemaVersion?: number, -): StoreIdentity { - const { l1ChainId, rollupAddress } = config; - if (l1ChainId === undefined || rollupAddress === undefined || schemaVersion === undefined) { - const missing = [ - l1ChainId === undefined && 'l1ChainId', - rollupAddress === undefined && 'rollupAddress', - schemaVersion === undefined && 'schemaVersion', - ].filter((component): component is string => component !== false); - throw new Error(`Cannot open store '${name}' without a complete identity: missing ${missing.join(', ')}`); - } - return { l1ChainId, rollupAddress, schemaVersion }; -} - /** * Belt-and-braces invariant check for stores whose physical name carries their identity: the recorded version can * only disagree with the expected one if there is a store-naming bug. Writes the marker on first open; refuses to From a00cf229959855bc93114ea1aa1655b5e687cfb5 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 11:35:25 +0000 Subject: [PATCH 42/48] polish --- yarn-project/pxe/src/storage/store_identity.ts | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/yarn-project/pxe/src/storage/store_identity.ts b/yarn-project/pxe/src/storage/store_identity.ts index 9d359c4c4ed9..6daeb4a2e5a7 100644 --- a/yarn-project/pxe/src/storage/store_identity.ts +++ b/yarn-project/pxe/src/storage/store_identity.ts @@ -27,10 +27,8 @@ export function effectiveStoreName(name: string, identity: StoreIdentity): strin } /** - * Belt-and-braces invariant check for stores whose physical name carries their identity: the recorded version can - * only disagree with the expected one if there is a store-naming bug. Writes the marker on first open; refuses to - * proceed on mismatch; never clears. The marker records `(schemaVersion, rollupAddress)` only — the store name - * carries the full identity. + * Invariant check for stores whose physical name carries their identity: the recorded version can + * only disagree with the expected one if there is a store-naming bug. */ export async function assertStoreIdentity( store: AztecAsyncKVStore, From 14b87838dc9d4bdbbabc12b768ab2bbecf5fa2b8 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 12:20:50 +0000 Subject: [PATCH 43/48] refactor(pxe): rename store helpers to openStore and openBrowserStore --- .../src/entrypoints/client/bundle/index.ts | 1 + .../src/entrypoints/client/bundle/utils.ts | 4 ++-- .../pxe/src/entrypoints/client/lazy/index.ts | 1 + .../pxe/src/entrypoints/client/lazy/utils.ts | 4 ++-- .../pxe/src/entrypoints/client/store.ts | 2 +- .../pxe/src/entrypoints/server/store.test.ts | 20 +++++++++---------- .../pxe/src/entrypoints/server/store.ts | 2 +- .../pxe/src/entrypoints/server/utils.ts | 4 ++-- 8 files changed, 20 insertions(+), 18 deletions(-) diff --git a/yarn-project/pxe/src/entrypoints/client/bundle/index.ts b/yarn-project/pxe/src/entrypoints/client/bundle/index.ts index 437bf2a74da3..12f498eabc41 100644 --- a/yarn-project/pxe/src/entrypoints/client/bundle/index.ts +++ b/yarn-project/pxe/src/entrypoints/client/bundle/index.ts @@ -6,3 +6,4 @@ export * from '../../../contract_logging.js'; export * from '../../../storage/index.js'; export * from './utils.js'; export type { PXECreationOptions } from '../../pxe_creation_options.js'; +export { openBrowserStore } from '../store.js'; diff --git a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts index 4b3ec173acc4..7961cc7155ec 100644 --- a/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/bundle/utils.ts @@ -11,7 +11,7 @@ import type { PXEConfig } from '../../../config/index.js'; import { PXE } from '../../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../../storage/metadata.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../../pxe_creation_options.js'; -import { openPXEBrowserStore } from '../store.js'; +import { openBrowserStore } from '../store.js'; /** * Create and start an PXE instance with the given AztecNode. @@ -43,7 +43,7 @@ export async function createPXE( const store = options.store ?? - (await openPXEBrowserStore( + (await openBrowserStore( 'pxe_data', PXE_DATA_SCHEMA_VERSION, { diff --git a/yarn-project/pxe/src/entrypoints/client/lazy/index.ts b/yarn-project/pxe/src/entrypoints/client/lazy/index.ts index 3f417a8b5209..d9a339a33458 100644 --- a/yarn-project/pxe/src/entrypoints/client/lazy/index.ts +++ b/yarn-project/pxe/src/entrypoints/client/lazy/index.ts @@ -6,3 +6,4 @@ export * from '../../../error_enriching.js'; export * from '../../../contract_logging.js'; export * from './utils.js'; export { type PXECreationOptions } from '../../pxe_creation_options.js'; +export { openBrowserStore } from '../store.js'; diff --git a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts index 24447c51b55e..f9f228db638a 100644 --- a/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts +++ b/yarn-project/pxe/src/entrypoints/client/lazy/utils.ts @@ -11,7 +11,7 @@ import type { PXEConfig } from '../../../config/index.js'; import { PXE } from '../../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../../storage/metadata.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../../pxe_creation_options.js'; -import { openPXEBrowserStore } from '../store.js'; +import { openBrowserStore } from '../store.js'; /** * Create and start an PXE instance with the given AztecNode. @@ -43,7 +43,7 @@ export async function createPXE( const store = options.store ?? - (await openPXEBrowserStore( + (await openBrowserStore( 'pxe_data', PXE_DATA_SCHEMA_VERSION, { diff --git a/yarn-project/pxe/src/entrypoints/client/store.ts b/yarn-project/pxe/src/entrypoints/client/store.ts index 14387f8c940a..13d8583cab0d 100644 --- a/yarn-project/pxe/src/entrypoints/client/store.ts +++ b/yarn-project/pxe/src/entrypoints/client/store.ts @@ -9,7 +9,7 @@ import { assertStoreIdentity, effectiveStoreName } from '../../storage/store_ide * config.rollupAddress, schemaVersion)` triple. A store exists per identity: reopening with the same identity returns * the same data, a different identity selects a different (possibly fresh) store. */ -export async function openPXEBrowserStore( +export async function openBrowserStore( name: string, schemaVersion: number, config: { l1ChainId: number; rollupAddress: EthAddress; dataStoreMapSizeKb?: number }, diff --git a/yarn-project/pxe/src/entrypoints/server/store.test.ts b/yarn-project/pxe/src/entrypoints/server/store.test.ts index 70df459d1227..15b15215d2d2 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.test.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.test.ts @@ -4,9 +4,9 @@ import { mkdtemp, rm, stat } from 'fs/promises'; import { tmpdir } from 'os'; import { join } from 'path'; -import { openPXEStore } from './store.js'; +import { openStore } from './store.js'; -describe('openPXEStore', () => { +describe('openStore', () => { let dataDirectory: string; beforeEach(async () => { @@ -28,15 +28,15 @@ describe('openPXEStore', () => { const addrA = EthAddress.random(); const addrB = EthAddress.random(); - const storeA = await openPXEStore('test_store', 1, configFor(addrA)); + const storeA = await openStore('test_store', 1, configFor(addrA)); await storeA.openSingleton('payload').set('data-for-A'); await storeA.close(); - const storeB = await openPXEStore('test_store', 1, configFor(addrB)); + const storeB = await openStore('test_store', 1, configFor(addrB)); expect(await storeB.openSingleton('payload').getAsync()).toBeUndefined(); await storeB.close(); - const reopenedA = await openPXEStore('test_store', 1, configFor(addrA)); + const reopenedA = await openStore('test_store', 1, configFor(addrA)); expect(await reopenedA.openSingleton('payload').getAsync()).toEqual('data-for-A'); await reopenedA.close(); }); @@ -44,21 +44,21 @@ describe('openPXEStore', () => { it('separates stores by schema version', async () => { const addr = EthAddress.random(); - const v1 = await openPXEStore('test_store', 1, configFor(addr)); + const v1 = await openStore('test_store', 1, configFor(addr)); await v1.openSingleton('k').set('v1-data'); await v1.close(); - const v2 = await openPXEStore('test_store', 2, configFor(addr)); + const v2 = await openStore('test_store', 2, configFor(addr)); expect(await v2.openSingleton('k').getAsync()).toBeUndefined(); await v2.close(); - const v1Again = await openPXEStore('test_store', 1, configFor(addr)); + const v1Again = await openStore('test_store', 1, configFor(addr)); expect(await v1Again.openSingleton('k').getAsync()).toEqual('v1-data'); await v1Again.close(); }); it('places stores under a sibling -stores directory, not nested in ', async () => { - const store = await openPXEStore('test_store', 1, configFor(EthAddress.random())); + const store = await openStore('test_store', 1, configFor(EthAddress.random())); await store.close(); await expect(stat(join(dataDirectory, 'test_store-stores'))).resolves.toBeDefined(); @@ -66,7 +66,7 @@ describe('openPXEStore', () => { }); it('falls back to an ephemeral tmp store when no data directory is configured', async () => { - const store = await openPXEStore('test_store', 1, { + const store = await openStore('test_store', 1, { dataStoreMapSizeKb: 10 * 1024, l1ChainId: 31337, rollupAddress: EthAddress.random(), diff --git a/yarn-project/pxe/src/entrypoints/server/store.ts b/yarn-project/pxe/src/entrypoints/server/store.ts index 51c0e64ce7b4..0374f6c77398 100644 --- a/yarn-project/pxe/src/entrypoints/server/store.ts +++ b/yarn-project/pxe/src/entrypoints/server/store.ts @@ -21,7 +21,7 @@ export type IdentityStoreConfig = { * A store exists per identity: reopening with the same identity returns the same data, a different identity selects * a different (possibly fresh) store. Falls back to an ephemeral tmp store when no data directory is configured. */ -export async function openPXEStore( +export async function openStore( name: string, schemaVersion: number, config: IdentityStoreConfig, diff --git a/yarn-project/pxe/src/entrypoints/server/utils.ts b/yarn-project/pxe/src/entrypoints/server/utils.ts index 01ae45ddd283..3eb40c46df75 100644 --- a/yarn-project/pxe/src/entrypoints/server/utils.ts +++ b/yarn-project/pxe/src/entrypoints/server/utils.ts @@ -14,7 +14,7 @@ import type { PXEConfig } from '../../config/index.js'; import { PXE } from '../../pxe.js'; import { PXE_DATA_SCHEMA_VERSION } from '../../storage/index.js'; import { type PXECreationOptions, isPrivateKernelProver } from '../pxe_creation_options.js'; -import { openPXEStore } from './store.js'; +import { openStore } from './store.js'; type PXEConfigWithoutDefaults = Omit< PXEConfig, @@ -46,7 +46,7 @@ export async function createPXE( if (!options.store) { const storeLogger = loggers.store ?? createLogger('pxe:data:lmdb', { actor }); - options.store = await openPXEStore( + options.store = await openStore( 'pxe_data', PXE_DATA_SCHEMA_VERSION, { From 4bf6008ed4ed809d5a63959a041b0777958f69ab Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 12:26:52 +0000 Subject: [PATCH 44/48] feat(wallets): partition wallet_data by store identity like pxe_data --- .../src/embedded/entrypoints/browser.ts | 31 ++++++++++--------- .../wallets/src/embedded/entrypoints/node.ts | 27 ++++++++-------- .../wallets/src/embedded/wallet_db.ts | 6 ---- 3 files changed, 31 insertions(+), 33 deletions(-) diff --git a/yarn-project/wallets/src/embedded/entrypoints/browser.ts b/yarn-project/wallets/src/embedded/entrypoints/browser.ts index 8f2a503ff879..2b34f83ffbca 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/browser.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/browser.ts @@ -1,7 +1,7 @@ import { type AztecNode, createAztecNodeClient } from '@aztec/aztec.js/node'; import { type Logger, createLogger } from '@aztec/foundation/log'; -import { AztecSQLiteOPFSStore, openTmpStore, storePoolDirectory } from '@aztec/kv-store/sqlite-opfs'; -import { type PXE, type PXECreationOptions, createPXE } from '@aztec/pxe/client/lazy'; +import { openTmpStore } from '@aztec/kv-store/sqlite-opfs'; +import { type PXE, type PXECreationOptions, createPXE, openBrowserStore } from '@aztec/pxe/client/lazy'; import { type PXEConfig, getPXEConfig } from '@aztec/pxe/config'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry/lazy'; import { getStandardHandshakeRegistry } from '@aztec/standard-contracts/handshake-registry/lazy'; @@ -10,7 +10,7 @@ import { getStandardMultiCallEntrypoint } from '@aztec/standard-contracts/multi- import { LazyAccountContractsProvider } from '../account-contract-providers/lazy.js'; import type { AccountContractsProvider } from '../account-contract-providers/types.js'; import { EmbeddedWallet, type EmbeddedWalletOptions, splitPxeOptions } from '../embedded_wallet.js'; -import { WALLET_DATA_STORE_NAME, WalletDB } from '../wallet_db.js'; +import { WALLET_DATA_SCHEMA_VERSION, WalletDB } from '../wallet_db.js'; export class BrowserEmbeddedWallet extends EmbeddedWallet { static async create( @@ -64,17 +64,20 @@ export class BrowserEmbeddedWallet extends EmbeddedWallet { const pxe = await createPXE(aztecNode, pxeConfig, pxeOptions); - const walletDBStore = - options.walletDb?.store ?? - (options.ephemeral - ? await openTmpStore(true) - : await AztecSQLiteOPFSStore.open( - // Chain-agnostic store: keyed by a fixed schema-versioned name, not by network identity. - rootLogger.createChild('wallet:data'), - WALLET_DATA_STORE_NAME, - false, - storePoolDirectory(WALLET_DATA_STORE_NAME), - )); + let walletDBStore = options.walletDb?.store; + if (!walletDBStore) { + if (options.ephemeral) { + walletDBStore = await openTmpStore(true); + } else { + const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); + walletDBStore = await openBrowserStore( + 'wallet_data', + WALLET_DATA_SCHEMA_VERSION, + { l1ChainId, rollupAddress: l1ContractAddresses.rollupAddress }, + rootLogger.createChild('wallet:data'), + ); + } + } const walletDB = new WalletDB(walletDBStore, rootLogger.createChild('wallet:db').info); const wallet = new this(pxe, aztecNode, walletDB, new LazyAccountContractsProvider(), rootLogger) as T; diff --git a/yarn-project/wallets/src/embedded/entrypoints/node.ts b/yarn-project/wallets/src/embedded/entrypoints/node.ts index 45ad80bd1017..50ad7c21d32f 100644 --- a/yarn-project/wallets/src/embedded/entrypoints/node.ts +++ b/yarn-project/wallets/src/embedded/entrypoints/node.ts @@ -1,20 +1,16 @@ import { createAztecNodeClient } from '@aztec/aztec.js/node'; import { type Logger, createLogger } from '@aztec/foundation/log'; -import { openStoreAt, openTmpStore } from '@aztec/kv-store/lmdb-v2'; import { type PXEConfig, getPXEConfig } from '@aztec/pxe/config'; -import { type PXE, type PXECreationOptions, createPXE } from '@aztec/pxe/server'; +import { type PXE, type PXECreationOptions, createPXE, openStore } from '@aztec/pxe/server'; import { getStandardAuthRegistry } from '@aztec/standard-contracts/auth-registry'; import { getStandardHandshakeRegistry } from '@aztec/standard-contracts/handshake-registry'; import { getStandardMultiCallEntrypoint } from '@aztec/standard-contracts/multi-call-entrypoint'; import type { AztecNode } from '@aztec/stdlib/interfaces/client'; -import { mkdir } from 'fs/promises'; -import { join } from 'path'; - import { BundleAccountContractsProvider } from '../account-contract-providers/bundle.js'; import type { AccountContractsProvider } from '../account-contract-providers/types.js'; import { EmbeddedWallet, type EmbeddedWalletOptions, splitPxeOptions } from '../embedded_wallet.js'; -import { WALLET_DATA_STORE_NAME, WalletDB } from '../wallet_db.js'; +import { WALLET_DATA_SCHEMA_VERSION, WalletDB } from '../wallet_db.js'; const DEFAULT_WALLET_DATA_DIRECTORY = 'aztec-wallet-data'; @@ -72,13 +68,18 @@ export class NodeEmbeddedWallet extends EmbeddedWallet { let walletDBStore = options.walletDb?.store; if (!walletDBStore) { const bindings = rootLogger.createChild('wallet:data').getBindings(); - if (options.ephemeral) { - walletDBStore = await openTmpStore('wallet_data', true, undefined, undefined, bindings); - } else { - const walletDataDir = join(pxeConfig.dataDirectory ?? DEFAULT_WALLET_DATA_DIRECTORY, WALLET_DATA_STORE_NAME); - await mkdir(walletDataDir, { recursive: true }); - walletDBStore = await openStoreAt(walletDataDir, pxeConfig.dataStoreMapSizeKb, undefined, bindings); - } + const { l1ChainId, l1ContractAddresses } = await aztecNode.getNodeInfo(); + walletDBStore = await openStore( + 'wallet_data', + WALLET_DATA_SCHEMA_VERSION, + { + dataDirectory: options.ephemeral ? undefined : (pxeConfig.dataDirectory ?? DEFAULT_WALLET_DATA_DIRECTORY), + dataStoreMapSizeKb: pxeConfig.dataStoreMapSizeKb, + l1ChainId, + rollupAddress: l1ContractAddresses.rollupAddress, + }, + bindings, + ); } const walletDB = new WalletDB(walletDBStore, rootLogger.createChild('wallet:db').info); diff --git a/yarn-project/wallets/src/embedded/wallet_db.ts b/yarn-project/wallets/src/embedded/wallet_db.ts index 24de2b2238e0..3feba81ab538 100644 --- a/yarn-project/wallets/src/embedded/wallet_db.ts +++ b/yarn-project/wallets/src/embedded/wallet_db.ts @@ -14,12 +14,6 @@ function accountKey(field: string, address: AztecAddress | string): string { /** Bump when the WalletDB layout changes; a new version selects a fresh store, leaving the old one intact. */ export const WALLET_DATA_SCHEMA_VERSION = 1; -/** - * Name of the wallet DB store. Chain-agnostic: accounts and aliases apply across networks, so the name carries - * only the schema version. - */ -export const WALLET_DATA_STORE_NAME = `wallet_data_v${WALLET_DATA_SCHEMA_VERSION}`; - export class WalletDB { private accounts: AztecAsyncMap; private aliases: AztecAsyncMap; From 02e6e5cc7c783e2d8bc4374299d915811d2739bb Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 12:35:14 +0000 Subject: [PATCH 45/48] docs: wallet_data is partitioned by identity in the store-selection migration note --- docs/docs-developers/docs/resources/migration_notes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 40092ac24532..318e370a00d6 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -11,9 +11,9 @@ Aztec is in active development. Each version may introduce breaking changes that ### [PXE] Stores are now selected by `(l1ChainId, rollupAddress, schemaVersion)` instead of being wiped on mismatch -Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores now exist per `(l1ChainId, rollupAddress, schemaVersion)` triple, and switching networks (or upgrading) selects or creates the matching store instead of overwriting previous ones. The embedded wallet's `wallet_data` store takes a different approach: since its contents (account secrets and aliases) are not network-specific, it is now chain-agnostic: a single schema-versioned store (`wallet_data_v1`) shared across networks, so stored accounts and aliases survive switching networks. +Previously, connecting a PXE or embedded wallet to a different or redeployed rollup, or bumping the store schema version, wiped the existing on-disk store in place. That meant master account keys could be destroyed simply by pointing a wallet at a different network. PXE data stores now exist per `(l1ChainId, rollupAddress, schemaVersion)` triple, and switching networks (or upgrading) selects or creates the matching store instead of overwriting previous ones. The embedded wallet's `wallet_data` store is partitioned the same way, so accounts and aliases are per network: switching networks starts with an empty account list until accounts are re-imported, and switching back finds the originals intact. -**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. On Node.js environments (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. The embedded Node.js wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where the old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with the chain-agnostic wallet store at `/wallet_data_v1` on Node.js and under the fixed name `wallet_data_v1` in browser OPFS. Browser apps can enumerate and clean up `pxe_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: +**Impact**: The first start after upgrading to this version begins with a fresh, empty store; the pre-upgrade data is not deleted. On Node.js environments (lmdb-v2) pre-upgrade data stays at `/` while new per-identity `pxe_data` stores live under `/-stores/`. The embedded Node.js wallet previously stored data in cwd-relative, rollup-address-suffixed directories instead (`pxe_data_/pxe_data` for the PXE store, `wallet_data_/wallet_data` for the wallet store): if you used it before this release, that is where the old data lives. The embedded wallet now defaults its data root to `aztec-wallet-data/`, with per-identity wallet stores under `/wallet_data-stores/` on Node.js and OPFS store names prefixed `wallet_data_` in the browser. Browser apps can enumerate and clean up `pxe_data` and `wallet_data` stores for networks no longer in use with the new `listStores()` / `deleteStore()` utilities: ```ts import { deleteStore, listStores } from '@aztec/kv-store/sqlite-opfs'; From cf97f65785738e4f678cc01fd26ba7ea32a2d4a3 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 12:50:11 +0000 Subject: [PATCH 46/48] docs: restore the indexeddb migration entry verbatim Migration notes are append-only, including entries under TBD that this branch does not own; the createStore guidance correction will ride the entry that documents the API change instead. --- docs/docs-developers/docs/resources/migration_notes.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 318e370a00d6..5c317c39baeb 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -216,20 +216,14 @@ The browser PXE data store and the embedded wallet (`@aztec/wallets`) now persis ```diff - import { createStore } from '@aztec/kv-store/indexeddb'; -- const store = await createStore(name); -+ import { AztecSQLiteOPFSStore } from '@aztec/kv-store/sqlite-opfs'; -+ const store = await AztecSQLiteOPFSStore.open(logger, name); ++ import { createStore } from '@aztec/kv-store/sqlite-opfs'; ``` -Use `openTmpStore()` from the same entrypoint for an ephemeral store. - If you must stay on IndexedDB for now, import from the deprecated entrypoint instead: ```diff - import { createStore } from '@aztec/kv-store/indexeddb'; -- const store = await createStore(name); -+ import { AztecIndexedDBStore } from '@aztec/kv-store/deprecated/indexeddb'; -+ const store = await AztecIndexedDBStore.open(logger, name); ++ import { createStore } from '@aztec/kv-store/deprecated/indexeddb'; ``` **Impact**: Existing IndexedDB-backed data is not migrated, so browser PXE and wallet state starts fresh on SQLite-OPFS (the v5 protocol upgrade wipes local state regardless). SQLite-OPFS also holds an exclusive, origin-wide lock on its store directory, so a second browser tab opening the same store will fail. Consequently, we recommend to explicitly manage this case in your app if it uses `EmbeddedWallet`. From 1a0ba29246ba2c4caa9791f86d8bd21ab27101ec Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 12:51:59 +0000 Subject: [PATCH 47/48] docs: note the kv-store createStore removal in the store-selection entry --- docs/docs-developers/docs/resources/migration_notes.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 5c317c39baeb..f06ef174a9b3 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -22,6 +22,8 @@ const names = await listStores(); await deleteStore(names[0]); ``` +This change also removes `createStore` from `@aztec/kv-store/sqlite-opfs` and `@aztec/kv-store/deprecated/indexeddb` (superseding the import guidance in the SQLite-OPFS default entry below): stores are opened by name with `AztecSQLiteOPFSStore.open` / `AztecIndexedDBStore.open`, and PXE selects its per-identity stores internally. + ### [Aztec.nr] `TestEnvironmentOptions::with_tagging_secret_strategy` replaced `TestEnvironmentOptions::with_tagging_secret_strategy` is now `with_default_tag_secret_strategy_all_modes` for tests From c6a4241107d1133a0fe525bcfd7e77ccecfe9997 Mon Sep 17 00:00:00 2001 From: mverzilli Date: Fri, 10 Jul 2026 12:55:05 +0000 Subject: [PATCH 48/48] polish --- docs/docs-developers/docs/resources/migration_notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index f06ef174a9b3..81090922254b 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -22,7 +22,7 @@ const names = await listStores(); await deleteStore(names[0]); ``` -This change also removes `createStore` from `@aztec/kv-store/sqlite-opfs` and `@aztec/kv-store/deprecated/indexeddb` (superseding the import guidance in the SQLite-OPFS default entry below): stores are opened by name with `AztecSQLiteOPFSStore.open` / `AztecIndexedDBStore.open`, and PXE selects its per-identity stores internally. +This change also removes `createStore` from `@aztec/kv-store/sqlite-opfs` and `@aztec/kv-store/deprecated/indexeddb`: stores are now opened by name with `AztecSQLiteOPFSStore.open` / `AztecIndexedDBStore.open`. ### [Aztec.nr] `TestEnvironmentOptions::with_tagging_secret_strategy` replaced