extenddb-reference.md

ExtendDB Reference Notes

This file tracks ExtendDB source provenance, local setup, and ExtendDO compatibility patches. For user-facing behavior and scope, see ExtendDB Parity, API Surface, and Limitations.

Attribution Review

Reviewed on 2026-06-02 for public repository readiness.

• License: Apache License, Version 2.0.
• Local notice: pnpm run setup:extenddb retains third_party/extenddb/NOTICE unchanged.
• Local license: pnpm run setup:extenddb retains third_party/extenddb/LICENSE unchanged.
• Root notice: NOTICE includes ExtendDB attribution and points back to this patch ledger.
• File headers: materialized Rust files and crate manifests retain upstream copyright and SPDX headers. Some upstream storage files retain the DynamoDB Open contributors copyright line they carried in the imported source.
• Trademark note: the ExtendDB NOTICE trademark text is retained, and ExtendDO adds a non-affiliation note in NOTICE, README.md, and docs/public-release.md.

This review is not legal advice. The final release process should still include a full license scan and a decision on whether each modified local source file needs an additional per-file "modified by ExtendDO" notice beyond this public patch ledger.

Source And Version

The ignored external-projects/extenddb directory is not authoritative. Its local Git metadata resolves to the ExtendDO repository. Use the upstream URL and tag below for public provenance.

• Upstream repository: https://github.com/ExtendDB/extenddb.git.
• Upstream reference tag: v0.1.0.
• Upstream reference commit: c20b2d6767f234fd67dc111c5c6a1b50342c4847.
• Historical initial source import in this repository: 470d600b9b18020f7fc58a08c2d4cf5579937fc8.
• Historical last tracked source patch in this repository: b354e0a7c6103acfa2629ac694799aa65282a201.
• Historical vendor metadata introduction commit: 535ad0913aefea80000ed9847e41dd42ba41b6cb.
• Imported ExtendDB workspace package version: 0.1.0.
• Imported ExtendDB workspace metadata in the reference copy:
  • Edition: 2024
  • Rust version: 1.85
  • License: Apache-2.0
• ExtendDO workspace metadata used by the local crates:
  • Version: 0.1.0
  • Edition: 2021
  • Rust version: 1.80
  • License: Apache-2.0
• Ignored local source materialized by pnpm run setup:extenddb:
  • third_party/extenddb/crates/core
  • third_party/extenddb/crates/engine
  • third_party/extenddb/crates/storage
  • third_party/extenddb/crates/auth
  • third_party/extenddb/LICENSE
  • third_party/extenddb/NOTICE
• Tracked setup manifest and patches:
  • patches/extenddb/manifest.json
  • patches/extenddb/compatibility.patch
• Generated local setup metadata:
  • third_party/extenddb/EXTENDDO_VENDOR.json

The previous vendor metadata incorrectly recorded dbe61d21f0ce65afea3e7dea7f239b8270bf98cd as an upstream reference commit. That hash is an ExtendDO repository commit, not an ExtendDB upstream commit for the local source. The current public upstream baseline is the v0.1.0 tag and commit listed above.

The upstream source copy also contains server, CLI, PostgreSQL storage, docs, samples, and tests. Those components are not copied into the runtime build.

How ExtendDO References ExtendDB

third_party/extenddb is an ignored local directory, but once materialized it is part of this repository's Rust workspace. It is not a Git submodule and it is not downloaded during the Worker build. The root Cargo.toml declares the local crates as workspace members and exposes them as path dependencies:

• extenddb-core = { path = "third_party/extenddb/crates/core" }
• extenddb-auth = { path = "third_party/extenddb/crates/auth" }
• extenddb-engine = { path = "third_party/extenddb/crates/engine" }
• extenddb-storage = { path = "third_party/extenddb/crates/storage" }

crates/wasm depends on those path dependencies and is the bridge between ExtendDB Rust and the Worker host. Its wasm exports are generated by wasm-pack into worker/src/generated/wasm and imported by worker/src/wasm.ts.

The main runtime references are:

• crates/wasm/src/lib.rs calls extenddb_engine::dispatch for DynamoDB operation handling.
• crates/wasm/src/lib.rs calls extenddb-auth for identity policy, resource policy, trust policy, wildcard, principal, and condition evaluation helpers.
• crates/wasm/src/lib.rs exposes helper exports for condition AST evaluation, update-action application, key-condition matching, and query access planning using ExtendDB core expression types.
• crates/wasm/src/worker_storage.rs implements the ExtendDB storage traits from extenddb-storage and forwards each trait method to the JavaScript host through host.callStorage(method, payload).
• worker/src/engine-host.ts receives those storage method names and routes them to Cloudflare Durable Objects.

This means the default DynamoDB data plane is not:

TypeScript parses DynamoDB -> TypeScript implements DynamoDB behavior

It is:

DynamoDB JSON -> ExtendDB engine in wasm -> ExtendDB storage trait callback -> Durable Objects

TypeScript is responsible for Cloudflare platform adaptation, persistence, auth request plumbing, management routes, and error translation at the host boundary. ExtendDB remains responsible for the operation semantics that run inside the wasm dispatcher.

Materializing Local ExtendDB

ExtendDO intentionally keeps ExtendDB source out of Git. A developer or CI job must materialize the ignored local copy before building:

pnpm run setup:extenddb

The default setup downloads the pinned archive URL recorded in patches/extenddb/manifest.json, extracts only the runtime paths, applies patches/extenddb/compatibility.patch, and writes generated local metadata to third_party/extenddb/EXTENDDO_VENDOR.json.

To use an existing local checkout instead of downloading:

pnpm run setup:extenddb -- --source ../extenddb

To materialize from a remote Git checkout:

pnpm run setup:extenddb -- --repo <extenddb-git-url> --ref <tag-or-sha>

Build scripts run scripts/check-extenddb-source.mjs and fail with setup instructions when the local source is missing. They do not fetch source implicitly.

Updating ExtendDB

Recommended update procedure:

1. Review the upstream ExtendDB release or commit range.
2. Update patches/extenddb/manifest.json with the new repository/ref metadata.
3. Regenerate patches/extenddb/compatibility.patch by comparing the upstream runtime paths with the desired ExtendDO-compatible local source.
4. Run pnpm run setup:extenddb -- --force from a clean ExtendDO worktree or an isolated branch.
5. Update this patch ledger with the new reference commit, version metadata, and local patch inventory.
6. Run pnpm run test and pnpm run build.
7. Commit manifest and patch updates separately from unrelated ExtendDO changes when the diff is large enough to benefit review.

Runtime Use

• extenddo-protocol uses third_party/extenddb/crates/core for DynamoDB wire-compatible item and table types.
• extenddo-wasm uses third_party/extenddb/crates/engine for DynamoDB operation dispatch, validation, expression parsing, update evaluation, and response shaping.
• extenddo-wasm implements third_party/extenddb/crates/storage traits with a Cloudflare Worker host callback backed by CatalogDO and TableDO.
• extenddo-wasm uses third_party/extenddb/crates/auth for IAM identity policy document parsing, validation, condition evaluation, effective identity-policy decisions, and resource/trust-policy decisions.

Local Compatibility Patches

The patch inventory below is based on comparing third_party/extenddb with https://github.com/ExtendDB/extenddb.git tag v0.1.0 (c20b2d6767f234fd67dc111c5c6a1b50342c4847) on 2026-06-05.

Crate	Local changes
core	Added serde derives to expression AST, key-condition, update, stream, TTL, ListTables, and UpdateTable types that cross the wasm host boundary. Added TableStatus::Restoring so Worker restore/import lifecycle rows can preserve ExtendDB table-description shape while hiding partially initialized tables. Some additional diffs are Rust formatting/import-order churn.
engine	Disabled native filesystem import/export in the default Worker build, added Worker-safe unsupported stubs for native import/export handlers, recognized DescribeExport, ListExports, DescribeImport, and ListImports as known operations for Worker-side handling, and replaced direct SystemTime stream iterator timestamps with a wasm-safe js_sys::Date helper on wasm32.
storage	Made native server, diagnostics, hooks, settings, operations, and component-registry modules feature-gated behind native; removed the unused auth dependency from the Worker storage build; made native-only dependencies optional; and uses LocalBoxFuture for storage trait futures on wasm32.
auth	Added PolicyParseOptions, strict-by-default parser compatibility hooks, additional condition operators used by ExtendDO policy handling, typed condition value coercion when explicitly enabled, optional missing-action/resource defaults, optional non-AWS principal object keys, and IP-address/CIDR condition evaluation. The Worker public API does not expose AWS-only context-extension mode.

Worker-Specific Adaptations

• worker/src/import-export.ts keeps the ExtendDB/DynamoDB import-export response shape but stores DynamoDB JSON line files in R2, because the upstream ExtendDB implementation reads and writes local filesystem paths. It intentionally completes through asynchronous Worker control-plane jobs instead of mirroring ExtendDB's synchronous filesystem completion, and R2 exports write to generated per-export object keys under the requested destination to avoid concurrent overwrite collisions. It keeps required InputFormat, ImportTable unknown-field rejection, ignored nested table-creation extras, and import-row validation/count behavior from external-projects/extenddb/crates/engine/src/import_export.rs.

Management Behavior Ports

Management API behavior is modeled after the ExtendDB management store and management routes, but the Worker exposes curl-friendly JSON endpoints rather than depending on the extenddb manage CLI.

• Account create/delete behavior mirrors external-projects/extenddb/crates/server/src/management/account.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/accounts.rs: duplicate create reports "Account already exists", missing delete reports "Account not found", and account delete is rejected while catalog tables still exist with "Cannot delete account with existing tables. Delete all tables first."
• Admin user create/delete/password behavior mirrors external-projects/extenddb/crates/server/src/management/admin.rs and external-projects/extenddb/crates/storage-postgres/src/admin_store.rs: empty admin names/passwords return the same validation messages, duplicate admin create reports "Admin user already exists", and missing delete or password change reports "Admin user not found".
• Settings list/get/set behavior mirrors external-projects/extenddb/crates/server/src/management/settings.rs, external-projects/extenddb/crates/server/src/management/ops_settings.rs, and external-projects/extenddb/crates/storage-postgres/src/catalog_store.rs: list ordering is by key, sensitive values are redacted with the same key substring rules, missing settings report "Setting not found", writable keys and read-only keys match ExtendDB, and numeric validation follows the Rust f64/u32 parser split between parse errors and range errors.
• Inline policy behavior mirrors external-projects/extenddb/crates/server/src/management/iam_policy.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/policies.rs, and policy document parsing now directly reuses external-projects/extenddb/crates/auth/src/policy/document.rs: policy writes validate the policy name and document before account storage checks, use ExtendDB's exact validation messages, and require a valid account; policies are keyed by principal type/name without requiring the user, group, or role row to exist; listing an unknown principal returns an empty list; deleting a missing user/group/role policy reports "Policy not found". Runtime identity-policy authorization now routes through third_party/extenddb/crates/auth via wasm instead of the previous TypeScript evaluator.
• Permissions-boundary management mirrors external-projects/extenddb/crates/server/src/management/permissions_boundary.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/policies.rs: principal-name and document validation run before account storage checks, with ExtendDB's exact management messages.
• User/group/role list behavior mirrors external-projects/extenddb/crates/storage-postgres/src/management_store/users.rs, external-projects/extenddb/crates/storage-postgres/src/management_store/groups.rs, and external-projects/extenddb/crates/storage-postgres/src/management_store/roles.rs: list operations query by account id directly and return an empty list when the account has no matching principals, including when the account id does not exist.
• IAM group membership create/delete behavior mirrors external-projects/extenddb/crates/server/src/management/iam_group.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/groups.rs: group create uses ExtendDB's exact group-name validation, duplicate, and missing-account errors; missing group delete reports "IAM group not found"; adding an existing membership reports "User is already a member of this group", missing group or user on add reports "Group or user not found", and deleting a missing membership reports "Membership not found".
• Management error status mapping follows external-projects/extenddb/crates/server/src/management/ops.rs for not-found and conflict errors.
• User/role tag behavior mirrors external-projects/extenddb/crates/server/src/management/iam_user.rs, external-projects/extenddb/crates/server/src/management/iam_role.rs, external-projects/extenddb/crates/storage-postgres/src/management_store/users.rs, and external-projects/extenddb/crates/storage-postgres/src/management_store/roles.rs: tag writes validate tag key/value lengths with ExtendDB's exact messages and require the referenced user or role through storage FK behavior, while list and delete query/delete by principal key and return an empty list/success for missing principals.
• User creation mirrors external-projects/extenddb/crates/server/src/management/iam_user.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/users.rs by using ExtendDB's exact user-name/password validation messages, duplicate and missing-account create errors, and by seeding SelfServicePolicy for each new user, allowing that user's own access-key and password self-service operations. User password changes mirror external-projects/extenddb/crates/server/src/management/iam_user_self.rs and the same storage implementation for empty-password and missing-user behavior.
• Role creation/deletion mirrors external-projects/extenddb/crates/server/src/management/iam_role.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/roles.rs by using ExtendDB's exact role-name and trust-policy validation messages, duplicate and missing-account create errors, and missing-role delete error. Runtime trust-policy authorization for both authenticated user assume-role and management caller-ARN assume-role now routes through the Rust-backed resource-policy evaluator, including AWS root principal, wildcard principal, object-form statement, and explicit-deny behavior.
• User access-key create/list/delete behavior mirrors external-projects/extenddb/crates/server/src/management/iam_user_self.rs and external-projects/extenddb/crates/storage-postgres/src/management_store/access_keys.rs: create/import require a real user through FK-style storage behavior, create reports "User not found" for missing users, import reports "IAM user not found" or "Access key ID already exists" for the corresponding storage errors, import validation uses the same access-key and secret-length messages and setting gate, list queries access keys directly and returns an empty list for missing or deleted users, and delete reports "Access key not found" when no matching key row exists.
• User/group/role deletion mirrors external-projects/extenddb/crates/storage-postgres/src/management_store/users.rs, external-projects/extenddb/crates/storage-postgres/src/management_store/groups.rs, external-projects/extenddb/crates/storage-postgres/src/management_store/roles.rs, and the migration schema: FK-backed rows such as tags, access keys, group memberships, and role sessions are removed with the principal, while inline policies and permissions boundaries remain keyed by account/principal name.

If future work copies additional source snippets or closely ports designs from ExtendDB, record the source path, source commit, summary of copied/adapted material, and any NOTICE or license attribution changes in this file.