feat(data-connections): store backend credentials in Cloudflare Secrets Store instead of DynamoDB

[alukach]

Problem

S3AccessKeyAuthenticationSchema (src/types/data-connection.ts) holds access_key_id and secret_access_key inline, and the whole authentication object is persisted on the data connection row in DynamoDB. src/components/features/data-connections/redact.ts strips it before anything crosses into a client component, which is the right guard for the RSC boundary — but the credential is still sitting in plaintext at rest, in table scans, in backups, and one careless select away from a response body.

This gets more pressing as S3-compatible backends without a federation story come online. Cloudflare R2 in particular has no AssumeRoleWithWebIdentity equivalent, so s3_access_key is the only way to reach it — key-based auth is not a legacy path being wound down, it's the path forward for R2.

Ask

Move backend credential material out of DynamoDB and into Cloudflare Secrets Store. source.coop is write-only against the store: it creates, rotates, and deletes secrets, and never reads a value back. The data proxy (source-cooperative/data.source.coop) holds the read side. Companion issue: source-cooperative/data.source.coop#210.

The capability split is enforced by Cloudflare's own token model rather than by convention:

• source.coop gets an API token with Account Secrets Store Edit only.
• The value cannot be read back through the API by anyone — Account Secrets Store Read grants metadata only, never the value. Values are only consumable through a Worker binding.

That last point is a feature here: it makes "source.coop cannot read the credential it wrote" a property of the platform, not something to review for.

Consequences for the UI

Edit becomes write-only-replace. The form cannot pre-fill or display an existing key, so the secret fields render empty with a "leave blank to keep the current credential" affordance. redact.ts gets simpler in the process — S3AccessKey already redacts to a bare { type }, and once the value never enters the record there is nothing left to strip.

⚠️ The part that needs a decision first

Secrets Store secrets are bound to a Worker statically, by name, at deploy time (secrets_store_secrets in Wrangler config, read as await env.BINDING.get()). There is no runtime lookup by name. So "one secret per data connection" means the proxy needs a new binding and a redeploy for every connection created — plus the account limits bite: one store per account in beta, 100 secrets per account, which becomes a hard ceiling on how many key-based connections the platform can ever hold.

The alternative is envelope encryption: one long-lived key in Secrets Store bound to the proxy, used to encrypt each connection's credentials. DynamoDB stores only the ciphertext, and the API serves it as part of authentication. That keeps plaintext out of the database (the actual goal), scales to any number of connections, and needs no redeploy on connection create.

Recommendation: envelope encryption, with an asymmetric keypair. source.coop holds the public half (encrypt only), the proxy the private half (decrypt only) — the read/write split then holds by construction rather than by IAM scoping alone. Wire shape:

{ "type": "s3_access_key", "ciphertext": "<base64>", "key_id": "dataconn-v1" }

key_id names the bound secret so rotation is a config change plus a re-encrypt sweep, not a flag day.

Work

• Settle the shape (one-secret-per-connection vs. envelope) jointly with the proxy issue — the authentication wire shape depends on it.
• Provision the store and a scoped write token; document the naming convention (e.g. dataconn-{data_connection_id}) and the workers scope required for the proxy to consume it.
• Write on connection create; rotate on update; delete on connection delete. Handle the partial-failure case (secret written, row write failed → orphaned secret; row written, secret failed → connection that cannot authenticate).
• Extend the create/update actions and the /api/v1/data-connections routes so authentication never round-trips a plaintext credential.
• Update DataConnectionForm for write-only-replace, and simplify redact.ts accordingly.
• Migration for existing s3_access_key connections; verify none are left holding plaintext afterwards.
• Extend Azure SAS token the same way, or state explicitly that it is out of scope for now.

Acceptance

• Creating an R2 data connection through the UI stores no credential material in DynamoDB.
• Reading the connection back — API, RSC payload, or admin UI — never yields the secret.
• The proxy can serve signed reads from that connection (validated jointly with the companion issue).

Related: #432 (GitHub Actions OIDC epic — same "who may act on a product" territory), source-cooperative/data.source.coop#207.

[jedsundwall]

To the extent I understand it, this seems like the right approach – specifically the "envelope" option – but I defer to Tyler on the final decision. I really like the idea of having no one ever having to handle keys directly. That said, I feel like we should require users to have been two-factor authenticated to ever upload data using any method.