From eaefddc4f87689ed109767f68080d9e46d85fca0 Mon Sep 17 00:00:00 2001 From: Agents Agent Date: Thu, 25 Jun 2026 15:10:27 +0200 Subject: [PATCH 1/2] feat(agents-login): store captured credentials in agents-api Postgres, not Vault The login worker no longer writes Claude/Codex tokens to Vault. On a successful capture it POSTs them to agents-api's in-cluster ingest endpoint (POST /api/v1/internal/credentials, bearer-authenticated), which persists them per user in Postgres; capture fails loudly and stores nothing when no token is produced. - New agentsApiClient replaces vaultClient; config gains AGENTS_API_INTERNAL_URL + AGENTS_API_INTERNAL_BEARER and drops the Vault settings. - Worker manifest wires the agents-api URL + internal bearer and drops the Vault role/env; egress to agents-system retained. - The Vault claude-oauth / codex-oauth VaultStaticSecrets, their kustomization entries, and the agents/{claude,codex}-oauth Vault policy paths + agents-oauth-writer role are removed. - Docs updated to describe Postgres-via-agents-api storage. --- .../cluster/flux/apps/agents/AGENT-PARITY.md | 7 +- platform/cluster/flux/apps/agents/README.md | 9 +- platform/cluster/flux/apps/agents/SETUP.md | 82 +++----- .../agents/agents-login/network-policy.yaml | 23 ++- .../flux/apps/agents/agents-login/rbac.yaml | 4 +- .../agents/agents-login/serviceaccount.yaml | 7 +- .../flux/apps/agents/agents-login/worker.yaml | 24 ++- .../agents/credentials/claude-oauth-vss.yaml | 29 --- .../agents/credentials/codex-oauth-vss.yaml | 30 --- .../agents/credentials/kustomization.yaml | 2 - .../flux/apps/data/vault/bootstrap-auth.sh | 42 +--- .../agents-api/deployment-enschede-ws.yaml | 4 +- .../apps/stateless/agents-api/deployment.yaml | 4 +- services/agents-login/Dockerfile | 2 +- services/agents-login/README.md | 62 +++--- services/agents-login/src/index.ts | 2 +- services/agents-login/src/shared/config.ts | 20 +- services/agents-login/src/shared/types.ts | 4 +- .../src/worker/agentsApiClient.ts | 80 ++++++++ .../agents-login/src/worker/credentials.ts | 5 +- services/agents-login/src/worker/index.ts | 19 +- services/agents-login/src/worker/lease.ts | 2 +- services/agents-login/src/worker/server.ts | 19 +- services/agents-login/src/worker/session.ts | 22 +-- .../agents-login/src/worker/vaultClient.ts | 172 ---------------- .../agents-login/test/agentsApiClient.test.ts | 70 +++++++ services/agents-login/test/config.test.ts | 29 +-- .../agents-login/test/helpers/fakeVault.ts | 121 ------------ .../agents-login/test/integration.pty.test.ts | 41 ++-- services/agents-login/test/session.test.ts | 92 ++++----- .../agents-login/test/vaultClient.test.ts | 183 ------------------ .../agents-login/test/workerServer.test.ts | 40 +--- services/agents-login/vitest.config.ts | 2 +- 33 files changed, 374 insertions(+), 880 deletions(-) delete mode 100644 platform/cluster/flux/apps/agents/credentials/claude-oauth-vss.yaml delete mode 100644 platform/cluster/flux/apps/agents/credentials/codex-oauth-vss.yaml create mode 100644 services/agents-login/src/worker/agentsApiClient.ts delete mode 100644 services/agents-login/src/worker/vaultClient.ts create mode 100644 services/agents-login/test/agentsApiClient.test.ts delete mode 100644 services/agents-login/test/helpers/fakeVault.ts delete mode 100644 services/agents-login/test/vaultClient.test.ts diff --git a/platform/cluster/flux/apps/agents/AGENT-PARITY.md b/platform/cluster/flux/apps/agents/AGENT-PARITY.md index a736c59b..e628adf5 100644 --- a/platform/cluster/flux/apps/agents/AGENT-PARITY.md +++ b/platform/cluster/flux/apps/agents/AGENT-PARITY.md @@ -29,10 +29,9 @@ default home. The `agents-login` portal keeps the two in lockstep too: its image bakes both CLIs, the worker captures both credential sets in the same -login session, and it writes both Vault paths -(`secret/agents/claude-oauth` with `credentials_json`/`claude_json`, -`secret/agents/codex-oauth` with `auth_json`/`config_toml`) — a login -that refreshed only one provider is a parity regression. +login session, and it posts both provider payloads to agents-api +(`CLAUDE` with `oauth_token`, `CODEX` with `auth_json`/`config_toml`) — +a login flow that cannot refresh both providers is a parity regression. The `auth-bootstrap` Pod populates both with one human exec-and-paste session each (see `README.md`). The diff --git a/platform/cluster/flux/apps/agents/README.md b/platform/cluster/flux/apps/agents/README.md index 0ef05cd6..afce2b05 100644 --- a/platform/cluster/flux/apps/agents/README.md +++ b/platform/cluster/flux/apps/agents/README.md @@ -40,13 +40,12 @@ Deployment), plus the Docker socket gid and runner egress node IP. The browser-driven login portal (`agents-login`, controller + worker Deployments in this directory) is the preferred way to refresh credentials: it runs the CLI logins server-side and writes the captured -OAuth to Vault (`secret/agents/claude-oauth`, `secret/agents/codex-oauth`) -from where VSO projects them to every runner — no terminal copy-paste, -one sign-in for the whole fleet. See `SETUP.md` for the portal flow, the -Vault schema, the Lease+CAS writeback model, and the node-pin +OAuth credentials to agents-api, which persists them in Postgres — no +terminal copy-paste, one sign-in for the whole fleet. See `SETUP.md` for +the portal flow, the internal ingest model, and the node-pin classification. The terminal bootstrap below remains the fallback until the companion runner change (in the `agents` repo) makes runners consume -the projected Secrets instead of the PVCs. +credentials from agents-api instead of the PVCs. ## First-time auth bootstrap diff --git a/platform/cluster/flux/apps/agents/SETUP.md b/platform/cluster/flux/apps/agents/SETUP.md index 42fb52b0..ed34618f 100644 --- a/platform/cluster/flux/apps/agents/SETUP.md +++ b/platform/cluster/flux/apps/agents/SETUP.md @@ -20,25 +20,24 @@ for when the portal is not yet deployed. ### Login portal (preferred) `agents-login.jorisjonkers.dev` is a browser-driven portal that runs -`claude /login` and `codex login --device` server-side and writes the -captured OAuth credentials to Vault, from where VSO projects them to -every runner — so one re-sign-in renews the whole fleet and no text has -to be copied out of a terminal. It is forward-auth protected by a +`claude setup-token` and `codex login --device` server-side and posts the +captured OAuth credentials to agents-api, which persists them in Postgres +for runner consumption — so one re-sign-in renews the whole fleet and no +text has to be copied out of a terminal. It is forward-auth protected by a dedicated `AGENTS_LOGIN` permission, NOT the shared `AGENTS` one: a holder of that permission can mint the fleet's root credentials, so it is granted deliberately. The portal is two Deployments in `agents-system`: -- **controller** — the public UI. Holds no Vault token and stores no +- **controller** — the public UI. Holds no credential store access and stores no credential files. Uses short HTTP polling (not a WebSocket: the Cloudflare edge kills idle sockets ~100 s and the OAuth approval idles longer) and proxies the flow to the worker over an internal token. - **worker** — owns the PTY that runs the CLI login, captures the - credential files, and writes Vault under a single-writer Kubernetes - `Lease` (`agents-login-write`) with KV v2 Compare-And-Set. Bound to the - `agents-oauth-writer` Vault policy via the `agents-login-worker` - ServiceAccount. + credentials, and posts them to agents-api under a single-writer Kubernetes + `Lease` (`agents-login-write`). The worker uses the same + `agents-login-internal` Secret token that agents-api uses for worker calls. To re-authenticate: open the portal, start the Claude login (copy the authorize URL into a browser tab signed in to the Claude Pro account, @@ -61,77 +60,60 @@ codex login --device # enter the device code at chatgpt.com/codex/auth infinity`, a later exec session's stdout is not the Pod's logs, and the Claude flow needs the redirect URL fed back on stdin. -## Vault schema for the OAuth credentials +## agents-api credential ingest -The portal writes two KV v2 paths under `secret/agents`: +The worker calls agents-api's internal ingest endpoint: -- `secret/agents/claude-oauth` — fields `credentials_json`, `claude_json` - (+ `schema_version`, `updated_at`, `updated_by`). -- `secret/agents/codex-oauth` — fields `auth_json`, `config_toml` (+ the - same metadata). +- `POST http://agents-api.agents-system.svc.cluster.local:8082/api/v1/internal/credentials` +- `Authorization: Bearer ` +- Body: `userId`, uppercase `provider`, and a provider payload. -Field names are underscored because `vault kv put` with leading-dot keys -is awkward; the `VaultStaticSecret` transformation templates re-emit the -exact CLI filenames (`.credentials.json`, `.claude.json`, `auth.json`, -`config.toml`) into the projected Secrets `agents-claude-oauth` / -`agents-codex-oauth`. `.claude.json` is a SIBLING of `~/.claude/`, not a -file inside it, so a consumer init-copy must place it at -`$HOME/.claude.json`. +Claude payloads contain `oauth_token`. Codex payloads contain `auth_json` +and `config_toml`. agents-api owns persistence in Postgres; the worker does +not read back stored credential status. The shared controller↔worker token lives at -`secret/agents/login-internal-token` (field `token`). The portal 403s on -writes until the `agents-oauth-writer` policy and the worker auth role -are live, so re-run the `vault-bootstrap-auth` Job first if a fresh init -is involved, then seed: +`secret/agents/login-internal-token` (field `token`) and is projected into +the Kubernetes Secret `agents-login-internal`. Seed it once: ```sh vault kv put -mount=secret agents/login-internal-token token="$(openssl rand -hex 32)" ``` -VSO reconciles every `refreshAfter: 1h`. To pull a fresh login through -immediately: - -```sh -kubectl -n agents-system annotate vaultstaticsecret agents-claude-oauth force-sync="$(date +%s)" --overwrite -kubectl -n agents-system annotate vaultstaticsecret agents-codex-oauth force-sync="$(date +%s)" --overwrite -``` - ## Writeback and failure modes -Both CLIs rotate their refresh token in place on disk, so a read-only -projected Secret would go stale and force a premature re-login. The -worker is the single writer that pushes a rotated credential back to -Vault, serialized by the `agents-login-write` Lease and KV v2 CAS against -`metadata.version`: +The worker is the single writer that posts captured credentials to agents-api, +serialized by the `agents-login-write` Lease: -- **CAS conflict** — the loser re-reads and retries with bounded backoff; - a persistent conflict surfaces an error rather than blind-overwriting. +- **No captured credential** — the session fails and nothing is posted. +- **agents-api ingest failure** — the session fails and nothing is persisted by + the worker. - **Worker down** — credentials go stale until the next login; the refresh-ping CronJob is the early-warning signal. - **Two writers** — prevented by `replicas: 1` + `Recreate` + the Lease. ## Node-pin classification -Moving credentials into Vault does NOT by itself let runners schedule +Moving credentials into Postgres does NOT by itself let runners schedule anywhere. `AGENT_RUNTIME_NODE=enschede-gtx-960m-1` is **multi-reason**: the docker-socket GID, the hard-coded egress callback IP in `network-policy/runner-egress.yaml`, and the untrusted-workloads-at-home -policy. Only the **credential** justification is removed by the Vault +policy. Only the **credential** justification is removed by the credential-store migration; the others keep the pin until the companion runner change tolerates capability scheduling. The agents-login worker itself is **not** -pinned — it reads Vault, so it runs anywhere. +pinned — it posts to agents-api, so it runs anywhere. ## Companion change (agents repo) -The cluster projects `agents-claude-oauth` / `agents-codex-oauth` as -read-only Secrets. Making runners consume them instead of the PVCs is a +Making runners consume agents-api-stored credentials instead of the PVCs is a companion change in the `agents` repo: - `services/agent-runner/Dockerfile` + `/opt/entrypoint.sh` — create a - writable `$HOME/.claude` / `$CODEX_HOME` and init-copy the projected - files into place (`.claude.json` as a sibling of `.claude/`). -- `Fabric8AgentRunnerOrchestrator.kt` — swap the two `claude-credentials` - / `codex-credentials` PVC volume sources for the projected Secrets. + writable `$HOME/.claude` / `$CODEX_HOME` and materialize the fetched + credential files into place. +- `Fabric8AgentRunnerOrchestrator.kt` — stop mounting the two + `claude-credentials` / `codex-credentials` PVC volume sources once runners + can fetch credentials through agents-api. Until that lands, runners still read the PVCs, so the legacy bootstrap (sections 3-5 below) and the `claude-credentials` / `codex-credentials` diff --git a/platform/cluster/flux/apps/agents/agents-login/network-policy.yaml b/platform/cluster/flux/apps/agents/agents-login/network-policy.yaml index 38e44a0a..1d6fa862 100644 --- a/platform/cluster/flux/apps/agents/agents-login/network-policy.yaml +++ b/platform/cluster/flux/apps/agents/agents-login/network-policy.yaml @@ -1,13 +1,13 @@ -# The worker is the sensitive half of the credential portal: it holds the -# Vault-writer token and the Lease. It is reached only by agents-api, which -# proxies the login flow from the agents-ui Credentials page (served at +# The worker is the sensitive half of the credential portal: it runs the +# login CLIs and posts captured credentials to agents-api. It is reached only by +# agents-api, which proxies the login flow from the agents-ui Credentials page (served at # agents.jorisjonkers.dev). Ingress is therefore restricted to the agents-api # pods on 8081; there is no public-facing component in this namespace. # -# Egress: DNS, the in-cluster Vault and Kubernetes API (the service CIDR on -# 443/8200), and outbound HTTPS for the Anthropic/OpenAI OAuth flows — RFC1918 +# Egress: DNS, agents-api, the Kubernetes API (the service CIDR on +# 443/6443), and outbound HTTPS for the Anthropic/OpenAI OAuth flows — RFC1918 # is excluded from the internet rule so the worker cannot reach arbitrary -# cluster-private hosts (the explicit service-CIDR rule re-allows Vault + apiserver). +# cluster-private hosts (explicit rules re-allow agents-api + apiserver). apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -45,14 +45,17 @@ spec: port: 53 - protocol: TCP port: 53 - # In-cluster Vault (KV v2 writes + kubernetes auth login). + # In-cluster agents-api credential ingest. - to: - namespaceSelector: matchLabels: - kubernetes.io/metadata.name: data-system + kubernetes.io/metadata.name: agents-system + podSelector: + matchLabels: + app.kubernetes.io/name: agents-api ports: - protocol: TCP - port: 8200 + port: 8082 # Kubernetes API (the Lease lock) reached via the kubernetes.default # ClusterIP in the service CIDR. NetworkPolicy cannot select the apiserver # by label, so allow the service CIDR on the apiserver ports. @@ -66,7 +69,7 @@ spec: port: 6443 # Outbound HTTPS for the Claude/Codex OAuth endpoints. RFC1918 + the # link-local range are excluded so this rule cannot reach cluster-private - # hosts; Vault and the apiserver are covered by the explicit rules above. + # hosts; agents-api and the apiserver are covered by the explicit rules above. - to: - ipBlock: cidr: 0.0.0.0/0 diff --git a/platform/cluster/flux/apps/agents/agents-login/rbac.yaml b/platform/cluster/flux/apps/agents/agents-login/rbac.yaml index b1c1634b..406ae1c2 100644 --- a/platform/cluster/flux/apps/agents/agents-login/rbac.yaml +++ b/platform/cluster/flux/apps/agents/agents-login/rbac.yaml @@ -1,6 +1,6 @@ # Single-writer lock for credential writeback. The worker holds this one named -# Lease while it writes Vault, so two worker instances (e.g. across a Recreate -# rollover) cannot race a Compare-And-Set. Scoped to the single Lease name the +# Lease while it posts to agents-api, so two worker instances (e.g. across a +# Recreate rollover) cannot race. Scoped to the single Lease name the # worker uses (LEASE_NAME=agents-login-write) — not the whole leases resource. apiVersion: rbac.authorization.k8s.io/v1 kind: Role diff --git a/platform/cluster/flux/apps/agents/agents-login/serviceaccount.yaml b/platform/cluster/flux/apps/agents/agents-login/serviceaccount.yaml index ab578f87..2d779a74 100644 --- a/platform/cluster/flux/apps/agents/agents-login/serviceaccount.yaml +++ b/platform/cluster/flux/apps/agents/agents-login/serviceaccount.yaml @@ -1,8 +1,5 @@ -# The worker is the only identity that writes the OAuth credential paths in -# Vault (bound to the agents-oauth-writer policy via the agents-login-worker -# kubernetes auth role in bootstrap-auth.sh) and the only one that takes the -# write Lease. agents-api proxies the browser login flow to it but holds no -# Vault token itself. +# The worker runs the OAuth CLIs, posts captured credentials to agents-api, and +# takes the write Lease. agents-api proxies the browser login flow to it. apiVersion: v1 kind: ServiceAccount metadata: diff --git a/platform/cluster/flux/apps/agents/agents-login/worker.yaml b/platform/cluster/flux/apps/agents/agents-login/worker.yaml index 612bc1f2..a7a5b7ef 100644 --- a/platform/cluster/flux/apps/agents/agents-login/worker.yaml +++ b/platform/cluster/flux/apps/agents/agents-login/worker.yaml @@ -1,10 +1,11 @@ -# Private half of the portal. Owns the PTY that runs `claude /login` and -# `codex login --device`, captures the credential files, and writes them to -# Vault via KV v2 Compare-And-Set under a Lease lock. Single instance + Recreate -# so the Lease is never contended by two writers across a rollover. +# Private half of the portal. Owns the PTY that runs `claude setup-token` and +# `codex login --device`, captures credentials, and posts them to agents-api +# under a Lease lock. Single instance + Recreate so the Lease is never +# contended by two writers across a rollover. # -# HOME is a writable emptyDir, not the legacy credential PVCs: Vault is the -# source of truth, so the worker needs no node pin and can schedule anywhere. +# HOME is a writable emptyDir, not the legacy credential PVCs: agents-api +# persists credentials in Postgres, so the worker needs no node pin and can +# schedule anywhere. apiVersion: apps/v1 kind: Deployment metadata: @@ -61,10 +62,13 @@ spec: value: /home/agent - name: CODEX_HOME value: /home/agent/.codex - - name: VAULT_ADDR - value: http://vault.data-system.svc.cluster.local:8200 - - name: VAULT_K8S_ROLE - value: agents-login-worker + - name: AGENTS_API_INTERNAL_URL + value: http://agents-api.agents-system.svc.cluster.local:8082 + - name: AGENTS_API_INTERNAL_BEARER + valueFrom: + secretKeyRef: + name: agents-login-internal + key: token - name: LEASE_NAME value: agents-login-write - name: LEASE_NAMESPACE diff --git a/platform/cluster/flux/apps/agents/credentials/claude-oauth-vss.yaml b/platform/cluster/flux/apps/agents/credentials/claude-oauth-vss.yaml deleted file mode 100644 index 084a48e7..00000000 --- a/platform/cluster/flux/apps/agents/credentials/claude-oauth-vss.yaml +++ /dev/null @@ -1,29 +0,0 @@ -# Claude Code OAuth credential projected from Vault into a Secret the runner -# consumes. The credential-login portal uses `claude setup-token`, whose product -# is a long-lived OAuth token (the `oauth_token` field) meant for the -# CLAUDE_CODE_OAUTH_TOKEN env var — that is the primary key here. The -# leading-dot file keys are kept for the interactive `claude login` shape -# (.credentials.json inside ~/.claude/, .claude.json as its sibling); they -# render empty when only a token was captured, which the runner ignores. -apiVersion: secrets.hashicorp.com/v1beta1 -kind: VaultStaticSecret -metadata: - name: agents-claude-oauth - namespace: agents-system -spec: - vaultAuthRef: vso-system/default - type: kv-v2 - mount: secret - path: agents/claude-oauth - destination: - name: agents-claude-oauth - create: true - transformation: - templates: - oauth_token: - text: '{{- get .Secrets "oauth_token" -}}' - .credentials.json: - text: '{{- get .Secrets "credentials_json" -}}' - .claude.json: - text: '{{- get .Secrets "claude_json" -}}' - refreshAfter: 1h diff --git a/platform/cluster/flux/apps/agents/credentials/codex-oauth-vss.yaml b/platform/cluster/flux/apps/agents/credentials/codex-oauth-vss.yaml deleted file mode 100644 index 2acc694d..00000000 --- a/platform/cluster/flux/apps/agents/credentials/codex-oauth-vss.yaml +++ /dev/null @@ -1,30 +0,0 @@ -# OAuth credential files for Codex CLI, projected from Vault into a -# Secret whose keys are the exact filenames used by the CLI. -# A consumer init-copy places auth.json inside ~/.codex/ and -# config.toml alongside it. -# -# Populate the Vault path before the first runner Pod starts: -# -# vault kv put -mount=secret agents/codex-oauth \ -# auth_json="$(cat ~/.codex/auth.json)" \ -# config_toml="$(cat ~/.codex/config.toml)" -apiVersion: secrets.hashicorp.com/v1beta1 -kind: VaultStaticSecret -metadata: - name: agents-codex-oauth - namespace: agents-system -spec: - vaultAuthRef: vso-system/default - type: kv-v2 - mount: secret - path: agents/codex-oauth - destination: - name: agents-codex-oauth - create: true - transformation: - templates: - auth.json: - text: '{{- get .Secrets "auth_json" -}}' - config.toml: - text: '{{- get .Secrets "config_toml" -}}' - refreshAfter: 1h diff --git a/platform/cluster/flux/apps/agents/credentials/kustomization.yaml b/platform/cluster/flux/apps/agents/credentials/kustomization.yaml index 4cdb7c5b..76c36e68 100644 --- a/platform/cluster/flux/apps/agents/credentials/kustomization.yaml +++ b/platform/cluster/flux/apps/agents/credentials/kustomization.yaml @@ -8,5 +8,3 @@ resources: - kb-bearer-vss.yaml - github-app-vss.yaml - knowledge-vault-deploy-key-vss.yaml - - claude-oauth-vss.yaml - - codex-oauth-vss.yaml diff --git a/platform/cluster/flux/apps/data/vault/bootstrap-auth.sh b/platform/cluster/flux/apps/data/vault/bootstrap-auth.sh index 179bc1a7..ab3c237c 100644 --- a/platform/cluster/flux/apps/data/vault/bootstrap-auth.sh +++ b/platform/cluster/flux/apps/data/vault/bootstrap-auth.sh @@ -114,11 +114,8 @@ path "secret/data/platform/postgres" { # under KV v2. # # Scoped to repositories/* + projects/* on purpose. A blanket -# secret/{data,metadata}/agents/* grant also reached the OAuth credential -# paths (agents/claude-oauth, agents/codex-oauth) — and the metadata -# delete capability would DESTROY their KV v2 version history, losing the -# ability to roll back to a known-good login. Those paths are owned solely -# by the agents-oauth-writer policy below. +# secret/{data,metadata}/agents/* grant would also reach unrelated agent +# subsystem secrets and allow metadata deletes beyond repository keys. path "secret/data/agents/repositories/*" { capabilities = ["create", "read", "update", "delete"] } @@ -144,31 +141,6 @@ path "database/creds/agents-api" { } EOF -# The credential-login portal worker (SA agents-login-worker in -# agents-system, bound to the agents-oauth-writer policy via the -# kubernetes auth role below) is the ONLY identity allowed to write the -# captured Claude/Codex OAuth files. It also reads the KV v2 metadata to -# run Compare-And-Set version checks on each writeback. No delete/destroy: -# OAuth version history must survive a bad login so a prior known-good -# version can be restored. -cat <<'EOF' >/tmp/agents-oauth-writer.hcl -path "secret/data/agents/claude-oauth" { - capabilities = ["create", "read", "update"] -} - -path "secret/data/agents/codex-oauth" { - capabilities = ["create", "read", "update"] -} - -path "secret/metadata/agents/claude-oauth" { - capabilities = ["read"] -} - -path "secret/metadata/agents/codex-oauth" { - capabilities = ["read"] -} -EOF - cat <<'EOF' >/tmp/n8n.hcl path "secret/data/platform/automation" { capabilities = ["read"] @@ -304,7 +276,6 @@ EOF vault policy write auth-api /tmp/auth-api.hcl vault policy write agents-api /tmp/agents-api.hcl -vault policy write agents-oauth-writer /tmp/agents-oauth-writer.hcl vault policy write n8n /tmp/n8n.hcl vault policy write postgres /tmp/postgres.hcl vault policy write rabbitmq /tmp/rabbitmq.hcl @@ -325,15 +296,6 @@ vault write auth/kubernetes/role/agents-api \ policies="agents-api" \ ttl="1h" -# Credential-login portal worker. Only this SA may write the OAuth -# credential paths; the controller half of the portal holds no Vault -# token at all. -vault write auth/kubernetes/role/agents-login-worker \ - bound_service_account_names="agents-login-worker" \ - bound_service_account_namespaces="agents-system" \ - policies="agents-oauth-writer" \ - ttl="1h" - vault write auth/kubernetes/role/n8n \ bound_service_account_names="n8n" \ bound_service_account_namespaces="automation-system" \ diff --git a/platform/cluster/flux/apps/stateless/agents-api/deployment-enschede-ws.yaml b/platform/cluster/flux/apps/stateless/agents-api/deployment-enschede-ws.yaml index 74fa8a34..37d34419 100644 --- a/platform/cluster/flux/apps/stateless/agents-api/deployment-enschede-ws.yaml +++ b/platform/cluster/flux/apps/stateless/agents-api/deployment-enschede-ws.yaml @@ -182,8 +182,8 @@ spec: optional: true # Credential-login worker — agents-api proxies the agents-ui # Credentials page to it. INTERNAL_TOKEN is the shared bearer - # projected from Vault into agents-login-internal; optional so the - # Deployment still starts before the path is seeded. + # projected into agents-login-internal; optional so the Deployment + # still starts before the path is seeded. - name: CREDENTIAL_WORKER_URL value: http://agents-login-worker.agents-system.svc.cluster.local:8081 - name: INTERNAL_TOKEN diff --git a/platform/cluster/flux/apps/stateless/agents-api/deployment.yaml b/platform/cluster/flux/apps/stateless/agents-api/deployment.yaml index b234a394..e83454d0 100644 --- a/platform/cluster/flux/apps/stateless/agents-api/deployment.yaml +++ b/platform/cluster/flux/apps/stateless/agents-api/deployment.yaml @@ -198,8 +198,8 @@ spec: optional: true # Credential-login worker — agents-api proxies the agents-ui # Credentials page to it. INTERNAL_TOKEN is the shared bearer - # projected from Vault into agents-login-internal; optional so the - # Deployment still starts before the path is seeded (the page just + # projected into agents-login-internal; optional so the Deployment + # still starts before the path is seeded (the page just # errors until then). - name: CREDENTIAL_WORKER_URL value: http://agents-login-worker.agents-system.svc.cluster.local:8081 diff --git a/services/agents-login/Dockerfile b/services/agents-login/Dockerfile index de1682f1..023c1019 100644 --- a/services/agents-login/Dockerfile +++ b/services/agents-login/Dockerfile @@ -1,5 +1,5 @@ # The credential-login worker: drives the Claude Code and Codex CLIs over a PTY -# to capture OAuth credentials into Vault. Both CLIs are baked into the image. +# to capture OAuth credentials and hand them to agents-api. Both CLIs are baked into the image. # The browser-facing half of the portal lives in agents-ui / agents-api. # ---- build stage ---------------------------------------------------------- diff --git a/services/agents-login/README.md b/services/agents-login/README.md index f9531961..8711de5b 100644 --- a/services/agents-login/README.md +++ b/services/agents-login/README.md @@ -8,7 +8,8 @@ and copy/pasting a long OAuth URL out of a terminal that cannot copy text. The browser-driven replacement is the **Credentials** page in agents-ui (served at `agents.jorisjonkers.dev`), which calls **agents-api**, which proxies to this worker. The worker is the only privileged piece: it drives the CLI logins over -a PTY and writes the captured credentials to Vault for fan-out to every runner. +a PTY and posts the captured credentials to agents-api, which persists them in +Postgres for runner consumption. This service is internal (ClusterIP only) and has no public surface — the UI, forward-auth gating, and CSRF all live in agents-ui / agents-api. agents-api @@ -16,45 +17,42 @@ authenticates to the worker with a shared `INTERNAL_TOKEN`. ## Flow -### Claude (`claude /login`) +### Claude (`claude setup-token`) 1. Worker spawns the CLI; the authorize URL is parsed from the PTY output. 2. agents-ui shows the URL in a copyable element. The operator opens it, approves. 3. The operator pastes the post-approval redirect URL back in the page; agents-api forwards it to the worker, which writes it to the child's stdin. -4. On the CLI success line, the worker captures the credentials and writes Vault. +4. On the CLI success line, the worker captures the OAuth token and posts it to + agents-api. ### Codex (`codex login --device`) 1. Worker spawns the CLI; the verification URL and device code are parsed out. 2. agents-ui shows both. The operator enters the code in the browser (no paste-back). -3. On the CLI success line, the worker captures the credentials and writes Vault. +3. On the CLI success line, the worker captures the credential files and posts + them to agents-api. -## Vault paths and fields +## agents-api ingest payload -Written with KV v2 Compare-And-Set against `metadata.version`. The KV mount is -configurable (`VAULT_KV_MOUNT`, default `secret`, matching the rest of the -agents stack). All values are UTF-8 text. +The worker calls `POST {AGENTS_API_INTERNAL_URL}/api/v1/internal/credentials` +with `Authorization: Bearer {AGENTS_API_INTERNAL_BEARER}`. -| Path (relative to the KV mount) | Fields | -| ------------------------------- | ------------------------------------------------------------------------------- | -| `agents/claude-oauth` | `credentials_json`, `claude_json`, `schema_version`, `updated_at`, `updated_by` | -| `agents/codex-oauth` | `auth_json`, `config_toml`, `schema_version`, `updated_at`, `updated_by` | +| Provider | Payload fields | +| -------- | ------------------------------ | +| `CLAUDE` | `oauth_token` | +| `CODEX` | `auth_json`, `config_toml` | -Field keys are underscored because `vault kv put` with leading-dot keys is -awkward; the VaultStaticSecret transformation templates re-emit the exact CLI -filenames into the projected Secrets. Captured from the live HOME: +Captured from the live HOME: -- `$HOME/.claude/.credentials.json` -- `$HOME/.claude.json` — a **sibling** of `.claude/`, at `$HOME/.claude.json`, - **not** inside the `.claude/` directory. +- Claude `setup-token` stdout (`oauth_token`). A completed Claude flow without + a token is treated as a failed capture and nothing is posted. - `$CODEX_HOME/auth.json` (`CODEX_HOME` defaults to `$HOME/.codex`) - `$CODEX_HOME/config.toml` -The write critical section is guarded by a named Kubernetes Lease. On a CAS -conflict the worker re-reads and retries with bounded backoff; on a persistent -conflict it fails loudly (surfacing an error to agents-api) rather than -blind-overwriting. +The write critical section is guarded by a named Kubernetes Lease. On an ingest +failure the worker fails loudly, surfaces the error to agents-api, and persists +nothing locally. ## Environment variables @@ -63,24 +61,20 @@ blind-overwriting. | `PORT` | `8081` | Listen port. | | `HOST` | `0.0.0.0` | Listen address. | | `INTERNAL_TOKEN` | _(required)_ | Shared token agents-api presents. | +| `AGENTS_API_INTERNAL_URL` | `http://agents-api.agents-system.svc.cluster.local:8082` | Internal agents-api base URL. | +| `AGENTS_API_INTERNAL_BEARER` | _(required)_ | Bearer for agents-api ingest. | | `SESSION_TTL_MS` | `900000` | Login-session timeout. | | `HOME` | OS home | Base for Claude credential capture. | | `CODEX_HOME` | `$HOME/.codex` | Base for Codex credential capture. | -| `VAULT_ADDR` | `http://vault.data-system.svc.cluster.local:8200` | Vault address. | -| `VAULT_K8S_ROLE` | `agents-login-worker` | Vault Kubernetes-auth role. | -| `VAULT_K8S_MOUNT` | `kubernetes` | Vault Kubernetes-auth mount. | -| `VAULT_KV_MOUNT` | `secret` | KV v2 mount. | -| `VAULT_CLAUDE_PATH` | `agents/claude-oauth` | KV path for Claude. | -| `VAULT_CODEX_PATH` | `agents/codex-oauth` | KV path for Codex. | -| `VAULT_SA_TOKEN_PATH` | `/var/run/secrets/kubernetes.io/serviceaccount/token` | SA JWT path for Vault + Lease auth. | -| `VAULT_CAS_MAX_RETRIES` | `5` | CAS retry budget before failing loudly. | +| `SA_TOKEN_PATH` | `/var/run/secrets/kubernetes.io/serviceaccount/token` | SA JWT path for Lease auth. | | `LEASE_NAME` | `agents-login-write` | Coordination Lease name. | | `LEASE_NAMESPACE` | `agents-system` | Coordination Lease namespace. | | `AGENTS_LOGIN_DEBUG` | unset | `1` enables debug log lines. | ## Security posture -- The worker holds the only Vault-writer token in the flow; agents-api holds none. +- The worker holds the internal agents-api ingest bearer, not database + credentials or a Vault writer token. - Logs and HTTP responses are deep-redacted of token/credential-shaped material. - The worker rejects any request lacking the `INTERNAL_TOKEN`, and its NetworkPolicy admits ingress only from the agents-api pods. @@ -96,9 +90,9 @@ npm run build # tsc -> dist/ ``` Tests use **fake CLI binaries** (shell scripts created in a temp dir and put on -`PATH`) and a fake in-process Vault HTTP server, so the suite runs with no -network and no real CLIs. The PTY logic is covered both by a deterministic fake -PTY (state machine, parsing, cancellation, timeout, CAS retry/conflict) and, +`PATH`) and fake agents-api clients, so the suite runs with no network and no +real CLIs. The PTY logic is covered both by a deterministic fake PTY (state +machine, parsing, cancellation, timeout, ingest failure) and, where the host supports it, an end-to-end pass over a real pseudo-terminal driving the fake CLI scripts. diff --git a/services/agents-login/src/index.ts b/services/agents-login/src/index.ts index 506b43e0..4d222b4b 100644 --- a/services/agents-login/src/index.ts +++ b/services/agents-login/src/index.ts @@ -2,7 +2,7 @@ import { runWorker } from './worker/index.js' // The credential-login worker: the only process this image runs. It owns the // PTY that drives `claude /login` / `codex login --device`, captures the -// credential files, and writes them to Vault under a Lease + CAS. The browser +// credential files, and posts them to agents-api under a Lease. The browser // UI and its auth live in agents-ui / agents-api, which proxy to this worker // over the internal token. runWorker().catch((err) => { diff --git a/services/agents-login/src/shared/config.ts b/services/agents-login/src/shared/config.ts index ee10de87..62626b23 100644 --- a/services/agents-login/src/shared/config.ts +++ b/services/agents-login/src/shared/config.ts @@ -35,17 +35,12 @@ export interface WorkerConfig { internalToken: string home: string codexHome: string - vaultAddr: string - vaultK8sRole: string - vaultK8sMount: string - vaultKvMount: string - vaultClaudePath: string - vaultCodexPath: string + agentsApiInternalUrl: string + agentsApiInternalBearer: string saTokenPath: string leaseName: string leaseNamespace: string sessionTtlMs: number - casMaxRetries: number } export function loadWorkerConfig(): WorkerConfig { @@ -56,16 +51,11 @@ export function loadWorkerConfig(): WorkerConfig { internalToken: env('INTERNAL_TOKEN'), home, codexHome: optEnv('CODEX_HOME', join(home, '.codex')), - vaultAddr: optEnv('VAULT_ADDR', 'http://vault.data-system.svc.cluster.local:8200'), - vaultK8sRole: env('VAULT_K8S_ROLE', 'agents-login-worker'), - vaultK8sMount: optEnv('VAULT_K8S_MOUNT', 'kubernetes'), - vaultKvMount: optEnv('VAULT_KV_MOUNT', 'secret'), - vaultClaudePath: optEnv('VAULT_CLAUDE_PATH', 'agents/claude-oauth'), - vaultCodexPath: optEnv('VAULT_CODEX_PATH', 'agents/codex-oauth'), - saTokenPath: optEnv('VAULT_SA_TOKEN_PATH', '/var/run/secrets/kubernetes.io/serviceaccount/token'), + agentsApiInternalUrl: optEnv('AGENTS_API_INTERNAL_URL', 'http://agents-api.agents-system.svc.cluster.local:8082'), + agentsApiInternalBearer: env('AGENTS_API_INTERNAL_BEARER'), + saTokenPath: optEnv('SA_TOKEN_PATH', '/var/run/secrets/kubernetes.io/serviceaccount/token'), leaseName: optEnv('LEASE_NAME', 'agents-login-write'), leaseNamespace: optEnv('LEASE_NAMESPACE', 'agents-system'), sessionTtlMs: intEnv('SESSION_TTL_MS', 15 * 60 * 1000), - casMaxRetries: intEnv('VAULT_CAS_MAX_RETRIES', 5), } } diff --git a/services/agents-login/src/shared/types.ts b/services/agents-login/src/shared/types.ts index 0b7cb302..c479dfbb 100644 --- a/services/agents-login/src/shared/types.ts +++ b/services/agents-login/src/shared/types.ts @@ -10,8 +10,8 @@ export type Provider = 'claude' | 'codex' * awaiting_device — Codex only: device code emitted, operator approves in the * browser; no paste-back needed * finalizing — login accepted by the CLI, capturing + writing credentials - * succeeded — credentials written to Vault - * failed — error (parse failure, child crash, persistent CAS conflict…) + * succeeded — credentials posted to agents-api + * failed — error (parse failure, child crash, ingest failure…) * cancelled — operator cancelled or session timed out */ export type SessionPhase = diff --git a/services/agents-login/src/worker/agentsApiClient.ts b/services/agents-login/src/worker/agentsApiClient.ts new file mode 100644 index 00000000..c78273ca --- /dev/null +++ b/services/agents-login/src/worker/agentsApiClient.ts @@ -0,0 +1,80 @@ +import type { Provider } from '../shared/types.js' +import type { CredentialBundle } from './credentials.js' + +export type AgentsApiProvider = 'CLAUDE' | 'CODEX' + +export interface AgentsApiClientOptions { + baseUrl: string + bearer: string +} + +export interface CredentialIngestRequest { + userId: string + provider: AgentsApiProvider + payload: Record +} + +export interface StoredCredentialStatus { + status: 'unknown' +} + +export class AgentsApiClient { + private readonly endpoint: string + + constructor( + opts: AgentsApiClientOptions, + private readonly fetchImpl: typeof fetch = fetch, + ) { + this.endpoint = new URL('/api/v1/internal/credentials', opts.baseUrl).toString() + this.bearer = opts.bearer + } + + private readonly bearer: string + + async postCredentials(req: CredentialIngestRequest): Promise { + const res = await this.fetchImpl(this.endpoint, { + method: 'POST', + headers: { + authorization: `Bearer ${this.bearer}`, + 'content-type': 'application/json', + }, + body: JSON.stringify(req), + }) + if (!res.ok) { + throw new Error(`agents-api credential ingest failed: ${res.status} ${await safeText(res)}`) + } + } + + storedStatus(): StoredCredentialStatus { + return { status: 'unknown' } + } +} + +export function providerForApi(provider: Provider): AgentsApiProvider { + return provider === 'claude' ? 'CLAUDE' : 'CODEX' +} + +export function payloadForApi(provider: Provider, bundle: CredentialBundle): Record { + if (provider === 'claude') { + const oauthToken = bundle.data.oauth_token + if (!oauthToken) { + throw new Error('no Claude OAuth token captured') + } + return { oauth_token: oauthToken } + } + + const authJson = bundle.data.auth_json + const configToml = bundle.data.config_toml + if (!authJson || !configToml) { + throw new Error('no Codex credential captured') + } + return { auth_json: authJson, config_toml: configToml } +} + +async function safeText(res: Response): Promise { + try { + return await res.text() + } catch { + return '' + } +} diff --git a/services/agents-login/src/worker/credentials.ts b/services/agents-login/src/worker/credentials.ts index 20648a06..8675c928 100644 --- a/services/agents-login/src/worker/credentials.ts +++ b/services/agents-login/src/worker/credentials.ts @@ -10,7 +10,7 @@ export interface CredentialPaths { } export interface CredentialBundle { - // Vault KV v2 path-relative field map plus metadata. + // Captured provider fields plus local bookkeeping used before ingest. data: Record } @@ -52,9 +52,6 @@ export async function captureClaude( if (!oauthToken && credentials === undefined) { throw new Error('no Claude credential captured: setup-token printed no token and no .credentials.json was written') } - // Vault KV v2 field keys are underscored; the VaultStaticSecret templates - // re-emit the consumer-facing shape (CLAUDE_CODE_OAUTH_TOKEN env + the - // dot-named files) into the projected Secret. const data: Record = { schema_version: SCHEMA_VERSION, updated_at: now().toISOString(), diff --git a/services/agents-login/src/worker/index.ts b/services/agents-login/src/worker/index.ts index db4ebd0c..5505a829 100644 --- a/services/agents-login/src/worker/index.ts +++ b/services/agents-login/src/worker/index.ts @@ -4,21 +4,16 @@ import { K8sLeaseLock } from './lease.js' import { createNodePtySpawner } from './pty.js' import { buildWorkerServer } from './server.js' import { SessionManager } from './session.js' -import { VaultClient } from './vaultClient.js' +import { AgentsApiClient } from './agentsApiClient.js' export async function runWorker(): Promise { const cfg = loadWorkerConfig() const logger = createLogger() - const vault = new VaultClient({ - addr: cfg.vaultAddr, - k8sRole: cfg.vaultK8sRole, - k8sMount: cfg.vaultK8sMount, - kvMount: cfg.vaultKvMount, - saTokenPath: cfg.saTokenPath, - maxCasRetries: cfg.casMaxRetries, + const agentsApi = new AgentsApiClient({ + baseUrl: cfg.agentsApiInternalUrl, + bearer: cfg.agentsApiInternalBearer, }) - await vault.login() const lease = new K8sLeaseLock({ name: cfg.leaseName, @@ -31,10 +26,9 @@ export async function runWorker(): Promise { const sessions = new SessionManager({ spawner, - vault, + agentsApi, lease, paths: { home: cfg.home, codexHome: cfg.codexHome }, - vaultPaths: { claude: cfg.vaultClaudePath, codex: cfg.vaultCodexPath }, logger, ttlMs: cfg.sessionTtlMs, }) @@ -43,8 +37,7 @@ export async function runWorker(): Promise { sessions, internalToken: cfg.internalToken, logger, - vault, - vaultPaths: { claude: cfg.vaultClaudePath, codex: cfg.vaultCodexPath }, + agentsApi, }) await app.listen({ host: cfg.host, port: cfg.port }) logger.info('worker listening', { host: cfg.host, port: cfg.port }) diff --git a/services/agents-login/src/worker/lease.ts b/services/agents-login/src/worker/lease.ts index 6857567a..06a70331 100644 --- a/services/agents-login/src/worker/lease.ts +++ b/services/agents-login/src/worker/lease.ts @@ -1,4 +1,4 @@ -// Named-lock abstraction backing the Vault write critical section. The real +// Named-lock abstraction backing the credential ingest critical section. The real // implementation acquires/renews a coordination.k8s.io Lease via the Kubernetes // API; tests inject a fake. diff --git a/services/agents-login/src/worker/server.ts b/services/agents-login/src/worker/server.ts index eef15e54..51ec4094 100644 --- a/services/agents-login/src/worker/server.ts +++ b/services/agents-login/src/worker/server.ts @@ -3,15 +3,13 @@ import type { Logger } from '../shared/log.js' import type { Provider, SessionStatus } from '../shared/types.js' import { redactString } from '../shared/redact.js' import type { SessionManager } from './session.js' -import type { VaultClient } from './vaultClient.js' +import type { AgentsApiClient } from './agentsApiClient.js' export interface WorkerServerDeps { sessions: SessionManager internalToken: string logger: Logger - // The stored-credential check reads non-secret bookkeeping from these paths. - vault: Pick - vaultPaths: { claude: string; codex: string } + agentsApi: Pick } function isProvider(v: unknown): v is Provider { @@ -19,7 +17,7 @@ function isProvider(v: unknown): v is Provider { } // The status payload never carries a captured credential — those go straight to -// Vault. authorizeUrl / verificationUrl / deviceCode are exactly what the UI +// agents-api. authorizeUrl / verificationUrl / deviceCode are exactly what the UI // must show the operator, and the OAuth `state` + PKCE `code_challenge` are long // base64url strings the generic token redactor would otherwise mask, breaking // the URL the operator has to open. Only the free-text message/error fields @@ -63,14 +61,11 @@ export function buildWorkerServer(deps: WorkerServerDeps): FastifyInstance { app.get('/healthz', async () => ({ ok: true })) - // The credential-status check: what is currently stored in Vault for each - // provider (version + when/who last refreshed), so the UI can confirm a login - // landed without exposing any secret material. + // There is no read-side agents-api credential endpoint. Report only a local + // placeholder so the UI can remain explicit that stored state is unknown. app.get('/status', async () => { - const [claude, codex] = await Promise.all([ - deps.vault.readStatus(deps.vaultPaths.claude), - deps.vault.readStatus(deps.vaultPaths.codex), - ]) + const claude = deps.agentsApi.storedStatus() + const codex = deps.agentsApi.storedStatus() return { claude, codex } }) diff --git a/services/agents-login/src/worker/session.ts b/services/agents-login/src/worker/session.ts index 61f5c7d6..adabda43 100644 --- a/services/agents-login/src/worker/session.ts +++ b/services/agents-login/src/worker/session.ts @@ -6,14 +6,13 @@ import { capture, type CredentialPaths } from './credentials.js' import { detectFailure, detectSuccess, parseClaude, parseClaudeToken, parseCodex } from './parse.js' import type { PtyProcess, PtySpawner } from './pty.js' import type { LeaseLock } from './lease.js' -import { CasConflictError, type VaultClient } from './vaultClient.js' +import { payloadForApi, providerForApi, type AgentsApiClient } from './agentsApiClient.js' export interface SessionDeps { spawner: PtySpawner - vault: VaultClient + agentsApi: Pick lease: LeaseLock paths: CredentialPaths - vaultPaths: { claude: string; codex: string } logger: Logger ttlMs: number now?: () => Date @@ -224,20 +223,17 @@ export class LoginSession { // prints to stdout, not a file; parse it from the accumulated buffer. const claudeToken = this.provider === 'claude' ? parseClaudeToken(this.buffer) : undefined const bundle = await capture(this.provider, this.deps.paths, updatedBy, this.now, claudeToken) - const path = this.provider === 'claude' ? this.deps.vaultPaths.claude : this.deps.vaultPaths.codex + const provider = providerForApi(this.provider) + const payload = payloadForApi(this.provider, bundle) await this.deps.lease.withLock(async () => { - await this.deps.vault.writeCas(path, bundle.data) + await this.deps.agentsApi.postCredentials({ userId: updatedBy, provider, payload }) }) this.clearTimer() this.proc?.kill() - this.setPhase('succeeded', 'Credentials written to Vault.') + this.setPhase('succeeded', 'Credentials written.') this.deps.logger.info('login session succeeded', { sessionId: this.id, provider: this.provider }) } catch (err) { - if (err instanceof CasConflictError) { - this.fail(`Vault write conflict — another writer is active. ${err.message}`) - } else { - this.fail(err instanceof Error ? err.message : 'failed to finalize login') - } + this.fail(err instanceof Error ? err.message : 'failed to finalize login') } } @@ -262,8 +258,8 @@ export class LoginSession { /** * Holds at most one active session per provider. Claude and Codex run their - * CLIs against separate HOME subtrees (`.claude` vs `CODEX_HOME`) and the Vault - * write is serialized by the Lease, so the two providers can log in + * CLIs against separate HOME subtrees (`.claude` vs `CODEX_HOME`) and the + * ingest write is serialized by the Lease, so the two providers can log in * concurrently; a second start for a provider whose session is still live * re-attaches to it rather than spawning a duplicate. */ diff --git a/services/agents-login/src/worker/vaultClient.ts b/services/agents-login/src/worker/vaultClient.ts deleted file mode 100644 index 6dd8a911..00000000 --- a/services/agents-login/src/worker/vaultClient.ts +++ /dev/null @@ -1,172 +0,0 @@ -// Vault KV v2 client with Kubernetes auth and Compare-And-Set writes. - -export interface VaultClientOptions { - addr: string - k8sRole: string - k8sMount: string - kvMount: string - saTokenPath: string - maxCasRetries: number -} - -export class CasConflictError extends Error { - constructor( - readonly path: string, - readonly attempts: number, - ) { - super(`persistent CAS conflict writing ${path} after ${attempts} attempts`) - this.name = 'CasConflictError' - } -} - -interface AuthResponse { - auth: { client_token: string; lease_duration: number } -} - -interface MetadataResponse { - data: { current_version: number } -} - -interface DataResponse { - data: { data: Record; metadata: { version: number } } -} - -/** Non-secret summary of a stored credential, for the UI's "check" panel. */ -export interface CredentialStatus { - exists: boolean - version: number - updatedAt?: string - updatedBy?: string - schemaVersion?: string -} - -export class VaultClient { - private token?: string - - constructor( - private readonly opts: VaultClientOptions, - private readonly fetchImpl: typeof fetch = fetch, - private readonly readToken: (p: string) => Promise = defaultReadToken, - ) {} - - /** Authenticate via the Kubernetes auth backend; caches the client token. */ - async login(): Promise { - const jwt = await this.readToken(this.opts.saTokenPath) - const url = `${this.opts.addr}/v1/auth/${this.opts.k8sMount}/login` - const res = await this.fetchImpl(url, { - method: 'POST', - headers: { 'content-type': 'application/json' }, - body: JSON.stringify({ role: this.opts.k8sRole, jwt }), - }) - if (!res.ok) { - throw new Error(`vault k8s login failed: ${res.status} ${await safeText(res)}`) - } - const body = (await res.json()) as AuthResponse - if (!body.auth?.client_token) { - throw new Error('vault k8s login returned no client_token') - } - this.token = body.auth.client_token - } - - private headers(): Record { - if (!this.token) { - throw new Error('vault client not authenticated; call login() first') - } - return { 'x-vault-token': this.token, 'content-type': 'application/json' } - } - - /** Current metadata version of a KV v2 secret, or 0 if it does not exist. */ - async currentVersion(path: string): Promise { - const url = `${this.opts.addr}/v1/${this.opts.kvMount}/metadata/${path}` - const res = await this.fetchImpl(url, { headers: this.headers() }) - if (res.status === 404) { - return 0 - } - if (!res.ok) { - throw new Error(`vault metadata read failed: ${res.status} ${await safeText(res)}`) - } - const body = (await res.json()) as MetadataResponse - return body.data?.current_version ?? 0 - } - - /** - * Read the non-secret bookkeeping fields (version + who/when last wrote) for - * the UI's credential-status check. The credential blobs in `data` are never - * returned. Reports `exists: false` when the path has no live secret. - */ - async readStatus(path: string): Promise { - const url = `${this.opts.addr}/v1/${this.opts.kvMount}/data/${path}` - const res = await this.fetchImpl(url, { headers: this.headers() }) - if (res.status === 404) { - return { exists: false, version: 0 } - } - if (!res.ok) { - throw new Error(`vault data read failed: ${res.status} ${await safeText(res)}`) - } - const body = (await res.json()) as DataResponse - const data = body.data?.data ?? {} - return { - exists: true, - version: body.data?.metadata?.version ?? 0, - updatedAt: data.updated_at, - updatedBy: data.updated_by, - schemaVersion: data.schema_version, - } - } - - private async putOnce(path: string, data: Record, casVersion: number): Promise { - const url = `${this.opts.addr}/v1/${this.opts.kvMount}/data/${path}` - const res = await this.fetchImpl(url, { - method: 'POST', - headers: this.headers(), - body: JSON.stringify({ options: { cas: casVersion }, data }), - }) - // Vault returns 400 with check-and-set messaging on a CAS mismatch. - if (res.status === 400) { - const text = await safeText(res) - if (/check-and-set|cas/i.test(text)) { - return false - } - throw new Error(`vault write failed: 400 ${text}`) - } - if (!res.ok) { - throw new Error(`vault write failed: ${res.status} ${await safeText(res)}`) - } - return true - } - - /** - * Write a KV v2 secret using Compare-And-Set against the current metadata - * version. On CAS conflict, re-read and retry with bounded backoff; on - * persistent conflict, throw CasConflictError rather than blind-overwrite. - */ - async writeCas(path: string, data: Record): Promise { - const max = Math.max(1, this.opts.maxCasRetries) - for (let attempt = 1; attempt <= max; attempt += 1) { - const version = await this.currentVersion(path) - const ok = await this.putOnce(path, data, version) - if (ok) { - return - } - await sleep(Math.min(2000, 50 * 2 ** (attempt - 1))) - } - throw new CasConflictError(path, max) - } -} - -async function defaultReadToken(path: string): Promise { - const { readFile } = await import('node:fs/promises') - return (await readFile(path, 'utf8')).trim() -} - -async function safeText(res: Response): Promise { - try { - return await res.text() - } catch { - return '' - } -} - -function sleep(ms: number): Promise { - return new Promise((r) => setTimeout(r, ms)) -} diff --git a/services/agents-login/test/agentsApiClient.test.ts b/services/agents-login/test/agentsApiClient.test.ts new file mode 100644 index 00000000..5652ca2b --- /dev/null +++ b/services/agents-login/test/agentsApiClient.test.ts @@ -0,0 +1,70 @@ +import { describe, expect, it } from 'vitest' +import { AgentsApiClient, payloadForApi, providerForApi } from '../src/worker/agentsApiClient.js' + +describe('AgentsApiClient', () => { + it('posts credentials to the internal ingest endpoint', async () => { + const calls: Array<{ url: string; init: RequestInit }> = [] + const client = new AgentsApiClient( + { baseUrl: 'http://agents-api.local:8082/base', bearer: 'secret-bearer' }, + async (url, init) => { + calls.push({ url: String(url), init: init ?? {} }) + return new Response(null, { status: 204 }) + }, + ) + + await client.postCredentials({ userId: 'alice', provider: 'CLAUDE', payload: { oauth_token: 'tok' } }) + + expect(calls).toHaveLength(1) + expect(calls[0].url).toBe('http://agents-api.local:8082/api/v1/internal/credentials') + expect(calls[0].init.method).toBe('POST') + expect(calls[0].init.headers).toEqual({ + authorization: 'Bearer secret-bearer', + 'content-type': 'application/json', + }) + expect(JSON.parse(String(calls[0].init.body))).toEqual({ + userId: 'alice', + provider: 'CLAUDE', + payload: { oauth_token: 'tok' }, + }) + }) + + it('fails loudly on non-2xx ingest responses', async () => { + const client = new AgentsApiClient( + { baseUrl: 'http://agents-api.local:8082', bearer: 'secret-bearer' }, + async () => new Response('nope', { status: 503 }), + ) + + await expect( + client.postCredentials({ userId: 'alice', provider: 'CLAUDE', payload: { oauth_token: 'tok' } }), + ).rejects.toThrow(/503 nope/) + }) + + it('maps provider names to the uppercase API enum', () => { + expect(providerForApi('claude')).toBe('CLAUDE') + expect(providerForApi('codex')).toBe('CODEX') + }) + + it('extracts only the required provider payload fields', () => { + expect( + payloadForApi('claude', { + data: { oauth_token: 'tok', credentials_json: '{}', updated_by: 'alice' }, + }), + ).toEqual({ oauth_token: 'tok' }) + + expect( + payloadForApi('codex', { + data: { auth_json: '{}', config_toml: 'model="x"', updated_by: 'bob' }, + }), + ).toEqual({ auth_json: '{}', config_toml: 'model="x"' }) + }) + + it('rejects incomplete provider payloads before posting', () => { + expect(() => payloadForApi('claude', { data: { credentials_json: '{}' } })).toThrow(/Claude OAuth token/) + expect(() => payloadForApi('codex', { data: { auth_json: '{}' } })).toThrow(/Codex credential/) + }) + + it('reports only unknown local stored status', () => { + const client = new AgentsApiClient({ baseUrl: 'http://agents-api.local:8082', bearer: 'secret-bearer' }) + expect(client.storedStatus()).toEqual({ status: 'unknown' }) + }) +}) diff --git a/services/agents-login/test/config.test.ts b/services/agents-login/test/config.test.ts index 8fd8dcfe..4be6c43f 100644 --- a/services/agents-login/test/config.test.ts +++ b/services/agents-login/test/config.test.ts @@ -9,14 +9,14 @@ describe('config', () => { k.startsWith('AGENTS_LOGIN') || [ 'INTERNAL_TOKEN', + 'AGENTS_API_INTERNAL_URL', + 'AGENTS_API_INTERNAL_BEARER', 'PORT', 'HOST', - 'VAULT_ADDR', - 'VAULT_K8S_ROLE', 'SESSION_TTL_MS', 'CODEX_HOME', 'LEASE_NAME', - 'VAULT_CAS_MAX_RETRIES', + 'SA_TOKEN_PATH', ].includes(k) ) { delete process.env[k] @@ -29,35 +29,42 @@ describe('config', () => { it('loads worker config with derived CODEX_HOME default', () => { process.env.INTERNAL_TOKEN = 'tok' + process.env.AGENTS_API_INTERNAL_BEARER = 'bearer' process.env.HOME = '/home/agent' const cfg = loadWorkerConfig() expect(cfg.codexHome).toBe('/home/agent/.codex') - expect(cfg.vaultKvMount).toBe('secret') - expect(cfg.vaultClaudePath).toBe('agents/claude-oauth') - expect(cfg.vaultCodexPath).toBe('agents/codex-oauth') + expect(cfg.agentsApiInternalUrl).toBe('http://agents-api.agents-system.svc.cluster.local:8082') + expect(cfg.agentsApiInternalBearer).toBe('bearer') expect(cfg.leaseName).toBe('agents-login-write') expect(cfg.leaseNamespace).toBe('agents-system') }) - it('honours an explicit CODEX_HOME and vault overrides', () => { + it('honours an explicit CODEX_HOME and agents-api URL override', () => { process.env.INTERNAL_TOKEN = 'tok' + process.env.AGENTS_API_INTERNAL_BEARER = 'bearer' process.env.HOME = '/home/agent' process.env.CODEX_HOME = '/custom/codex' - process.env.VAULT_K8S_ROLE = 'role-x' - process.env.VAULT_CAS_MAX_RETRIES = '9' + process.env.AGENTS_API_INTERNAL_URL = 'http://agents-api.local:8082' const cfg = loadWorkerConfig() expect(cfg.codexHome).toBe('/custom/codex') - expect(cfg.vaultK8sRole).toBe('role-x') - expect(cfg.casMaxRetries).toBe(9) + expect(cfg.agentsApiInternalUrl).toBe('http://agents-api.local:8082') }) it('throws when INTERNAL_TOKEN is missing', () => { + process.env.AGENTS_API_INTERNAL_BEARER = 'bearer' process.env.HOME = '/home/agent' expect(() => loadWorkerConfig()).toThrow(/INTERNAL_TOKEN/) }) + it('throws when AGENTS_API_INTERNAL_BEARER is missing', () => { + process.env.INTERNAL_TOKEN = 'tok' + process.env.HOME = '/home/agent' + expect(() => loadWorkerConfig()).toThrow(/AGENTS_API_INTERNAL_BEARER/) + }) + it('throws on a non-integer port', () => { process.env.INTERNAL_TOKEN = 'tok' + process.env.AGENTS_API_INTERNAL_BEARER = 'bearer' process.env.HOME = '/home/agent' process.env.PORT = 'abc' expect(() => loadWorkerConfig()).toThrow(/integer/) diff --git a/services/agents-login/test/helpers/fakeVault.ts b/services/agents-login/test/helpers/fakeVault.ts deleted file mode 100644 index ba8ffa3c..00000000 --- a/services/agents-login/test/helpers/fakeVault.ts +++ /dev/null @@ -1,121 +0,0 @@ -import { createServer, type Server } from 'node:http' -import type { AddressInfo } from 'node:net' - -interface SecretEntry { - version: number - data: Record -} - -export interface FakeVaultOptions { - // Force the first N writes to a path to fail with a CAS mismatch (simulating - // a racing writer), to exercise the retry / persistent-conflict paths. - conflictCount?: number - // Reject the k8s login with this status (default 200 OK). - loginStatus?: number -} - -/** - * Minimal in-process Vault KV v2 server speaking just enough of the API for the - * client: k8s login, metadata read, and CAS data write. - */ -export class FakeVault { - private server: Server - readonly store = new Map() - private remainingConflicts: number - readonly logins: Array<{ role: string; jwt: string }> = [] - url = '' - - constructor(private readonly opts: FakeVaultOptions = {}) { - this.remainingConflicts = opts.conflictCount ?? 0 - this.server = createServer((req, res) => this.handle(req, res)) - } - - async start(): Promise { - await new Promise((resolve) => this.server.listen(0, '127.0.0.1', resolve)) - const addr = this.server.address() as AddressInfo - this.url = `http://127.0.0.1:${addr.port}` - } - - async stop(): Promise { - await new Promise((resolve) => this.server.close(() => resolve())) - } - - private async body(req: import('node:http').IncomingMessage): Promise { - const chunks: Buffer[] = [] - for await (const c of req) { - chunks.push(c as Buffer) - } - return Buffer.concat(chunks).toString('utf8') - } - - private handle(req: import('node:http').IncomingMessage, res: import('node:http').ServerResponse): void { - void this.route(req, res).catch(() => { - res.statusCode = 500 - res.end('{"errors":["internal"]}') - }) - } - - private async route( - req: import('node:http').IncomingMessage, - res: import('node:http').ServerResponse, - ): Promise { - const url = req.url ?? '' - res.setHeader('content-type', 'application/json') - - if (url.endsWith('/login')) { - if (this.opts.loginStatus && this.opts.loginStatus !== 200) { - res.statusCode = this.opts.loginStatus - res.end('{"errors":["denied"]}') - return - } - const parsed = JSON.parse((await this.body(req)) || '{}') - this.logins.push({ role: parsed.role, jwt: parsed.jwt }) - res.statusCode = 200 - res.end(JSON.stringify({ auth: { client_token: 'fake-token', lease_duration: 3600 } })) - return - } - - // /v1//metadata/ - const metaMatch = url.match(/\/v1\/[^/]+\/metadata\/(.+)$/) - if (metaMatch && req.method === 'GET') { - const entry = this.store.get(metaMatch[1]) - if (!entry) { - res.statusCode = 404 - res.end('{"errors":[]}') - return - } - res.statusCode = 200 - res.end(JSON.stringify({ data: { current_version: entry.version } })) - return - } - - // /v1//data/ - const dataMatch = url.match(/\/v1\/[^/]+\/data\/(.+)$/) - if (dataMatch && req.method === 'POST') { - const path = dataMatch[1] - const payload = JSON.parse((await this.body(req)) || '{}') - const cas = payload.options?.cas - const existing = this.store.get(path) - const currentVersion = existing?.version ?? 0 - - if (this.remainingConflicts > 0) { - this.remainingConflicts -= 1 - res.statusCode = 400 - res.end('{"errors":["check-and-set parameter did not match the current version"]}') - return - } - if (cas !== currentVersion) { - res.statusCode = 400 - res.end('{"errors":["check-and-set parameter did not match the current version"]}') - return - } - this.store.set(path, { version: currentVersion + 1, data: payload.data }) - res.statusCode = 200 - res.end(JSON.stringify({ data: { version: currentVersion + 1 } })) - return - } - - res.statusCode = 404 - res.end('{"errors":["not found"]}') - } -} diff --git a/services/agents-login/test/integration.pty.test.ts b/services/agents-login/test/integration.pty.test.ts index 1a7cb5a5..03b7e684 100644 --- a/services/agents-login/test/integration.pty.test.ts +++ b/services/agents-login/test/integration.pty.test.ts @@ -2,11 +2,9 @@ import { describe, it, expect, beforeAll } from 'vitest' import { rmSync } from 'node:fs' import { SessionManager } from '../src/worker/session.js' import { NoopLeaseLock } from '../src/worker/lease.js' -import { VaultClient } from '../src/worker/vaultClient.js' import { createLogger } from '../src/shared/log.js' import { createNodePtySpawner } from '../src/worker/pty.js' import type { PtySpawner } from '../src/worker/pty.js' -import { FakeVault } from './helpers/fakeVault.js' import { makeFakeCliEnv, type FakeCliEnv } from './helpers/fakeCli.js' // Drives the worker through a REAL pseudo-terminal against fake `claude` / @@ -42,7 +40,7 @@ describe('real-PTY integration with fake CLIs', () => { // Local-only smoke test. It drives a REAL pseudo-terminal against fake CLI // scripts, which is timing-sensitive under CI load; the deterministic FakePty // tests in session.test.ts cover the same state machine, parsing, capture and - // Vault write. Skipped when CI is set so a PTY timing flake never gates merges. + // agents-api ingest. Skipped when CI is set so a PTY timing flake never gates merges. it.runIf(!process.env.CI)( 'captures Claude + Codex creds end to end via PTY', async () => { @@ -52,29 +50,17 @@ describe('real-PTY integration with fake CLIs', () => { } const cli: FakeCliEnv = makeFakeCliEnv() process.env.PATH = cli.pathEnv - - const vault = new FakeVault() - await vault.start() - const vaultClient = new VaultClient( - { - addr: vault.url, - k8sRole: 'r', - k8sMount: 'kubernetes', - kvMount: 'secret', - saTokenPath: '/x', - maxCasRetries: 3, - }, - fetch, - async () => 'jwt', - ) - await vaultClient.login() + const posts: Array<{ userId: string; provider: string; payload: Record }> = [] const mgr = new SessionManager({ spawner, - vault: vaultClient, + agentsApi: { + async postCredentials(req) { + posts.push(req) + }, + }, lease: new NoopLeaseLock(), paths: { home: cli.home, codexHome: cli.codexHome }, - vaultPaths: { claude: 'agents/claude-oauth', codex: 'agents/codex-oauth' }, logger: createLogger(), ttlMs: 30_000, }) @@ -86,19 +72,16 @@ describe('real-PTY integration with fake CLIs', () => { const sub = mgr.submitRedirectUrl(claude.id, 'https://claude.ai/redirect?code=2') expect(sub.ok).toBe(true) await waitFor(() => mgr.status(claude.id)!.phase === 'succeeded') - - const claudeSecret = vault.store.get('agents/claude-oauth') - expect(claudeSecret?.data['credentials_json']).toContain('claude-secret-token') - expect(claudeSecret?.data['claude_json']).toContain('oauthAccount') + expect(posts[0]).toMatchObject({ userId: 'alice', provider: 'CLAUDE' }) + expect(posts[0].payload.oauth_token).toContain('sk-ant-oat01') // Codex: device flow, no paste-back. const codex = mgr.start('codex', 'alice') await waitFor(() => mgr.status(codex.id)!.phase === 'succeeded') - const codexSecret = vault.store.get('agents/codex-oauth') - expect(codexSecret?.data['auth_json']).toContain('codex-secret') - expect(codexSecret?.data['config_toml']).toContain('gpt-5') + expect(posts[1]).toMatchObject({ userId: 'alice', provider: 'CODEX' }) + expect(posts[1].payload.auth_json).toContain('codex-secret') + expect(posts[1].payload.config_toml).toContain('gpt-5') - await vault.stop() rmSync(cli.binDir.replace(/\/bin$/, ''), { recursive: true, force: true }) }, 20_000, diff --git a/services/agents-login/test/session.test.ts b/services/agents-login/test/session.test.ts index 54f80e91..2aec3c3c 100644 --- a/services/agents-login/test/session.test.ts +++ b/services/agents-login/test/session.test.ts @@ -4,10 +4,8 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { SessionManager, type SessionDeps } from '../src/worker/session.js' import { NoopLeaseLock } from '../src/worker/lease.js' -import { VaultClient } from '../src/worker/vaultClient.js' import { createLogger } from '../src/shared/log.js' import { fakeSpawner, type Action } from './helpers/fakePty.js' -import { FakeVault } from './helpers/fakeVault.js' async function tick(times = 8): Promise { for (let i = 0; i < times; i += 1) { @@ -15,8 +13,27 @@ async function tick(times = 8): Promise { } } +interface PostedCredential { + userId: string + provider: 'CLAUDE' | 'CODEX' + payload: Record +} + +function fakeAgentsApi(fail?: Error): { postCredentials: (req: PostedCredential) => Promise; posts: PostedCredential[] } { + const posts: PostedCredential[] = [] + return { + posts, + async postCredentials(req: PostedCredential) { + if (fail) { + throw fail + } + posts.push(req) + }, + } +} + // Poll until the session reaches one of the expected phases (handles the async -// finalize that awaits credential capture + the lease + the Vault round-trip). +// finalize that awaits credential capture + the lease + the agents-api POST). async function waitPhase( mgr: { status: (id?: string) => { phase: string } | undefined }, id: string, @@ -37,34 +54,18 @@ describe('LoginSession state machine', () => { let root: string let home: string let codexHome: string - let vault: FakeVault - let vaultClient: VaultClient + let agentsApi: ReturnType - beforeEach(async () => { + beforeEach(() => { root = mkdtempSync(join(tmpdir(), 'sess-')) home = join(root, 'home') codexHome = join(home, '.codex') mkdirSync(join(home, '.claude'), { recursive: true }) mkdirSync(codexHome, { recursive: true }) - vault = new FakeVault() - await vault.start() - vaultClient = new VaultClient( - { - addr: vault.url, - k8sRole: 'r', - k8sMount: 'kubernetes', - kvMount: 'secret', - saTokenPath: '/x', - maxCasRetries: 3, - }, - fetch, - async () => 'jwt', - ) - await vaultClient.login() + agentsApi = fakeAgentsApi() }) - afterEach(async () => { - await vault.stop() + afterEach(() => { rmSync(root, { recursive: true, force: true }) }) @@ -79,10 +80,9 @@ describe('LoginSession state machine', () => { return { deps: { spawner, - vault: vaultClient, + agentsApi, lease, paths: { home, codexHome }, - vaultPaths: { claude: 'agents/claude-oauth', codex: 'agents/codex-oauth' }, logger: createLogger(), ttlMs: 60_000, }, @@ -90,9 +90,10 @@ describe('LoginSession state machine', () => { } } - it('drives the full Claude flow: authorize URL → redirect paste-back → success → Vault write', async () => { + it('drives the full Claude flow: authorize URL → redirect paste-back → success → agents-api POST', async () => { writeFileSync(join(home, '.claude', '.credentials.json'), '{"accessToken":"secret"}') writeFileSync(join(home, '.claude.json'), '{"oauthAccount":{}}') + const token = 'sk-ant-oat01-FullFlowToken1234567890_-abcXYZ' const { deps: d } = deps(() => [ { type: 'emit', data: 'Visit https://claude.ai/oauth/authorize?code=1\r\n' }, @@ -100,7 +101,7 @@ describe('LoginSession state machine', () => { type: 'expectStdin', match: (i) => i.includes('https://claude.ai/redirect'), then: [ - { type: 'emit', data: 'Login successful.\r\n' }, + { type: 'emit', data: `Login successful.\r\nYour OAuth token: ${token}\r\n` }, { type: 'exit', code: 0 }, ], }, @@ -119,8 +120,9 @@ describe('LoginSession state machine', () => { expect(sub.ok).toBe(true) expect(await waitPhase(mgr, started.id, ['succeeded', 'failed'])).toBe('succeeded') - expect(vault.store.get('agents/claude-oauth')?.data['claude_json']).toBe('{"oauthAccount":{}}') - expect(vault.store.get('agents/claude-oauth')?.data.updated_by).toBe('alice') + expect(agentsApi.posts).toEqual([ + { userId: 'alice', provider: 'CLAUDE', payload: { oauth_token: token } }, + ]) }) it('captures the setup-token OAuth token from stdout when no credentials file is written', async () => { @@ -145,8 +147,9 @@ describe('LoginSession state machine', () => { await tick() expect(mgr.submitRedirectUrl(started.id, 'paste-code-xyz').ok).toBe(true) expect(await waitPhase(mgr, started.id, ['succeeded', 'failed'])).toBe('succeeded') - expect(vault.store.get('agents/claude-oauth')?.data['oauth_token']).toBe(token) - expect(vault.store.get('agents/claude-oauth')?.data['credentials_json']).toBeUndefined() + expect(agentsApi.posts).toEqual([ + { userId: 'alice', provider: 'CLAUDE', payload: { oauth_token: token } }, + ]) }) it('drives the Codex device flow with no paste-back', async () => { @@ -161,7 +164,9 @@ describe('LoginSession state machine', () => { const mgr = new SessionManager(d) const started = mgr.start('codex', 'bob') expect(await waitPhase(mgr, started.id, ['succeeded', 'failed'])).toBe('succeeded') - expect(vault.store.get('agents/codex-oauth')?.data['auth_json']).toBe('{"tokens":{}}') + expect(agentsApi.posts).toEqual([ + { userId: 'bob', provider: 'CODEX', payload: { auth_json: '{"tokens":{}}', config_toml: 'model="x"\n' } }, + ]) }) it('surfaces the device code before success', async () => { @@ -277,29 +282,24 @@ describe('LoginSession state machine', () => { expect(instances.length).toBe(1) }) - it('surfaces a persistent Vault CAS conflict as a failed session', async () => { + it('surfaces an agents-api ingest failure as a failed session', async () => { writeFileSync(join(home, '.claude', '.credentials.json'), '{}') writeFileSync(join(home, '.claude.json'), '{}') - await vault.stop() - vault = new FakeVault({ conflictCount: 99 }) - await vault.start() - const conflictClient = new VaultClient( - { addr: vault.url, k8sRole: 'r', k8sMount: 'kubernetes', kvMount: 'secret', saTokenPath: '/x', maxCasRetries: 2 }, - fetch, - async () => 'jwt', - ) - await conflictClient.login() + const failingAgentsApi = fakeAgentsApi(new Error('agents-api credential ingest failed: 503 unavailable')) const { spawner } = fakeSpawner(() => [ { type: 'emit', data: 'Visit https://claude.ai/oauth/authorize?code=1\r\n' }, - { type: 'expectStdin', match: () => true, then: [{ type: 'emit', data: 'Login successful.\r\n' }] }, + { + type: 'expectStdin', + match: () => true, + then: [{ type: 'emit', data: 'Login successful.\r\nYour OAuth token: sk-ant-oat01-FailureToken1234567890\r\n' }], + }, ]) const mgr = new SessionManager({ spawner, - vault: conflictClient, + agentsApi: failingAgentsApi, lease: new NoopLeaseLock(), paths: { home, codexHome }, - vaultPaths: { claude: 'agents/claude-oauth', codex: 'agents/codex-oauth' }, logger: createLogger(), ttlMs: 60_000, }) @@ -307,7 +307,7 @@ describe('LoginSession state machine', () => { await tick() mgr.submitRedirectUrl(started.id, 'https://claude.ai/redirect') expect(await waitPhase(mgr, started.id, ['failed', 'succeeded'])).toBe('failed') - expect(mgr.status(started.id)!.error).toMatch(/conflict/i) + expect(mgr.status(started.id)!.error).toMatch(/agents-api credential ingest failed/) }) it('returns undefined status for an unknown id', () => { diff --git a/services/agents-login/test/vaultClient.test.ts b/services/agents-login/test/vaultClient.test.ts deleted file mode 100644 index 63e45b02..00000000 --- a/services/agents-login/test/vaultClient.test.ts +++ /dev/null @@ -1,183 +0,0 @@ -import { describe, it, expect, afterEach } from 'vitest' -import { VaultClient, CasConflictError } from '../src/worker/vaultClient.js' -import { FakeVault } from './helpers/fakeVault.js' - -function client(vault: FakeVault, maxCasRetries = 5): VaultClient { - return new VaultClient( - { - addr: vault.url, - k8sRole: 'agents-login', - k8sMount: 'kubernetes', - kvMount: 'secret', - saTokenPath: '/unused', - maxCasRetries, - }, - fetch, - async () => 'fake-sa-jwt', - ) -} - -describe('VaultClient', () => { - let vault: FakeVault - afterEach(async () => { - await vault.stop() - }) - - it('authenticates via the kubernetes auth backend', async () => { - vault = new FakeVault() - await vault.start() - const c = client(vault) - await c.login() - expect(vault.logins).toHaveLength(1) - expect(vault.logins[0]).toEqual({ role: 'agents-login', jwt: 'fake-sa-jwt' }) - }) - - it('throws when k8s login is rejected', async () => { - vault = new FakeVault({ loginStatus: 403 }) - await vault.start() - await expect(client(vault).login()).rejects.toThrow(/login failed/) - }) - - it('writes a fresh secret with CAS version 0', async () => { - vault = new FakeVault() - await vault.start() - const c = client(vault) - await c.login() - await c.writeCas('agents/claude-oauth', { '.claude.json': '{}', schema_version: '1' }) - expect(vault.store.get('agents/claude-oauth')?.version).toBe(1) - }) - - it('retries on a transient CAS conflict then succeeds', async () => { - vault = new FakeVault({ conflictCount: 2 }) - await vault.start() - const c = client(vault) - await c.login() - await c.writeCas('agents/codex-oauth', { 'auth.json': '{}' }) - expect(vault.store.get('agents/codex-oauth')?.version).toBe(1) - }) - - it('fails loudly with CasConflictError on persistent conflict', async () => { - vault = new FakeVault({ conflictCount: 99 }) - await vault.start() - const c = client(vault, 3) - await c.login() - await expect(c.writeCas('agents/claude-oauth', { a: 'b' })).rejects.toBeInstanceOf(CasConflictError) - }) - - it('reports current version 0 for a missing secret', async () => { - vault = new FakeVault() - await vault.start() - const c = client(vault) - await c.login() - expect(await c.currentVersion('agents/missing')).toBe(0) - }) - - it('refuses to write before login', async () => { - vault = new FakeVault() - await vault.start() - const c = client(vault) - await expect(c.writeCas('x', { a: 'b' })).rejects.toThrow(/not authenticated/) - }) - - it('throws on a non-CAS 400 write error', async () => { - vault = new FakeVault() - await vault.start() - const badFetch = (async (input: string | URL | Request) => { - const url = input.toString() - if (url.endsWith('/login')) { - return new Response(JSON.stringify({ auth: { client_token: 't', lease_duration: 1 } }), { - status: 200, - }) - } - if (url.includes('/metadata/')) { - return new Response('{}', { status: 404 }) - } - return new Response('{"errors":["malformed request"]}', { status: 400 }) - }) as unknown as typeof fetch - const c = new VaultClient( - { addr: vault.url, k8sRole: 'r', k8sMount: 'kubernetes', kvMount: 'secret', saTokenPath: '/x', maxCasRetries: 2 }, - badFetch, - async () => 'jwt', - ) - await c.login() - await expect(c.writeCas('p', { a: 'b' })).rejects.toThrow(/400/) - }) - - it('throws when metadata read fails non-404', async () => { - vault = new FakeVault() - await vault.start() - const badFetch = (async (input: string | URL | Request) => { - const url = input.toString() - if (url.endsWith('/login')) { - return new Response(JSON.stringify({ auth: { client_token: 't', lease_duration: 1 } }), { - status: 200, - }) - } - return new Response('boom', { status: 500 }) - }) as unknown as typeof fetch - const c = new VaultClient( - { addr: vault.url, k8sRole: 'r', k8sMount: 'kubernetes', kvMount: 'secret', saTokenPath: '/x', maxCasRetries: 2 }, - badFetch, - async () => 'jwt', - ) - await c.login() - await expect(c.currentVersion('p')).rejects.toThrow(/metadata read failed/) - }) - - it('reads non-secret status fields and never returns the secret blob', async () => { - vault = new FakeVault() - await vault.start() - const stubFetch = (async (input: string | URL | Request) => { - const url = input.toString() - if (url.endsWith('/login')) { - return new Response(JSON.stringify({ auth: { client_token: 't', lease_duration: 1 } }), { status: 200 }) - } - if (url.includes('/data/agents/claude-oauth')) { - return new Response( - JSON.stringify({ - data: { - data: { - credentials_json: 'TOP-SECRET-BLOB', - updated_at: '2026-06-23T10:00:00Z', - updated_by: 'ExtraToast', - schema_version: '1', - }, - metadata: { version: 4 }, - }, - }), - { status: 200 }, - ) - } - return new Response('{}', { status: 404 }) - }) as unknown as typeof fetch - const c = new VaultClient( - { addr: vault.url, k8sRole: 'r', k8sMount: 'kubernetes', kvMount: 'secret', saTokenPath: '/x', maxCasRetries: 2 }, - stubFetch, - async () => 'jwt', - ) - await c.login() - const st = await c.readStatus('agents/claude-oauth') - expect(st).toEqual({ - exists: true, - version: 4, - updatedAt: '2026-06-23T10:00:00Z', - updatedBy: 'ExtraToast', - schemaVersion: '1', - }) - expect(JSON.stringify(st)).not.toContain('TOP-SECRET-BLOB') - expect(await c.readStatus('agents/missing')).toEqual({ exists: false, version: 0 }) - }) - - it('throws when login returns no client_token', async () => { - vault = new FakeVault() - await vault.start() - const badFetch = (async () => - new Response(JSON.stringify({ auth: {} }), { status: 200 })) as unknown as typeof fetch - const c = new VaultClient( - { addr: vault.url, k8sRole: 'r', k8sMount: 'kubernetes', kvMount: 'secret', saTokenPath: '/x', maxCasRetries: 2 }, - badFetch, - async () => 'jwt', - ) - await expect(c.login()).rejects.toThrow(/no client_token/) - }) -}) diff --git a/services/agents-login/test/workerServer.test.ts b/services/agents-login/test/workerServer.test.ts index b4944636..94c0abf6 100644 --- a/services/agents-login/test/workerServer.test.ts +++ b/services/agents-login/test/workerServer.test.ts @@ -9,28 +9,19 @@ function manager() { { type: 'emit', data: 'Visit https://claude.ai/oauth/authorize?code=1\r\n' }, { type: 'expectStdin', match: () => true, then: [] }, ]) - // vault/lease are never reached in these route tests (no success emitted). + // agentsApi/lease are never reached in these route tests (no success emitted). return new SessionManager({ spawner, - vault: {} as never, + agentsApi: {} as never, lease: {} as never, paths: { home: '/tmp', codexHome: '/tmp/.codex' }, - vaultPaths: { claude: 'agents/claude-oauth', codex: 'agents/codex-oauth' }, logger: createLogger(), ttlMs: 60_000, }) } const TOKEN = 'internal-secret' -const VAULT_PATHS = { claude: 'agents/claude-oauth', codex: 'agents/codex-oauth' } - -function fakeVault( - overrides: Record = {}, -) { - return { - readStatus: async (path: string) => overrides[path] ?? { exists: false, version: 0 }, - } -} +const fakeAgentsApi = { storedStatus: () => ({ status: 'unknown' as const }) } describe('worker HTTP server', () => { let app: ReturnType @@ -40,8 +31,7 @@ describe('worker HTTP server', () => { sessions: manager(), internalToken: TOKEN, logger: createLogger(), - vault: fakeVault(), - vaultPaths: VAULT_PATHS, + agentsApi: fakeAgentsApi, }) }) @@ -125,27 +115,19 @@ describe('worker HTTP server', () => { expect(res.json().ok).toBe(true) }) - it('reports per-provider stored-credential status (no secrets)', async () => { + it('reports local unknown stored-credential status (no secrets)', async () => { const sessions = manager() const local = buildWorkerServer({ sessions, internalToken: TOKEN, logger: createLogger(), - vault: fakeVault({ - 'agents/claude-oauth': { exists: true, version: 3, updatedAt: '2026-06-23T10:00:00Z', updatedBy: 'ExtraToast' }, - }), - vaultPaths: VAULT_PATHS, + agentsApi: fakeAgentsApi, }) const res = await local.inject({ method: 'GET', url: '/status', headers: { 'x-internal-token': TOKEN } }) expect(res.statusCode).toBe(200) const body = res.json() - expect(body.claude).toEqual({ - exists: true, - version: 3, - updatedAt: '2026-06-23T10:00:00Z', - updatedBy: 'ExtraToast', - }) - expect(body.codex).toEqual({ exists: false, version: 0 }) + expect(body.claude).toEqual({ status: 'unknown' }) + expect(body.codex).toEqual({ status: 'unknown' }) }) it('requires the internal token for /status', async () => { @@ -169,10 +151,9 @@ describe('worker HTTP server', () => { ]) const sessions = new SessionManager({ spawner, - vault: {} as never, + agentsApi: {} as never, lease: {} as never, paths: { home: '/tmp', codexHome: '/tmp/.codex' }, - vaultPaths: { claude: 'agents/claude-oauth', codex: 'agents/codex-oauth' }, logger: createLogger(), ttlMs: 60_000, }) @@ -180,8 +161,7 @@ describe('worker HTTP server', () => { sessions, internalToken: TOKEN, logger: createLogger(), - vault: fakeVault(), - vaultPaths: VAULT_PATHS, + agentsApi: fakeAgentsApi, }) const headers = { 'x-internal-token': TOKEN } const start = await local.inject({ method: 'POST', url: '/sessions', headers, payload: { provider: 'claude' } }) diff --git a/services/agents-login/vitest.config.ts b/services/agents-login/vitest.config.ts index eb6fe54c..18720d89 100644 --- a/services/agents-login/vitest.config.ts +++ b/services/agents-login/vitest.config.ts @@ -8,7 +8,7 @@ export default defineConfig({ provider: 'v8', reporter: ['text', 'text-summary'], include: ['src/**/*.ts'], - // Entrypoints wire real Vault / node-pty / network and are exercised via + // Entrypoints wire node-pty / network and are exercised via // their composed units; they hold no branching logic worth gating on. exclude: ['src/index.ts', 'src/**/index.ts', 'src/worker/pty.ts'], thresholds: { From cd3107b995a6a2616f223f709fc622e0b928e011 Mon Sep 17 00:00:00 2001 From: Agents Agent Date: Thu, 25 Jun 2026 15:13:38 +0200 Subject: [PATCH 2/2] ci: drop agents OAuth Vault assertions from bootstrap-validate The agents/claude-oauth + agents/codex-oauth Vault paths and the agents-oauth-writer role were removed (OAuth tokens now persist in agents-api Postgres), so the bootstrap-validate workflow no longer asserts them. --- .../workflows/vault-bootstrap-validate.yml | 49 ------------------- 1 file changed, 49 deletions(-) diff --git a/.github/workflows/vault-bootstrap-validate.yml b/.github/workflows/vault-bootstrap-validate.yml index a8187a36..8d8399b3 100644 --- a/.github/workflows/vault-bootstrap-validate.yml +++ b/.github/workflows/vault-bootstrap-validate.yml @@ -201,7 +201,6 @@ jobs: # with the script when a policy lands or is retired. required_policies=( agents-api - agents-oauth-writer auth-api downloads knowledge-api @@ -228,51 +227,3 @@ jobs: exit 1 fi echo "all required policies uploaded successfully" - - - name: Assert the OAuth-credential RBAC boundary - # The bootstrap log only proves a policy was uploaded, not that - # its capabilities are correct. Mint a token under each policy and - # exercise the boundary that matters: agents-api (deploy keys) must - # NOT be able to write the OAuth paths or destroy their version - # history, and agents-oauth-writer must be able to write the OAuth - # data and read its metadata but NOT delete it. A helper inverts - # the exit code so a *successful* denied-operation fails the step. - env: - VAULT_ADDR: http://127.0.0.1:8200 - VAULT_TOKEN: dev-root-token - run: | - set -euo pipefail - - must_deny() { - # Runs "$@" with a scoped token; fails the step if it SUCCEEDS. - if VAULT_TOKEN="${SCOPED}" "$@" >/dev/null 2>&1; then - echo "FAIL: expected permission denied but the operation succeeded: $*" >&2 - exit 1 - fi - } - - AGENTS_API=$(vault token create -policy=agents-api -ttl=5m -field=token) - OAUTH_WRITER=$(vault token create -policy=agents-oauth-writer -ttl=5m -field=token) - - # agents-api keeps its deploy-key write. - VAULT_TOKEN="${AGENTS_API}" vault kv put secret/agents/repositories/ci-test key=value >/dev/null - echo "ok: agents-api can write agents/repositories/*" - - # agents-api must NOT reach the OAuth data or destroy its metadata. - SCOPED="${AGENTS_API}" must_deny vault kv put secret/agents/claude-oauth credentials_json=nope - echo "ok: agents-api denied write on agents/claude-oauth" - vault kv put secret/agents/claude-oauth credentials_json=seed >/dev/null - SCOPED="${AGENTS_API}" must_deny vault kv metadata delete secret/agents/claude-oauth - echo "ok: agents-api denied metadata delete on agents/claude-oauth" - - # agents-oauth-writer can write data + read metadata for CAS, - # but must NOT be able to delete (destroy) the version history. - VAULT_TOKEN="${OAUTH_WRITER}" vault kv put secret/agents/claude-oauth credentials_json=v1 >/dev/null - VAULT_TOKEN="${OAUTH_WRITER}" vault kv put secret/agents/codex-oauth auth_json=v1 >/dev/null - echo "ok: agents-oauth-writer can write the OAuth data paths" - VAULT_TOKEN="${OAUTH_WRITER}" vault kv metadata get secret/agents/claude-oauth >/dev/null - echo "ok: agents-oauth-writer can read OAuth metadata (CAS version checks)" - SCOPED="${OAUTH_WRITER}" must_deny vault kv metadata delete secret/agents/claude-oauth - echo "ok: agents-oauth-writer denied metadata delete on agents/claude-oauth" - - echo "OAuth-credential RBAC boundary holds"