feat(index): reclaim superseded index versions, and refuse to delete the live one

.s2s/BACKLOG.md

@@ -899,8 +899,9 @@ Swagger at `/docs` is auto-generated and complete, but the markdown reference do
 
 ### DEBT-032: no way to clean up an old index version after a reindex
 
-**Status**: planned | **Priority**: high | **Created**: 2026-07-14 | **Raised to high**: 2026-07-14
+**Status**: completed (2026-07-14) | **Priority**: high | **Created**: 2026-07-14 | **Raised to high**: 2026-07-14 | **PR**: #108
 **Origin**: DOCS-009 (2026-07-14), found while documenting the reindex flow end to end.
+**Resolution**: `VectorStoreProvider` gained `delete_index_version(namespace, index_version)`, implemented by Qdrant and pgvector, exposed as `DELETE /api/v1/index-versions/{version}` (admin). The guard lives in the providers, not the endpoint: a store is the only component that knows which version it reads, so it raises `ActiveIndexVersionError` (409) rather than trusting callers to check first. `scripts/reindex.sh --cleanup OLD_VERSION` completes the lifecycle as a second run, because the switch sits between the two and the API correctly refuses a cleanup before it. Verified on the dev stack: reindexing `default` to v2 doubled the namespace (12 -> 24 points), the switch left live search byte-identical, the cleanup reclaimed exactly the 12 old points, and the same call against the *active* version was refused with 409 while the six other namespaces (all at version 1) lost nothing.
 
 **Why high, and why this is not really "debt"**: REQ-064 spells the cleanup out as part of the requirement ("new chunks created with incremented version alongside old, atomic switch via config change, **cleanup of old version afterwards**"). So this is not a suboptimal-but-working solution: it is an acceptance criterion of a shipped requirement that was never built, while the module docstring tells the operator it exists. Code that promises a capability it does not have is the same disease as BUG-023, one level up.
 
@@ -913,10 +914,10 @@ The module docstring in `vektra-index/src/vektra_index/reindex.py` promises the
 Consequences: every reindex permanently doubles the storage for that namespace, and the only way to reclaim it is a hand-written delete against the store (a Qdrant filter delete, or `DELETE FROM document_chunks WHERE index_version = N`). That is an irreversible operation with no namespace guard, run by hand, at exactly the moment the operator is least sure of what is live — the failure mode being the deletion of the *active* version, which empties the live index. `docs/reference/api.md` documents the manual procedure with the warnings it needs, but the procedure should not be manual.
 
 **Acceptance criteria**:
-- [ ] `VectorStoreProvider` gains a version-scoped delete, implemented by both pgvector and Qdrant
-- [ ] An admin endpoint exposes it and **refuses to delete the version the system is currently serving**, with a test that proves the refusal: the destructive failure mode here is not "an old version survives", it is "the live index is emptied"
-- [ ] `scripts/reindex.sh` can complete the lifecycle
-- [ ] `docs/reference/api.md` replaces the manual store-level procedure with the endpoint
+- [x] `VectorStoreProvider` gains a version-scoped delete, implemented by both pgvector and Qdrant
+- [x] An admin endpoint exposes it and **refuses to delete the version the system is currently serving**, with a test that proves the refusal: the destructive failure mode here is not "an old version survives", it is "the live index is emptied"
+- [x] `scripts/reindex.sh` can complete the lifecycle
+- [x] `docs/reference/api.md` replaces the manual store-level procedure with the endpoint
 
 **Traceability**: REQ-064 (unimplemented acceptance criterion), ARCH-045 (index versioning), ADR-0026 (the Protocol that needs the version-scoped delete), BUG-023 (whose fix made this live)
 
@@ -929,6 +930,8 @@ Consequences: every reindex permanently doubles the storage for that namespace,
 
 **Context**: REQ-010 defines a single error envelope (`{"error": {category, code, message, remediation, request_id, details}}`), and `docs/reference/api.md` presented it as universal. It is not. Reindex (`400`, `404`), conversations (`404`, `503`) and admin conversation turns (`404`, `501`, `503`) raise `HTTPException` with a plain string detail, so they return FastAPI's bare `{"detail": "..."}` — no code, no remediation, no request id.
 
+**Scope reduced (2026-07-14, DEBT-032 / #108)**: the new `DELETE /api/v1/index-versions/{version}` was built enveloped from the start (`ERR-INDEX-001`, `ERR-INDEX-002`), so it is *not* part of this cleanup. What remains is the pre-existing set: `POST /reindex`, `GET /reindex/{job_id}/status`, conversations, admin conversation turns.
+
 A client cannot branch on an error code for these endpoints, and the operator-facing reindex failures are exactly the ones where a machine-readable code would be worth having. The doc now warns that both shapes exist; the fix is to make the envelope actually universal.
 
 **Acceptance criteria**:

CHANGELOG.md

@@ -22,6 +22,7 @@ Convention (Keep a Changelog 1.1.0):
 - **ci**: publish versioned container images to GHCR on tag push (INFRA-007). `v*` tags trigger `.github/workflows/publish.yml`, which builds and pushes `ghcr.io/vektralabs/vektra:{version}` and `:{version}-ocr` (GHA build cache, OCI version/revision labels, built-in `GITHUB_TOKEN`, no new secrets). Deployment docs and a new `deploy/docker-compose.image.yml.example` overlay cover the resulting `docker compose pull && docker compose up -d` flow as an alternative to building from source. No `latest` tag. The manual tagging flow is unchanged.
 - **index**: `GET /api/v1/documents/{document_id}/chunks` lists a document's stored chunks (text, position, parent link, metadata) at the active index version, read from the active vector store. Any valid scope.
 - **index**: `reindex_jobs.chunks_reindexed` (migration `0007`) records how many chunks a reindex actually re-embedded and wrote, exposed on `GET /api/v1/reindex/{job_id}/status`. A job that walked every document and stored nothing used to be indistinguishable from a real one.
+- **index**: `DELETE /api/v1/index-versions/{version}` (admin) reclaims a superseded index version, completing the reindex lifecycle REQ-064 always specified ("cleanup of old version afterwards") and nothing implemented (DEBT-032). A reindex writes a second copy of every chunk under the target version and leaves both in the store; until now nothing removed the loser, so **every reindex doubled a namespace's storage permanently** and the only way back was a hand-written delete against the store. The gap was live only because BUG-023 was fixed: while reindex wrote nothing, there was never an old version to clean up. `VectorStoreProvider` grows `delete_index_version(namespace, index_version)`, implemented by both Qdrant and pgvector, and `scripts/reindex.sh --cleanup OLD_VERSION [NAMESPACE]` drives it. **The store refuses to delete the version it is currently serving** (409, `ERR-INDEX-001`), because the failure mode that matters here is not "an old version survives" but "the live index is emptied": the hand-written delete this replaces has no idea which version is live and is run at precisely the moment the operator is least sure. The refusal lives in the providers, not the endpoint, since a store is the only component that knows which version it reads — so it holds for every caller, and it is why the cleanup is a second run of the script rather than a flag on the first: the switch sits between them. The call is idempotent (a second one returns `chunks_removed: 0`) and namespace-bound keys cannot clean up another namespace's version (403). Verified against the live Qdrant stack: reindexing a namespace to v2 doubled it (12 → 24 points), the switch left search byte-identical, the cleanup reclaimed exactly the 12 old points, deleting the *active* version was refused, and the six other namespaces — all sitting at version 1 — lost nothing.
 
 ### Fixed
 

docs/reference/api.md

@@ -941,7 +941,7 @@ Re-embed a namespace under a **new index version** without downtime (REQ-064, AR
 
 The mechanism: chunks are versioned. Every read (search, stats, chunk listing) filters on the **active** index version, set by `VEKTRA_ACTIVE_INDEX_VERSION` (default `1`). A reindex writes a second copy of the chunks under the target version, **alongside** the live ones, which keep serving traffic. The new version becomes live only when the operator changes the env var and restarts. Nothing is deleted along the way, so a bad reindex is rolled back by simply not switching.
 
-The switch and the cleanup are manual. There is **no** API to activate a version or to delete an old one: activation is an env var plus a restart, and cleanup is a store-level delete (see [Step 5](#step-5-clean-up-the-old-version)).
+The switch is manual: there is **no** API to activate a version, it is an env var plus a restart. The cleanup afterwards is an API call, `DELETE /api/v1/index-versions/{version}` (see [Step 5](#step-5-clean-up-the-old-version)), which refuses to delete whichever version is currently being served.
 
 ### POST /api/v1/reindex
 
@@ -1022,9 +1022,44 @@ Errors: `404` if the job id is unknown, or belongs to a different namespace than
 
 **Why `chunks_reindexed` matters.** Until BUG-023 the job read chunks from a Postgres table that is empty in Qdrant mode, re-embedded nothing, wrote nothing, and still reported `completed`. An operator polling `status` alone would have seen a green result and switched the active version to an **empty index**. The job now refuses to report success when it wrote nothing: a namespace with documents but zero reindexed chunks fails with `error_message` explaining that the store returned no chunks at the source version. `chunks_reindexed` makes the distinction visible rather than implied — treat a completed job whose `chunks_reindexed` is 0 (against a non-empty namespace) as a bug, not a no-op.
 
+### DELETE /api/v1/index-versions/{index_version}
+
+Reclaim the storage held by a superseded index version (REQ-064). Reindex leaves both versions in the store; this is the only thing that removes one.
+
+**Scopes**: `admin`
+
+```bash
+curl -s -X DELETE \
+  -H "Authorization: Bearer $VEKTRA_API_KEY" \
+  "http://localhost:8000/api/v1/index-versions/1?namespace=default" | python3 -m json.tool
+```
+
+| Parameter | In | Default | Description |
+|-----------|-----|---------|-------------|
+| `index_version` | path | (required) | The version to delete. Must be `>= 1`, and must **not** be the active one. |
+| `namespace` | query | `default` | Namespace to clean up. A namespace-bound key may only name its own; naming another returns `403`. |
+
+```json
+{
+  "namespace": "default",
+  "index_version": 1,
+  "chunks_removed": 12
+}
+```
+
+Errors (all in the REQ-010 envelope):
+
+- **`409` `ERR-INDEX-001`** if `index_version` is the version the system is currently serving. **This is the guard that matters.** The store refuses before deleting anything, so the request is a no-op, not a partial wipe. The failure mode it exists to prevent is not "an old version survives", it is "the live index is emptied": get the version number wrong by one and every query stops finding anything. `details` carries the `namespace` and `index_version` that were refused.
+- `400` `ERR-INDEX-002` if `index_version` is below 1.
+- `403` `ERR-AUTH-003` for a namespace-bound key naming another namespace. The delete is by filter, so a silently retargeted namespace would drop an entire version of a namespace the caller never named.
+
+The call is **idempotent**: deleting a version that is already gone returns `200` with `chunks_removed: 0`. That is the cheapest way to confirm a cleanup really happened, and it is safe to repeat.
+
+It is also the one **irreversible** step in the whole flow. Everything before it can be undone by not switching, or by switching back. Once the old version is deleted, rolling back means reindexing again from scratch.
+
 ### End-to-end operator flow
 
-`scripts/reindex.sh TARGET_VERSION [NAMESPACE]` automates steps 1 and 2. The steps below are what it does, plus the manual part it deliberately stops short of.
+`scripts/reindex.sh TARGET_VERSION [NAMESPACE]` automates steps 1 and 2; `scripts/reindex.sh --cleanup OLD_VERSION [NAMESPACE]` does step 5. They are two separate runs on purpose: between them sits the switch, which only a human can do, and until it happens the API will (correctly) refuse the cleanup.
 
 #### Step 1: trigger
 
@@ -1083,24 +1118,39 @@ To roll back, set the variable back to `1` and recreate again: version 1 is stil
 
 #### Step 5: clean up the old version
 
-**There is no API for this.** The old version's chunks stay in the store, invisible to reads but occupying space, until they are removed at the store level. Once the new version is verified in production, delete them directly:
+Until this step, the old version is still in the store: invisible to reads, but occupying exactly as much space as the live one. Every reindex you never clean up doubles the namespace again.
+
+Do it **after** step 4, once the new version has served real traffic and you are no longer going to roll back:
 
 ```bash
-# Qdrant
-curl -s -X POST "http://localhost:6333/collections/vektra/points/delete?wait=true" \
-  -H "Content-Type: application/json" \
-  -d '{"filter":{"must":[
-        {"key":"namespace_id","match":{"value":"default"}},
-        {"key":"index_version","match":{"value":1}}]}}'
+curl -s -X DELETE \
+  -H "Authorization: Bearer $VEKTRA_API_KEY" \
+  "http://localhost:8000/api/v1/index-versions/1?namespace=default"
+# {"namespace":"default","index_version":1,"chunks_removed":12}
 ```
 
-```sql
--- pgvector
-DELETE FROM document_chunks
-WHERE namespace_id = 'default' AND index_version = 1;
+Or: `scripts/reindex.sh --cleanup 1 default`.
+
+Ordering is not left to your memory. If you run this **before** the switch, version 1 is still the active one and the API refuses with `409` — the request deletes nothing:
+
+```json
+{
+  "error": {
+    "category": "PERMANENT",
+    "code": "ERR-INDEX-001",
+    "message": "Refusing to delete index version 1 of namespace 'default': it is the version currently being served. Switch VEKTRA_ACTIVE_INDEX_VERSION to the new version and restart before cleaning up the old one.",
+    "remediation": "Switch VEKTRA_ACTIVE_INDEX_VERSION to the new version and restart, then delete the old one. To check which version is live, see VEKTRA_ACTIVE_INDEX_VERSION in the running configuration.",
+    "request_id": "550e8400-e29b-41d4-a716-446655440000",
+    "details": {"namespace": "default", "index_version": 1}
+  }
+}
 ```
 
-Both are irreversible and neither is namespace-safe by default: **always filter on both `namespace_id` and `index_version`**, and count the matching rows/points before deleting. Deleting the version that is currently active empties the live index.
+That refusal is the whole reason this is an endpoint rather than a store-level delete. Reclaiming space is the small half of the job; the large half is that the obvious hand-written version of it — a Qdrant filter delete, or `DELETE FROM document_chunks WHERE index_version = 1` — has no idea which version is live, and is run by an operator at precisely the moment they are least sure. One wrong number and the live index is empty, with no error and nothing to roll back to.
+
+To confirm it is really gone, run it again: a second call returns `chunks_removed: 0`.
+
+This is the one irreversible step in the flow. Everything before it is undone by not switching, or by switching back.
 
 ## Analytics
 
@@ -1282,10 +1332,10 @@ Most errors follow a standard envelope (REQ-010):
 
 See [error codes reference](error-codes.md) for the complete list.
 
-Not every endpoint uses the envelope. Reindex, conversations and admin conversation turns return FastAPI's bare shape instead:
+Not every endpoint uses the envelope. `POST /api/v1/reindex`, `GET /api/v1/reindex/{job_id}/status`, conversations and admin conversation turns return FastAPI's bare shape instead:
 
 ```json
 {"detail": "Reindex job not found"}
 ```
 
-Clients that parse errors must handle both. The envelope is the intended contract; the bare form is a gap, not a second contract to rely on.
+Clients that parse errors must handle both. The envelope is the intended contract; the bare form is a gap (DEBT-033), not a second contract to rely on. `DELETE /api/v1/index-versions/{version}` uses the envelope throughout, including its `409`.

docs/reference/error-codes.md

@@ -92,6 +92,8 @@ These codes are used by specific components and are not part of the REQ-011 regi
 | ERR-LEARN-004 | vektra-learn | 409 | Duplicate enrollment (student already enrolled in course) |
 | ERR-LEARN-005 | vektra-learn | 404 | Conversation not found (GET conversations/turns) |
 | ERR-LEARN-006 | vektra-learn | 403 | Conversation belongs to a different course/namespace |
+| ERR-INDEX-001 | vektra-index | 409 | Refused: the index version named for deletion is the one currently being served. Nothing was deleted (REQ-064) |
+| ERR-INDEX-002 | vektra-index | 400 | Invalid index version (below 1) |
 
 ## Adding new error codes