Commit ba9dcb6
authored
feat(stovepipe): add storage extension with RequestStore + RequestURIStore (MySQL) (#277)
## Summary
### Why?
The ingest controller (PR #276) mints a `Request` but is log-only. For
the pipeline to do real work it needs to persist requests and look them
up two ways, both flowing from the workflow RFC: by **request ID**
(every downstream stage reloads the entity), and by **(queue, commit
URI)** to find whether a commit is already being validated — the RFC's
`(Queue, head URI)` dedup key.
### What?
A new `stovepipe/extension/storage` extension plus its first backend
(MySQL), mirroring the SubmitQueue storage conventions: a factory
`Storage` interface,
`ErrNotFound`/`ErrAlreadyExists`/`ErrVersionMismatch` sentinels,
metrics-wrapped MySQL ops, and optimistic-locking CAS with version
arithmetic owned by the caller.
Two stores, **one per table**:
- **`RequestStore`** (`request` table) — `Create` (ErrAlreadyExists on
dup ID); `Get` by ID (ErrNotFound); `Update`, a pure conditional write
taking the whole `Request` plus `oldVersion`/`newVersion`, persisting
the mutable fields (uri, state) only if the stored version matches (else
ErrVersionMismatch).
- **`RequestURIStore`** (`request_uri` table) — the reverse index from a
validated commit to its request, keyed by `(queue, uri)`: `Create`
(ErrAlreadyExists on a duplicate `(queue, uri)` — the dedup signal) and
`GetIDByURI` (ErrNotFound). It's a separate store because it's a
separate table; the two are written independently (no cross-table
transaction) so the contract stays satisfiable by key/value backends.
Ships the MySQL impl, schema (`request.sql`, `request_uri.sql`),
generated mocks, and a docker-compose integration test.
Follow-ups: wire `storage` into `ingest` (dedup via `GetIDByURI`, then
`Create`); an in-memory backend + shared contract suite.
## Test Plan
- ✅ `bazel test
//test/integration/stovepipe/extension/storage/mysql:mysql_test` — real
MySQL via docker-compose: create/get/update-CAS (success, stale-version
mismatch, missing-row mismatch), not-found, duplicate-ID, and the URI
mapping (round-trip, not-found, duplicate dedup, per-queue isolation).
- ✅ `bazel build //stovepipe/...`; `make check-gazelle`, `make
check-tidy`, `make lint` clean. Mocks regenerated and idempotent.
## Issues
## Stack
1. #276
1. @ #277
1. #278
1. #2791 parent c13d330 commit ba9dcb6
18 files changed
Lines changed: 946 additions & 0 deletions
File tree
- stovepipe/extension/storage
- mock
- mysql
- schema
- test/integration/stovepipe/extension/storage/mysql
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 70 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
0 commit comments