Merge pull request #1 from nullclaw/feat/orchestration

Store API, orchestration fields, and per-state concurrency

Author: DonPrus

AGENTS.md

@@ -18,7 +18,12 @@ Primary modules:
 - `src/store.zig` - SQLite access, transactions, migrations, ownership/free helpers
 - `src/domain.zig` - pipeline FSM parse/validation/transition logic
 - `src/ids.zig` - UUID/token/hash/time helpers
+- `src/config.zig` - config loading and resolution
+- `src/export_manifest.zig` - nullhub manifest export
+- `src/from_json.zig` - JSON config bootstrap
 - `src/migrations/001_init.sql` - schema
+- `src/migrations/003_store.sql` - KV store table
+- `src/migrations/004_store_fts.sql` - FTS5 search index
 
 Baseline commands:
 

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 nullclaw contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -19,7 +19,7 @@ stages, and attach artifacts.
 - `nullTickets` (this repository) is responsible for durable task state:
   - pipelines, stages, transitions
   - runs, leases, events, artifacts
-  - dependencies, quality gates, assignments
+  - dependencies, assignments
   - idempotent writes and optimistic transition checks
 - `nullTickets` is intentionally orchestration-light:
   - it does not decide global scheduling strategy
@@ -40,7 +40,7 @@ Practical architecture:
 You do not have to use all three components.
 
 - `nullclaw` + `nullTickets` is a valid setup for sequential execution.
-- `nullTickets` can be used with other agent runtimes as long as they implement the tracker contract (`claim -> events/gates -> transition/fail`).
+- `nullTickets` can be used with other agent runtimes as long as they implement the tracker contract (`claim -> events -> transition/fail`).
 - `nullboiler` is optional and is mainly needed for advanced multi-agent orchestration.
 
 ## Adoption Path
@@ -87,10 +87,15 @@ bash tests/test_e2e.sh
 
 - `src/main.zig` - process entrypoint, argument parsing, socket accept loop
 - `src/api.zig` - HTTP routing, request validation, response serialization
-- `src/store.zig` - SQLite queries, transactions, ownership/free helpers
+- `src/store.zig` - SQLite queries, transactions, migrations, ownership/free helpers
 - `src/domain.zig` - pipeline FSM parsing and validation
 - `src/ids.zig` - UUID/token/hash/time helpers
+- `src/config.zig` - config loading and resolution
+- `src/export_manifest.zig` - nullhub manifest export
+- `src/from_json.zig` - JSON config bootstrap
 - `src/migrations/001_init.sql` - database schema
+- `src/migrations/003_store.sql` - KV store table
+- `src/migrations/004_store_fts.sql` - FTS5 full-text search index
 - `tests/test_e2e.sh` - end-to-end API flow
 
 ## API Surface
@@ -116,15 +121,26 @@ bash tests/test_e2e.sh
 | `DELETE` | `/tasks/{id}/assignments/{agent_id}` | Unassign task |
 | `POST` | `/leases/claim` | Claim next task by role |
 | `POST` | `/leases/{id}/heartbeat` | Extend lease |
+| `GET` | `/tasks/{id}/run-state` | Get task run_id |
 | `POST` | `/runs/{id}/events` | Append run event |
 | `GET` | `/runs/{id}/events?limit=&cursor=` | List run events (cursor paginated) |
-| `POST` | `/runs/{id}/gates` | Add quality gate result |
-| `GET` | `/runs/{id}/gates` | List quality gate results |
 | `POST` | `/runs/{id}/transition` | Move task to next stage |
 | `POST` | `/runs/{id}/fail` | Mark run as failed |
 | `POST` | `/artifacts` | Attach artifact |
 | `GET` | `/artifacts?task_id=&run_id=&limit=&cursor=` | List artifacts (cursor paginated) |
 | `GET` | `/ops/queue` | Per-role queue stats for orchestrator |
+| `PUT` | `/store/{namespace}/{key}` | Put KV store entry |
+| `GET` | `/store/{namespace}/{key}` | Get KV store entry |
+| `DELETE` | `/store/{namespace}/{key}` | Delete KV store entry |
+| `GET` | `/store/{namespace}` | List entries in namespace |
+| `DELETE` | `/store/{namespace}` | Delete all entries in namespace |
+| `GET` | `/store/search?q=&namespace=&limit=&filter_path=&filter_value=` | Full-text search store |
+
+### Store API Notes
+
+- Store path segments are URL-decoded by the server. Clients should percent-encode reserved characters in `namespace` and `key` (for example spaces or `/`).
+- The namespace name `search` is reserved for `GET /store/search` and cannot be listed via `GET /store/{namespace}`.
+- `GET /store/search` also supports exact JSON filtering with `filter_path` and `filter_value` in addition to FTS search.
 
 ## Agent Loop
 
@@ -134,7 +150,6 @@ POST /leases/claim { agent_id, agent_role, lease_ttl_ms? }
 -> 204 (no work)
 
 POST /runs/{run_id}/events      (Bearer <lease_token>)
-POST /runs/{run_id}/gates       (Bearer <lease_token>)
 POST /runs/{run_id}/transition  (Bearer <lease_token>)
 POST /runs/{run_id}/fail        (Bearer <lease_token>)
 

deps/sqlite/build.zig

@@ -14,6 +14,7 @@ pub fn build(b: *std.Build) void {
     });
     lib.root_module.addCSourceFile(.{
         .file = b.path("sqlite3.c"),
+        .flags = &.{"-DSQLITE_ENABLE_FTS5"},
     });
     lib.installHeader(b.path("sqlite3.h"), "sqlite3.h");
     lib.installHeader(b.path("sqlite3ext.h"), "sqlite3ext.h");

docs/api.md

@@ -32,12 +32,6 @@ OTLP attribute mapping keys:
 - `GET /pipelines`
 - `GET /pipelines/{id}`
 
-Pipeline transitions support `required_gates`:
-
-```json
-{ "from": "coding", "to": "review", "trigger": "complete", "required_gates": ["tests_passed"] }
-```
-
 ## Tasks
 
 - `POST /tasks`
@@ -71,8 +65,6 @@ Pipeline transitions support `required_gates`:
 
 - `POST /runs/{id}/events` (Bearer)
 - `GET /runs/{id}/events?limit=&cursor=`
-- `POST /runs/{id}/gates` (Bearer)
-- `GET /runs/{id}/gates`
 - `POST /runs/{id}/transition` (Bearer)
 - `POST /runs/{id}/fail` (Bearer)
 
@@ -86,7 +78,6 @@ Pipeline transitions support `required_gates`:
 
 Transition returns `409` when:
 
-- required gates are not passed
 - `expected_stage` does not match
 - `expected_task_version` does not match
 
@@ -95,6 +86,21 @@ Transition returns `409` when:
 - `POST /artifacts`
 - `GET /artifacts?task_id=&run_id=&limit=&cursor=`
 
+## Store (KV)
+
+- `PUT /store/{namespace}/{key}` with `{ "value": ... }`
+- `GET /store/{namespace}/{key}`
+- `DELETE /store/{namespace}/{key}`
+- `GET /store/{namespace}` (list entries)
+- `DELETE /store/{namespace}` (delete all entries in namespace)
+- `GET /store/search?q=&namespace=&limit=&filter_path=&filter_value=` (FTS5 full-text search)
+
+Notes:
+
+- `namespace` and `key` path segments are URL-decoded server-side, so clients should percent-encode reserved characters such as spaces or `/`.
+- `search` is a reserved namespace name because `GET /store/search` is the full-text search endpoint.
+- `filter_path` and `filter_value` apply an exact JSON filter on top of FTS results.
+
 ## Ops
 
 - `GET /ops/queue?near_expiry_ms=&stuck_ms=`

docs/architecture.md

@@ -18,8 +18,8 @@ In scope:
 - Lease ownership and retries
 - Run events and artifacts
 - Task dependencies (DAG)
-- Quality gate evidence and enforcement
 - Optional task assignments
+- Key-value store with full-text search
 - Orchestrator-facing queue metrics (`/ops/queue`)
 
 Out of scope:
@@ -44,26 +44,24 @@ Out of scope:
 - `events`: append-only run timeline
 - `artifacts`: task/run outputs
 - `task_dependencies`: DAG edges (`task -> depends_on_task`)
-- `gate_results`: pass/fail evidence per run and gate
 - `task_assignments`: optional explicit owner binding for agents
+- `store`: namespaced key-value entries with FTS5 search
 - `idempotency_keys`: deduplication store for write retries
 
 ## Execution Semantics
 
 1. Agent claims work using role (`/leases/claim`).
 2. Service starts a new run and grants a lease token.
 3. Agent sends events and periodic heartbeats.
-4. Agent (or orchestrator) may submit gate results for run quality checks.
-5. Agent either transitions the run to the next stage or fails it.
-6. Transition can require gate pass state and optimistic checks (`expected_stage`, `expected_task_version`).
-7. Failures apply retry policy and optional dead-letter routing.
-8. Lease is released on transition/failure or expires automatically.
+4. Agent either transitions the run to the next stage or fails it.
+5. Transition can enforce optimistic checks (`expected_stage`, `expected_task_version`).
+6. Failures apply retry policy and optional dead-letter routing.
+7. Lease is released on transition/failure or expires automatically.
 
 ## Safety and Correctness
 
 - State-changing paths use SQL transactions (`BEGIN IMMEDIATE`) to avoid double-claim races.
 - Lease tokens are stored as SHA-256 hashes, not plaintext.
 - Pipeline transitions are validated against declared FSM definitions.
-- Required quality gates are enforced server-side on transition.
 - Claim excludes blocked dependencies, non-eligible retries, and foreign assignments.
 - API string fields are JSON-escaped at serialization time.

docs/workflows.md

@@ -19,10 +19,6 @@ State metadata supports:
 - `description`: optional description
 - `terminal`: whether this stage is terminal
 
-Transition metadata supports:
-
-- `required_gates`: quality gates that must be passed before transition
-
 ## Transition Rules
 
 - Every transition must reference existing states.
@@ -53,13 +49,6 @@ Transition metadata supports:
 
 Dependencies are resolved only when the upstream task reaches a terminal pipeline stage.
 
-## Quality Gates
-
-- Add gate result: `POST /runs/{id}/gates`
-- Inspect gate history: `GET /runs/{id}/gates`
-
-`/runs/{id}/transition` returns `409` if required gates are not passed.
-
 ## Assignments (Optional)
 
 - Assign: `POST /tasks/{id}/assignments`