diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/CHUNK_MAP.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/CHUNK_MAP.md index ea79bc58..9ac2ce5c 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/CHUNK_MAP.md @@ -3,10 +3,15 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| | `WS-CI-001-01` | Parallel Full-Suite Coverage | L1 | Completed and merged in PR #163 | -| `WS-CI-001-01R1` | Timeout Cleanup Repair | L1 | Implemented and internally reviewed; awaiting hosted repair PR proof/human review | -| `WS-CI-001-02` | Safe Routing, Cache, and Timing Refinement | L1 | Future; not started and requires separate approval | +| `WS-CI-001-01R1` | Timeout Cleanup Repair | L1 | Completed and merged in PR #164 | +| `WS-CI-001-02` | Safe Routing, Cache, and Timing Refinement | L1 | Active signed planning; measured decision defers routing/cache/timing and plans 02A/02B | +| `WS-CI-001-02A` | Safe Migrate-Once Database Reset | L1/P0 | Proposed successor; requires separate signed implementation start | +| `WS-CI-001-02B` | Exact-Custody Semantic Test Lanes | L1/P0 | Proposed after 02A evidence; not yet eligible to start | +| `WS-CI-001-03` | Safe Routing, Cache, and Timing Reassessment | L1 | Future planning after 02B evidence; not a declared successor | Each chunk maps to one PR. Chunk 01 preserves the full suite and every coverage -gate. Chunk 01R1 is the bounded post-merge response to CodeRabbit's timeout -cleanup finding. Chunk 02 may be planned only from measured 01 evidence and -cannot be activated without separate human approval. +gate. Chunk 01R1 repaired timeout cleanup. Chunk 02 converts measured evidence +and PR #180 into two prospective implementation contracts. Chunk 02A first +proves the destructive reset and fixture migration under the existing CI +topology. Chunk 02B may then change runner/workflow topology using 02A evidence. +Only 02A is the declared successor of this planning PR. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DECISIONS.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DECISIONS.md index 7fe13ee5..96e1ad8f 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DECISIONS.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DECISIONS.md @@ -37,3 +37,24 @@ node IDs are therefore not executable cross-process authority. Shards execute validated whole modules, record final collection and completion in the same pytest process, require those exact sets to match, and bind their stable test-base cardinalities to the authenticated preflight manifest. + +## 2026-07-22 - Split reset safety from semantic-lane topology + +PR #180 is discovery and an immutable contributor source, not authorized +implementation. Review showed its destructive migrate-once reset and its CI +lane/workflow rewrite cross separable L1/P0 boundaries. Chunk 02A must first +prove runner-owned reset containment, canonical trigger restoration, migration +state, and fixture equivalence under the existing CI topology. Only later chunk +02B may replace that topology, with independent exact-node validation and +isolated PostgreSQL and MinIO custody. This limits rollback and review scope +while preserving Konan's authorship. + +## 2026-07-22 - Defer routing, dependency cache, and durable timing weights + +Measured evidence shows repeated migrations and service-heavy execution are the +current bottleneck. Path routing, dependency caching, sampling, and durable +timing weights add invalidation and silent-suppression risk without addressing +that cause. Chunk 02 therefore makes a reviewed no-implementation decision for +those original options. They may be reassessed only in future planning chunk +`WS-CI-001-03`, after 02B exact-head evidence exists; 03 is not a successor of +this PR and has no start authority. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DISCOVERY.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DISCOVERY.md index 6df4f21e..721ad532 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DISCOVERY.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DISCOVERY.md @@ -33,6 +33,19 @@ PR #161 Backend run `29752233451`, job `88385435082`: Recent successful Backend workflows have ranged from roughly 24 to 52 minutes. The full-suite process is therefore the dominant wall-clock cost. +Contributor PR #180 subsequently supplied local migrate-once measurements: + +- project-client tests improved from 706–729 seconds to 179.53 seconds; +- task-client tests improved from 535–798 seconds to 145.89 seconds; +- the complete unsharded suite reported 1,815 passed in 919.48 seconds; +- global coverage remained 87.58 percent and protected modules remained + between 90.90 and 100 percent. + +Those numbers are discovery inputs, not exact-head hosted acceptance evidence. +The PR was opened before a prospective signed implementation start, so its code +cannot be treated as authorized implementation or merged as-is. Its author, +Konan, must retain implementation attribution if the work is later adopted. + ## Repository inventory - `backend/pyproject.toml` declares pytest, pytest-asyncio, pytest-cov, coverage, @@ -80,6 +93,14 @@ The full-suite process is therefore the dominant wall-clock cost. - No live per-shard timing evidence exists. - No current workflow test proves a failed/cancelled shard makes the final required job fail. +- PR #180's semantic-lane inventory proves file assignment but does not yet + prove that every collected node completed exactly once. +- Its reset regression test covers five of the seven canonical trigger-guarded + tables, leaving incomplete restoration proof. +- Several exact Boolean identity assertions were changed to truthiness checks, + weakening test contracts without a product reason. +- The contributor diff and original retrospective scope do not match exactly; + the successor contract must authorize the complete file set prospectively. ## Risks discovered @@ -103,6 +124,9 @@ The full-suite process is therefore the dominant wall-clock cost. - Whether all shards require MinIO or only the artifact shard. - Practical concurrency available to the repository/account. - Aggregate Actions-minute increase for four shards. +- Exact hosted wall time of migrate-once semantic lanes at the reviewed head. +- Whether four dependency lanes remain balanced after exact node custody and + all required real-service contracts are restored. ## Conventions to preserve diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/PLAN.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/PLAN.md index b5d247be..36dde35d 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/PLAN.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/PLAN.md @@ -148,3 +148,36 @@ thresholds, disable shards, or silently fall back inside the workflow. Implementation does not begin until the reviewed 01 contract receives explicit human approval. + +## 02 planning amendment — migrate-once semantic lanes + +PR #180 supplied useful discovery and local measurements but is not authorized +implementation evidence. It showed repeated per-test migrations dominate wall +time and proposed four dependency-based processes in one job, each PostgreSQL +lane owning a private migrated database and role. The immediate implementation +successor is the reset-only `WS-CI-001-02A`; lane orchestration is deferred to +`WS-CI-001-02B`. Routing, cache suppression, sampling, and test skipping are not +part of either chunk. + +The work is split at the destructive-reset/workflow boundary. 02A may adopt the +database reset and fixture changes from Konan's immutable commits +`e22e9fba681f6f076d67433c90a268218be7662b` and +`c73b28931b3b2a68a9ee2389181220132efaa41e`, with exact file scope and preserved +authorship, only after this planning chunk merges and 02A receives a signed +implementation start. It must prove database ownership, reset containment, +canonical trigger restoration, migration-state preservation, and unchanged +test behavior under the existing CI topology. + +02B is a later, separately started successor that may adopt the semantic-lane +runner and workflow topology only after 02A merges. It must prove exact +collected-node-to-completed-node custody, isolated PostgreSQL and MinIO state, +unchanged coverage floors, failure custody, and exact hosted timing. Four +semantic lanes are a dependency partition, not arbitrary shards and not a +substitute for exact execution proof. + +The target remains under eight minutes for the complete Backend required job, +but correctness, real PostgreSQL/MinIO contracts, 78% global coverage, and all +protected 90% floors are hard gates. No timing claim is accepted from a local or +different commit. Exceeding eight minutes does not permit gate weakening: 02B +must stop for explicit human risk acceptance or further optimization, with wall +time, slowest-lane time, and aggregate runner minutes recorded. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/RISKS.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/RISKS.md index 46196ab5..d9927924 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/RISKS.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/RISKS.md @@ -16,3 +16,7 @@ | R12 | Plan scope expands into path-based skipping | High | Defer routing to separately approved 02 contract | | R13 | Mutable PostgreSQL tag changes CI behavior | High | Replace `postgres:16` with a reviewed digest pin in 01 | | R14 | Parameter display values change across pytest processes | Critical | Execute whole modules; compare exact collection/completion within one process and bind only stable test-base cardinalities across processes | +| R15 | Semantic lanes prove files but silently lose test nodes | Critical | Record canonical collection and completed-node custody per lane; reject missing, duplicate, foreign, deselected, zero-collection, and unexpected skip evidence | +| R16 | Fast reset leaves a guarded trigger disabled | Critical | Use one canonical seven-table guard inventory and prove every trigger is enabled after success and rollback | +| R17 | Mechanical test edits weaken behavioral contracts | Critical | Restore strict Boolean identity assertions and require test-delta review against current main | +| R18 | Contributor implementation is retroactively authorized | Critical | Preserve PR #180 as discovery evidence; merge planning 02 first, then signed-start prospective 02A before adoption | diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md index 08d75b0c..c30e141a 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md @@ -1,9 +1,12 @@ # STATUS: WS-CI-001 - Backend CI Acceleration -- Phase: post-merge timeout cleanup repair -- Active implementation chunk: `WS-CI-001-01R1` -- Proposed first chunk: `WS-CI-001-01` -- Human direction: prioritize full-suite CI acceleration before - `WS-ENG-001-04B` -- Current gate: exact rebased-head review, repair PR CI, and human review -- `WS-ENG-001-04B`: remains inactive +- Phase: signed planning amendment +- Active planning chunk: `WS-CI-001-02` +- Active implementation chunk: none +- Proposed implementation successor: `WS-CI-001-02A` +- Later proposed chunk: `WS-CI-001-02B` (cannot start before 02A evidence) +- Human direction: preserve Konan's authorship and measured work from PR #180, + but adopt it only under prospective zero-trust scope and evidence +- Current gate: internal review passed; planning-only PR and user-owned PR decision +- Deferred option: routing/cache/timing reassessment is future `WS-CI-001-03`, + with no start or successor declaration diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02-safe-routing-cache-timing.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02-safe-routing-cache-timing.md index 90354075..f1362db5 100644 --- a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02-safe-routing-cache-timing.md +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02-safe-routing-cache-timing.md @@ -6,8 +6,9 @@ ## Goal -Use measured 01 evidence to consider fail-closed path routing, dependency cache, -and durable shard weights without weakening full-suite requirements. +Use measured evidence to decide whether fail-closed path routing, dependency +cache, or durable timing weights should be implemented now, and prospectively +contract the safer optimization that addresses the observed bottleneck. ## Why this chunk exists @@ -43,13 +44,34 @@ approval; any coverage/test weakening; backend product changes; 04B activation. ## Acceptance criteria -- [ ] A separate reviewed amendment defines exact files and fail-closed routing. -- [ ] Cache and timing provenance cannot cross dependency or commit boundaries. -- [ ] Full-suite-required change classes default closed on ambiguity. +- [x] Routing is not implemented: current evidence does not justify suppression + risk, and every change class continues to run the full required suite. +- [x] Dependency cache and durable timing weights are not implemented: their + provenance/invalidation boundaries remain unresolved and cannot cross + dependency or commit boundaries. +- [x] Future reassessment is identified as planning chunk `WS-CI-001-03`; it + requires evidence after 02B, its own signed planning start, and fail-closed + defaults. It is not this PR's successor. +- [x] Prospective `WS-CI-001-02A` and `WS-CI-001-02B` contracts split the + migrate-once reset from semantic-lane orchestration without treating PR + #180 as authorized implementation evidence. +- [x] The contracts require destructive-reset ownership, exact node custody, + strict test assertions, all guarded-trigger restoration, unchanged + coverage floors, isolated services, and exact hosted proof. ## Verification commands -To be defined from 01 hosted evidence. +```bash +python3 scripts/update_post_merge_memory.py validate-merge-intent --repository-root . --base-ref origin/main +python3 scripts/test_agent_gates.py +python3 scripts/check_loop_memory_state.py +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_stale_authorization_docs.py +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_internal_review_evidence.py +git diff --check origin/main...HEAD +``` ## Required reviewers @@ -70,5 +92,5 @@ telemetry complexity is justified by measured results. ## Stop conditions -Stop if 01 is not merged and stable, if scope is not explicit, or if the user has -not separately approved this chunk. +Stop if the plan attempts routing/cache/timing implementation, if successor +scope is not explicit, or if implementation begins before its signed start. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02A-safe-migrate-once-database-reset.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02A-safe-migrate-once-database-reset.md new file mode 100644 index 00000000..05b1a50d --- /dev/null +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02A-safe-migrate-once-database-reset.md @@ -0,0 +1,138 @@ +# Chunk Contract: WS-CI-001-02A — Safe Migrate-Once Database Reset + +## Parent initiative + +`WS-CI-001` — Backend CI Acceleration + +## Goal + +Adopt and repair Konan's migrate-once reset and fixture work under the existing +CI topology, proving destructive reset containment before lane orchestration. + +## Immutable contributor source + +- `e22e9fba681f6f076d67433c90a268218be7662b` +- `c73b28931b3b2a68a9ee2389181220132efaa41e` + +Only reviewed changes from these commits and the exact allowlist below are +eligible. Later PR #180 commits are new discovery. The resulting implementation +commit and PR evidence must preserve Konan's authorship. + +## Risk class + +L1 / P0 CI integrity and destructive test-data handling + +## SLA + +P1 + +## Start phase + +`implementation` + +## Allowed files + +- `backend/pyproject.toml` +- `backend/tests/conftest.py` +- `backend/tests/test_actors.py` +- `backend/tests/test_alembic.py` +- `backend/tests/test_api_rate_controls.py` +- `backend/tests/test_artifact_admission.py` +- `backend/tests/test_artifact_recovery.py` +- `backend/tests/test_audit.py` +- `backend/tests/test_auth.py` +- `backend/tests/test_authorization.py` +- `backend/tests/test_checkers.py` +- `backend/tests/test_database_reset.py` +- `backend/tests/test_outbox.py` +- `backend/tests/test_projects.py` +- `backend/tests/test_tasks.py` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02A-internal-review-evidence.md` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02A-pr-trust-bundle.md` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02A-external-review-response.md` +- `.agent-loop/merge-intents/WS-CI-001-02A.json` + +## Not allowed changes + +- workflow topology, lane/shard runners, coverage commands, product code, + production migrations, API behavior, or service architecture; +- removed, skipped, deselected, or weakened tests or behavioral assertions; +- destructive reset against an admin, default, non-test, unexpected, or + runner-unowned database/role; +- dynamic acceptance of unknown tables, duplicated guard inventories, or + triggers left disabled on any exit path. + +## Acceptance criteria + +- [ ] The reset authenticates the expected runner-generated database name and + least-privileged owner role before any trigger, truncate, or schema action + and rejects admin/default/non-test/wrong-role targets. +- [ ] One canonical reset inventory defines resettable tables, protected + migration state, and all seven trigger-guarded tables; implementation and + tests import the same source and fail closed on unknown schema objects. +- [ ] Success, repeated reset, injected failure, transaction rollback, signal, + wrong database, wrong role, admin URL, and unexpected-table tests prove no + out-of-lane data is reset and no trigger remains disabled. +- [ ] Alembic state and immutable actor migration state remain byte-for-byte + unchanged, and schema-owning migration tests rebuild safely. +- [ ] Every fixture-migrated module retains the same collected tests and no + assertion is weakened; exact Boolean identity regressions are restored. +- [ ] Existing real PostgreSQL, MinIO, API, migration, lock, trigger, + constraint, transaction, concurrency, coverage, and failure gates pass + under the existing CI topology. +- [ ] Global 78% and every protected 90% coverage floor remain unchanged. + +## Verification commands + +```bash +cd backend +ci_verify_dir="$(mktemp -d)" +trap 'rm -rf -- "$ci_verify_dir"' EXIT +test -n "${WORKSTREAM_TEST_ADMIN_DATABASE_URL:-}" +ruff check app tests scripts +python -m pytest -q tests/test_isolated_database_runner.py +python scripts/run_isolated_tests.py --metadata-json "$ci_verify_dir/reset.json" --timeout-seconds 1200 -- python -m pytest -q tests/test_database_reset.py +python scripts/run_isolated_tests.py --metadata-json "$ci_verify_dir/collect.json" --timeout-seconds 12600 -- python -m pytest --collect-only -q +python scripts/run_isolated_tests.py --metadata-json "$ci_verify_dir/suite.json" --timeout-seconds 12600 -- coverage run -m pytest +coverage report --precision=2 --fail-under=78 +coverage report --include='app/adapters/artifacts/*,app/core/cancellation.py,app/core/file_locks.py,app/interfaces/artifact_operations.py,app/interfaces/artifacts.py,app/modules/artifacts/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/external_services.py' --precision=2 --fail-under=90 +coverage report --include='app/core/config.py' --precision=2 --fail-under=90 +coverage report --include='app/workers/*' --precision=2 --fail-under=90 +coverage report --include='app/main.py' --precision=2 --fail-under=90 +coverage report --include='app/adapters/artifacts/s3_compatible.py' --precision=2 --fail-under=90 +coverage report --include='app/core/s3_validation.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/actors/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/authorization/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/api_controls/*,app/api/deps/api_controls.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/tasks/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/auth.py,app/core/auth.py,app/adapters/auth/dev.py,app/adapters/auth/flow.py' --precision=2 --fail-under=90 +cd .. +python3 scripts/test_agent_gates.py +python3 scripts/check_internal_review_evidence.py +git diff --check +``` + +## Required reviewers + +- [ ] senior engineering +- [ ] QA/test +- [ ] security/auth +- [ ] product/ops +- [ ] architecture +- [ ] CI integrity +- [ ] reuse/dedup +- [ ] test delta + +## Human review focus + +Target ownership proof, protected/resettable inventory, rollback and trigger +restoration, test-delta equivalence, and contributor attribution. + +## Stop conditions + +Stop if signed implementation custody is absent, target ownership cannot be +proved, unknown schema state must be accepted, tests/coverage must be weakened, +or the existing CI topology cannot prove the reset safely. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02B-exact-custody-semantic-test-lanes.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02B-exact-custody-semantic-test-lanes.md new file mode 100644 index 00000000..784bae53 --- /dev/null +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-02B-exact-custody-semantic-test-lanes.md @@ -0,0 +1,133 @@ +# Chunk Contract: WS-CI-001-02B — Exact-Custody Semantic Test Lanes + +## Parent initiative + +`WS-CI-001` — Backend CI Acceleration + +## Goal + +Replace arbitrary shards with dependency-based lanes after 02A proves the +migrate-once reset, while preserving exact execution and coverage custody. + +## Preconditions + +02A is merged with hosted reset evidence. This contract then requires its own +signed implementation start. Eligible contributor source is limited to commits +`e22e9fba681f6f076d67433c90a268218be7662b` and +`c73b28931b3b2a68a9ee2389181220132efaa41e`. Later commits are ineligible +unless a subsequent reviewed contract names their immutable SHAs and exact file +scope. + +## Risk class + +L1 / P0 CI integrity + +## Start phase + +`implementation` + +## Allowed files + +- `.github/workflows/backend.yml` +- `backend/scripts/ci_test_shards.py` (deletion only) +- `backend/scripts/run_isolated_tests.py` +- `backend/scripts/run_test_lanes.py` +- `backend/scripts/validate_test_lane_evidence.py` +- `backend/tests/test_ci_test_shards.py` (deletion only) +- `backend/tests/test_ci_test_lanes.py` +- `backend/tests/test_isolated_database_runner.py` +- `backend/tests/test_test_lane_evidence.py` +- `docs/operations_backend_testing.md` +- `scripts/test_agent_gates.py` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/STATUS.md` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02B-internal-review-evidence.md` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02B-pr-trust-bundle.md` +- `.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02B-external-review-response.md` +- `.agent-loop/merge-intents/WS-CI-001-02B.json` + +## Not allowed changes + +Product/test behavior, production code or migrations, fixture/reset semantics, +test skipping or sampling, lower/non-blocking coverage, shared mutable database +or storage namespaces, unpinned dependencies/actions, or alternate ownership +and provisioning paths outside `run_isolated_tests.py`. + +## Acceptance criteria + +- [ ] Canonical discovery recursively collects repository `backend/tests/**/test_*.py` + nodes at the exact head; an immutable manifest binds head SHA, module, + node ID, lane, collection exit, completion exit, skip/deselect state, and + artifact digest. +- [ ] An independent validator rejects missing, duplicate, foreign, deselected, + unexpectedly skipped, zero-collected, interrupted, and partially completed + nodes and proves every canonical node completed exactly once. +- [ ] Four dependency lanes run concurrently in one job with distinct + runner-created PostgreSQL databases/roles and distinct MinIO bucket/prefix + namespaces; concurrency and bounded-cleanup tests reject collisions. +- [ ] Per-lane coverage artifacts are exact-head/digest bound and combined once; + global 78% and all protected 90% commands remain unchanged and blocking. +- [ ] Real PostgreSQL/MinIO/API contracts, cleanup, timeout, signals, redaction, + permissions, and upstream failure/cancellation custody remain mandatory. +- [ ] Exact hosted head records total Backend wall time, slowest lane, aggregate + runner minutes, collected/completed counts, and coverage. More than eight + minutes blocks completion unless the user explicitly accepts the measured + risk; no gate may be weakened to meet the target. +- [ ] Konan remains the implementation author/contributor in Git history and PR + evidence. + +## Verification commands + +```bash +cd backend +ci_verify_dir="$(mktemp -d)" +trap 'rm -rf -- "$ci_verify_dir"' EXIT +test -n "${WORKSTREAM_TEST_ADMIN_DATABASE_URL:-}" +ruff check app tests scripts +python -m pytest -q tests/test_ci_test_lanes.py tests/test_isolated_database_runner.py +python scripts/run_test_lanes.py --collect-only --metadata-dir "$ci_verify_dir/collect" --summary-json "$ci_verify_dir/collect.json" +python scripts/validate_test_lane_evidence.py --metadata-dir "$ci_verify_dir/collect" --summary-json "$ci_verify_dir/collect.json" +python scripts/run_test_lanes.py --metadata-dir "$ci_verify_dir/run" --summary-json "$ci_verify_dir/run.json" --timeout-seconds 1200 +python scripts/validate_test_lane_evidence.py --metadata-dir "$ci_verify_dir/run" --summary-json "$ci_verify_dir/run.json" +coverage combine +coverage report --precision=2 --fail-under=78 +coverage report --include='app/adapters/artifacts/*,app/core/cancellation.py,app/core/file_locks.py,app/interfaces/artifact_operations.py,app/interfaces/artifacts.py,app/modules/artifacts/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/external_services.py' --precision=2 --fail-under=90 +coverage report --include='app/core/config.py' --precision=2 --fail-under=90 +coverage report --include='app/workers/*' --precision=2 --fail-under=90 +coverage report --include='app/main.py' --precision=2 --fail-under=90 +coverage report --include='app/adapters/artifacts/s3_compatible.py' --precision=2 --fail-under=90 +coverage report --include='app/core/s3_validation.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/actors/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/authorization/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/api_controls/*,app/api/deps/api_controls.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/tasks/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/auth.py,app/core/auth.py,app/adapters/auth/dev.py,app/adapters/auth/flow.py' --precision=2 --fail-under=90 +cd .. +python3 scripts/test_agent_gates.py +python3 scripts/check_internal_review_evidence.py +git diff --check +``` + +## Required reviewers + +- [ ] senior engineering +- [ ] QA/test +- [ ] security/auth +- [ ] product/ops +- [ ] architecture +- [ ] CI integrity +- [ ] docs +- [ ] reuse/dedup +- [ ] test delta + +## Human review focus + +Exact node custody, isolated services, coverage combination, fail-closed fan-in, +contributor attribution, and exact hosted resource/time evidence. + +## Stop conditions + +Stop if 02A evidence is incomplete, independent validation scope remains +unresolved, exact-node or coverage custody fails, services share mutable state, +or the hosted target requires a weakened gate. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-03-safe-routing-cache-timing-reassessment.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-03-safe-routing-cache-timing-reassessment.md new file mode 100644 index 00000000..2291e158 --- /dev/null +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/chunks/WS-CI-001-03-safe-routing-cache-timing-reassessment.md @@ -0,0 +1,23 @@ +# Chunk Contract: WS-CI-001-03 — Safe Routing, Cache, and Timing Reassessment + +## Phase + +Future planning only. This chunk is not started, is not a declared successor, +and authorizes no implementation. + +## Preconditions + +02A and 02B are merged, and exact hosted evidence identifies a remaining +bottleneck that routing, cache, or durable timing data could address. + +## Planning scope + +Reassess fail-closed path routing, dependency-cache provenance/invalidation, +and exact-head timing weights. Produce a separately reviewed implementation +contract if any option is justified. + +## Hard boundaries + +No sampling, skipped tests, ambiguous full-suite routing, cross-commit cache or +timing authority, coverage weakening, implementation, or successor declaration +without its own signed planning event and review. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-external-review-response.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-external-review-response.md new file mode 100644 index 00000000..433b9ab6 --- /dev/null +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-external-review-response.md @@ -0,0 +1,49 @@ +# External Review Response + +## Chunk + +`WS-CI-001-02` + +## Source + +CodeRabbit review of PR #185 at `8391a7ccddd493de310fa3685c6398324d80785e`. + +## Comments addressed + +- Replaced broad successor governance-file categories with exact status, + internal-review, trust-bundle, external-response, and merge-intent paths for + 02A and 02B. +- Stated that later contributor commits are ineligible unless a subsequent + reviewed contract names immutable SHAs and exact file scope. +- Reworded the remaining-risk statement to require a signed 02A start. + +## Comments deferred + +None. + +## Comments rejected + +- The suggested trust-bundle and merge-intent rename would make them diverge + from the canonical signed `WS-CI-001-02` contract title. The existing title is + intentionally synchronized with signed state; the goal and PR title describe + the planning amendment and 02A reset-safety successor. + +## Human decisions needed + +None for these findings. + +## Commands rerun + +```bash +python3 scripts/update_post_merge_memory.py validate-merge-intent --repository-root . --base-ref origin/main +python3 scripts/test_agent_gates.py +python3 scripts/check_loop_memory_state.py +python3 scripts/check_markdown_links.py +python3 scripts/check_internal_review_evidence.py +git diff --check origin/main...HEAD +``` + +## Remaining risks + +The contract edits passed internal exact-head re-review. Fresh hosted checks are +still required on the pushed repair commit before merge readiness is reported. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-internal-review-evidence.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-internal-review-evidence.md new file mode 100644 index 00000000..d8012480 --- /dev/null +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-internal-review-evidence.md @@ -0,0 +1,87 @@ +# Internal Review Evidence + +## Chunk + +`WS-CI-001-02` + +This planning review also covers the prospective 02A, 02B, and future 03 +contracts. It does not authorize any implementation chunk to start. + +## Required Statements + +open sub-agent sessions: none + +valid findings addressed: yes + +## Reviewed Revision + +Reviewed code SHA: 060b0f37e2c6880ecd3c1d88d28d089d47133076 + +Reviewed at: 2026-07-22T11:57:27Z + +Reviewer run IDs: eng006_senior_arch_docs, eng006_qa_ci_tests, eng006_security_ops_reuse + +After the reviewed SHA, only review evidence files changed. + +## Reviewer Results + +| Reviewer | Result | Blocking findings | Notes | +|---|---:|---|---| +| senior engineering | PASS AFTER FIXES | None | Split reset safety from workflow topology and bounded future routing work. | +| QA/test | PASS AFTER FIXES | None | Exact reset, node custody, coverage, and rerunnable proof commands reviewed. | +| security/auth | PASS AFTER FIXES | None | Destructive target ownership and fail-closed refusal requirements reviewed. | +| product/ops | PASS | None | No product behavior changes; contributor attribution is explicit. | +| architecture | PASS AFTER FIXES | None | 02A and 02B cross separate, ordered boundaries. | +| CI integrity | PASS AFTER FIXES | None | Full suite and all 78/90 coverage gates remain blocking. | +| docs | PASS AFTER FIXES | None | Active 02 scope, successor, and future reassessment now agree. | +| reuse/dedup | PASS AFTER FIXES | None | Canonical runner and one reset inventory are required. | +| test delta | PASS AFTER FIXES | None | Removed, skipped, deselected, or weakened tests are prohibited. | + +## Valid Findings Addressed + +- Replaced retrospective PR scope with exact file allowlists and immutable + contributor commits. +- Split the destructive reset/fixture migration (02A) from semantic lane and + workflow adoption (02B). +- Added reset target ownership, wrong-target refusal, canonical seven-table + inventory, rollback, failure, signal, and protected-state requirements. +- Added exact recursive node custody, an independent evidence validator, + PostgreSQL/MinIO isolation, complete protected coverage commands, and hosted + timing/resource disposition. +- Converted planning verification prose to executable commands with isolated, + rerunnable temporary outputs. +- Explicitly deferred routing/cache/timing implementation to future planning 03 + and aligned the merge-intent title with the signed 02 contract. +- Removed the stale cross-initiative status claim. +- Addressed CodeRabbit's exact-governance-scope, immutable-later-commit, and + custody-wording findings; retained the canonical signed chunk title after + exact-head reviewers confirmed that renaming would create scope drift. + +## Commands Run + +```bash +python3 scripts/update_post_merge_memory.py validate-merge-intent --repository-root . --base-ref origin/main +python3 scripts/test_agent_gates.py +python3 scripts/check_loop_memory_state.py +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_stale_authorization_docs.py +python3 scripts/check_stale_artifact_contracts.py +git diff --check origin/main...HEAD +``` + +## Results + +- Merge intent passed for `WS-CI-001-02`. +- 95 Agent Gate tests passed. +- Loop memory state passed. +- Markdown links passed. +- All stale-contract scans passed. +- Diff integrity passed. + +## Remaining Risks + +- PR #180 remains open and mutable; only its two pinned contributor commits are + eligible discovery sources, and adoption still requires a signed 02A start. +- Hosted implementation timing and exact test equivalence remain future 02A/02B + evidence, not claims made by this planning PR. diff --git a/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-pr-trust-bundle.md b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-pr-trust-bundle.md new file mode 100644 index 00000000..d1556ed9 --- /dev/null +++ b/.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/reviews/WS-CI-001-02-pr-trust-bundle.md @@ -0,0 +1,137 @@ +# PR Trust Bundle + +## Chunk + +`WS-CI-001-02` — Safe Routing, Cache, and Timing Refinement + +Merge intent: `.agent-loop/merge-intents/WS-CI-001-02.json` + +## Goal + +Turn measured CI evidence and contributor PR #180 into prospective, bounded +implementation contracts without retroactively authorizing its code. + +## Human-approved intent + +The user directed the orchestrator to preserve strict zero-trust loop +engineering, review PR #180, and make the contributor work adoptable through +the normal plan, chunk, evidence, and PR process. + +## What changed + +- Recorded PR #180 measurements and review gaps as discovery. +- Decided not to implement routing, caching, sampling, or durable timing now. +- Defined reset-safety chunk 02A as the only declared successor. +- Defined later exact-custody semantic-lane chunk 02B after 02A evidence. +- Reserved routing/cache/timing reassessment for future planning chunk 03. + +## Why it changed + +PR #180 demonstrates a promising migrate-once direction, but its original diff +crosses destructive database reset and workflow-topology boundaries and lacked +prospective signed scope and complete exact-execution evidence. + +## Design chosen + +First prove runner-owned destructive reset containment under the existing CI +topology. Then, under a separate signed chunk, change to dependency-based lanes +with independent node, coverage, PostgreSQL, and MinIO custody. + +## Alternatives rejected + +- Merging PR #180 as-is was rejected because chat and retrospective artifacts + cannot authorize implementation. +- One large reset-plus-workflow chunk was rejected as unreviewable L1/P0 scope. +- More arbitrary infrastructure shards were rejected because they add cost and + do not address repeated migrations. +- Routing or sampling was rejected because it could suppress required proof. + +## Scope control + +Only WS-CI-001 planning, decision, risk, status, chunk-contract, review, and one +merge-intent file change. No workflow, backend, test, dependency, coverage, or +product runtime file changes. + +## Product behavior + +- [x] No Workstream product behavior changed. + +## Acceptance criteria proof + +- [x] Original routing/cache/timing options received an evidence-backed decision. +- [x] 02A and 02B have exact prospective file scope and immutable source commits. +- [x] Reset ownership, node custody, service isolation, coverage, and timing are explicit. +- [x] Only same-initiative 02A is declared and still requires a signed start. + +## Tests/checks run + +```bash +python3 scripts/update_post_merge_memory.py validate-merge-intent --repository-root . --base-ref origin/main +python3 scripts/test_agent_gates.py +python3 scripts/check_loop_memory_state.py +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_stale_authorization_docs.py +python3 scripts/check_stale_artifact_contracts.py +git diff --check origin/main...HEAD +``` + +All passed; 95 Agent Gate tests passed. + +## Test delta + +No executable tests changed. Future contracts prohibit removed, skipped, +deselected, or weakened tests and require exact collection/completion proof. + +## CI integrity + +- [x] No workflow or package script changed +- [x] Full suite remains required +- [x] Global 78% and all protected 90% floors remain blocking +- [x] No cache, routing, sampling, or timing authority introduced + +## Reviewer results + +Reviewed code SHA: 060b0f37e2c6880ecd3c1d88d28d089d47133076 + +Reviewed at: 2026-07-22T11:57:27Z + +Reviewer run IDs: eng006_senior_arch_docs, eng006_qa_ci_tests, eng006_security_ops_reuse + +All nine required tracks passed after the documented repairs, with no remaining +condition or blocker. + +## External review + +PR #180 supplied discovery and contributor measurements. Its review findings +were incorporated into prospective contracts; it remains unauthorized for +merge under this planning chunk. CodeRabbit's three valid scope/wording findings +were repaired. Its title-change suggestion was rejected because the trust bundle +and merge intent intentionally match the canonical signed chunk title; the goal +and PR title describe the planning amendment and reset-safety successor. + +## Remaining risks + +02A must prove destructive reset safety, and 02B must prove hosted exact-node, +coverage, isolation, and timing custody. Neither implementation is authorized +by this PR alone. + +## Follow-up work + +After this PR merges and automation reconciles it, dispatch a signed start for +02A only. Do not start 02B or 03 automatically. + +## Human review focus + +- Prospective authorization rather than retroactive adoption. +- 02A/02B boundary and exact file scope. +- Destructive reset containment and independent node custody. +- Preservation of Konan's authorship. + +## Human merge ownership + +- [ ] I can explain what changed. +- [ ] I can explain why it changed. +- [ ] I know what could break. +- [ ] I accept the remaining risks. +- [ ] The user explicitly approved this specific PR for merge. diff --git a/.agent-loop/merge-intents/WS-CI-001-02.json b/.agent-loop/merge-intents/WS-CI-001-02.json new file mode 100644 index 00000000..c568ee8a --- /dev/null +++ b/.agent-loop/merge-intents/WS-CI-001-02.json @@ -0,0 +1,9 @@ +{ + "chunk_id": "WS-CI-001-02", + "chunk_title": "Safe Routing, Cache, and Timing Refinement", + "initiative_id": "WS-CI-001", + "next_chunk_id": "WS-CI-001-02A", + "next_chunk_title": "Safe Migrate-Once Database Reset", + "next_requires_explicit_start": true, + "schema_version": 2 +}