Skip to content

Enforce Go cutover completion gates#108

Merged
mrjf merged 2 commits into
mainfrom
codex/go-cutover-completion-gate
Jun 5, 2026
Merged

Enforce Go cutover completion gates#108
mrjf merged 2 commits into
mainfrom
codex/go-cutover-completion-gate

Conversation

@mrjf
Copy link
Copy Markdown
Contributor

@mrjf mrjf commented Jun 5, 2026

TL;DR

This PR makes the Python -> Go migration completion claim deterministic. It adds a Go-only cutover gate that runs with Python runtime env vars cleared, requires every checked-in legacy Python pytest node to have an explicit Go replacement mapping, and turns the real command behavior/state-diff checks into hard failures.

Normal PRs still collect this evidence without failing solely because the migration is incomplete. Crane PRs and manual enforce_completion=true runs now hard-fail until the real command gates, golden replay gates, no-Python gate, and test-conversion manifest all pass.

Related to #78. This does not mark #78 done; it prevents future false completion.

Problem

  • Crane has been able to report Score: 1.0 and crane-completed while core Go CLI behavior still fails real command assertions.
  • The final migration gate must not depend on a live Python CLI once the Go port is cut over.
  • The existing evidence path did not require a complete, explicit mapping from legacy Python tests to Go replacements.

Approach

Area Change
Cutover tests Add TestGoCutover* tests that run the Go binary with Python hidden and emit Crane gates.
Real behavior Expand real command scenarios to 20 and make failures hard t.Errorf results.
Coverage manifest Add a checked-in JSON manifest that must map every legacy Python pytest node to Go test names.
Workflow Run normal Go tests separately from TestGoCutover, append both JSON streams to scorer input, and enforce cutover status only in completion-enforced mode.
Scorer Allow a complete all-Go golden replay plus no-Python gate to satisfy final Python-reference requirements.

Implementation

  • .github/workflows/migration-ci.yml: runs go test -skip '^TestGoCutover' ./..., then runs go test ./cmd/apm -run '^TestGoCutover' with Python env vars cleared. GO_CUTOVER_STATUS is hard-gated only when migration completion is enforced.
  • .crane/scripts/score.go: treats Python reference/test gates as satisfied only when the Go-only real behavior, golden fixture, all-Go replay, and no-Python gates are all green.
  • cmd/apm/go_cutover_coverage_test.go: discovers Python pytest nodes under tests/ except tests/parity/, compares them to the Go cutover manifest, and emits python_behavior_contracts, golden_fixture_corpus, all_go_golden_tests, and no_python_runtime_dependency gates.
  • cmd/apm/real_behavior_test.go: expands the real behavior contract to install/update/compile/pack/unpack/run/audit/policy/mcp/runtime/plugin/marketplace/config/cache/uninstall/prune/deps/view scenarios.
  • cmd/apm/CUTOVER.md: documents the new completion gate command and the final deletion-grade checklist.
  • tests/unit/test_crane_score.py: verifies the scorer can reach 1.0 without live Python only when all-Go replay evidence is complete.

Trade-offs

  • The new manifest starts empty on purpose, so the cutover gate reports 0/23766 mapped Python tests today. That is evidence, not noise.
  • The gate is non-enforcing for ordinary PRs so this guardrail can merge before the Go port is complete. It is enforcing for crane/* PRs and manual completion checks.
  • This edits .github/workflows/migration-ci.yml, not the Agentic Workflow markdown sources, so gh aw compile is not required by AGENTS.md.

Benefits

  1. Crane cannot call Crane Migration: Python to Go -- Full APM CLI Rewrite #78 complete just because facade parity or old scorer gates are green.
  2. Reviewers get a precise count of unmigrated Python tests from Go-only CI evidence.
  3. Final cutover can prove the Go binary runs without Python runtime support.
  4. Real command behavior now has state assertions instead of success-message assertions.

Validation

$ git diff --check
# passed
$ .venv/bin/python -m pytest tests/unit/test_crane_score.py -q
24 passed in 77.30s (0:01:17)
$ APM_PYTHON_BIN= APM_PYTHON_CONTRACT_INVENTORY= PYTHONPATH= VIRTUAL_ENV= \
  go test ./cmd/apm -run '^TestGoCutoverNoPythonRuntimeDependency$' -v
=== RUN   TestGoCutoverNoPythonRuntimeDependency
{"crane":"gate","name":"no_python_runtime_dependency","passed":true}
--- PASS: TestGoCutoverNoPythonRuntimeDependency (0.19s)
PASS
ok  github.com/githubnext/apm/cmd/apm  0.730s

Expected incomplete-cutover result on current main behavior:

$ APM_PYTHON_BIN= APM_PYTHON_CONTRACT_INVENTORY= PYTHONPATH= VIRTUAL_ENV= \
  go test ./cmd/apm -run '^TestGoCutover' -json
Go cutover coverage incomplete: 0/23766 Python tests mapped to Go tests; 23766 missing.
{"crane":"gate","name":"functional","passing":1,"total":20}
{"crane":"gate","name":"state_diff","passing":1,"total":20}
FAIL github.com/githubnext/apm/cmd/apm

Mermaid validation was attempted for a PR-body diagram, but local mmdc required Chrome 148.0.7778.97 while Puppeteer installed 149.0.7827.22; this body omits the diagram rather than including an unvalidated one.

How to test

  1. Run .venv/bin/python -m pytest tests/unit/test_crane_score.py -q.
  2. Run APM_PYTHON_BIN= APM_PYTHON_CONTRACT_INVENTORY= PYTHONPATH= VIRTUAL_ENV= go test ./cmd/apm -run '^TestGoCutover' -json and confirm it fails with unmapped Python tests plus real behavior gate ratios.
  3. Trigger Migration Parity and Benchmarks manually with enforce_completion=true to verify incomplete cutover blocks completion.
  4. Trigger the same workflow without enforcement to verify the evidence artifact is collected without failing solely because the migration is incomplete.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 5, 2026

Migration Benchmark Results

Migration CLI Benchmark

Includes fixture-backed commands that must read, write, execute, or fail against real project state. The installed-project fixture contains apm.yml, apm.lock.yaml, apm_modules packages, local .apm primitives, target directories, deployed prompt files, and sample source files.
The harness checks return-code parity for each command. Detailed stdout/stderr byte counts are kept in the JSON samples, but this is not an output-parity test.

Max allowed Go/Python median ratio: 5.00

Benchmark Command Fixture Python median Go median Go/Python Result Return codes
init scaffold init --yes empty-project 0.4360s 0.0014s 0.00x 315.53x faster {'python': [0], 'go': [0]}
targets json targets --json installed-project 0.4260s 0.0013s 0.00x 338.91x faster {'python': [0], 'go': [0]}
script list list installed-project 0.4297s 0.0013s 0.00x 335.46x faster {'python': [0], 'go': [0]}
deps list deps list installed-project 0.4369s 0.0013s 0.00x 331.64x faster {'python': [0], 'go': [0]}
deps tree deps tree installed-project 0.4401s 0.0015s 0.00x 301.93x faster {'python': [0], 'go': [0]}
install local package install --no-policy ./packages/local-tools local-install-project 0.4747s 0.0013s 0.00x 364.29x faster {'python': [0], 'go': [0]}
compile copilot target compile --target copilot compilation-project 0.4560s 0.0013s 0.00x 344.14x faster {'python': [0], 'go': [0]}
pack output pack --output dist installed-project 0.4530s 0.0013s 0.00x 344.38x faster {'python': [0], 'go': [0]}
run script run stamp runnable-project 0.4439s 0.0013s 0.00x 346.38x faster {'python': [0], 'go': [0]}
audit hidden unicode audit --ci audit-finding-project 0.4456s 0.0013s 0.00x 331.59x faster {'python': [1], 'go': [0]}

Workloads

  • init scaffold: Creates a new apm.yml in an otherwise empty project directory.
  • targets json: Reads configured project targets from apm.yml and emits machine output.
  • script list: Reads apm.yml scripts and renders the runnable script inventory.
  • deps list: Scans apm_modules package directories and apm.lock.yaml metadata.
  • deps tree: Builds a dependency tree from apm.lock.yaml and installed package metadata.
  • install local package: Installs a local package and materializes lock/module state.
  • compile copilot target: Discovers local primitives and writes the Copilot target artifact.
  • pack output: Resolves local package contents and writes a distributable artifact.
  • run script: Executes a project script and writes the script's side-effect file.
  • audit hidden unicode: Scans a real installed file and fails on planted hidden Unicode.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 5, 2026

Migration Benchmark Results

Migration CLI Benchmark

Includes fixture-backed commands that must read, write, execute, or fail against real project state. The installed-project fixture contains apm.yml, apm.lock.yaml, apm_modules packages, local .apm primitives, target directories, deployed prompt files, and sample source files.
The harness checks return-code parity for each command. Detailed stdout/stderr byte counts are kept in the JSON samples, but this is not an output-parity test.

Max allowed Go/Python median ratio: 5.00

Benchmark Command Fixture Python median Go median Go/Python Result Return codes
init scaffold init --yes empty-project 0.4270s 0.0013s 0.00x 334.32x faster {'python': [0], 'go': [0]}
targets json targets --json installed-project 0.4196s 0.0013s 0.00x 322.30x faster {'python': [0], 'go': [0]}
script list list installed-project 0.4213s 0.0012s 0.00x 337.86x faster {'python': [0], 'go': [0]}
deps list deps list installed-project 0.4277s 0.0013s 0.00x 332.77x faster {'python': [0], 'go': [0]}
deps tree deps tree installed-project 0.4234s 0.0014s 0.00x 309.89x faster {'python': [0], 'go': [0]}
install local package install --no-policy ./packages/local-tools local-install-project 0.4651s 0.0013s 0.00x 368.01x faster {'python': [0], 'go': [0]}
compile copilot target compile --target copilot compilation-project 0.4436s 0.0013s 0.00x 343.99x faster {'python': [0], 'go': [0]}
pack output pack --output dist installed-project 0.4436s 0.0013s 0.00x 329.38x faster {'python': [0], 'go': [0]}
run script run stamp runnable-project 0.4254s 0.0012s 0.00x 349.48x faster {'python': [0], 'go': [0]}
audit hidden unicode audit --ci audit-finding-project 0.4358s 0.0013s 0.00x 337.48x faster {'python': [1], 'go': [0]}

Workloads

  • init scaffold: Creates a new apm.yml in an otherwise empty project directory.
  • targets json: Reads configured project targets from apm.yml and emits machine output.
  • script list: Reads apm.yml scripts and renders the runnable script inventory.
  • deps list: Scans apm_modules package directories and apm.lock.yaml metadata.
  • deps tree: Builds a dependency tree from apm.lock.yaml and installed package metadata.
  • install local package: Installs a local package and materializes lock/module state.
  • compile copilot target: Discovers local primitives and writes the Copilot target artifact.
  • pack output: Resolves local package contents and writes a distributable artifact.
  • run script: Executes a project script and writes the script's side-effect file.
  • audit hidden unicode: Scans a real installed file and fails on planted hidden Unicode.

@mrjf mrjf merged commit 6bc2cdc into main Jun 5, 2026
6 checks passed
@mrjf mrjf deleted the codex/go-cutover-completion-gate branch June 5, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant