Skip to content

Require golden fixture cutover gates#105

Merged
mrjf merged 4 commits into
mainfrom
codex/golden-fixture-completion-gate
Jun 4, 2026
Merged

Require golden fixture cutover gates#105
mrjf merged 4 commits into
mainfrom
codex/golden-fixture-completion-gate

Conversation

@mrjf
Copy link
Copy Markdown
Contributor

@mrjf mrjf commented Jun 3, 2026

Description

Require golden-fixture cutover evidence before migration completion can reach migration_score = 1.0 by enforcing three hard gates: committed golden corpus, all-Go golden replay, and no Python runtime dependency.
Also includes follow-up CI stabilization: updated tests/parity/python_contract_coverage.yml to map newly added Python tests so the Python behavior contracts parity gate and Go test workflow pass consistently.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

@mrjf
Copy link
Copy Markdown
Contributor Author

mrjf commented Jun 3, 2026

@copilot fix tests

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Jun 3, 2026

@copilot fix tests

Fixed in 0fbef5d by applying Ruff formatting to tests/unit/test_crane_score.py, which resolves the failing test/lint check.

Copilot finished work on behalf of mrjf June 3, 2026 22:20
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Migration Benchmark Results

Migration CLI Benchmark

Max allowed Go/Python median ratio: 5.00

Command Python median Go median Go/Python Result Return codes
--help 0.4365s 0.0012s 0.00x 359.44x faster {'python': [0], 'go': [0]}
--version 0.4401s 0.0011s 0.00x 382.78x faster {'python': [0], 'go': [0]}
compile --help 0.4505s 0.0012s 0.00x 371.87x faster {'python': [0], 'go': [0]}
install --help 0.4363s 0.0013s 0.00x 344.97x faster {'python': [0], 'go': [0]}
pack --help 0.4360s 0.0013s 0.00x 339.08x faster {'python': [0], 'go': [0]}
audit --help 0.4338s 0.0012s 0.00x 354.97x faster {'python': [0], 'go': [0]}
init --yes 0.4446s 0.0012s 0.00x 358.44x faster {'python': [0], 'go': [0]}

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Migration Benchmark Results

Migration CLI Benchmark

Max allowed Go/Python median ratio: 5.00

Command Python median Go median Go/Python Result Return codes
--help 0.4305s 0.0013s 0.00x 337.23x faster {'python': [0], 'go': [0]}
--version 0.4333s 0.0012s 0.00x 361.36x faster {'python': [0], 'go': [0]}
compile --help 0.4275s 0.0013s 0.00x 332.93x faster {'python': [0], 'go': [0]}
install --help 0.4406s 0.0013s 0.00x 336.18x faster {'python': [0], 'go': [0]}
pack --help 0.4471s 0.0013s 0.00x 354.24x faster {'python': [0], 'go': [0]}
audit --help 0.4464s 0.0013s 0.00x 354.99x faster {'python': [0], 'go': [0]}
init --yes 0.4517s 0.0013s 0.00x 353.72x faster {'python': [0], 'go': [0]}

@mrjf
Copy link
Copy Markdown
Contributor Author

mrjf commented Jun 3, 2026

@copilot fix tests

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Jun 3, 2026

@copilot fix tests

Fixed in c4ac5bc by updating tests/parity/python_contract_coverage.yml with the missing Python test mappings so parity/Go test gates pass again.

Copilot finished work on behalf of mrjf June 3, 2026 22:35
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 4, 2026

Migration Benchmark Results

Migration CLI Benchmark

Max allowed Go/Python median ratio: 5.00

Command Python median Go median Go/Python Result Return codes
--help 0.3844s 0.0012s 0.00x 333.44x faster {'python': [0], 'go': [0]}
--version 0.3763s 0.0011s 0.00x 335.90x faster {'python': [0], 'go': [0]}
compile --help 0.3728s 0.0012s 0.00x 315.15x faster {'python': [0], 'go': [0]}
install --help 0.3799s 0.0012s 0.00x 323.00x faster {'python': [0], 'go': [0]}
pack --help 0.3799s 0.0012s 0.00x 316.91x faster {'python': [0], 'go': [0]}
audit --help 0.3753s 0.0011s 0.00x 344.35x faster {'python': [0], 'go': [0]}
init --yes 0.3806s 0.0012s 0.00x 321.30x faster {'python': [0], 'go': [0]}

…completion-gate

# Conflicts:
#	cmd/apm/CUTOVER.md
#	tests/parity/python_contract_coverage.yml
#	tests/unit/test_crane_score.py
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 4, 2026

Migration Benchmark Results

Migration CLI Benchmark

Includes fixture-backed commands that must read, write, execute, or fail against real project state. The installed-project fixture contains apm.yml, apm.lock.yaml, apm_modules packages, local .apm primitives, target directories, deployed prompt files, and sample source files.
The harness checks return-code parity for each command. Detailed stdout/stderr byte counts are kept in the JSON samples, but this is not an output-parity test.

Max allowed Go/Python median ratio: 5.00

Benchmark Command Fixture Python median Go median Go/Python Result Return codes
init scaffold init --yes empty-project 0.4476s 0.0013s 0.00x 336.18x faster {'python': [0], 'go': [0]}
targets json targets --json installed-project 0.4372s 0.0014s 0.00x 309.71x faster {'python': [0], 'go': [0]}
script list list installed-project 0.4560s 0.0015s 0.00x 309.50x faster {'python': [0], 'go': [0]}
deps list deps list installed-project 0.4706s 0.0014s 0.00x 329.98x faster {'python': [0], 'go': [0]}
deps tree deps tree installed-project 0.4645s 0.0015s 0.00x 304.51x faster {'python': [0], 'go': [0]}
install local package install --no-policy ./packages/local-tools local-install-project 0.5101s 0.0013s 0.00x 384.53x faster {'python': [0], 'go': [0]}
compile copilot target compile --target copilot compilation-project 0.4814s 0.0014s 0.00x 347.51x faster {'python': [0], 'go': [0]}
pack output pack --output dist installed-project 0.4717s 0.0014s 0.00x 343.57x faster {'python': [0], 'go': [0]}
run script run stamp runnable-project 0.4362s 0.0013s 0.00x 340.82x faster {'python': [0], 'go': [0]}
audit hidden unicode audit --ci audit-finding-project 0.4456s 0.0014s 0.00x 315.88x faster {'python': [1], 'go': [0]}

Workloads

  • init scaffold: Creates a new apm.yml in an otherwise empty project directory.
  • targets json: Reads configured project targets from apm.yml and emits machine output.
  • script list: Reads apm.yml scripts and renders the runnable script inventory.
  • deps list: Scans apm_modules package directories and apm.lock.yaml metadata.
  • deps tree: Builds a dependency tree from apm.lock.yaml and installed package metadata.
  • install local package: Installs a local package and materializes lock/module state.
  • compile copilot target: Discovers local primitives and writes the Copilot target artifact.
  • pack output: Resolves local package contents and writes a distributable artifact.
  • run script: Executes a project script and writes the script's side-effect file.
  • audit hidden unicode: Scans a real installed file and fails on planted hidden Unicode.

@mrjf mrjf merged commit c39f1b2 into main Jun 4, 2026
6 checks passed
@mrjf mrjf deleted the codex/golden-fixture-completion-gate branch June 4, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants