Skip to content

feat: expand scratch codegen and reduction evidence#39

Merged
wesleysimplicio merged 3 commits into
masterfrom
codex/scratch-codegen-foundation
May 30, 2026
Merged

feat: expand scratch codegen and reduction evidence#39
wesleysimplicio merged 3 commits into
masterfrom
codex/scratch-codegen-foundation

Conversation

@wesleysimplicio
Copy link
Copy Markdown
Owner

@wesleysimplicio wesleysimplicio commented May 30, 2026

Summary

  • add deterministic scratch codegen executors with LibCST/ts-morph paths and executor metrics
  • add planner completion caching keyed by provider/model/prompt/settings/template version
  • add cache telemetry (hits, misses, puts, hit_rate) for cold/warm evidence
  • add rust-axum and php-laravel pilot stacks and include nested stacks/recipes in wheels
  • add inline SkillOpt support through optional task required_skill
  • add synthetic benchmark harnesses and generated evidence for codegen and static fixers

Refs #32
Refs #33
Refs #37

Evidence

  • bench/results_scratch_codegen.md: 50/50 synthetic codegen cases, 100% executor match, 100% codegen share, 0 planner/LLM calls
  • bench/results_static_fixers.md: 50/50 synthetic verify-loop cases, 80% fixed before retry, 40% retry-call reduction

Validation

  • python -m pytest tests/python -q -> 157 passed, 1 skipped
  • focused ruff check / ruff format --check on touched Python files -> passed
  • git diff --check -> no whitespace errors; only Windows LF-to-CRLF warnings
  • cargo test --manifest-path <rendered rust-axum>/Cargo.toml -> passed
  • rendered php-laravel and ran php -l over all rendered PHP files -> passed
  • python -m build --wheel --outdir <tmp> -> built successfully and included nested stack/recipe templates

Remaining Release Gates

This PR advances the local implementation and synthetic evidence, but the tracking issues still need external release evidence before closure:

  • real 50-scratch corpus with planner/doer credentials
  • real LLM baseline pass-rate and latency comparison
  • real package-manager execution for Laravel requires Composer on PATH

@wesleysimplicio wesleysimplicio marked this pull request as ready for review May 30, 2026 07:27
@wesleysimplicio wesleysimplicio changed the title feat: add scratch codegen executor registry feat: expand scratch codegen and reduction evidence May 30, 2026
@wesleysimplicio wesleysimplicio merged commit 0262e29 into master May 30, 2026
1 check passed
@wesleysimplicio wesleysimplicio deleted the codex/scratch-codegen-foundation branch May 30, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant