feat(cli): add cap-evolve quickstart — free/local presets to a runnable project (#133) - #248
feat(cli): add cap-evolve quickstart — free/local presets to a runnable project (#133)#248OsherElhadad wants to merge 2 commits into
cap-evolve quickstart — free/local presets to a runnable project (#133)#248Conversation
…able project (#133) Closes #133. There was no path from a fresh install to a real run without hand-editing capevolve.yaml, and no path at all without paid credentials — the only zero-friction option was `examples/toy_calc/run.sh`, which runs inside the repo and scaffolds nothing you own. `quickstart` closes that: pick a free or local preset, get a project that is ALREADY `cap-evolve check`-green, and the next command is `cap-evolve run`. Presets (a dict in `quickstart.PRESETS`, deliberately not a plugin framework): mock (default) $0, fully offline, no credential — deterministic stand-in local $0, a local OpenAI-compatible server (Ollama/llama.cpp/vLLM) free $0, Gemini free tier via its OpenAI-compatible endpoint Distinct from `intake`, and no code is shared with it. `intake` is the guided INTERVIEW: it mines the working dir, asks what capability to optimize, and leaves an adapter STUB you must implement before `implement-and-check` opens the gate. `quickstart` asks nothing and writes a working adapter, so a user sees a sealed test number before hitting a credential or cost wall. A quickstart project is a normal project intake could have produced. Non-interactive contract: `--yes`, `--preset`, or a non-TTY stdin/stderr all mean "use defaults and never read stdin", so a piped or CI invocation cannot hang. The TTY decision reuses #215's `eventstream.capability` ladder rather than sniffing isatty locally. Secrets: a credential's env var NAME is resolved via #190's `model_config` (so provider-scoping is not re-implemented) and only PRESENCE is reported — never a value, a prefix, or a length. URL userinfo is stripped at resolution, a non-default base URL renders as `<custom>`, and everything printed goes through `dashboard.redact`. The scaffolded adapter stores the variable's NAME and reads `os.environ` at run time; nothing written contains a credential. Health checks come from #121's `doctor` (`run_doctor` + branch on `rep.ok`), not a reimplementation. The spec is produced by patching the shipped `templates/project/capevolve.yaml`, which OMITS `protected_paths` — #197 makes an empty list a hard error — and the seed task set is 16 tasks so the val split clears #195's `MIN_VAL_TASKS` floor with room to spare. Stdout is exactly one JSON object (#217); the human summary goes to stderr. Stdlib only, zero new runtime deps. cli.py touched in three places only, 12 insertions and 1 deletion: the `_cmd_quickstart` handler, one `COMMANDS` row, and replacing the literal usage string with one joined from `COMMANDS` (#214 removes that block entirely; the interim version is COMMANDS-derived so it cannot drift either).
🔬 EvidenceEvery command, with full output. Worktree 1. Baseline before any change — 179 tests2. #124 — the branch does not existNothing on origin. Not built on; 3. THE MONEY EVIDENCE — quickstart → check → run → sealed test numberFresh scratch dir, whole transcript, nothing elided: Sealed number: 4. Stdout is exactly one JSON object; human output on stderrThe two streams were captured to separate files above, so the separation is structural, not inferred: 5. Piped / non-TTY stdin does not hang, and uses defaults0.086s and 0.078s. The piped 6. Appears in #214's generated listing with ZERO editsOne conflict, one hunk, resolution = take #214's side wholesale. My handler and The #214's own tests, with quickstart registered: Includes Full suite on the #214 merge: Per-subcommand 7. #193/#121
|
|
❌ Automatic Labeling Failed An error occurred while trying to automatically label this pull request. Please check the workflow logs for details and add labels manually. |
🔍 Review — PR #248Verdict: CHANGES REQUESTED — one blocking defect. The central claim reproduces exactly as written when you follow the JSON Everything else held up under attack: 196/207 tests reproduce, no canary of mine leaked, stdout is one object in every mode I could construct, the over-redaction fix is safe, all four BlockingB1. The doc block is: mkdir ~/my-run && cd ~/my-run
cap-evolve quickstart --yes
cap-evolve check .capevolve/project # already green
cap-evolve run # sealed test number, $0
vs. with the export the JSON told you to make: Root cause, not symptom. Consequence. The first-run experience this PR is for ends in Fix (either, or both — the first is a one-line doc edit and is enough for the doc claim):
Non-blockingN1.
Consequence: a user setting up the N2.
N3. Consequence: someone passing N4.
N5. Still one JSON object, still exit 1, so the contracts hold — but the message names an internal path and not the actual problem. Fix: in the Does quickstart→run actually work from clean?Yes — with the export from the JSON Full transcript, clean scratch dir, nothing elided:
Re-run / existing-state probes — this is where I expected to find bad state, and mostly didn't:
No partial-write state found in any failure path: Presets under adversity:
Canary sweepMy own canaries, none of them theirs — five shapes × secret-looking and innocent names, plus URL userinfo. Swept all three presets, both streams, and every written file (values, 16-char prefix, 12-char prefix), on the branch alone and again on the real #190 merge so the
Zero leaks. The hostname-in-adapter case is correct and correctly scoped: the adapter must be able to reach the endpoint, and the credential inside the URL dies at Is tolerating #190's stdout violation defensible?Yes. Defensible, and I'd keep it exactly as written. I reproduced the conflict, and I reproduced it without #248 in the tree: So the defect is entirely #190's, exists without this PR, and #248 cannot fix it — the offending The mitigation is narrow in the way that matters. It does not mask the defect either: #217's own
|
| dependency | absent → behaviour | correct? |
|---|---|---|
eventstream (#215) |
interactive() falls back to sys.stdin.isatty() and sys.stderr.isatty(), wrapped in try/except → False. Verified False under both-non-TTY, and test_interactive_requires_a_tty_on_both_streams covers all four TTY combinations. |
Yes — and it fails safe: the degradation is "never prompt", which cannot hang. The and (not or) matches #215's own rung semantics. |
model_config (#190) |
_resolve_provider uses a hardcoded name table (quickstart.py:289) and reports reason: "model_config unavailable — env-name lookup only". Verified: free preset → credential_env: "GEMINI_API_KEY", credential_present: true, no value. Userinfo stripping falls back to a manual partition("//") split (:360-362) — verified it strips svc-acct:RVdeepSecret42@. |
Yes — same shape, same fields, no value; the reason string makes the degradation visible rather than silent. Not a worse default: still name-only, still provider-scoped by construction (a two-row literal table cannot cross providers). |
doctor (#121/#193) |
_health() returns None, and _main omits the health key rather than emitting a fake {"ok": true}. Verified: absent → no health key; present → health: {"ok": true, "failed": []}. |
Yes — this is the one that would be a silent degradation if it stubbed ok: true, and it deliberately doesn't. Omission is honest. |
dashboard.safe_url (#190) |
Local fallback is a strict allowlist over _PUBLIC_DEFAULTS, not a heuristic. Verified: preset URLs verbatim, https://internal.acme/v1 → <custom>, http://127.0.0.1:8080/v1 → <custom> (a different loopback port is still <custom>). |
Yes, and stricter than the real thing — the fallback is more conservative than #190's, which is the correct direction for a security fallback. The post-merge loosening is N1, and it's in the wrong direction — worth fixing before #190 lands. |
I checked the one thing that would break this whole pattern: quickstart calls mc.strip_url_userinfo (quickstart.py:359), and on #190 that function lives in dashboard, not model_config. It resolves anyway, because #190 imports it into model_config's namespace (model_config.py:51). Verified on the real merge: strip_url_userinfo: True, safe_url: True. This is a latent coupling to #190's import list, not its public API — if #190 ever switches to from . import dashboard + dashboard.strip_url_userinfo(...), mc.strip_url_userinfo disappears and quickstart's elif fallback silently takes over. Not a bug today; worth one line of comment at :359.
#197 / #195 compliance
Both verified with the real functions, not by reading the template.
$ python -c "from cap_evolve import quickstart, specfile; ...; read_yaml(spec)"
protected_paths in spec: False
optimizer_skill= mock dataset_source= adapter max_iterations= 3 stall= 2
protected_paths is absent, not present-and-empty — which is what #197 hard-errors on. It holds through #190's merge too (#190 adds 10 lines to templates/project/capevolve.yaml and none of them is protected_paths). Patching the shipped template rather than authoring a spec is the right call for exactly this reason.
$ make_splits(16 ids, seed=0, ratios=(0.5,0.25,0.25)) -> train/val/test = 8 4 4
$ make_splits(8 ids, seed=0, ratios=(0.5,0.25,0.25)) -> val = 2 # exactly #195's floor
val=4 clears MIN_VAL_TASKS=2 with margin. The comment at quickstart.py:105-108 explaining why the seed set is 16 and not 8 is load-bearing and correct — I confirmed 8 tasks lands val on the floor exactly, one rounding change from a hard fail. Note MIN_VAL_TASKS does not exist yet on any branch I can see, so the test's getattr(_splits, "MIN_VAL_TASKS", 2) is doing real work; when #195 lands with a different floor the assertion adapts.
Presets
PRESETS (quickstart.py:68-96) is a plain dict and genuinely one-row-extensible: a preset is consumed only through row["runner"]/["base_url"]/["model"]/["optimizer"]/["needs"]/["summary"], --preset choices come from sorted(PRESETS), _PUBLIC_DEFAULTS is derived from it, and the --help epilog is generated from it. I confirmed #124's branch is absent (git branch -r | grep -iE "124|cheap-onramp" → nothing), so the claim isn't validated against a real second consumer — but adding a row requires editing nothing else, and I traced every read.
Usable, or aspirational?
mock— genuinely usable, end to end, offline, and it is the default. This is the preset the central claim rests on.local— usable if you have a server. Fails gracefully without one (scored 0.0 withrunner error: HTTPErrorper rollout, not a crash), andneedsis honest and surfaced innext. Aspirational in one respect: the model is pinned toqwen2.5:3b, which most people won't have pulled even withollama serveup — that produces a 404 from a running server, indistinguishable in the trace from a dead one (both arerunner error: HTTPError). Non-blocking; worth mentioningollama pull qwen2.5:3binneeds.free— usable, but with the credential-baking wrinkle above:_CRED_ENVis fixed at scaffold time (quickstart.py:370), so scaffolding before exportingGEMINI_API_KEYwrites_CRED_ENV = ''and the adapter then sends noAuthorizationheader at all — a 401 that presents asrunner error: HTTPError, identical to a dead endpoint, with no hint that the key was the problem. Theneedsline and the# target runner needs:echo make it discoverable, so not blocking. Cleanest fix: have the adapter resolve the name at run time (_CRED_ENV or os.environlookup over the provider's known names) rather than baking a possibly-empty string, so exporting the key afterwards just works.
A missing prerequisite therefore fails with an actionable message at scaffold time (needs, in both the JSON and stderr) and a confusing failure at run time (HTTPError). The first is good; the second is the honest limit of a scaffold that can't probe. --no-doctor exists and #193's doctor is wired in, which is the right place for a real reachability probe — worth a follow-up to have quickstart --preset local actually probe the endpoint via #190's model_config.probe() (which exists) and say "nothing is answering at 127.0.0.1:11434" before you spend a run finding out.
Test quality
17 cases from 12 functions (4 from one parametrize, 3 from another). They assert outcomes, not presence, which is the right instinct — test_scaffolded_project_runs_to_a_sealed_test_number drives the actual pipeline and asserts baseline_val == 0.0 and test_reward == 1.0, so a scaffold with no headroom or a broken adapter fails there rather than in a user's terminal. test_never_hangs_and_uses_defaults uses timeout= as the assertion, which is the only way to test "does not hang". The canary test checks 16- and 12-char prefixes, not just whole values, which is the failure mode that got past three earlier tests in this epic.
Untested paths, in rough priority order:
_ask_preset(quickstart.py:262-274) is never executed. It is only reachable wheninteractive()is true, which no test can be. So the one interactive path the command has — the prompt, the default-on-empty, the default-on-invalid-answer, and theexcept → defaultwhen stdin vanishes mid-prompt — is entirely uncovered. Cheap fix: call_ask_presetdirectly withmonkeypatch.setattr(sys, "stdin", io.StringIO("free\n"))and assert"free", then""→"mock", then"garbage"→"mock". Three asserts, no subprocess._patch_spec(quickstart.py:303-323) — the whole reason Protected-paths tamper guard: verify the optimizer never edited scoring/eval/task files #197 compliance holds, and only tested via its output on the current template. I probed it directly: it correctly skips comments and indented keys, and appends the missing key at top level (which the flat reader then wins on — verifiedread_yamlreturns the appendedmax_iterations: 3over an indentedmax_iterations: 10). Behaviour is right; it just isn't pinned._template_spec's installed-wheel fallback (quickstart.py:333-340) — a second, hand-maintained copy of the spec keys that no test ever renders. If it drifts from the template, wheel installs get a subtly different project than repo installs and nothing catches it. One test asserting the fallback string parses and has the same key set as the template would close it._next_steps,safe_url's fallback branch,_optionalwith a dependency present, and the_resolve_providerexceptbranch (:298-300) — all exercised by me manually, none pinned.
Merge-order note
compileall clean. Suite: 196 passed on the branch (179 baseline + 17 new, as claimed), 207 passed with #214 merged (as claimed). test_dashboard_launch.py did not flake for me in any of the three runs (#200).
Recommended order: #214 → #197 → #195 → #193 → #190 → #215 → #245 → #248 last.
Rationale, from the merges I actually performed:
- CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214 first, non-negotiable. feat(cli): add
cap-evolve quickstart— free/local presets to a runnable project (#133) #248 vs CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214 is one conflict, one hunk, and the resolution is take CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214's side wholesale — confirmed, and confirmed nothing is lost:_usage()is untouched by feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248, theCOMMANDSrow and the handler merge cleanly, and the hunk feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248 removes is a literal usage list that CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214 replaces with the generated one anyway. feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248'scli.py:655→662-666hunk is a deletion of a literal list, which is why it resolves so cleanly. - feat(cli): add
cap-evolve quickstart— free/local presets to a runnable project (#133) #248 last because it is the only branch that reads four siblings' APIs. Merged last, every_optional()returns a real module and no fallback ships; merged earlier, the fallbacks carry it correctly (verified) but N1'ssafe_urlregression appears the moment feat(config): provider-scoped credential resolution + auto provider probing (closes #134) #190 lands afterwards. - Add
cap-evolve doctorinstall/health diagnostic #193 before feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248 but after CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214. If Addcap-evolve doctorinstall/health diagnostic #193 lands before CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214, then feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248 vs Addcap-evolve doctorinstall/health diagnostic #193 is three conflicts, not one — including the usage line, where Addcap-evolve doctorinstall/health diagnostic #193 reintroduces a literal{version|splits|check|doctor|run|estimate|dashboard}, i.e. exactly the string CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214 exists to delete. Resolvable (I did it: keep both handlers, keep feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248's COMMANDS-derived line), but it re-creates the conflict CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214 fixes. Landing CLI ergonomics: per-subcommand --help, did-you-mean, arg validation, UTF-8 hardening #214 first makes Addcap-evolve doctorinstall/health diagnostic #193 and feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248 conflict-free with each other. - feat(config): provider-scoped credential resolution + auto provider probing (closes #134) #190's stdout defect must be fixed in feat(config): provider-scoped credential resolution + auto provider probing (closes #134) #190 before it merges, per the orchestrator's ruling — otherwise cap-evolve run prints two JSON objects to stdout — the machine-readable contract has never been parseable #217's
test_run_stdout_is_a_single_json_objectis red from the moment feat(config): provider-scoped credential resolution + auto provider probing (closes #134) #190 lands, regardless of feat(cli): addcap-evolve quickstart— free/local presets to a runnable project (#133) #248. - N1 should be resolved in whichever of feat(config): provider-scoped credential resolution + auto provider probing (closes #134) #190/feat(cli): add
cap-evolve quickstart— free/local presets to a runnable project (#133) #248 lands second, since it only manifests on the merge. - Cross-host parity: single-source per-host metadata + a stdlib-only fallback that is proven (and the install.sh smoke job that makes any host ✅ honest) #245 is compatible: quickstart is stdlib-only, so Cross-host parity: single-source per-host metadata + a stdlib-only fallback that is proven (and the install.sh smoke job that makes any host ✅ honest) #245's
test_stdlib_only.pyadditions pass over it. Terminal degradation ladder + redacted forensic crash log (#144) #215 and Cross-host parity: single-source per-host metadata + a stdlib-only fallback that is proven (and the install.sh smoke job that makes any host ✅ honest) #245 conflicted only on docs/CHANGELOG in my merge — trivial.
Verification I re-ran
$ cd /tmp/rv-248 && PYTHONPATH=/tmp/rv-248/core pytest core/tests -q
196 passed in 80.23s (0:01:20)
$ git merge origin/feat/issue-137-cli-ergonomics # one conflict, one hunk, take #214's side
$ PYTHONPATH=/tmp/rv-248/core pytest core/tests -q
207 passed in 71.78s (0:01:11)
$ python -m compileall -q core/cap_evolve core/tests
compileall clean
$ pytest core/tests/test_quickstart.py core/tests/test_doctor.py -q # on the #193 merge
62 passed in 6.50s
$ cap-evolve --help # #214 merged, ZERO edits to #214
usage: cap-evolve {version|splits|check|quickstart|run|estimate|dashboard} [args]
commands:
...
quickstart Scaffold a ready-to-run project from a free/local preset (zero questions).
...
$ cap-evolve --help # #193 merged (no #214): still COMMANDS-derived
usage: cap-evolve {version|splits|check|quickstart|doctor|run|estimate|dashboard} [args]
# stdout is exactly one JSON object — every mode, via json.loads
default exit=0 ONE OBJECT
yes exit=0 ONE OBJECT
preset-free exit=0 ONE OBJECT
preset-local exit=0 ONE OBJECT
no-doctor exit=0 ONE OBJECT
force exit=0 ONE OBJECT
piped-stdin exit=0 ONE OBJECT (preset used: mock — the pipe was never read)
existing-project exit=1 ONE OBJECT (ok=False, actionable error)
read-only cwd exit=1 ONE OBJECT (ok=False, Permission denied)
unknown-preset exit=2 EMPTY stdout (argparse; stderr: "invalid choice: 'nope'")
# money path, clean scratch dir
baseline_val= 0.0 test_reward= 1.0 # with the export from `next`
baseline_val= 0.0 test_reward= 0.0 # following docs/GETTING_STARTED.md:52-56 ← B1
# #190's stdout violation, WITHOUT #248 in the tree
$ git checkout -b rv-noqs origin/feat/issue-137-cli-ergonomics && git merge origin/feat/issue-134-provider-creds
$ pytest core/tests/test_documented_cli.py::test_run_stdout_is_a_single_json_object -q
1 failed in 3.62s
# my canaries, on the #190 merge (real model_config path)
NO LEAK
hostname NOT in streams
# _optional() fallbacks, all four absent on this branch
doctor: ABSENT (fallback active) -> _health() -> None, no `health` key emitted
model_config: ABSENT -> credential_env='GEMINI_API_KEY', present=True, no value
eventstream: ABSENT -> interactive() False under non-TTY
dashboard.safe_url: absent -> 'https://internal.acme/v1' -> '<custom>', preset URLs verbatim
# #197 / #195
protected_paths in spec: False
train/val/test = 8 4 4 (8 tasks would give val=2, exactly #195's floor)
BLOCKING: docs/GETTING_STARTED.md's 4-command path omitted `export CAPEVOLVE_MOCK_SCRIPT=...`, so following the docs literally sealed `test_reward 0.0` instead of the promised 1.0 — with exit 0, a green `check`, and not a word on either stream. Two halves, both fixed: 1. The export is in the documented path, matching the `next` list quickstart already printed, with a line saying what happens if you skip it. 2. The `mock` optimizer's no-script branch WARNS on stderr instead of returning 0 with only a JSON `note` nobody surfaces. Fixed once, in `_mock_apply.py`, so `toy_calc` benefits identically. Three layers of `capture_output=True` (run-optimizer, harness, cli) were swallowing the optimizer's stderr on success; each now relays it. stdout is untouched — still exactly one report object (#217). Review findings: 1. `safe_url` checked our OWN preset defaults first, so both non-mock presets' shipped endpoints print verbatim instead of masking the value quickstart itself chose. Everything else still delegates to #190 / falls back to `<custom>`. 2. A `doctor` failure deliberately does not change the exit code: the scaffold succeeded, health is advisory. Made explicit in code and docs. 3. `--model`/`--base-url` with `mock` are refused, not silently dropped — a dead flag is the same defect family as a silent no-op optimizer. 4. `val_tasks` asks `make_splits` instead of re-deriving `round(n * 0.25)`. 5. `--force` over a plain file gives an actionable message, not `[Errno 20]`. 6. An existing `.capevolve/mock_script.json` is inside the `--force` guard; it used to be replaced unannounced, outside the project dir the guard covered. Also: the `free` adapter resolves its credential at RUN time from provider-scoped names fixed at scaffold time, so exporting the key after scaffolding works without a `--force` re-scaffold. `local`'s `needs` mentions `ollama pull`. And a coupling note at the `strip_url_userinfo` call: it resolves only because #190 imports it into `model_config`'s namespace, so the lookup is a `getattr` and the degradation is a documented branch. 6 new tests (202 passed, was 196).
🔧 Review fixesNew commit B1 (blocking) — the documented path, copy-pasted verbatimI put
The export now sits in the documented block, matching the B1, second half — the silent branch is no longer silentYou were right that this belongs in one place. Without the export, from a clean dir: The wrong number still comes out, but it is no longer wearing success's clothes — and stdout is still exactly one object.
(that probe script is not committed) The 6 non-blocking findingsN1 — N2 — a N3 — Refuse rather than warn: with N4 — N5 — N6 (the wrinkle) — the junk Also from your presets section: the No Coupling notes recorded1. 2. Corrected merge order#214 → #197 → #195 → #193 → #190 → #215 → #245 → #248 last. Your correction stands and I re-verified the part that motivates it: #214 must go first. If #193 lands before it, #248-vs-#193 is 3 conflicts including the usage line, where #193 reintroduces the literal #248-vs-#214 is still one conflict, one hunk, take #214's side wholesale — confirmed again on this commit, in a throwaway worktree: The three Verification re-run
Files touched
|
Closes #133.
What
cap-evolve quickstart— the zero-question fast path from a fresh install to a real, sealed run. Before this, reaching a run meant hand-editingcapevolve.yaml, and reaching a real one meant paid credentials; the only zero-friction option wasexamples/toy_calc/run.sh, which runs inside the repo and scaffolds nothing you own.quickstartwrites a project into a directory of your own that is alreadycap-evolve check-green, so the next command iscap-evolve run.Presets
A plain dict in
quickstart.PRESETS. No plugin framework — a preset is one row.mock(default)local127.0.0.1freeGEMINI_API_KEYexportedHow this differs from
intakeNo code is shared, and neither invokes the other.
intakeis the guided interview: it mines your working dir, asks what capability to optimize, and leaves an adapter stub you must implement beforeimplement-and-checkopens the gate.quickstartasks nothing and writes a working adapter, so a curious installer sees a sealed test number before hitting a credential or cost wall.A quickstart project is an ordinary project that intake could have produced. Use quickstart to see the pipeline work; use intake when you have real work to optimize.
Non-interactive contract
--yes,--preset, or a non-TTY stdin/stderr all mean "use defaults, never read stdin". A piped or CI invocation cannot hang — proven with a subprocess timeout, since "hangs forever" is not a value an assertion can inspect. The TTY decision reuses #215'seventstream.capabilityladder (pipe/noneare the non-TTY rungs) rather than sniffingisattylocally;plain/dumbare real terminals that merely refuse colour, so they still get the prompt. There is a documented fallback for while #215 is unmerged.No-secret guarantee
model_config, so provider-scoping is not re-implemented here.os.environat run time. Nothing written to disk contains a credential.https://user:token@host/) is stripped at the single point of resolution, before anything is stored, printed, or reported.<custom>— a real internal gateway URL already leaked into a public PR in this epic.dashboard.redact. One deliberate exception, re-stamped after redaction:credential_envandcredential_present.redactmasks any value under a key that merely looks secret, so the provider block came out ascredential_env: «redacted», hiding the one thing the user needs (which var to export) for zero security gain.credential_envis a NAME andcredential_presenta bool, secret-free by construction inmodel_config. Add cap-evolve doctor install/health diagnostic #121'sdoctorsolves this identically (render names post-redaction).Verified with multi-shape canaries under innocent-looking env names (
BUILD_NUMBER,DEPLOY_TAG,REGION_HINT,TELEMETRY_ID) — a key-name heuristic missed exactly that case earlier in this epic. Shapes: bare high-entropy, UUID,ghp_,sk-, opaque watsonx-style base64. Prefixes (16- and 12-char) are checked too, since a truncated fragment still identifies the account and no longer matches a shape rule.Reuse rather than reimplementation
doctorrun_doctor()+ branch onrep.ok. No health checks reimplemented;format_reportgoes to stderr on failure.model_config.resolve(require_credential=False)+strip_url_userinfo. Provider-scoping and precedence not duplicated.eventstream.capabilityfor the interactive decision.templates/project/capevolve.yaml, which deliberately OMITSprotected_paths. Asserted, so a future template change cannot reintroduce an empty list (a hard error).MIN_VAL_TASKSexactly — one rounding change from a hard-failing scaffold.Each optional dependency is reached through a single
_optional()import lookup with a documented fallback, so this merges in any order.#124
No — the branch does not exist.
git branch -r | grep 124returns nothing on origin, so there was no entry point to build on. Rather than invent a parallel preset,PRESETSis a dict whose rows carryprovider/runner/base_url/model/optimizer/needs: #124's cheap-real-run preset is one more row, no structural change. Happy to add it in this PR if the branch lands first.Exact
cli.pylines touched12 insertions, 1 deletion, three places — nothing else:
_cmd_run(new lines 469-474)_cmd_quickstarthandler + blank linesCOMMANDS(new line 652)"quickstart": _cmd_quickstart,main()(line 655 → 662-666)usage: cap-evolve {version|splits|...}string with one joined fromCOMMANDSThat third change is not a new literal — it removes one. #214 deletes that whole block and replaces it with
_usage(); until it lands, joiningCOMMANDSmeans a newly registered subcommand appears with zero edits either way. On merge with #214, that hunk is the single conflict and the resolution is "take #214's side wholesale".Expected merge order
Order-independent. Recommended: #214 → #193/#121 → #190 → this. Verified against all three merged locally (see Verification). #214 is the only conflict, and it is one hunk with an obvious resolution.
Note found while verifying: #214 and #190 conflict with each other on #217's contract —
test_run_stdout_is_a_single_json_objectfails with both merged, because #190 prints a{"step": "provider"}line torun's stdout. Pre-existing, reproduced without any of my changes applied, and not mine to fix; flagging it for whoever merges second.Files touched
core/cap_evolve/quickstart.py— new, 430 linescore/tests/test_quickstart.py— new, 17 testscore/cap_evolve/cli.py— +12 / -1docs/GETTING_STARTED.md— new section 4 (preset table, intake contrast, contracts)Verification
Full suite: 196 passed, 0 failed (baseline 179 + 17 new).
compileallclean. On the #214 merge: 207 passed, 0 failed, including its documented-CLI checker over the new docs.The money evidence — quickstart → check → run → sealed test number
baseline_val 0.0 → sealed test_reward 1.0, gate-accepted, $0, zero model calls.
Stdout is exactly one JSON object
Human output is on stderr (see the transcript above — the two streams were captured separately).
Appears in #214's generated listing with ZERO edits
Merged
origin/feat/issue-137-cli-ergonomicslocally._usage()untouched:The description is pulled from the handler docstring. All 11 of #214's tests pass, including
test_every_subcommand_renders_helpforquickstart.Piped / non-TTY stdin does not hang
0.08s, defaults used. Four stdin shapes are covered by a parametrized test with
timeout=60.No canary leaks — including innocent key names
No value, no prefix, no length, in output or in any written file.
#197
protected_pathsand #195 val floortest_spec_omits_protected_pathsparses the written spec and asserts the key is absent (not present-and-empty, which #197 hard-errors on).test_val_split_clears_the_min_val_floorruns the realmake_splitsat the spec's seed/ratios and assertslen(val) > MIN_VAL_TASKS.Full suite