Context
The CLI promptkit eval command prints a plain-text table to stdout. That's fine for local use, but in CI you often want a machine-readable artifact to upload, trend over time, or render in a PR check.
Proposal
Add a --reporter flag to promptkit eval:
promptkit eval ./evals/sentiment.eval.ts --reporter json --out results.json
promptkit eval ./evals/sentiment.eval.ts --reporter html --out results.html
Reporters:
text (default, current behaviour)
json — full EvalResult serialised, plus a small summary header
html — a self-contained static HTML page with a per-case table
Implementation notes
- Create
src/cli/reporters/ with text.ts, json.ts, html.ts.
- Define a
Reporter interface: report(result: EvalResult): string.
- The HTML reporter should be a single string template — no external CSS/JS deps. Keep it ~100 lines.
- Add a
--out flag that writes to a file instead of stdout (text reporter still goes to stdout by default).
Acceptance
/label "good first issue" "help wanted" "area:cli" "area:eval" "enhancement"
Context
The CLI
promptkit evalcommand prints a plain-text table to stdout. That's fine for local use, but in CI you often want a machine-readable artifact to upload, trend over time, or render in a PR check.Proposal
Add a
--reporterflag topromptkit eval:Reporters:
text(default, current behaviour)json— fullEvalResultserialised, plus a small summary headerhtml— a self-contained static HTML page with a per-case tableImplementation notes
src/cli/reporters/withtext.ts,json.ts,html.ts.Reporterinterface:report(result: EvalResult): string.--outflag that writes to a file instead of stdout (text reporter still goes to stdout by default).Acceptance
--reporterand--outflags wired into the eval command.docs/cli.mdupdated with examples./label "good first issue" "help wanted" "area:cli" "area:eval" "enhancement"