This repository is a local reproduction of a DSPy GEPA workflow for AIME-style math problems. It demonstrates how to use Laminar tracing and OneQuery-backed reflection to inspect LLM telemetry while GEPA improves prompts.
The tutorial program runs a two-step solver:
solve: produces an initial answer and reasoning.review: checks the initial answer and returns the final answer.
Both calls are traced under the same Laminar trace, so GEPA reflection can inspect the full problem trajectory. When enabled, the OneQuery reflection agent queries the Laminar source for relevant spans before proposing prompt updates.
The latest kept experiment used OneQuery + Laminar telemetry reflection with a small AIME evaluation setup.
- Result JSON:
reports/gepa_onequery_bigger_result.json - Markdown report:
reports/gepa_onequery_bigger_report.md - Reflection transcripts:
reports/onequery_reflection_transcripts_bigger/ - Final test score:
5/15(33.33%)
uv run python aime_gepa_tutorial.py \
--mode gepa \
--limit-train 24 \
--limit-val 12 \
--limit-test 15 \
--num-threads 2 \
--gepa-auto none \
--max-metric-calls 72 \
--no-cache \
--reflection-agent-onequery \
--reflection-agent-onequery-org onequery-demo \
--reflection-agent-onequery-source laminar-aime-gepa \
--reflection-agent-transcripts-dir reports/onequery_reflection_transcripts_bigger \
--result-json reports/gepa_onequery_bigger_result.json \
--report reports/gepa_onequery_bigger_report.md \
--plot reports/gepa_onequery_bigger_progress.png \
--log-dir gepa_logs/gepa_onequery_bigger- Python
>=3.10,<3.15 uv- OpenAI API credentials for DSPy
- Laminar credentials unless running with
--no-laminar - OneQuery CLI/source access when using
--reflection-agent-onequery