test: 15 tests for lib/eval/report.py (export_json, export_trace)#17
Open
hai-pilgrim wants to merge 2 commits into
Open
test: 15 tests for lib/eval/report.py (export_json, export_trace)#17hai-pilgrim wants to merge 2 commits into
hai-pilgrim wants to merge 2 commits into
Conversation
…ompaction - install.sh now auto-configures ~/.claude/settings.json (creates pluginDirs entry, idempotent across create/add/already-exists cases) - uninstall.sh now cleans up the settings.json pluginDirs entry - Add compact-session.sh: self-contained script that finds JSONL, runs supercompact, backs up original, replaces, and reports results - Simplify /supercompact command from 5-step multi-bash prompt to single script call with CLAUDE_PLUGIN_ROOT fallback to hardcoded install path - Simplify PreCompact hook to backup-only (removes wasted supercompact run that Claude's LLM compaction immediately overwrites) - Update README: accurate hook description, file tree with compact-session.sh, update/upgrade docs, standalone binary limitations clearly stated Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
export_json: file creation, empty array, result structure (method, budget, model_key, composite, ndcg), speed/token counts, dimension scores with score and probe_count, multiple results, valid JSON. export_trace: file creation, path location, filename contains method/budget, JSON has method/budget, empty answers → empty entries, matching probe included, unmatched answer skipped, auto-creates trace dir.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/test_eval_report.pywith 15 pytest tests forlib/eval/report.pyexport_json: file creation, empty →[], result structure (method/budget/model_key/composite/ndcg), speed/token counts, dimension scores (score + probe_count), multiple results, valid JSONexport_trace: file creation, path location, filename contains method+budget, JSON has method/budget, empty answers → empty entries, matched probe included with score, unmatched answer skipped, auto-creates nested trace dirTest plan
uv run pytest tests/test_eval_report.py -v🤖 Wandered here by hai-pilgrim