Skip to content

docs: add reports output formats documentation page#96

Merged
himanshu231204 merged 1 commit into
OpenAgentHQ:mainfrom
PrinceThummar011:docs/reports-output-formats
Jul 14, 2026
Merged

docs: add reports output formats documentation page#96
himanshu231204 merged 1 commit into
OpenAgentHQ:mainfrom
PrinceThummar011:docs/reports-output-formats

Conversation

@PrinceThummar011

Copy link
Copy Markdown
Contributor

Summary

Adds a new documentation page (docs/reports.md) that explains all 5 report
output formats produced by OpenAgent Eval: Terminal, Markdown, HTML, JSON,
and Comparison. Previously these formats were only discoverable by reading
source code — this page gives users a single reference for what each format
looks like, how to generate it, and when to use it.

Closes #95

What's included

For each of the 5 formats:

  • Description of what it is and when to use it
  • Exact CLI command to generate it
  • What the output contains (summary, metrics, errors, samples, config, etc.)
  • A real example output block, generated from actual code / test fixtures

Formats covered:

  • Terminal Report – Rich-formatted console output (oaeval report)
  • Markdown Report.md file with tables (--output markdown)
  • HTML Report – Styled Jinja2 web page (--output html), including
    template customization
  • JSON Report – Machine-readable schema (--output json)
  • Comparison Report – Side-by-side experiment diff (oaeval compare)

Other changes

  • Registered docs/reports.md in mkdocs.yml navigation
  • Linked the new page from docs/index.md (Beautiful Reports feature card)

Testing

  • Verified sample output against actual oaeval report / oaeval compare
    runs (or tests/unit/test_reports/ fixtures where a live run wasn't
    feasible)
  • Built docs locally with mkdocs serve to confirm nav and rendering

@himanshu231204

Copy link
Copy Markdown
Contributor

Hey @PrinceThummar011! 👋 Welcome to OpenAgent Eval, and thank you so much for this fantastic first contribution!

This is a really well-crafted documentation page — I love the attention to detail:

  • Comprehensive coverage of all 5 report formats (Terminal, Markdown, HTML, JSON, Comparison) with clear CLI commands and sample output for each.
  • Real, verifiable examples — the sample Terminal output with
    ich\ formatting, the JSON schema, and the Comparison Report delta table all look spot-on.
  • The practical touches — covering custom Jinja2 templates for HTML reports, piping output to files, and configuration reference table — that's the kind of detail that makes docs genuinely useful.
  • Clean integration with the existing site via \mkdocs.yml\ nav registration and the link from \docs/index.md.

I've verified that:
✅ All CI workflows are passing (Lint & Format, Tests on 3.10/3.11/3.12, Coverage, Documentation build, CodeQL, Build Package)
✅ No merge conflicts — branch is clean against \main$
✅ The \mkdocs serve\ build will render correctly with the new nav entry

Really impressed with how thorough this is for a first PR. This closes #95 perfectly.

Looking forward to more contributions from you! 🚀

@himanshu231204 himanshu231204 merged commit c7f2718 into OpenAgentHQ:main Jul 14, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: Add reports output formats documentation page

2 participants