v0.4.6 – Bug Fixes & Improvements

What's Changed

✨ Features

• Retriever Settings Validation — validate retriever settings keys to catch typos early (#172)
• Standardized Provider Errors — standardize provider error _str_\ format for better debugging (#167)
• Environment Variables Reference — new docs page covering all environment variables (#176)
• Package Release Rules — add release workflow documentation to AGENT.md (#183)

🐛 Bug Fixes

• pgvector Async Connection — use async psycopg connection in pgvector retriever (#182)
• Report max_examples Respect — HTML and JSON generators now respect \ReportConfig.max_examples\ (#181)
• Synthesis Duplicate Functions — extract duplicate inner generation functions (#180)
• Synthesis Parallel Generation — parallelize adversarial test case generation with \�syncio.gather\ (#178)
• Synthesis Premature Return — fix Strategy 0 in _parse_response\ returning prematurely (#175)
• Synthesis Corpus Errors — _read_corpus\ no longer silently swallows file reading errors (#149)
• CLI Version Flag — add consistent --version\ flag to all CLI commands (#157)
• CLI Export Commands — export all CLI commands (#147)
• CLI Dry-run Warning — interpolate timeout in dry-run warning (#145)
• CLI Mock Provider — use configured provider for synth (#148)
• Progress Bar Reset — fix progress bar reset to 1/total_items on completion (#162)
• Context Variable Shadowing — rename ctx variables to avoid shadowing CLIContext and click.Context (#153, #162)
• Report max_examples Hardcoded — use \ReportConfig.max_examples\ instead of hardcoded limits (#161)
• Comparison Winner Logic — use common metrics only in comparison report (#159)
• Dataset Input Validation — add input validation for dataset path with helpful error message (#158)
• Non-ASCII Keywords — support non-ASCII characters in content gap analysis (#156)
• Failure Analysis Metrics — pass \metric_scores\ from error entries in _compute_failure_analysis\ (#155)
• Synthesis Curly Braces — escape curly braces in context before \str.format()\ in synthesis (#152)
• Report Config Validation — validate config key in \ReportManager.reconstruct()\ (#150)
• Metrics Zero Timeout — preserve zero timeout details (#146)
• Missing Command Imports — add missing imports and _all_\ entries for 6 commands (#168)
• Orphan LLMResponse — remove orphan LLMResponse construction in anthropic generate() (#166)
• Dead Regex — remove dead _SIMPLE_PATTERNS\ regex (#165)
• Corpus Naive Timestamps — normalize naive staleness timestamps (#170)
• Configuration Validation — enhance error messages for missing required config fields (#140)

📚 Documentation

• Env-var Documentation — correct env-var and config claims in docs (#179)
• Quickstart Guide — add QUICKSTART.md for coding agents (#144)
• Context Files Compressed — compress context files to <100 lines for coding agent efficiency (#142)
• AI Files Reorganized — move .ai/ files to root, add INSTRUCTIONS.md writing rules (#139)

⚙️ Testing

• Synth CLI Unit Tests — add unit tests for the synth CLI command (#174)
• Report Edge Cases — cover \ReportManager.reconstruct()\ edge cases (#173)
• Pipeline Integration Test — add full-pipeline e2e test with mock providers (#171)

⚙️ Internal

• Ignore Local Configs — ignore local config files in git (#143)
• Remove Local Artifacts — remove local artifacts from tracking (#141)

❤️ Contributors

• @himanshu231204
• @Nitjsefnie
• @fazalpsinfo-cmyk
• @Sanjays2402
• @lesbass
• @PrinceThummar011
• @Silvren
• @hkJerryLeung
• @1-gokul

Full Changelog: v0.4.5...v0.4.6

v0.4.5

Release v0.4.5

v0.4.4

Bug Fix\n\n- Added individual JSON object parsing for malformed responses\n- 4-strategy fallback for JSON parsing\n- Handles escaped characters and unescaped quotes in LLM responses\n\nFull Changelog: v0.4.3...v0.4.4

v0.4.3

Bug Fix\n\n- Simplified JSON parsing with 3-strategy fallback\n- Strategy 1: Clean and parse JSON\n- Strategy 2: Regex extraction of Q&A pairs\n- Strategy 3: Individual field extraction\n- Handles malformed LLM responses gracefully\n\nFull Changelog: v0.4.2...v0.4.3

v0.4.2

Bug Fix\n\n- Add regex fallback for JSON parsing when LLM returns malformed JSON\n- Improved control character handling in JSON responses\n- Better error handling with graceful degradation\n\nFull Changelog: v0.4.1...v0.4.2

v0.4.1

Bug Fix\n\n- Improve JSON parsing resilience in synthesis module\n- Handle control characters and unescaped newlines in LLM responses\n\nFull Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed

Bug Fixes

• Improve JSON parsing resilience in synthesis module (_parse_response)
• Handle malformed JSON from LLM (trailing commas, extra text, single quotes)

New Features

• Added CI/CD integration module for automated evaluation in pipelines
• Added CLI test command for running evaluations
• Added corpus and related modules example notebook

Tests

• Added 10 new tests for JSON parsing edge cases
• All 31 synthesis tests pass

Full Changelog: v0.3.0...v0.4.0

v0.3.0 - Production-Grade Features

What's Changed

• feat: Phase 9 - Corpus Health Auditor (THE DIFFERENTIATOR) by @himanshu231204 in #26
• feat: Phase 10 - LLM-as-Judge Metrics (NLI-based scoring) by @himanshu231204 in #27
• feat: Phase 11 — Component Diagnosis by @himanshu231204 in #28
• feat: Phase 12 — Synthetic Test Data Generator by @himanshu231204 in #29

Full Changelog: v0.2.0...v0.3.0

v0.2.0

Release v0.2.0

Features:
- Comprehensive CLI UX improvements (global flags, dry-run, shell completion, JSON output)
- Enhanced validate, delete, list, init, and doctor commands

Bug Fixes:
- Comprehensive bug hunt (Critical through Low severity)
- Dataset path config and undeclared provider dependencies
- Hallucination test fixture deepeval module replacement
- CLI ANSI code stripping in tests and --no-interactive flag

Documentation:
- Provider guides for all LLM, retriever, and embedder providers
- Examples section with RAG tutorial guide
- CLI improvements documentation

v0.1.1

Full Changelog: v0.1.0...v0.1.1