Skip to content

Add golden fixtures and contract tests for structured route logs#59

Merged
Disaster-Terminator merged 1 commit intoreview/route-contract-hardening-2026-05-05from
add-tests-for-structured-route-logs-gpx9y3
May 5, 2026
Merged

Add golden fixtures and contract tests for structured route logs#59
Disaster-Terminator merged 1 commit intoreview/route-contract-hardening-2026-05-05from
add-tests-for-structured-route-logs-gpx9y3

Conversation

@Disaster-Terminator
Copy link
Copy Markdown
Owner

Motivation

  • Freeze the structured route log shape with a stable golden fixture so downstream observability work can rely on a fixed contract.
  • Make log-shape drift obvious in CI without including prompts, secrets, Authorization headers, or raw request bodies.
  • Ensure existing summary and budgeting tooling (router_log_summary.py and check_route_error_budget.py) handle mixed records (explicit route_id, null/passthrough, and missing route_id) and malformed trailing JSON gracefully.

Description

  • Add a golden NDJSON fixture at tests/samples/router_logs_contract.ndjson that includes route_complete, route_error, a passthrough (route_id: null), a legacy log missing route_id, a startup non-route line, and a trailing malformed JSON line.
  • Add contract tests to tests/test_router_log_summary.py that parse the fixture with parse_route_records and assert totals, routes vs targets buckets, error_types, and parse diagnostics, and a safety test that asserts forbidden sensitive patterns are not present in the fixture.
  • Add a contract test to tests/test_check_route_error_budget.py that exercises check_budget on the fixture with permissive BudgetConfig and asserts target_error_rates, route_error_rates (including unknown for missing route IDs), error_types, and parse diagnostics.
  • Keep this as a test/contract layer only and do not change routing logic or introduce any secrets or production logs.

Testing

  • Ran uv run python -m pytest -q and the test suite passed (153 passed, 1 warning).
  • Ran uv run python scripts/router_log_summary.py --json < tests/samples/router_logs_contract.ndjson which produced a JSON summary successfully showing expected counts and diagnostics.
  • Ran uv run python scripts/check_route_error_budget.py < tests/samples/router_logs_contract.ndjson which executed the CLI reading from stdin and printed a failing budget result (non-zero exit) because the fixture intentionally contains route errors; the fixture-driven unit test for check_budget uses a permissive BudgetConfig and passed under pytest.

Codex Task

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Disaster-Terminator! 👋

Your private repo does not have access to Sourcery.

Please upgrade to continue using Sourcery ✨

@qodo-code-review
Copy link
Copy Markdown

ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one.

@Disaster-Terminator Disaster-Terminator merged commit dc103cd into review/route-contract-hardening-2026-05-05 May 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant