refactor(data): unify source-health tracking on one shared router (P1.3)#66
Merged
Conversation
Closes #49 (P1.3). There were two overlapping failover/health mechanisms, and no single source of truth: the daily fetcher's inline cascade recorded NO health, while realtime/news each created their own DataSourceRouter and the /admin view created a third, empty one — so source health was fragmented and the dashboard showed nothing real. Unify on a process-wide shared router without touching the proven cascade logic: - source_router.py: add get_source_router() singleton + reset_source_router(). - fetcher.py: record per-source success (on fresh data) / failure (on exception) into the shared router at the 4 cascade points. Stale data is data lag, not a source failure, so it is not recorded. Recording is wrapped defensively — it can never break a data fetch. - realtime.py / news_fetcher.py / admin.py: default to the shared router, so all data sources now feed (and /admin reads) one health view. - conftest.py: autouse reset of the shared router for test isolation. Behavior-preserving: the cascade order/retry/freshness logic is unchanged — only additive health instrumentation. ruff clean; 69 data/router/realtime/news tests green (incl. 5 new). Routing-aware cascade (skipping DOWN sources via the router) is a deferred Phase-2 follow-up noted on the issue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #49. There were two overlapping failover/health mechanisms with no single source of truth: the daily fetcher's inline cascade recorded no health, while realtime/news each built their own
DataSourceRouterand/admincreated a third, empty one — so health was fragmented and the dashboard showed nothing real.Fix — unify on a shared router, without touching the proven cascade
source_router.py:get_source_router()process-wide singleton +reset_source_router().fetcher.py: record per-source success (on fresh data) / failure (on exception) at the 4 cascade points. Stale data is data lag, not a source failure → not recorded. Recording is wrapped defensively so it can never break a fetch.realtime.py/news_fetcher.py/admin.py: default to the shared router, so all sources feed (and/adminreads) one health view.conftest.py: autouse reset for test isolation (health is in-memory + shared).Behavior-preserving
Cascade order / retry / freshness logic is unchanged — only additive health instrumentation on a hot path, wrapped to be failure-proof.
ruffclean; 69 data/router/realtime/news tests green (incl. 5 new proving the singleton is shared and the fetcher records into it).Scope note: making the cascade consult the router to skip DOWN sources (routing-aware failover) is a deliberate Phase-2 follow-up — higher risk on the hot path; this PR establishes the single health brain first. Simulation-only; not investment advice.