Skip to content

Isaac/feat/query mechanism#43

Open
isaaccodekill wants to merge 10 commits intomainfrom
isaac/feat/query-mechanism
Open

Isaac/feat/query mechanism#43
isaaccodekill wants to merge 10 commits intomainfrom
isaac/feat/query-mechanism

Conversation

@isaaccodekill
Copy link
Copy Markdown

No description provided.

Copy link
Copy Markdown
Collaborator

@piotrnowakowski piotrnowakowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High: the freshness gate is effectively disabled.

  • In backend/modules/web_researcher/freshness.py (check_freshness, around line 45), CCC values are looked up by (city, field).
  • But _extract_ccc_values() in the same file (around line 185) only stores keys shaped like (city, "_has_data").
  • In practice there are no real overlaps, freshness_results stays empty, and backend/modules/web_researcher/context_merger.py (around line 118) promotes any web finding without freshness data straight to resolved.
  • I reproduced this with a direct Dresden/capex probe; it returned [] instead of running freshness classification.
    Local repro example:
  • After start_step("s1", "Step 1"), the written progress.json contained one running step with "items": [].
  • After add_item("s1", "hello"), the in-memory tracker state changed to include {"text": "hello"} under that step.
  • But the on-disk progress.json still showed "items": [] instead of the new item.
    So the failure is not theoretical: the state advances in memory, but the file that the status endpoint reads does not. That is why the frontend ends up seeing stale progress even though the pipeline continues.

I also validated the UI side separately with a synthetic local run: when valid steps data was manually present in progress.json, the progress panel rendered correctly in the browser. That narrows the problem to the backend write path rather than the React component.

For the freshness issue, a similarly concrete repro was:

  • web finding: Dresden / capex = 50
  • context bundle excerpt containing city_key="Dresden" and matching partial_answer text
  • check_freshness() still returned []
    That happens because the CCC map is populated under ("dresden", "_has_data"), while the lookup is performed under ("dresden", "capex").

Comment thread .idea/vcs.xml Outdated
Copy link
Copy Markdown
Collaborator

@piotrnowakowski piotrnowakowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinning the reproduced issues inline on the changed lines.

Comment thread backend/services/progress_tracker.py Outdated
Comment thread backend/modules/web_researcher/freshness.py Outdated
Comment thread backend/modules/web_researcher/freshness.py Outdated
Comment thread backend/modules/web_researcher/context_merger.py Outdated
Comment thread backend/api/services/pdf_export.py Outdated
…ichment to markdown-only

Main removed the SQL researcher and db_models in fa3b039. Adapted this branch's
enrichment layer to work on markdown-only:

- Dropped sql_researcher imports, parallel SQL execution, and SQL result
  processing from the orchestrator.
- Rewrote check_freshness to pull per-city markdown excerpts as CCC evidence;
  the freshness LLM now extracts ccc_value_extracted alongside the
  classification so CCC-vs-web comparison still works without structured rows.
- Removed sql references from gap_analysis and assumptions_estimator context
  slimmers.
- Removed source_db_path/source_db_url/enable_sql from AppConfig and the
  DATABASE_URL/ENABLE_SQL env overrides; dropped update_sql_bundle on
  RunLogger.
- Kept enrichment config + benchmark_fact_judge config side-by-side in
  config.py, llm_config.yaml, and __all__.
- Rewrote tests/test_freshness.py around markdown evidence; updated the
  phase-2 freshness test to match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants