Skip to content

fix: persist feedback model and ruleset versions#47

Merged
sapnilbiswas merged 4 commits into
Stanzin7:masterfrom
Th-Shivam:fix/feedback-metadata-versions
Apr 24, 2026
Merged

fix: persist feedback model and ruleset versions#47
sapnilbiswas merged 4 commits into
Stanzin7:masterfrom
Th-Shivam:fix/feedback-metadata-versions

Conversation

@Th-Shivam
Copy link
Copy Markdown
Contributor

@Th-Shivam Th-Shivam commented Mar 30, 2026

Closes #44

Summary by CodeRabbit

  • Bug Fixes

    • Fixed version tracking in scan feedback: model and ruleset versions are now reliably extracted and stored from scan results.
    • Normalized executive summary storage so summary metadata (including ruleset version) is consistently populated.
    • Treats anonymous feedback as unauthenticated (user cleared when anonymous).
  • Improvements

    • Summary generation now surfaces the resolved model version in executive summaries.
  • Tests

    • Added endpoint tests to verify feedback persistence, version resolution, nested metadata handling, and anonymous-user normalization.

Copilot AI review requested due to automatic review settings March 30, 2026 12:59
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ensures scan feedback records include the associated model_version and ruleset_version so feedback can be audited and analyzed against the exact scan context (closes #44).

Changes:

  • Add safe extraction/resolution helpers to pull model_version/ruleset_version from cached or persisted scan payloads before saving feedback.
  • Populate ruleset_version into the stored scan “summary” payload and set a default model_version in generated executive summaries.
  • Add API tests asserting feedback persistence of version fields and safe handling when metadata is missing.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
tests/api/test_feedback_endpoint.py Adds endpoint tests verifying save_feedback() receives extracted model_version/ruleset_version and handles missing metadata.
src/extension_shield/core/summary_generator.py Adds model_version defaulting in the executive summary payload.
src/extension_shield/api/main.py Adds version extraction helpers, sets ruleset_version on scan summary payloads, and persists versions on feedback save.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

)


summary.setdefault("model_version", model_name)
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

summary.setdefault("model_version", model_name) may be misleading for auditability because invoke_with_fallback() can map the requested model_name to a provider-specific model name (and may fall back across providers). If you intend model_version to represent the actual model used, consider plumbing the resolved provider/model back from invoke_with_fallback (or otherwise recording it) rather than the requested env value.

Suggested change
summary.setdefault("model_version", model_name)
# Record the actual model used, if provided by the client response
resolved_model = getattr(response, "model", None) or getattr(response, "model_name", None)
if resolved_model:
summary.setdefault("model_version", resolved_model)

Copilot uses AI. Check for mistakes.
ruleset_version="v1",
)


Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new feedback version behavior is only tested with payloads that have model_version in summary and scoring_v2 at the top level. In the SQLite backend, scoring_v2/report_view_model are commonly embedded under the summary JSON (and legacy rows may only have versions under summary["report_view_model"]["meta"]). Adding a test that exercises the DB payload shape (nested summary fields) would help ensure versions are persisted correctly across backends and older scan rows.

Suggested change
def test_feedback_persists_versions_from_nested_summary_payload(
client: TestClient,
) -> None:
scan_id = "nestedabcdefghijnestedabcdefghij"
scan_results.pop(scan_id, None)
# Simulate a DB-style payload where scoring_v2 is embedded under summary
scan_payload = {
"summary": {
"model_version": "gpt-4.1-mini",
"scoring_v2": {
"scoring_version": "2.1.0",
"weights_version": "rules-v3",
},
"report_view_model": {
"meta": {
"model_version": "legacy-model",
"ruleset_version": "legacy-rules",
}
},
}
}
with patch("extension_shield.api.main.db") as mock_db:
mock_db.get_scan_result = MagicMock(return_value=scan_payload)
mock_db.save_feedback = MagicMock()
response = client.post(
"/api/feedback",
json={
"scan_id": scan_id,
"helpful": False,
"reason": "score_off",
"suggested_score": 85,
"comment": "Nested summary payload",
},
)
assert response.status_code == 200
mock_db.save_feedback.assert_called_once_with(
scan_id=scan_id,
helpful=False,
reason="score_off",
suggested_score=85,
comment="Nested summary payload",
user_id=None,
model_version="gpt-4.1-mini",
ruleset_version="rules-v3",
)

Copilot uses AI. Check for mistakes.
Comment thread src/extension_shield/api/main.py Outdated

metadata = _coerce_json_dict(payload.get("metadata"))
summary = _coerce_json_dict(payload.get("summary"))
report_view_model = _coerce_json_dict(payload.get("report_view_model"))
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_extract_feedback_versions() looks for report_view_model/meta only at the top level (payload["report_view_model"]). For the SQLite backend, report_view_model is persisted inside the summary JSON (see Database.save_scan_result storing report_view_model under summary), so report_meta will always be empty when feedback resolves via db.get_scan_result(). Consider also checking summary.get("report_view_model") (and then its meta) so version extraction works consistently across backends/legacy rows.

Suggested change
report_view_model = _coerce_json_dict(payload.get("report_view_model"))
report_view_model_source = payload.get("report_view_model")
if not report_view_model_source:
report_view_model_source = summary.get("report_view_model")
report_view_model = _coerce_json_dict(report_view_model_source)

Copilot uses AI. Check for mistakes.
@Stanzin7
Copy link
Copy Markdown
Owner

Stanzin7 commented Apr 5, 2026

@Th-Shivam can you make sure all the checks are passing and resubmit, thanks

@Stanzin7 Stanzin7 added help wanted Extra attention is needed good first issue Good for newcomers labels Apr 6, 2026
@github-actions github-actions Bot added the area: backend Changes to the Python backend and scanning pipeline label Apr 8, 2026
@sapnilbiswas
Copy link
Copy Markdown
Collaborator

@Th-Shivam Any updates?

@Th-Shivam
Copy link
Copy Markdown
Contributor Author

I am really very sorry im in the middle of my term end examinations , i will solve these issues once my exams are over , most probably on 29th i will fix and raise the pr .
Thanks for understanding . @sapnilbiswas

@sapnilbiswas
Copy link
Copy Markdown
Collaborator

@Th-Shivam That's completely fine; however, I don't think this needs much time. You can just run. npm audit --fix to get it fixed

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 24, 2026

📝 Walkthrough

Walkthrough

Adds helpers to extract model_version and ruleset_version from scan payloads, normalizes executive summary structure on scan completion, and persists resolved version metadata with submitted feedback; also augments summary generation to inject a resolved model identifier into executive summaries and adds tests for feedback endpoint behavior.

Changes

Cohort / File(s) Summary
API Feedback & Payload Normalization
src/extension_shield/api/main.py
Adds helpers to coerce JSON-like fields and to resolve model_version/ruleset_version from scan payloads (searching metadata, summary, report_view_model.meta, scoring_v2). Normalizes executive_summary to a dict on scan completion and writes resolved ruleset_version into the payload. Updates submit_feedback() to treat "anon" as unauthenticated, resolve versions via _resolve_feedback_versions(feedback.scan_id), and persist them with feedback instead of None.
Summary Generation: model resolution
src/extension_shield/core/summary_generator.py
Adds best-effort extraction of a resolved model identifier from raw LLM responses and injects it as model_version into executive summaries across success and fallback code paths; introduces small helpers to augment summary shape consistently.
Endpoint Tests for Feedback
tests/api/test_feedback_endpoint.py
New test module (4 tests) that patches extension_shield.api.main.db to control get_scan_result and capture save_feedback calls. Tests verify resolved model_version/ruleset_version persistence from various payload shapes and anonymous user normalization (user_id=None).

Sequence Diagram(s)

sequenceDiagram
  participant Client as Client
  participant API as API (/api/feedback)
  participant ScanStore as Scan Store (get_scan_result)
  participant DB as DB (save_feedback)

  Client->>API: POST /api/feedback {scan_id, feedback, user:"anon" / user_id}
  API->>ScanStore: get_scan_result(scan_id)
  ScanStore-->>API: scan_payload (summary, metadata, scoring_v2...)
  API->>API: _coerce_json_fields() / _resolve_feedback_versions()
  API->>DB: save_feedback({..., model_version, ruleset_version, user_id=None})
  DB-->>API: saved
  API-->>Client: 200 OK
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 I hopped through payloads, sniffed each nest,
Found model and ruleset where they rest,
No more None left to baffle the night,
Feedback now stores the versions just right.
A tiny thump — audit trails take flight! 🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix: persist feedback model and ruleset versions' accurately describes the main change in the pull request, which focuses on persisting model and ruleset version metadata in feedback records.
Linked Issues check ✅ Passed The pull request successfully addresses all acceptance criteria from issue #44: model_version and ruleset_version are extracted from scan metadata, missing metadata is handled safely, and comprehensive tests verify the new behavior.
Out of Scope Changes check ✅ Passed All changes are directly aligned with issue #44 requirements: helper functions for version extraction, feedback payload updates, version resolution logic, and test coverage for the new behavior.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added bug Bug report or bug fix related work area: frontend Changes to the React frontend area: tests Changes to test coverage or test infrastructure and removed bug Bug report or bug fix related work labels Apr 24, 2026
Copilot AI review requested due to automatic review settings April 24, 2026 17:09
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot removed the area: frontend Changes to the React frontend label Apr 24, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/extension_shield/core/summary_generator.py (1)

499-613: ⚠️ Potential issue | 🟠 Major

Don't drop model_version on post-invoke failures.

Once invoke_with_fallback() succeeds, a parse/validation exception still falls through to return None. The scan payload then gets an empty summary, so feedback loses model_version even though it was already resolved from the response. Prefer returning a deterministic fallback summary with the attached version in this branch too.

Suggested fix
-        try:
+        resolved_model_version = None
+        try:
             # Format prompt to messages
             formatted_prompt = prompt.format_prompt()
             messages = formatted_prompt.to_messages()

             # Invoke with fallback
             response = invoke_with_fallback(
                 messages=messages,
                 model_name=model_name,
                 model_parameters=model_parameters,
             )
             resolved_model_version = _extract_response_model_version(response)
@@
         except Exception as exc:
             # Avoid noisy stack traces in normal operation; callers can decide how to handle None.
             logger.warning("Failed to generate executive summary: %s", exc)
-            return None
+            if facts_score is not None and facts_score_label is not None:
+                from extension_shield.core.report_view_model import _fallback_executive_summary
+                return _attach_model_version(
+                    _fallback_executive_summary(
+                        score=facts_score,
+                        score_label=facts_score_label,
+                        host_scope_label=facts_host_scope_label or "UNKNOWN",
+                    ),
+                    resolved_model_version,
+                )
+            return {"model_version": resolved_model_version} if resolved_model_version else None
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/extension_shield/core/summary_generator.py` around lines 499 - 613,
Initialize resolved_model_version = None before the try block, then after
invoke_with_fallback keep assigning resolved_model_version as you already do; in
the except Exception handler, if resolved_model_version is not None import and
return the deterministic fallback via
_attach_model_version(_fallback_executive_summary(...), resolved_model_version)
(use the same score/score_label/host_scope_label logic you use earlier or pass
defaults), otherwise keep the current logger.warning and return None;
references: invoke_with_fallback, resolved_model_version, _attach_model_version,
_fallback_executive_summary.
🧹 Nitpick comments (2)
tests/api/test_feedback_endpoint.py (1)

14-126: Add one cache-hit case for scan_results too.

These tests only cover the DB fallback, but _resolve_feedback_versions() checks the in-memory scan_results cache first. A small happy-path case there would lock down the primary lookup path as well.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/api/test_feedback_endpoint.py` around lines 14 - 126, Add a new unit
test that exercises the in-memory cache path by populating the global
scan_results dict for a test scan_id with a payload containing model_version and
ruleset_version, then POST to /api/feedback and assert mock_db.save_feedback is
called with those versions; ensure you reference the in-memory cache symbol
scan_results and the feedback resolver (_resolve_feedback_versions) behavior,
and optionally assert that mock_db.get_scan_result was not called to confirm the
cache hit.
src/extension_shield/api/main.py (1)

1566-1574: Reuse the normalized summary when building report_view_model.

This block adds ruleset_version to executive_summary, but the report_view_model built below still uses final_state.get("executive_summary"). That leaves the two payload shapes out of sync for the same scan.

Suggested fix
-                "report_view_model": build_report_view_model_safe(
-                    manifest=manifest,
-                    analysis_results={**analysis_results, "executive_summary": final_state.get("executive_summary") or {}},
+                "report_view_model": build_report_view_model_safe(
+                    manifest=manifest,
+                    analysis_results={**analysis_results, "executive_summary": executive_summary},
                     metadata=metadata,
                     extension_id=extension_id,
                     scan_id=extension_id,
                 ),
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/extension_shield/api/main.py` around lines 1566 - 1574, The
executive_summary dict is normalized and updated with ruleset_version but the
later report_view_model still reads final_state.get("executive_summary"),
causing inconsistent payloads; update the code to reuse the normalized
executive_summary when building report_view_model (either assign
final_state["executive_summary"]=executive_summary after normalization or have
report_view_model reference the local executive_summary variable) so both places
use the same mutated object (refer to the executive_summary variable and the
report_view_model construction).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/extension_shield/api/main.py`:
- Around line 1036-1043: _resolve_feedback_versions currently only checks the
in-memory scan_results and db.get_scan_result, so file-backed scan payloads
loaded by get_scan_results(scan_id) are ignored; update
_resolve_feedback_versions to call get_scan_results(scan_id) as a final fallback
when payload is not found in scan_results and db.get_scan_result returns falsy,
then pass that payload into _extract_feedback_versions (handle None safely).
Refer to symbols: _resolve_feedback_versions, scan_results, db.get_scan_result,
get_scan_results, and _extract_feedback_versions.

In `@src/extension_shield/core/summary_generator.py`:
- Around line 64-69: The function _attach_model_version currently uses
normalized.setdefault("model_version", model_version) which preserves any
existing model_version in the parsed summary; instead, make the
transport/client-provided model_version authoritative by assigning it directly.
In _attach_model_version, keep the shallow-copy behavior (normalized =
dict(summary) if isinstance(summary, dict) else {}), then replace the setdefault
call with a direct assignment normalized["model_version"] = model_version when
model_version is truthy so the returned summary always reflects the resolved
model_version from the client.

In `@tests/api/test_feedback_endpoint.py`:
- Around line 39-47: The test assertions use the wrong anonymous user_id;
submit_feedback() calls _get_user_id() which returns "anon" for unauthenticated
requests, so update the mock_db.save_feedback assertions in these tests
(referencing the mock_db.save_feedback call in
tests/api/test_feedback_endpoint.py) to expect user_id="anon" (or alternatively
stub/authenticate _get_user_id in the test setup) so the asserted arguments
match submit_feedback() behavior.

---

Outside diff comments:
In `@src/extension_shield/core/summary_generator.py`:
- Around line 499-613: Initialize resolved_model_version = None before the try
block, then after invoke_with_fallback keep assigning resolved_model_version as
you already do; in the except Exception handler, if resolved_model_version is
not None import and return the deterministic fallback via
_attach_model_version(_fallback_executive_summary(...), resolved_model_version)
(use the same score/score_label/host_scope_label logic you use earlier or pass
defaults), otherwise keep the current logger.warning and return None;
references: invoke_with_fallback, resolved_model_version, _attach_model_version,
_fallback_executive_summary.

---

Nitpick comments:
In `@src/extension_shield/api/main.py`:
- Around line 1566-1574: The executive_summary dict is normalized and updated
with ruleset_version but the later report_view_model still reads
final_state.get("executive_summary"), causing inconsistent payloads; update the
code to reuse the normalized executive_summary when building report_view_model
(either assign final_state["executive_summary"]=executive_summary after
normalization or have report_view_model reference the local executive_summary
variable) so both places use the same mutated object (refer to the
executive_summary variable and the report_view_model construction).

In `@tests/api/test_feedback_endpoint.py`:
- Around line 14-126: Add a new unit test that exercises the in-memory cache
path by populating the global scan_results dict for a test scan_id with a
payload containing model_version and ruleset_version, then POST to /api/feedback
and assert mock_db.save_feedback is called with those versions; ensure you
reference the in-memory cache symbol scan_results and the feedback resolver
(_resolve_feedback_versions) behavior, and optionally assert that
mock_db.get_scan_result was not called to confirm the cache hit.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 870aa146-113a-4dac-90a5-e09589d5965c

📥 Commits

Reviewing files that changed from the base of the PR and between 2149cba and 2afbcc9.

⛔ Files ignored due to path filters (1)
  • frontend/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (3)
  • src/extension_shield/api/main.py
  • src/extension_shield/core/summary_generator.py
  • tests/api/test_feedback_endpoint.py

Comment on lines +1036 to +1043
def _resolve_feedback_versions(scan_id: str) -> tuple[Optional[str], Optional[str]]:
"""Look up the related scan payload and extract version metadata safely."""
payload = scan_results.get(scan_id)
if isinstance(payload, dict):
return _extract_feedback_versions(payload)

db_payload = db.get_scan_result(scan_id)
return _extract_feedback_versions(db_payload)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Include the existing file-backed scan fallback here too.

get_scan_results() can still load <scan_id>_results.json when the DB row is missing, but _resolve_feedback_versions() stops after memory/DB. That means feedback for file-only scans regresses to None again even though the metadata is still available.

Suggested fix
 def _resolve_feedback_versions(scan_id: str) -> tuple[Optional[str], Optional[str]]:
     """Look up the related scan payload and extract version metadata safely."""
     payload = scan_results.get(scan_id)
     if isinstance(payload, dict):
         return _extract_feedback_versions(payload)

-    db_payload = db.get_scan_result(scan_id)
-    return _extract_feedback_versions(db_payload)
+    try:
+        db_payload = db.get_scan_result(scan_id)
+        if isinstance(db_payload, dict):
+            return _extract_feedback_versions(db_payload)
+    except Exception as exc:
+        logger.debug("[FEEDBACK] Failed DB lookup for %s: %s", scan_id, exc)
+
+    result_file = RESULTS_DIR / f"{scan_id}_results.json"
+    if result_file.exists():
+        try:
+            with open(result_file, "r", encoding="utf-8") as f:
+                return _extract_feedback_versions(json.load(f))
+        except (OSError, json.JSONDecodeError) as exc:
+            logger.debug("[FEEDBACK] Failed file lookup for %s: %s", scan_id, exc)
+
+    return None, None
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/extension_shield/api/main.py` around lines 1036 - 1043,
_resolve_feedback_versions currently only checks the in-memory scan_results and
db.get_scan_result, so file-backed scan payloads loaded by
get_scan_results(scan_id) are ignored; update _resolve_feedback_versions to call
get_scan_results(scan_id) as a final fallback when payload is not found in
scan_results and db.get_scan_result returns falsy, then pass that payload into
_extract_feedback_versions (handle None safely). Refer to symbols:
_resolve_feedback_versions, scan_results, db.get_scan_result, get_scan_results,
and _extract_feedback_versions.

Comment thread src/extension_shield/core/summary_generator.py
Comment on lines +39 to +47
mock_db.save_feedback.assert_called_once_with(
scan_id=scan_id,
helpful=False,
reason="score_off",
suggested_score=72,
comment="Score feels too strict",
user_id=None,
model_version="gpt-4o",
ruleset_version="v1",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

These assertions expect the wrong anonymous user_id.

submit_feedback() calls _get_user_id(), and unauthenticated requests currently resolve to "anon", not None. Without stubbing auth or sending a user header, this assertion pattern will fail here and in the two tests below.

Suggested fix
-        user_id=None,
+        user_id="anon",
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
mock_db.save_feedback.assert_called_once_with(
scan_id=scan_id,
helpful=False,
reason="score_off",
suggested_score=72,
comment="Score feels too strict",
user_id=None,
model_version="gpt-4o",
ruleset_version="v1",
mock_db.save_feedback.assert_called_once_with(
scan_id=scan_id,
helpful=False,
reason="score_off",
suggested_score=72,
comment="Score feels too strict",
user_id="anon",
model_version="gpt-4o",
ruleset_version="v1",
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/api/test_feedback_endpoint.py` around lines 39 - 47, The test
assertions use the wrong anonymous user_id; submit_feedback() calls
_get_user_id() which returns "anon" for unauthenticated requests, so update the
mock_db.save_feedback assertions in these tests (referencing the
mock_db.save_feedback call in tests/api/test_feedback_endpoint.py) to expect
user_id="anon" (or alternatively stub/authenticate _get_user_id in the test
setup) so the asserted arguments match submit_feedback() behavior.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
src/extension_shield/api/main.py (1)

1036-1043: ⚠️ Potential issue | 🟠 Major

Keep the file fallback and swallow lookup failures here.

Line 1042 still falls straight through to db.get_scan_result(scan_id). If the scan only exists as <scan_id>_results.json, feedback will still persist None versions, and a DB lookup exception can now fail /api/feedback outright instead of degrading gracefully.

Patch sketch
 def _resolve_feedback_versions(scan_id: str) -> tuple[Optional[str], Optional[str]]:
     """Look up the related scan payload and extract version metadata safely."""
     payload = scan_results.get(scan_id)
     if isinstance(payload, dict):
         return _extract_feedback_versions(payload)

-    db_payload = db.get_scan_result(scan_id)
-    return _extract_feedback_versions(db_payload)
+    try:
+        db_payload = db.get_scan_result(scan_id)
+        if isinstance(db_payload, dict):
+            return _extract_feedback_versions(db_payload)
+    except Exception as exc:
+        logger.debug("[FEEDBACK] Failed DB lookup for %s: %s", scan_id, exc)
+
+    result_file = RESULTS_DIR / f"{scan_id}_results.json"
+    if result_file.exists():
+        try:
+            with open(result_file, "r", encoding="utf-8") as f:
+                return _extract_feedback_versions(json.load(f))
+        except (OSError, json.JSONDecodeError) as exc:
+            logger.debug("[FEEDBACK] Failed file lookup for %s: %s", scan_id, exc)
+
+    return None, None
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/extension_shield/api/main.py` around lines 1036 - 1043, The current
_resolve_feedback_versions falls through to db.get_scan_result(scan_id) without
handling DB lookup failures or missing DB rows; wrap the DB call in a try/except
and also treat a falsy db result as a fallback to the in-memory payload: keep
the initial payload = scan_results.get(scan_id) check and return
_extract_feedback_versions(payload) if it's a dict, otherwise try to get
db_payload = db.get_scan_result(scan_id) inside a try block, and if that call
raises or returns None/false, return _extract_feedback_versions(payload) (or
_extract_feedback_versions(None) if payload is absent); reference symbols:
_resolve_feedback_versions, scan_results, db.get_scan_result, and
_extract_feedback_versions.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/extension_shield/api/main.py`:
- Around line 1566-1574: The current use of
executive_summary.setdefault("ruleset_version", ...) will leave an upstream None
or empty-string in place; change the logic in the block that builds
executive_summary so you explicitly check for a falsy value (e.g., if not
executive_summary.get("ruleset_version")) and then assign
executive_summary["ruleset_version"] = scoring_v2_payload.get("weights_version")
or scoring_v2_payload.get("scoring_version"). Update the code around the
executive_summary creation (the executive_summary variable and the setdefault
call) to use this truthy check so a None/"" is overwritten by the resolved
scoring_v2_payload value.

---

Duplicate comments:
In `@src/extension_shield/api/main.py`:
- Around line 1036-1043: The current _resolve_feedback_versions falls through to
db.get_scan_result(scan_id) without handling DB lookup failures or missing DB
rows; wrap the DB call in a try/except and also treat a falsy db result as a
fallback to the in-memory payload: keep the initial payload =
scan_results.get(scan_id) check and return _extract_feedback_versions(payload)
if it's a dict, otherwise try to get db_payload = db.get_scan_result(scan_id)
inside a try block, and if that call raises or returns None/false, return
_extract_feedback_versions(payload) (or _extract_feedback_versions(None) if
payload is absent); reference symbols: _resolve_feedback_versions, scan_results,
db.get_scan_result, and _extract_feedback_versions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 521f30e9-76ae-437d-9e56-b6394b281bc8

📥 Commits

Reviewing files that changed from the base of the PR and between 2afbcc9 and bf8597b.

📒 Files selected for processing (3)
  • src/extension_shield/api/main.py
  • src/extension_shield/core/summary_generator.py
  • tests/api/test_feedback_endpoint.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/extension_shield/core/summary_generator.py
  • tests/api/test_feedback_endpoint.py

Comment on lines +1566 to +1574
executive_summary = final_state.get("executive_summary") or {}
if isinstance(executive_summary, dict):
executive_summary = dict(executive_summary)
else:
executive_summary = {}
executive_summary.setdefault(
"ruleset_version",
scoring_v2_payload.get("weights_version") or scoring_v2_payload.get("scoring_version"),
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

setdefault() leaves null/blank ruleset_version untouched.

If upstream already includes ruleset_version: None or "", Line 1571 treats that as present and persists the empty value even though scoring_v2_payload already resolved the version. A truthy check is safer here.

Patch sketch
-            executive_summary.setdefault(
-                "ruleset_version",
-                scoring_v2_payload.get("weights_version") or scoring_v2_payload.get("scoring_version"),
-            )
+            resolved_ruleset_version = (
+                scoring_v2_payload.get("weights_version")
+                or scoring_v2_payload.get("scoring_version")
+            )
+            if resolved_ruleset_version and not executive_summary.get("ruleset_version"):
+                executive_summary["ruleset_version"] = resolved_ruleset_version
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/extension_shield/api/main.py` around lines 1566 - 1574, The current use
of executive_summary.setdefault("ruleset_version", ...) will leave an upstream
None or empty-string in place; change the logic in the block that builds
executive_summary so you explicitly check for a falsy value (e.g., if not
executive_summary.get("ruleset_version")) and then assign
executive_summary["ruleset_version"] = scoring_v2_payload.get("weights_version")
or scoring_v2_payload.get("scoring_version"). Update the code around the
executive_summary creation (the executive_summary variable and the setdefault
call) to use this truthy check so a None/"" is overwritten by the resolved
scoring_v2_payload value.

@sapnilbiswas
Copy link
Copy Markdown
Collaborator

@Th-Shivam Thanks for taking the initiative, I have fixed this on your behalf

@sapnilbiswas sapnilbiswas merged commit 84f335c into Stanzin7:master Apr 24, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: backend Changes to the Python backend and scanning pipeline area: tests Changes to test coverage or test infrastructure bug Bug report or bug fix related work good first issue Good for newcomers help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persist model_version and ruleset_version in feedback records

4 participants