RPM checker: VERSION GUARD clause and format_for_prompt fix#288
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
zvigrinberg
left a comment
zvigrinberg
left a comment
There was a problem hiding this comment.
Hi @TamarW0 ,
Please take a look on my comments.
Thank you.
| # Emit version-in-range so the L1 agent can apply the VERSION GUARD | ||
| # and VERSION-BASED FALLBACK rules defined in the Case B thought instructions. | ||
| # Without this, the agent is told to check TARGET_IN_VULNERABLE_RANGE but never sees it. | ||
| if self.target_version_in_vulnerable_range is not None: |
There was a problem hiding this comment.
@TamarW0 What happens if self.target_version_in_vulnerable_range is None?
it doesn't include the TARGET_IN_VULNERABLE_RANGE in the formatted prompt.
There was a problem hiding this comment.
@TamarW0 - agree with zvi recommend placing UNKNOWN in that case
| @@ -512,12 +519,19 @@ | |||
| - ALL AFFECTED_FILES have been searched | |||
| - Evidence is sufficient for confident verdict | |||
|
|
|||
| **VERSION-BASED FALLBACK (when code search is inconclusive):** | |||
| **VERSION-BASED FALLBACK:** | |||
| If code searches found no fix pattern or no fix call-site verification: | |||
| - Check TARGET_IN_VULNERABLE_RANGE in VULNERABILITY_INTEL | |||
| - If TARGET_IN_VULNERABLE_RANGE: YES and no fix was verified in code: | |||
| → Conclude VULNERABLE based on version evidence | |||
| → Reason: "Target version is within affected range, and no fix was verified in code." | |||
|
|
|||
| **VERSION GUARD (when fix pattern WAS found):** | |||
| If TARGET_IN_VULNERABLE_RANGE is YES AND a fix-pattern grep match was found: | |||
| - The match may be a pre-existing similar check, not the actual CVE fix. | |||
| - Verify the match is at the EXACT function/location described in the CVE. | |||
| - If the match is ambiguous (e.g. a common pattern like a bounds check that could | |||
| pre-date the fix), conclude VULNERABLE (version-based). | |||
There was a problem hiding this comment.
@TamarW0 If TARGET_IN_VULNERABLE_RANGE field is absent in the data in both guards( because of the code that i commented on it previously) , then the LLM is being asked to lookup a field that doesn't exists might hallucinate it wrongly or fallback to some undesired behavior...
Better to explicitly put in the prompts that if field TARGET_IN_VULNERABLE_RANGE is absent from data context , then treat as unknown , or to conclude vulnerable unless fix was verified at the exact CVE location
@RedTanny WDYT?
There was a problem hiding this comment.
@zvigrinberg Agree — omitting the field when it’s None is the wrong default.
The agent is already instructed to check TARGET_IN_VULNERABLE_RANGE, so if the value isn’t in the prompt it may invent YES/NO or skip the guard incorrectly.
I’d prefer we always emit it:
YES / NO when known
TARGET_IN_VULNERABLE_RANGE: UNKNOWN when None
And update both VERSION GUARD and VERSION-BASED FALLBACK to say explicitly: if the field is UNKNOWN/absent, treat as unknown — do not assume YES/NO; conclude VULNERABLE unless the fix is verified at the exact CVE location.
That keeps the VERSION GUARD intent (avoid false PATCHED from a generic grep match) without creating a blind spot when range data is missing.
| # Emit version-in-range so the L1 agent can apply the VERSION GUARD | ||
| # and VERSION-BASED FALLBACK rules defined in the Case B thought instructions. | ||
| # Without this, the agent is told to check TARGET_IN_VULNERABLE_RANGE but never sees it. | ||
| if self.target_version_in_vulnerable_range is not None: |
There was a problem hiding this comment.
@TamarW0 - agree with zvi recommend placing UNKNOWN in that case
|
Hi @TamarW0 Please check out why 2 tests are failing. |
zvigrinberg
left a comment
zvigrinberg
left a comment
There was a problem hiding this comment.
@TamarW0 LGTM Approved.
4 participants
No description provided.