fix(opencode): distinguish prompt defense scopes - #3677
fix(opencode): distinguish prompt defense scopes#3677Kerberosmansour (kerberosmansour) wants to merge 1 commit into
Conversation
Signed-off-by: kerberosmansour <13433538+kerberosmansour@users.noreply.github.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
|
🟡 Contributor Check: MEDIUM
Automated check by AGT Contributor Check. |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Imran Siddique (imran-siddique)
left a comment
There was a problem hiding this comment.
This is a metric-honesty fix, and for a governance toolkit that is a category worth taking as seriously as a control fix. A number that flatters is worse than no number, because it terminates the operator's investigation.
The specific confusion is the one that matters: a grade computed over the fully compiled context, which already includes bundled guardrails, reads as a grade of what the operator wrote. Someone tuning additionalContext sees a good letter and stops. Reporting both, with effective-context and operator-additional-context named explicitly in the payload rather than implied by field naming, is the right shape. Keeping the existing fields untouched so consumers do not break is the right call for a change whose whole point is that a field name misled people.
One question rather than a change request, and it is for the maintainers as much as you.
promptDefenseBlockingScope: "effective-context" is an honest label, and what it documents is that minimumPromptDefenseGrade still gates on the compiled context. So an operator who sets a minimum grade intending to hold their own context to a standard is still being gated on the measurement that includes the bundled defaults. Their configured content could be empty and the gate would pass on the strength of guardrails they did not write.
Not changing enforcement inside a reporting PR is the conservative call and I agree with it. But the gap now exists in a field name and nowhere else, and field names are not a backlog. MohammadHaroonAbuomar liamcrumm worth deciding whether minimumPromptDefenseGrade should key on the configured scope, or whether a second threshold is the answer. If neither, the docs should say which scope the gate measures, because the two-scope split makes the question obvious to the next reader.
No approve bit on this repo, so a comment, but I would merge it as it stands.
Summary
Root cause
The previous grade measured the fully compiled context, which includes bundled guardrails, but its name made it easy to interpret as a grade of operator-supplied policy content.
Validation
cd agent-governance-opencode && npm run check— 26/26 tests passedCloses #3664
Tracking and related work