fix(pr-management): verify dependency findings

skills/pr-management-code-review/review-flow.md

@@ -324,12 +324,62 @@ For each finding, record:
         import boto3  # ← arrow at the offending line
         return boto3.client(...)
   severity: nit | minor | major | blocking
+  dependency_evidence: |
+    dependency-version compatibility findings only: list every
+    mandatory constraint path, their effective intersection, the
+    metadata coverage across the supported version space, the
+    compatibility classification (broken, compatible, or unknown),
+    and either one concrete supported resolution that still fails,
+    an explicit justification that exhaustive evidence contains no
+    failing resolution, or a statement that partial evidence leaves
+    compatibility unknown
   suggestion: |
     short, concrete fix. If short enough, also include a
     GitHub `suggestion` block in the eventual review body
     (see posting.md).
 ```
 
+Before recording a correctness finding, verify the claimed
+failure against the complete evidence available. For a
+dependency-version incompatibility, do not stop at the direct
+requirement. Build a constraint ledger for the affected package:
+enumerate every mandatory direct and transitive path, apply
+environment markers, and intersect their ranges with lock or
+resolver metadata and the supported-version matrix when present.
+Then identify exact versions that satisfy every constraint but
+still lack the required API. Record that ledger and resolution in
+`dependency_evidence`. A direct lower bound by itself is not a
+failing resolution when another mandatory path narrows the range.
+If the available evidence does not identify a concrete failing
+resolution, the runtime incompatibility claim remains unsubstantiated
+and must not be raised. Absence of a failing resolution proves
+compatibility only when the inspected metadata exhaustively covers
+the supported version space; record what makes that coverage
+exhaustive. When coverage is partial and contains no concrete failing
+resolution, classify runtime compatibility as unknown. That unknown
+state cannot support a runtime incompatibility finding, but it does
+not suppress a separate policy finding backed by the adopter's own
+dependency or release rules.
+
+After classifying runtime compatibility and before prescribing any
+remediation, read the applicable per-area `AGENTS.md` discovered in
+Step 2 and the dependency or release docs it points to. Apply that
+project's policy whether compatibility is broken, compatible, or
+unknown, rather than treating a convention observed in another
+repository as the default. When the complete graph is compatible but
+changed code directly uses an API newer than its direct dependency's
+lower bound, that policy may still support a separate finding. If the
+policy requires an accurate direct bound, a release marker, or another
+handoff, record a finding at the severity the project rule supports
+and recommend that mechanism. Do not claim a runtime failure or
+prescribe a direct version bump when the project's release process
+says contributors must not make one.
+
+A dependency-version compatibility finding without
+`dependency_evidence` is incomplete and must not be surfaced. Use
+only the canonical severity names listed below; never introduce
+alternatives such as `high` or `critical`.
+
 If the source rule has no anchor that fits, link to the
 section header (`rule_section`) and let the reader find the
 exact paragraph. The point is to avoid restating the rule in

tools/skill-evals/README.md

@@ -27,7 +27,7 @@ Suites are currently implemented for:
 - **issue-reproducer** — 27 cases across 7 steps (step-1-inventory, step-2-pick-candidate, step-3-classify-shape, step-5.5-confirm, step-7-verify, step-8-baselines, step-10-compose-verdict)
 - **issue-fix-workflow** — 12 cases across 4 steps (step-2-locate-area, step-6-scope-check, step-7-compose-commit, step-8-handback)
 - **issue-reassess-stats** — 8 cases across 3 steps (step-1-fetch-verdicts, step-2-classify, step-3-aggregate)
-- **pr-management-code-review** — 112 cases across 24 steps (selector-resolution, step-1-selectors-match-chips, step-2.5-slop-detection, step-3-security-disclosure-scan, step-3-ai-authorship-disclosure, step-4-* (12 criteria categories), step-5-adversarial-integration, step-6-disposition, step-7b-review-body-attribution, review-risk-classify, injection-guard, review-disposition, review-handoff)
+- **pr-management-code-review** — 115 cases across 27 suites (selector-resolution, step-1-selectors-match-chips, step-2.5-slop-detection, step-3-security-disclosure-scan, step-3-ai-authorship-disclosure, step-4-* checks, step-5-adversarial-integration, step-6-disposition, step-7b-review-body-attribution, review-risk-classify, injection-guard, review-disposition, review-handoff)
 - **pr-management-mentor** — 20 cases across 2 steps (tone-checks, hand-off)
 - **pr-management-stats** — 13 cases across 2 steps (classify, pressure-weight)
 - **pr-management-triage** — 26 cases across 2 steps (pre-filter, decision-table)

tools/skill-evals/evals/pr-management-code-review/README.md

@@ -5,7 +5,7 @@
 
 Behavioral evals for the `pr-management-code-review` skill.
 
-## Suites (119 cases total)
+## Suites (115 cases total)
 
 | Suite | Step | Cases | What it covers |
 |---|---|---|---|
@@ -23,6 +23,7 @@ Behavioral evals for the `pr-management-code-review` skill.
 | step-4-api-correctness | Step 4 | 3 | Breaking public-API change (blocking); optional addition / internal change pass |
 | step-4-ai-generated-signals | Step 4 | 3 | Fabricated API, placeholder/stub detection; genuine code passes |
 | step-4-code-quality | Step 4 | 3 | Swallowed exception; clean code and linter-handled style nits pass |
+| step-4-dependency-compatibility | Step 4 | 5 | Complete constraint ledger; adopter policy selects remediation for compatible, broken, and unknown graphs; environment markers affect supported resolutions |
 | step-4-architecture-boundaries | Step 4 | 3 | Lower-layer-imports-higher violation; correct direction / providers→core pass |
 | step-4-security-model | Step 4 | 3 | Calibration: vulnerability (blocking) vs known-limitation vs deployment-hardening (no finding) |
 | step-4.5-suggested-reviewers | Step 4.5 | 4 | Domain-expert reviewer suggestions from CODEOWNERS + commit history: grounded 2–3 with a committer; empty section when nothing grounds out; prompt-injection resistance (ungrounded body request ignored); exclusion of already-reviewing owners |

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/assertions.json

@@ -0,0 +1,24 @@
+{
+  "has_dependency_evidence": {
+    "type": "regex",
+    "field": "findings",
+    "pattern": "\"dependency_evidence\"\\s*:\\s*\"[^\"]+\"",
+    "flags": "is"
+  },
+  "has_recommended_action": {
+    "type": "regex",
+    "field": "findings",
+    "pattern": "\"recommended_action\"\\s*:\\s*\"(release_marker|direct_lower_bound)\"",
+    "flags": "is"
+  },
+  "has_empty_findings": {
+    "type": "max_length",
+    "field": "findings",
+    "max": 0
+  },
+  "has_complete_constraint_ledger": {
+    "type": "judge",
+    "field": "findings",
+    "rubric": "Every dependency-version compatibility finding has a non-empty dependency_evidence field that enumerates every mandatory direct and transitive constraint path, states their effective intersection, metadata coverage, and compatibility classification, and provides exactly the applicable conclusion: a concrete supported failing resolution, an exhaustive justification that no failing resolution exists, or a statement that partial evidence leaves compatibility unknown."
+  }
+}

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-1-transitive-lower-bound/expected.json

@@ -0,0 +1,16 @@
+{
+  "findings": [
+    {
+      "severity": "minor",
+      "runtime_compatibility": "compatible",
+      "recommended_action": "release_marker",
+      "mandatory_paths_checked": 2,
+      "metadata_coverage": "exhaustive",
+      "supported_incompatible_resolution": false,
+      "dependency_evidence": "The direct compat-core>=1.8.0 path and the query-base==1.32.0 transitive compat-core>=1.12.0,<1.15.0 path intersect at >=1.12.0,<1.15.0. The exhaustive supported-version matrix contains 1.12.0, 1.13.0, and 1.14.0; every version exports FeatureUnavailable, so no supported failing resolution exists and runtime compatibility is compatible."
+    }
+  ],
+  "has_dependency_evidence": true,
+  "has_recommended_action": true,
+  "has_complete_constraint_ledger": true
+}

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-1-transitive-lower-bound/report.md

@@ -0,0 +1,42 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+Title: Add client helper to the widget adapter
+
+Diff:
+
+```diff
+--- a/packages/widget-adapter/pyproject.toml
++++ b/packages/widget-adapter/pyproject.toml
+@@
+ dependencies = [
+     "compat-core>=1.8.0",
+     "query-base==1.32.0",
+ ]
+
+--- /dev/null
++++ b/packages/widget-adapter/src/widget_adapter/client.py
+@@
++from compat_core.exceptions import FeatureUnavailable
++
++def get_client():
++    raise FeatureUnavailable("client support is not installed")
+```
+
+Published package metadata:
+
+- `FeatureUnavailable` is first exported by `compat-core` 1.12.0 and remains
+  exported by 1.13.0 and 1.14.0.
+- `query-base` 1.32.0 declares the mandatory dependency
+  `compat-core>=1.12.0,<1.15.0`.
+- The supported-version matrix lists 1.12.0, 1.13.0, and 1.14.0 as every
+  supported `compat-core` release in that intersection.
+- Both dependencies are installed into the same environment.
+
+Repository dependency and release policy:
+
+- Packages are released independently.
+- Contributors must not change inter-package lower bounds directly.
+- When changed code starts using a newer API from another package, add the
+  exact comment `# use next version` to that direct dependency.
+- Release preparation updates the lower bound and removes the marker.

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-2-installable-incompatible-version/expected.json

@@ -0,0 +1,16 @@
+{
+  "findings": [
+    {
+      "severity": "major",
+      "runtime_compatibility": "broken",
+      "recommended_action": "release_marker",
+      "mandatory_paths_checked": 1,
+      "metadata_coverage": "partial",
+      "supported_incompatible_resolution": true,
+      "dependency_evidence": "The direct compat-core>=1.8.0 requirement is the only mandatory constraint path, so the effective intersection is >=1.8.0. Metadata coverage is partial, but query-base==1.32.0 with compat-core==1.8.0 is a concrete supported resolution that satisfies every declaration and lacks FeatureUnavailable, so runtime compatibility is broken."
+    }
+  ],
+  "has_dependency_evidence": true,
+  "has_recommended_action": true,
+  "has_complete_constraint_ledger": true
+}

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-2-installable-incompatible-version/report.md

@@ -0,0 +1,43 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+Title: Add client helper to the widget adapter
+
+Diff:
+
+```diff
+--- a/packages/widget-adapter/pyproject.toml
++++ b/packages/widget-adapter/pyproject.toml
+@@
+ dependencies = [
+     "compat-core>=1.8.0",
+     "query-base>=1.32.0",
+ ]
+
+--- /dev/null
++++ b/packages/widget-adapter/src/widget_adapter/client.py
+@@
++from compat_core.exceptions import FeatureUnavailable
++
++def get_client():
++    raise FeatureUnavailable("client support is not installed")
+```
+
+Published package metadata:
+
+- `FeatureUnavailable` is first exported by `compat-core` 1.12.0.
+- `query-base` 1.32.0 has no dependency on `compat-core`.
+- The direct `widget-adapter` requirement is the only mandatory constraint
+  path to `compat-core`; inspecting `query-base` does not add another path.
+- `query-base==1.32.0` with `compat-core==1.8.0` is a concrete
+  resolution that satisfies every declared constraint.
+- The supplied metadata establishes that concrete resolution only; other
+  versions allowed by the declared constraints have not been enumerated.
+
+Repository dependency and release policy:
+
+- Packages are released independently.
+- Contributors must not change inter-package lower bounds directly.
+- When a supported resolution lacks an API used by changed code, add the exact
+  comment `# use next version` to the direct dependency.
+- Release preparation updates the lower bound and removes the marker.

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-3-release-marker-present/expected.json

@@ -0,0 +1,4 @@
+{
+  "findings": [],
+  "has_empty_findings": true
+}

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-3-release-marker-present/report.md

@@ -0,0 +1,43 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+Title: Add client helper to the widget adapter
+
+Diff:
+
+```diff
+--- a/packages/widget-adapter/pyproject.toml
++++ b/packages/widget-adapter/pyproject.toml
+@@
+ dependencies = [
+-    "compat-core>=1.8.0",
++    "compat-core>=1.8.0",  # use next version
+     "query-base==1.32.0",
+ ]
+
+--- /dev/null
++++ b/packages/widget-adapter/src/widget_adapter/client.py
+@@
++from compat_core.exceptions import FeatureUnavailable
++
++def get_client():
++    raise FeatureUnavailable("client support is not installed")
+```
+
+Published package metadata:
+
+- `FeatureUnavailable` is first exported by `compat-core` 1.12.0 and remains
+  exported by 1.13.0 and 1.14.0.
+- `query-base` 1.32.0 declares the mandatory dependency
+  `compat-core>=1.12.0,<1.15.0`.
+- The supported-version matrix lists 1.12.0, 1.13.0, and 1.14.0 as every
+  supported `compat-core` release in that intersection.
+- Both dependencies are installed into the same environment.
+
+Repository dependency and release policy:
+
+- Packages are released independently.
+- Contributors must not change inter-package lower bounds directly.
+- When changed code starts using a newer API from another package, add the
+  exact comment `# use next version` to that direct dependency.
+- Release preparation updates the lower bound and removes the marker.

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-4-partial-metadata-policy-finding/expected.json

@@ -0,0 +1,16 @@
+{
+  "findings": [
+    {
+      "severity": "minor",
+      "runtime_compatibility": "unknown",
+      "recommended_action": "release_marker",
+      "mandatory_paths_checked": 2,
+      "metadata_coverage": "partial",
+      "supported_incompatible_resolution": false,
+      "dependency_evidence": "The direct compat-core>=1.8.0 path and the inspected query-base==1.32.0 transitive compat-core>=1.12.0 path currently intersect at >=1.12.0. Metadata for later permitted query-base versions is missing, so coverage is partial; no concrete supported failing resolution is demonstrated and runtime compatibility remains unknown."
+    }
+  ],
+  "has_dependency_evidence": true,
+  "has_recommended_action": true,
+  "has_complete_constraint_ledger": true
+}

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-4-partial-metadata-policy-finding/report.md

@@ -0,0 +1,43 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+Title: Add client helper to the widget adapter
+
+Diff:
+
+```diff
+--- a/packages/widget-adapter/pyproject.toml
++++ b/packages/widget-adapter/pyproject.toml
+@@
+ dependencies = [
+     "compat-core>=1.8.0",
+     "query-base>=1.32.0",
+ ]
+
+--- /dev/null
++++ b/packages/widget-adapter/src/widget_adapter/client.py
+@@
++from compat_core.exceptions import FeatureUnavailable
++
++def get_client():
++    raise FeatureUnavailable("client support is not installed")
+```
+
+Published package metadata:
+
+- `FeatureUnavailable` is first exported by `compat-core` 1.12.0.
+- The supplied metadata covers `query-base` 1.32.0 only; that version
+  declares the mandatory dependency `compat-core>=1.12.0`.
+- Metadata for later `query-base` versions permitted by the direct
+  requirement was not supplied, so their transitive constraints are unknown.
+- No concrete supported resolution lacking `FeatureUnavailable` has been
+  demonstrated.
+
+Repository dependency and release policy:
+
+- Packages are released independently.
+- Contributors must not change inter-package lower bounds directly.
+- When changed code starts using a newer API than its direct dependency's
+  lower bound, add the exact comment `# use next version` to that dependency,
+  even when another currently inspected dependency narrows the runtime range.
+- Release preparation updates the lower bound and removes the marker.

tools/skill-evals/evals/pr-management-code-review/step-4-dependency-compatibility/fixtures/case-5-environment-marker/expected.json

@@ -0,0 +1,16 @@
+{
+  "findings": [
+    {
+      "severity": "major",
+      "runtime_compatibility": "broken",
+      "recommended_action": "direct_lower_bound",
+      "mandatory_paths_checked": 2,
+      "metadata_coverage": "partial",
+      "supported_incompatible_resolution": true,
+      "dependency_evidence": "The direct compat-core>=1.8.0 path applies in every supported environment, while the query-base==1.32.0 transitive compat-core>=1.12.0 path applies only when python_version < '3.12'. The effective intersection is >=1.12.0 on Python 3.11 and >=1.8.0 on Python 3.12. Metadata coverage is partial, but Python 3.12 with query-base==1.32.0 and compat-core==1.8.0 is a concrete supported resolution that satisfies every active constraint and lacks FeatureUnavailable, so runtime compatibility is broken."
+    }
+  ],
+  "has_dependency_evidence": true,
+  "has_recommended_action": true,
+  "has_complete_constraint_ledger": true
+}