Skip to content

Harden and document the policy-engine contract (conformance, fail-closed, path-safety, imports, parsing)#190

Merged
dgenio merged 6 commits into
mainfrom
claude/issue-triage-grouping-s6ah9x
Jun 19, 2026
Merged

Harden and document the policy-engine contract (conformance, fail-closed, path-safety, imports, parsing)#190
dgenio merged 6 commits into
mainfrom
claude/issue-triage-grouping-s6ah9x

Conversation

@dgenio

@dgenio dgenio commented Jun 19, 2026

Copy link
Copy Markdown
Owner

Summary

This PR locks the documented policy-engine contract with conformance, edge-case, and invariant tests, plus the doc clarifications they anchor to. It is the coherent group identified during issue triage: six issues that all live in internal/policy + internal/engine + docs/, share one implementation path (pin documented behavior with additive tests + docs), and carry near-zero interaction risk.

What changed (file by file):

  • internal/engine/engine.go — path-safety now rejects Windows drive-relative paths (C:foo), not only drive-absolute (C:\…) and bare-drive (C:). Renamed isWindowsAbsPathisWindowsDrivePath (+ isASCIILetter) and made the check host-independent. (behavior change — see Tradeoffs)
  • internal/engine/conformance_test.go (new)#139: a constraint conformance matrix derived from docs/policy-language.md covering paths, args, urls, command, and memory_write.
  • internal/engine/failclosed_test.go (new)#155: regression tests for the deny-by-default invariants (no-rule→default, omitted-default→deny, missing constrained argument→deny, unknown memory scope→deny, parser rejects unknown decision, glob never panics/over-matches).
  • internal/engine/path_safety_test.go (new)#175/#170: a cross-platform Windows-input matrix and tests pinning the lexical-only / no-symlink-resolution / case-sensitive non-guarantees.
  • internal/policy/parsing_edgecases_test.go (new)#178: empty / whitespace / tab / comment-only / multi-document / duplicate-key handling.
  • internal/policy/imports_semantics_test.go (new)#142: importer-wins scalar precedence and OR-semantics for the redaction/audit enable flags.
  • docs/policy-language.md — path-safety guarantees/non-guarantees, an import merge-precedence table, and a parsing edge-cases section.
  • docs/threat-model.md — a Fail-closed invariants section and the lexical/symlink/case path-safety limits.
  • CHANGELOG.md[Unreleased] entries.

Related issues

Closes #139
Closes #142
Closes #155
Closes #170
Closes #175
Closes #178

Why

These issues were selected as the single most coherent group from a triage of the 78 open issues (no open PRs existed): all are about pinning the documented policy/engine contract to the code with additive tests and doc clarifications, in two adjacent packages, with no runtime behavior change — except one deliberate Mode-B hardening surfaced by the #175 work (the C:foo gap).

How verified

  • make ci (fmt-check + vet + go test -race with coverage) — all packages pass; internal/engine coverage 94.6%, total 81.5%.
  • go test ./internal/engine ./internal/policy -v — all new subtests run and pass (conformance matrix 24 cases, fail-closed invariants, Windows-input matrix 12 cases, parser edge cases, import semantics).
  • The C:foo drive-relative case fails against the pre-change engine and passes after the fix (test + fix are coupled).
  • Parser edge-case expectations were empirically confirmed before writing assertions (spaces/newlines/comment → empty default-deny policy; tabs/multi-doc/duplicate-key → error).

Tradeoffs / risks

  • One intentional behavior change (Mode B): drive-relative C:foo paths are now denied where they were previously allowed on non-Windows hosts. This only tightens path-safety for a security tool; documented in the CHANGELOG under Changed. No legitimate project-relative path begins with a single letter followed by :.
  • The conformance test couples to the doc examples by design; future edits to docs/policy-language.md examples must keep the matrix in sync (called out in the doc/section).
  • Everything else is additive tests + docs.

Checklist

  • Tests added or updated
  • Documentation updated if needed
  • CI passes
  • Issue number included

🤖 Generated with Claude Code


Generated by Claude Code

claude added 4 commits June 19, 2026 06:01
The path-safety pre-check denied drive-absolute (C:\...) and bare-drive
(C:) forms but treated a drive-relative path such as C:foo as an ordinary
relative name on non-Windows hosts, letting it through. Detect any
single-letter drive prefix (X:) host-independently and deny it.

Refs #175, #170
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RosRSbDXFWCMFbrWqbHRLS
Add three additive test suites that lock the documented engine contract:

- path_safety_test.go: a cross-platform Windows-input matrix (drive,
  UNC, traversal, mixed separators) plus tests pinning the lexical-only,
  no-symlink-resolution, case-sensitive non-guarantees. (#175, #170)
- failclosed_test.go: regression tests for the deny-by-default
  invariants (no-rule→default, omitted-default→deny, missing constrained
  argument→deny, unknown memory scope→deny, parser rejects unknown
  decision, glob never panics/over-matches). (#155)
- conformance_test.go: a constraint conformance matrix derived from the
  examples in docs/policy-language.md for every constraint family. (#139)

Refs #139, #155, #170, #175
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RosRSbDXFWCMFbrWqbHRLS
- parsing_edgecases_test.go: pin empty/whitespace/tab/comment-only,
  multi-document, and duplicate-key handling for policy YAML. (#178)
- imports_semantics_test.go: cover importer-wins scalar precedence and
  OR-semantics for the redaction/audit enable flags, complementing the
  existing import tests. (#142)

Refs #142, #178
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RosRSbDXFWCMFbrWqbHRLS
- policy-language.md: add path-safety guarantees/non-guarantees,
  an import merge-precedence table, and a parsing edge-cases section.
- threat-model.md: add a Fail-closed invariants section and document
  the lexical-only/symlink/case path-safety limits under "What MVP does
  not yet mitigate".
- CHANGELOG: record the contract-hardening work and the drive-relative
  path-safety change.

Refs #139, #142, #155, #170, #175, #178
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RosRSbDXFWCMFbrWqbHRLS
Copilot AI review requested due to automatic review settings June 19, 2026 06:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR formalizes and hardens the policy-engine contract by turning the documented policy semantics (constraints, imports, parsing behavior, and fail-closed invariants) into executable tests, and updating the documentation and changelog accordingly. It also tightens path-safety by denying Windows drive-relative paths (C:foo) host-independently.

Changes:

  • Add doc-derived conformance and invariant regression tests across internal/engine and internal/policy.
  • Harden path-safety to deny Windows drive-qualified paths including drive-relative forms, regardless of host OS.
  • Expand documentation and changelog to explicitly specify guarantees/non-guarantees and edge-case parsing/import semantics.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
internal/engine/engine.go Tightens Windows path-safety detection to deny drive-relative paths host-independently.
internal/engine/conformance_test.go Adds doc-derived constraint conformance matrix across constraint families.
internal/engine/failclosed_test.go Adds regression tests pinning deny-by-default / fail-closed invariants.
internal/engine/path_safety_test.go Adds OS-independent Windows-input matrix + tests documenting lexical-only limits.
internal/policy/parsing_edgecases_test.go Adds strict parsing edge-case tests (blank/comment-only, tabs, multi-doc, duplicates).
internal/policy/imports_semantics_test.go Adds tests pinning importer-wins scalars and OR semantics for security flags.
docs/policy-language.md Documents path-safety limits, import merge precedence table, and parsing edge-case contract.
docs/threat-model.md Documents fail-closed invariants and path-safety lexical/symlink/case limitations.
CHANGELOG.md Adds unreleased entries including the path-safety hardening behavior change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/policy-language.md Outdated
Comment thread internal/policy/imports_semantics_test.go Outdated
claude added 2 commits June 19, 2026 06:08
ParsePolicy applies per-file defaults (defaults.decision→deny,
audit.format→jsonl) before imports are merged, so omitting those in the
importing file resolves to the default and overrides the base rather than
inheriting it. Only `version` (no per-file default) truly inherits when
omitted. Fix the merge-precedence table to state this and replace the
misleading "inherited from base" test assertion with one that pins the
actual contract.

Addresses Copilot review feedback on #190.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RosRSbDXFWCMFbrWqbHRLS
Audit follow-up to PR #190.

- docs/threat-model.md: the "Fail-closed invariants" list enumerated the
  engine/parser invariants but omitted two that issue #155 explicitly names
  and that already have regression tests: audit sequence/chain state advances
  only after a successful write (TestWriteFailureDoesNotAdvanceSequence/
  ChainState) and the deny-all approver default for ask decisions
  (TestHTTPAskDeniedByDefaultApprover). Added both with file references so the
  documented list matches #155's "enumerate each invariant" acceptance.
- internal/engine/conformance_test.go: assert a reason substring alongside the
  decision for each case (#139 execution note), so a decision-right/
  reason-wrong regression is caught. Reasons were captured empirically; the
  "rm -rf /" case is denied by the executable allowlist (not the deny pattern),
  so its assertion and name reflect the real reason.

make ci passes (vet, go test -race); engine coverage 94.6%, total 81.5%.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016T54tPTSQ2XRYmpiVhJzn1
@dgenio dgenio merged commit fc87a13 into main Jun 19, 2026
4 checks passed
@dgenio dgenio deleted the claude/issue-triage-grouping-s6ah9x branch June 19, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment