Skip to content

Add implementation plan for universal harness access#722

Open
ggallen wants to merge 40 commits into
fullsend-ai:mainfrom
ggallen:feat/universal-harness-access
Open

Add implementation plan for universal harness access#722
ggallen wants to merge 40 commits into
fullsend-ai:mainfrom
ggallen:feat/universal-harness-access

Conversation

@ggallen
Copy link
Copy Markdown
Contributor

@ggallen ggallen commented May 7, 2026

Proposes making harnesses and agents universally accessible via HTTP(S) URLs, absolute paths, or relative paths. This enables community sharing, decentralized evolution of agent capabilities, and runtime adaptation.

Key features:

  • URL support for declarative resources (agents, skills, policies)
  • Scripts/binaries must remain local for security
  • Content-addressed caching with integrity verification
  • SSRF protection (domain allowlists, no redirects, internal IP blocking)
  • Transitive dependency resolution
  • Stricter security scanning for remote resources (higher LLM Guard threshold)

Security concerns prominently documented:

  • TOCTOU mitigation via content-addressed cache
  • Dependency confusion prevention via explicit URLs and domain allowlists
  • Prompt injection scanning required for all remote resources
  • Access policies constrain runtime resource loading

Implementation changes detailed in docs/plans/universal-harness-access.md covering resource fetcher, dependency resolver, cache layer, audit logging, and migration path across four phases.

Additional changes:

  • README updated to add docs/plans/ section with links to universal-harness-access plan (new) and pre-existing plan files (agent-execution-environment, vertex-inference-provisioning, drift-scanner) for complete documentation index
  • Cross-references Adopt provider-backed policy composition to reduce harness policy duplication #776 (provider-backed sandbox policy composition) as complementary to harness-level composability

Status: Proposed (all design questions answered, ready for acceptance review)

@fullsend-ai-review
Copy link
Copy Markdown

fullsend-ai-review Bot commented May 7, 2026

Review: #722

Head SHA: 88c5bca
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This PR adds ADR-0037 (universal harness access via URLs and paths), a comprehensive implementation plan, and a README update indexing the docs/plans/ directory. The design is thorough and well-structured. The ADR correctly follows repo conventions (problem-oriented with multiple options and trade-offs), the security analysis covers the key threat vectors (SSRF, TOCTOU, dependency confusion, prompt injection, DNS rebinding), and the illustrative Go code demonstrates sound security patterns (content-addressed caching with re-verification, IP pinning for DNS rebinding protection, prefix matching with URL normalization). No critical or high findings.

Findings

Info

  • [Correctness] README.md:13 — The new README entry for the drift scanner describes it as "Implementation plan for ADR-0046 drift detection tool," but no docs/ADRs/0046-*.md file exists in the repository. The pre-existing plan file docs/plans/2026-03-06-adr46-drift-scanner.md references ADR-0046 in its title, but that ADR has not been written. This is a pre-existing issue propagated by this PR's README indexing, not introduced by it. Consider noting the ADR as pending or updating the description.

  • [Style/conventions] docs/plans/universal-harness-access.md:463-473 — The "minimal wrapper pattern" section describes a fullsend fetch-and-run command that downloads and executes remote scripts, then immediately notes this requires in-sandbox execution that doesn't exist yet and "should either be scoped to future in-sandbox execution work or removed until that capability exists." The document recommends removing it but keeps it. Consider moving this to an appendix or future-work section to avoid suggesting an insecure pattern as a current mitigation.

  • [Correctness] docs/plans/universal-harness-access.md — The CachePut function correctly self-documents its non-atomic write limitation (metadata and content written separately). The implementation should use atomic writes (write-to-temp + rename) as noted. This is appropriate documentation for a plan-phase artifact.

  • [Intent alignment] The PR body says "Status: Proposed (all design questions answered, ready for acceptance review)" and the ADR status is correctly set to "Proposed." The scope (new ADR + plan + README index) is appropriate for a design proposal with no linked issue.

Footer

Outcome: approve
This review applies to SHA 88c5bca93f17f270d39ac0a40307566f2235ee0f. Any push to the PR head clears this review and requires a new evaluation.

Previous run

Review: #722

Head SHA: 4abf3e7
Timestamp: 2026-05-12T18:35:01Z
Outcome: comment-only

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs and paths, plus README updates to surface the docs/plans/ section. The design is thorough, with extensive security analysis covering SSRF, TOCTOU, dependency confusion, and prompt injection threats. The decision to restrict executable resources (scripts/binaries) to local-only while allowing declarative resources (agents, skills, policies) to be URL-referenced is sound and aligns with the project's security threat model. No blocking issues found; a few minor inconsistencies between the ADR and the illustrative implementation code are noted below.

Findings

Medium

  • [correctness] docs/plans/universal-harness-access.md:1021-1027 — The DefaultPolicy in the illustrative Go code includes cdn.fullsend.ai as a default allowed domain, but the ADR's "Uniform security with user-controlled trust" section states the default configuration includes only fullsend-ai/library (a GitHub URL prefix). This creates ambiguity about what the actual shipped defaults should be. If cdn.fullsend.ai is intended as a default, it should be documented in the ADR; if not, it should be removed from the illustrative DefaultPolicy.
    Remediation: Align the ADR and implementation plan on the exact set of default allowed domains.

Low

  • [correctness] docs/plans/universal-harness-access.md:1306-1340 — The ResolveHarness function and resolveResourceWithLimits in the illustrative code include depth tracking and transitive dependency comments, but Phase 1 explicitly states "No transitive dependency resolution." Implementers could misread the illustrative code as Phase 1 scope. Consider annotating these sections with explicit phase markers (e.g., // Phase 2: transitive resolution).
    Remediation: Add phase annotations to the illustrative code, or split the resolver code into Phase 1 (single-level) and Phase 2 (transitive) sections.

  • [style] docs/plans/universal-harness-access.md:603-612 — The SSRF domain allowlist example includes cdn.fullsend.ai as an exact-match entry. Given the ADR's principle of "no special treatment for first-party resources," having a first-party CDN domain in examples could imply preferential defaults. Consider using a generic placeholder or explicitly noting this is user-configured.

Info

  • [correctness] docs/ADRs/0037-universal-harness-access.md:41 — The numbering note about ADRs 0029 and 0035 being reserved/skipped is verified as accurate against the existing ADR sequence (0028 → 0030, 0034 → 0036). Helpful context for future contributors.

  • [style] docs/plans/universal-harness-access.md:1240-1269 — The CachePut function's comment about non-atomic writes is good practice for illustrative code. The recommendation to use write-to-temp-then-rename in production is the correct pattern.

Footer

Outcome: comment-only
This review applies to SHA 4abf3e728ae857c34c631225d6e1e24cd19f9995. Any push to the PR head clears this review and requires a new evaluation.

Previous run (2)

Review: #722

Head SHA: 8778d1f
Timestamp: 2026-05-12T00:00:00Z
Outcome: comment-only

Summary

This PR adds two new design documents (ADR-0037 and an implementation plan) for universal harness access via URLs and paths, plus a README update linking existing plan docs. The documents are well-structured, thoroughly address security concerns (SSRF, TOCTOU, dependency confusion, prompt injection), and the illustrative Go code demonstrates sound defensive patterns (DNS rebinding protection, content-addressed caching, integrity verification). The change is documentation-only with no runtime impact. A few medium and low-severity observations are noted below regarding ADR numbering, code illustrations, and minor editorial items.

Findings

Medium

  • [Correctness] docs/ADRs/0037-universal-harness-access.md — ADR is numbered 0037 but ADR 0035 does not exist on the main branch (jumps from 0034 to 0036 to 0037). This may indicate a numbering gap or a collision with a concurrent PR. Verify this is the intended number to avoid conflicts.
    Remediation: Check if another PR is adding ADR 0035 concurrently, or if this should be renumbered to fill the gap per project conventions.

  • [Correctness] docs/plans/universal-harness-access.md:1076-1080 — The DialContext implementation pins to ips[0] only. The inline comment acknowledges this is simplified, but the plan text should be clearer that production code MUST iterate through all resolved IPs with fallback, since pinning to only IPv6 (or only IPv4) will cause connection failures in mixed-stack environments. As written, an implementer could miss this nuance.
    Remediation: Elevate the inline comment to a numbered requirement in the SSRF protection section, or add a TODO item in the Phase 1 deliverables.

Low

  • [Correctness] docs/plans/universal-harness-access.md:1159-1163 — The isInternalIP function parses CIDR ranges on every call. The comment notes this should be done at init time, but since this is an implementation plan (not just illustrative code), the plan should explicitly list "pre-parse CIDR ranges at package init" as a requirement to avoid performance regression in production.
    Remediation: Add to the Phase 1 implementation checklist.

  • [Style/conventions] docs/plans/universal-harness-access.md — Open questions 2, 3, and 4 are marked as resolved in ADR-0037 but still appear as open questions in the plan document with full option analysis. The note at line 1671 explains this, but the presentation is confusing — a reader might think these are still unresolved. Consider moving resolved questions to a "Resolved Questions" section or marking each one inline as "[RESOLVED in ADR-0037]".
    Remediation: Restructure open questions section to distinguish truly open items (docs: Add agent-compatible code problem document #1, docs: add agent infrastructure problem document #5) from resolved ones (Add problem areas: Tekton pipeline review, migration path, multi-tenancy #2, docs: Add codebase context problem document and trim CLAUDE.md #3, Use AI to help formalise intent after rapid local prototyping #4).

  • [Correctness] docs/plans/universal-harness-access.md:1084client.Get(rawURL) sends the original URL (with hostname) to the HTTP client, but the custom DialContext pins to a pre-resolved IP. This works because DialContext intercepts the connection, but the Host header will contain the original hostname (correct for TLS SNI). Worth a brief note in the plan confirming this is intentional for TLS certificate validation.
    Remediation: Add a one-line comment in the code illustration confirming Host header behavior is correct for SNI/TLS.

Info

  • [Intent alignment] README.md — The README update adds links to three pre-existing plan files (agent-execution-environment, vertex-inference-provisioning, drift-scanner) that are not part of the universal harness access feature. The PR body mentions this. These are documentation hygiene improvements bundled with the feature PR — acceptable but worth noting.

  • [Style/conventions] docs/ADRs/0037-universal-harness-access.md — The ADR status is "Proposed" but the PR body says "all design questions answered, ready for acceptance review." This is consistent — the ADR should remain Proposed until formally accepted.

Footer

Outcome: comment-only
This review applies to SHA 8778d1f5e45022d6f2c45ac9e6df2e21ff867c28. Any push to the PR head clears this review and requires a new evaluation.

Previous run (3)

Review: #722

Head SHA: 6fe6d2d
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs and paths. The design is thorough — it presents multiple options with clear trade-offs, addresses security concerns comprehensively (SSRF, DNS rebinding, TOCTOU, dependency confusion, prompt injection), and proposes a sensible phased implementation. The ADR numbering is correct (follows 0036), all cross-referenced documents exist, and the README update is appropriate housekeeping. A few minor informational items are noted below but none are blocking.

Findings

Medium

(none)

Low

  • [correctness] docs/ADRs/0037-universal-harness-access.md:159 — The Consequences section header reads "If Option A (URL support everywhere with security extensions) is accepted:" but the Decision section chose a hybrid of Option A + Option C. The body of the Consequences section correctly describes the hybrid decision; only the header is stale.
    Remediation: Update the header to match the actual decision, e.g., "With the hybrid approach (Option A for declarative resources, Option C restriction for executables) accepted:"

Info

  • [correctness] docs/plans/universal-harness-access.md — The .fullsend-cache/ directory introduced by this design should likely be added to .gitignore (or documented as requiring that). The plan does not mention this.
  • [style] docs/plans/universal-harness-access.md — The Go pseudocode snippets are illustrative and contain minor implementation concerns (non-atomic cache writes in CachePut, per-call net.ParseCIDR allocations in isInternalIP) that should be addressed during actual implementation. These are expected for a plan document.

Footer

Outcome: approve
This review applies to SHA 6fe6d2d0ec37788935aebef94d7334d669d92d77. Any push to the PR head clears this review and requires a new evaluation.

Previous run (4)

Review: #722

Head SHA: 74321e2
Timestamp: 2026-05-12T17:15:00Z
Outcome: comment-only

Summary

This PR adds a well-structured ADR (0037) and implementation plan for universal harness access via URLs and paths. The design is thorough, with strong security analysis covering SSRF, TOCTOU, dependency confusion, and prompt injection threats. The decision to restrict executable resources (scripts/binaries) to local-only while allowing declarative resources (agents, skills, policies) via URL is sound and aligns with the repo's threat priority model. Three medium findings relate to internal consistency between the ADR and plan documents, and two issues in the illustrative Go code that should be corrected before the code samples are used as implementation guidance.

Findings

Medium

  • [Correctness] docs/ADRs/0037-universal-harness-access.md vs docs/plans/universal-harness-access.md — The ADR contains a "Resolved design questions" section (§§1–7) that presents firm decisions for namespace governance, version resolution, signature verification, offline mode, lock file format, and URL scheme. However, the implementation plan's "Open Questions" section (§§1–5) re-presents several of the same topics (signature verification §2, namespace governance §3, version resolution §4) as still-open with multiple options. Readers following the plan will not realize these were already decided in the ADR.
    Remediation: Either remove the duplicated topics from the plan's Open Questions (replacing them with back-references to the ADR decisions), or demote the ADR section to "Recommended resolutions" if these truly remain open. The two documents should agree on what is decided vs. open.

  • [Correctness] docs/plans/universal-harness-access.md:1069-1076 — The DNS-rebinding-safe DialContext uses only ips[0] (the first resolved IP). If that IP is an IPv6 address and the network path doesn't support IPv6 (or vice versa), the connection will fail even though a valid IPv4 address exists in the resolved set. Production HTTP clients try all resolved addresses.
    Remediation: Document that the implementation should iterate through all validated IPs (trying each in order) rather than pinning to ips[0] alone. Go's net.Dialer supports this via a custom DialContext that attempts each address sequentially.

  • [Correctness] docs/plans/universal-harness-access.md:1107-1110 — The isAllowedDomain function makes *.example.com match both any.example.com and the bare domain example.com. This dual behavior is documented in the comment but is semantically surprising — a wildcard prefix *. conventionally means "any subdomain of" not "the domain itself or any subdomain." This could lead to unintended allowlisting.
    Remediation: Consider splitting the behavior: *.example.com matches only subdomains; to also match the bare domain, users should add both example.com and *.example.com to the allowlist. This matches how TLS wildcard certificates and DNS wildcards work.

Low

  • [Correctness] docs/plans/universal-harness-access.md:1256-1266 — The import block for internal/resolve/resolve.go includes "path" and "path/filepath", but path is never used in the function bodies (URL normalization uses url.ResolveReference instead). In Go, unused imports are compile errors.
    Remediation: Remove the "path" import from the illustrative code.

  • [Style/conventions] docs/plans/universal-harness-access.md:1655 — The "Open Questions" section header at line 1655 is followed by the note "These implementation-level questions remain open and can be resolved during Phase 1/2 development" — but per CLAUDE.md, problem documents should have an "Open questions" section for unresolved issues. Since the plan is an implementation document (not a problem doc), this is acceptable, but the overlap with the ADR's resolved questions (see medium finding above) creates confusion about the document hierarchy.
    Remediation: Add a brief note at the top of the plan's Open Questions clarifying which questions from ADR-0037 are settled and which remain implementation-level.

Info

  • [Intent alignment] The PR body references Adopt provider-backed policy composition to reduce harness policy duplication #776 (provider-backed sandbox policy composition) as complementary work. This cross-reference is appropriate and the ADR correctly scopes the boundary: this ADR covers what the agent is (composability of definitions), while Adopt provider-backed policy composition to reduce harness policy duplication #776 covers where the agent runs (sandbox policy composability).

  • [Style/conventions] The ADR follows the repository's convention of presenting multiple options (A through D plus Z) with trade-offs before stating a decision. The implementation plan provides illustrative Go code that is internally consistent with the ADR's decisions. Both documents are well-organized and accessible.

  • [Platform security] The SSRF protection design is comprehensive: HTTPS-only, domain allowlists, internal IP blocking (including CGN, benchmark, IPv4-mapped IPv6), DNS rebinding protection via DialContext pinning, no redirects, timeouts, and size limits. The mandatory hash pinning decision (stronger than GitHub Actions' "recommended" pinning) is a good security posture for a platform that runs autonomous agents.

Footer

Outcome: comment-only
This review applies to SHA 74321e2a58b53b2d86affa13ab99101f64ee4532. Any push to the PR head clears this review and requires a new evaluation.

Previous run (5)

Review: #722

Head SHA: ad75dd5
Timestamp: 2026-05-12T00:00:00Z
Outcome: comment-only

Summary

This PR adds ADR-0037 and a detailed implementation plan for universal harness access via URLs. The documents are thorough, well-structured, and the security analysis is comprehensive — covering SSRF, TOCTOU, dependency confusion, DNS rebinding, and prompt injection threats with appropriate mitigations. The change is documentation-only with no Go code modifications. Two minor quality issues were found: a duplicated paragraph in the ADR Decision section and the README change bundles links to 3 pre-existing plan files alongside the new one (not harmful but worth noting for commit hygiene).

Findings

Medium

  • [Correctness] docs/ADRs/0037-universal-harness-access.md:148-150 — The Decision section contains a duplicated paragraph: "Hybrid approach: Option A for declarative resources combined with Option C's restriction on executable resources:" appears twice consecutively.
    Remediation: Remove the duplicate line.

Low

  • [Style/conventions] README.md:42-46 — The README diff adds a new docs/plans/ section with 4 plan links, but 3 of these plans (agent-execution-environment.md, vertex-inference-provisioning.md, 2026-03-06-adr46-drift-scanner.md) already exist on the base branch and are unrelated to this PR's universal-harness-access feature. Bundling unrelated README additions makes the change harder to attribute in git history.
    Remediation: Consider splitting the 3 existing plan links into a separate commit or PR for cleaner history, or note in the PR description that these are intentionally bundled.

Info

  • [Correctness] docs/ADRs/0037-universal-harness-access.md — ADR-0037 skips number 0035. Existing ADRs go from 0034 to 0036 (0035 is already absent). The numbering is consistent with the project's pattern of non-contiguous ADR numbers, so this is not a problem — just noting for awareness.

  • [Correctness] docs/plans/universal-harness-access.md — The implementation plan contains Go code examples for proposed packages (internal/fetch/, internal/resolve/, internal/audit/, internal/harness/url.go). These are illustrative code in a design document, not actual implementation. The code examples are well-structured and demonstrate the security patterns described in the ADR. One minor note: the matchesAllowedPrefix function uses url.ResolveReference for normalization which handles percent-encoding correctly — good design choice over path.Clean().

  • [Intent alignment] No linked issue for this PR. The ADR status is "Proposed" which is appropriate — it requires security review before acceptance per the PR body.

Footer

Outcome: comment-only
This review applies to SHA ad75dd5b9aa335511ddcec8da26e21777ddc7820. Any push to the PR head clears this review and requires a new evaluation.

Previous run (6)

Review: #722

Head SHA: 3ae4aad
Timestamp: 2026-05-12T00:00:00Z
Outcome: comment-only

Summary

This PR adds ADR-0037 (universal harness access via URLs and paths), a comprehensive implementation plan, and README updates linking the new plan and three pre-existing plans. The design is thorough, with well-considered security mitigations (SSRF protection, mandatory hash pinning, DNS rebinding prevention, content-addressed caching). The code examples are illustrative and demonstrate sound security practices. Two findings are worth noting: the ADR declares itself "Accepted" while the PR body requests security review before acceptance, and the README change bundles index entries for three pre-existing plan documents alongside the new one.

Findings

Medium

  • [Intent alignment] docs/ADRs/0037-universal-harness-access.md:24-25 — The ADR frontmatter and body declare status: Accepted and state "This ADR has been reviewed and accepted. The implementation plan provides detailed guidance for phased implementation. Security considerations... have been addressed during the review process." However, the PR body says "Status: Proposed (requires security review before acceptance)." These claims contradict each other. If this PR is the mechanism for proposing the ADR, the ADR should not claim it has already been accepted — that status should be set only after the PR is approved and merged. Merging as-is would create the false impression that a formal review has already occurred.
    Remediation: Change the ADR status to Proposed and update the Decision section to reflect that acceptance is pending review. The status can be updated to Accepted in a follow-up commit after reviewers approve.

Low

  • [Correctness] README.md — The README diff adds a new docs/plans/ section with four entries, but only one (universal-harness-access.md) is introduced by this PR. The other three (agent-execution-environment.md, vertex-inference-provisioning.md, 2026-03-06-adr46-drift-scanner.md) are pre-existing files that were previously unlisted. While not incorrect (the files do exist), this bundles unrelated documentation hygiene into a feature PR, expanding the scope beyond what the title and description indicate.
    Remediation: Either update the PR description to note the README cleanup, or split the pre-existing plan entries into a separate commit/PR for clarity.

Info

  • [Style/conventions] docs/plans/universal-harness-access.md — The implementation plan includes illustrative Go code referencing packages that don't yet exist (internal/fetch/, internal/resolve/, internal/security/). This is appropriate for a plan document and does not indicate an error. Implementers should treat these as directional guidance, not final API designs.

  • [Correctness] docs/plans/universal-harness-access.md — The isInternalIP function in the code example parses CIDR strings (net.ParseCIDR) on every invocation. In production, these should be package-level variables initialized once. The current pattern is fine for illustrative code in a plan document.

  • [Platform security] docs/ADRs/0037-universal-harness-access.md — The security analysis is comprehensive. Notable strengths: mandatory hash pinning for all sources (no tiered trust), DNS rebinding protection via DialContext pinning, URL prefix matching with trailing-slash enforcement to prevent prefix confusion, cache integrity re-verification on hits, and the intersection model for URL-fetched harness allowlists. The open questions around insider threat governance for allowed_remote_resources and signature verification are appropriately deferred.

Footer

Outcome: comment-only
This review applies to SHA 3ae4aad1805f6928ed69d9e1f8178384970606bd. Any push to the PR head clears this review and requires a new evaluation.

Previous run (7)

Review: #722

Head SHA: c5b4086
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This is a well-structured documentation PR that adds a new ADR (0037, Proposed status) and a detailed implementation plan for universal harness access via URLs and paths. The PR follows project conventions: the ADR presents multiple options with trade-offs (Options A–D and Z), explicitly marks the decision as "Proposed — pending security review," and the implementation plan provides phased delivery with thorough security analysis. The README update correctly links to the new plan document and three pre-existing plan documents that were previously unlinked. All cross-referenced ADRs (0024, 0022, 0017) and issue (#776) exist and are relevant. No code changes are introduced — all Go code in the plan is illustrative. No critical or high findings.

Findings

Medium

  • [Correctness] docs/plans/universal-harness-access.md (Section 5: Dependency Resolver) — The resolve.go code example imports "path" (unused) and omits "net/url" (required by matchesAllowedPrefix which calls url.Parse and uses url.URL). Since this is illustrative code in a design document rather than production code, it won't cause build failures, but inaccurate examples may mislead implementers.
    Remediation: Fix the import list in the code example: remove "path", add "net/url".

  • [Correctness] docs/ADRs/0037-universal-harness-access.md (Open questions section) — The ADR identifies a significant insider threat concern: allowed_remote_resources in harness YAML is editable by anyone with write access to .fullsend, potentially bypassing org-level domain allowlists. The implementation plan describes four phases but doesn't indicate which open questions must be resolved before which phases. Since the ADR is "Proposed" status, this is expected, but implementers would benefit from explicit phase-gating of unresolved security questions.
    Remediation: Add a note in the plan's Phase 1 section listing which ADR open questions must be resolved before Phase 1 begins (at minimum, the allowed_remote_resources governance question).

Low

  • [Correctness] docs/plans/universal-harness-access.md (Section 4: Cache) — The CacheEntry struct declares a ContentType field, but CachePut never populates it (always zero-value). Minor inconsistency in the illustrative code.
    Remediation: Either populate ContentType from the HTTP response header in the example, or remove the field from the struct.

Info

Footer

Outcome: approve
This review applies to SHA c5b4086726f2ea84154c3cad14c09f5cbf15527c. Any push to the PR head clears this review and requires a new evaluation.

Previous run (8)

Review: #722

Head SHA: 5e7c2e6
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs and paths. The documents are internally consistent, thoroughly analyze security implications (SSRF, TOCTOU, dependency confusion, prompt injection), and follow the repo's convention of presenting multiple options with trade-offs. The ADR is correctly marked "Proposed" with explicit gates before acceptance. No blocking findings — the design is sound and the documentation is high quality.

Findings

Critical

(none)

High

(none)

Medium

(none)

Low

  • [Correctness] docs/plans/universal-harness-access.md — The matchesAllowedPrefix code example uses path.Clean() for URL path normalization. While this works for most cases, path.Clean uses POSIX path semantics (e.g., collapses // to /, removes trailing /) which may differ subtly from RFC 3986 path normalization. When implementing, consider using url.Parse roundtrip or net/url's ResolveReference for canonical URL construction to ensure consistent behavior with percent-encoded paths and edge cases like empty path segments.

Info

  • [Style] docs/plans/universal-harness-access.md — The implementation plan includes ~500 lines of Go code across 9 sections (harness schema, URL detection, resource fetcher, cache, dependency resolver, CLI integration, security scanner, audit logging, offline mode). While these concrete examples aid understanding and will accelerate implementation, they may become stale as the design evolves through review. The ADR's "Proposed" status and explicit implementation gates mitigate this risk.
  • [Intent] README.md — The diff adds a docs/plans/ section linking to 4 plan documents, 3 of which pre-exist on main. This is appropriate since the docs/plans/ section was entirely absent from the README, and adding it alongside the new plan document is the natural time to list all existing plans.
  • [Correctness] docs/ADRs/0037-universal-harness-access.md — ADR-0035 does not exist on main (sequence goes 0034 → 0036 → 0037). The numbering gap is pre-existing and unrelated to this PR.

Footer

Outcome: approve
This review applies to SHA 5e7c2e606ea26558d3c96f66c9278ff4bd9f246e. Any push to the PR head clears this review and requires a new evaluation.

Previous run (9)

Review: #722

Head SHA: 9175242
Timestamp: 2026-05-12T16:00:00Z
Outcome: comment-only

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs and paths. The documents are thorough, internally consistent, and demonstrate careful security analysis across SSRF, TOCTOU, dependency confusion, and prompt injection vectors. The ADR correctly marks itself as "Proposed" with explicit gates before acceptance. Two non-blocking findings are noted: the new README docs/plans/ section omits the three pre-existing plan files in that directory, and a cross-reference to PR #776 cannot be verified.

Findings

Medium

  • [Correctness] README.md:42-43 — The new docs/plans/ section in the README lists only the new universal-harness-access plan, but the directory already contains three other files (2026-03-06-adr46-drift-scanner.md, agent-execution-environment.md, vertex-inference-provisioning.md). This creates an incomplete index that may confuse readers expecting a comprehensive listing.
    Remediation: Either add entries for the existing plans in this PR, or note that the section is a partial listing. Consider whether the other plan files were intentionally unlisted.

Low

  • [Correctness] docs/ADRs/0037-universal-harness-access.md:236 — The ADR cross-references #776 ("provider-backed sandbox policy composition") but this PR is not accessible. If it doesn't exist yet, the reference is forward-looking and should be noted as such to avoid confusion.
    Remediation: Clarify whether Adopt provider-backed policy composition to reduce harness policy duplication #776 exists. If it's a planned future PR, mark the reference as "(planned)" or "(future)".

Info

  • [Style] docs/plans/universal-harness-access.mdmake lint could not be verified due to network restrictions in the sandbox environment (pre-commit hooks unable to fetch from GitHub). This is an infrastructure limitation, not a code issue. The author should confirm lint passes locally before merge.

Footer

Outcome: comment-only
This review applies to SHA 91752421665da26e5169055c59445bc52d3d3311. Any push to the PR head clears this review and requires a new evaluation.

Previous run (10)

Review: #722

Head SHA: 4a50255
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This is a well-structured documentation PR adding ADR-0037 and an implementation plan for universal harness access via URLs and paths. The ADR thoroughly analyzes security implications (SSRF, TOCTOU, dependency confusion, prompt injection) with concrete mitigations. The implementation plan includes detailed Go code sketches, a phased migration path, and a comprehensive testing strategy. No code changes are included — only documentation, a README index update, and a minor .gitignore housekeeping addition. The documents are internally consistent, reference existing ADRs correctly, and follow repo conventions.

Findings

Medium

  • [style/conventions] .gitignore — The /fullsend binary ignore is described as "unrelated housekeeping" in the PR body. Bundling unrelated changes with a feature PR is minor scope creep. Not blocking since the change is trivial and correct (ignoring a compiled binary at repo root).

Low

  • [correctness] docs/ADRs/0037-universal-harness-access.md — ADR numbering skips 0035 (existing ADRs go 0034 → 0036 → 0037). The repo already has gaps (0029 missing), so this appears to be an existing pattern, possibly from concurrent PRs. Worth confirming no collision exists on the target branch.

Info

  • [correctness] docs/plans/universal-harness-access.md — The plan references internal/security/scan.go and security.ScanResource() which don't exist yet. This is expected for a plan document describing future implementation, but implementers should note these are proposed interfaces, not existing code.
  • [intent-alignment] The ADR status is correctly marked "Proposed" with explicit gates before acceptance (security review, consensus on trust model). This aligns with the repo convention that documents should "present multiple options with trade-offs, not prescribe single solutions."

Footer

Outcome: approve
This review applies to SHA 4a50255c1f9757ff5145eb4ed5e7e11e176654db. Any push to the PR head clears this review and requires a new evaluation.

Previous run (11)

Review: #722

Head SHA: 067a668
Timestamp: 2026-05-12T00:00:00Z
Outcome: comment-only

Summary

This PR adds a well-structured ADR (0037) and implementation plan for universal harness access via URLs, along with a README update documenting the docs/plans/ section and a .gitignore entry for the compiled binary. The documents are thorough, with extensive security analysis. A few medium and low findings are noted around the bundled .gitignore change (unrelated to the main topic), a potential ADR numbering collision risk, and minor documentation observations.

Findings

Medium

  • [style/conventions] README.md — The PR bundles a docs/plans/ section into the README that documents 3 pre-existing plan files (2026-03-06-adr46-drift-scanner.md, agent-execution-environment.md, vertex-inference-provisioning.md) alongside the new one. While all referenced files exist, adding previously-undocumented files to the README in a PR about a specific new feature muddies the PR scope. Consider splitting the README section for pre-existing plans into a separate PR for cleaner history.
    Remediation: Split the README additions for pre-existing plan files into a separate commit or PR.

  • [intent-alignment] .gitignore — Adding /fullsend to .gitignore is unrelated to universal harness access. This is a housekeeping change bundled into a feature PR.
    Remediation: Consider splitting this into a separate commit with its own description, or note it explicitly in the PR body as an unrelated cleanup (which the PR body does under "Additional changes").

Low

  • [correctness] docs/ADRs/0037-universal-harness-access.md — ADR number 0037 is currently available (highest on main is 0036), but there is a general risk of ADR number collisions with concurrent PRs. This is an existing repo-wide concern, not specific to this PR.
    Remediation: Use the renumber-adr process before merge if other ADR PRs are in flight.

Info

  • [correctness] docs/plans/universal-harness-access.md — The implementation plan includes substantial Go code snippets (fetch.go, cache.go, resolve.go, url.go, audit logging). These are illustrative pseudocode in a design document, not actual source files. The code is well-thought-out and covers SSRF protection, DNS rebinding, content-addressed caching, and integrity verification. The isInternalIP function correctly handles IPv4-mapped IPv6 addresses and covers ranges beyond what Go's stdlib IsPrivate() checks (CGN, benchmark testing ranges).

  • [security] docs/ADRs/0037-universal-harness-access.md — The ADR correctly identifies the key security threats (TOCTOU, dependency confusion, SSRF, prompt injection via skills) and proposes appropriate mitigations. The decision to require mandatory hash pinning for all remote resources (including first-party) is a strong security stance. The open question about allowed_remote_resources governance (whether it should be CODEOWNERS-protected) is important and correctly flagged.

  • [style/conventions] docs/plans/universal-harness-access.md — The plan document cross-references Adopt provider-backed policy composition to reduce harness policy duplication #776 which is verified to exist ("Adopt provider-backed policy composition to reduce harness policy duplication"). The relationship is clearly articulated.

Footer

Outcome: comment-only
This review applies to SHA 067a668ab44658c9aff57a740f87f60118c2ca36. Any push to the PR head clears this review and requires a new evaluation.

Previous run (12)

Review: #722

Head SHA: 5e54842
Timestamp: 2026-05-12T00:00:00Z
Outcome: comment-only

Summary

This is a well-constructed documentation PR adding ADR-0037 and a detailed implementation plan for universal harness access. The security analysis is thorough (SSRF, TOCTOU, dependency confusion, prompt injection), the phased migration is practical, and the code sketches are illustrative without being premature. A few minor issues are worth noting but none are blocking.

Findings

Medium

  • [correctness] docs/plans/universal-harness-access.md (Open Questions section) — Duplicate heading number: there are two sections numbered "### 2." ("Signature verification" at ~line 1612 and "Namespace governance" at ~line 1625 in the diff). The subsequent sections ("### 3. Version resolution" and "### 4. Cache eviction") continue from the wrong count. Should be 2, 3, 4, 5.
    Remediation: Renumber the Open Questions to 1–5.

Low

  • [intent-alignment] .gitignore — The addition of /fullsend to .gitignore appears unrelated to the universal harness access feature. This looks like it's ignoring a built binary, which is reasonable but is scope creep for this PR.
    Remediation: Consider splitting this into a separate commit or PR, or mention it in the PR description.

  • [correctness] README.md — The PR adds the entire docs/plans/ section to the README, listing four plans including three that already exist (ADR-0046 Drift Scanner, Agent Execution Environment, Vertex AI Inference Provisioning). While these links are valid and the section is useful, this is additional scope beyond what the PR title suggests ("Add implementation plan for universal harness access").
    Remediation: Acknowledge in the PR description that the README is being updated to document the full plans directory, not just the new plan.

  • [style] docs/ADRs/0037-universal-harness-access.md — The ADR references #776 in the Consequences section (line 248 of the ADR). This appears to be a GitHub issue reference. Verify it exists and is the correct cross-reference for sandbox-level policy composability.

Info

  • [correctness] docs/plans/universal-harness-access.md — The Go code sketches in the implementation plan reference packages like internal/security with types like security.ScanResource and security.RemoteResourcePolicy that don't exist yet. This is expected for a plan document, but worth noting that these are aspirational interfaces, not existing code.

  • [style] docs/ADRs/0037-universal-harness-access.md — The ADR is numbered 0037, which follows the existing sequence (latest on main is 0036). Note that 0035 appears to be missing/skipped, but this is a pre-existing gap not introduced by this PR.

Footer

Outcome: comment-only
This review applies to SHA 5e5484228f7184ea14bb48a9c13573fe181384c7. Any push to the PR head clears this review and requires a new evaluation.

Previous run (13)

Review: #722

Head SHA: 5e54842
Timestamp: 2026-05-12T00:00:00Z
Outcome: comment-only

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs, along with README updates to surface the existing docs/plans/ directory. The design is thorough, with strong security analysis covering SSRF, TOCTOU, dependency confusion, and prompt injection threats. No executable code is changed. Two minor issues noted: duplicate section numbering in the implementation plan's Open Questions, and an unrelated .gitignore stowaway change.

Findings

Medium

(none)

Low

  • [style/conventions] docs/plans/universal-harness-access.md Open Questions section — Duplicate heading number: two sections are numbered ### 2. ("Signature verification" and "Namespace governance"). Should be numbered sequentially 1–5.
    Remediation: Renumber "Namespace governance" to ### 3., "Version resolution" to ### 4., and "Cache eviction" to ### 5..

Info

  • [intent alignment] .gitignore — Addition of /fullsend is unrelated to the universal harness access feature. This ignores a root-level binary that would be produced by running go build without the -o flag (the Makefile already builds to bin/fullsend which is covered by the existing bin/ entry). Harmless but a stowaway change.

  • [correctness] docs/ADRs/0037-universal-harness-access.md — The ADR references #776 in the Consequences section ("This complements sandbox-level policy composability (Adopt provider-backed policy composition to reduce harness policy duplication #776)"). Verify this issue number is correct and intentional as a forward reference.

  • [correctness] README.md — The plans directory section is a welcome addition. The 3 pre-existing plans (2026-03-06-adr46-drift-scanner.md, agent-execution-environment.md, vertex-inference-provisioning.md) were not previously listed in the README, so this change documents more than just the new plan.

Footer

Outcome: comment-only
This review applies to SHA 5e5484228f7184ea14bb48a9c13573fe181384c7. Any push to the PR head clears this review and requires a new evaluation.

Previous run (14)

Review: #722

Head SHA: c12fbab
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs and paths. The documents are thorough, correctly follow project conventions, and explicitly gate implementation behind security review. ADR numbering (0037) correctly follows the existing 0036. All cross-referenced ADRs (0017, 0022, 0024) and plan files exist. One minor out-of-scope change (.gitignore) is noted but non-blocking.

Findings

Medium / Low / Info

  • [intent-alignment] .gitignore — The addition of /fullsend to .gitignore is unrelated to the universal harness access feature. Consider separating this into its own commit for cleaner history.

  • [correctness] docs/plans/universal-harness-access.md — The isAllowedDomain wildcard matching (*.example.com) also matches the bare domain example.com itself (line: hostname == domain). This is a design choice worth calling out explicitly in the plan, as it differs from typical wildcard semantics where *.example.com means "subdomains only." Since this is a plan document and not merged code, this is informational.

  • [correctness] docs/plans/universal-harness-access.md — The DNS rebinding protection pins to ips[0] only. If the first IP is IPv6 and the client cannot connect via IPv6, the fetch will fail even though a valid IPv4 address was available. The implementation should consider falling back to other validated IPs. Informational since this is a plan document.

Footer

Outcome: approve
This review applies to SHA c12fbab3ee0516374d1c189d5a03565a701de2ad. Any push to the PR head clears this review and requires a new evaluation.

Previous run (15)

Review: #722

Head SHA: 7078c83
Timestamp: 2026-05-12T00:00:00Z
Outcome: approve

Summary

This PR adds a well-structured ADR (0037) and implementation plan for universal harness access via URLs and paths. The documents follow repo conventions—presenting multiple options with trade-offs, maintaining open questions, and staying organization-agnostic. The security analysis is thorough: SSRF protection with DNS rebinding defense, mandatory hash pinning, content-addressed caching, domain allowlists, internal IP blocking, and no-redirect policy. The ADR is clearly marked "Proposed" and explicitly states it should not be implemented until security review is complete. A few minor observations are noted below but nothing blocks approval.

Findings

Medium / Low / Info

  • [intent-alignment] .gitignore — The addition of /fullsend to .gitignore appears unrelated to the universal harness access feature. Consider splitting into a separate commit or PR for cleaner history.

  • [intent-alignment] README.md — The README diff adds a new docs/plans/ section listing 4 plans, but only universal-harness-access.md is introduced by this PR. The other 3 plans (drift scanner, agent execution environment, vertex inference provisioning) already existed without README entries. This is useful housekeeping but expands scope beyond the stated purpose.

  • [correctness] docs/plans/universal-harness-access.md — Minor inconsistency between prose and illustrative code: the "Content-Addressed Caching" section describes cache lookup as SHA256(URL + hash) → cache_manifest.db → SHA256(content) → cached file, but the illustrative CacheGet implementation uses the content hash directly as the directory key with no manifest DB. This is fine for illustrative code but worth aligning if the plan is used as an implementation reference.

  • [correctness] docs/plans/universal-harness-access.md — The ADR describes "Visited node tracking" for cycle detection in transitive dependency resolution, but the illustrative resolveResourceWithLimits function only implements depth and count limits without a visited-set. The comment // Parse skill to extract transitive dependencies is a TODO placeholder. Worth noting for implementers.

  • [style] docs/ADRs/0037-universal-harness-access.md — The ADR references ADR-0024 and ADR-0022 but neither has a number matching existing ADR files (existing ADRs go up to 0036). If these refer to planned or in-progress ADRs, consider noting that, or verify the numbers are correct.

Footer

Outcome: approve
This review applies to SHA 7078c83d77803c047b5d95d88e105a752a89a00c. Any push to the PR head clears this review and requires a new evaluation.

Previous run (16)

Review: #722

Head SHA: 8cd3799
Timestamp: 2026-05-11T00:00:00Z
Outcome: comment-only

Summary

This PR adds a well-structured proposed ADR (0037) and detailed implementation plan for universal harness access via URLs. The documents follow repo conventions (multiple options with trade-offs, open questions sections, "Proposed" status with explicit acceptance prerequisites). All cross-referenced documents (ADR-0022, ADR-0024, security-threat-model, agent-architecture) exist in the repo. The security analysis is thorough, covering SSRF, TOCTOU, dependency confusion, prompt injection, and DNS rebinding. No critical or high issues found. A few observations worth noting below.

Findings

Medium

  • [intent-alignment] .gitignore — The addition of /fullsend to .gitignore is unrelated to universal harness access. This appears to be a housekeeping change to ignore the compiled cmd/fullsend/ binary at the repo root. Consider splitting this into a separate commit or noting it in the PR description to keep the PR scope clean.

Low

  • [correctness] docs/plans/universal-harness-access.md:860-870 — The illustrative ParseIntegrityHash function does not validate hash format (hex characters, expected length of 64 for SHA256). A malformed hash like #sha256=notahash would be accepted. Since this is a design illustration, not production code, flagging for awareness when implementing.

  • [correctness] docs/plans/universal-harness-access.md:992-998 — The isAllowedDomain wildcard matching (*.example.com) also matches the bare domain example.com itself (line: hostname == domain). This is documented in the code comment but could be surprising — typically *.example.com means "subdomains only." Worth a design note clarifying the intended semantics when implementing.

Info

  • [correctness] docs/ADRs/0037-universal-harness-access.md — ADR numbering is correct: 0037 follows the highest existing ADR (0036-agent-execution-sandbox.md). No conflicts detected.

  • [style] docs/plans/universal-harness-access.md — The plan's related documents section references ADR-0017 (credential isolation) which also exists. All five cross-references from the plan and the three relates_to entries in the ADR frontmatter correspond to real documents in the repo.

  • [correctness] docs/plans/universal-harness-access.md:946-963 — The DNS rebinding protection correctly validates all resolved IPs before connecting, then pins to ips[0] in the custom DialContext. The implementation pattern is sound — all IPs are checked against the blocklist before any connection is established.

Footer

Outcome: comment-only
This review applies to SHA 8cd3799f5382f73d6c3338a3e6fa3718eaa140b4. Any push to the PR head clears this review and requires a new evaluation.

Previous run (17)

Review: #722

Head SHA: 8c0a4e9
Timestamp: 2026-05-11T00:00:00Z
Outcome: approve

Summary

This is a well-structured documentation PR adding an ADR (0037) and implementation plan for universal harness access via URLs and paths. The documents follow repo conventions — multiple options with trade-offs, open questions sections, and clear status gating ("Proposed" with explicit prerequisites before acceptance). The security analysis is thorough, covering SSRF protection, TOCTOU mitigation, dependency confusion, and prompt injection via remote resources. No executable code is changed. One minor scope observation noted below.

Findings

Low

  • [intent alignment] .gitignore:22 — The addition of /fullsend to .gitignore (ignoring the compiled binary at repo root) is unrelated to the universal harness access feature. This is benign but would be cleaner as a separate commit for traceability.

Footer

Outcome: approve
This review applies to SHA 8c0a4e9684b3d93a0f6f6e4637ab1c6181249cd5. Any push to the PR head clears this review and requires a new evaluation.

Previous run (18)

Review: #722

Head SHA: 3003b05
Timestamp: 2026-05-11T00:00:00Z
Outcome: comment-only

Summary

This is a well-written documentation PR that adds a Proposed ADR (0037) and a detailed implementation plan for universal harness access via URLs. The documents are thorough, security-aware, and internally consistent. All referenced ADRs exist and the README link is correctly placed. One medium finding notes that the SSRF IP blocklist in the plan should be more comprehensive to prevent bypass when implemented. Two low/info items note an unrelated .gitignore change and a minor ADR numbering gap. No blocking issues.

Findings

Medium

  • [platform security] docs/plans/universal-harness-access.md — The SSRF internal IP blocklist and illustrative isInternalIP() function (§3 Resource Fetcher) rely solely on Go's IsLoopback(), IsPrivate(), and IsLinkLocalUnicast(). These miss several address ranges that should be blocked for robust SSRF defense: 0.0.0.0/8 (current network — curl http://0.0.0.0:8080 hits localhost on Linux), 100.64.0.0/10 (Carrier-Grade NAT / shared address space, RFC 6598), 198.18.0.0/15 (benchmark testing, RFC 2544), and IPv4-mapped IPv6 addresses (::ffff:127.0.0.1 can bypass IPv4-only checks). Since this plan will guide implementation, the blocklist enumeration in both the prose and code examples should be explicit and complete.
    Remediation: Add the missing ranges to the "Internal IP rejection" list in the SSRF Protection section and update the illustrative isInternalIP() to check them explicitly rather than relying solely on stdlib helpers.

Low

  • [style/conventions] .gitignore:22 — The /fullsend entry (ignoring the built binary) is unrelated to universal harness access. Bundling unrelated changes makes commit history harder to bisect.
    Remediation: Move to a separate commit or PR, or mention the change in the PR description.

Info

  • [correctness] docs/ADRs/0037-universal-harness-access.md — ADR 0037 skips number 0035 (0029 is also missing from the sequence). This may collide with in-flight ADRs on other branches. Consider checking for conflicts before merge using the renumber-adr process.

Footer

Outcome: comment-only
This review applies to SHA 3003b051f2a55e31ddfb11b53804f28cc51caf51. Any push to the PR head clears this review and requires a new evaluation.

Previous run (19)

Review: #722

Head SHA: 665c4cd
Timestamp: 2026-05-11T00:00:00Z
Outcome: comment-only

Summary

This is a well-structured documentation PR that adds ADR-0037 (Proposed) and a detailed implementation plan for making harness resources accessible via URLs. The security analysis is thorough — SSRF protection, mandatory hash pinning, DNS rebinding defense, and the declarative-only restriction for remote resources are all well-reasoned. The design correctly rejects tiered trust in favor of uniform security. No critical or high issues found. A few minor observations are noted below.

Findings

Medium

  • [intent-alignment] .gitignore — The addition of fullsend to .gitignore is unrelated to universal harness access. This is a drive-by change that should ideally be in a separate commit or PR for traceability. The bare entry fullsend will match any file or directory named fullsend at any depth in the repo tree; if the intent is to ignore a built binary at the repo root, /fullsend (with leading slash) would be more precise.
    Remediation: Consider using /fullsend to scope the ignore to the repo root, or split this into a separate PR.

Low

  • [correctness] docs/plans/universal-harness-access.md — The illustrative matchesAllowedPrefix function relies on strings.HasPrefix for URL prefix matching. The ADR's Validate() method requires trailing slashes on allowed_remote_resources entries, which prevents prefix confusion attacks (e.g., library-evil/ won't match prefix library/). This coupling between validation and matching is correct but implicit — when this is implemented, a code comment or test should make the invariant explicit to prevent future regressions where one is changed without the other.
    Remediation: Add a comment in the implementation noting the trailing-slash invariant, and ensure unit tests cover the prefix confusion case.

  • [correctness] docs/plans/universal-harness-access.md — The ResolveHarness function signature passes workspaceRoot and the harness, but the CLI integration snippet (section 6) calls resolve.ResolveHarness(ctx, h, fetchPolicy) without workspaceRoot, mismatching the function definition in section 5. This is a minor inconsistency in illustrative code.
    Remediation: Align the CLI snippet with the function signature when implementing.

  • [style] docs/plans/universal-harness-access.md — Significant content overlap with the ADR (security implications, TOCTOU mitigations, dependency confusion). The plan is 1,199 lines, partly because threat analysis is duplicated from ADR-0037. Consider cross-referencing the ADR for shared security analysis rather than restating it.
    Remediation: After the ADR is accepted, consolidate shared security analysis into one location with cross-references.

Info

  • [correctness] docs/ADRs/0037-universal-harness-access.md — ADR number 0037 does not conflict with any existing ADR on main (latest is 0036). Numbers 0029 and 0035 are unused gaps, which is consistent with prior numbering patterns in this repo.

  • [correctness] All cross-references check out: ADR-0024, ADR-0022, ADR-0017, security-threat-model.md, and agent-architecture.md all exist. The README link is inserted in correct alphabetical order.

  • [intent-alignment] The ADR is correctly marked Proposed with explicit gates before acceptance (security review, open questions, consensus on trust model). This follows the repo convention of presenting options with trade-offs rather than prescribing solutions.

Footer

Outcome: comment-only
This review applies to SHA 665c4cd7184c79e46d47dde84a3010ed9a1707be. Any push to the PR head clears this review and requires a new evaluation.

Previous run (20)

Review: #722

Head SHA: 4bafeab
Timestamp: 2026-05-11T00:00:00Z
Outcome: comment-only

Summary

This PR adds a well-structured ADR (0037) and detailed implementation plan for universal harness access via URLs and paths. The design documents are thorough, present multiple options with trade-offs (consistent with repo conventions), and include extensive security analysis covering SSRF, TOCTOU, dependency confusion, DNS rebinding, and prompt injection. The ADR is correctly marked as "Proposed" with explicit gates before acceptance. Two minor findings: an unrelated .gitignore change bundled into the PR, and a redundant conditional in the illustrative Go code that could confuse implementers. No blocking issues.

Findings

Medium

  • [correctness] docs/plans/universal-harness-access.md:1196-1205 — In the illustrative resolveResourceWithLimits Go code, the mandatory hash check on line ~1193 (if !hasHash { return error }) guarantees hasHash is always true past that point. The subsequent if hasHash guard on line ~1197 (cache lookup) and line ~1215 (hash comparison) are therefore redundant — they will always be true. While this is design-doc code (not compiled), implementers following this as a template would carry forward dead conditionals that obscure the actual control flow and could mask bugs if the mandatory check is later relaxed.
    Remediation: Remove the redundant if hasHash guards in the illustrative code since mandatory hash pinning ensures they are always true.

Low

  • [intent-alignment] .gitignore:22 — Adding fullsend (the binary) to .gitignore is unrelated to the universal harness access design documents. This change should be in a separate commit or PR for clearer change tracking.
    Remediation: Split the .gitignore change into its own commit or PR.

Info

  • [style] docs/ADRs/0037-universal-harness-access.md — The ADR is numbered 0037. Existing ADRs go up to 0036. There are existing gaps in the sequence (no 0029, 0035). 0037 is the correct next number and does not conflict with any existing ADR on main.

  • [correctness] docs/plans/universal-harness-access.md — All referenced documents (ADR-0024, ADR-0022, ADR-0017, security-threat-model.md, agent-architecture.md) exist in the repository. Cross-references are valid.

  • [style] README.md:45 — The new entry is correctly placed in alphabetical order within the plans section. The ADR itself is not individually linked in the README's ADR section, which is consistent with how other ADRs are handled (the section links to the directory, not individual ADRs).

Footer

Outcome: comment-only
This review applies to SHA 4bafeab69af4760e464411ea282f6a512d6cdfc8. Any push to the PR head clears this review and requires a new evaluation.

ggallen added a commit to ggallen/fullsend that referenced this pull request May 7, 2026
Fixes from review comment #4397770354:

1. Fix io.LimitReader truncation bug - read one extra byte and reject if
   response exceeds MaxSizeBytes instead of silently truncating

2. Add dependency resolution limits - MaxDepth (10) and MaxResources (50)
   to prevent runaway transitive dependency chains

3. Fix ADR numbering gap - rename ADR-0030 to ADR-0029 and update all
   references

4. Remove unrelated .gitignore change - remove 'fullsend' binary entry

5. Fix hardcoded audit log path - use ~/.cache/fullsend/audit/ instead
   of /var/log/fullsend/

6. Add error check for os.UserHomeDir() call

7. Add missing import declarations (fmt, path/filepath) to audit package

8. Clarify HTTP/HTTPS classification in SSRF protection section

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@ggallen ggallen force-pushed the feat/universal-harness-access branch from f2047b5 to 1314189 Compare May 7, 2026 15:29
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

Site preview

Preview: https://1160005f-site.fullsend-ai.workers.dev

Commit: 4abf3e728ae857c34c631225d6e1e24cd19f9995

@ggallen ggallen changed the title Add ADR-0030 and implementation plan for universal harness access Add implementation plan for universal harness access May 7, 2026
Copy link
Copy Markdown

@fullsend-ai-review fullsend-ai-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Copy link
Copy Markdown
Contributor

@maruiz93 maruiz93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the approach. The other day I mentioned this in the call that I really think external references make sense. There are a couple of things that we should acknowledge in the ADR though:

  • Barak mentioned that this is really similar to package management. But here we are not packaging the harnesses as they are a set of files that could actually be in different parts and not a blackbox downloaded from somewhere. I prefer this idea as it's more flexible and more likely to encourage people to reuse different pieces but there are cons like attack surface increased and need to check different things to download all the pieces that conforms the agent. We should at least mention this in the ADR.

  • This approach makes me think that it makes more sense to have our own set of harnes files and harness components in a separate github repo within the fullsend-ai. Maybe even more than one repo so it allows fullsend to provide different tiers of agents (first-class fully supported and evaluated) and second class but available for others and more likely to accept external suggestions and build community without being so strict with the evaluation .

Comment thread docs/ADRs/0037-universal-harness-access.md
Comment thread docs/plans/universal-harness-access.md
Comment thread docs/ADRs/0037-universal-harness-access.md
Comment thread docs/ADRs/0037-universal-harness-access.md Outdated
ggallen added a commit to ggallen/fullsend that referenced this pull request May 7, 2026
Fixes from review comment #4397770354:

1. Fix io.LimitReader truncation bug - read one extra byte and reject if
   response exceeds MaxSizeBytes instead of silently truncating

2. Add dependency resolution limits - MaxDepth (10) and MaxResources (50)
   to prevent runaway transitive dependency chains

3. Fix ADR numbering gap - rename ADR-0030 to ADR-0029 and update all
   references

4. Remove unrelated .gitignore change - remove 'fullsend' binary entry

5. Fix hardcoded audit log path - use ~/.cache/fullsend/audit/ instead
   of /var/log/fullsend/

6. Add error check for os.UserHomeDir() call

7. Add missing import declarations (fmt, path/filepath) to audit package

8. Clarify HTTP/HTTPS classification in SSRF protection section

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@ggallen ggallen force-pushed the feat/universal-harness-access branch from 749b70c to 2372441 Compare May 7, 2026 23:24
Copy link
Copy Markdown
Contributor Author

@ggallen ggallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review feedback addressed

I've addressed all the issues from the review:

1. Missing imports in implementation plan (Medium - Correctness)

Fixed the missing strings import in the resolve.go snippet. The url.go snippet already had all necessary imports.

2. Incomplete docs/plans/ index (Info - Style/conventions)

Added the missing plan files to README.md:

  • ADR46 Drift Scanner
  • Vertex Inference Provisioning

3. Cache persistence question (from @ralphbean)

Added clarification in the ADR's "Proposed approach" section about cache location and persistence.

Cache semantics: The cache is stored in the repository's workspace (e.g., .fullsend-cache/). In ephemeral CI/CD environments like GitHub Actions where each workflow run gets a fresh runner, the cache is rebuilt on each run. However, the implementation should leverage platform-native caching mechanisms (GitHub Actions cache, GitLab CI cache) to persist the content-addressed cache across workflow runs.

This means:

  • First run: fetch remote resources and store in local cache, then upload to platform cache
  • Subsequent runs: restore from platform cache, avoiding re-fetch unless resources have changed

This is similar to how dependency managers (npm, pip) work in CI - the dependencies are rebuilt each run, but restored from cache to avoid re-downloading.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

fullsend review is working on this — view logs

Comment thread docs/ADRs/0037-universal-harness-access.md
Copy link
Copy Markdown

@fullsend-ai-review fullsend-ai-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

ggallen added a commit to ggallen/fullsend that referenced this pull request May 8, 2026
Added a "Trade-off" section to the implementation plan explaining
that the "scripts must be local" requirement means the .fullsend repo
will still contain script files, leading to potentially large diffs
when scripts are updated.

Proposed mitigations:
- Minimal wrapper pattern: Local scripts become thin wrappers that
  download and verify URL-sourced scripts, minimizing diff size
- Vendoring with lock files: Use a lock file approach where diffs
  show only URL+hash updates rather than full script content
- Future sandboxing: If URL-sourced scripts are permitted later,
  they would run in heavily restricted sandboxes

Addresses fullsend-ai#722 (comment)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

fullsend review is working on this — view logs

@ggallen ggallen requested review from maruiz93 and ralphbean May 8, 2026 01:19
Changes based on ralphbean's review:
- Change ADR status from Proposed to Accepted in frontmatter and body
- Update Decision section to reflect acceptance following security review
- Add cache integrity re-verification to TOCTOU mitigation section
- Add preamble to Open Questions explaining intentional deferral
- Clarify that open questions don't block core design acceptance

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

Revert status back to Proposed and provide concrete answers to all
design questions:

1. Insider threat governance: Harness allowlists must be subset of
   org-level config.yaml allowlist (validated at load time)

2. Signature verification: Phase 1 uses hash pinning only; signatures
   deferred to Phase 3 as optional enhancement

3. Namespace governance: Decentralized publishing model; no central
   registry; consumers control trust via allowlists

4. Version resolution: No magic resolution; all references must be
   full URLs with explicit hashes

5. Offline mode: Support via --offline flag; fail if resources not
   cached; useful for air-gapped CI/CD

6. Lock file format: Phase 3 adds .fullsend/lock.yaml with full
   schema for transitive dependency pinning

7. URL scheme: Phase 1 supports bare https:// only; git+https://
   structured refs deferred to Phase 2/3

All questions now have concrete decisions with rationale, enabling
the ADR to move toward acceptance.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

Remove duplicated "Hybrid approach" heading that appeared twice
consecutively in the Decision section.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

MEDIUM fixes:
1. ADR vs Plan inconsistency: Add note to Open Questions clarifying
   which questions are resolved in ADR-0037 vs truly open for
   implementation (top-level harness protection, cache eviction)

2. DNS rebinding IP handling: Document that production implementations
   should iterate through all validated IPs sequentially to handle
   IPv4/IPv6 network configurations

3. Wildcard domain matching: Fix *.example.com to match subdomains
   only (not bare domain), following TLS certificate conventions

LOW fixes:
1. Remove unused "path" import from resolve.go example code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

MEDIUM fix:
- Update Consequences section header to reflect hybrid approach
  (was "Option A" but decision is hybrid A+C)

LOW fix:
- Document that .fullsend-cache/ should be added to .gitignore
  to prevent cache artifacts from being committed

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

Add notes to illustrative code examples flagging implementation
concerns that should be addressed in production:

1. CachePut: Non-atomic writes (metadata.json, content written
   separately). Production should use atomic writes via temp file
   + rename to prevent partial cache entries.

2. isInternalIP: Repeated net.ParseCIDR allocations per call.
   Production should parse CIDR ranges once at package init and
   reuse them.

These are acceptable for plan documents but worth documenting for
implementers.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

MEDIUM fixes:
1. ADR numbering gap: Add note explaining 0029 and 0035 are
   intentionally reserved/skipped per project conventions

2. DNS rebinding protection: Update code to iterate through all
   validated IPs with fallback instead of pinning to ips[0] only

3. CIDR parsing performance: Move CIDR range parsing to package-level
   variables instead of parsing on every isInternalIP call

LOW fixes:
1. Question organization: Move resolved questions (signature
   verification, namespace governance, version resolution) to new
   "Resolved Questions" section with [RESOLVED in ADR-0037] markers

2. Host header behavior: Add comment documenting that Host header
   uses original URL hostname (intentional for TLS SNI/cert validation)
   while DialContext pins to pre-validated IPs

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

MEDIUM fixes:
1. Default allowed domains: Remove cdn.fullsend.ai from DefaultPolicy
   and add note that default shipped config includes github.com/
   fullsend-ai/library but has no special privilege (user-editable)

2. Phase annotations: Add phase markers to code sections to clearly
   delineate Phase 1 (single-level) vs Phase 2+ (transitive) features:
   - Added "Phase 1/2+" markers to skill resolution code
   - Added phase comments to depth limit checks
   - Marked cycle detection tests as Phase 2+

LOW fix:
1. Misleading SSRF examples: Replace cdn.fullsend.ai with generic
   placeholders (example.org, myorg) in allowed_domains and
   allowed_remote_resources examples to avoid implying preferential
   treatment

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

fullsend review is working on this — view logs

Copy link
Copy Markdown
Contributor

@maruiz93 maruiz93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@ralphbean ralphbean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thorough and well-designed ADR + implementation plan. The security analysis is strong, the phased rollout is pragmatic, and the uniform trust model is the right call. A few issues to address before merging — mostly consistency between the ADR prose and the pseudocode.

Findings summary:

  • 4 requesting changes (cache key description, offline mode, resolver signature, cache permissions)
  • 1 noted/deferred (SHA algorithm agility)
  • 1 discussion point (script locality restriction)

See inline comments for details.

3. **Dependency confusion:** An attacker publishes a malicious skill at `https://attacker.com/skills/common-name` and tricks a harness into referencing it instead of the legitimate `https://fullsend.ai/skills/common-name`. **Mitigations:**
- Explicit URL references (no auto-resolution of names to URLs).
- User-controlled URL allowlists per organization (configurable in `config.yaml`). Fetches to URLs outside the allowlist are rejected.
- Mandatory hash pinning: The attacker cannot substitute content for an already-pinned URL.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[important] This line says "Every path field (agent, policy, skills[], host_files[].src, pre_script, post_script, etc.) accepts URLs" — but the Decision section (line 153) and the plan's resource type table explicitly restrict host_files, pre_script, post_script, and validation scripts to local-only.

This will confuse implementers. The sentence should exclude the local-only fields.


Discussion (non-blocking): More broadly — is the local-only restriction for scripts the right long-term call? The hash pinning + allowlist + CODEOWNERS protections that make remote declarative resources trustworthy apply equally to scripts. Sandboxing scripts doesn't really work either: post_script typically needs host access (push results, run validation, collect artifacts), so a sandbox with "no secrets, no network, no writes" makes it useless.

It's going to be really unsatisfying for users to define everything about an agent remotely except scripts — those still have to be copied everywhere. The "wall of text" diff problem on fullsend update that @ralphbean flagged remains unsolved for scripts.

Not blocking this PR on it, but worth a follow-up discussion on whether hash-pinned remote scripts (with the same allowlist/CODEOWNERS governance) should be permitted without sandboxing.

5. **Executable code from URLs:** Pre/post scripts fetched from URLs run on the runner host with full privileges. **Mitigation:** Apply **Option C** restriction: scripts and binaries must be local files. Only declarative resources (agents, skills, policies, schemas) can be URLs. **Alternative (future):** URL-sourced scripts could run in a restricted sandbox with no access to secrets, no network, and no filesystem writes outside `/tmp`. This requires designing an in-sandbox pre/post command execution mechanism (something like `pre_commands`/`post_commands` that run inside the sandbox before/after the agent's main execution). Today, `pre_script` and `post_script` run outside the sandbox. Any relaxation of the "scripts must be local" restriction depends on this prerequisite capability being implemented first.

6. **Runtime dependency discovery increases attack surface:** If agents can fetch resources at runtime based on dynamic input (e.g., "I need a Python linting skill for this repo"), an attacker can manipulate input to trigger fetch of a malicious resource. **Mitigations:**
- Runtime resource loading is opt-in per harness (disabled by default).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[important] This says "The cache key is SHA256(URL + hash)" but the implementation plan's CachePath function uses only the content's SHA256 hash as the cache key:

func CachePath(workspaceRoot, hash string) string {
    return filepath.Join(workspaceRoot, ".fullsend-cache", "resources", "sha256", hash)
}

The implementation is correct — content-addressed storage means two URLs with identical content share a cache entry. The ADR text should match: the cache key is SHA256(content), not SHA256(URL + hash).

}

logPath := filepath.Join(logDir, "fetches.jsonl")
f, err := os.OpenFile(logPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[important] This offline mode pseudocode rejects all remote references:

if offline && hasRemoteReferences(h) {
    return fmt.Errorf("harness references remote resources but --offline is set")
}

But both the ADR (line 310-311) and the plan prose (line 577) say offline mode should use cached resources and only fail on cache misses:

fullsend run --offline <harness> disables network fetches. If any required resource is not in cache, the run fails.

The pseudocode should pass an offline flag into the resolver, which attempts cache lookups and only errors on cache misses — not reject all remote references wholesale.

if h.Policy != "" {
resolved.PolicyPath, err = resolveResourceWithLimits(ctx, workspaceRoot, h.Policy, h.AllowedRemoteResources, policy, 0, &resourceCount)
if err != nil {
return nil, fmt.Errorf("resolving policy: %w", err)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] This function signature has no "parent context" parameter:

func resolveResourceWithLimits(ctx context.Context, workspaceRoot, ref string, ...)

The design (lines 100-149) describes resolving relative paths against a parent URL's base path for URL-sourced resources (e.g., ../policies/rust-sandbox.yaml relative to the skill's URL). But there's no way to pass that parent URL here.

Phase 2 will require a breaking signature change. Worth adding a parentRef string parameter (unused in Phase 1) or a resolverState struct now so the Phase 1 implementation doesn't need a breaking change later.

```yaml
# Generated by fullsend lock harness/code.yaml
# DO NOT EDIT - This file is auto-generated
version: 1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] Cache directories are created with 0755 and content files with 0644:

os.MkdirAll(dir, 0755)
os.WriteFile(..., content, 0644)

On shared runners, any user can read cached policies and agent definitions. These may contain organizational configuration. Should use 0700/0600 to restrict access to the owning user.

// Example: https://example.com/file.md#sha256=abc123... -> "abc123..."
// Returns an error if the hash is not a valid 64-character lowercase hex string.
func ParseIntegrityHash(rawURL string) (urlWithoutHash, hash string, hasHash bool) {
u, err := url.Parse(rawURL)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate, deferred] The design hardcodes SHA-256 throughout — URL fragment format (#sha256=...), hash validation (64 hex chars), cache paths (sha256/). SHA-256 is still secure, but since this is greenfield design with no backward compatibility constraints, consider supporting algorithm-agile prefixes from day one (e.g., #sha512=... as a future option). The URL fragment already namespaces with sha256=, so the extensibility point is there — ParseIntegrityHash and CachePath would just need to handle the algorithm prefix. This avoids a painful migration if the project ever needs to upgrade hash algorithms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants