feat: add docs-review skill for documentation staleness detection

[csoceanu]

Add a new docs-review skill to the review agent that detects stale documentation caused by code changes. The skill builds an identifier checklist from every changed file, greps documentation for matches, and evaluates candidates in two passes (quick scan then deep read).

Changes:

• Create skills/docs-review/SKILL.md with 6-step process
• Add docs-review to review harness and agent definition
• Add documentation currency as 7th review dimension
• Update pr-review to delegate to docs-review after code-review

[fullsend-ai-review]

Review: #717

Head SHA: ce183f5
Timestamp: 2026-05-10T00:00:00Z
Outcome: comment-only

Summary

This PR cleanly adds a new docs-review skill to the review agent, introducing documentation currency as a 7th review dimension. The changes are internally consistent: the agent definition, harness config, pr-review skill, and new skill file all align correctly. Step renumbering and cross-references in pr-review are accurate. No security, injection, or correctness issues were found. One medium finding notes an inconsistency between the review agent's tools frontmatter and the docs-review skill's subagent dispatch requirement.

Findings

Critical

None.

High

None.

Medium

• [config-inconsistency] internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md:24 — The "Context management" section instructs the agent to "Dispatch a subagent to carry out the process below," but the review agent's tools frontmatter (agents/review.md) lists only Read, Grep, Glob, Bash and does not include Agent. Since Agent is not in disallowedTools, it is likely available at runtime, so the skill should function — but the tools list is incomplete and could mislead future contributors about the agent's actual capabilities. Consider either adding Agent to the review agent's tools list, or revising the context management section to describe the subagent dispatch as optional/conditional.

Low / Info

None.

Footer

Outcome: comment-only
This review applies to SHA ce183f558b367a06aa416c756145dd9cc700084a. Any push to the PR head clears this review and requires a new evaluation.

Previous run

Review: #717

Head SHA: b27626b
Timestamp: 2026-05-10T00:00:00Z
Outcome: approve

Summary

This PR cleanly adds a docs-review skill as a 7th review dimension, with well-designed integration into the review agent, harness config, and pr-review orchestration. The new skill follows established patterns (frontmatter, numbered process, constraints), includes sensible false-positive safeguards (two-pass evaluation, severity cap at high, exclusion of already-modified docs and changelogs), and all cross-references and step renumbering in pr-review are correct. No security, correctness, or injection concerns were found.

Findings

Low

• [style] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:3-6 — The frontmatter description field is not updated to mention docs-review delegation, while the body text (lines 11-13) and the agent definition's frontmatter description are both updated for consistency. Consider updating the pr-review frontmatter description to mention docs-review as well.

Footer

Outcome: approve
This review applies to SHA b27626b084e449fbe234408d1aadd24f19a432b0. Any push to the PR head clears this review and requires a new evaluation.

Previous run (2)

Review: #717

Head SHA: e4633fa
Timestamp: 2026-05-07T00:00:00Z
Outcome: comment-only

Summary

This PR cleanly adds a new docs-review skill for detecting documentation staleness caused by code changes, integrating it into the review agent definition, harness config, and pr-review orchestration. The skill design is well-structured with a clear six-step process, appropriate severity capping, and read-only constraints. One medium finding exists: the grep example in the new skill contradicts the filtering instructions in a prior step, which could lead agents to search vendored documentation files. No security, injection, or correctness issues were found.

Findings

Medium

• [correctness] internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md:96 — Step 4's grep example searches . (the entire repo tree) with --include patterns but does not add --exclude-dir flags for .git, vendor, node_modules. Step 3 instructs the agent to exclude these directories when discovering docs, but the step 4 example ignores that filtering. An agent following the example literally would grep through vendored or dependency markdown files, producing false-positive matches.
  Remediation: Add --exclude-dir flags to the example grep command: grep -rl "$id" --include="*.md" --include="*.rst" --include="*.adoc" --exclude-dir=.git --exclude-dir=vendor --exclude-dir=node_modules . 2>/dev/null

Info

• [style] internal/scaffold/fullsend-repo/agents/review.md:3 — The frontmatter description field still reads "Reviews for correctness, security, intent alignment, and style" and does not mention the new documentation currency dimension. Consider updating to reflect the expanded scope.

Footer

Outcome: comment-only
This review applies to SHA e4633fa5557a57f1e1b35929480afcfbb6ef32d4. Any push to the PR head clears this review and requires a new evaluation.

Previous run (3)

Review: #717

Head SHA: 87e4dac
Timestamp: 2026-05-07T00:00:00Z
Outcome: comment-only

Summary

The PR adds a well-structured docs-review skill with a clear six-step process (identifier extraction, doc discovery, grep, two-pass evaluation, findings) and integrates it correctly into the review agent definition, harness, and pr-review orchestration skill. The skill routing, dimension numbering, step renumbering, and finding aggregation across steps are all consistent. One medium finding: the pr-review constraint section now requires docs-review completion before posting, but step 4 defines skip conditions where docs-review is intentionally not run — these two instructions contradict each other and could cause the agent to emit a spurious failure or ignore the constraint.

Findings

Critical

(none)

High

(none)

Medium

• [instruction-conflict] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md — The constraint "Never post without completing the code-review and docs-review skills first" (added at the end of the Constraints section) contradicts step 4's skip conditions ("Skip this step if: docs-only PRs, reverts, >200 files"). When a docs-only PR reaches step 4, the skip conditions say don't run docs-review, but the constraint says the review is invalid without completing it. An LLM agent following these instructions literally could either (a) run docs-review despite the skip condition, defeating its purpose, or (b) refuse to post and emit a failure, producing a false negative. The constraint needs to account for legitimate skips.
  Remediation: Qualify the constraint to acknowledge step 4 skip conditions, e.g., "Never post without completing the code-review skill and, when applicable per step 4, the docs-review skill."

Low

(none)

Info

(none)

Footer

Outcome: comment-only
This review applies to SHA 87e4dac41289ae1d41534e8ca40117cd40bbd563. Any push to the PR head clears this review and requires a new evaluation.

[ralphbean]

So, this might blow the context window of the agent. At the very top of the skill, WDYT about adding an exhortation to encourage the agent to spin up a sub-agent to carry out this entire skill?

[csoceanu]

@ralphbean - Great suggestion! Added a "Context management" section at the top of the skill encouraging the agent to dispatch a subagent

[github-actions]

Site preview

Preview: https://ef94a6f5-site.fullsend-ai.workers.dev

Commit: 20831f571ec5bdf41056ef5355930fcdd3b0a412

[github-actions]

fullsend review is working on this — view logs

[github-actions]

fullsend review is working on this — view logs