feat(autoreview): add reference support by coygeek · Pull Request #23 · openclaw/agent-skills

coygeek · 2026-06-11T22:29:09Z

Closes #24

Summary

add autoreview reference inputs for local paths, Git repositories, references files, hidden references, and strict reference mode
wire reference access through Codex, Claude, OpenCode, and prompt-manifest fallback modes
add reference fixture coverage to the review harness and document the workflow in the autoreview skill
harden reference handling for local-path safety, hidden metadata redaction, dry-run behavior, pinned Git refs, file reference materialization, and alias validation

Verification

python3 skills/autoreview/scripts/autoreview --help
python3 skills/autoreview/scripts/autoreview --self-test-config-defaults
python3 skills/autoreview/scripts/autoreview --self-test-engine-isolation
python3 skills/autoreview/scripts/autoreview --self-test-opencode-isolation
python3 skills/autoreview/scripts/autoreview --self-test-json-array-parser
python3 skills/autoreview/scripts/autoreview --self-test-references
skills/autoreview/scripts/test-review-harness --reference-fixtures
python3 -m compileall -q skills/autoreview/scripts
scripts/validate-skills
git diff --check -- skills/autoreview/SKILL.md skills/autoreview/scripts/autoreview skills/autoreview/scripts/test-review-harness.py skills/autoreview/scripts/test-review-harness skills/autoreview/scripts/test-review-harness.ps1
python3 skills/autoreview/scripts/autoreview --mode branch --base origin/main --engine codex --model gpt-5.5 --thinking high reported clean with no accepted/actionable findings

Notes

Optional ruby scripts/install-skills.test.rb still fails on the pre-existing Ruby 2.6 assert_path_exists incompatibility in scripts/install-skills.test.rb.

Add shared reference parsing, validation, JSONC loading, prompt-manifest rendering, and metadata scaffolding for the autoreview helper.\n\nThe new contract keeps dataset inputs file-only while allowing validated read-only path and Git references for later engine wiring.

Route normalized review references through engine-specific access paths: OpenCode generated config references, Claude and Codex add-dir support, and prompt-manifest fallback for prompt-only engines. Preserve existing isolation defaults while making strict reference mode fail before review execution when native directory access is unavailable.

Document review references in the public autoreview skill and add deterministic fake-engine harness coverage for native and prompt-manifest reference modes.

Replace Path.write_text(newline=...) with an explicit open call so the reference fixture harness works on the Python runtime used by this repo.\n\nThis coordinator fix was found by rerunning the PLAN-03 reference fixture gate before integration.

Fix accepted final autoreview findings by resolving existing local reference paths before Git shorthand parsing and by preventing AUTOREVIEW_STRICT_REFERENCES from overriding explicit CLI reference sources.\n\nAdds focused self-test coverage for relative local references and strict env precedence.

Reject exact broad filesystem roots before granting native reference access so reviewers cannot be handed overly broad local directories. Redact hidden reference details from normal run metadata while keeping explicit dry-run output useful for diagnostics.

Native directory modes now only receive references that materialize to directories. File references remain valid prompt-manifest references and strict native reference mode rejects them instead of passing unusable file paths to add-dir style engines.

Wrap all post-materialization setup and validation in the reference workspace cleanup guard so strict-mode, target-selection, reviewer-option, dry-run, and review execution exits all remove cloned reference repositories.

Dry-run now parses and validates git reference specs without cloning them, preserving the inspection-only contract while real review paths still materialize references before execution.

Dry-run now treats unmaterialized git references as intended native directories for mode planning without cloning them. Reference path safety also rejects every .env-prefixed filename so secret-bearing dotenv variants cannot be exposed as review context.

Git references pinned to commit SHAs now fall back to clone, fetch, and checkout when branch-style shallow clone cannot resolve the ref. The reference self-test covers immutable commit pins using a local fixture repository.

Hidden references are now rendered alias-only in reviewer prompts, with explicit instructions not to repeat hidden locations or descriptions. The reference self-test verifies hidden git references do not leak repository, runtime path, or description details.

Local file references now contribute bounded prompt content instead of only an inaccessible path. Native-capable engines can combine embedded file references with native directory references, while actual add-dir arguments still include only directories.

Local file references now grant native access to their containing directory while retaining bounded prompt excerpts. Validation rejects file references whose parent directory is too broad, preserving the strict-reference guarantee without passing file paths to add-dir.

Local file references are copied into per-alias temp directories before native engine access is granted. This preserves strict-reference completeness without exposing unrelated sibling files from the original directory.

Reference aliases now require an alphanumeric character and reject dot segments before they are used as materialization path components. The reference self-test covers the path traversal case.

Reference dry-runs now report planned Codex native directory mode without invoking codex exec --help. The reference self-test covers previewing strict git references with a missing Codex binary.

clawsweeper · 2026-06-11T22:30:04Z

Codex review: needs real behavior proof before merge. Reviewed June 11, 2026, 6:38 PM ET / 22:38 UTC.

Summary
Review failed before ClawSweeper could summarize the requested change.

Reproducibility: unclear. The review failed before ClawSweeper could establish a reproduction path.

Review metrics: none identified.

Merge readiness
Overall: 🌊 off-meta tidepool
Proof: 🌊 off-meta tidepool
Patch quality: 🌊 off-meta tidepool
Result: rating does not apply to this item.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] No close action taken because the review did not complete.

Maintainer options:

Decide the mitigation before merge
Retry the Codex review after fixing the execution failure.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] Review did not complete, so no work-lane recommendation was made.

Review details

Best possible solution:

Retry the Codex review after fixing the execution failure.

Do we have a high-confidence way to reproduce the issue?

Unclear. The review failed before ClawSweeper could establish a reproduction path.

Is this the best way to solve the issue?

Unclear. Retry the review first so ClawSweeper can evaluate the actual issue and fix direction.

AGENTS.md: unclear because the file could not be read completely.

Codex review notes: model internal, reasoning high; reviewed against 720c704bfe92.

Label changes

Label justifications:

rating: 🌊 off-meta tidepool: Overall readiness is 🌊 off-meta tidepool; proof is 🌊 off-meta tidepool and patch quality is 🌊 off-meta tidepool.

Evidence reviewed

What I checked:

failure reason: retryable codex transport failure.
codex failure detail: Codex review failed for this PR with exit 1.
codex stderr: es not include a built-in permission system for restricting filesystem, process, network, or credential access and runs with the permissions of the launching process. The current autoreview Pi runner already uses isolation flags such as --no-approve, --no-session, --no-context-files, --no-extensions, --no-skills, --no-prompt-templates, and --no-themes, then runs from a temporary directory.\n\nFor Pi, reference support should be prompt/tool based:\n\n- Include a reference manifest with aliases, paths, descriptions, and read-only instructions.\n- Prefer absolute resolved paths.\n- Keep current isolation flags.\n- Do not imply that Pi enforces reference-only access; it does not.\n- Add a smoke test that a Pi review prompt can inspect a referenced path when tools are enabled.\n\n### Droid: cwd/worktree support plus hook-based context injection; no native reference catalog found\n\nFactory Droid docs show:\n\n- droid exec --cwd <path> executes from a specific working directory.\n- droid exec -f <prompt.md> loads a prompt from file.\n- droid exec supports JSON/stream-JSON output formats.\n- Droid hooks can inject additional context through UserPromptSubmit and SessionStart.\n- Hook output can include hookSpecificOutput.additionalContext.\n- Hooks have access to cwd and project-root environment variables.\n\nI did not find a first-class OpenCode-style reference catalog in the Droid docs checked.
codex stdout: No stdout captured.

Likely related people:

unknown: Codex failed before it could trace repository history. (role: review did not complete; confidence: low)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

coygeek · 2026-06-11T22:33:17Z

Related issue: #24

steipete · 2026-06-16T02:53:20Z

@clawsweeper re-review

Default-branch exact-review capacity fix is now landed in openclaw/clawsweeper@c88270b. Triggering one normal exact review to verify repository_dispatch -> capacity gate -> Codex review end to end.

clawsweeper · 2026-06-16T02:53:23Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Failed
Detail: The targeted re-review timed out or failed while waiting for exact-review Codex capacity. Retry after capacity recovers.
Run: https://github.com/openclaw/clawsweeper/actions/runs/27590892083
Updated: 2026-06-16T03:15:12.906Z

coygeek added 20 commits June 11, 2026 14:10

merge: integrate autoreview reference model

da05c2a

merge: integrate autoreview engine reference modes

ba04645

test(autoreview): add reference fixture harness docs

bba756e

Document review references in the public autoreview skill and add deterministic fake-engine harness coverage for native and prompt-manifest reference modes.

merge: integrate autoreview reference docs and harness

9aaa6a1

fix(autoreview): clean reference workspace on early exits

84e7ac9

Wrap all post-materialization setup and validation in the reference workspace cleanup guard so strict-mode, target-selection, reviewer-option, dry-run, and review execution exits all remove cloned reference repositories.

fix(autoreview): keep dry-run reference checks side-effect free

eb834be

Dry-run now parses and validates git reference specs without cloning them, preserving the inspection-only contract while real review paths still materialize references before execution.

fix(autoreview): materialize pinned git references

6dc44fd

Git references pinned to commit SHAs now fall back to clone, fetch, and checkout when branch-style shallow clone cannot resolve the ref. The reference self-test covers immutable commit pins using a local fixture repository.

fix(autoreview): isolate native file references

2dbb796

Local file references are copied into per-alias temp directories before native engine access is granted. This preserves strict-reference completeness without exposing unrelated sibling files from the original directory.

fix(autoreview): reject unsafe reference aliases

04dd189

Reference aliases now require an alphanumeric character and reject dot segments before they are used as materialization path components. The reference self-test covers the path traversal case.

fix(autoreview): avoid codex probes in dry-run

aff0c65

Reference dry-runs now report planned Codex native directory mode without invoking codex exec --help. The reference self-test covers previewing strict git references with a missing Codex binary.

coygeek mentioned this pull request Jun 11, 2026

Add reference-directory support to autoreview across OpenCode, Claude Code, Codex, Pi, and Droid #24

Open

clawsweeper Bot added the rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. label Jun 11, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(autoreview): add reference support#23

feat(autoreview): add reference support#23
coygeek wants to merge 20 commits into
openclaw:mainfrom
coygeek:autoreview-reference-support

coygeek commented Jun 11, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 11, 2026 •

edited

Loading

Uh oh!

coygeek commented Jun 11, 2026

Uh oh!

steipete commented Jun 16, 2026

Uh oh!

clawsweeper Bot commented Jun 16, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

coygeek commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Notes

Uh oh!

clawsweeper Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coygeek commented Jun 11, 2026

Uh oh!

steipete commented Jun 16, 2026

Uh oh!

clawsweeper Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coygeek commented Jun 11, 2026 •

edited

Loading

clawsweeper Bot commented Jun 11, 2026 •

edited

Loading

clawsweeper Bot commented Jun 16, 2026 •

edited

Loading