docs: annotate the recorder's agent claim and give the gate a locator asterisk - #160
Conversation
… asterisk The README's hero line said Paragent "records an agent's successful trajectory." Reading src/recorder/cli.ts and session.ts end to end confirms there is no agent: a developer types literal Playwright calls, and provenance.agent_model is "human" on every trajectory recorded so far (contracts/trajectory.schema.json already anticipates the distinction). - README.md: hero line now says "a developer's" trajectory (true today); the demo caption and the "How it works" table annotate the gap and point at the unbuilt agent-driven on-ramp (#127) instead of erasing the intent. - docs/gate/testbed.md: adds a second "Honesty" section, sibling to the existing version-bump-churn-as-proxy asterisk, recording that hand-picked locators make any gate survival number an upper bound, not an estimate — a developer reading the DOM picks differently than an agent acting at runtime. Cross-references docs/gate/recorder.md. - docs/gate/recorder.md: cross-references the new asterisk at the point where the ADR-0006 task's locators are chosen. - docs/INTEGRITY-AUDIT.md: adds A-11 for this claim-vs-reality gap, marked resolved by this PR, and bumps the category-A and total counts. No metric is invented or changed; this only adds caveats to existing, unmeasured gate numbers per CONTRIBUTING rules 3-4. Refs #127 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Opus review — changes requested (1 must-fix)Reviewed the full diff and verified the claims against HEAD independently. What holds up. The substantive contribution here is the Must fix — the tagline still makes the claim this PR corrects
This is the same claim as "records an agent's successful trajectory," in bold, eleven lines above the sentence the PR did fix to "a developer's." As it stands the file answers the question two different ways depending on where you stop reading, and the more prominent answer is the wrong one. The checklist item is that the README's claim be true or annotated; a fix that leaves the headline copy asserting it isn't finished. Also fix —
|
…left standing Review on PR #160 caught two spots the first pass missed: - README line 3, the bold tagline, still said "Record a browser agent once" — the same claim as the hero paragraph, contradicting the "a developer's" fix eleven lines below. Now "Record a browser task once," staying punchy with no disclaimer hung off it; the #127 detail already lives in the demo-caption annotation. - "Where this fits" ("record whatever agent you already trust, then replay its output") was flagged but left as an acknowledged scope call in the first pass. Rather than leave two contradicting copies of the claim in the file, it's now marked explicitly as the target, not what ships today, with the same #127 pointer. docs/INTEGRITY-AUDIT.md's A-11 row updated to name all four spots this PR touches, matching what "Resolved" now covers. Refs #127 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Addressed both findings from review, pushed as a second commit (7adef8a): Finding 1 (must fix) — README line 3 tagline now reads "Record a browser task once. Replay it deterministically, without the model. Repair it when the page changes." Dropped "agent" entirely rather than hanging a disclaimer off the tagline — the #127 detail already lives in the demo-caption annotation below, and the tagline no longer asserts anything false. Finding 2 — "Where this fits" ("record whatever agent you already trust, then replay its output") is kept as the positioning statement but now explicitly marked as the target, not what ships today: "That's the target, not what ships today — the recorder captures a developer's hand-typed actions, not an arbitrary agent's, until #127 lands." Chose to annotate rather than rewrite since the paragraph's point (composability with whatever agent framework you already trust) is legitimate roadmap positioning, not a present-tense factual claim once marked as such. Also updated the INTEGRITY-AUDIT.md A-11 row to name all four spots now covered (tagline, hero, "How it works" table, "Where this fits") since the scope of what "Resolved" covers changed. Re-ran both gates after the fix:
Grepped the full README afterward for any remaining bare "agent" claims about Paragent's own recording capability — the only other hits are line 9 ("Browser agents re-derive the same work on every run," describing the general problem class) and line 133 ("agent, a contributor, a candidate, or an investor," addressing the reader), neither of which claims Paragent itself records via an agent today. Not merging — still Refs #127, issue stays open. |
Opus re-review — approvedBoth findings fixed and verified against the pushed branch.
Swept the file myself: no remaining claim that Paragent records via an agent. Required checks green ( Merging via admin bypass — #127 stays open for the agent-driven recorder itself. |
Summary
Ships the two unblocked "honesty" items from #127's checklist. Issue #127 is size/L and scoped as an agent-driven recorder — that is not built here. This PR is documentation-only: it makes the README's recorder claim true and records the gate-validity consequence it names. The issue stays open (
Refs #127, notCloses) for the agent-driven recorder itself.docs/gate/notes the hand-picked-locator asterisk on any pre-existing gate number"Verified against HEAD first
Read
src/recorder/cli.tsandsrc/recorder/session.tsend to end, as instructed. The issue's description still holds:src/recorder/cli.tsis a hand-written script — a developer types literal Playwright calls, one per step, with hardcoded selectors and a hardcodedTASK_KEY. There is no model client in the recording path. Confirmed independently by the contract itself:contracts/trajectory.schema.json'sprovenance.agent_modelfield is described as "Model id used for the driving agent, or 'human' for manual capture," and every trajectory recorded so far — including the committed ADR-0006 example — carries"agent_model": "human".One thing has changed since the issue was filed: #151 landed
AnthropicRepairModelClient(src/runner/repair-anthropic.ts), a real repair client. That's the repair beat, not record, andStubRepairModelClientis still the default insrc/runner/replay.ts. It doesn't touch this PR's scope and I left the README's existing repair (#27) annotation as-is.What changed and why
README.md— the hero line said Paragent "records an agent's successful trajectory." It now says "a developer's," which is true today. Per the task's guidance not to turn the landing page into a disclaimer, I kept the hero line itself short and put the roadmap annotation where the README already annotates gaps between the demo and shipped reality — the caption under the demo GIF, right next to the existing repair (#27) note, extended to explain thatrecordtoday means a developer typing the trajectory by hand and that an agent-driven on-ramp is scoped but not built (#127). The "How it works" table's Record row and "Model involved?" column were the same false claim in tabular form directly below the fixed prose (agent / "Yes") — left uncorrected it would have contradicted the hero line one screen down, so it's now "a developer... by hand" / "No".docs/gate/testbed.md— this is the substantive point: a gate measuring whether hand-picked locators survive churn is measuring a better-than-real case, because a developer reading the DOM picks stable selectors on sight and an agent acting at runtime has no equivalent look-ahead. Added a new## Honesty second — hand-picked locators, not agent-pickedsection immediately after the existing## Honesty first — proxy, not organic churnsection, so a reader of one caveat lands on the other. States plainly that any survival number M2–M4 report is an upper bound, not an estimate, and that both asterisks travel with the number until #127 closes this one. Cross-referencesdocs/gate/recorder.md.docs/gate/recorder.md— also relevant (it's where the ADR-0006 task's 12 locators are actually chosen and where the doc already explains why they're deliberately not version-tolerant), so I added a short paragraph right there noting the locators are hand-picked too, withprovenance.agent_model: "human"as the receipt, and a link back to the new testbed.md section for the full argument. Splitting it this way avoided duplicating the whole argument in two files.docs/INTEGRITY-AUDIT.md— this tracks exactly this category of claim-vs-reality gap, so I added row A-11 (bumping the category-A and total counts from 10/44 to 11/45), following the existing "(was) ... Resolved" format used elsewhere in that table (e.g. D-02–D-08), since this PR resolves the gap in the same change that logs it.No
src/files touched, no gate number invented or altered — only caveats added to numbers that are alreadyno_data/unmeasured, per CONTRIBUTING rules 3–4.Test plan
npm run lint:docs— clean (61 docs)npm run ci— green (secret-scan, validate:contracts, lint, lint:docs, typecheck, 453 unit tests, 26 integration tests)🤖 Generated with Claude Code