Skip to content

docs: annotate the recorder's agent claim and give the gate a locator asterisk - #160

Merged
myselfsiddharth merged 2 commits into
mainfrom
track1/b4-recorder-annotation
Aug 14, 2026
Merged

docs: annotate the recorder's agent claim and give the gate a locator asterisk#160
myselfsiddharth merged 2 commits into
mainfrom
track1/b4-recorder-annotation

Conversation

@myselfsiddharth

Copy link
Copy Markdown
Contributor

Summary

Ships the two unblocked "honesty" items from #127's checklist. Issue #127 is size/L and scoped as an agent-driven recorder — that is not built here. This PR is documentation-only: it makes the README's recorder claim true and records the gate-validity consequence it names. The issue stays open (Refs #127, not Closes) for the agent-driven recorder itself.

  • "README's 'records an agent's successful trajectory' is either true or annotated"
  • "docs/gate/ notes the hand-picked-locator asterisk on any pre-existing gate number"

Verified against HEAD first

Read src/recorder/cli.ts and src/recorder/session.ts end to end, as instructed. The issue's description still holds: src/recorder/cli.ts is a hand-written script — a developer types literal Playwright calls, one per step, with hardcoded selectors and a hardcoded TASK_KEY. There is no model client in the recording path. Confirmed independently by the contract itself: contracts/trajectory.schema.json's provenance.agent_model field is described as "Model id used for the driving agent, or 'human' for manual capture," and every trajectory recorded so far — including the committed ADR-0006 example — carries "agent_model": "human".

One thing has changed since the issue was filed: #151 landed AnthropicRepairModelClient (src/runner/repair-anthropic.ts), a real repair client. That's the repair beat, not record, and StubRepairModelClient is still the default in src/runner/replay.ts. It doesn't touch this PR's scope and I left the README's existing repair (#27) annotation as-is.

What changed and why

README.md — the hero line said Paragent "records an agent's successful trajectory." It now says "a developer's," which is true today. Per the task's guidance not to turn the landing page into a disclaimer, I kept the hero line itself short and put the roadmap annotation where the README already annotates gaps between the demo and shipped reality — the caption under the demo GIF, right next to the existing repair (#27) note, extended to explain that record today means a developer typing the trajectory by hand and that an agent-driven on-ramp is scoped but not built (#127). The "How it works" table's Record row and "Model involved?" column were the same false claim in tabular form directly below the fixed prose (agent / "Yes") — left uncorrected it would have contradicted the hero line one screen down, so it's now "a developer... by hand" / "No".

docs/gate/testbed.md — this is the substantive point: a gate measuring whether hand-picked locators survive churn is measuring a better-than-real case, because a developer reading the DOM picks stable selectors on sight and an agent acting at runtime has no equivalent look-ahead. Added a new ## Honesty second — hand-picked locators, not agent-picked section immediately after the existing ## Honesty first — proxy, not organic churn section, so a reader of one caveat lands on the other. States plainly that any survival number M2–M4 report is an upper bound, not an estimate, and that both asterisks travel with the number until #127 closes this one. Cross-references docs/gate/recorder.md.

docs/gate/recorder.md — also relevant (it's where the ADR-0006 task's 12 locators are actually chosen and where the doc already explains why they're deliberately not version-tolerant), so I added a short paragraph right there noting the locators are hand-picked too, with provenance.agent_model: "human" as the receipt, and a link back to the new testbed.md section for the full argument. Splitting it this way avoided duplicating the whole argument in two files.

docs/INTEGRITY-AUDIT.md — this tracks exactly this category of claim-vs-reality gap, so I added row A-11 (bumping the category-A and total counts from 10/44 to 11/45), following the existing "(was) ... Resolved" format used elsewhere in that table (e.g. D-02–D-08), since this PR resolves the gap in the same change that logs it.

No src/ files touched, no gate number invented or altered — only caveats added to numbers that are already no_data/unmeasured, per CONTRIBUTING rules 3–4.

Test plan

  • npm run lint:docs — clean (61 docs)
  • npm run ci — green (secret-scan, validate:contracts, lint, lint:docs, typecheck, 453 unit tests, 26 integration tests)

🤖 Generated with Claude Code

… asterisk

The README's hero line said Paragent "records an agent's successful
trajectory." Reading src/recorder/cli.ts and session.ts end to end confirms
there is no agent: a developer types literal Playwright calls, and
provenance.agent_model is "human" on every trajectory recorded so far
(contracts/trajectory.schema.json already anticipates the distinction).

- README.md: hero line now says "a developer's" trajectory (true today);
  the demo caption and the "How it works" table annotate the gap and point
  at the unbuilt agent-driven on-ramp (#127) instead of erasing the intent.
- docs/gate/testbed.md: adds a second "Honesty" section, sibling to the
  existing version-bump-churn-as-proxy asterisk, recording that hand-picked
  locators make any gate survival number an upper bound, not an estimate —
  a developer reading the DOM picks differently than an agent acting at
  runtime. Cross-references docs/gate/recorder.md.
- docs/gate/recorder.md: cross-references the new asterisk at the point
  where the ADR-0006 task's locators are chosen.
- docs/INTEGRITY-AUDIT.md: adds A-11 for this claim-vs-reality gap, marked
  resolved by this PR, and bumps the category-A and total counts.

No metric is invented or changed; this only adds caveats to existing,
unmeasured gate numbers per CONTRIBUTING rules 3-4.

Refs #127

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@myselfsiddharth
myselfsiddharth requested a review from a team as a code owner August 14, 2026 18:53
@github-actions github-actions Bot added the size/M <= 200 changed lines label Aug 14, 2026
@github-actions
github-actions Bot requested a review from OM152002 August 14, 2026 18:53
@github-actions github-actions Bot added documentation Improvements or additions to documentation gate PRD section 9 gate measurement labels Aug 14, 2026
@myselfsiddharth

Copy link
Copy Markdown
Contributor Author

Opus review — changes requested (1 must-fix)

Reviewed the full diff and verified the claims against HEAD independently.

What holds up. The substantive contribution here is the docs/gate/testbed.md asterisk, and the argument is right: a developer reading the DOM picks stable attributes on sight, an agent recording at runtime has no equivalent look-ahead, so a survival number over hand-picked locators is an upper bound rather than an estimate. Siting it directly after the existing proxy-churn caveat is the correct call — a reader who finds one finds the other. The provenance.agent_model: "human" receipt is the strongest available evidence and I confirmed it: contracts/trajectory.schema.json:90 defines the field as "Model id used for the driving agent, or 'human' for manual capture", and contracts/examples/trajectory.example.json:10 carries "human". Splitting the argument (full version in testbed.md, pointer from recorder.md where the 12 locators are actually chosen) avoids a duplicate that would drift. Anchor link resolves.

Must fix — the tagline still makes the claim this PR corrects

README.md:3, unchanged by this PR:

Record a browser agent once. Replay it deterministically, without the model. Repair it when the page changes.

This is the same claim as "records an agent's successful trajectory," in bold, eleven lines above the sentence the PR did fix to "a developer's." As it stands the file answers the question two different ways depending on where you stop reading, and the more prominent answer is the wrong one. The checklist item is that the README's claim be true or annotated; a fix that leaves the headline copy asserting it isn't finished.

Also fix — README.md:108

It composes with them rather than replacing them: record whatever agent you already trust, then replay its output.

Third copy of the claim, present tense. The author flagged this one and left it on scope grounds, which was a fair reading of the instruction — but internal consistency in a single file beats a minimal diff. This one sits closer to legitimate positioning than line 3 does, so annotating it with the #127 pointer is acceptable if a rewrite would cost the paragraph its meaning.

Not blocking

INTEGRITY-AUDIT.md's A-11 row uses the (was) … **Resolved** form borrowed from the D rows rather than the plain form the other A rows use. Precedent exists (D-02–D-08) and the count bump 10→11 / 44→45 is arithmetically right, so I'm leaving it — but if the tagline fix changes what A-11 claims was resolved, that row's text needs to follow.

Verdict: hold for the tagline fix, then merge. Everything else is sound and CI is green.

…left standing

Review on PR #160 caught two spots the first pass missed:

- README line 3, the bold tagline, still said "Record a browser agent
  once" — the same claim as the hero paragraph, contradicting the "a
  developer's" fix eleven lines below. Now "Record a browser task once,"
  staying punchy with no disclaimer hung off it; the #127 detail already
  lives in the demo-caption annotation.
- "Where this fits" ("record whatever agent you already trust, then
  replay its output") was flagged but left as an acknowledged scope call
  in the first pass. Rather than leave two contradicting copies of the
  claim in the file, it's now marked explicitly as the target, not what
  ships today, with the same #127 pointer.

docs/INTEGRITY-AUDIT.md's A-11 row updated to name all four spots this PR
touches, matching what "Resolved" now covers.

Refs #127

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@myselfsiddharth

Copy link
Copy Markdown
Contributor Author

Addressed both findings from review, pushed as a second commit (7adef8a):

Finding 1 (must fix) — README line 3 tagline now reads "Record a browser task once. Replay it deterministically, without the model. Repair it when the page changes." Dropped "agent" entirely rather than hanging a disclaimer off the tagline — the #127 detail already lives in the demo-caption annotation below, and the tagline no longer asserts anything false.

Finding 2 — "Where this fits" ("record whatever agent you already trust, then replay its output") is kept as the positioning statement but now explicitly marked as the target, not what ships today: "That's the target, not what ships today — the recorder captures a developer's hand-typed actions, not an arbitrary agent's, until #127 lands." Chose to annotate rather than rewrite since the paragraph's point (composability with whatever agent framework you already trust) is legitimate roadmap positioning, not a present-tense factual claim once marked as such.

Also updated the INTEGRITY-AUDIT.md A-11 row to name all four spots now covered (tagline, hero, "How it works" table, "Where this fits") since the scope of what "Resolved" covers changed.

Re-ran both gates after the fix:

  • npm run lint:docs — clean (61 docs)
  • npm run ci — green (secret-scan, validate:contracts, lint, lint:docs, typecheck, 453 unit tests, 26 integration tests)

Grepped the full README afterward for any remaining bare "agent" claims about Paragent's own recording capability — the only other hits are line 9 ("Browser agents re-derive the same work on every run," describing the general problem class) and line 133 ("agent, a contributor, a candidate, or an investor," addressing the reader), neither of which claims Paragent itself records via an agent today.

Not merging — still Refs #127, issue stays open.

@myselfsiddharth

Copy link
Copy Markdown
Contributor Author

Opus re-review — approved

Both findings fixed and verified against the pushed branch.

  • Tagline (README.md:3) now reads "Record a browser task once" — drops the claim entirely rather than hanging a disclaimer off the repo's one-line pitch. Correct call.
  • "Where this fits" keeps the composability positioning (which is legitimate roadmap intent, not a false claim about today) and marks it "That's the target, not what ships today" with the Trajectories are hand-written, not agent-recorded — run 1 costs a developer-day and zero tokens #127 pointer. Right resolution for that one.
  • A-11 was widened to name all four sites, so the audit row now matches what was actually resolved.

Swept the file myself: no remaining claim that Paragent records via an agent. Required checks green (lint-typecheck-test-secrets, privacy-canary, conventional-commit title).

Merging via admin bypass — #127 stays open for the agent-driven recorder itself.

@myselfsiddharth
myselfsiddharth merged commit 1ee5009 into main Aug 14, 2026
12 checks passed
@myselfsiddharth
myselfsiddharth deleted the track1/b4-recorder-annotation branch August 14, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation gate PRD section 9 gate measurement size/M <= 200 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant