From 14e981e47e1f54dc797371c93239593333e16af3 Mon Sep 17 00:00:00 2001 From: myselfsiddharth Date: Fri, 14 Aug 2026 11:53:03 -0700 Subject: [PATCH 1/2] docs: annotate the recorder's agent claim and give the gate a locator asterisk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README's hero line said Paragent "records an agent's successful trajectory." Reading src/recorder/cli.ts and session.ts end to end confirms there is no agent: a developer types literal Playwright calls, and provenance.agent_model is "human" on every trajectory recorded so far (contracts/trajectory.schema.json already anticipates the distinction). - README.md: hero line now says "a developer's" trajectory (true today); the demo caption and the "How it works" table annotate the gap and point at the unbuilt agent-driven on-ramp (#127) instead of erasing the intent. - docs/gate/testbed.md: adds a second "Honesty" section, sibling to the existing version-bump-churn-as-proxy asterisk, recording that hand-picked locators make any gate survival number an upper bound, not an estimate — a developer reading the DOM picks differently than an agent acting at runtime. Cross-references docs/gate/recorder.md. - docs/gate/recorder.md: cross-references the new asterisk at the point where the ADR-0006 task's locators are chosen. - docs/INTEGRITY-AUDIT.md: adds A-11 for this claim-vs-reality gap, marked resolved by this PR, and bumps the category-A and total counts. No metric is invented or changed; this only adds caveats to existing, unmeasured gate numbers per CONTRIBUTING rules 3-4. Refs #127 Co-Authored-By: Claude Opus 5 --- README.md | 13 ++++++++----- docs/INTEGRITY-AUDIT.md | 7 ++++--- docs/gate/recorder.md | 13 ++++++++++++- docs/gate/testbed.md | 31 ++++++++++++++++++++++++++++++- 4 files changed, 54 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 41cc886..9c5aee2 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ a dashboard, and it re-reads the DOM, re-plans, and re-infers every click — bu tokens and wall-clock to rediscover a path it already found yesterday, with a fresh chance of getting it wrong. -Paragent takes the model out of the second run. It records an agent's **successful** trajectory +Paragent takes the model out of the second run. It records a developer's **successful** trajectory through a web UI, compiles it into a deterministic replayable script with a **post-condition assertion on every step**, and replays it with no model in the loop. When an assertion fails — a button moved, a label changed — the model is called back @@ -25,9 +25,12 @@ in to repair just that step, and the repaired script is what runs next time. Real run against the bundled fixture, driven by an illustrative script rather than the documented npm run commands. The record and replay beats are what - the repo ships. The repair beat is not: Paragent ships - StubRepairModelClient, which proposes nothing, so repair needs a model client - that is not yet wired (#27). + the repo ships — record today means a developer typing the trajectory by hand + (src/recorder/cli.ts), not an agent proposing it; that on-ramp is scoped but + not yet built (#127). The + repair beat is not: Paragent ships StubRepairModelClient, which proposes + nothing, so repair needs a model client that is + not yet wired (#27).

@@ -37,7 +40,7 @@ in to repair just that step, and the repaired script is what runs next time. | Stage | What happens | Model involved? | | --- | --- | --- | -| **Record** | An agent completes the task once. Every action and its post-condition is captured as a trajectory. | Yes | +| **Record** | A developer completes the task once, by hand. Every action and its post-condition is captured as a trajectory. | No | | **Compile** | The trajectory becomes a bundle of cache rows — one per step, each with its assertion. Typed values become parameter slots. | No | | **Replay** | Steps execute in order. Each asserts its post-condition before the next runs. | No | | **Repair** | On assertion failure, the model is called to fix that step; the updated script is written back. | Only on failure | diff --git a/docs/INTEGRITY-AUDIT.md b/docs/INTEGRITY-AUDIT.md index e92ac55..95c5bc2 100644 --- a/docs/INTEGRITY-AUDIT.md +++ b/docs/INTEGRITY-AUDIT.md @@ -4,7 +4,7 @@ doc_type: brief status: draft owner: D2 created: 2026-07-25 -updated: 2026-07-29 +updated: 2026-08-14 confidence: HIGH supersedes: null sources_verified: true @@ -18,12 +18,12 @@ D2 consolidator pass (2026-07-25), **after** C4 + C5 merges. **Surfaces conflict | Category | Count | | --- | --- | -| A. Unsourced / under-sourced claims | 10 | +| A. Unsourced / under-sourced claims | 11 | | B. Placeholder / unmeasured metrics | 8 | | C. LOW-confidence findings load-bearing in a decision | 6 | | D. Document disagreements (5 resolved) | 8 | | E. Documentation-standard violations (fixed or flagged) | 12 | -| **Total** | **44** | +| **Total** | **45** | --- @@ -41,6 +41,7 @@ D2 consolidator pass (2026-07-25), **after** C4 + C5 merges. **Surfaces conflict | A-08 | Proof-points F10 | Solo self-kill as diligence asset | Interpretation ASSUMED | | A-09 | Pitch Ask | Financing ask | Founder TBD | | A-10 | Any spoken gate % outside measured artifact | Performance | Must cite Track-1 output; none yet | +| A-11 | (was) Root README hero | "records an agent's successful trajectory" | **Resolved** — recorder is human-driven (`agent_model: "human"`), not agent-driven; README now says "developer's" plus a #127 annotation, and the locator-choice consequence is recorded in `docs/gate/testbed.md` / `recorder.md` | --- diff --git a/docs/gate/recorder.md b/docs/gate/recorder.md index c15c541..58e54bc 100644 --- a/docs/gate/recorder.md +++ b/docs/gate/recorder.md @@ -4,7 +4,7 @@ doc_type: runbook status: draft owner: B2 created: 2026-07-25 -updated: 2026-08-12 +updated: 2026-08-14 confidence: MED supersedes: null sources_verified: true @@ -232,6 +232,17 @@ it back. Version tolerance belongs in the preamble and nowhere else. Running the against a later version is therefore expected to exit **4** (`STEP NOT RECORDABLE`) naming the step — and nothing is written, because a hand-patched trajectory invalidates the gate. +**These locators are also hand-picked, and that is a second, separate asterisk on whatever +survival number the matrix eventually reports.** The 12 selectors above were chosen by a +developer reading the 9.5.21 DOM — picking `data-testid`/`aria-label` where one exists, falling +back to structure only where neither does (step 7's note above) — not proposed by an agent acting +at runtime with no such look-ahead. `provenance.agent_model` is `"human"` on this trajectory for +exactly that reason. See +[`docs/gate/testbed.md` § Honesty second](./testbed.md#honesty-second--hand-picked-locators-not-agent-picked) +for why that makes the gate number an upper bound, not an estimate, and +[issue #127](https://github.com/DevToolie/Paragent/issues/127) for the unbuilt agent-driven +on-ramp that would close it. + ### Two recordings, diffed Recorded three times, each on a **freshly created container** (`--down` then up, so the Grafana diff --git a/docs/gate/testbed.md b/docs/gate/testbed.md index b623004..fbae254 100644 --- a/docs/gate/testbed.md +++ b/docs/gate/testbed.md @@ -4,7 +4,7 @@ doc_type: spec status: draft owner: B1 created: 2026-07-25 -updated: 2026-07-28 +updated: 2026-08-14 confidence: HIGH supersedes: null sources_verified: true @@ -29,6 +29,35 @@ Do not present matrix results as “we measured real production churn.” Presen them as “we measured survival across known OSS console redesigns under accelerated upgrades.” Gate numbers derived here inherit this limitation. +## Honesty second — hand-picked locators, not agent-picked + +A second asterisk sits alongside the one above, and it attaches to the *recorded trajectory* +rather than to the matrix. `src/recorder/cli.ts` is a hand-written script: a developer reads the +live DOM and types literal Playwright locators, one per step (`RECORDED_AGAINST = "9.5.21"`, +[`docs/gate/recorder.md`](./recorder.md)). `contracts/trajectory.schema.json`'s own +`provenance.agent_model` field anticipates the distinction — "Model id used for the driving +agent, or 'human' for manual capture" — and every trajectory this repo has recorded carries +`"agent_model": "human"` (`contracts/examples/trajectory.example.json`). + +That matters because a developer and an agent do not pick locators the same way. A developer +reading the DOM can see which attributes are stable — `data-testid`, `aria-label`, structural +position — before typing a single selector, and picks accordingly. An agent recording the same +task at runtime sees only what is rendered in the moment; it has no equivalent look-ahead. A gate +that measures whether *hand-picked* locators survive a version bump is therefore measuring a +better-than-real case: the survival rate M2–M4 report is an upper bound on what an agent-recorded +trajectory would achieve, not an estimate of it. This is not a defect in the harness — it is a +property of who is recording, and it holds regardless of how the version-bump-churn proxy above +resolves. + +There is no fix in this repo yet. An agent-driven recorder is scoped as +[issue #127](https://github.com/DevToolie/Paragent/issues/127) and is not built; the checklist +item that would close this asterisk is "an agent-recorded number exists to compare against the +hand-recorded one." Until then, every gate number this harness produces carries **both** +asterisks — proxy churn (above) and hand-picked locators (here) — and neither should be dropped +when the number is quoted. See also [`docs/gate/recorder.md`](./recorder.md) for where the +locators in the committed trajectory were chosen and why they are deliberately not +version-tolerant. + ## Choice See [ADR-0003](../decisions/ADR-0003-testbed-grafana-oss.md). Target: From 7adef8af9f54715535a5e397d5c451d04ab5a9cb Mon Sep 17 00:00:00 2001 From: myselfsiddharth Date: Fri, 14 Aug 2026 11:57:19 -0700 Subject: [PATCH 2/2] docs: fix the two remaining agent claims the tagline and positioning left standing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review on PR #160 caught two spots the first pass missed: - README line 3, the bold tagline, still said "Record a browser agent once" — the same claim as the hero paragraph, contradicting the "a developer's" fix eleven lines below. Now "Record a browser task once," staying punchy with no disclaimer hung off it; the #127 detail already lives in the demo-caption annotation. - "Where this fits" ("record whatever agent you already trust, then replay its output") was flagged but left as an acknowledged scope call in the first pass. Rather than leave two contradicting copies of the claim in the file, it's now marked explicitly as the target, not what ships today, with the same #127 pointer. docs/INTEGRITY-AUDIT.md's A-11 row updated to name all four spots this PR touches, matching what "Resolved" now covers. Refs #127 Co-Authored-By: Claude Opus 5 --- README.md | 9 +++++---- docs/INTEGRITY-AUDIT.md | 2 +- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 9c5aee2..4b955c5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Paragent -**Record a browser agent once. Replay it deterministically, without the model. Repair it when the page changes.** +**Record a browser task once. Replay it deterministically, without the model. Repair it when the page changes.** [![CI](https://github.com/DevToolie/Paragent/actions/workflows/ci.yml/badge.svg)](https://github.com/DevToolie/Paragent/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE) @@ -103,9 +103,10 @@ persisted to disk. Full command list: [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) ## Where this fits Most browser-agent frameworks optimise the *first* run — better planning, better DOM -grounding, better recovery. Paragent assumes you already have one that works and -optimises every run after it. It composes with them rather than replacing them: record -whatever agent you already trust, then replay its output. +grounding, better recovery. Paragent is meant to compose with them rather than replace them: +record whatever agent you already trust, then replay its output. That's the target, not what +ships today — the recorder captures a developer's hand-typed actions, not an arbitrary agent's, +until [#127](https://github.com/DevToolie/Paragent/issues/127) lands. It's aimed at work that is **repeated**, in a **browser**, where **no clean API exists** — the cases where you'd write a script if the UI would just hold still. diff --git a/docs/INTEGRITY-AUDIT.md b/docs/INTEGRITY-AUDIT.md index 95c5bc2..940e3a5 100644 --- a/docs/INTEGRITY-AUDIT.md +++ b/docs/INTEGRITY-AUDIT.md @@ -41,7 +41,7 @@ D2 consolidator pass (2026-07-25), **after** C4 + C5 merges. **Surfaces conflict | A-08 | Proof-points F10 | Solo self-kill as diligence asset | Interpretation ASSUMED | | A-09 | Pitch Ask | Financing ask | Founder TBD | | A-10 | Any spoken gate % outside measured artifact | Performance | Must cite Track-1 output; none yet | -| A-11 | (was) Root README hero | "records an agent's successful trajectory" | **Resolved** — recorder is human-driven (`agent_model: "human"`), not agent-driven; README now says "developer's" plus a #127 annotation, and the locator-choice consequence is recorded in `docs/gate/testbed.md` / `recorder.md` | +| A-11 | (was) Root README tagline (L3), hero (L14), "How it works" table, "Where this fits" (L105-108) | "Record a browser agent once" / "records an agent's successful trajectory" / "record whatever agent you already trust" | **Resolved** — recorder is human-driven (`agent_model: "human"`), not agent-driven; all four spots now say so (tagline drops "agent," hero says "developer's," table says "A developer... by hand" / "No," positioning paragraph is marked target-not-shipped) with a #127 annotation, and the locator-choice consequence is recorded in `docs/gate/testbed.md` / `recorder.md` | ---