You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(skill): noun-phrase guide titles, demo voice robust to prompt wording
Tier 1 + Tier 2 evals on Sonnet surfaced two skill gaps:
- guide titles: the example told agents to write "How to X", an
instructional sentence that contradicts the line-115 noun-phrase
rule. Replaced with a noun-phrase example + explicit counter-example.
- demo voice: when the user said "walkthrough" / "show what's in it"
instead of "Loom", agents dropped to flat third-person product copy.
The run-type table called the demo voice merely "presenter"; the
Step 4 block calls it "Loom recording". Aligned the table, and added
that the voice is fixed by the run type, not by prompt wording.
- both verifier gates now check voice_matches_register, not just
voice_consistent — four uniformly-flat captions used to pass.
guide seed 0.91 -> 0.93; fallback demo seed 0.76 fail -> 0.91 pass.
@@ -222,6 +222,8 @@ Read every caption out loud in your head before shipping it. TTS will say what y
222
222
223
223
Voice: someone who just shipped something and hit record on Loom to show you. Casual. First-person ("I'll show you," "we just shipped"). Acknowledges the viewer ("watch this," "you can see"). Mild editorial reactions allowed ("which I really like," "took us a few iterations"). Contractions throughout. **Do not** sound like marketing copy.
224
224
225
+
This voice is fixed by the run type, **not** by how the user phrased the request. "Demo it like a quick walkthrough," "just show what's in it," "give me a tour" — all still get the full Loom voice. A flat request never licenses flat narration: if the run type is `demo`, the captions are first-person and casual, period. Neutral product narration ("The form contains three fields") is a failure, not a safe default.
226
+
225
227
Example clip (use this as a reference, not as a template — write fresh for the actual content):
226
228
227
229
```
@@ -287,7 +289,7 @@ When you didn't pick a type or none fit cleanly, use `demo`. The Loom voice read
287
289
288
290
-**demo:***"New signup flow"*, *"Signup, redesigned"*, *"Walking through the new signup"*. **Not***"Signup flow on staging"* — that leaks the QA frame into a demo title.
289
291
-**qa:***"Project-create flow QA on staging"*, *"Checkout smoke test"*, *"Login regression"*. Engineer-readable is correct here — the audience is another engineer.
290
-
-**guide:***"How to inspect internet traffic with Cloudflare Radar"*, *"Linear vs Notion: project switching"*. Often phrased as "How to X" or "X vs Y."
292
+
-**guide:***"ISP-level attribution with Cloudflare Radar"*, *"Linear vs Notion: project switching"*. Name the payoff or the comparison — still a noun phrase. **Not***"How to inspect internet traffic with Cloudflare Radar"* — that's an instructional sentence, not a title.
291
293
-**bug:***"Login regression repro"*, *"Rate-limit 503 on signup"*. Tight, blunt, names the bug.
292
294
293
295
### Universal narration tips (apply to every type)
@@ -321,6 +323,7 @@ prompt: |
321
323
322
324
Step 1. Enumerate every distinct UI element visible in the frame. Don't write a sentence; list them.
323
325
Step 2. Decide whether the caption (a) names something *not* obvious from the frame's literal contents (good), or (b) reads as a description of what's visible (bad — that's the screenshot's job). For qa, additionally require an explicit verdict (worked/passed/broken/clean). For bug, require factual past-tense framing. For guide, require a specific finding, not a UX observation.
326
+
Step 2b. Check the caption's voice register against the run type. For demo, the caption must sound like a person narrating a Loom — first-person and/or direct viewer address, contractions, casual. A demo caption written as detached third-person product copy ("The form contains three fields," "The submit button is disabled") fails here even if it lands a point — set matches=false and rewrite it in voice.
324
327
Step 3. Check the caption against the banned-phrase list: "welcome to", "today we'll", "I'm excited to show", "in this walkthrough", "I am now clicking", "let me navigate", "as we can see", "observe how", "notice that", "in conclusion", "seamless", "robust", "leverage", "powerful", "intuitive". Case-insensitive substring.
325
328
Step 4. If it fails on any axis, write a one-line replacement caption in the run-type's voice that does land a point. Stay close to the original facts.
326
329
@@ -367,6 +370,7 @@ prompt: |
367
370
368
371
Check:
369
372
- voice_consistent: does one voice run through intro + every caption? If even one slide breaks register (analyst voice in a demo clip, court-reporter past tense in a present-tense run), flag it.
373
+
- voice_matches_register: do the captions actually land the voice in the definition above — not just match each other? Captions that are uniformly flat still fail this. A demo clip whose captions read as detached third-person product copy ("The form contains three fields") is wrong even if all N captions are wrong the same way. The intro and outro carrying the voice does not cover for flat captions. Flag every caption that doesn't carry the run-type voice.
370
374
- opener_shape: does the description avoid banned openers ("welcome to", "today we'll", "I'm excited to show", "in this walkthrough", "A 30-second look at...", and meta-framing in general)?
371
375
- banned_phrases: scan title, description, and every caption against the universal banned-phrase list. Return every match.
372
376
- spine_adherence: does the clip actually deliver on the picked spine? If three slides chase a different thread, flag it.
@@ -375,6 +379,7 @@ prompt: |
375
379
Return ONLY JSON matching this schema:
376
380
{
377
381
"voice_consistent": true | false,
382
+
"voice_matches_register": true | false,
378
383
"voice_issues": ["slide N: reason", ...],
379
384
"opener_shape_ok": true | false,
380
385
"opener_issue": "..." | null,
@@ -386,7 +391,7 @@ prompt: |
386
391
```
387
392
388
393
**Acting on the result:**
389
-
-`voice_consistent: false`or `banned_phrases_found` nonempty → call `slideshow_update_slide` on each flagged slide with a rewritten caption that fixes the issue. One pass — don't loop.
394
+
-`voice_consistent: false`, `voice_matches_register: false`, or `banned_phrases_found` nonempty → call `slideshow_update_slide` on each slide named in `voice_issues` with a rewritten caption that fixes the issue. One pass — don't loop.
390
395
-`opener_shape_ok: false` → call `slideshow_create`'s update path or, if no such path, accept that the description is locked at create time and note in the final report that next clip should pick a better opener.
391
396
- Always use `suggested_summary` as the input to `slideshow_set_summary` unless you have a strong reason not to. The reviewer wrote it in the right voice; trust it.
0 commit comments