Record how every screenshot scene was made

store_assets/screenshot-pipeline/README.md

@@ -110,6 +110,45 @@ English callout text (CAMERA, REMOTE, …) to the localized pill text.
   whatever `AVCaptureDevice.localizedName` returns, so a capture made against a
   USB webcam has to have that chip blanked.
 
+## Changing a scene
+
+Every image `manifest.js` renders traces back to a recorded recipe in
+`generate.mjs`, so a scene can be changed rather than reverse-engineered:
+
+```bash
+node generate.mjs derive            # list every recipe + coverage check
+node generate.mjs derive slot3_preview.jpg --as try.jpg   # remake, don't clobber
+```
+
+Three kinds of recipe:
+
+- **`PROMPTS`** — the base scenes. `CHOSEN` records which candidate the manifest
+  kept, since `node generate.mjs <id> N` writes `<id>_c1..cN`.
+- **`DERIVED`** — the "what the camera sees" previews and the edits that fixed a
+  scene's staging (a phone rotated to portrait, a feeder moved outside a window).
+- **crops** — exact boxes, recovered by matching each committed crop against its
+  parent.
+
+`node generate.mjs derive` with no argument also checks that every manifest
+reference has a recipe, and exits non-zero if one doesn't. Add a scene, add its
+recipe — otherwise the gap stays invisible until someone tries to change it.
+
+**The `DERIVED` prompts are reconstructions.** The originals were typed as argv
+strings and never recorded; these were rebuilt by reading the committed images.
+They produce an equivalent asset, not the same pixels — the model is not
+deterministic. Always `--as` and compare before overwriting.
+
+Worked example — swapping the cardinal for another animal:
+
+1. Edit the `slot3_ots`, `slot3_ipad` and `mac3_direct` prompts in `PROMPTS`.
+2. `node generate.mjs slot3_ots 3`, pick a candidate, update `CHOSEN`.
+3. Re-run the staging edits that sit on top (`slot3_ots_c1p`, `mac3_direct_e1`).
+4. Re-measure the quads — the phone lands somewhere new. `tools.py detect`, then
+   ALWAYS `tools.py overlay` to check.
+5. Re-run the previews (`slot3_preview`, `mac3_preview`, `mac3_preview_port`)
+   so the remote screens show the same animal the in-scene camera is pointed at.
+6. `./ship-locales.sh`.
+
 ## Adding a new screenshot (the Claude workflow)
 
 This pipeline was built with Claude Code and is easiest to extend the same way.
@@ -127,13 +166,15 @@ Example prompts that work well:
 What Claude does under the hood (or do it manually):
 
 1. **Generate the scene** — add a prompt to `generate.mjs` (`PROMPTS`), run
-   `node generate.mjs <sceneId> 2`. Scenes must show device screens *black/off*,
-   subjects in the upper two thirds, no logos (reject candidates with Apple logos).
-   Requires a Google AI Studio key in `AI_STUDIO`.
+   `node generate.mjs <sceneId> 2`, then record the candidate you keep in
+   `CHOSEN`. Scenes must show device screens *black/off*, subjects in the upper
+   two thirds, no logos (reject candidates with Apple logos). Requires a Google
+   AI Studio key in `AI_STUDIO`.
 2. **Derive the preview** — what the camera device sees, seeded from the scene
-   itself for consistency: `node generate.mjs edit "<scene>,<crop>" out.jpg "<prompt>"`.
-   Crop the subject with `python3 tools.py crop` and pass it as the second
-   reference so pose/orientation match exactly.
+   itself for consistency. Add a `DERIVED` entry and run
+   `node generate.mjs derive <out.jpg>`; use `edit` only while you are still
+   iterating on the wording. Crop the subject with `python3 tools.py crop` and
+   pass it as a second reference so pose/orientation match exactly.
 3. **Find the screen quads** — `python3 tools.py detect <scene> X0 Y0 X1 Y1`,
    then ALWAYS verify with `python3 tools.py overlay <scene> TLx TLy ... --out q.png`.
    AI-image device edges bow slightly: trust locally-measured corners over global

store_assets/screenshot-pipeline/generate.mjs

@@ -5,7 +5,14 @@
 // Usage:
 //   node generate.mjs <sceneId> [count]                 e.g. node generate.mjs slot0_v1 2
 //   node generate.mjs edit <inputImg> <outFile> <prompt> [aspect]
-//     image-to-image, e.g. deriving "what the camera phone sees" from a scene
+//     image-to-image, ad hoc. Anything the manifest ends up using should be
+//     recorded in DERIVED instead, so it can be remade.
+//   node generate.mjs derive [outFile]
+//     re-run a recorded derivation; no argument lists them all
+//
+// Every file manifest.js references traces back to one of three things: a
+// PROMPTS entry (with CHOSEN recording which candidate was kept), a DERIVED
+// entry, or a crop. Adding a scene means adding it here too — see README.
 import { readFileSync, writeFileSync, mkdirSync } from "node:fs";
 import { dirname, join } from "node:path";
 import { fileURLToPath } from "node:url";
@@ -118,6 +125,17 @@ rustic wooden bird feeder with a bright red cardinal perched on its ledge, and a
 second unbranded smartphone mounted on a small flexible tripod clamped right next
 to the feeder, its screen dark — both clearly recognizable, gently soft from
 distance. Morning light, dewy greens. ${STYLE}`,
+  slot3_ipad: `Over-the-shoulder shot from directly behind a woman standing on a
+cozy wooden porch on a bright morning. Her shoulder and hair are softly out of
+focus at the right edge of the near foreground. She holds an unbranded modern
+tablet upright in portrait orientation in one hand, screen facing the viewer,
+completely black and switched off, no reflections, shown large in the
+center-left of the frame. Beyond the porch railing, across the garden a few
+steps away, a rustic wooden bird feeder with a bright red cardinal perched on
+its ledge, and an unbranded smartphone mounted in PORTRAIT orientation on a
+small flexible tripod clamped right next to the feeder, its screen dark — both
+clearly recognizable, gently soft from distance. Morning light, dewy greens.
+${STYLE}`,
   slot4_watch: `Night scene, close-up of a raised wrist wearing an unbranded modern
 smartwatch with a rectangular rounded screen, completely black and off, facing the
 viewer, in the upper half of the frame. In the softly blurred background, an
@@ -138,6 +156,151 @@ the other hand. Both device screens dark and off. Airy daylight, white brick and
 wood floor. ${STYLE}`,
 };
 
+// Which candidate each base scene the manifest uses came from. `node
+// generate.mjs <id> N` writes <id>_c1..cN; this records which one was kept, so
+// a regenerated set can be compared against the same slot.
+const CHOSEN = {
+  slot0_ots: "slot0_ots_c1.jpg",
+  slot1_group: "slot1_group_c2.jpg",
+  slot3_ipad: "slot3_ipad_c1.jpg",
+  slot3_ots: "slot3_ots_c1.jpg",
+  slot4_watch: "slot4_watch_c1.jpg",
+  mac0_studio: "mac0_studio_c2.jpg",
+  mac2_cook: "mac2_cook_c1.jpg",
+  mac3_direct: "mac3_direct_c1.jpg",
+};
+
+// Everything the manifest uses that is NOT a raw candidate: the "what the
+// camera sees" previews, and the edits that fixed a scene's staging.
+//
+// IMPORTANT — these prompts are RECONSTRUCTED by reading the committed images,
+// not the verbatim text originally typed. That was passed as an argv string and
+// never recorded anywhere. They describe the same transformation and produce an
+// equivalent asset; they will NOT reproduce the committed file pixel for pixel,
+// because the model is not deterministic. Treat them as the recipe, not the
+// receipt. Crop entries, by contrast, are exact: their boxes were recovered by
+// matching the committed crop against its parent.
+const DERIVED = {
+  // ---- Scene edits ----
+  "slot3_ots_c1p.jpg": {
+    from: ["slot3_ots_c1.jpg"],
+    aspect: "9:16",
+    prompt: `Keep this photograph exactly as it is, with one change: rotate the
+small smartphone clamped to the flexible tripod beside the bird feeder so it
+stands in PORTRAIT orientation, taller than it is wide, its screen still facing
+the viewer and completely black and switched off. Do not change the woman, the
+phone in her hand, the feeder, the cardinal, the railing, the light, or the
+background.`,
+  },
+  "mac3_direct_e1.jpg": {
+    from: ["mac3_direct_c1.jpg"],
+    aspect: "9:16",
+    prompt: `Keep this room, window, laptop, desk and the woman exactly as they
+are, with two changes: move the wooden bird feeder and the cardinal perched on
+it OUTSIDE the window, into the garden behind the glass, so the window frame
+clearly separates them from the room; and rotate the smartphone clamped to the
+flexible tripod beside the feeder into PORTRAIT orientation. Every screen stays
+completely black and switched off.`,
+  },
+  "mac0_studio_e2.jpg": {
+    from: ["mac0_studio_c2.jpg"],
+    aspect: "9:16",
+    prompt: `Keep this studio scene exactly as it is and add the subject being
+photographed: on the white product table in front of the professional camera, a
+luxury wristwatch with a dark leather strap, being adjusted by a hand in a white
+cotton glove reaching in from the right. The camera on the tripod points at the
+watch. Every device screen stays completely black and switched off.`,
+  },
+
+  // ---- "What the camera sees" previews ----
+  // Each renders the scene's subject alone, from the camera device's position,
+  // at the aspect that device would actually deliver. Crop the subject out of
+  // the scene with `tools.py crop` and pass it as a second reference when the
+  // pose needs to match exactly.
+  "slot0_ots_preview.jpg": {
+    from: ["slot0_ots_c1.jpg"],
+    aspect: "9:16",
+    prompt: `Render only what the tripod-mounted camera phone in this scene is
+pointed at, as a clean full-frame product photograph: the luxury wristwatch
+standing on its small white plinth against the seamless white studio backdrop,
+softbox-lit. No devices, no people, no tripod in frame.`,
+  },
+  "slot1_preview.jpg": {
+    from: ["slot1_group_c2.jpg"],
+    aspect: "16:9",
+    prompt: `Render only what the tripod-mounted camera phone in this scene is
+pointed at, as a clean full-frame photograph: the whole multi-generation family
+sitting together on the picnic blanket, everyone looking at the camera, the
+picnic basket in front of them. Same warm golden light and string lights and
+greenery behind. No devices, no tripod in frame.`,
+  },
+  "slot3_preview.jpg": {
+    from: ["slot3_ots_c1p.jpg"],
+    aspect: "9:16",
+    prompt: `Render only what the tripod-mounted camera phone in this scene is
+pointed at, as a clean full-frame photograph shot on a telephoto lens: the
+rustic wooden bird feeder with the bright red cardinal perched on its ledge,
+framed TIGHT so the bird and the feeder ledge fill most of the frame and the
+bird is large enough to read at a glance. Same morning light, dewy greens,
+softly blurred garden behind. No devices, no people, no porch railing in
+frame.`,
+  },
+  "slot4_preview.jpg": {
+    from: ["slot4_watch_c1.jpg"],
+    aspect: "9:16",
+    prompt: `Render only what the phone clamped to the telescope eyepiece in this
+scene is pointed at, as a clean full-frame photograph: the bright full moon,
+sharp and detailed, filling much of the frame against a black night sky. No
+devices, no people, no telescope in frame.`,
+  },
+  "mac0_preview.jpg": {
+    from: ["mac0_studio_e2.jpg"],
+    aspect: "16:9",
+    prompt: `Render only what the professional camera on the tripod in this scene
+is pointed at, as a clean full-frame product photograph: the luxury wristwatch
+with the dark leather strap lying on the white product table, a hand in a white
+cotton glove adjusting it. Softbox-lit, bright and premium. No devices, no
+tripod in frame.`,
+  },
+  "mac2_preview.jpg": {
+    from: ["mac2_cook_c1.jpg"],
+    aspect: "16:9",
+    prompt: `Render only what the overhead phone on the under-cabinet arm in this
+scene is pointed at, as a clean full-frame overhead photograph shot straight
+down: the round wooden charcuterie board on the kitchen counter, loaded with
+cured meats, cheeses, figs, grapes, nuts, olives and edible flowers, with a hand
+reaching in to place a sprig of rosemary. No devices, no arm in frame.`,
+  },
+  "mac3_preview.jpg": {
+    from: ["mac3_direct_e1.jpg"],
+    aspect: "16:9",
+    prompt: `Render only what the phone clamped beside the feeder in this scene is
+pointed at, as a clean full-frame photograph: the rustic wooden bird feeder with
+the bright red cardinal perched on its ledge, framed TIGHT so the feeder and the
+bird fill most of the frame, softly blurred garden foliage behind. No devices,
+no window frame, no people in frame.`,
+  },
+  "mac3_preview_port.jpg": {
+    from: ["mac3_direct_e1.jpg"],
+    aspect: "9:16",
+    // Committed at 1295x2752 (aspect 0.47) rather than the 1536x2752 the model
+    // returns: that near-phone aspect is what makes the frame reach the top and
+    // bottom edges of a portrait screen in slots 2 and 2i. The trim was not
+    // recorded; a centred crop reproduces the committed framing.
+    crop: { from: "mac3_preview_port.jpg", box: [120, 0, 1415, 2752] },
+    prompt: `Render a tight close-up of what the phone clamped beside the feeder
+in this scene is pointed at, as a clean full-frame photograph: the bright red
+cardinal filling most of the frame, perched on the weathered wooden ledge with
+seed scattered around it, softly blurred green foliage and a wooden post behind.
+No devices, no window frame, no people in frame.`,
+  },
+
+  // ---- Deterministic crops (exact; boxes recovered from the committed files) ----
+  "mac2_cook_c1_crop.jpg": {
+    crop: { from: "mac2_cook_c1.jpg", box: [0, 440, 1536, 2210] },
+  },
+};
+
 async function callModel(parts, aspectRatio) {
   const res = await fetch(
     `https://generativelanguage.googleapis.com/v1beta/models/${MODEL}:generateContent`,
@@ -167,7 +330,49 @@ function save(file, inline) {
 
 mkdirSync(OUT_DIR, { recursive: true });
 
-if (process.argv[2] === "edit") {
+if (process.argv[2] === "derive") {
+  const want = process.argv[3];
+  if (!want) {
+    console.log("recorded derivations:\n");
+    for (const [out, d] of Object.entries(DERIVED)) {
+      const how = d.prompt ? `edit of ${d.from.join(", ")} @ ${d.aspect}` : "crop";
+      console.log(`  ${out.padEnd(26)} ${how}${d.crop && d.prompt ? " + crop" : ""}`);
+    }
+    console.log("\nbase scenes kept from PROMPTS:\n");
+    for (const [id, file] of Object.entries(CHOSEN)) console.log(`  ${file.padEnd(26)} node generate.mjs ${id}`);
+
+    // Coverage: every scene the manifest renders must be remakeable. Without
+    // this the gap is invisible until someone tries to change a scene and finds
+    // the recipe was never written down.
+    const manifest = readFileSync(join(here, "manifest.js"), "utf8");
+    const used = [...new Set(manifest.match(/\.\.\/ai-scenes\/[\w.-]+/g) || [])]
+      .map((p) => p.replace("../ai-scenes/", ""));
+    const known = new Set([...Object.keys(DERIVED), ...Object.values(CHOSEN)]);
+    const orphans = used.filter((f) => !known.has(f));
+    console.log(`\ncoverage: ${used.length - orphans.length}/${used.length} manifest scenes have a recipe`);
+    for (const f of orphans) console.log(`  NO RECIPE: ${f}`);
+    process.exit(orphans.length ? 1 : 0);
+  }
+  const d = DERIVED[want];
+  if (!d) { console.error(`no recorded derivation for ${want}; have: ${Object.keys(DERIVED).join(", ")}`); process.exit(1); }
+  // `--as <name>` writes somewhere else, so a regeneration can be compared
+  // against the committed asset before it replaces it. The model is not
+  // deterministic; you always want to look before you overwrite.
+  const asIdx = process.argv.indexOf("--as");
+  const outName = asIdx > 0 ? process.argv[asIdx + 1] : want;
+  if (d.prompt) {
+    const parts = d.from.map((f) => ({
+      inlineData: { mimeType: "image/jpeg", data: readFileSync(join(OUT_DIR, f)).toString("base64") },
+    }));
+    parts.push({ text: d.prompt });
+    save(join(OUT_DIR, outName), await callModel(parts, d.aspect));
+  }
+  if (d.crop) {
+    const [x0, y0, x1, y1] = d.crop.box;
+    // tools.py owns image manipulation; this script stays dependency-free.
+    console.log(`then crop:\n  python3 tools.py crop ../ai-scenes/${d.crop.from} ${x0} ${y0} ${x1} ${y1} ../ai-scenes/${want}`);
+  }
+} else if (process.argv[2] === "edit") {
   // input may be several comma-separated reference image paths
   const [input, outFile, prompt, aspect = "9:16"] = process.argv.slice(3);
   const parts = input.split(",").map((p) => ({