diff --git a/bin/native/darwin-arm64/loaf b/bin/native/darwin-arm64/loaf index 14e7eb7e..05a6a6f2 100755 Binary files a/bin/native/darwin-arm64/loaf and b/bin/native/darwin-arm64/loaf differ diff --git a/content/skills/loaf-reference/SKILL.md b/content/skills/loaf-reference/SKILL.md index 2761a397..2e214cfe 100644 --- a/content/skills/loaf-reference/SKILL.md +++ b/content/skills/loaf-reference/SKILL.md @@ -61,10 +61,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive | diff --git a/dist/amp/skills/loaf-reference/SKILL.md b/dist/amp/skills/loaf-reference/SKILL.md index 1b417bb2..bc0e384d 100644 --- a/dist/amp/skills/loaf-reference/SKILL.md +++ b/dist/amp/skills/loaf-reference/SKILL.md @@ -67,10 +67,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive | diff --git a/dist/codex/skills/loaf-reference/SKILL.md b/dist/codex/skills/loaf-reference/SKILL.md index 1b417bb2..bc0e384d 100644 --- a/dist/codex/skills/loaf-reference/SKILL.md +++ b/dist/codex/skills/loaf-reference/SKILL.md @@ -67,10 +67,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive | diff --git a/dist/cursor/skills/loaf-reference/SKILL.md b/dist/cursor/skills/loaf-reference/SKILL.md index 1b417bb2..bc0e384d 100644 --- a/dist/cursor/skills/loaf-reference/SKILL.md +++ b/dist/cursor/skills/loaf-reference/SKILL.md @@ -67,10 +67,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive | diff --git a/dist/opencode/skills/loaf-reference/SKILL.md b/dist/opencode/skills/loaf-reference/SKILL.md index 1b417bb2..bc0e384d 100644 --- a/dist/opencode/skills/loaf-reference/SKILL.md +++ b/dist/opencode/skills/loaf-reference/SKILL.md @@ -67,10 +67,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive | diff --git a/dist/skills/loaf-reference/SKILL.md b/dist/skills/loaf-reference/SKILL.md index 331f4adb..bf8ef066 100644 --- a/dist/skills/loaf-reference/SKILL.md +++ b/dist/skills/loaf-reference/SKILL.md @@ -66,10 +66,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive | diff --git a/docs/changes/20260710-journal-reliability-foundation/research/u8-claude-code-2.1.215-candidate-smoke.json b/docs/changes/20260710-journal-reliability-foundation/research/u8-claude-code-2.1.215-candidate-smoke.json index 30fde737..859d4f49 100644 --- a/docs/changes/20260710-journal-reliability-foundation/research/u8-claude-code-2.1.215-candidate-smoke.json +++ b/docs/changes/20260710-journal-reliability-foundation/research/u8-claude-code-2.1.215-candidate-smoke.json @@ -1,6 +1,6 @@ { "evidence_version": 2, - "timestamp": "2026-07-19T15:09:50.471Z", + "timestamp": "2026-07-19T20:41:33.994Z", "target": "claude-code", "surface": "cli", "version": "2.1.215", @@ -41,7 +41,7 @@ "stderr_empty": true, "model_visible_marker_observed": true, "assistant_marker_match": true, - "marker": "LOAF_U8_SMOKE_71B170407D8B", + "marker": "LOAF_U8_SMOKE_1E69C06FCEDC", "hook_observation": { "event_name": "SessionStart:startup", "native_json": true, @@ -52,6 +52,6 @@ "hooks_path": "plugins/loaf/hooks/hooks.json", "hooks_sha256": "5c7343a34af37d6c6e930981f71e405b8023975de4146822058f98c15251b7c2", "native_binary_path": "plugins/loaf/bin/native/darwin-arm64/loaf", - "native_binary_sha256": "949936e1c17cde73cfdea9b717e2f431854aebcb1641a947286dd2eb9b8a5ee7" + "native_binary_sha256": "0aa8f3f4babf64a9eb34dbcba9fb90b0f09b7978e6e2d6eadb395d7636908bd9" } } diff --git a/docs/changes/20260710-journal-reliability-foundation/research/u8-codex-0.144.5-isolated-smoke.json b/docs/changes/20260710-journal-reliability-foundation/research/u8-codex-0.144.5-isolated-smoke.json index 0412f8df..04543657 100644 --- a/docs/changes/20260710-journal-reliability-foundation/research/u8-codex-0.144.5-isolated-smoke.json +++ b/docs/changes/20260710-journal-reliability-foundation/research/u8-codex-0.144.5-isolated-smoke.json @@ -1,6 +1,6 @@ { "evidence_version": 2, - "timestamp": "2026-07-19T15:09:58.049Z", + "timestamp": "2026-07-19T20:41:22.121Z", "target": "codex", "surface": "cli", "version": "0.144.5", @@ -40,7 +40,7 @@ "stderr": "Reading additional input from stdin...", "model_visible_marker_observed": true, "assistant_marker_match": true, - "marker": "LOAF_CODEX_U8_9BDAC4356EBF", + "marker": "LOAF_CODEX_U8_45E98BA70001", "hook_observation": { "event_name": "SessionStart:startup", "native_json": true, @@ -51,6 +51,6 @@ "hooks_path": "dist/codex/.codex/hooks.json", "hooks_sha256": "0f9e3feb7204c3309a7db6a02224ad881f6801a8cdd126ec6222e6d9a804c33d", "native_binary_path": "bin/native/darwin-arm64/loaf", - "native_binary_sha256": "949936e1c17cde73cfdea9b717e2f431854aebcb1641a947286dd2eb9b8a5ee7" + "native_binary_sha256": "0aa8f3f4babf64a9eb34dbcba9fb90b0f09b7978e6e2d6eadb395d7636908bd9" } } diff --git a/docs/changes/20260710-journal-reliability-foundation/research/u8-opencode-1.18.3-isolated-smoke.json b/docs/changes/20260710-journal-reliability-foundation/research/u8-opencode-1.18.3-isolated-smoke.json index 57accdf8..103ce642 100644 --- a/docs/changes/20260710-journal-reliability-foundation/research/u8-opencode-1.18.3-isolated-smoke.json +++ b/docs/changes/20260710-journal-reliability-foundation/research/u8-opencode-1.18.3-isolated-smoke.json @@ -1,6 +1,6 @@ { "evidence_version": 2, - "timestamp": "2026-07-19T15:10:10.630Z", + "timestamp": "2026-07-19T20:41:39.037Z", "target": "opencode", "surface": "cli", "version": "1.18.3", @@ -39,11 +39,11 @@ "root_session_lookup_proven": true, "no_auth_supplied": true, "cleanup_succeeded": true, - "marker": "LOAF_OPENCODE_U8_38E9391C6EB4", + "marker": "LOAF_OPENCODE_U8_870D0055A736", "candidate_artifacts": { "hooks_path": "dist/opencode/plugins/hooks.ts", "hooks_sha256": "7b4e2b359bd1c9904fabe6e6778082084bae794e757e7a383fabddbcddf857e0", "native_binary_path": "bin/native/darwin-arm64/loaf", - "native_binary_sha256": "949936e1c17cde73cfdea9b717e2f431854aebcb1641a947286dd2eb9b8a5ee7" + "native_binary_sha256": "0aa8f3f4babf64a9eb34dbcba9fb90b0f09b7978e6e2d6eadb395d7636908bd9" } } diff --git a/docs/changes/20260719-path-stability-and-origin-hygiene/change.md b/docs/changes/20260719-path-stability-and-origin-hygiene/change.md new file mode 100644 index 00000000..aa8f50f4 --- /dev/null +++ b/docs/changes/20260719-path-stability-and-origin-hygiene/change.md @@ -0,0 +1,156 @@ +--- +change: path-stability-and-origin-hygiene +created: 2026-07-19 +branch: path-stability-and-origin-hygiene +--- + + + +# Path Stability and Origin Hygiene + +## Problem + +Codex's post-ship review of intent-exploration-foundation surfaced four defects that make installed Loaf policy and diagnostics untrustworthy between releases. First, the Codex installer resolves the `loaf` PATH entry through `EvalSymlinks` into the versioned Homebrew Cellar target, so every rendered surface — `CODEX_HOME/AGENTS.md` guidance, the SessionStart hook, and every execpolicy prefix in `loaf.rules` — is invalidated by every release, and Codex tasks already running with injected policy strand on paths that no longer exist. Second, the Intent, Exploration, and legacy-conversion writers introduced relationship origin values (`intent-create`, `exploration-create`, `legacy-conversion`) that `doctor` rejects — its invariant allows only `imported`/`manual`/`command` — so a healthy schema-12 database permanently warns on ten real rows, and the recommended `state repair relationship-origin` matches zero of them because it only backfills missing origins. The writer, validator, and repair plan each hold a private copy of the vocabulary and have drifted. Third, live help disagrees with implemented commands: `loaf state migrate --help` lists three of six dispatchable sources, and leaf subcommands like `loaf conversation handle add --help` fail with `unknown option`, undermining the maintenance skill's discover-syntax-from-live-help rule. Fourth, `journal context` still presents five historically-resolved blockers as unresolved and its newest project synthesis is from June — closure facts were never written. + +## Hypothesis + +Rendering the stable PATH entrypoint instead of its versioned resolution, closing the origin vocabulary behind one registry that writers, doctor, and repair all consume, and restoring help parity makes installed Codex policy survive upgrades and makes doctor and live help trustworthy authorities again — hardening the exact surfaces `change-native-execution-migration` will build on, before it is shaped. + +## Scope + +**In** + +- A single relationship-origin registry in `internal/state` feeding the writers, the doctor invariant SQL, and `state repair relationship-origin`; the Intent, Exploration, and legacy-conversion writers normalize to `command`; repair gains a reclassify mode that migrates existing unknown-origin rows backup-first; doctor-clean tests cover Intent creation, Exploration creation, and legacy conversion. +- Codex-managed rendering (AGENTS.md guidance, hooks, execpolicy prefixes) uses the un-canonicalized PATH entrypoint (e.g. `/opt/homebrew/bin/loaf`); `EvalSymlinks` canonicalization is retained for forbidden-roots validation only; `loaf install --upgrade` repins existing installs. +- Help-surface parity: `state migrate` help lists all six sources with a dispatcher/help parity test; leaf subcommands (`conversation handle add`, and the Exploration conversation equivalent) honor `-h`/`--help`. +- Real-state remediation as Definition of Done evidence: reclassify the live unknown-origin rows, write `unblock` closures for the five stale blockers, and write a fresh `wrap(project)` synthesis. + +**Out** (deferred, not rejected) + +- Retained cross-context Exploration dogfood — not a code change; tracked as an Intent that gates shaping of `change-native-execution-migration` (candidate topic: the release-finalization-in-worktree-topologies question discovered 2026-07-19). +- Handoff and research artifact authority (`.agents/handoffs/`, `.agents/reports/` vs SQLite) — owned by the terminal guidance/artifact sweep, per the successor protocol. +- Structural staleness semantics for block/task/handoff/wrap surfaces (expiry, closure prompts) — terminal sweep; this Change only writes the missing closure facts. +- Non-Codex harness adapters for execpolicy-style trust — the central registry already anticipates them; nothing is claimed here. + +**Cut** (explicitly rejected) + +- Rendering literal `loaf` and trusting PATH resolution in execpolicy prefixes — it would authorize any earlier-on-PATH binary named `loaf`, surrendering the known-binary posture Decision 19 of journal-reliability-foundation established. +- Restoring versioned absolute paths with a repin-on-upgrade workflow — the review's core complaint (every release invalidates rendered policy and strands running tasks) would remain by design. +- Expanding the origin vocabulary with per-ceremony values — an open-ended registry recreates the drift class this Change exists to kill; operation detail belongs in `reason`. + +## Observable Workflow + +```text +$ loaf install --upgrade # Codex files now carry the stable entrypoint +$ grep -c Cellar ~/.config/codex/rules/loaf.rules +0 +$ brew upgrade loaf # next release lands +$ grep loaf ~/.config/codex/hooks.json # rendered command still resolves — nothing strands + +$ loaf state repair relationship-origin --dry-run + 10 relationship row(s) with unknown origin would be reclassified to 'command' +$ loaf state repair relationship-origin --apply # backup-first +$ loaf doctor # zero relationship-origin diagnostics + +$ loaf state migrate --help # lists markdown, storage-home, schema, + # lifecycle-statuses, journal-first, deferrals +$ loaf conversation handle add --help # prints leaf usage, exit 0 +``` + +## Rabbit Holes and No-Gos + +- Do not redesign the execpolicy trust model or add new basic leaves — only change which path string is rendered; classification and gating stay byte-identical. +- Do not touch installed-distribution-authority's resolver (PR #120) — `EvalSymlinks` remains correct for finding the distribution root; this Change only changes what is rendered into Codex-managed files. +- Do not build staleness detection, blocker expiry, or wrap reminders — that is terminal-sweep territory; here, only the missing closure facts get written. +- Do not migrate the seven pre-existing `command`/`imported` writers or invent a writer-side enforcement framework beyond the registry and its tests. +- Do not grow repair into a general origin-rewriting tool — reclassify maps exactly the three named legacy values to `command`; foreign origins stay visible as warnings, never silently rewritten. + +## Decisions + +Provenance: accepted 2026-07-19 during hotfix shaping, interviewed against Codex's post-ship review of intent-exploration-foundation; PATH rendering and origin vocabulary were explicit user decisions, the rest shaped autonomously. Decision 4 was amended during implementation (journaled `decision(state)` 2026-07-19) when the source-literal parity scan exposed a fourth legacy origin. + +1. **Codex-managed surfaces render the stable PATH entrypoint; canonicalization is validation-only.** This amends Decision 19 of journal-reliability-foundation, which pinned "the canonical absolute Loaf executable" into rendered prefixes. The un-canonicalized `LookPath` result (`/opt/homebrew/bin/loaf`) is absolute and survives upgrades because Homebrew repoints the symlink; `EvalSymlinks` still runs to enforce forbidden-roots and existence checks against the real target. Forecloses literal-`loaf` PATH trust and versioned Cellar pins alike. Journaled as `decision(codex-policy)` 2026-07-19. Trust boundary, stated for H1: rendered policy now trusts wherever the entrypoint symlink points at execution time, not a fixed target — validation gates install/upgrade time (both the entrypoint and its canonical target are checked against forbidden roots), and a post-install retarget of the symlink is out of scope by design, exactly as a post-install replacement of the Cellar binary was under the old pin. Both models trust the writability of a user-owned Homebrew prefix; the amendment trades a version-frozen path for upgrade continuity without narrowing what an attacker with prefix write access could already do. +2. **The relationship-origin vocabulary is closed at mechanism level: `imported`, `manual`, `command`.** `origin` answers "by what mechanism did this row appear"; `reason` carries the operation ("recorded by intent create"). The three new writers normalize to `command`, matching every pre-existing CLI writer. Forecloses per-ceremony origin values and the open-ended registry they imply. Journaled as `decision(state)` 2026-07-19. Implementation note: the parity scan (Decision 3) found the closed vocabulary already breached beyond the shaped three — `run.go` and `finding.go` inlined origin `system` on five run/finding/verdict relationship writers; all five were normalized to `command` in this Change. +3. **One registry, three consumers, and the invariant is executable.** Writers, the doctor SQL, and repair all derive from a single Go registry, and each state-creating ceremony gets a doctor-clean-after-writer test — the only structure that prevents recurrence when the next ceremony lands. +4. **Repair reclassifies the named legacy values; it does not invent.** `state repair relationship-origin` keeps its missing-origin backfill and adds reclassification of exactly `intent-create`, `legacy-conversion`, `exploration-create`, and `system` to `command` — dry-run first, backup-first on apply, idempotent on rerun. Bare invocation (no `--origin`) is reclassification-only, which is what the workflow above runs; the backfill requires an explicit `--origin imported|manual` because inventing missing provenance is an operator judgment, while reclassifying the named legacy values is not. `system` was amended in during implementation: the parity scan exposed it as a shipped writer-invented origin (run/finding ceremonies in released alphas), so user databases need the same reclassify path; the local database was confirmed clean of it (its ten unknowns are `legacy-conversion`×8 + `intent-create`×2). Genuinely foreign origins keep warning rather than being laundered into `command`; surfacing them is doctor's job. The live row count is treated as a floor, not an exact target — any Intent/Exploration ceremony run before this ships adds rows the repair must also catch. +5. **Journal staleness is remediated as data, not solved as code.** Five `unblock` closures and one `wrap(project)` synthesis are Definition of Done evidence; the structural fix is named for the terminal sweep. +6. **Exploration dogfood is a successor gate, not hotfix scope.** A tracked Intent requires one real Exploration spanning two conversations/harnesses, resumed from its checkpoint, before `change-native-execution-migration` is shaped — honoring the reviewer's sequencing without padding this hotfix. + +## Planning Contract + +### Approach — origin registry (U1) + +Add the registry to `internal/state` (single source: allowed origins plus the doctor `NOT IN` clause built from it). Change the three INSERT literals — `intent.go` (`'intent-create'`), `intent_conversion.go` (`'legacy-conversion'`), `exploration.go` (`'exploration-create'`) — to `'command'`. Extend `inspectRelationshipOriginInvariants` in `status.go` to build its allowed set from the registry. Extend the `relationship-origin` repair plan: dry-run reports each of the three named legacy origins with count and target value; apply reclassifies them to `command` after the standard backup, and reruns are no-ops; origins outside the named set are reported but never rewritten. Tests: doctor-clean after Intent create, Exploration create, and legacy conversion on fresh fixtures; a registry parity test asserting every origin literal written by `internal/state` code is registry-listed; repair fixture seeded with all three legacy values proving dry-run match, apply, idempotency, and post-repair doctor cleanliness. + +### Approach — Codex stable-path rendering (U2) + +Split `trustedCodexJournalExecutable` into resolution and rendering concerns: validation continues to canonicalize (`EvalSymlinks` + `Abs`) and enforce forbidden roots and guidance-character checks against the real target; the returned render path becomes the absolute un-canonicalized `LookPath` result. All rendered surfaces — AGENTS.md fenced guidance, `hooks.json` command, `loaf.rules` prefixes — consume the render path. Unit tests use a fake symlink layout (entrypoint symlink → versioned target) asserting rendered content carries the entrypoint, validation still rejects forbidden-root targets, and retargeting the symlink (simulated upgrade) leaves previously rendered files valid without rewrite. `loaf install --upgrade` repins existing installs through the existing digest-owned managed-content merge; the isolated `CODEX_HOME` runtime smoke re-runs to prove model-visible startup is unchanged. + +### Approach — help parity (U3) + +`writeStateMigrateHelp` lists all six sources; a parity test asserts the dispatcher's source set and the help text agree, so the next source cannot ship unlisted. Leaf subcommand help: route `-h`/`--help` before unknown-option rejection for `conversation handle add` and the Exploration conversation equivalent, with tests asserting exit 0 and usage output. + +### Risks + +- **Digest ownership on upgrade.** Rendered Codex files are digest-owned; the merge policy refuses unowned or locally modified content. Repinning must flow through the recognized-ownership path — preflight reads the existing ownership tests before touching the renderer. +- **Real-state repair timing.** The repair ships in the next alpha; running it against the live database with a stale installed binary is the known stale-binary hazard. The DoD run uses the checkout binary explicitly or waits for the released binary, and says which. +- **`~/.local/bin` copies.** Where the PATH entry is a real file (launcher + `native/`), render path and canonical path coincide — behavior is unchanged by construction, but a test pins it. +- **Row-count drift.** Every Intent/Exploration ceremony before ship adds unknown-origin rows (this shaping itself may add one via Intent creation). Repair and its tests target the pattern, never a hardcoded count. + +### Adjacent-surface notes + +Two edits sit adjacent to the unit letter and are deliberate, not drift. Bare `loaf migrate --help` (top level) was fixed alongside U3's contracted surfaces because it fails the identical discover-syntax-from-live-help rule the Problem names and was flagged in this Change's own review round; it received the same registry-derived help treatment and a parity test. The three u8 smoke-evidence files under `docs/changes/20260710-journal-reliability-foundation/research/` were regenerated because capability evidence pins the native binary's SHA-256 and fails the suite on drift by design; regeneration ran exclusively through the sanctioned smoke writers (as in the installed-distribution-authority Change), and the diffs carry only hash, timestamp, and marker. + +### Sequencing + +U1 and U2 are independent and can proceed in parallel; U3 trails as mechanical cleanup. Real-state DoD steps (repair apply, closures, wrap, live repin) run after implementation lands, with the release. The gating Intent for Exploration dogfood is created during shaping, before implementation starts. + +## Implementation Units + +- **U1 — Relationship-origin registry and normalization.** One registry consumed by writers, doctor, and repair; three writers normalized to `command`; repair reclassify mode; doctor-clean and parity tests. +- **U2 — Codex stable-path rendering.** Render the un-canonicalized PATH entrypoint across AGENTS.md, hooks, and execpolicy prefixes; validation keeps canonicalizing; upgrade repins; symlink-retarget test proves upgrade survival. +- **U3 — Help-surface parity.** Six migrate sources listed with a dispatcher/help parity test; leaf `-h`/`--help` honored on `conversation handle add` and the Exploration equivalent. + +## Verification Contract + + + +- **V1.** `go test ./... -count=1` and `npm run typecheck` pass; `loaf build` succeeds. +- **V2.** Doctor-clean-after-writer tests: after Intent create, Exploration create, and legacy conversion on fresh fixtures, `inspectRelationshipOriginInvariants` returns zero diagnostics. +- **V3.** Repair fixture seeded with `intent-create`, `legacy-conversion`, and `exploration-create` rows: dry-run reports them all with target `command`; apply reclassifies backup-first; rerun is a no-op; doctor is clean afterward. +- **V4.** Renderer test with a symlinked entrypoint: rendered AGENTS.md, hooks, and rules contain the entrypoint path and no canonicalized segment; forbidden-root targets still rejected; symlink retarget leaves rendered files valid. +- **V5.** Dispatcher/help parity test passes; `loaf state migrate --help` lists all six sources; `loaf conversation handle add --help` and the Exploration equivalent exit 0 with usage. + + + +- **H1.** A reviewer confirms the Decision 19 amendment language accurately preserves the validation posture — no security property is silently dropped in the rendered-vs-validated split. +- **H2.** A reviewer confirms the real-state DoD evidence (backup filename, doctor output, closure entries, repinned file diff) is recorded in the PR body. + +## Definition of Done + +- All Verification Contract items pass; `loaf change check` reports zero violations. +- Real database: `state repair relationship-origin --apply` run backup-first, `loaf doctor` clean of relationship-origin diagnostics. +- Journal hygiene: `unblock` closures written for the five stale blockers, plus a current `wrap(project)` synthesis. +- Real Codex install repinned via `loaf install --upgrade`; no Cellar-versioned path remains in managed files. +- The Exploration-dogfood gating Intent exists and names its revisit trigger (shaping of `change-native-execution-migration`). +- PR merged per project conventions (squash, PR# suffix, no auto-push). + +## Durable Outputs + +After ship, `/loaf:reflect` decides whether two learnings earn ARCHITECTURE.md entries: the closed mechanism-level origin vocabulary (writers never invent provenance values) and the rendered-vs-validated path split for harness-managed trust surfaces. The Decision 19 amendment lives in this Change and the journal; the original journal-reliability-foundation document remains an unedited historical record. + +## Open Questions + +- [KU] Does Codex re-read execpolicy rules at task start (making stable paths sufficient without any rewrite), or cache them per session? → U2 isolated `CODEX_HOME` runtime smoke, recorded with the DoD evidence. + +Resolved during implementation (routes retained for provenance): + +- ~~[KU] digest-owned merge policy~~ → resolved by U2's upgrade-survival test: an owned upgrade after a symlink retarget converges byte-identically with no rewrite, and the existing ownership tests all pass against the render-path split. +- ~~[KU] is `manual` code-written?~~ → resolved yes: `Store.CreateLink` (`internal/state/link.go`) writes `manual` on every explicit `loaf link` relationship, and repair writes it under `--origin manual`; recorded in the registry's doc comment. + +## Source Inputs + +- Codex post-ship review of intent-exploration-foundation, delivered in conversation 2026-07-19, with reproduced anchors: `internal/state/status.go` (allowed-origin SQL), `internal/cli/install_codex_rules.go` (EvalSymlinks rendering), `internal/cli/cli.go` (migrate help vs dispatcher), and the failing `loaf conversation handle add --help`. +- Journal decisions 2026-07-19: `decision(codex-policy)` (Decision 19 amendment) and `decision(state)` (closed origin vocabulary). +- Prior Changes: `20260710-journal-reliability-foundation` (Decision 19, the amended contract), `20260718-installed-distribution-authority` (PR #120 — executable provenance, explicitly untouched), `20260717-intent-exploration-foundation` (PR #122 — the reviewed work). +- Live-state evidence: ten unknown-origin relationship rows on the real schema-12 database; five unresolved-blocker entries and a June-dated project synthesis in `loaf journal context`. diff --git a/internal/cli/agent_help.go b/internal/cli/agent_help.go index 860c1ca7..d53f5499 100644 --- a/internal/cli/agent_help.go +++ b/internal/cli/agent_help.go @@ -89,12 +89,14 @@ func agentHelpCommands() []agentHelpCommand { {Name: "doctor", Description: "Diagnose native state health", Options: []agentHelpOption{{Flags: "--fix", Description: "Apply safe repairs"}, {Flags: "--dry-run", Description: "Preview repairs without writing"}, {Flags: "--json", Description: "Output diagnostics, repair plan, global database scope, and project identity as JSON"}}}, {Name: "repair", Description: "Repair guarded SQLite data drift"}, {Name: "repair legacy-project-database", Description: "Archive migrated per-project SQLite leftovers", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview archive paths without writing"}, {Flags: "--apply", Description: "Move legacy SQLite files into the archive directory"}, {Flags: "--json", Description: "Output archive plan/result, global database scope, and project identity as JSON"}}}, - {Name: "repair relationship-origin", Description: "Backfill missing relationship provenance", Options: []agentHelpOption{{Flags: "--origin ", Description: "Provenance value to set"}, {Flags: "--dry-run", Description: "Preview affected rows without writing"}, {Flags: "--apply", Description: "Apply the backfill"}, {Flags: "--json", Description: "Output repair plan/result, global database scope, and project identity as JSON"}}}, + {Name: "repair relationship-origin", Description: "Reclassify retired legacy origins to 'command'; foreign origins are reported, never rewritten. Bare invocation is reclassify-only and leaves missing origins untouched; --origin adds the backfill", Options: []agentHelpOption{{Flags: "--origin ", Description: "Enable the missing-origin backfill with this provenance value; omit for reclassify-only"}, {Flags: "--dry-run", Description: "Preview affected rows without writing"}, {Flags: "--apply", Description: "Apply the reclassification, and the backfill when --origin is given, after a backup"}, {Flags: "--json", Description: "Output repair mode, plan/result, global database scope, and project identity as JSON"}}}, {Name: "repair journal-search", Description: "Rebuild the derived journal search index from canonical journal entries", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview canonical/index parity counts without writing"}, {Flags: "--apply", Description: "Create a verified backup, rebuild the index, and verify exact parity"}, {Flags: "--json", Description: "Output parity counts, backup verification, and repair result as JSON"}}}, {Name: "migrate", Description: "Run state migrations"}, {Name: "migrate markdown", Description: "Import markdown artifacts into native SQLite state", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview import work without creating SQLite state"}, {Flags: "--apply", Description: "Initialize SQLite and apply the import"}, {Flags: "--resume", Description: "Resume an interrupted import"}, {Flags: "--backup", Description: "Create SQLite and .agents rollback backups during apply or resume"}, {Flags: "--remove-source", Description: "Remove ephemeral Markdown sources after a rollback backup"}, {Flags: "--rollback ", Description: "Restore .agents files from a rollback manifest"}, {Flags: "--json", Description: "Output migration contract, scope, project context, counts, and rollback fields as JSON"}}}, {Name: "migrate storage-home", Description: "Copy legacy XDG_STATE_HOME SQLite state into the global XDG_DATA_HOME database", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview migration work without copying"}, {Flags: "--apply", Description: "Copy or merge eligible legacy state without deleting the source"}, {Flags: "--json", Description: "Output migration contract, global database paths, action, and project identity when available"}}}, {Name: "migrate schema", Description: "Preview or apply pending SQLite schema upgrades with a verified backup before mutation", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview pending schema upgrades without writing"}, {Flags: "--apply", Description: "Apply pending schema upgrades after creating and verifying a backup"}, {Flags: "--json", Description: "Output schema upgrade action, versions, pending migrations, backup, and verification as JSON"}}}, + {Name: "migrate lifecycle-statuses", Description: "Normalize legacy lifecycle statuses in SQLite with a backup and rollback manifest", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview status normalization on a temporary database copy"}, {Flags: "--apply", Description: "Normalize live SQLite statuses after creating a backup"}, {Flags: "--rollback ", Description: "Restore statuses from a lifecycle-statuses rollback manifest"}, {Flags: "--json", Description: "Output migration contract, project context, counts, backup, and rollback fields as JSON"}}}, + {Name: "migrate journal-first", Description: "Transform the global database to the journal-first model: purge lifecycle noise, drop the session entity, rekey journal search; destructive by consent", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview counts against a temporary database copy without mutation or backup"}, {Flags: "--apply", Description: "Take a mandatory backup, then apply the migration to the live database"}, {Flags: "--json", Description: "Output migration contract, counts, backup path, and schema version as JSON"}}}, {Name: "migrate deferrals", Description: "Convert historical journal deferrals into canonical deferred Intents; apply is backup-first, provenance-linking, legacy-preserving, and rerunnable", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Report the project-specific conversion manifest without writing"}, {Flags: "--apply", Description: "Convert after creating and verifying a whole-database backup"}, {Flags: "--json", Description: "Output the conversion manifest, counts, backup, and project identity as JSON"}}}, {Name: "backup", Description: "Create a SQLite database backup with local rollback or operator-selected non-temporary external destination classification", Options: []agentHelpOption{{Flags: "--to ", Description: "Operator-selected non-temporary external destination directory; not proof of off-device protection"}, {Flags: "--json", Description: "Output backup verification, classification, readiness, checksum, journal watermark, and current project identity as JSON"}}}, {Name: "backup verify", Description: "Verify an existing SQLite database backup and report retrieval/recovery readiness", Options: []agentHelpOption{{Flags: "--json", Description: "Output schema version, SQLite validity, journal retrieval readiness, recovery readiness, watermark, and captured project identities as JSON"}}}, @@ -142,6 +144,8 @@ func agentHelpCommands() []agentHelpCommand { {Name: "markdown", Description: "Import markdown artifacts into native SQLite state", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview import work without creating SQLite state"}, {Flags: "--apply", Description: "Initialize SQLite and apply the import"}, {Flags: "--resume", Description: "Resume an interrupted import"}, {Flags: "--json", Description: "Output migration contract, scope, project context, and counts as JSON"}}}, {Name: "storage-home", Description: "Copy legacy XDG_STATE_HOME SQLite state into the global XDG_DATA_HOME database", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview migration work without copying"}, {Flags: "--apply", Description: "Copy or merge eligible legacy state without deleting the source"}, {Flags: "--json", Description: "Output migration contract, global database paths, action, and project identity when available"}}}, {Name: "schema", Description: "Preview or apply pending SQLite schema upgrades with a verified backup before mutation", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview pending schema upgrades without writing"}, {Flags: "--apply", Description: "Apply pending schema upgrades after creating and verifying a backup"}, {Flags: "--json", Description: "Output schema upgrade action, versions, pending migrations, backup, and verification as JSON"}}}, + {Name: "lifecycle-statuses", Description: "Normalize legacy lifecycle statuses in SQLite with a backup and rollback manifest", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview status normalization on a temporary database copy"}, {Flags: "--apply", Description: "Normalize live SQLite statuses after creating a backup"}, {Flags: "--rollback ", Description: "Restore statuses from a lifecycle-statuses rollback manifest"}, {Flags: "--json", Description: "Output migration contract, project context, counts, backup, and rollback fields as JSON"}}}, + {Name: "journal-first", Description: "Transform the global database to the journal-first model: purge lifecycle noise, drop the session entity, rekey journal search; destructive by consent", Options: []agentHelpOption{{Flags: "--dry-run", Description: "Preview counts against a temporary database copy without mutation or backup"}, {Flags: "--apply", Description: "Take a mandatory backup, then apply the migration to the live database"}, {Flags: "--json", Description: "Output migration contract, counts, backup path, and schema version as JSON"}}}, {Name: "worktree-storage", Description: "Move linked-worktree .agents content to the main checkout", Options: []agentHelpOption{{Flags: "--apply", Description: "Perform the migration; dry-run is the default"}, {Flags: "--force-from-worktree", Description: "On conflict, keep the worktree-local copy"}, {Flags: "--force-from-main", Description: "On conflict, keep the main-worktree copy"}}}, }, }, diff --git a/internal/cli/cli.go b/internal/cli/cli.go index da38a546..daf9361f 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -1667,7 +1667,7 @@ func writeStateRepairHelp(out io.Writer) { fmt.Fprintln(out) fmt.Fprintln(out, "Targets:") fmt.Fprintln(out, " legacy-project-database Archive migrated per-project SQLite leftovers") - fmt.Fprintln(out, " relationship-origin Backfill missing relationship provenance") + fmt.Fprintln(out, " relationship-origin Backfill missing relationship provenance and reclassify retired legacy origins") fmt.Fprintln(out, " journal-search Rebuild derived journal search from canonical entries") fmt.Fprintln(out) fmt.Fprintln(out, "Options:") @@ -1680,9 +1680,12 @@ func writeStateMigrateHelp(out io.Writer) { fmt.Fprintln(out, "Run state migrations.") fmt.Fprintln(out) fmt.Fprintln(out, "Sources:") - fmt.Fprintln(out, " markdown Import .agents Markdown artifacts into SQLite") - fmt.Fprintln(out, " storage-home Copy legacy XDG_STATE_HOME state into XDG_DATA_HOME") - fmt.Fprintln(out, " schema Preview or apply pending SQLite schema upgrades") + fmt.Fprintln(out, " markdown Import .agents Markdown artifacts into SQLite") + fmt.Fprintln(out, " storage-home Copy legacy per-project state into the global XDG data-home database") + fmt.Fprintln(out, " schema Preview or apply pending SQLite schema upgrades") + fmt.Fprintln(out, " lifecycle-statuses Normalize legacy lifecycle statuses in SQLite") + fmt.Fprintln(out, " journal-first Transform the global database to the journal-first model") + fmt.Fprintln(out, " deferrals Convert historical journal deferrals into canonical deferred Intents") fmt.Fprintln(out) fmt.Fprintln(out, "Options:") fmt.Fprintln(out, " -h, --help Show help") @@ -2540,7 +2543,7 @@ func writeStateRepairLegacyProjectDatabaseHelp(out io.Writer) { } func writeStateRepairRelationshipOriginHelp(out io.Writer) { - writeUsageHelp(out, "loaf state repair relationship-origin --origin [--dry-run|--apply] [--json]", "Backfill missing relationship provenance for the current project.", "--origin Provenance value to set: imported or manual", "--dry-run Preview affected rows without writing", "--apply Apply the backfill", "--json Output repair plan/result, global database scope, and project identity as JSON") + writeUsageHelp(out, "loaf state repair relationship-origin [--origin ] [--dry-run|--apply] [--json]", "Reclassify the retired legacy origins (intent-create, legacy-conversion, exploration-create, system) to 'command' for the current project; foreign origins are reported, never rewritten. Without --origin this reclassification is all that runs and rows with no origin are left untouched; --origin additionally backfills them.", "--origin Enable the missing-origin backfill with this provenance value: imported or manual (omit for reclassify-only)", "--dry-run Preview affected rows without writing", "--apply Apply the reclassification, and the backfill when --origin is given, after a backup", "--json Output repair mode, plan/result, global database scope, and project identity as JSON") } func writeStateRepairJournalSearchHelp(out io.Writer) { @@ -2617,7 +2620,7 @@ func (r Runner) runStateRepairRelationshipOrigin(args []string, out io.Writer, r } return err } - result, err := state.RepairMissingRelationshipOrigins(context.Background(), projectRoot, state.PathResolver{StateHome: r.StateHome}, state.RelationshipOriginRepairOptions{ + result, err := state.RepairRelationshipOrigins(context.Background(), projectRoot, state.PathResolver{StateHome: r.StateHome}, state.RelationshipOriginRepairOptions{ Origin: options.origin, Apply: options.apply, }) @@ -2644,12 +2647,43 @@ func (r Runner) runStateRepairRelationshipOrigin(args []string, out io.Writer, r if result.ProjectCurrentPath != "" { fmt.Fprintf(out, "project path: %s\n", result.ProjectCurrentPath) } - fmt.Fprintf(out, "origin: %s\n", result.Origin) - fmt.Fprintf(out, "matched: %d\n", result.Matched) - fmt.Fprintf(out, "updated: %d\n", result.Updated) + fmt.Fprintf(out, "mode: %s\n", result.Mode) + backfilling := result.Mode == state.RelationshipOriginRepairModeBackfillAndReclassify + if backfilling { + fmt.Fprintf(out, "origin: %s\n", result.Origin) + fmt.Fprintf(out, "matched: %d\n", result.Matched) + fmt.Fprintf(out, "updated: %d\n", result.Updated) + } else if result.Matched > 0 { + // Reclassify-only never writes these rows, so they are reported as + // untouched state rather than as a backfill that matched nothing. + fmt.Fprintf(out, "missing origin: %d row(s) left untouched; pass --origin imported|manual to backfill them\n", result.Matched) + } + reclassifiable := 0 + reclassifyTarget := "" + for _, reclassification := range result.Reclassified { + reclassifiable += reclassification.Matched + // Every legacy origin reclassifies to the same registry target, so any + // group's target names the dry-run summary value. + reclassifyTarget = reclassification.Target + fmt.Fprintf(out, "reclassify %s -> %s: matched %d, updated %d\n", reclassification.Origin, reclassification.Target, reclassification.Matched, reclassification.Updated) + } + for _, foreign := range result.ForeignOrigins { + fmt.Fprintf(out, "warn: foreign origin %q on %d relationship row(s); left for doctor, never rewritten\n", foreign.Origin, foreign.Count) + } fmt.Fprintf(out, "applied: %t\n", result.Applied) - if !result.Applied && result.Matched > 0 { - fmt.Fprintln(out, "next: rerun with --apply after reviewing the selected origin") + if !result.Applied && reclassifiable > 0 { + fmt.Fprintf(out, "%d relationship row(s) with unknown origin would be reclassified to '%s'\n", reclassifiable, reclassifyTarget) + } + pendingBackfill := 0 + if backfilling { + pendingBackfill = result.Matched + } + if !result.Applied && (pendingBackfill > 0 || reclassifiable > 0) { + if backfilling { + fmt.Fprintln(out, "next: rerun with --apply after reviewing the selected origin") + } else { + fmt.Fprintln(out, "next: rerun with --apply after reviewing the reclassification plan") + } } return nil } @@ -3135,36 +3169,59 @@ func (r Runner) runStateExport(args []string, out io.Writer, runtime state.Runti } } +// stateMigrateSource binds one dispatchable `loaf state migrate` source to its +// runner and its --help writer, so a source cannot gain dispatch without help +// routing or vice versa. +type stateMigrateSource struct { + run func(Runner, []string, io.Writer, state.Runtime) error + help func(io.Writer) +} + +// stateMigrateSources is the single registry runStateMigrate dispatches +// through; nested ` --help` routing is derived from it. The source +// list in writeStateMigrateHelp must name every key; +// TestStateMigrateHelpListsEveryDispatchableSource enforces that parity. +var stateMigrateSources = map[string]stateMigrateSource{ + "lifecycle-statuses": {run: Runner.runStateMigrateLifecycleStatuses, help: writeStateMigrateLifecycleStatusesHelp}, + "journal-first": { + run: func(r Runner, args []string, out io.Writer, runtime state.Runtime) error { + return r.runJournalFirstMigration(args, out, runtime, "loaf state migrate journal-first") + }, + help: writeStateMigrateJournalFirstHelp, + }, + "schema": { + run: func(r Runner, args []string, out io.Writer, runtime state.Runtime) error { + return r.runSchemaUpgrade(args, out, runtime, "loaf state migrate schema") + }, + help: writeStateMigrateSchemaHelp, + }, + "markdown": {run: Runner.runStateMigrateMarkdown, help: writeStateMigrateMarkdownHelp}, + "storage-home": {run: Runner.runStateMigrateStorageHome, help: writeStateMigrateStorageHomeHelp}, + "deferrals": {run: Runner.runStateMigrateDeferrals, help: writeStateMigrateDeferralsHelp}, +} + +// stateMigrateSourceHelp derives the `loaf state migrate --help` +// routing map from the dispatch registry. +func stateMigrateSourceHelp() map[string]func(io.Writer) { + writers := make(map[string]func(io.Writer), len(stateMigrateSources)) + for name, source := range stateMigrateSources { + writers[name] = source.help + } + return writers +} + func (r Runner) runStateMigrate(args []string, out io.Writer, runtime state.Runtime) error { if len(args) == 0 { return fmt.Errorf("state migrate requires a source") } - if writeNestedHelp(out, args, map[string]func(io.Writer){ - "lifecycle-statuses": writeStateMigrateLifecycleStatusesHelp, - "journal-first": writeStateMigrateJournalFirstHelp, - "schema": writeStateMigrateSchemaHelp, - "markdown": writeStateMigrateMarkdownHelp, - "storage-home": writeStateMigrateStorageHomeHelp, - "deferrals": writeStateMigrateDeferralsHelp, - }) { + if writeNestedHelp(out, args, stateMigrateSourceHelp()) { return nil } - switch args[0] { - case "lifecycle-statuses": - return r.runStateMigrateLifecycleStatuses(args[1:], out, runtime) - case "journal-first": - return r.runJournalFirstMigration(args[1:], out, runtime, "loaf state migrate journal-first") - case "schema": - return r.runSchemaUpgrade(args[1:], out, runtime, "loaf state migrate schema") - case "deferrals": - return r.runStateMigrateDeferrals(args[1:], out, runtime) - case "markdown": - return r.runStateMigrateMarkdown(args[1:], out, runtime) - case "storage-home": - return r.runStateMigrateStorageHome(args[1:], out, runtime) - default: + source, ok := stateMigrateSources[args[0]] + if !ok { return fmt.Errorf("state migrate source %q is not implemented yet", args[0]) } + return source.run(r, args[1:], out, runtime) } func writeStateMigrateMarkdownHelp(out io.Writer) { @@ -3274,16 +3331,45 @@ func (r Runner) runSchemaUpgrade(args []string, out io.Writer, runtime state.Run return nil } +// writeMigrateHelp lists the sources the runMigrate switch dispatches. It is +// the top-level sibling of writeStateMigrateHelp and covers a different source +// set: worktree-storage is top-level only, and deferrals is state-level only. +// TestRunnerMigrateHelpListsEveryDispatchableSource enforces that parity. +func writeMigrateHelp(out io.Writer) { + fmt.Fprintln(out, "Usage: loaf migrate [options]") + fmt.Fprintln(out) + fmt.Fprintln(out, "Run native migration workflows.") + fmt.Fprintln(out) + fmt.Fprintln(out, "Sources:") + fmt.Fprintln(out, " markdown Import .agents Markdown artifacts into SQLite") + fmt.Fprintln(out, " storage-home Copy legacy per-project state into the global XDG data-home database") + fmt.Fprintln(out, " schema Preview or apply pending SQLite schema upgrades") + fmt.Fprintln(out, " lifecycle-statuses Normalize legacy lifecycle statuses in SQLite") + fmt.Fprintln(out, " journal-first Transform the global database to the journal-first model") + fmt.Fprintln(out, " worktree-storage Move linked-worktree .agents content to the main checkout") + fmt.Fprintln(out) + fmt.Fprintln(out, "Options:") + fmt.Fprintln(out, " -h, --help Show help") +} + func (r Runner) runMigrate(args []string, out io.Writer, runtime state.Runtime) error { if len(args) == 0 { + // Bare invocation stays an error, but the source list goes to stderr + // so the failure is self-correcting instead of a dead end. + writeMigrateHelp(firstWriter(r.Stderr, os.Stderr)) return fmt.Errorf("migrate requires a source") } + if isHelpArg(args) { + writeMigrateHelp(out) + return nil + } if writeNestedHelp(out, args, map[string]func(io.Writer){ "lifecycle-statuses": writeMigrateLifecycleStatusesHelp, "journal-first": writeMigrateJournalFirstHelp, "schema": writeMigrateSchemaHelp, "markdown": writeMigrateMarkdownHelp, "storage-home": writeMigrateStorageHomeHelp, + "worktree-storage": writeWorktreeMigrationHelp, }) { return nil } @@ -13264,6 +13350,7 @@ func parseJournalSearchRepairArgs(args []string) (journalSearchRepairOptions, er func parseRelationshipOriginRepairArgs(args []string) (relationshipOriginRepairOptions, error) { var options relationshipOriginRepairOptions + originSet := false for i := 0; i < len(args); i++ { arg := args[i] switch arg { @@ -13279,6 +13366,7 @@ func parseRelationshipOriginRepairArgs(args []string) (relationshipOriginRepairO return relationshipOriginRepairOptions{}, err } options.origin = value + originSet = true default: return relationshipOriginRepairOptions{}, fmt.Errorf("unknown option %q", arg) } @@ -13286,10 +13374,11 @@ func parseRelationshipOriginRepairArgs(args []string) (relationshipOriginRepairO if options.apply && options.dryRun { return relationshipOriginRepairOptions{}, fmt.Errorf("state repair relationship-origin cannot combine --apply and --dry-run") } - if options.origin == "" { - return relationshipOriginRepairOptions{}, fmt.Errorf("state repair relationship-origin requires --origin imported|manual") - } - if options.origin != "imported" && options.origin != "manual" { + // Omitting --origin is valid: it selects reclassify-only, the mechanical + // registry-derived repair. Supplying it opts into the missing-origin + // backfill, which needs a real mechanism value — an explicitly empty one is + // an operator mistake, not a mode selection. + if originSet && options.origin != "imported" && options.origin != "manual" { return relationshipOriginRepairOptions{}, fmt.Errorf("relationship origin must be imported or manual") } return options, nil diff --git a/internal/cli/cli_reference.go b/internal/cli/cli_reference.go index 0a41dee2..9e47fbae 100644 --- a/internal/cli/cli_reference.go +++ b/internal/cli/cli_reference.go @@ -171,11 +171,11 @@ func cliReferenceCommands() []cliReferenceCommand { {Flags: "--apply", Description: "Move legacy SQLite files into the archive directory"}, {Flags: "--json", Description: "Output archive plan/result, global database scope, and project identity as JSON"}, }}, - {Name: "repair relationship-origin", Description: "Preview or apply guarded relationship provenance backfills", Options: []cliReferenceOption{ - {Flags: "--origin ", Description: "Provenance value to backfill"}, + {Name: "repair relationship-origin", Description: "Reclassify retired legacy origins to 'command'; foreign origins are reported, never rewritten. Bare invocation is reclassify-only and leaves missing origins untouched; --origin adds the backfill", Options: []cliReferenceOption{ + {Flags: "--origin ", Description: "Enable the missing-origin backfill with this provenance value; omit for reclassify-only"}, {Flags: "--dry-run", Description: "Preview affected rows without writing"}, - {Flags: "--apply", Description: "Backfill missing origins after creating a SQLite backup"}, - {Flags: "--json", Description: "Output repair plan/result, global database scope, and project identity as JSON"}, + {Flags: "--apply", Description: "Reclassify retired legacy origins, and backfill missing origins when --origin is given, after creating a SQLite backup"}, + {Flags: "--json", Description: "Output repair mode, plan/result, global database scope, and project identity as JSON"}, }}, {Name: "repair journal-search", Description: "Preview or apply a backup-first rebuild of the derived journal search index", Options: []cliReferenceOption{ {Flags: "--dry-run", Description: "Preview canonical/index parity counts without writing"}, @@ -212,6 +212,11 @@ func cliReferenceCommands() []cliReferenceCommand { {Flags: "--rollback ", Description: "Restore statuses from a lifecycle-statuses rollback manifest"}, {Flags: "--json", Description: "Output migration contract, project context, counts, backup, and rollback fields as JSON"}, }}, + {Name: "migrate journal-first", Description: "Transform the global database to the journal-first model: purge lifecycle noise, drop the session entity, rekey journal search; destructive by consent", Options: []cliReferenceOption{ + {Flags: "--dry-run", Description: "Preview counts against a temporary database copy without mutation or backup"}, + {Flags: "--apply", Description: "Take a mandatory backup, then apply the migration to the live database"}, + {Flags: "--json", Description: "Output migration contract, counts, backup path, and schema version as JSON"}, + }}, {Name: "backup", Description: "Create a SQLite database backup with local rollback or operator-selected non-temporary external destination classification", Options: []cliReferenceOption{{Flags: "--to ", Description: "Operator-selected non-temporary external destination directory; not proof of off-device protection"}, {Flags: "--json", Description: "Output backup verification, classification, readiness, checksum, journal watermark, and current project identity as JSON"}}}, {Name: "backup verify", Description: "Verify an existing SQLite database backup and report retrieval/recovery readiness", Options: []cliReferenceOption{{Flags: "--json", Description: "Output schema version, SQLite validity, journal retrieval readiness, recovery readiness, watermark, and captured project identities as JSON"}}}, {Name: "backup restore", Description: "Run an isolated disposable restore rehearsal without activating or replacing the live database", Options: []cliReferenceOption{{Flags: "", Description: "Verified backup path"}, {Flags: "--to ", Description: "Required empty disposable restore target; never the live database"}, {Flags: "--json", Description: "Output isolated disposable rehearsal, exact-copy, integrity, retrieval, watermark, and live-database safety evidence; never activates the live database"}}}, @@ -342,6 +347,11 @@ func cliReferenceCommands() []cliReferenceCommand { {Flags: "--rollback ", Description: "Restore statuses from a lifecycle-statuses rollback manifest"}, {Flags: "--json", Description: "Output migration contract, project context, counts, backup, and rollback fields as JSON"}, }}, + {Name: "journal-first", Description: "Transform the global database to the journal-first model: purge lifecycle noise, drop the session entity, rekey journal search; destructive by consent", Options: []cliReferenceOption{ + {Flags: "--dry-run", Description: "Preview counts against a temporary database copy without mutation or backup"}, + {Flags: "--apply", Description: "Take a mandatory backup, then apply the migration to the live database"}, + {Flags: "--json", Description: "Output migration contract, counts, backup path, and schema version as JSON"}, + }}, {Name: "worktree-storage", Description: "Move linked-worktree .agents state to the main worktree", Options: []cliReferenceOption{ {Flags: "--apply", Description: "Perform the migration; dry-run is the default"}, {Flags: "--force-from-worktree", Description: "On conflict, keep the worktree-local copy"}, diff --git a/internal/cli/cli_reference_test.go b/internal/cli/cli_reference_test.go index 9177b52b..5f3a326f 100644 --- a/internal/cli/cli_reference_test.go +++ b/internal/cli/cli_reference_test.go @@ -63,7 +63,7 @@ func TestRunnerGenerateCLIReferenceWritesSkillNatively(t *testing.T) { "| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database,", "repair journal-search", "migrate schema", - "| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage |", + "| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage |", "| `loaf doctor` | Diagnose Loaf project alignment (symlinks, stale files, version drift) | — |", "## Topics", "[references/configuration.md](references/configuration.md)", diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index d5affe43..5d1039bf 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -2788,7 +2788,7 @@ func TestRunnerStateHelpIsNative(t *testing.T) { {name: "state doctor", args: []string{"state", "doctor", "--help"}, want: "Usage: loaf state doctor [--fix] [--dry-run] [--json]"}, {name: "state repair", args: []string{"state", "repair", "--help"}, want: "Usage: loaf state repair [options]"}, {name: "state repair legacy-project-database", args: []string{"state", "repair", "legacy-project-database", "--help"}, want: "Usage: loaf state repair legacy-project-database [--dry-run|--apply] [--json]"}, - {name: "state repair relationship-origin", args: []string{"state", "repair", "relationship-origin", "--help"}, want: "Usage: loaf state repair relationship-origin --origin [--dry-run|--apply] [--json]"}, + {name: "state repair relationship-origin", args: []string{"state", "repair", "relationship-origin", "--help"}, want: "Usage: loaf state repair relationship-origin [--origin ] [--dry-run|--apply] [--json]"}, {name: "state repair journal-search", args: []string{"state", "repair", "journal-search", "--help"}, want: "Usage: loaf state repair journal-search [--dry-run|--apply] [--json]"}, {name: "state migrate", args: []string{"state", "migrate", "--help"}, want: "Usage: loaf state migrate [options]"}, {name: "project list", args: []string{"project", "list", "--help"}, want: "Usage: loaf project list [--json]"}, @@ -2837,7 +2837,7 @@ func TestRunnerStateAndProjectJSONHelpNamesContracts(t *testing.T) { {name: "project rename", args: []string{"project", "rename", "--help"}, wants: []string{"--json", "friendly name", "database path", "applied status"}}, {name: "project move", args: []string{"project", "move", "--help"}, wants: []string{"--json", "current path", "database path", "applied status"}}, {name: "state repair legacy", args: []string{"state", "repair", "legacy-project-database", "--help"}, wants: []string{"--json", "archive plan/result", "global database scope", "project identity"}}, - {name: "state repair relationship", args: []string{"state", "repair", "relationship-origin", "--help"}, wants: []string{"--json", "repair plan/result", "global database scope", "project identity"}}, + {name: "state repair relationship", args: []string{"state", "repair", "relationship-origin", "--help"}, wants: []string{"--json", "repair mode, plan/result", "global database scope", "project identity", "omit for reclassify-only"}}, {name: "state repair journal-search", args: []string{"state", "repair", "journal-search", "--help"}, wants: []string{"--json", "parity counts", "backup verification", "exact parity"}}, {name: "state migrate markdown", args: []string{"state", "migrate", "markdown", "--help"}, wants: []string{"--json", "migration contract", "project context", "counts"}}, {name: "state migrate storage-home", args: []string{"state", "migrate", "storage-home", "--help"}, wants: []string{"--json", "migration contract", "global database paths", "project identity"}}, @@ -3785,6 +3785,202 @@ VALUES ('relationship-without-origin', ?, 'task', 'task-one', 'spec', 'spec-one' } } +// The Change's Observable Workflow invokes the repair without --origin. This +// pins that exact contract end-to-end: bare dry-run reports the reclassifiable +// rows, bare apply reclassifies backup-first, and a rerun is a no-op — all +// without ever backfilling the row that carries no origin. +func TestRunnerStateRepairRelationshipOriginBareInvocationReclassifiesOnly(t *testing.T) { + workingDir := realpath(t, t.TempDir()) + stateHome := t.TempDir() + + var initOut bytes.Buffer + if err := (Runner{Stdout: &initOut, WorkingDir: workingDir, StateHome: stateHome}).Run([]string{"state", "init", "--json"}); err != nil { + t.Fatalf("state init error = %v", err) + } + initialized := decodeStateStatus(t, initOut.Bytes()) + db, err := sql.Open("sqlite3", initialized.DatabasePath) + if err != nil { + t.Fatalf("sql.Open() error = %v", err) + } + defer db.Close() + for id, origin := range map[string]string{ + "relationship-intent-create": "intent-create", + "relationship-legacy-conversion": "legacy-conversion", + } { + if _, err := db.Exec(` +INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) +VALUES (?, ?, 'task', 'task-one', 'spec', 'spec-one', 'implements', 'seeded row', ?, '2026-07-19T10:00:00Z', '2026-07-19T10:00:00Z') +`, id, initialized.ProjectID, origin); err != nil { + t.Fatalf("insert %s error = %v", id, err) + } + } + if _, err := db.Exec(` +INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, created_at, updated_at) +VALUES ('relationship-without-origin', ?, 'task', 'task-two', 'spec', 'spec-two', 'implements', 'legacy row', '2026-06-13T10:00:00Z', '2026-06-13T10:00:00Z') +`, initialized.ProjectID); err != nil { + t.Fatalf("insert relationship without origin error = %v", err) + } + + var dryRunOut bytes.Buffer + if err := (Runner{Stdout: &dryRunOut, WorkingDir: workingDir, StateHome: stateHome}).Run([]string{"state", "repair", "relationship-origin", "--dry-run"}); err != nil { + t.Fatalf("bare state repair relationship-origin --dry-run error = %v", err) + } + for _, want := range []string{ + "mode: reclassify-only", + "reclassify intent-create -> command: matched 1, updated 0", + "reclassify legacy-conversion -> command: matched 1, updated 0", + "missing origin: 1 row(s) left untouched; pass --origin imported|manual to backfill them", + "2 relationship row(s) with unknown origin would be reclassified to 'command'", + "next: rerun with --apply after reviewing the reclassification plan", + } { + if !strings.Contains(dryRunOut.String(), want) { + t.Fatalf("bare dry-run output = %q, want %q", dryRunOut.String(), want) + } + } + // No backfill ran, so no line may report one — 'matched'/'updated' are the + // backfill's own counters and would read as a backfill that did nothing. + for _, unwanted := range []string{"matched: ", "updated: ", "origin: imported"} { + if strings.Contains(dryRunOut.String(), unwanted) { + t.Fatalf("bare dry-run output = %q, must not contain backfill line %q", dryRunOut.String(), unwanted) + } + } + if got := sqliteCount(t, db, `SELECT COUNT(*) FROM relationships WHERE origin = 'command'`); got != 0 { + t.Fatalf("reclassified rows after dry-run = %d, want 0", got) + } + + var applyOut bytes.Buffer + if err := (Runner{Stdout: &applyOut, WorkingDir: workingDir, StateHome: stateHome}).Run([]string{"state", "repair", "relationship-origin", "--apply", "--json"}); err != nil { + t.Fatalf("bare state repair relationship-origin --apply error = %v", err) + } + applied := decodeRelationshipOriginRepairResult(t, applyOut.Bytes()) + if applied.Mode != state.RelationshipOriginRepairModeReclassifyOnly { + t.Fatalf("apply Mode = %q, want %q", applied.Mode, state.RelationshipOriginRepairModeReclassifyOnly) + } + if applied.Origin != "" { + t.Fatalf("apply Origin = %q, want empty without --origin", applied.Origin) + } + if !applied.Applied { + t.Fatal("apply Applied = false, want true") + } + if applied.Updated != 0 { + t.Fatalf("apply Updated = %d, want 0 — bare invocation must not backfill", applied.Updated) + } + if applied.BackupPath == "" { + t.Fatal("apply BackupPath is empty, want a backup-first reclassification") + } + if _, err := os.Stat(applied.BackupPath); err != nil { + t.Fatalf("apply backup does not exist: %v", err) + } + if got := sqliteCount(t, db, `SELECT COUNT(*) FROM relationships WHERE origin = 'command'`); got != 2 { + t.Fatalf("reclassified rows after apply = %d, want 2", got) + } + if got := sqliteCount(t, db, `SELECT COUNT(*) FROM relationships WHERE origin IS NULL OR TRIM(origin) = ''`); got != 1 { + t.Fatalf("missing-origin rows after apply = %d, want the row left untouched", got) + } + + var rerunOut bytes.Buffer + if err := (Runner{Stdout: &rerunOut, WorkingDir: workingDir, StateHome: stateHome}).Run([]string{"state", "repair", "relationship-origin", "--apply", "--json"}); err != nil { + t.Fatalf("bare state repair relationship-origin rerun error = %v", err) + } + rerun := decodeRelationshipOriginRepairResult(t, rerunOut.Bytes()) + if rerun.BackupPath != "" { + t.Fatalf("rerun BackupPath = %q, want no backup for a no-op", rerun.BackupPath) + } + for _, reclassification := range rerun.Reclassified { + if reclassification.Matched != 0 || reclassification.Updated != 0 { + t.Fatalf("rerun reclassification = %#v, want zero matched and updated", reclassification) + } + } + if got := sqliteCount(t, db, `SELECT COUNT(*) FROM relationships WHERE origin = 'command'`); got != 2 { + t.Fatalf("reclassified rows after rerun = %d, want 2", got) + } +} + +func TestRunnerStateRepairRelationshipOriginTextReportsReclassifyAndForeign(t *testing.T) { + workingDir := realpath(t, t.TempDir()) + stateHome := t.TempDir() + + var initOut bytes.Buffer + if err := (Runner{Stdout: &initOut, WorkingDir: workingDir, StateHome: stateHome}).Run([]string{"state", "init", "--json"}); err != nil { + t.Fatalf("state init error = %v", err) + } + initialized := decodeStateStatus(t, initOut.Bytes()) + db, err := sql.Open("sqlite3", initialized.DatabasePath) + if err != nil { + t.Fatalf("sql.Open() error = %v", err) + } + defer db.Close() + for id, origin := range map[string]string{ + "relationship-intent-create": "intent-create", + "relationship-exploration-create": "exploration-create", + "relationship-mystery-import": "mystery-import", + } { + if _, err := db.Exec(` +INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) +VALUES (?, ?, 'task', 'task-one', 'spec', 'spec-one', 'implements', 'seeded row', ?, '2026-07-19T10:00:00Z', '2026-07-19T10:00:00Z') +`, id, initialized.ProjectID, origin); err != nil { + t.Fatalf("insert %s error = %v", id, err) + } + } + + // Text-mode dry-run must disclose the reclassification plan and the + // preserved foreign origins, not just the missing-origin backfill. + var dryRunOut bytes.Buffer + err = Runner{ + Stdout: &dryRunOut, + WorkingDir: workingDir, + StateHome: stateHome, + }.Run([]string{"state", "repair", "relationship-origin", "--origin", "imported", "--dry-run"}) + if err != nil { + t.Fatalf("state repair relationship-origin text dry-run error = %v", err) + } + for _, want := range []string{ + "matched: 0", + "reclassify exploration-create -> command: matched 1, updated 0", + "reclassify intent-create -> command: matched 1, updated 0", + "reclassify legacy-conversion -> command: matched 0, updated 0", + `warn: foreign origin "mystery-import" on 1 relationship row(s); left for doctor, never rewritten`, + "2 relationship row(s) with unknown origin would be reclassified to 'command'", + "next: rerun with --apply", + } { + if !strings.Contains(dryRunOut.String(), want) { + t.Fatalf("text dry-run output = %q, want %q", dryRunOut.String(), want) + } + } + + // Text-mode apply must disclose the backup and the reclassified counts + // even when the missing-origin backfill matches nothing. + var applyOut bytes.Buffer + err = Runner{ + Stdout: &applyOut, + WorkingDir: workingDir, + StateHome: stateHome, + }.Run([]string{"state", "repair", "relationship-origin", "--origin", "imported", "--apply"}) + if err != nil { + t.Fatalf("state repair relationship-origin text apply error = %v", err) + } + for _, want := range []string{ + "backup: ", + "reclassify exploration-create -> command: matched 1, updated 1", + "reclassify intent-create -> command: matched 1, updated 1", + `warn: foreign origin "mystery-import" on 1 relationship row(s); left for doctor, never rewritten`, + "applied: true", + } { + if !strings.Contains(applyOut.String(), want) { + t.Fatalf("text apply output = %q, want %q", applyOut.String(), want) + } + } + if strings.Contains(applyOut.String(), "would be reclassified") { + t.Fatalf("text apply output = %q, want no dry-run reclassify summary", applyOut.String()) + } + if got := sqliteCount(t, db, `SELECT COUNT(*) FROM relationships WHERE origin = 'command'`); got != 2 { + t.Fatalf("command-origin relationships after apply = %d, want 2", got) + } + if got := sqliteCount(t, db, `SELECT COUNT(*) FROM relationships WHERE origin = 'mystery-import'`); got != 1 { + t.Fatalf("foreign-origin relationships after apply = %d, want 1", got) + } +} + func TestRunnerStateRepairJournalSearchHumanJSONAndErrorEnvelope(t *testing.T) { workingDir := realpath(t, t.TempDir()) stateHome := t.TempDir() @@ -6952,17 +7148,25 @@ func TestRunnerStateJSONValidationErrorsAreMachineReadable(t *testing.T) { want: "cannot combine --apply and --dry-run", }, { - name: "relationship repair missing origin", - args: []string{"state", "repair", "relationship-origin", "--dry-run", "--json"}, + name: "relationship repair invalid origin", + args: []string{"state", "repair", "relationship-origin", "--origin", "external", "--json"}, command: "state repair relationship-origin", - want: "requires --origin", + want: "must be imported or manual", }, { - name: "relationship repair invalid origin", - args: []string{"state", "repair", "relationship-origin", "--origin", "external", "--json"}, + // Omitting --origin selects reclassify-only, but supplying it empty + // is an operator mistake and must not fall through to that mode. + name: "relationship repair empty origin", + args: []string{"state", "repair", "relationship-origin", "--origin", "", "--json"}, command: "state repair relationship-origin", want: "must be imported or manual", }, + { + name: "relationship repair conflicting flags", + args: []string{"state", "repair", "relationship-origin", "--apply", "--dry-run", "--json"}, + command: "state repair relationship-origin", + want: "cannot combine --apply and --dry-run", + }, } for _, tc := range tests { @@ -14463,14 +14667,14 @@ func TestRunnerAgentHelpIsNative(t *testing.T) { if got := commands["state"].optionDescriptions["state repair legacy-project-database --json"]; !strings.Contains(got, "archive plan/result") || !strings.Contains(got, "project identity") { t.Fatalf("legacy repair json description = %q, want archive/project identity guidance", got) } - if got := commands["state"].optionDescriptions["state repair relationship-origin --origin "]; !strings.Contains(got, "Provenance value") { - t.Fatalf("relationship repair origin description = %q, want provenance guidance", got) + if got := commands["state"].optionDescriptions["state repair relationship-origin --origin "]; !strings.Contains(got, "provenance value") || !strings.Contains(got, "omit for reclassify-only") { + t.Fatalf("relationship repair origin description = %q, want provenance guidance naming the reclassify-only default", got) } if got := commands["state"].optionDescriptions["state repair relationship-origin --dry-run"]; !strings.Contains(got, "without writing") { t.Fatalf("relationship repair dry-run description = %q, want non-mutating preview", got) } - if got := commands["state"].optionDescriptions["state repair relationship-origin --json"]; !strings.Contains(got, "repair plan/result") || !strings.Contains(got, "global database scope") { - t.Fatalf("relationship repair json description = %q, want repair/scope guidance", got) + if got := commands["state"].optionDescriptions["state repair relationship-origin --json"]; !strings.Contains(got, "repair mode, plan/result") || !strings.Contains(got, "global database scope") { + t.Fatalf("relationship repair json description = %q, want mode/repair/scope guidance", got) } if got := commands["state"].optionDescriptions["state repair journal-search --dry-run"]; !strings.Contains(got, "without writing") { t.Fatalf("journal-search repair dry-run description = %q, want non-mutating preview", got) diff --git a/internal/cli/exploration.go b/internal/cli/exploration.go index 1106d8b8..ec2ca1bb 100644 --- a/internal/cli/exploration.go +++ b/internal/cli/exploration.go @@ -299,6 +299,10 @@ func (r Runner) runExplorationConversation(args []string, out io.Writer, runtime return fmt.Errorf("usage: loaf exploration conversation add [--json]") } args = args[1:] + if isHelpArg(args) { + writeExplorationConversationHelp(out) + return nil + } jsonOutput := false positional := []string{} for _, arg := range args { @@ -587,6 +591,10 @@ func (r Runner) runConversationHandle(args []string, out io.Writer, runtime stat return fmt.Errorf("usage: loaf conversation handle add --harness --handle [options]") } args = args[1:] + if isHelpArg(args) { + writeConversationHandleHelp(out) + return nil + } options := state.ConversationHandleAddOptions{} jsonOutput := false for i := 0; i < len(args); i++ { diff --git a/internal/cli/help_parity_test.go b/internal/cli/help_parity_test.go new file mode 100644 index 00000000..9660460c --- /dev/null +++ b/internal/cli/help_parity_test.go @@ -0,0 +1,265 @@ +package cli + +import ( + "bytes" + "path/filepath" + "strings" + "testing" +) + +// parseHelpSectionNames extracts the first token of each indented line in the +// named help section, e.g. the source names under "Sources:". +func parseHelpSectionNames(t *testing.T, help string, section string) map[string]bool { + t.Helper() + names := map[string]bool{} + inSection := false + for _, line := range strings.Split(help, "\n") { + if strings.TrimSpace(line) == section { + inSection = true + continue + } + if !inSection { + continue + } + if strings.TrimSpace(line) == "" { + break + } + fields := strings.Fields(line) + if len(fields) > 0 { + names[fields[0]] = true + } + } + if len(names) == 0 { + t.Fatalf("help output has no %q section entries:\n%s", section, help) + } + return names +} + +func TestStateMigrateHelpListsEveryDispatchableSource(t *testing.T) { + // stateMigrateSources is the registry runStateMigrate dispatches through, + // so its key set IS the dispatcher's source set: a source cannot become + // dispatchable without an entry here, and this test then forces it into + // the writeStateMigrateHelp source list. + var help bytes.Buffer + writeStateMigrateHelp(&help) + listed := parseHelpSectionNames(t, help.String(), "Sources:") + for source, entry := range stateMigrateSources { + if entry.run == nil { + t.Errorf("stateMigrateSources[%q] has no run func", source) + } + if entry.help == nil { + t.Errorf("stateMigrateSources[%q] has no help func", source) + } + if !listed[source] { + t.Errorf("writeStateMigrateHelp does not list dispatchable source %q", source) + } + } + for source := range listed { + if _, ok := stateMigrateSources[source]; !ok { + t.Errorf("writeStateMigrateHelp lists %q, which is not dispatchable via stateMigrateSources", source) + } + } +} + +func TestStateMigrateEveryListedSourceDispatches(t *testing.T) { + // Behavioral probe of the dispatcher: every source runStateMigrate accepts + // must reach its runner instead of the not-implemented fallthrough. Each + // runner rejects the unknown option during arg parsing, before any state + // access, so no database is ever touched. + t.Setenv("LOAF_DB", filepath.Join(t.TempDir(), "loaf.sqlite")) + for source := range stateMigrateSources { + t.Run(source, func(t *testing.T) { + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run([]string{"state", "migrate", source, "--no-such-option"}) + if err == nil { + t.Fatalf("Run(state migrate %s --no-such-option) error = nil, want unknown-option rejection from the source runner", source) + } + if strings.Contains(err.Error(), "is not implemented yet") { + t.Fatalf("state migrate %q did not dispatch: %v", source, err) + } + }) + } +} + +func TestRunnerStateMigrateHelpListsAllSources(t *testing.T) { + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run([]string{"state", "migrate", "--help"}) + if err != nil { + t.Fatalf("Run(state migrate --help) error = %v", err) + } + wants := []string{ + "Usage: loaf state migrate [options]", + "markdown", + "storage-home", + "schema", + "lifecycle-statuses", + "journal-first", + "deferrals", + } + for _, want := range wants { + if !strings.Contains(stdout.String(), want) { + t.Fatalf("stdout = %q, want %q", stdout.String(), want) + } + } +} + +func TestRunnerStateMigrateSourceHelpIsNative(t *testing.T) { + for source := range stateMigrateSources { + t.Run(source, func(t *testing.T) { + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run([]string{"state", "migrate", source, "--help"}) + if err != nil { + t.Fatalf("Run(state migrate %s --help) error = %v", source, err) + } + want := "Usage: loaf state migrate " + source + if !strings.Contains(stdout.String(), want) { + t.Fatalf("stdout = %q, want %q", stdout.String(), want) + } + }) + } +} + +// migrateDispatchableSources is anchored to the runMigrate switch in cli.go. +// runMigrate dispatches through a switch rather than a registry, so this slice +// is the test-side transcription the help surface is held against; adding a +// case to that switch without adding it here leaves the new source unprobed, +// and adding it here without listing it in writeMigrateHelp fails parity. +var migrateDispatchableSources = []string{"lifecycle-statuses", "journal-first", "schema", "markdown", "storage-home", "worktree-storage"} + +func TestRunnerMigrateHelpExitsZeroAndListsEverySource(t *testing.T) { + // Bare `loaf migrate --help` used to fall through the switch and exit + // non-zero with `migrate source "--help" is not implemented yet`. + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run([]string{"migrate", "--help"}) + if err != nil { + t.Fatalf("Run(migrate --help) error = %v, want nil", err) + } + if !strings.Contains(stdout.String(), "Usage: loaf migrate [options]") { + t.Fatalf("stdout = %q, want migrate usage line", stdout.String()) + } + listed := parseHelpSectionNames(t, stdout.String(), "Sources:") + for _, source := range migrateDispatchableSources { + if !listed[source] { + t.Errorf("writeMigrateHelp does not list dispatchable source %q", source) + } + } + for source := range listed { + found := false + for _, dispatchable := range migrateDispatchableSources { + if source == dispatchable { + found = true + break + } + } + if !found { + t.Errorf("writeMigrateHelp lists %q, which the runMigrate switch does not dispatch", source) + } + } +} + +func TestRunnerMigrateBareInvocationErrorsAndPrintsHelpToStderr(t *testing.T) { + // Bare `loaf migrate` keeps its non-zero exit; the source list goes to + // stderr so the error is self-correcting. + var stdout, stderr bytes.Buffer + err := Runner{ + Stdout: &stdout, + Stderr: &stderr, + WorkingDir: t.TempDir(), + }.Run([]string{"migrate"}) + if err == nil { + t.Fatalf("Run(migrate) error = nil, want missing-source error") + } + if !strings.Contains(err.Error(), "migrate requires a source") { + t.Fatalf("Run(migrate) error = %v, want %q", err, "migrate requires a source") + } + if !strings.Contains(stderr.String(), "Usage: loaf migrate [options]") { + t.Fatalf("stderr = %q, want migrate usage line", stderr.String()) + } + if stdout.String() != "" { + t.Fatalf("stdout = %q, want empty on the error path", stdout.String()) + } +} + +func TestRunnerMigrateEveryListedSourceDispatches(t *testing.T) { + // Behavioral probe: every source writeMigrateHelp advertises must reach a + // runner instead of the not-implemented fallthrough. Each runner rejects + // the unknown option during arg parsing, before any state access, so no + // database is ever touched. + t.Setenv("LOAF_DB", filepath.Join(t.TempDir(), "loaf.sqlite")) + for _, source := range migrateDispatchableSources { + t.Run(source, func(t *testing.T) { + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run([]string{"migrate", source, "--no-such-option"}) + if err == nil { + t.Fatalf("Run(migrate %s --no-such-option) error = nil, want unknown-option rejection from the source runner", source) + } + if strings.Contains(err.Error(), "is not implemented yet") { + t.Fatalf("migrate %q did not dispatch: %v", source, err) + } + }) + } +} + +func TestRunnerMigrateSourceHelpIsNative(t *testing.T) { + // Anchored to the runMigrate switch in cli.go: every dispatchable + // top-level migrate source must answer --help with its usage. + for _, source := range migrateDispatchableSources { + t.Run(source, func(t *testing.T) { + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run([]string{"migrate", source, "--help"}) + if err != nil { + t.Fatalf("Run(migrate %s --help) error = %v", source, err) + } + want := "Usage: loaf migrate " + source + if !strings.Contains(stdout.String(), want) { + t.Fatalf("stdout = %q, want %q", stdout.String(), want) + } + }) + } +} + +func TestRunnerConversationLeafHelpIsNative(t *testing.T) { + cases := []struct { + name string + args []string + want string + }{ + {name: "conversation handle add --help", args: []string{"conversation", "handle", "add", "--help"}, want: "Usage: loaf conversation handle add "}, + {name: "conversation handle add -h", args: []string{"conversation", "handle", "add", "-h"}, want: "Usage: loaf conversation handle add "}, + {name: "exploration conversation add --help", args: []string{"exploration", "conversation", "add", "--help"}, want: "Usage: loaf exploration conversation add "}, + {name: "exploration conversation add -h", args: []string{"exploration", "conversation", "add", "-h"}, want: "Usage: loaf exploration conversation add "}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + var stdout bytes.Buffer + err := Runner{ + Stdout: &stdout, + WorkingDir: t.TempDir(), + }.Run(tc.args) + if err != nil { + t.Fatalf("Run(%v) error = %v", tc.args, err) + } + if !strings.Contains(stdout.String(), tc.want) { + t.Fatalf("stdout = %q, want %q", stdout.String(), tc.want) + } + }) + } +} diff --git a/internal/cli/install_codex_rules.go b/internal/cli/install_codex_rules.go index 8abe44e7..267bc21f 100644 --- a/internal/cli/install_codex_rules.go +++ b/internal/cli/install_codex_rules.go @@ -504,6 +504,14 @@ func validateCodexJournalExecutable(projectRoot string) error { return err } +// trustedCodexJournalExecutable resolves the loaf PATH entry into the path +// rendered across Codex-managed surfaces (AGENTS.md guidance, the hooks.json +// command, and loaf.rules execpolicy prefixes). The returned render path is +// the absolute PATH entrypoint without symlink resolution, so upgrades that +// retarget the entrypoint (a Homebrew Cellar repoint) never invalidate +// rendered policy. Validation strictness is unchanged: the canonical +// EvalSymlinks target must exist and stay outside forbidden roots, and both +// the render and canonical paths must be guidance-safe. func trustedCodexJournalExecutable(projectRoot string, operations *codexRuleInstallOperations) (string, error) { if projectRoot == "" { projectRoot, _ = os.Getwd() @@ -522,6 +530,10 @@ func trustedCodexJournalExecutable(projectRoot string, operations *codexRuleInst if err != nil { return "", fmt.Errorf("cannot trust Codex basic command policy: loaf executable is not on PATH: %w", err) } + render, err := filepath.Abs(path) + if err != nil { + return "", fmt.Errorf("cannot trust Codex basic command policy: absolutize loaf executable %s: %w", path, err) + } canonical, err := filepath.EvalSymlinks(path) if err != nil { return "", fmt.Errorf("cannot trust Codex basic command policy: resolve loaf executable %s: %w", path, err) @@ -530,7 +542,7 @@ func trustedCodexJournalExecutable(projectRoot string, operations *codexRuleInst if err != nil { return "", fmt.Errorf("cannot trust Codex basic command policy: canonicalize loaf executable: %w", err) } - if strings.ContainsAny(canonical, "`\r\n") { + if strings.ContainsAny(render, "`\r\n") || strings.ContainsAny(canonical, "`\r\n") { return "", fmt.Errorf("cannot trust Codex basic command policy: executable path contains unsupported guidance characters") } for _, forbidden := range forbiddenRoots { @@ -548,8 +560,17 @@ func trustedCodexJournalExecutable(projectRoot string, operations *codexRuleInst if pathWithinInstall(forbiddenPath, canonical) { return "", fmt.Errorf("cannot trust Codex basic command policy: loaf executable %s is inside forbidden path %s", canonical, forbiddenPath) } + // The render path is what rendered policy ultimately trusts; an + // entrypoint inside a forbidden root is rejected even when its + // canonical target is not. + if absForbidden, absErr := filepath.Abs(forbidden); absErr == nil && pathWithinInstall(absForbidden, render) { + return "", fmt.Errorf("cannot trust Codex basic command policy: loaf executable %s is inside forbidden path %s", render, absForbidden) + } + if pathWithinInstall(forbiddenPath, render) { + return "", fmt.Errorf("cannot trust Codex basic command policy: loaf executable %s is inside forbidden path %s", render, forbiddenPath) + } } - return canonical, nil + return render, nil } func codexJournalForbiddenExecutableRoots(projectRoot string) []string { diff --git a/internal/cli/install_codex_rules_test.go b/internal/cli/install_codex_rules_test.go index 2893ee0f..f7ecbf1b 100644 --- a/internal/cli/install_codex_rules_test.go +++ b/internal/cli/install_codex_rules_test.go @@ -7,6 +7,7 @@ import ( "os" "os/exec" "path/filepath" + "strconv" "strings" "testing" ) @@ -94,6 +95,185 @@ func TestValidateCodexJournalExecutableRejectsMissingAndDisposablePaths(t *testi }) } +func TestTrustedCodexJournalExecutableRendersEntrypointNotCanonicalTarget(t *testing.T) { + root := realpath(t, t.TempDir()) + target := filepath.Join(root, "cellar-1.0", "loaf") + writeCodexExecutableFixture(t, target) + entrypoint := filepath.Join(root, "bin", "loaf") + symlinkCodexEntrypoint(t, target, entrypoint) + got, err := trustedCodexJournalExecutable(filepath.Join(root, "project"), codexEntrypointOperations(root, entrypoint)) + if err != nil { + t.Fatalf("trusted executable error = %v", err) + } + if got != entrypoint || !filepath.IsAbs(got) { + t.Fatalf("trusted executable = %q, want absolute stable entrypoint %q", got, entrypoint) + } + if strings.Contains(got, "cellar-1.0") { + t.Fatalf("trusted executable = %q, want no canonicalized segment", got) + } +} + +func TestTrustedCodexJournalExecutablePlainFileRenderEqualsCanonical(t *testing.T) { + root := realpath(t, t.TempDir()) + path := filepath.Join(root, "trusted-bin", "loaf") + writeCodexExecutableFixture(t, path) + got, err := trustedCodexJournalExecutable(filepath.Join(root, "project"), codexEntrypointOperations(root, path)) + if err != nil { + t.Fatalf("trusted executable error = %v", err) + } + if got != path { + t.Fatalf("trusted executable = %q, want plain-file path %q where render equals canonical", got, path) + } +} + +func TestTrustedCodexJournalExecutableRejectsForbiddenTargetBehindSymlink(t *testing.T) { + root := realpath(t, t.TempDir()) + project := filepath.Join(root, "project") + target := filepath.Join(project, "bin", "loaf") + writeCodexExecutableFixture(t, target) + entrypoint := filepath.Join(root, "bin", "loaf") + symlinkCodexEntrypoint(t, target, entrypoint) + if _, err := trustedCodexJournalExecutable(project, codexEntrypointOperations(root, entrypoint)); err == nil || !strings.Contains(err.Error(), "forbidden path") { + t.Fatalf("trust validation error = %v, want forbidden-target refusal", err) + } +} + +func TestTrustedCodexJournalExecutableRejectsForbiddenEntrypointWithLegitimateTarget(t *testing.T) { + // The mirror of the forbidden-target case: the PATH entrypoint itself sits + // inside a forbidden root while its canonical target is outside. Only the + // render path is disposable here, and the render path is what rendered + // policy ultimately trusts, so canonical-only validation would pass it. + root := realpath(t, t.TempDir()) + project := filepath.Join(root, "project") + target := filepath.Join(root, "cellar-1.0", "loaf") + writeCodexExecutableFixture(t, target) + entrypoint := filepath.Join(project, "bin", "loaf") + symlinkCodexEntrypoint(t, target, entrypoint) + _, err := trustedCodexJournalExecutable(project, codexEntrypointOperations(root, entrypoint)) + if err == nil || !strings.Contains(err.Error(), "forbidden path") { + t.Fatalf("trust validation error = %v, want forbidden-entrypoint refusal", err) + } + if !strings.Contains(err.Error(), entrypoint) { + t.Fatalf("trust validation error = %v, want the rejected entrypoint %q named", err, entrypoint) + } + if strings.Contains(err.Error(), "cellar-1.0") { + t.Fatalf("trust validation error = %v, want no legitimate canonical target blamed", err) + } +} + +func TestTrustedCodexJournalExecutableRejectsGuidanceCharactersInEitherPath(t *testing.T) { + t.Run("entrypoint", func(t *testing.T) { + root := realpath(t, t.TempDir()) + target := filepath.Join(root, "cellar-1.0", "loaf") + writeCodexExecutableFixture(t, target) + entrypoint := filepath.Join(root, "bin`tick", "loaf") + symlinkCodexEntrypoint(t, target, entrypoint) + if _, err := trustedCodexJournalExecutable(filepath.Join(root, "project"), codexEntrypointOperations(root, entrypoint)); err == nil || !strings.Contains(err.Error(), "unsupported guidance characters") { + t.Fatalf("trust validation error = %v, want guidance-character refusal for entrypoint", err) + } + }) + t.Run("canonical target", func(t *testing.T) { + root := realpath(t, t.TempDir()) + target := filepath.Join(root, "cellar`1.0", "loaf") + writeCodexExecutableFixture(t, target) + entrypoint := filepath.Join(root, "bin", "loaf") + symlinkCodexEntrypoint(t, target, entrypoint) + if _, err := trustedCodexJournalExecutable(filepath.Join(root, "project"), codexEntrypointOperations(root, entrypoint)); err == nil || !strings.Contains(err.Error(), "unsupported guidance characters") { + t.Fatalf("trust validation error = %v, want guidance-character refusal for canonical target", err) + } + }) +} + +func TestInstallCodexJournalRuleRendersSymlinkedEntrypointAcrossSurfaces(t *testing.T) { + fixture := newCodexRuleInstallFixture(t) + target := filepath.Join(fixture.root, "cellar-1.0", "loaf") + writeCodexExecutableFixture(t, target) + entrypoint := filepath.Join(fixture.root, "bin", "loaf") + symlinkCodexEntrypoint(t, target, entrypoint) + operations := codexEntrypointOperations(fixture.root, entrypoint) + if err := installCodexJournalRuleWithOperations(fixture.options(true, false), fixture.codexHome, operations); err != nil { + t.Fatalf("install with symlinked entrypoint: %v", err) + } + rule, err := os.ReadFile(fixture.dest()) + if err != nil { + t.Fatalf("read rendered rule: %v", err) + } + if !strings.Contains(string(rule), strconv.Quote(entrypoint)) || strings.Contains(string(rule), "cellar-1.0") { + t.Fatalf("rendered rule = %q, want entrypoint prefixes without canonicalized segment", rule) + } + guidance, err := os.ReadFile(filepath.Join(fixture.codexHome, codexJournalGuidanceRelativePath)) + if err != nil { + t.Fatalf("read rendered guidance: %v", err) + } + if !strings.Contains(string(guidance), journalContextShellQuote(entrypoint)+" journal log --execpolicy-safe") || strings.Contains(string(guidance), "cellar-1.0") { + t.Fatalf("rendered guidance = %q, want entrypoint command without canonicalized segment", guidance) + } + hooksDest := filepath.Join(fixture.codexHome, "hooks.json") + loafHooks := filepath.Join(fixture.dist, ".codex", "hooks.json") + writeInstallFile(t, loafHooks, `{"hooks":{"SessionStart":[{"matcher":"startup|resume|clear|compact","hooks":[{"type":"command","command":"{{LOAF_EXECUTABLE}} journal context --from-hook --codex-hook"}]}]}}`) + writeInstallFile(t, hooksDest, `{"hooks":{}}`) + if err := mergeCodexHookFilesForOS(hooksDest, loafHooks, filepath.Join(fixture.root, "project"), operations, "darwin"); err != nil { + t.Fatalf("merge hooks with symlinked entrypoint: %v", err) + } + hooks, err := os.ReadFile(hooksDest) + if err != nil { + t.Fatalf("read rendered hooks: %v", err) + } + if !strings.Contains(string(hooks), journalContextShellQuote(entrypoint)+codexJournalHookCommandSuffix) || strings.Contains(string(hooks), "cellar-1.0") { + t.Fatalf("rendered hooks = %q, want entrypoint command without canonicalized segment", hooks) + } +} + +func TestTrustedCodexJournalExecutableSurvivesEntrypointRetarget(t *testing.T) { + fixture := newCodexRuleInstallFixture(t) + oldTarget := filepath.Join(fixture.root, "cellar-1.0", "loaf") + writeCodexExecutableFixture(t, oldTarget) + entrypoint := filepath.Join(fixture.root, "bin", "loaf") + symlinkCodexEntrypoint(t, oldTarget, entrypoint) + operations := codexEntrypointOperations(fixture.root, entrypoint) + rendered, err := trustedCodexJournalExecutable(filepath.Join(fixture.root, "project"), operations) + if err != nil { + t.Fatalf("trusted executable error = %v", err) + } + if err := installCodexJournalRuleWithOperations(fixture.options(true, false), fixture.codexHome, operations); err != nil { + t.Fatalf("install with symlinked entrypoint: %v", err) + } + installedRule, err := os.ReadFile(fixture.dest()) + if err != nil { + t.Fatalf("read installed rule: %v", err) + } + + // Simulate the Homebrew upgrade: the versioned target moves, the stable + // entrypoint is repointed, and the old Cellar directory disappears. + newTarget := filepath.Join(fixture.root, "cellar-2.0", "loaf") + writeCodexExecutableFixture(t, newTarget) + if err := os.Remove(entrypoint); err != nil { + t.Fatalf("remove entrypoint symlink: %v", err) + } + symlinkCodexEntrypoint(t, newTarget, entrypoint) + if err := os.RemoveAll(filepath.Join(fixture.root, "cellar-1.0")); err != nil { + t.Fatalf("remove stale versioned target: %v", err) + } + + if _, err := os.Stat(rendered); err != nil { + t.Fatalf("previously rendered executable path stat = %v, want upgrade survival", err) + } + if _, err := os.Stat(oldTarget); !os.IsNotExist(err) { + t.Fatalf("stale canonical target stat = %v, want removal proving a canonical pin would strand", err) + } + // The owned upgrade converges to byte-identical content: nothing to rewrite. + if err := installCodexJournalRuleWithOperations(fixture.options(false, true), fixture.codexHome, operations); err != nil { + t.Fatalf("upgrade after retarget: %v", err) + } + upgradedRule, err := os.ReadFile(fixture.dest()) + if err != nil { + t.Fatalf("read upgraded rule: %v", err) + } + if string(upgradedRule) != string(installedRule) { + t.Fatalf("upgraded rule = %q, want previously rendered content untouched %q", upgradedRule, installedRule) + } +} + func TestCodexJournalRuleExecpolicyClassification(t *testing.T) { codex, err := exec.LookPath("codex") if err != nil { @@ -646,6 +826,36 @@ func (f codexRuleInstallFixture) installWithExecutable(t *testing.T, path string return installCodexJournalRuleWithOperations(f.options(autoJournal, upgrade), f.codexHome, operations) } +func writeCodexExecutableFixture(t *testing.T, path string) { + t.Helper() + writeInstallFile(t, path, "#!/bin/sh\nexit 0\n") + if err := os.Chmod(path, 0o755); err != nil { + t.Fatalf("chmod executable fixture: %v", err) + } +} + +func symlinkCodexEntrypoint(t *testing.T, target string, entrypoint string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(entrypoint), 0o755); err != nil { + t.Fatalf("create entrypoint directory: %v", err) + } + if err := os.Symlink(target, entrypoint); err != nil { + t.Fatalf("symlink entrypoint: %v", err) + } +} + +func codexEntrypointOperations(root string, entrypoint string) *codexRuleInstallOperations { + return &codexRuleInstallOperations{ + lookPath: func(name string) (string, error) { + if name != "loaf" { + return "", fmt.Errorf("unexpected executable %q", name) + } + return entrypoint, nil + }, + forbiddenRoots: []string{filepath.Join(root, "project")}, + } +} + func readCodexRuleManifestTest(t *testing.T, path string) codexManagedRuleManifest { t.Helper() manifest, err := readCodexManagedRuleManifest(path) diff --git a/internal/state/exploration.go b/internal/state/exploration.go index 67c3cad7..b1907a6e 100644 --- a/internal/state/exploration.go +++ b/internal/state/exploration.go @@ -602,8 +602,8 @@ func writeExplorationSourceRelationshipsTx(ctx context.Context, tx *sql.Tx, proj relationshipID := stableMigrationID("relationship", projectID, "exploration", explorationID, relationshipType, entity.Kind, entity.ID) if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, 'exploration', ?, ?, ?, ?, 'recorded by exploration create', 'exploration-create', ?, ?) -`, relationshipID, projectID, explorationID, entity.Kind, entity.ID, relationshipType, now, now); err != nil { +VALUES (?, ?, 'exploration', ?, ?, ?, ?, 'recorded by exploration create', ?, ?, ?) +`, relationshipID, projectID, explorationID, entity.Kind, entity.ID, relationshipType, relationshipOriginCommand, now, now); err != nil { return &ExplorationTransactionError{Stage: "relationship", Err: err} } } diff --git a/internal/state/finding.go b/internal/state/finding.go index 42b8e452..376f84c8 100644 --- a/internal/state/finding.go +++ b/internal/state/finding.go @@ -236,7 +236,7 @@ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) } if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, 'report', ?, 'finding', ?, 'contains', 'recorded by finding create', 'system', ?, ?) +VALUES (?, ?, 'report', ?, 'finding', ?, 'contains', 'recorded by finding create', 'command', ?, ?) ON CONFLICT(id) DO NOTHING `, stableMigrationID("relationship", projectID, "report", report.ID, "contains", "finding", findingID), projectID, report.ID, findingID, timestamp, timestamp); err != nil { return FindingCreateResult{}, fmt.Errorf("record report finding relationship: %w", err) @@ -244,7 +244,7 @@ ON CONFLICT(id) DO NOTHING if run != nil { if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, 'run', ?, 'finding', ?, 'produces', 'recorded by finding create', 'system', ?, ?) +VALUES (?, ?, 'run', ?, 'finding', ?, 'produces', 'recorded by finding create', 'command', ?, ?) ON CONFLICT(id) DO NOTHING `, stableMigrationID("relationship", projectID, "run", run.ID, "produces", "finding", findingID), projectID, run.ID, findingID, timestamp, timestamp); err != nil { return FindingCreateResult{}, fmt.Errorf("record run finding relationship: %w", err) @@ -447,7 +447,7 @@ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) } if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, 'finding', ?, 'verdict', ?, 'adjudicated_by', 'recorded by finding verdict', 'system', ?, ?) +VALUES (?, ?, 'finding', ?, 'verdict', ?, 'adjudicated_by', 'recorded by finding verdict', 'command', ?, ?) ON CONFLICT(id) DO NOTHING `, stableMigrationID("relationship", projectID, "finding", finding.ID, "adjudicated_by", "verdict", verdictID), projectID, finding.ID, verdictID, now, now); err != nil { return FindingVerdictResult{}, fmt.Errorf("record finding verdict relationship: %w", err) @@ -455,7 +455,7 @@ ON CONFLICT(id) DO NOTHING if run != nil { if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, 'run', ?, 'verdict', ?, 'records', 'recorded by finding verdict', 'system', ?, ?) +VALUES (?, ?, 'run', ?, 'verdict', ?, 'records', 'recorded by finding verdict', 'command', ?, ?) ON CONFLICT(id) DO NOTHING `, stableMigrationID("relationship", projectID, "run", run.ID, "records", "verdict", verdictID), projectID, run.ID, verdictID, now, now); err != nil { return FindingVerdictResult{}, fmt.Errorf("record run verdict relationship: %w", err) diff --git a/internal/state/intent.go b/internal/state/intent.go index 26d36461..3a06d57a 100644 --- a/internal/state/intent.go +++ b/internal/state/intent.go @@ -1001,8 +1001,8 @@ func writeIntentSourceRelationshipsTx(ctx context.Context, tx *sql.Tx, projectID relationshipID := stableMigrationID("relationship", projectID, entity.Kind, entity.ID, "source-of", "intent", intentID) if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, ?, ?, 'intent', ?, 'source-of', 'recorded by intent create', 'intent-create', ?, ?) -`, relationshipID, projectID, entity.Kind, entity.ID, intentID, now, now); err != nil { +VALUES (?, ?, ?, ?, 'intent', ?, 'source-of', 'recorded by intent create', ?, ?, ?) +`, relationshipID, projectID, entity.Kind, entity.ID, intentID, relationshipOriginCommand, now, now); err != nil { return nil, &IntentTransactionError{Stage: "relationship", Err: err} } sources = append(sources, TraceRelationship{Direction: "inbound", Type: "source-of", Entity: entity, Reason: "recorded by intent create"}) diff --git a/internal/state/intent_conversion.go b/internal/state/intent_conversion.go index da907649..19818f56 100644 --- a/internal/state/intent_conversion.go +++ b/internal/state/intent_conversion.go @@ -270,9 +270,9 @@ VALUES (?, ?, ?, ?, ?, ?, 1, ?, ?) relationshipID := stableMigrationID("relationship", projectID, sourceEdge.kind, sourceEdge.id, "source-of", "intent", intentID) if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, ?, ?, 'intent', ?, 'source-of', 'legacy deferral conversion', 'legacy-conversion', ?, ?) +VALUES (?, ?, ?, ?, 'intent', ?, 'source-of', 'legacy deferral conversion', ?, ?, ?) ON CONFLICT(id) DO NOTHING -`, relationshipID, projectID, sourceEdge.kind, sourceEdge.id, intentID, timestamp, timestamp); err != nil { +`, relationshipID, projectID, sourceEdge.kind, sourceEdge.id, intentID, relationshipOriginCommand, timestamp, timestamp); err != nil { return result, fmt.Errorf("insert converted relationship %s: %w", row.OperationKey, err) } } diff --git a/internal/state/relationship_origin.go b/internal/state/relationship_origin.go new file mode 100644 index 00000000..75bb02b9 --- /dev/null +++ b/internal/state/relationship_origin.go @@ -0,0 +1,89 @@ +package state + +import ( + "sort" + "strings" +) + +// Relationship origin is closed at mechanism level: origin answers "by what +// mechanism did this row appear" while reason carries the operation detail +// ("recorded by intent create"). Writers, the doctor invariant SQL, and +// `state repair relationship-origin` all derive from this registry instead of +// restating the literals, so the vocabulary cannot drift again +// (path-stability-and-origin-hygiene, Decisions 2-4). +const ( + // relationshipOriginImported marks rows carried in by imports and migrations. + relationshipOriginImported = "imported" + // relationshipOriginManual marks rows written by explicit operator link + // commands (Store.CreateLink) or operator-selected backfills; it is a live + // code-written value, not a hand-authored-only one. + relationshipOriginManual = "manual" + // relationshipOriginCommand marks rows written by CLI ceremonies. + relationshipOriginCommand = "command" +) + +// allowedRelationshipOrigins returns the closed mechanism-level vocabulary in +// registration order. Consumers build SQL and validation from this list. +func allowedRelationshipOrigins() []string { + return []string{relationshipOriginImported, relationshipOriginManual, relationshipOriginCommand} +} + +// legacyRelationshipOriginReclassifications maps the retired origin values — +// written before the vocabulary closed — to their mechanism-level replacement. +// Repair reclassifies exactly these; any other unknown origin is foreign +// provenance that stays visible as a doctor warning and is never rewritten. +// +// 'intent-create', 'exploration-create', and 'legacy-conversion' are the +// retired per-ceremony values from the Intent, Exploration, and +// legacy-conversion writers. +// +// 'system' is retired writer provenance of a different shape: run.go and +// finding.go inlined it on the run→report, report→finding, run→finding, +// finding→verdict, and run→verdict relationships until 2026-07-19, when the +// source-literal parity scan exposed them and this Change normalized those +// writers to 'command'. Released alphas shipped the 'system' writers, so user +// databases can hold 'system' rows that no live writer produces — they are +// reclassifiable legacy, not foreign provenance. +func legacyRelationshipOriginReclassifications() map[string]string { + return map[string]string{ + "intent-create": relationshipOriginCommand, + "legacy-conversion": relationshipOriginCommand, + "exploration-create": relationshipOriginCommand, + "system": relationshipOriginCommand, + } +} + +// legacyRelationshipOrigins returns the reclassifiable legacy origins in +// deterministic order for planning and reporting. +func legacyRelationshipOrigins() []string { + legacy := legacyRelationshipOriginReclassifications() + origins := make([]string, 0, len(legacy)) + for origin := range legacy { + origins = append(origins, origin) + } + sort.Strings(origins) + return origins +} + +// relationshipOriginNotAllowedFragment builds a parameterized `column NOT IN +// (...)` fragment covering the allowed vocabulary, plus its bind arguments, so +// SQL consumers never inline the origin literals. +func relationshipOriginNotAllowedFragment(column string) (string, []any) { + return parameterizedNotInFragment(column, allowedRelationshipOrigins()) +} + +// legacyRelationshipOriginNotInFragment builds the same parameterized fragment +// over the reclassifiable legacy origins. +func legacyRelationshipOriginNotInFragment(column string) (string, []any) { + return parameterizedNotInFragment(column, legacyRelationshipOrigins()) +} + +func parameterizedNotInFragment(column string, values []string) (string, []any) { + placeholders := make([]string, len(values)) + args := make([]any, len(values)) + for i, value := range values { + placeholders[i] = "?" + args[i] = value + } + return column + " NOT IN (" + strings.Join(placeholders, ", ") + ")", args +} diff --git a/internal/state/relationship_origin_test.go b/internal/state/relationship_origin_test.go new file mode 100644 index 00000000..5fe65455 --- /dev/null +++ b/internal/state/relationship_origin_test.go @@ -0,0 +1,469 @@ +package state + +import ( + "context" + "fmt" + "go/ast" + "go/parser" + "go/token" + "os" + "reflect" + "regexp" + "strconv" + "strings" + "testing" +) + +func TestRelationshipOriginRegistryPinsClosedVocabulary(t *testing.T) { + // The vocabulary is closed at mechanism level; any drift from the + // documented values is a contract break, not a refactor. + wantAllowed := []string{"imported", "manual", "command"} + if got := allowedRelationshipOrigins(); !reflect.DeepEqual(got, wantAllowed) { + t.Fatalf("allowedRelationshipOrigins() = %#v, want %#v", got, wantAllowed) + } + + wantLegacy := map[string]string{ + "intent-create": "command", + "legacy-conversion": "command", + "exploration-create": "command", + "system": "command", + } + if got := legacyRelationshipOriginReclassifications(); !reflect.DeepEqual(got, wantLegacy) { + t.Fatalf("legacyRelationshipOriginReclassifications() = %#v, want %#v", got, wantLegacy) + } + + fragment, args := relationshipOriginNotAllowedFragment("origin") + if fragment != "origin NOT IN (?, ?, ?)" { + t.Fatalf("relationshipOriginNotAllowedFragment() = %q, want parameterized NOT IN over three origins", fragment) + } + if !reflect.DeepEqual(args, []any{"imported", "manual", "command"}) { + t.Fatalf("relationshipOriginNotAllowedFragment() args = %#v, want allowed origins", args) + } +} + +// Source-scan patterns for statements that write the relationships.origin +// column. The scan below keeps Decision 3 executable: the next ceremony cannot +// ship a novel provenance value without failing the parity test. +var ( + relationshipInsertPattern = regexp.MustCompile(`(?is)insert\s+into\s+relationships\s*\(`) + relationshipUpdateSetPattern = regexp.MustCompile(`(?is)update\s+relationships\s+set\b`) + sqlValuesOpenPattern = regexp.MustCompile(`(?is)^\s*values\s*\(`) + sqlWherePattern = regexp.MustCompile(`(?i)\bwhere\b`) + sqlOriginAssignPattern = regexp.MustCompile(`(?i)\borigin\s*=\s*`) +) + +// relationshipOriginWrite describes one origin write found inside a SQL string +// literal: either an inline SQL literal value, or the ordinal of its `?` among +// every placeholder in the statement for mapping onto the enclosing call's +// bind arguments. +type relationshipOriginWrite struct { + sqlLiteral string + isLiteral bool + placeholder int +} + +// findRelationshipOriginWrites locates every write to relationships.origin in +// one SQL string: INSERT column/VALUES pairs and UPDATE ... SET assignments. +// Unrecognizable origin expressions are errors so the invariant fails closed +// instead of silently skipping a statement the scan cannot classify. +func findRelationshipOriginWrites(query string) ([]relationshipOriginWrite, error) { + writes := []relationshipOriginWrite{} + for _, match := range relationshipInsertPattern.FindAllStringIndex(query, -1) { + columnsEnd := strings.Index(query[match[1]:], ")") + if columnsEnd < 0 { + return nil, fmt.Errorf("unterminated relationships column list in %q", query) + } + columns := strings.Split(query[match[1]:match[1]+columnsEnd], ",") + originIndex := -1 + for i, column := range columns { + if strings.EqualFold(strings.TrimSpace(column), "origin") { + originIndex = i + } + } + if originIndex == -1 { + continue + } + afterColumns := match[1] + columnsEnd + 1 + valuesOpen := sqlValuesOpenPattern.FindStringIndex(query[afterColumns:]) + if valuesOpen == nil { + return nil, fmt.Errorf("origin-writing INSERT without a recognizable VALUES tuple in %q", query) + } + items, offsets, err := splitSQLTupleItems(query, afterColumns+valuesOpen[1]) + if err != nil { + return nil, err + } + if originIndex >= len(items) { + return nil, fmt.Errorf("VALUES tuple shorter than column list in %q", query) + } + write, err := classifySQLOriginValue(query, items[originIndex], offsets[originIndex]) + if err != nil { + return nil, err + } + writes = append(writes, write) + } + for _, match := range relationshipUpdateSetPattern.FindAllStringIndex(query, -1) { + clauseEnd := len(query) + if where := sqlWherePattern.FindStringIndex(query[match[1]:]); where != nil { + clauseEnd = match[1] + where[0] + } + assign := sqlOriginAssignPattern.FindStringIndex(query[match[1]:clauseEnd]) + if assign == nil { + continue + } + valueStart := match[1] + assign[1] + value := query[valueStart:clauseEnd] + if comma := indexTopLevelComma(value); comma >= 0 { + value = value[:comma] + } + write, err := classifySQLOriginValue(query, value, valueStart) + if err != nil { + return nil, err + } + writes = append(writes, write) + } + return writes, nil +} + +// splitSQLTupleItems splits a parenthesized SQL tuple starting just after its +// opening parenthesis, honoring single-quoted literals and nested parentheses. +// It returns each item trimmed alongside the absolute offset of its first +// non-space character. +func splitSQLTupleItems(query string, start int) ([]string, []int, error) { + depth := 1 + inQuote := false + items := []string{} + offsets := []int{} + itemStart := start + record := func(end int) { + raw := query[itemStart:end] + leading := len(raw) - len(strings.TrimLeft(raw, " \t\r\n")) + items = append(items, strings.TrimSpace(raw)) + offsets = append(offsets, itemStart+leading) + } + for i := start; i < len(query); i++ { + switch query[i] { + case '\'': + inQuote = !inQuote + case '(': + if !inQuote { + depth++ + } + case ')': + if !inQuote { + depth-- + if depth == 0 { + record(i) + return items, offsets, nil + } + } + case ',': + if !inQuote && depth == 1 { + record(i) + itemStart = i + 1 + } + } + } + return nil, nil, fmt.Errorf("unterminated VALUES tuple in %q", query) +} + +// indexTopLevelComma returns the offset of the first comma outside quotes and +// parentheses, or -1. +func indexTopLevelComma(fragment string) int { + depth := 0 + inQuote := false + for i := 0; i < len(fragment); i++ { + switch fragment[i] { + case '\'': + inQuote = !inQuote + case '(': + if !inQuote { + depth++ + } + case ')': + if !inQuote { + depth-- + } + case ',': + if !inQuote && depth == 0 { + return i + } + } + } + return -1 +} + +// classifySQLOriginValue resolves one origin value expression to either its +// inline SQL literal or the ordinal of its placeholder within the statement. +func classifySQLOriginValue(query string, item string, offset int) (relationshipOriginWrite, error) { + trimmed := strings.TrimSpace(item) + switch { + case trimmed == "?": + return relationshipOriginWrite{placeholder: strings.Count(query[:offset], "?")}, nil + case len(trimmed) >= 2 && strings.HasPrefix(trimmed, "'") && strings.HasSuffix(trimmed, "'"): + return relationshipOriginWrite{isLiteral: true, sqlLiteral: trimmed[1 : len(trimmed)-1]}, nil + default: + return relationshipOriginWrite{}, fmt.Errorf("unrecognized origin value %q in %q; keep origin values scannable (literal or ?)", trimmed, query) + } +} + +// relationshipOriginRuntimeBinds classifies the origin binds that are runtime +// values by construction and therefore cannot be resolved statically. Each key +// is file:function:identifier — stable across edits, unlike a line number — and +// each entry is a deliberate decision that the named value is validated against +// the registry before it reaches SQL: +// +// - repair's backfill origin is the operator's --origin selection, rejected by +// RepairRelationshipOrigins unless it is 'imported' or 'manual'. +// - repair's reclassification target is read from +// legacyRelationshipOriginReclassifications, whose values are registry +// constants pinned by TestRelationshipOriginRegistryPinsClosedVocabulary. +// +// Any other unresolvable bind fails the scan. Adding a key here is the explicit +// way to say "this one is checked elsewhere"; silence is not. +var relationshipOriginRuntimeBinds = map[string]bool{ + "repair.go:backfillMissingRelationshipOrigins:origin": true, + "repair.go:reclassifyLegacyRelationshipOrigin:target": true, +} + +// enclosingFunctionName names the function declaration containing pos, or "" +// for package-level expressions. +func enclosingFunctionName(file *ast.File, pos token.Pos) string { + for _, decl := range file.Decls { + funcDecl, ok := decl.(*ast.FuncDecl) + if !ok || pos < funcDecl.Pos() || pos > funcDecl.End() { + continue + } + return funcDecl.Name.Name + } + return "" +} + +// TestRelationshipOriginSourceLiteralsAreRegistryListed is the registry parity +// scan: every non-test source file in this package is parsed and every origin +// literal written to relationships.origin — inline in SQL, bound as a Go +// string literal, or bound through a package-level string constant — must be +// registry-listed, which also keeps the retired legacy origins from +// reappearing in origin position. +// +// The scan fails closed. A bind it cannot classify — a local variable, a field, +// a call result — is an error, not a skip, because `origin := "new-ceremony"` +// would otherwise evade the invariant entirely. The only escape is an explicit +// relationshipOriginRuntimeBinds entry naming a caller-validated runtime value. +func TestRelationshipOriginSourceLiteralsAreRegistryListed(t *testing.T) { + allowed := map[string]bool{} + for _, origin := range allowedRelationshipOrigins() { + allowed[origin] = true + } + + entries, err := os.ReadDir(".") + if err != nil { + t.Fatalf("read package directory: %v", err) + } + fset := token.NewFileSet() + files := map[string]*ast.File{} + for _, entry := range entries { + name := entry.Name() + if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") { + continue + } + file, err := parser.ParseFile(fset, name, nil, 0) + if err != nil { + t.Fatalf("parse %s: %v", name, err) + } + files[name] = file + } + + // Package-level string constants and variables resolve by name so a writer + // cannot launder a novel origin through one level of const indirection. + stringConstants := map[string]string{} + for _, file := range files { + for _, decl := range file.Decls { + genDecl, ok := decl.(*ast.GenDecl) + if !ok || (genDecl.Tok != token.CONST && genDecl.Tok != token.VAR) { + continue + } + for _, spec := range genDecl.Specs { + valueSpec, ok := spec.(*ast.ValueSpec) + if !ok || len(valueSpec.Names) != len(valueSpec.Values) { + continue + } + for i, ident := range valueSpec.Names { + literal, ok := valueSpec.Values[i].(*ast.BasicLit) + if !ok || literal.Kind != token.STRING { + continue + } + if value, err := strconv.Unquote(literal.Value); err == nil { + stringConstants[ident.Name] = value + } + } + } + } + } + + scanned := 0 + for fileName, file := range files { + processedQueries := map[token.Pos]bool{} + ast.Inspect(file, func(node ast.Node) bool { + call, ok := node.(*ast.CallExpr) + if !ok { + return true + } + for argIndex, arg := range call.Args { + queryLiteral, ok := arg.(*ast.BasicLit) + if !ok || queryLiteral.Kind != token.STRING { + continue + } + query, err := strconv.Unquote(queryLiteral.Value) + if err != nil { + continue + } + writes, err := findRelationshipOriginWrites(query) + if err != nil { + t.Errorf("%s: %v", fset.Position(queryLiteral.Pos()), err) + continue + } + if len(writes) > 0 { + processedQueries[queryLiteral.Pos()] = true + } + for _, write := range writes { + scanned++ + if write.isLiteral { + if !allowed[write.sqlLiteral] { + t.Errorf("%s: SQL origin literal %q is not registry-listed", fset.Position(queryLiteral.Pos()), write.sqlLiteral) + } + continue + } + boundIndex := argIndex + 1 + write.placeholder + if boundIndex >= len(call.Args) { + t.Errorf("%s: cannot statically resolve the origin bind argument (placeholder %d); keep bind arguments inline for this scan", fset.Position(queryLiteral.Pos()), write.placeholder) + continue + } + switch bound := call.Args[boundIndex].(type) { + case *ast.BasicLit: + if bound.Kind != token.STRING { + t.Errorf("%s: origin bound to non-string literal %s", fset.Position(bound.Pos()), bound.Value) + continue + } + value, err := strconv.Unquote(bound.Value) + if err != nil || !allowed[value] { + t.Errorf("%s: bound origin literal %s is not registry-listed", fset.Position(bound.Pos()), bound.Value) + } + case *ast.Ident: + if value, ok := stringConstants[bound.Name]; ok { + if !allowed[value] { + t.Errorf("%s: origin constant %s = %q is not registry-listed", fset.Position(bound.Pos()), bound.Name, value) + } + continue + } + key := fileName + ":" + enclosingFunctionName(file, bound.Pos()) + ":" + bound.Name + if !relationshipOriginRuntimeBinds[key] { + t.Errorf("%s: origin bound to unresolvable identifier %s (scan key %q); bind a registry constant, or add the key to relationshipOriginRuntimeBinds with a comment if this is a caller-validated runtime value", fset.Position(bound.Pos()), bound.Name, key) + } + default: + t.Errorf("%s: origin bound to unclassifiable expression %T; bind a registry constant or a caller-validated runtime value the scan can key on", fset.Position(call.Args[boundIndex].Pos()), call.Args[boundIndex]) + } + } + } + return true + }) + // A second pass over raw string literals: origin-writing SQL that is + // not an inline call argument evades placeholder mapping, so it must + // not exist. + ast.Inspect(file, func(node ast.Node) bool { + literal, ok := node.(*ast.BasicLit) + if !ok || literal.Kind != token.STRING { + return true + } + value, err := strconv.Unquote(literal.Value) + if err != nil { + return true + } + if processedQueries[literal.Pos()] { + return true + } + writes, err := findRelationshipOriginWrites(value) + if err == nil && len(writes) > 0 { + t.Errorf("%s: origin-writing SQL must be an inline argument of its exec call so this scan can resolve its bind arguments", fset.Position(literal.Pos())) + } + return true + }) + } + // The package currently holds 21 origin-writing statements; a collapsing + // scan would silently vacate the invariant, so pin a generous floor. + if scanned < 15 { + t.Fatalf("origin-write scan found %d statement(s), want at least 15 — the scanner or the writers regressed", scanned) + } +} + +func assertZeroRelationshipOriginDiagnostics(t *testing.T, store *Store, wantRelationships int) { + t.Helper() + ctx := context.Background() + var count int + if err := store.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM relationships WHERE origin = ?`, relationshipOriginCommand).Scan(&count); err != nil { + t.Fatalf("count command-origin relationships error = %v", err) + } + if count != wantRelationships { + t.Fatalf("command-origin relationships = %d, want %d", count, wantRelationships) + } + diagnostics, err := inspectRelationshipOriginInvariants(ctx, store) + if err != nil { + t.Fatalf("inspectRelationshipOriginInvariants() error = %v", err) + } + if len(diagnostics) != 0 { + t.Fatalf("diagnostics = %#v, want none after writer", diagnostics) + } +} + +func TestDoctorCleanAfterIntentCreateWithSource(t *testing.T) { + root, _, store := intentTestFixture(t) + ctx := context.Background() + spark, err := store.CaptureSpark(ctx, root, SparkCaptureOptions{Text: "origin hygiene source"}) + if err != nil { + t.Fatalf("CaptureSpark() error = %v", err) + } + if _, err := store.CreateIntent(ctx, root, IntentCreateOptions{ + Title: "Origin hygiene intent", + Body: "Body.", + Sources: []string{spark.Spark.Alias}, + }); err != nil { + t.Fatalf("CreateIntent() error = %v", err) + } + assertZeroRelationshipOriginDiagnostics(t, store, 1) +} + +func TestDoctorCleanAfterExplorationCreateWithSource(t *testing.T) { + root, _, store := explorationFixture(t) + ctx := context.Background() + intent, err := store.CreateIntent(ctx, root, IntentCreateOptions{Title: "Exploration source intent", Body: "Body."}) + if err != nil { + t.Fatalf("CreateIntent() error = %v", err) + } + if _, err := store.CreateExploration(ctx, root, ExplorationCreateOptions{ + Title: "Origin hygiene exploration", + Sources: []string{intent.Intent.Alias}, + }); err != nil { + t.Fatalf("CreateExploration() error = %v", err) + } + assertZeroRelationshipOriginDiagnostics(t, store, 1) +} + +func TestDoctorCleanAfterLegacyConversion(t *testing.T) { + root, resolver, store, databasePath := conversionFixture(t) + projectID := projectIDForTest(t, store, root) + seedLegacyDeferral(t, store, projectID, "legacy-origin", "Intent: legacy body\nWhy: why\nBoundary: boundary\nTrigger: trigger") + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + if _, err := ConvertLegacyDeferrals(context.Background(), root, resolver, true); err != nil { + t.Fatalf("ConvertLegacyDeferrals(apply) error = %v", err) + } + + reopened, err := OpenStore(databasePath) + if err != nil { + t.Fatalf("OpenStore() error = %v", err) + } + defer reopened.Close() + // Conversion writes one source-of edge per decision and spark source. + assertZeroRelationshipOriginDiagnostics(t, reopened, 2) +} diff --git a/internal/state/repair.go b/internal/state/repair.go index 0f69c378..56c23831 100644 --- a/internal/state/repair.go +++ b/internal/state/repair.go @@ -208,26 +208,94 @@ func inspectJournalSearchParityReadOnly(ctx context.Context, databasePath string return parity, nil } -// RelationshipOriginRepairOptions controls a guarded relationship provenance backfill. +// Relationship origin repair runs in one of two modes. Reclassify-only is the +// default because retiring the legacy origins is a mechanical, registry-derived +// rewrite that needs no operator judgement; backfilling a missing origin does +// need it, so it is opt-in through an explicit origin selection. +const ( + // RelationshipOriginRepairModeReclassifyOnly reclassifies the retired legacy + // origins and reports foreign ones. Rows with no origin are counted and + // reported but left untouched. + RelationshipOriginRepairModeReclassifyOnly = "reclassify-only" + // RelationshipOriginRepairModeBackfillAndReclassify additionally writes the + // operator-selected origin onto rows that have none. + RelationshipOriginRepairModeBackfillAndReclassify = "backfill-and-reclassify" +) + +// RelationshipOriginRepairOptions controls a guarded relationship provenance +// repair. An empty Origin selects reclassify-only mode; setting it to +// 'imported' or 'manual' also enables the missing-origin backfill. Any other +// value is rejected. type RelationshipOriginRepairOptions struct { Origin string Apply bool } -// RelationshipOriginRepairResult describes a dry-run or applied relationship provenance repair. +// RelationshipOriginReclassification reports one retired legacy origin group +// the repair plan reclassifies to its mechanism-level replacement. +type RelationshipOriginReclassification struct { + Origin string `json:"origin"` + Target string `json:"target"` + Matched int `json:"matched"` + Updated int `json:"updated"` +} + +// RelationshipOriginForeignGroup reports one origin value outside both the +// allowed vocabulary and the reclassifiable legacy set. Foreign provenance is +// surfaced for visibility and never rewritten; doctor keeps warning about it. +type RelationshipOriginForeignGroup struct { + Origin string `json:"origin"` + Count int `json:"count"` +} + +// RelationshipOriginRepairError preserves the partial repair result, including +// the pre-repair backup path, when an apply fails after the backup is taken. +// Without it a post-backup failure would surface a zero-value result and the +// operator would lose the reference to the backup that protects their data. +type RelationshipOriginRepairError struct { + Result RelationshipOriginRepairResult + Err error +} + +func (e *RelationshipOriginRepairError) Error() string { + if e == nil || e.Err == nil { + return "relationship origin repair failed" + } + message := e.Err.Error() + if e.Result.BackupPath != "" { + message += fmt.Sprintf("; preserved backup: %s", e.Result.BackupPath) + } + return message +} + +func (e *RelationshipOriginRepairError) Unwrap() error { + if e == nil { + return nil + } + return e.Err +} + +// RelationshipOriginRepairResult describes a dry-run or applied relationship +// provenance repair. Mode names which repair ran, so a consumer can never read +// Matched and Updated as a backfill that matched rows and failed to write them: +// in reclassify-only mode Matched counts rows the backfill would have covered +// had it been enabled, and Updated is always zero. type RelationshipOriginRepairResult struct { - ContractVersion int `json:"contract_version"` - DatabaseScope string `json:"database_scope"` - DatabasePath string `json:"database_path"` - BackupPath string `json:"backup_path,omitempty"` - ProjectID string `json:"project_id"` - ProjectName string `json:"project_name"` - ProjectCurrentPath string `json:"project_current_path"` - Origin string `json:"origin"` - Matched int `json:"matched"` - Updated int `json:"updated"` - Applied bool `json:"applied"` - GeneratedAt string `json:"generated_at"` + ContractVersion int `json:"contract_version"` + DatabaseScope string `json:"database_scope"` + DatabasePath string `json:"database_path"` + BackupPath string `json:"backup_path,omitempty"` + ProjectID string `json:"project_id"` + ProjectName string `json:"project_name"` + ProjectCurrentPath string `json:"project_current_path"` + Mode string `json:"mode"` + Origin string `json:"origin"` + Matched int `json:"matched"` + Updated int `json:"updated"` + Reclassified []RelationshipOriginReclassification `json:"reclassified"` + ForeignOrigins []RelationshipOriginForeignGroup `json:"foreign_origins"` + Applied bool `json:"applied"` + GeneratedAt string `json:"generated_at"` } // LegacyProjectDatabaseArchiveResult describes a guarded legacy project database archive. @@ -249,10 +317,22 @@ type LegacyProjectDatabaseArchiveResult struct { Warnings []string `json:"warnings"` } -// RepairMissingRelationshipOrigins backfills missing relationship origin values -// for the current project only. It is dry-run unless options.Apply is true. -func RepairMissingRelationshipOrigins(ctx context.Context, root project.Root, resolver PathResolver, options RelationshipOriginRepairOptions) (RelationshipOriginRepairResult, error) { - if options.Origin != "imported" && options.Origin != "manual" { +// RepairRelationshipOrigins reclassifies the retired legacy origins to +// 'command' for the current project and, when options.Origin selects a +// mechanism, also backfills rows that have no origin at all. Origins outside +// the allowed and legacy vocabularies are reported, never rewritten. It is +// dry-run unless options.Apply is true. A failure once the pre-repair backup +// exists returns a *RelationshipOriginRepairError carrying the partial result +// and backup path. +func RepairRelationshipOrigins(ctx context.Context, root project.Root, resolver PathResolver, options RelationshipOriginRepairOptions) (RelationshipOriginRepairResult, error) { + mode := RelationshipOriginRepairModeReclassifyOnly + switch options.Origin { + case "": + // No mechanism selected: reclassify the legacy origins and leave rows + // that carry no origin for an explicit, operator-chosen backfill. + case relationshipOriginImported, relationshipOriginManual: + mode = RelationshipOriginRepairModeBackfillAndReclassify + default: return RelationshipOriginRepairResult{}, fmt.Errorf("relationship origin must be imported or manual") } @@ -281,6 +361,14 @@ func RepairMissingRelationshipOrigins(ctx context.Context, root project.Root, re if err != nil { return RelationshipOriginRepairResult{}, err } + reclassifications, err := store.planLegacyRelationshipOriginReclassifications(ctx, identity.ID) + if err != nil { + return RelationshipOriginRepairResult{}, err + } + foreignOrigins, err := store.listForeignRelationshipOrigins(ctx, identity.ID) + if err != nil { + return RelationshipOriginRepairResult{}, err + } result := RelationshipOriginRepairResult{ ContractVersion: StateJSONContractVersion, @@ -289,26 +377,54 @@ func RepairMissingRelationshipOrigins(ctx context.Context, root project.Root, re ProjectID: identity.ID, ProjectName: identity.FriendlyName, ProjectCurrentPath: identity.CurrentPath, + Mode: mode, Origin: options.Origin, Matched: matched, + Reclassified: reclassifications, + ForeignOrigins: foreignOrigins, Applied: options.Apply, GeneratedAt: time.Now().UTC().Format(time.RFC3339Nano), } - if !options.Apply || matched == 0 { + reclassifiable := 0 + for _, reclassification := range reclassifications { + reclassifiable += reclassification.Matched + } + // Missing origins are only pending work when the backfill is enabled; + // reclassify-only leaves them alone, so they must not trigger a backup. + pendingBackfill := 0 + if mode == RelationshipOriginRepairModeBackfillAndReclassify { + pendingBackfill = matched + } + if !options.Apply || (pendingBackfill == 0 && reclassifiable == 0) { return result, nil } backup, err := Backup(ctx, root, resolver) if err != nil { - return RelationshipOriginRepairResult{}, fmt.Errorf("backup state database before relationship origin repair: %w", err) + result.BackupPath = backup.BackupPath + return result, &RelationshipOriginRepairError{Result: result, Err: fmt.Errorf("backup state database before relationship origin repair: %w", err)} } result.BackupPath = backup.BackupPath - updated, err := store.backfillMissingRelationshipOrigins(ctx, identity.ID, options.Origin, result.GeneratedAt) - if err != nil { - return RelationshipOriginRepairResult{}, err + // Every failure past this point carries the backup path so a partially + // applied repair still tells the operator where their pre-repair copy is. + if pendingBackfill > 0 { + updated, err := store.backfillMissingRelationshipOrigins(ctx, identity.ID, options.Origin, result.GeneratedAt) + if err != nil { + return result, &RelationshipOriginRepairError{Result: result, Err: err} + } + result.Updated = updated + } + for i := range result.Reclassified { + if result.Reclassified[i].Matched == 0 { + continue + } + updated, err := store.reclassifyLegacyRelationshipOrigin(ctx, identity.ID, result.Reclassified[i].Origin, result.Reclassified[i].Target, result.GeneratedAt) + if err != nil { + return result, &RelationshipOriginRepairError{Result: result, Err: err} + } + result.Reclassified[i].Updated = updated } - result.Updated = updated return result, nil } @@ -401,6 +517,88 @@ WHERE project_id = ? return count, nil } +// planLegacyRelationshipOriginReclassifications counts the rows carrying each +// retired legacy origin, in deterministic origin order, with the registry +// target each group reclassifies to. +func (s *Store) planLegacyRelationshipOriginReclassifications(ctx context.Context, projectID string) ([]RelationshipOriginReclassification, error) { + legacy := legacyRelationshipOriginReclassifications() + reclassifications := []RelationshipOriginReclassification{} + for _, origin := range legacyRelationshipOrigins() { + var count int + if err := s.db.QueryRowContext(ctx, ` +SELECT COUNT(*) +FROM relationships +WHERE project_id = ? + AND origin = ? +`, projectID, origin).Scan(&count); err != nil { + return nil, fmt.Errorf("count legacy relationship origin %q: %w", origin, err) + } + reclassifications = append(reclassifications, RelationshipOriginReclassification{ + Origin: origin, + Target: legacy[origin], + Matched: count, + }) + } + return reclassifications, nil +} + +// listForeignRelationshipOrigins groups origin values outside both the allowed +// vocabulary and the reclassifiable legacy set. These rows carry provenance +// this repair does not understand and must never launder into 'command'. +func (s *Store) listForeignRelationshipOrigins(ctx context.Context, projectID string) ([]RelationshipOriginForeignGroup, error) { + notAllowed, notAllowedArgs := relationshipOriginNotAllowedFragment("origin") + notLegacy, notLegacyArgs := legacyRelationshipOriginNotInFragment("origin") + args := []any{projectID} + args = append(args, notAllowedArgs...) + args = append(args, notLegacyArgs...) + rows, err := s.db.QueryContext(ctx, fmt.Sprintf(` +SELECT origin, COUNT(*) +FROM relationships +WHERE project_id = ? + AND origin IS NOT NULL AND TRIM(origin) != '' + AND %s + AND %s +GROUP BY origin +ORDER BY origin +`, notAllowed, notLegacy), args...) + if err != nil { + return nil, fmt.Errorf("list foreign relationship origins: %w", err) + } + defer rows.Close() + groups := []RelationshipOriginForeignGroup{} + for rows.Next() { + var group RelationshipOriginForeignGroup + if err := rows.Scan(&group.Origin, &group.Count); err != nil { + return nil, fmt.Errorf("scan foreign relationship origin: %w", err) + } + groups = append(groups, group) + } + if err := rows.Err(); err != nil { + return nil, fmt.Errorf("iterate foreign relationship origins: %w", err) + } + return groups, nil +} + +// reclassifyLegacyRelationshipOrigin rewrites exactly one retired legacy origin +// value to its registry target; the reason column is preserved untouched. +func (s *Store) reclassifyLegacyRelationshipOrigin(ctx context.Context, projectID string, origin string, target string, updatedAt string) (int, error) { + result, err := s.db.ExecContext(ctx, ` +UPDATE relationships +SET origin = ?, + updated_at = ? +WHERE project_id = ? + AND origin = ? +`, target, updatedAt, projectID, origin) + if err != nil { + return 0, fmt.Errorf("reclassify legacy relationship origin %q: %w", origin, err) + } + rows, err := result.RowsAffected() + if err != nil { + return 0, fmt.Errorf("count reclassified relationship origins: %w", err) + } + return int(rows), nil +} + func (s *Store) backfillMissingRelationshipOrigins(ctx context.Context, projectID string, origin string, updatedAt string) (int, error) { result, err := s.db.ExecContext(ctx, ` UPDATE relationships diff --git a/internal/state/repair_test.go b/internal/state/repair_test.go index 951327d1..da29a9d6 100644 --- a/internal/state/repair_test.go +++ b/internal/state/repair_test.go @@ -328,7 +328,7 @@ func TestRepairJournalSearchHoldsImmediateLockThroughBackupAndCommit(t *testing. } } -func TestRepairMissingRelationshipOriginsDryRunDoesNotWrite(t *testing.T) { +func TestRepairRelationshipOriginsDryRunDoesNotWrite(t *testing.T) { root := projectRoot(t) stateHome := t.TempDir() if _, err := Initialize(context.Background(), root, PathResolver{StateHome: stateHome}); err != nil { @@ -341,9 +341,9 @@ func TestRepairMissingRelationshipOriginsDryRunDoesNotWrite(t *testing.T) { t.Fatalf("Close() error = %v", err) } - result, err := RepairMissingRelationshipOrigins(context.Background(), root, PathResolver{StateHome: stateHome}, RelationshipOriginRepairOptions{Origin: "imported"}) + result, err := RepairRelationshipOrigins(context.Background(), root, PathResolver{StateHome: stateHome}, RelationshipOriginRepairOptions{Origin: "imported"}) if err != nil { - t.Fatalf("RepairMissingRelationshipOrigins() error = %v", err) + t.Fatalf("RepairRelationshipOrigins() error = %v", err) } if result.ContractVersion != StateJSONContractVersion { t.Fatalf("ContractVersion = %d, want %d", result.ContractVersion, StateJSONContractVersion) @@ -375,7 +375,7 @@ func TestRepairMissingRelationshipOriginsDryRunDoesNotWrite(t *testing.T) { assertMissingRelationshipOrigins(t, store, projectID, 1) } -func TestRepairMissingRelationshipOriginsApplyBackfillsCurrentProject(t *testing.T) { +func TestRepairRelationshipOriginsApplyBackfillsCurrentProject(t *testing.T) { root := projectRoot(t) stateHome := t.TempDir() if _, err := Initialize(context.Background(), root, PathResolver{StateHome: stateHome}); err != nil { @@ -388,9 +388,9 @@ func TestRepairMissingRelationshipOriginsApplyBackfillsCurrentProject(t *testing t.Fatalf("Close() error = %v", err) } - result, err := RepairMissingRelationshipOrigins(context.Background(), root, PathResolver{StateHome: stateHome}, RelationshipOriginRepairOptions{Origin: "imported", Apply: true}) + result, err := RepairRelationshipOrigins(context.Background(), root, PathResolver{StateHome: stateHome}, RelationshipOriginRepairOptions{Origin: "imported", Apply: true}) if err != nil { - t.Fatalf("RepairMissingRelationshipOrigins() error = %v", err) + t.Fatalf("RepairRelationshipOrigins() error = %v", err) } if result.ContractVersion != StateJSONContractVersion { t.Fatalf("ContractVersion = %d, want %d", result.ContractVersion, StateJSONContractVersion) @@ -435,19 +435,347 @@ func TestRepairMissingRelationshipOriginsApplyBackfillsCurrentProject(t *testing } } -func TestRepairMissingRelationshipOriginsRejectsUnknownOrigin(t *testing.T) { +func TestRepairRelationshipOriginsRejectsUnknownOrigin(t *testing.T) { root := projectRoot(t) stateHome := t.TempDir() if _, err := Initialize(context.Background(), root, PathResolver{StateHome: stateHome}); err != nil { t.Fatalf("Initialize() error = %v", err) } - _, err := RepairMissingRelationshipOrigins(context.Background(), root, PathResolver{StateHome: stateHome}, RelationshipOriginRepairOptions{Origin: "guessed", Apply: true}) + _, err := RepairRelationshipOrigins(context.Background(), root, PathResolver{StateHome: stateHome}, RelationshipOriginRepairOptions{Origin: "guessed", Apply: true}) if err == nil { - t.Fatal("RepairMissingRelationshipOrigins() error = nil, want invalid origin error") + t.Fatal("RepairRelationshipOrigins() error = nil, want invalid origin error") } } +func TestRepairRelationshipOriginReclassifiesLegacyOriginsAndPreservesForeign(t *testing.T) { + root := projectRoot(t) + stateHome := t.TempDir() + resolver := PathResolver{StateHome: stateHome} + ctx := context.Background() + if _, err := Initialize(ctx, root, resolver); err != nil { + t.Fatalf("Initialize() error = %v", err) + } + store := openTestStore(t, root, stateHome) + projectID := projectIDForTest(t, store, root) + insertRelationshipWithOrigin(t, store, projectID, "relationship-intent-create", "intent-create") + insertRelationshipWithOrigin(t, store, projectID, "relationship-legacy-conversion", "legacy-conversion") + insertRelationshipWithOrigin(t, store, projectID, "relationship-exploration-create", "exploration-create") + // 'system' is retired writer provenance released alphas wrote onto run and + // finding edges; it must reclassify like any other legacy origin rather + // than linger as a foreign-origin doctor warning. + insertRelationshipWithOrigin(t, store, projectID, "relationship-system", "system") + insertRelationshipWithOrigin(t, store, projectID, "relationship-mystery-import", "mystery-import") + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + wantPlan := []RelationshipOriginReclassification{ + {Origin: "exploration-create", Target: "command", Matched: 1}, + {Origin: "intent-create", Target: "command", Matched: 1}, + {Origin: "legacy-conversion", Target: "command", Matched: 1}, + {Origin: "system", Target: "command", Matched: 1}, + } + wantForeign := []RelationshipOriginForeignGroup{{Origin: "mystery-import", Count: 1}} + + dryRun, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Origin: "imported"}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(dry-run) error = %v", err) + } + if dryRun.Applied || dryRun.BackupPath != "" { + t.Fatalf("dry-run result = applied %t backup %q, want unapplied without backup", dryRun.Applied, dryRun.BackupPath) + } + if !reflect.DeepEqual(dryRun.Reclassified, wantPlan) { + t.Fatalf("dry-run Reclassified = %#v, want %#v", dryRun.Reclassified, wantPlan) + } + if !reflect.DeepEqual(dryRun.ForeignOrigins, wantForeign) { + t.Fatalf("dry-run ForeignOrigins = %#v, want %#v", dryRun.ForeignOrigins, wantForeign) + } + store = openTestStore(t, root, stateHome) + assertRelationshipOrigin(t, store, "relationship-intent-create", "intent-create") + assertRelationshipOrigin(t, store, "relationship-legacy-conversion", "legacy-conversion") + assertRelationshipOrigin(t, store, "relationship-exploration-create", "exploration-create") + assertRelationshipOrigin(t, store, "relationship-system", "system") + assertRelationshipOrigin(t, store, "relationship-mystery-import", "mystery-import") + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + applied, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Origin: "imported", Apply: true}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(apply) error = %v", err) + } + if !applied.Applied { + t.Fatal("Applied = false, want true") + } + if applied.BackupPath == "" { + t.Fatal("BackupPath is empty for applied reclassification") + } + if _, err := os.Stat(applied.BackupPath); err != nil { + t.Fatalf("repair backup does not exist: %v", err) + } + wantApplied := []RelationshipOriginReclassification{ + {Origin: "exploration-create", Target: "command", Matched: 1, Updated: 1}, + {Origin: "intent-create", Target: "command", Matched: 1, Updated: 1}, + {Origin: "legacy-conversion", Target: "command", Matched: 1, Updated: 1}, + {Origin: "system", Target: "command", Matched: 1, Updated: 1}, + } + if !reflect.DeepEqual(applied.Reclassified, wantApplied) { + t.Fatalf("apply Reclassified = %#v, want %#v", applied.Reclassified, wantApplied) + } + if !reflect.DeepEqual(applied.ForeignOrigins, wantForeign) { + t.Fatalf("apply ForeignOrigins = %#v, want %#v", applied.ForeignOrigins, wantForeign) + } + store = openTestStore(t, root, stateHome) + assertRelationshipOrigin(t, store, "relationship-intent-create", "command") + assertRelationshipOrigin(t, store, "relationship-legacy-conversion", "command") + assertRelationshipOrigin(t, store, "relationship-exploration-create", "command") + assertRelationshipOrigin(t, store, "relationship-system", "command") + assertRelationshipOrigin(t, store, "relationship-mystery-import", "mystery-import") + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + rerun, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Origin: "imported", Apply: true}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(rerun) error = %v", err) + } + if rerun.BackupPath != "" { + t.Fatalf("rerun BackupPath = %q, want no backup for a no-op", rerun.BackupPath) + } + for _, reclassification := range rerun.Reclassified { + if reclassification.Matched != 0 || reclassification.Updated != 0 { + t.Fatalf("rerun reclassification = %#v, want zero matched and updated", reclassification) + } + } + if !reflect.DeepEqual(rerun.ForeignOrigins, wantForeign) { + t.Fatalf("rerun ForeignOrigins = %#v, want foreign origin still surfaced", rerun.ForeignOrigins) + } + + store = openTestStore(t, root, stateHome) + defer store.Close() + assertRelationshipOrigin(t, store, "relationship-mystery-import", "mystery-import") + diagnostics, err := inspectRelationshipOriginInvariants(ctx, store) + if err != nil { + t.Fatalf("inspectRelationshipOriginInvariants() error = %v", err) + } + if len(diagnostics) != 1 { + t.Fatalf("diagnostics = %#v, want exactly one foreign-origin warning", diagnostics) + } + if diagnostics[0].Code != "relationship-origin-unknown" || !strings.Contains(diagnostics[0].Message, "mystery-import") { + t.Fatalf("diagnostic = %#v, want relationship-origin-unknown naming mystery-import", diagnostics[0]) + } +} + +// Bare invocation is the Change's Observable Workflow form: it reclassifies the +// retired legacy origins without touching rows that carry no origin at all, +// because choosing a provenance value for those needs operator judgement. +func TestRepairRelationshipOriginsReclassifyOnlyLeavesMissingOriginsUntouched(t *testing.T) { + root := projectRoot(t) + stateHome := t.TempDir() + resolver := PathResolver{StateHome: stateHome} + ctx := context.Background() + if _, err := Initialize(ctx, root, resolver); err != nil { + t.Fatalf("Initialize() error = %v", err) + } + store := openTestStore(t, root, stateHome) + projectID := projectIDForTest(t, store, root) + insertRelationshipWithOrigin(t, store, projectID, "relationship-intent-create", "intent-create") + insertRelationshipWithoutOrigin(t, store, projectID) + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + dryRun, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(bare dry-run) error = %v", err) + } + if dryRun.Mode != RelationshipOriginRepairModeReclassifyOnly { + t.Fatalf("dry-run Mode = %q, want %q", dryRun.Mode, RelationshipOriginRepairModeReclassifyOnly) + } + if dryRun.Origin != "" { + t.Fatalf("dry-run Origin = %q, want empty in reclassify-only mode", dryRun.Origin) + } + if dryRun.Matched != 1 { + t.Fatalf("dry-run Matched = %d, want the missing-origin row reported", dryRun.Matched) + } + if dryRun.Updated != 0 { + t.Fatalf("dry-run Updated = %d, want 0", dryRun.Updated) + } + wantPlan := []RelationshipOriginReclassification{ + {Origin: "exploration-create", Target: "command", Matched: 0}, + {Origin: "intent-create", Target: "command", Matched: 1}, + {Origin: "legacy-conversion", Target: "command", Matched: 0}, + {Origin: "system", Target: "command", Matched: 0}, + } + if !reflect.DeepEqual(dryRun.Reclassified, wantPlan) { + t.Fatalf("dry-run Reclassified = %#v, want %#v", dryRun.Reclassified, wantPlan) + } + + applied, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Apply: true}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(bare apply) error = %v", err) + } + if applied.Mode != RelationshipOriginRepairModeReclassifyOnly { + t.Fatalf("apply Mode = %q, want %q", applied.Mode, RelationshipOriginRepairModeReclassifyOnly) + } + if applied.BackupPath == "" { + t.Fatal("BackupPath is empty for an applied reclassification") + } + if _, err := os.Stat(applied.BackupPath); err != nil { + t.Fatalf("repair backup does not exist: %v", err) + } + if applied.Updated != 0 { + t.Fatalf("apply Updated = %d, want 0 — reclassify-only must never backfill", applied.Updated) + } + + store = openTestStore(t, root, stateHome) + assertRelationshipOrigin(t, store, "relationship-intent-create", "command") + // The missing-origin row is still missing: bare invocation reports it and + // leaves it for an explicit --origin run. + assertMissingRelationshipOrigins(t, store, projectID, 1) + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + rerun, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Apply: true}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(bare rerun) error = %v", err) + } + if rerun.BackupPath != "" { + t.Fatalf("rerun BackupPath = %q, want no backup for a no-op", rerun.BackupPath) + } + for _, reclassification := range rerun.Reclassified { + if reclassification.Matched != 0 || reclassification.Updated != 0 { + t.Fatalf("rerun reclassification = %#v, want zero matched and updated", reclassification) + } + } + if rerun.Matched != 1 { + t.Fatalf("rerun Matched = %d, want the missing-origin row still reported", rerun.Matched) + } +} + +// V3's doctor-clean requirement: a database whose only unknown origins are the +// retired legacy values is fully healed by one apply, with no residual warning. +// The mixed fixture above proves foreign origins survive; this one proves +// nothing else does. +func TestRepairRelationshipOriginsLegacyOnlyFixtureIsDoctorCleanAfterApply(t *testing.T) { + root := projectRoot(t) + stateHome := t.TempDir() + resolver := PathResolver{StateHome: stateHome} + ctx := context.Background() + if _, err := Initialize(ctx, root, resolver); err != nil { + t.Fatalf("Initialize() error = %v", err) + } + store := openTestStore(t, root, stateHome) + projectID := projectIDForTest(t, store, root) + for _, origin := range legacyRelationshipOrigins() { + insertRelationshipWithOrigin(t, store, projectID, "relationship-"+origin, origin) + } + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + applied, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Apply: true}) + if err != nil { + t.Fatalf("RepairRelationshipOrigins(apply) error = %v", err) + } + if applied.BackupPath == "" { + t.Fatal("BackupPath is empty for an applied reclassification") + } + if len(applied.ForeignOrigins) != 0 { + t.Fatalf("ForeignOrigins = %#v, want none in a legacy-only fixture", applied.ForeignOrigins) + } + for _, reclassification := range applied.Reclassified { + if reclassification.Matched != 1 || reclassification.Updated != 1 { + t.Fatalf("reclassification = %#v, want exactly one row matched and updated", reclassification) + } + } + + store = openTestStore(t, root, stateHome) + defer store.Close() + for _, origin := range legacyRelationshipOrigins() { + assertRelationshipOrigin(t, store, "relationship-"+origin, "command") + } + diagnostics, err := inspectRelationshipOriginInvariants(ctx, store) + if err != nil { + t.Fatalf("inspectRelationshipOriginInvariants() error = %v", err) + } + if len(diagnostics) != 0 { + t.Fatalf("diagnostics = %#v, want doctor clean after repairing a legacy-only database", diagnostics) + } +} + +func TestRepairRelationshipOriginsFailedWriteAfterBackupPreservesBackupPath(t *testing.T) { + root := projectRoot(t) + stateHome := t.TempDir() + resolver := PathResolver{StateHome: stateHome} + ctx := context.Background() + if _, err := Initialize(ctx, root, resolver); err != nil { + t.Fatalf("Initialize() error = %v", err) + } + store := openTestStore(t, root, stateHome) + projectID := projectIDForTest(t, store, root) + insertRelationshipWithoutOrigin(t, store, projectID) + // Abort every relationship update at the SQLite layer so the repair fails + // after its pre-repair backup exists. That window is the only one in which + // the backup path is the operator's sole recovery reference, so dropping it + // from the surfaced result would strand them. + if _, err := store.db.ExecContext(ctx, ` +CREATE TRIGGER forced_relationship_update_failure +BEFORE UPDATE ON relationships +BEGIN + SELECT RAISE(ABORT, 'forced post-backup failure'); +END +`); err != nil { + store.Close() + t.Fatalf("create forced failure trigger error = %v", err) + } + if err := store.Close(); err != nil { + t.Fatalf("Close() error = %v", err) + } + + result, err := RepairRelationshipOrigins(ctx, root, resolver, RelationshipOriginRepairOptions{Origin: "imported", Apply: true}) + if err == nil { + t.Fatal("RepairRelationshipOrigins() error = nil, want forced post-backup failure") + } + var repairErr *RelationshipOriginRepairError + if !errors.As(err, &repairErr) { + t.Fatalf("error = %T %v, want *RelationshipOriginRepairError", err, err) + } + if errors.Unwrap(repairErr) == nil { + t.Fatal("Unwrap() = nil, want the underlying repair failure") + } + if !strings.Contains(err.Error(), "forced post-backup failure") { + t.Fatalf("error = %v, want the underlying SQLite abort preserved", err) + } + if repairErr.Result.BackupPath == "" { + t.Fatal("preserved result BackupPath is empty after a post-backup failure") + } + if !strings.Contains(err.Error(), repairErr.Result.BackupPath) { + t.Fatalf("error = %v, want the preserved backup path named", err) + } + if result.BackupPath != repairErr.Result.BackupPath { + t.Fatalf("returned result BackupPath = %q, want %q from the preserved error", result.BackupPath, repairErr.Result.BackupPath) + } + if _, err := os.Stat(repairErr.Result.BackupPath); err != nil { + t.Fatalf("preserved backup does not exist: %v", err) + } + verification, err := VerifyBackup(ctx, repairErr.Result.BackupPath) + if err != nil { + t.Fatalf("VerifyBackup(preserved) error = %v", err) + } + if !verification.Verified { + t.Fatalf("preserved backup verification = %#v, want verified", verification) + } + + // The live row is untouched, so a rerun after the operator clears the fault + // still has the same work to do. + store = openTestStore(t, root, stateHome) + defer store.Close() + assertMissingRelationshipOrigins(t, store, projectID, 1) +} + func TestArchiveLegacyProjectDatabaseDryRunDoesNotMoveFiles(t *testing.T) { root := projectRoot(t) dataHome := t.TempDir() @@ -614,6 +942,27 @@ VALUES ('relationship-without-origin', ?, 'task', 'task-one', 'spec', 'spec-one' } } +func insertRelationshipWithOrigin(t *testing.T, store *Store, projectID string, id string, origin string) { + t.Helper() + if _, err := store.db.ExecContext(context.Background(), ` +INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) +VALUES (?, ?, 'spark', 'spark-one', 'intent', ?, 'source-of', 'seeded provenance row', ?, '2026-07-01T00:00:00Z', '2026-07-01T00:00:00Z') +`, id, projectID, "intent-for-"+id, origin); err != nil { + t.Fatalf("insert relationship with origin %q error = %v", origin, err) + } +} + +func assertRelationshipOrigin(t *testing.T, store *Store, id string, want string) { + t.Helper() + var got string + if err := store.db.QueryRowContext(context.Background(), `SELECT origin FROM relationships WHERE id = ?`, id).Scan(&got); err != nil { + t.Fatalf("read relationship %s origin error = %v", id, err) + } + if got != want { + t.Fatalf("relationship %s origin = %q, want %q", id, got, want) + } +} + func assertMissingRelationshipOrigins(t *testing.T, store *Store, projectID string, want int) { t.Helper() var got int diff --git a/internal/state/run.go b/internal/state/run.go index ada3512c..510868dd 100644 --- a/internal/state/run.go +++ b/internal/state/run.go @@ -187,7 +187,7 @@ VALUES (?, ?, ?, ?, ?, ?, ?, ?, NULL, ?, ?) if report != nil { if _, err := tx.ExecContext(ctx, ` INSERT INTO relationships (id, project_id, from_entity_kind, from_entity_id, to_entity_kind, to_entity_id, relationship_type, reason, origin, created_at, updated_at) -VALUES (?, ?, 'run', ?, 'report', ?, 'produces', 'recorded by run create', 'system', ?, ?) +VALUES (?, ?, 'run', ?, 'report', ?, 'produces', 'recorded by run create', 'command', ?, ?) ON CONFLICT(id) DO NOTHING `, stableMigrationID("relationship", projectID, "run", runID, "produces", "report", report.ID), projectID, runID, report.ID, timestamp, timestamp); err != nil { return RunCreateResult{}, fmt.Errorf("record run report relationship: %w", err) diff --git a/internal/state/status.go b/internal/state/status.go index 07a1b980..b974993a 100644 --- a/internal/state/status.go +++ b/internal/state/status.go @@ -851,13 +851,16 @@ WHERE origin IS NULL OR TRIM(origin) = '' }) } - unknownRows, err := store.db.QueryContext(ctx, ` + // The allowed set comes from the relationship-origin registry so the + // invariant cannot drift from what the writers and repair actually use. + notAllowed, notAllowedArgs := relationshipOriginNotAllowedFragment("origin") + unknownRows, err := store.db.QueryContext(ctx, fmt.Sprintf(` SELECT origin, COUNT(*) FROM relationships -WHERE origin IS NOT NULL AND TRIM(origin) != '' AND origin NOT IN ('imported', 'manual', 'command') +WHERE origin IS NOT NULL AND TRIM(origin) != '' AND %s GROUP BY origin ORDER BY origin -`) +`, notAllowed), notAllowedArgs...) if err != nil { return nil, fmt.Errorf("inspect unknown relationship origins: %w", err) } diff --git a/plugins/loaf/bin/native/darwin-arm64/loaf b/plugins/loaf/bin/native/darwin-arm64/loaf index 14e7eb7e..05a6a6f2 100755 Binary files a/plugins/loaf/bin/native/darwin-arm64/loaf and b/plugins/loaf/bin/native/darwin-arm64/loaf differ diff --git a/plugins/loaf/skills/loaf-reference/SKILL.md b/plugins/loaf/skills/loaf-reference/SKILL.md index e2bf5f75..a0746b4b 100644 --- a/plugins/loaf/skills/loaf-reference/SKILL.md +++ b/plugins/loaf/skills/loaf-reference/SKILL.md @@ -66,10 +66,10 @@ Names and one-line purposes only. Run `loaf --help` for options, argum | `loaf docs` | Manage docs/ indexing | index | | `loaf change` | Shape-first Change artifacts: git-canonical work context under docs/changes/ | init, check, list | | `loaf render` | Maintain committed durable Markdown renders | sweep | -| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | +| `loaf state` | Manage native SQLite state | path, status, init, doctor, repair legacy-project-database, repair relationship-origin, repair journal-search, migrate markdown, migrate storage-home, migrate schema, migrate deferrals, migrate lifecycle-statuses, migrate journal-first, backup, backup verify, backup restore, restore-ephemerals, verify-ephemerals, export, export all, export triage, export spec, export release-readiness | | `loaf journal` | Record and read the project-scoped journal (the durable record across all conversations) | log, recent, search, show, context, export, defer | | `loaf project` | Manage durable project identity | list, show, identity, rename, move, delete | -| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, worktree-storage | +| `loaf migrate` | Run native migration workflows | markdown, storage-home, schema, lifecycle-statuses, journal-first, worktree-storage | | `loaf task` | Manage project tasks | list, show, status, create, update, archive, refresh, sync | | `loaf spec` | Manage project specs | new, list, show, status, render, finalize, archive, delete | | `loaf report` | Manage durable reports (research, audits, investigations) | list, show, render, generate, create, finalize, archive |