feat(distillery-sync): ingest docs via distillery_ingest_doc, not distillery_store

[norrietaylor]

Closes #322.

What

Switch the distillery-sync document pass from distillery_store (single ad-hoc entries, fuzzy cosine dedup) to distillery_ingest_doc (norrietaylor/distillery#627), which is purpose-built for a re-running doc sync:

• Idempotent — external_id defaults to sha256(text): an unchanged file is a dedup_action: "skipped" no-op; changed content re-ingests under the same source.
• doctype facet — writes doctype/<v> tag + metadata.doctype (spec/adr/decision/doc), inferred from the doc path.
• Chunking — long docs split into chunk-linked entries.
• Provenance — metadata.source (repo-relative path) + metadata.title.

Changes (prompt-only, recompiled lock)

• Allowlist: +distillery_ingest_doc, -distillery_store/-distillery_update/-distillery_find_similar.
• Rewrote step 4 (Store/update/skip → single distillery_ingest_doc call with project/text/source/title/doctype; external_id defaults to the content hash).
• Replaced the srcpath/<slug> identity section with the content-hash idempotency model; updated steps 3/5/6 + the verification/idempotency prose.

Acceptance (#322)

• Re-running on an unchanged doc persists nothing (dedup_action: "skipped").
• Entries carry doctype/<v> + metadata.doctype + metadata.source + metadata.title.
• Long spec ingests as multiple chunk-linked entries.
• Needs the target Distillery to expose distillery_ingest_doc (present post-#627; gominimal's instance has it).

After merge: retag v0.3.0 to deploy.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

No new commits to review since the last review.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ebcc7d57-dee4-4635-a0a6-c0856ad48986

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The distillery-sync agent prompt and MCP tool configuration are updated in both the source .md and compiled .lock.yml to replace the distillery_store/distillery_update/srcpath-slug document-identity model with a single distillery_ingest_doc call per file that uses sha256(text) as external_id and metadata.source for provenance.

Changes

Switch to distillery_ingest_doc ingestion

Layer / File(s)	Summary
Document identity and idempotency model .github/workflows/distillery-sync.md, .github/workflows/distillery-sync.lock.yml	The srcpath/<slug> tag canonicalization algorithm and fuzzy cosine similarity backstop are removed. Idempotency is now defined by distillery_ingest_doc's external_id = sha256(text); provenance shifts to metadata.source (repo-relative path); backfill deduplication switches from srcpath-tag key to file-path-based deduplication.
Core ingestion procedure and per-file processing .github/workflows/distillery-sync.md, .github/workflows/distillery-sync.lock.yml	The per-file processing loop is replaced: explicit distillery_list/distillery_get/distillery_store/distillery_update create-vs-update logic is removed and replaced with one distillery_ingest_doc call per file using inferred doctype from path, deterministic title resolution, and dedup_action: "skipped" for unchanged content. Workflow callout text is updated to reflect the single per-file ingest behavior.
Run-summary and verification updates .github/workflows/distillery-sync.md, .github/workflows/distillery-sync.lock.yml	Run-summary and verification text are updated to report "ingested vs skipped" counts based on dedup_action, replacing prior create/update/skip wording. Verification for overlapping/re-run glob sets attributes the no-duplicate guarantee to distillery_ingest_doc's content-hash external_id returning dedup_action: "skipped" for byte-identical re-ingests.
MCP tool allowlist updates .github/workflows/distillery-sync.md, .github/workflows/distillery-sync.lock.yml	distillery_find_similar, distillery_store, and distillery_update are removed from the MCP server config and the Claude harness allowed-tools block in all three tool-list locations. distillery_ingest_doc is added.
Regenerated heredoc IDs and metadata hashes .github/workflows/distillery-sync.lock.yml	Workflow-level metadata hash, prompt heredoc boundary identifiers, safe-outputs config blob heredoc marker, and MCP gateway config heredoc labels are regenerated to reflect updated prompt and tool config contents.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• norrietaylor/spectacles#12: First introduced the Distillery MCP integration and distillery-sync workflow that this PR directly modifies.
• norrietaylor/spectacles#191: Previously modified the same distillery-sync agent prompt and document identity tagging that this PR replaces with content-hash ingestion.

Poem

🐇 Hoppy hash days are here at last,
No more slug tags from the past!
One ingest_doc call per file,
Content hashes do it in style.
dedup_action: "skipped" — hip hip hooray,
Identical bytes just stay away! 🎉

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: switching from distillery_store to distillery_ingest_doc for document ingestion.
Description check	✅ Passed	The description is directly related to the changeset, explaining the switch from distillery_store to distillery_ingest_doc and documenting the key benefits and changes.
Linked Issues check	✅ Passed	The PR fully addresses issue #322 requirements: idempotent re-sync via content-hash external_id, doctype faceting with tags and metadata, chunking support, and provenance metadata (source and title).
Out of Scope Changes check	✅ Passed	All changes are scoped to the distillery-sync workflow and documentation, directly addressing the tool switch from distillery_store to distillery_ingest_doc with no unrelated modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/distillery-ingest-doc

---

_{Comment @coderabbitai help to get the list of available commands.}

[norrietaylor]

@codereabbit review

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/distillery-sync.md:
- Around line 239-248: The documentation has conflicting descriptions of the
identity invariant for document ingestion. It states idempotency is based on
content hash (sha256), yet also describes source as an idempotency anchor and
references path-keyed identity elsewhere. Clarify the primary identity invariant
by explicitly stating that idempotency is determined by content hash
(sha256(text)), and that metadata.source tracks provenance but is not the
idempotency key. Ensure this invariant is consistently described throughout the
document, including the sections noted at lines 325-327, and remove or clarify
any statements that could be interpreted as path-based identity or source-based
deduplication being the primary mechanism.
- Around line 311-316: The doctype classification taxonomy is missing support
for the `feedback` doctype, which causes feedback documents to incorrectly
default to `doc` and breaks downstream facet filters. Update the doctype
enumeration from `adr|spec|decision|doc` to include `feedback`, and add a path
pattern rule (similar to the existing rules for `specs/**`, `decisions/*.md`,
and `adr/**`) that identifies which document paths should be classified as
`feedback` type. This ensures feedback documents are properly categorized rather
than collapsing into the generic `doc` classification.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2ed47eb2-9e28-4de1-b8c6-5c477e996b15

📥 Commits

Reviewing files that changed from the base of the PR and between eb8c272 and 33aad8c.

📒 Files selected for processing (2)

• .github/workflows/distillery-sync.lock.yml
• .github/workflows/distillery-sync.md

[norrietaylor]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.