feat(pipeline): add IDML/PDF ingest stage to the build pipeline#152
Merged
PAMulligan merged 2 commits intoJun 24, 2026
Conversation
Add an `ingest` ETL stage that extracts an InDesign source (.idml or .pdf)
into a generator-ready bundle on disk: ir.json (the intermediate IR the theme
generator consumes), assets/ (image bytes pulled from the source, named to
match the generator's planAssets output), and assets.manifest.json (per-frame
provenance plus unresolved-asset warnings).
This closes a real gap: embedded IDML image bytes previously had no automatic
path into a generated theme. The stage resolves them once and stages them
under the exact filenames generateTheme references, so the bundle feeds
straight into generation:
flavian pipeline ingest brochure.idml --out-dir build/brochure
flavian pipeline indesign build/brochure/ir.json --asset-dir build/brochure/assets
Details:
- Pure, source-agnostic transform (resolveAssets/buildIngestBundle) joins IR
image frames to an href->bytes resolver; runIngest wires in IDML zip
extraction and the PDF image cache.
- Runnable via `pnpm pipeline:ingest`, `flavian pipeline ingest`, and the
`flavian-ingest` bin.
- Tested on an in-memory IDML fixture: the transform, the manifest shape, and
an end-to-end run whose staged paths line up with generateTheme.
Redence proof token (Pipelines / Data-ETL): RDNC-NWYQDP
Closes #117
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…idmlpdf-ingest-stage-to-the-build-pipeline
This was referenced Jun 24, 2026
PAMulligan
added a commit
that referenced
this pull request
Jun 24, 2026
feat(pipeline): add asset-staging stage and repair #152 merge fallout
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
ingestETL stage to the InDesign pipeline that extracts a source document (.idml/.pdf) into a generator-ready bundle on disk:ir.json— the intermediate IR the theme generator consumesassets/— image bytes pulled from the source, named to match the generator'splanAssets()outputassets.manifest.json— per-frame provenance + unresolved-asset warningsWhy
Embedded IDML image bytes previously had no automatic path into a generated theme —
generate-themecould only stage images from a loose--asset-dir, and nothing extracted the bytes the IDML zip carries. This stage resolves them once and stages them under the exact filenamesgenerateTheme()references, so the bundle feeds straight into generation:What's in the stage
extractIdmlAssets, PDF via the parse-pdf image cache).resolveAssets/buildIngestBundlejoin IR image frames to anhref → bytesresolver (pure, source-agnostic), computing staged paths via the generator's ownplanAssetsso they line up byte-for-byte downstream.writeIngestBundlewritesir.json+assets/+assets.manifest.json.Runnable via
pnpm pipeline:ingest,flavian pipeline ingest, and theflavian-ingestbin. Adopts the in-progressidml-assets.jshelper (and its unit test).Test plan
ingest.test.mjs— transform, manifest shape, and an end-to-end IDML run whose staged paths line up withgenerateTheme()(3/3)assets-idml.test.mjs— embedded-asset extraction (2/2)scripts/indesign-fse/smoke-test.mjs✓tests/fixtures/indesign/fixtures.test.mjs(3/3)theme/assets/automaticallyThe full
@flavian/pipelinesuite is green except two pre-existing snapshot tests that only fail on a Windows (CRLF) checkout —.gitattributesstores them as LF (git ls-files --eol→i/lf), so they pass on the Ubuntu CI runner.Closes #117
🤖 Generated with Claude Code