refactor(extensions): move example YAML files out of the extensions folder - #1136
Open
nielspardon wants to merge 2 commits into
Open
refactor(extensions): move example YAML files out of the extensions folder#1136nielspardon wants to merge 2 commits into
nielspardon wants to merge 2 commits into
Conversation
nielspardon
requested review from
EpsilonPrime,
benbellick,
cpcloud,
jacques-n,
vbarua,
westonpace and
yongchul
as code owners
July 21, 2026 08:55
…older The extensions/ folder mixed official function extensions (functions_*.yaml) with example/placeholder files. Consumers filter these inconsistently, which led to the DuckDB extension ingesting placeholder definitions and producing bogus function registrations (substrait-io/duckdb-substrait-extension#239, issue substrait-io#214). Move the two remaining example files, type_variations.yaml and extension_types.yaml, to site/examples/extensions/ (the existing, schema-validated examples folder) and rename their URNs to the extension:example:* convention used there, so the core extensions/ folder contains only official function extensions. References to extension:io.substrait:extension_types (docs, the distance_functions example dependency and parser test fixtures) are updated to extension:example:extension_types. Note: extension_types.yaml is currently loaded by substrait-java and substrait-go for its point/line user-defined types; whether it should instead become a first-class official extension is left open for maintainer feedback. The third placeholder, extensions/unknown.yaml, was removed separately in substrait-io#1081, which replaced it with the built-in unbound type.
nielspardon
force-pushed
the
move-example-extensions-to-examples
branch
from
July 21, 2026 10:32
c2ed733 to
d2aeb13
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The core
extensions/folder mixes official function extensions (functions_*.yaml) with example/placeholder files:type_variations.yaml(extension:io.substrait:type_variations) — example type variations, no functionsextension_types.yaml(extension:io.substrait:extension_types) — thepoint/lineuser-defined types used as the docs' UDT exampleBecause they live alongside the real extensions, consumers ingest them inconsistently. This surfaced in duckdb-substrait-extension#239 (issue #214), where the now-removed
unknown.yamlplaceholder was expanded into bogus catch-all function registrations that clobbered valid ones.Filtering behavior across the ecosystem for the two remaining files is inconsistent:
functions_*.yamlfunctions*.yamlChange
Move the two example files to
site/examples/extensions/(the existing, schema-validated, docs-embeddable examples folder) and rename their URNs to theextension:example:*convention already used there. The coreextensions/folder now contains only officialfunctions_*.yamlextensions.References to
extension:io.substrait:extension_typesare updated toextension:example:extension_typesin the docs (extensions/index.md,types/type_parsing.md), thedistance_functions.yamlexample dependency, and the parser test fixtures intests/coverage/test_coverage.py. Thetype_variationsURN is not referenced anywhere else.Also adds a
.DS_Storeentry to.gitignore.No tooling/config changes were needed: the coverage tests and function-doc generator already filter to
functions_*,check-examplesnow validates the moved files against the same schemacheck-extensionsused,yamllintruns repo-wide, and the mkdocs--8<--include root is unchanged.Verification
pixi run yamllint— cleanpixi run check-jsonschema— examples + extensions + dialects allokpixi run pytest tests/test_extensions.py tests/coverage/test_coverage.py— 128 passedpixi run mkdocs build --strict— builds, includes resolvegit grep extension:io.substrait:{type_variations,extension_types}— no matches remainOpen questions for maintainers
extension_typesan example or an official extension? Unliketype_variations, it's genuinely load-bearing today (substrait-java and substrait-go load itspoint/linetypes). This PR treats it as an example; if we'd rather ship it officially, it should stay in / return toextensions/under an official URN and onlytype_variationsmoves.site/examples/extensions/, not shipped in the release artifact). An alternative is shipping examples underextensions/examples/so they travel with the extensions tree for downstream testing — the file moves and URN renames are identical; only the target directory and thecheck-extensionsglob differ.Downstream follow-ups (separate PRs)
Consumers that vendor the extensions folder will need updates when they next bump, since both files moved and the
extension_typesURN changed:extension_types.yaml+ defines theEXTENSION_TYPESURN constant; would lose thepoint/lineUDTs unless it vendors the example from the new location.extension_typestypes when it re-embeds.extension_typesbecomes dead (its cited bug docs: clarify u! prefix requirement for user-defined types in YAML #935 is already fixed).generate_extensions.shcopies top-levelextensions/*.yaml, so the moved files drop out naturally — no script change.functions*allowlist — no impact.🤖 Generated with AI
This change is