Skip to content

fix: accept exact loader bar schemas for feature inputs - #51

Merged
pjy008008 merged 1 commit into
developfrom
fix/feature-loader-schema-compatibility
Aug 8, 2026
Merged

fix: accept exact loader bar schemas for feature inputs#51
pjy008008 merged 1 commit into
developfrom
fix/feature-loader-schema-compatibility

Conversation

@pjy008008

Copy link
Copy Markdown
Contributor

Summary

  • accept the operating loader's explicit market-bars/1 base and derived Parquet schemas as feature inputs
  • bind legacy acceptance to exact producer metadata (Alpaca SIP/raw, resolution, manifest ID)
  • keep unknown versions, schema drift, and metadata drift fail-closed

Operational evidence

The development catalog has 96 AVAILABLE loader manifests and 768 receipts using market-bars/1. Live samples confirmed 30m uses the base schema while 1h/4h/1d use the derived schema with source_bar_count and source_minutes.

Validation

  • red test first: both exact legacy schemas were rejected before the change
  • 109 passed, 2 deselected across feature backfill/output trust-boundary tests
  • ruff check passed
  • mypy apps/pipeline_worker/feature_output.py passed

The deselected tests are Docker-backed integration tests; the local root backend pointer currently lags the vendored central-migration fixture (§2.7), which is unrelated to this patch.

@pjy008008
pjy008008 merged commit 89bc34e into develop Aug 8, 2026
10 checks passed
@pjy008008
pjy008008 deleted the fix/feature-loader-schema-compatibility branch August 8, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant