Skip to content

feat: add post-loader data contract validation for SFT/DPO/Online RL/Agentic - #429

Open
Hpward-Ling wants to merge 4 commits into
inclusionAI:mainfrom
Hpward-Ling:data
Open

feat: add post-loader data contract validation for SFT/DPO/Online RL/Agentic#429
Hpward-Ling wants to merge 4 commits into
inclusionAI:mainfrom
Hpward-Ling:data

Conversation

@Hpward-Ling

Copy link
Copy Markdown

What does this PR do?

Adds a post-loader data contract validator that checks required fields, types,
list lengths, and nested messages for SFT, DPO, Online RL, and agentic inputs
after the user dataset loader runs, before expensive model/worker
initialization. Errors are aggregated into a bounded report (instead of stopping
at the first), with sample index, field path, expected/actual type, and a
concrete correction hint per error.

Three entry points:

  1. Python APIfrom areno.api import validate_contract → returns a
    ContractReport with .ok, .errors, .warnings, .total_scanned.
  2. CLI standaloneareno data inspect --dataset-path ... --contract --mode {sft,dpo,online_rl,agentic}
    with --json for CI-friendly structured output.
  3. CLI integratedareno train ... --validate-data-contract runs the
    same validator after dataset loading, before model init. Default is
    --no-validate-data-contract (backward compatible).

Related issue

Fixes #214

Type of change

  • ✨ New feature

How was it tested?

All tests are CPU-only, no GPU or external services required.

# Core validator unit tests (all modes, boundary, malformed, disabled/default)
pytest tests/test_data_contract_cpu.py -v

# Data inspect CLI tests
pytest tests/test_data_inspect_cli_cpu.py -v

# Train CLI integration tests
pytest tests/test_train_cli_contract_integration_cpu.py -v

# Run all three together
pytest tests/test_data_contract_cpu.py tests/test_data_inspect_cli_cpu.py tests/test_train_cli_contract_integration_cpu.py -v

chengyan.ts added 4 commits July 30, 2026 18:16
…AI#214

Add explicit Limitations section covering bounded scan, post-loader-only
validation, structural (not semantic) checks, read-only behavior, and
missing-vs-null distinction. Completes the documentation requirements of
issue inclusionAI#214.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate post-loader data contracts by training mode

1 participant