Skip to content

Fix/issue214 Validate post-loader data contracts by training mode - #427

Open
pzx-dny wants to merge 2 commits into
inclusionAI:mainfrom
pzx-dny:fix/issue214
Open

Fix/issue214 Validate post-loader data contracts by training mode#427
pzx-dny wants to merge 2 commits into
inclusionAI:mainfrom
pzx-dny:fix/issue214

Conversation

@pzx-dny

@pzx-dny pzx-dny commented Jul 31, 2026

Copy link
Copy Markdown

What does this PR do?

Add post-loader data contract validation that checks dataset schema before expensive model/worker initialization. Supports SFT, DPO, online RL (GSPO/GRPO/PPO), and agentic modes with bounded error collection and human-readable/JSON output formatting.

Key changes:

  • New areno/api/data_validation.py — core validation module (stdlib only, no new dependencies)
  • Integrate validation into areno/cli/train.py:run() after dataset loading, before build_trainer()
  • New tests/test_data_validation_cpu.py — 26 CPU unit tests

Behavior: Valid data passes silently. Invalid data prints aggregated errors (sample index, field path, expected/actual type, fix hint) to stderr and raises click.UsageError to block training. No new CLI flags or subcommands.

Related issue

issue#214

How was it tested?

$ pytest tests/test_data_validation_cpu.py -v
============================== 26 passed in 0.04s ==============================

Test coverage:

  • SFT mode (7): valid data, missing field, empty value, wrong type, mixed errors, valid_rows count
  • DPO mode (5): prompt/response style, chat message list style, mismatched types, missing field, None value
  • Online RL mode (4): GSPO/GRPO/PPO valid, non-dict row detection
  • Agentic mode (1): auto-detection via agent_fn
  • Boundary (3): max_errors limit, empty dataset, unknown algo
  • Output format (6): human-readable and JSON output assertions

Hardware limitation: macOS without GPU — tests load the module directly to bypass the areno.api import chain (which requires torch). On Linux GPU environments, from areno.api.data_validation import ... works directly.

Checklist

  • The PR title summarizes the contribution.
  • Linked the related issue in the description (if any).
  • Existing tests pass (pytest tests/ -k cpu).
  • New behavior is covered by tests.
  • Described the test commands run and any hardware limitations.
  • Public API / CLI changes are additive and backward-compatible (see CONTRIBUTING.md).

Breaking change details

No breaking changes. Validation runs by default but is completely silent for valid data — no impact on existing behavior.

潘挚炫 added 2 commits July 29, 2026 15:03
Add post-loader data contract validation that checks dataset schema
before expensive model/worker initialization. Supports SFT, DPO,
online RL (GSPO/GRPO/PPO), and agentic modes with bounded error
collection and human-readable/JSON output formatting.
Fix spelling in filenames:
- areno/api/data_vaidation.py -> data_validation.py
- tests/test_data_vaildation_cpu.py -> test_data_validation_cpu.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant