Skip to content

Integrate official TOON spec tests (v1.4) and document deviations#1

Open
michal-ciechan wants to merge 9 commits into0xZunia:mainfrom
michal-ciechan:spec-tests-v1.3
Open

Integrate official TOON spec tests (v1.4) and document deviations#1
michal-ciechan wants to merge 9 commits into0xZunia:mainfrom
michal-ciechan:spec-tests-v1.3

Conversation

@michal-ciechan
Copy link

@michal-ciechan michal-ciechan commented Dec 14, 2025

Summary

This PR integrates the official TOON specification tests and documents deviations from the spec. This is mainly a precursor to upgrading to newer spec versions.

Changes

  • Updated SPEC.md from v1.2 to v1.4
  • Added official spec test fixtures (17 JSON files) from toon-format/spec
  • Created SpecTestRunner.cs that automatically loads and runs tests from JSON fixtures
  • Known deviations are marked as skipped with Skip.If() referencing the README
  • Documented 19 known spec deviations in README.md
  • Added link to official spec at https://github.com/toon-format/spec/blob/v1.4.0/SPEC.md

Open Questions

I'm still deciding whether to:

  • Fix some deviations - Some are likely bugs that should be fixed (e.g., hyphen quoting, quoted keys with brackets)
  • Accept some differences - Others might be acceptable implementation choices or .NET-specific behaviors (e.g., floating-point precision)

I haven't looked into each deviation in detail yet, but I definitely think we should be using the raw test files from the spec repo to ensure compliance.

Test Results

  • 269 tests passing
  • 19 spec test deviations (skipped)

Test plan

  • All spec tests pass or are explicitly skipped as known deviations
  • New spec tests load and run from JSON fixtures
  • Deviations documented in README with examples

🤖 Generated with Claude Code

mike.ciechan and others added 7 commits December 14, 2025 10:21
- Import 17 JSON test fixtures from toon-format-spec repository
- Add SpecTestRunner.cs for automated spec compliance testing
- Update SPEC.md from v1.2 to v1.3
- 257 of 273 tests passing (94% compliance)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add comprehensive Spec Deviations section documenting 16 known test failures
- Update spec version from v1.2 to v1.3 in Features
- Add link to official TOON spec at toon-format/spec

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove JsonPropertyName attributes and rely on JsonNamingPolicy.CamelCase

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace SPEC.md with v1.4.0 from toon-format/spec
- Update README references from v1.3 to v1.4
- Update version filter in SpecTestRunner to v1.4

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@michal-ciechan michal-ciechan changed the title Integrate official TOON spec tests (v1.3) and document deviations Integrate official TOON spec tests (v1.4) and document deviations Dec 14, 2025
mike.ciechan and others added 2 commits December 14, 2025 21:01
- Add Xunit.SkippableFact package for dynamic test skipping
- Mark 19 known deviations as skipped with reference to README
- Update README with correct deviation count (19)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Convert NormalizeLineEndings from private static helper to a string
extension method for cleaner test syntax.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant