Skip to content

test(grpo): document current surrogate ratio semantics - #438

Open
yaoyao12345666 wants to merge 1 commit into
inclusionAI:mainfrom
yaoyao12345666:test/grpo-surrogate-ratio-semantics
Open

test(grpo): document current surrogate ratio semantics#438
yaoyao12345666 wants to merge 1 commit into
inclusionAI:mainfrom
yaoyao12345666:test/grpo-surrogate-ratio-semantics

Conversation

@yaoyao12345666

@yaoyao12345666 yaoyao12345666 commented Aug 1, 2026

Copy link
Copy Markdown

What does this PR do?

Adds focused CPU tests that document the current GRPO surrogate semantics:

  • ratio statistics remain unit-valued when rollout logprobs differ;
  • rollout-vs-train drift remains visible through dedicated metrics; and
  • the unit-valued surrogate carries the expected gradients while prompt positions remain masked.

This changes tests only, not the GRPO implementation.

Related issue

Fixes #66

Type of change

  • Test coverage improvement

How was it tested?

  • python -m pytest tests/test_losses_rewards_cpu.py -q (13 passed)
  • ruff check tests/test_losses_rewards_cpu.py --no-cache
  • ruff format --check tests/test_losses_rewards_cpu.py --no-cache

The full CPU suite was attempted on Windows, but collection imports Linux-only pty/termios code. AReno officially targets Linux/WSL2; CI is the authoritative full-suite result.

Checklist

  • The PR title summarizes the contribution.
  • Linked the related issue in the description.
  • Existing tests pass (pytest tests/ -k cpu) - blocked locally by the Windows/Linux collection boundary described above.
  • New coverage tests the documented semantics.
  • Described the test commands run and hardware limitations.
  • No public API or CLI behavior changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document current GRPO surrogate ratio semantics

1 participant