Add audit-only dlogp diagnostics

[inaniloquentee]

Summary

• add a native audit-only dlogp diagnostics helper for training-vs-rollout logp drift over active response/action tokens
• report abs_dlogp mean/max/p50/p90/p99, ratio0, clipfrac0, approx_kl0, active-token coverage, warning counts, and numeric worst-token location fields
• wire audit mode into Megatron policy loss behind --rlk-consistency-mode/--rl-kernel-consistency-mode without changing loss behavior or enabling fast operators
• pass optional training metadata through rollout partitions and keep it out of rollout metric aggregation

Tests

• D:\RL-Kernel\.venv-py311\Scripts\python.exe -m pytest tests\utils\test_dlogp_diagnostics.py -m unit
• D:\RL-Kernel\.venv-py311\Scripts\python.exe -m pytest tests\test_metric_report.py -m unit
• D:\RL-Kernel\.venv-py311\Scripts\python.exe -m pytest tests\test_megatron_argument_validation.py -m unit
• pre-commit run --all-files --show-diff-on-failure --color=always

Closes #15

Co-authored-by: OpenAI Codex codex@openai.com

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ae96f669-a218-4294-bacb-90bb3b1b1905

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/issue15-dlogp-audit-diagnostics

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}