docs: fix reward function API contract drift - #450
Open
CalebWang0126 wants to merge 1 commit into
Open
Conversation
Document reward_fn(record) -> float over RewardRecord for prompt and agentic RL, including the dataset-loader metadata flow, and retire the old batch reward_fn(example, completions) shape from docs, README, AGENTS.md, and CONTRIBUTING.md. Closes inclusionAI#141
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #141.
Some documentation still described the reward contract as
reward_fn(example, completions) -> list[float], while the implementation loadsreward_fn(record) -> floatover aRewardRecord(areno/api/rewards.py). This PR retires the old batch shape everywhere and documents the current contract.Changes
docs/concepts/reward-functions.rst: rewritten aroundreward_fn(record) -> float. Explains the prompt-RL record fields, the agentic-RL trajectory fields (messages,trace,tool_calls,tool_results,rendered_completion,final_answer), and how dataset-loader rows reach rewards viasource_record/solutions->answer. Points at the existing example reward files.docs/reference/reward-function-api.rst: documents the fullRewardRecordfield table, the scalar return contract, and the--reward-fn-pathloading/validation behavior (must define callable reward_fn(record)).docs/troubleshooting/reward-function.rst: drops the batch-shape check ("reward list length matches the completions list length") in favor of the per-record contract.README.md,AGENTS.md,CONTRIBUTING.md: replace remainingreward_fn(example, completions) -> list[float]references. The README SDK quickstart additionally called the example reward file with the old shape, which no longer matchesexamples/math/math_verify_reward.py; it now builds oneRewardRecordper rollout sample, mirroring what the CLI trainer does. These three files are outsidedocs/but still presented the old shape as the current contract, so they are included to satisfy the acceptance criterion.Verification
grepacross docs, README, AGENTS.md, CONTRIBUTING.md, CODEMAP.md, and examples: noreward_fn(example, completions)references remain.sphinx-build -b html -E docsbuilds with zero warnings and no broken internal links.