Skip to content

docs: fix reward function API contract drift - #450

Open
CalebWang0126 wants to merge 1 commit into
inclusionAI:mainfrom
CalebWang0126:docs/reward-contract-drift
Open

docs: fix reward function API contract drift#450
CalebWang0126 wants to merge 1 commit into
inclusionAI:mainfrom
CalebWang0126:docs/reward-contract-drift

Conversation

@CalebWang0126

Copy link
Copy Markdown

Summary

Fixes #141.

Some documentation still described the reward contract as reward_fn(example, completions) -> list[float], while the implementation loads reward_fn(record) -> float over a RewardRecord (areno/api/rewards.py). This PR retires the old batch shape everywhere and documents the current contract.

Changes

  • docs/concepts/reward-functions.rst: rewritten around reward_fn(record) -> float. Explains the prompt-RL record fields, the agentic-RL trajectory fields (messages, trace, tool_calls, tool_results, rendered_completion, final_answer), and how dataset-loader rows reach rewards via source_record / solutions -> answer. Points at the existing example reward files.
  • docs/reference/reward-function-api.rst: documents the full RewardRecord field table, the scalar return contract, and the --reward-fn-path loading/validation behavior (must define callable reward_fn(record)).
  • docs/troubleshooting/reward-function.rst: drops the batch-shape check ("reward list length matches the completions list length") in favor of the per-record contract.
  • README.md, AGENTS.md, CONTRIBUTING.md: replace remaining reward_fn(example, completions) -> list[float] references. The README SDK quickstart additionally called the example reward file with the old shape, which no longer matches examples/math/math_verify_reward.py; it now builds one RewardRecord per rollout sample, mirroring what the CLI trainer does. These three files are outside docs/ but still presented the old shape as the current contract, so they are included to satisfy the acceptance criterion.

Verification

  • grep across docs, README, AGENTS.md, CONTRIBUTING.md, CODEMAP.md, and examples: no reward_fn(example, completions) references remain.
  • sphinx-build -b html -E docs builds with zero warnings and no broken internal links.
  • The rewritten README quickstart snippet passes Python syntax compilation.
  • Docs-only change; no code or public API changes.

Document reward_fn(record) -> float over RewardRecord for prompt and agentic RL, including the dataset-loader metadata flow, and retire the old batch reward_fn(example, completions) shape from docs, README, AGENTS.md, and CONTRIBUTING.md.

Closes inclusionAI#141
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: Fix reward function API contract drift

1 participant