feat(agentic): configurable trainable-turn selection and tool-call arg masking

[yourshana]

What

Make trainable turns configurable for agentic trajectories. Three modes + a tool-call arg masking option, applied at token granularity, plus turn-level validation and trainable-token observability.

Option	Default	Values
--trainable-turns	all_assistant	all_assistant | last_assistant | final_answer
--mask-tool-call-args	off	flag

• all_assistant — every assistant span contributes to policy loss (backward-compatible).
• last_assistant — only the final assistant span trainable.
• final_answer — only the assistant_text span after the last tool result; degenerates to last span when no tools; bare trailing tool call → zero signal (no error).
• --mask-tool-call-args — masks JSON-argument tokens within tool-call spans, keeps tool-name trainable. Research ablation; arg localization is approximate.

Why

LossMaskPolicy.final_assistant_text was a dead flag (defined at agentic.py:54, never read). No "last turn / final answer only" mode, no trainable-token count, no call/result pairing validation existed. The issue (#199) asks for exactly these capabilities using existing AReno contracts.

How

• last_assistant/final_answer need trajectory-level span info the per-span _response_loss_mask_for_span can't provide. A ResponseSpan list is captured during trajectory assembly (_set_sample_training_row, _append_sample_response) and consumed by _apply_trainable_turn_mode at the single chokepoint _train_rows_from_samples. The mode mask is composed on top of the existing per-span mask (including _tool_call_loss_mask's result-region suppression), never rebuilt from zero.
• _validate_call_result_pairing rejects mid-trajectory tool calls without matching tool results via ValueError before worker init. Trailing bare tool calls are exempt so final_answer can yield zero trainable signal without error. Orphan tool results are tolerated.
• mask_tool_call_args localizes JSON arguments via _tool_call_arg_token_range (decode + brace-match). Does NOT reuse _tool_call_loss_mask sentinel logic (different region). Localization is approximate; behavior pinned by per-token CPU tests.

Verification

• pytest tests/test_agentic_cpu.py → 63 passed (20 new + 43 regression)
• --trainable-turns bogus → click.UsageError (exit 2)
• TrainerConfig(trainable_turns="invalid") → ValueError
• Default reproduces pre-change loss mask exactly
• GPU smoke run on Colab T4 (Qwen3-0.6B + GSPO), confirmed trainable_turns config propagation across all three modes

Acceptance criteria

• Fixed multi-tool transcript with token-by-token mask assertions for every mode; trainable_tokens/masked_response_tokens in metrics; malformed call/result pairs rejected with turn-level error
• Uses existing AReno contracts; no external database or mandatory sandbox
• Default behavior remains backward compatible
• Focused automated tests cover success, invalid input, and boundary/failure paths
• User documentation includes a minimal runnable example (examples/agentic/trainable_turns_demo.py) and explains observable output

Files

areno/api/agentic.py · areno/api/trainer_config.py · areno/api/trainers/policy_only.py · areno/cli/train.py · tests/test_agentic_cpu.py · docs/cli/training.rst · docs/sdk/trainer.rst · docs/troubleshooting/agentic-rollout.rst · examples/agentic/trainable_turns_demo.py

Notes for reviewers

• No new dependencies; no trainer/rollout/dashboard changes.
• _tool_call_loss_mask sentinel logic left unchanged; mask_tool_call_args does not reuse it (different region).
• Removed final_assistant_text had no readers.

Closes #199