Add Nemotron 3.5 Lightning support and testing

[qgallouedec]

Released 2026-08-11: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 — 30B/3B-active hybrid Mamba-2 + MoE, 1M context, open weights published as an SFT/RL/distillation starting point.

• Tiny model script nemotron_3_5_lightning_for_causal_lm.py (NemotronHForCausalLM), mirroring the Ultra script — Mamba weights are bf16 in the checkpoint, so no fp32 restore.
• Vendored nemotron_3_5_lightning.jinja + _training.jinja (Ultra's template minus the medium_effort hint) and registered both in chat_template_utils.py (reuses qwen3_5_schema / nemotron_3_template).
• Test params added next to Nemotron Ultra in test_chat_template_utils.py (4 lists) and test_data_utils.py. Trainer tests keep Nano as the single NemotronH representative.
• Prerequisite: push trl-internal-testing/tiny-NemotronHForCausalLM-3.5-lightning with transformers==5.3.0 before merging.

---

Note

Low Risk
Additive model-family support (templates, registration, tests, docs) with no changes to existing Nemotron or training paths; merge depends on publishing the tiny Hub model first.

Overview
Adds Nemotron 3.5 Lightning to TRL’s supported chat-template families so SFT/GRPO can auto-swap training templates and tool/response parsing works like other Nemotron-H models.

Vendored nemotron_3_5_lightning.jinja (Ultra-style template without medium_effort) and nemotron_3_5_lightning_training.jinja (assistant {% generation %} wrappers only). chat_template_utils.py loads both, maps Lightning to qwen3_5_schema / nemotron_3_template, and returns the training template from get_training_chat_template.

A new tiny-model script nemotron_3_5_lightning_for_causal_lm.py builds/pushes trl-internal-testing/tiny-NemotronHForCausalLM-3.5-lightning from the NVIDIA checkpoint (transformers ≥ 5.3.0). Docs and README catalog the family; tests add the tiny fixture to chat-template and data-utils parametrizations alongside existing Nemotron Ultra cases.

^{Reviewed by Cursor Bugbot for commit 1360c7b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.