Do not quantize the teacher by default in the distillation scripts

[behroozazarkhalili]

What this does

Closes #6720.

Both distillation scripts default the teacher's quantization config to the student's, so --load_in_4bit (or any other ModelConfig quantization flag) quantizes the teacher as well. This drops quantization_config from the teacher's default init kwargs in both scripts, leaving the teacher in full precision unless the user asks otherwise.

The rationale is the one @cmpatino gave in #6642 (comment): the consensus in the literature is to avoid quantizing the teacher. The teacher defines the reference distribution the student is trained to match, so degrading it degrades the training signal itself, and since it receives no gradients the memory saving buys less than the same quantization does on the student.

Why no new config field

DistillationConfig.teacher_model_init_kwargs already exists, and both scripts merge it over their defaults. A per-model quantization choice was therefore already expressible; only the default coupled the two. Users who do want a quantized teacher can pass:

--teacher_model_init_kwargs '{"quantization_config": {"load_in_4bit": true}}'

from_pretrained accepts Union[QuantizationConfigMixin, Dict] for quantization_config (checked against transformers 5.11.0), and teacher_model_init_kwargs is listed in _VALID_DICT_FIELDS, so the JSON string form parses on the command line.

Scope note

Both copies carried the same default, so both are updated to keep them consistent:

• trl/scripts/distillation.py is unreleased (added in aacb260, not contained in v1.9.0), so this is not a behavior change for any released version.
• examples/scripts/distillation.py has carried the coupling since c475b97 and has shipped in v1.1.0 through v1.9.2. Changing it here is a real default change for users of that script. Happy to drop that hunk and land only the CLI half if you would rather keep the released example as is.

No test covers the scripts' quantization wiring and the docs do not mention it, so nothing else needed updating.

Checks

• ruff check and ruff format --check on both files, using the pinned v0.13.3: clean.
• doc-builder style --max_len 119 on both files: no changes needed, verified against a deliberately over-long fixture that the same invocation rejects.

---

Note

Low Risk
Script-only default changes for model loading; student quantization behavior is unchanged and teachers can still be quantized explicitly.

Overview
Stops coupling teacher quantization to the student in examples/scripts/distillation.py and trl/scripts/distillation.py. When users pass student quantization flags (e.g. --load_in_4bit), the teacher no longer inherits the same quantization_config and loads in full precision by default.

Comments document that quantizing the teacher can weaken the reference distribution for distillation, and that quantization can still be applied via --teacher_model_init_kwargs '{"quantization_config": {...}}'. The student path is unchanged (the example script still passes quantization_config into model_init_kwargs; the CLI script still passes it to DistillationTrainer).

^{Reviewed by Cursor Bugbot for commit 7e4aee0. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.