Skip to content

Ensure device agnostic for examples/docs/templates - #6770

Open
kaixuanliu wants to merge 4 commits into
huggingface:mainfrom
kaixuanliu:device-agnostic
Open

Ensure device agnostic for examples/docs/templates#6770
kaixuanliu wants to merge 4 commits into
huggingface:mainfrom
kaixuanliu:device-agnostic

Conversation

@kaixuanliu

@kaixuanliu kaixuanliu commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

@qgallouedec @albertvillanova @kashif pls help review, thx!


Note

Low Risk
Documentation, templates, and example-only code paths; the online DPO change is a like-for-like swap to an existing multi-device cache helper with no auth or data-handling impact.

Overview
Makes TRL docs, model cards, and vLLM examples work across accelerators instead of assuming CUDA.

Inference examples now pick the active device with torch.accelerator.current_accelerator() (with CPU fallback in the use-model doc) and use device_map="auto" in generated model-card pipeline snippets instead of device="cuda". Tests assert the updated card text.

Training/runtime: Online DPO vLLM colocate wake-up calls the shared empty_cache() helper instead of torch.cuda.empty_cache(). The VLLMClient docstring and __main__ block drop the cuda/xpu branch in favor of torch.accelerator.current_accelerator() and device_map for loading weights.

Reviewed by Cursor Bugbot for commit 89189cf. Bugbot is set up for automated code reviews on this repo. Configure here.

@kaixuanliu kaixuanliu changed the title Device agnostic Ensure device agnostic for examples/docs/templates Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant