feature: support eager as attn

[LovelyBuggies]

On some devices, using sdpa as the attention type during training can lead to strange issues. To address this, we support using the eager option.

While enabling this might slow down training and slightly increase VRAM usage, I haven't personally noticed any significant impact.

[LovelyBuggies]

This change is primarily to support development for a new repository. It involves using B200s to train 8B models, so there might be some unusual issues.

[LovelyBuggies]

A simple finding is that using "joint_mode: cross" doesn't significantly increase VRAM usage, but it does slow down the convergence speed.

I previously ran experiments on "Parallel" and "Multiple Parallel" configurations as well. The final performance was quite similar, though using only "Parallel" actually performed slightly worse.

Additionally, we didn't consider the samples in the "live" branch because some of the prompts there are extremely long, which would likely cause OOM errors.

[LovelyBuggies]

CoMLRL v1.4.1 Results on BFCL-v4 Parallel

MA-RLVR

MAGRPO: 4h training; 2xB200 (150G agent1 + 120G agent2)
MAAC: 9h training; 3xB200 (90G agent1 + 80G agent2 + 180G joint reward); can be optimized to be 2
IAC (shared-AC): 10h training; 2xB200 (125G agent1 + 125G agent2)

MA-DPO

MADPO: 6h training; 2xB200 (140G agent1 + 140G agent2)
MADPO-iter (decentralized): 10h training; 3xB200 (160G agent1 + 160G agent2 + 60 comp1,2,reward)
MADPO-iter (centralized): 9h training; 3xB200 (140G agent1 + 140G agent2 + 160G comp)

MA-RLHF

MARLHF: 7h training; 3xB200 (120G agent1 + 120G agent2+ 183G reward at peak)
MARLHF-iter (decentralized): 12h training; 3xB200 (170G agent1 + 150G agent2 + 180 comp1,2,reward)
MARLHF-iter (centralized): 12-15h training; 3xB200 (170G agent1 + 180G agent2 + 180G comp,reward)