Value Residual + Gated Attention + XSA + EMA + AdamW TTT — val_bpb pending H100#430

Draft

sahiee-dev wants to merge 12 commits intoopenai:mainfrom

sahiee-dev:main

Commits on Mar 22, 2026

Pivot to XSA+TTT+TrigramHash — QAT confirmed negative (PR #360)
sahiee-dev
committed

Commits on Mar 23, 2026

Add EMA + update submission: TrigramHash+XSA+EMA+TTT stack
sahiee-dev
committed
Remove TrigramHash — ablation confirms negative result
sahiee-dev
committed
Add Gated Attention + AdamW TTT — targeting sub-1.09
sahiee-dev
committed
Replace ReLU² with SwiGLU — iso-param, consistent with JoeProAI #1 open PR (1.0672)
sahiee-dev
committed
Fix serialization path + EMA every-10-steps + verify eval tokenizer
sahiee-dev
committed
Reduce artifact size: bigram 10240->7168, mlp_mult 3.0->2.75 to fit 16MB
sahiee-dev
committed
Replace AdamW TTT with LoRA TTT — consistent with PROTEUS v7 sub-1.0 approach
sahiee-dev
committed
Fix LoRA TTT dtype: cast tokens to long before embedding lookup
sahiee-dev
committed
Fix LoRA TTT device: create lora params on same device as base layer
sahiee-dev
committed
Fix LoRA dtype: cast to input dtype at runtime matching CastedLinear pattern
sahiee-dev
committed
Add per-document LoRA reset + multi-epoch TTT matching PROTEUS v7
sahiee-dev
committed