My blog posts:
📄 Early observations: QK Norm seems to hurt Muon optimizer LLM training: English | Chinese
📄 Early LLM (transformer) layers pay attention to many tokens to gain context, later layers focus on fewer "important" tokens: English | Bilingual Chinese
📄 Magnitude Attention: Don't Let Group of Similar Keys Steal Your Probability Mass: English | Chinese
📄 Theoretical Proposal for Hyperbolic Gated Delta Nets: English | Bilingual Chinese
- 📬 Contact: vukrosic1@gmail.com
Goal: Join a collaborative AI research lab. I aim to do high-quality research with like-minded people and maintain a daily social media presence, posting everything I research to accelerate the adoption and progress of Open Source AI.
Core Requirements:
- Maintain my current pace of sharing research content on social media without restrictions every day to promote Open Science.
- Retain research autonomy to ensure that outputs benefit the public to the greatest extent through social media.
- We may agree on potential publications and directions.
Languages:
- 🇷🇸 Serbian (Mother Tongue)
- 🇺🇸 English (Bilingual / Completely Fluent)
- 🇨🇳 Chinese (Learning/Progressing — Goal: Professional proficiency in 2027)
Let’s build open-source AI together. ☕️
目标: 加入一个协作式 AI 研究实验室。我希望与志同道合的伙伴共同开展高质量科研工作，并保持每日社交媒体活跃，发布我研究的所有内容，以此加速开源 AI 的普及与进步。
核心诉求:
- 保持目前的节奏，每天在社交媒体上无限制地分享研究内容，以促进开放科学 (Open Science)。
- 保持研究自主权，确保产出能通过社交媒体最大程度地惠及大众。
- 我们可以就潜在的论文发表和研究方向进行协商。
愿意搬迁至中国。
语言能力:
- 🇷🇸 塞尔维亚语 (母语)
- 🇺🇸 英语 (双语/完全精通)
- 🇨🇳 中文 (学习中 — 目标：2027 年达到专业水平)
让我们一起构建开源 AI。 ☕️