Hi AbanteAI team,
I'd like to request adding the GLM-4.6 model from Zhipu AI (Z.ai) to your LoCoDiff-bench evaluations. This benchmark seems perfect for testing GLM-4.6's strengths in long-context tracking and coding agent capabilities, as it handles up to 200K tokens and excels in code generation, reasoning, and agentic tasks.
Why add GLM-4.6?
- Top-tier performance in reviews: It's consistently pushed as a leading open-weight model, outperforming or matching models like Claude 3.5 Sonnet, GPT-4 Turbo, and DeepSeek-V3 in benchmarks for coding (e.g., LiveCodeBench, HumanEval), reasoning, and agent tools. It's SOTA among open-source LLMs for efficiency and multi-modal tasks.
- Relevance to LoCoDiff: With its MoE architecture (357B parameters, 32B active), fast inference (20-30% faster than peers), and strong long-context handling, it could provide interesting insights into state tracking over file histories and diffs—areas where it's already praised in real-world apps like Cursor and Cognition.
- Community interest: GLM-4.6 is open-source under MIT license on Hugging Face, with rapid adoption (65K+ downloads recently). Adding it would make your benchmark more comprehensive, especially for comparing Chinese AI tigers against Western models.
It would be awesome if you included GLM-4.6 in your next report or update, as it's everywhere being hyped as surpassing many "hypers" (like GPT variants) and others. Having real, independent tests in your system would help validate these claims and benefit the community.
Links for reference:
Thanks for considering this! Let me know if you need any help with integration or additional info.
Hi AbanteAI team,
I'd like to request adding the GLM-4.6 model from Zhipu AI (Z.ai) to your LoCoDiff-bench evaluations. This benchmark seems perfect for testing GLM-4.6's strengths in long-context tracking and coding agent capabilities, as it handles up to 200K tokens and excels in code generation, reasoning, and agentic tasks.
Why add GLM-4.6?
It would be awesome if you included GLM-4.6 in your next report or update, as it's everywhere being hyped as surpassing many "hypers" (like GPT variants) and others. Having real, independent tests in your system would help validate these claims and benefit the community.
Links for reference:
Thanks for considering this! Let me know if you need any help with integration or additional info.