GigaChat 3 Ultra vs. DeepScaleR

GigaChat 3 Ultra is a breakthrough open-source LLM, offering 702 billion parameters built on an advanced MoE architecture that keeps computation efficient while delivering frontier-level performance. Its design activates only 36 billion parameters per step, combining high intelligence with practical deployment speeds, even for research and enterprise workloads. The model is trained entirely from scratch on a 14-trillion-token dataset spanning ten+ languages, expansive natural corpora, technical literature, competitive programming problems, academic datasets, and more than 5.5 trillion synthetic tokens engineered to enhance reasoning depth. This approach enables the model to achieve exceptional Russian-language capabilities, strong multilingual performance, and competitive global benchmark scores across math (GSM8K, MATH-500), programming (HumanEval+), and domain-specific evaluations. GigaChat 3 Ultra is optimized for compatibility with modern open-source tooling, enabling fine-tuning, inference, and integration using standard frameworks without complex custom builds. Advanced engineering techniques—including MTP, MLA, expert balancing, and large-scale distributed training—ensure stable learning at enormous scale while preserving fast inference. Beyond raw intelligence, the model includes upgraded alignment, improved conversational behavior, and a refined chat template using TypeScript-based function definitions for cleaner, more efficient interactions. It also features a built-in code interpreter, enhanced search subsystem with query reformulation, long-term user memory capabilities, and improved Russian-language stylistic accuracy down to punctuation and orthography. With leading performance on Russian benchmarks and strong showings across international tests, GigaChat 3 Ultra stands among the top five largest and most advanced open-source LLMs in the world. It represents a major engineering milestone for the open community.

DeepScaleR is an advanced language model featuring 1.5 billion parameters, developed from DeepSeek-R1-Distilled-Qwen-1.5B through a unique blend of distributed reinforcement learning and a novel technique that gradually increases its context window from 8,000 to 24,000 tokens throughout training. The model was constructed using around 40,000 carefully curated mathematical problems taken from prestigious competition datasets, such as AIME (1984–2023), AMC (pre-2023), Omni-MATH, and STILL. With an impressive accuracy rate of 43.1% on the AIME 2024 exam, DeepScaleR exhibits a remarkable improvement of approximately 14.3 percentage points over its base version, surpassing even the significantly larger proprietary O1-Preview model. Furthermore, its outstanding performance on various mathematical benchmarks, including MATH-500, AMC 2023, Minerva Math, and OlympiadBench, illustrates that smaller, finely-tuned models enhanced by reinforcement learning can compete with or exceed the performance of larger counterparts in complex reasoning challenges. This breakthrough highlights the promising potential of streamlined modeling techniques in advancing mathematical problem-solving capabilities, encouraging further exploration in the field. Moreover, it opens doors for developing more efficient models that can tackle increasingly challenging problems with great efficacy.