Moonshot AI Unveils Kimi K3: A 2.8 Trillion Parameter Open Model with 1M Token Context

Moonshot AI has officially released Kimi K3, a groundbreaking 2.8 trillion-parameter model that pushes the boundaries of open-source artificial intelligence. This new iteration boasts native vision capabilities and an astonishing 1 million token context window, positioning it as the world's first open 3T-class model. The company aims to empower developers and researchers with a powerful tool for long-horizon coding, intricate knowledge work, and advanced reasoning tasks.

Details of Kimi K3

At its core, Kimi K3 is a sparse Mixture-of-Experts (MoE) model built upon two significant architectural innovations: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). KDA is a hybrid linear attention mechanism designed to accelerate decoding speeds, reportedly up to 6.3 times faster within million-token contexts. Complementing this, AttnRes optimizes information flow across the model's depth, enabling selective retrieval of representations rather than uniform accumulation, which Moonshot claims leads to approximately 25% higher training efficiency at a minimal additional cost. Sparsity is further leveraged through Stable LatentMoE, activating only 16 out of 896 experts, which introduces routing and optimization as key challenges addressed by techniques like Quantile Balancing and Per-Head Muon. The model also incorporates Sigmoid Tanh Unit (SiTU) and Gated MLA for improved activation control and attention selectivity, respectively. These architectural advancements, coupled with refined training and data recipes, result in an estimated 2.5x improvement in overall scaling efficiency compared to its predecessor, Kimi K2. For deployment, K3 utilizes quantization-aware training from the Supervised Fine-Tuning (SFT) stage, employing MXFP4 weights and MXFP8 activations for broad hardware compatibility, with Moonshot recommending configurations of 64 or more accelerators.

Performance Benchmarks and Competitive Landscape

Moonshot AI has provided performance data for Kimi K3 across a range of benchmarks, with reasoning effort set to maximum. The results indicate strong capabilities, particularly in areas like Program Bench (77.8) and OmniDocBench (91.1), where Kimi K3 leads or is highly competitive. It also shows leadership in BrowseComp (91.2) and Automation Bench (30.8). However, the company candidly acknowledges that Kimi K3's overall performance still trails the most advanced proprietary models, such as Claude Fable 5 and GPT 5.6 Sol. For instance, Kimi K3 scores lower than Fable 5 on FrontierSWE (81.2 vs. 86.6) and HLE-Full (43.5 vs. 53.3), and trails GPT 5.6 Sol on DeepSWE (67.5 vs. 73.0). It's important to note that the Fable 5 scores include a fallback mechanism where requests refused by Fable 5 are routed to Opus 4.8, and the BrowseComp benchmark utilized context compaction at 300K tokens. Despite these comparisons, Kimi K3 represents a significant leap forward for open-source models in terms of scale and context length.

Use Cases and Multimodal Capabilities

The architecture of Kimi K3 is designed to support a variety of demanding use cases. Its extensive context window is particularly beneficial for repo-scale engineering, allowing for long coding sessions with minimal human intervention, leveraging tools like Kimi Code. The model's native vision capabilities enable vision-in-the-loop applications, facilitating iterative development between code and live screenshots. Researchers can utilize Kimi K3 for research reproduction, handling extensive datasets and complex analyses, such as the "I–Love–Q relations" study involving over 20 papers and 3,000 lines of Python code. Furthermore, its capacity for deep research reports, exemplified by a 42-year ASIC study requiring over 2.8k fetches and 11k pages, showcases its utility in processing vast amounts of information. Document parsing is another key area, with Kimi K3 achieving a high score on OmniDocBench, demonstrating its proficiency in extracting structured information from complex documents. Moonshot AI emphasizes that a single, native multimodal architecture handles text, images, and video seamlessly, opening up new possibilities for AI applications.

Access and Pricing Model

Kimi K3 is now accessible through various Moonshot AI platforms, including Kimi.com, Kimi Work, Kimi Code, and via its API. Integration is streamlined through an OpenAI SDK-compatible interface, allowing developers to connect using a Moonshot base URL. Key parameters for API calls include reasoning_effort which is fixed to max, and users should omit parameters like thinking from older versions, temperature, top_p, and n. The max_completion_tokens parameter can be adjusted up to 1048576. Moonshot AI has adopted a flat pricing structure, irrespective of context length, which is a significant advantage for users dealing with large inputs. The pricing is set at $0.30 per million tokens for cache-hit inputs, $3.00 per million tokens for cache-miss inputs, and $15.00 per million tokens for output. The company highlights that achieving a high cache-hit rate is crucial for cost efficiency, reporting rates above 90% in coding workloads. This transparent and predictable pricing model aims to make the powerful capabilities of Kimi K3 more accessible to a wider range of users and organizations.

Future Implications and Open Source Contribution

The release of Kimi K3 marks a significant milestone in the advancement of open-source large language models. By offering a 2.8 trillion-parameter model with advanced architectural features and a massive context window, Moonshot AI is democratizing access to cutting-edge AI technology. The company's commitment to open-source is further demonstrated by its contribution of a KDA prefix caching implementation to vLLM, addressing a specific technical challenge posed by the new attention mechanism. This collaborative approach fosters innovation within the AI community, enabling developers to build upon and refine these powerful tools. While Kimi K3 may not yet surpass the performance of top-tier proprietary models, its scale, context length, and open accessibility provide a compelling alternative for many applications, particularly those requiring extensive data processing and long-form reasoning. The focus on efficiency and hardware compatibility suggests a path towards more practical and widespread adoption of extremely large models.