Thinking Machines Launches Inkling-Small Open Source AI Model
The new model delivers near-flagship performance at a fraction of the size.
1Introduction of Inkling-Small
Just two weeks after releasing its first open-source AI language model Inkling, Thinking Machines unveiled Inkling-Small. The new model offers comparable performance at a significantly smaller size. With 276 billion parameters, Inkling-Small processes multimodal inputs and supports context windows up to one million tokens. Available under Apache 2.0 license, the model's full weights are accessible on Hugging Face, providing developers with immediate access to this advanced AI technology.
2Performance Benchmarks
Inkling-Small scored 40 on the Artificial Analysis Intelligence Index, just one point behind its 975-billion-parameter predecessor. It outperformed the original Inkling on several tests: 80.2% on SWE-bench Verified and 64.7% on Terminal Bench 2.1. However, it lagged in factual knowledge assessments, indicating strengths in coding and multimodal tasks but limitations for high-stakes factual applications. This performance profile makes it particularly suitable for certain enterprise use cases while requiring caution in others.
3Technical Architecture
Inkling-Small employs a sparse Mixture-of-Experts architecture. Its 42-layer decoder routes each token to 6 out of 256 specialized experts, plus two shared experts active for every token. While the model contains 276 billion total parameters, it only uses 12 billion active parameters per token. This design enables the model to maintain performance while reducing size. Native multimodal capabilities allow joint processing of text, image, and audio inputs in a shared representation space, eliminating the need for separate external systems.
4Enterprise Advantages
Inkling-Small offers substantial compute cost savings for businesses. Its smaller footprint reduces hosting expenses and enables more organizations to run it on their own GPU infrastructure. Thinking Machines is offering launch discounts up to 50% on API pricing: $0.58 per million input tokens and $1.44 per million output tokens for the standard 64K-context model. These competitive rates, combined with the model's efficiency, make it an attractive option for cost-conscious enterprises seeking near-flagship performance.
5Use Case Applications
Thinking Machines positions Inkling-Small as ideal for coding assistants, chatbots, document analysis, and multimodal workflows. The model's variable reasoning effort feature allows developers to adjust compute resources based on task difficulty, enabling optimal balance between quality, latency, and cost across different workloads. However, applications requiring high factual accuracy still need supplemental retrieval systems and human review, as the model shows some limitations in pure knowledge tasks despite its low hallucination rate.
6Hardware Requirements
Despite its 'Small' designation, Inkling-Small isn't a consumer-scale model. The standard BF16 checkpoint requires at least 600GB aggregate GPU memory, with Thinking Machines recommending configurations using 4x NVIDIA B300 or 8x NVIDIA H200 GPUs. NVFP4 quantization reduces requirements to about 180GB VRAM, but even this exceeds the capabilities of typical laptops and workstations. Practical deployment targets include enterprise GPU servers and cloud clusters, reflecting its position in the professional AI infrastructure market.
7Architectural Innovations
Inkling-Small's expert-based architecture maintains a large learned capacity pool while activating only a fraction during each inference step. This approach improves energy efficiency and reduces computational costs. The model allows different experts to specialize in various tasks, optimizing overall performance. Such architectures represent a promising direction for making large language models more practical for real-world applications, particularly in resource-constrained environments where full model deployment would be prohibitively expensive.
8Industry Implications
The release of Inkling-Small marks a significant advancement for the open-source AI ecosystem. Its performance-to-size ratio could empower mid-sized enterprises to enhance their AI capabilities. Led by former OpenAI CTO Mira Murati, Thinking Machines is pursuing aggressive pricing and licensing strategies to accelerate enterprise adoption. This move may intensify competition between proprietary models and open-source alternatives, potentially reshaping market dynamics as more organizations gain access to high-performance AI tools without prohibitive costs.