# DeepSeek Open-Sources DeepInfer Inference Engine The framework powers DeepSeek-V4.1-Flash models with high throughput and low latency. - **Source:** [The Next Web](https://brifing.si/source/afe24b20-b275-4c19-b9a6-8bb3c8babae7) - **Published:** 2026-10-20 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/29f08f01-b887-41de-85e3-45d3a10c8b4b ## DeepInfer Framework Release DeepSeek has officially open-sourced DeepInfer, the internal engine used to run its AI models in production environments. Released under the Apache 2.0 licence on GitHub, the framework currently supports the DeepSeek-V4.1-Flash model. This move aims to help the broader community lower compute costs and reduce energy consumption during AI inference tasks. ## Technical Optimization Features DeepInfer bundles optimized code designed to split computational workloads across multiple chips efficiently. It utilizes speculative decoding, a technique that drafts several tokens ahead to check them in a single pass. Additionally, the software includes features to ensure output consistency and force specific response formats, as detailed on the project's official GitHub page. ## Hardware Support and Compatibility The framework currently supports two specific hardware architectures: Nvidia’s SM100 data-centre GPUs and Huawei’s Ascend 950 chips. DeepSeek confirmed that code for future models and further technical improvements will be added to the same repository. This hardware flexibility provides an alternative to proprietary stacks, following the company's recent release of Huawei chip tools. ## Scope and Limitations DeepInfer is not a complete, standalone service as it currently lacks a cache system, load balancing across machines, and the separation between prompt processing and answer generation. To assist users, DeepSeek also released the deepseek-recipe, which replicates how its official API converts text into tokens and back for consistent model interaction. ## Foundational Software Dependencies The development of DeepInfer was primarily built upon vLLM, a widely recognized open-source inference engine. DeepSeek also incorporated elements from SGLang and Nvidia’s TensorRT-LLM to enhance performance. The vLLM project acknowledged this contribution, noting that they intend to integrate ideas from DeepInfer to improve their own support for the V4.1-Flash model. ## Performance Benchmarking Context DeepSeek reported performance benchmarks based on four SM100 GPUs for prompt processing and 64 GPUs for output generation. These figures are internal company metrics, and there is currently no independent benchmarking available to verify the specific gains. Cloud hosting companies can now implement these methods to optimize their own model serving operations. ## Strategic Industry Implications AI companies typically guard their inference techniques as proprietary trade secrets, making this release a notable shift in transparency. By sharing these methods, DeepSeek allows organizations running models on private servers to potentially cut operational costs. This follows a broader trend where open-weight models now account for 23% of spending on Vercel’s AI Gateway. ## Regulatory and Market Environment The release occurred just one day after the United States ordered civilian federal agencies to remove DeepSeek products from their systems. Despite this regulatory pressure, the GitHub repository gained significant traction, surpassing 8,000 stars by Tuesday afternoon. The project remains a focal point for developers seeking to optimize large-scale model deployment. ## Sources covering this story - https://thenextweb.com/news/deepseek-deepinfer-open-source-inference-engine — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._