DeepInfer powers the company’s DeepSeek-V4.1-Flash model at scale with high throughput and low latency, DeepSeek said in a Tuesday (Oct. 20) post on X.
“We’re sharing our production stack to help the community lower compute costs, reduce energy use and make AI more affordable,” DeepSeek said in the post.
In a reply to its post, DeepSeek said the DeepInfer features high-performance kernels and an optimized inference engine, distributed inference and speculative decoding, and SWA Bounded Replay and structured outputs.
In another reply, DeepSeek said it is also open sourcing deepseek-recipe, the company’s production token encoding and decoding component that is aligned with the processing logic behind the official DeepSeek API to bring the same API behavior to community deployments.
We’d love to be your preferred source for news.
Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!
“From private deployments to public clouds, we’ll keep working with the community to make large-scale inference more efficient,” DeepSeek said in a third reply to its initial post.
Bloomberg reported Tuesday that DeepSeek’s release of DeepInfer reveals how the company contains the cost of inference, or running advanced AI; offers a way for cloud and hosting companies running DeepSeek’s models to cut their own costs; and offers a model for developers to reduce the costs of running their own inferencing systems.
PYMNTS reported in September 2025 that inference infrastructure is the next big layer in the generative AI tech stack because latency, cost, scale, energy use and deployment location all determine whether an AI service works or fails.
The report said that investors were already rewarding firms that capture inference demand, and that enterprises deploying generative AI solutions face inference costs that can reach hundreds of millions annually.
The Financial Times reported in June that companies are looking to better manage their use of artificial intelligence after seeing the costs of the technology rise and that this has created an opening for Chinese AI labs that are able to charge less than U.S. companies due to their more efficient models and China’s lower energy costs.
DeepSeek’s announcement of DeepInfer came a day after the U.S. Department of Homeland Security (DHS) issued an order banning federal civilian agencies from acquiring products and services developed by DeepSeek and requiring them to remove the company’s products from their information technology.
DHS said in its order that DeepSeek is a privately owned company located in the People’s Republic of China and that the company’s models “present multiple threat vectors.”
Add as Preferred Source
