Deepseek open-sources the inference engine that powers its fastest and cheapest AI model
Deepseek is open-sourcing DeepInfer, the inference framework behind its fast V4.1-Flash model. The vLLM-based system cuts compute costs and energy use when running AI models. The release adds more pricing pressure on Western providers like OpenAI and Anthropic.
Via The Decoder
Deepseek has open-sourced DeepInfer, its internal inference framework, under the Apache License 2.0. Inference is how a trained AI model handles requests and generates responses. DeepInfer is the serving software that lets Deepseek's V4.1-Flash model produce answers quickly at scale.
The framework builds on vLLM with optimized compute kernels, distributed processing across multiple chips, and faster text generation techniques. Code for future Deepseek models will also ship as open source. Deepseek is also releasing deepseek-recipe, which replicates the token processing…
Courtesy
This story was originally published by The Decoder. All rights belong to the original publisher.
Read the original on the-decoder.com ↗