SGLang and Miles Add Day-0 Support for DeepSeek-V4.1
Day‑0 inference support in SGLang and Day‑0 RL support in Miles.
Overview
DeepSeek-V4.1 Flash is the smallest model in DeepSeek's new architecture series: a 552B-parameter MoE with native multimodal vision. Its Causal-Encoder-Decoder activates 8B parameters on input and 16B on output.
DeepSeek reports that V4.1 Flash is more capable than V4 Pro, at lower cost, with faster generation and shorter task times. Weights are on Hugging Face; the architecture is described in the technical report.
What's New vs. V4
V4.1 introduces a new architecture rather than a V4 checkpoint. Flash is larger than V4 Flash, activates fewer parameters, includes native vision, and uses a smaller KV cache.
| V4 | V4.1 Flash | |
|---|---|---|
| Architecture | Decoder-only MoECSA / HCA, mHC, MTP | Causal encoder-decoderCompressed attention + SWA, mHC, Engram |
| Parameters | 284B Flash · 1.6T Pro13B and 49B active | 552B total8B active on input, 16B on output |
| Global KV per token | 3,514 bytes | 890 bytesAbout a quarter of V4 |
| Modalities | TextVision via the Flash-Vision-Exp add-on | Native vision |
SGLang Day-0 Support
SGLang provides Day-0 support for V4.1. The tables below list launch coverage for features and hardware.
Features
| Feature | Status | Coverage |
|---|---|---|
| Parallelism | Yes | TP, EP, DP attention, CP (text only), and MoRI (text only) |
| Speculative Decoding | Yes | Official DSpark implementation |
| Caching | Yes | Radix prefix cache, chunked prefill, Mooncake, and PD disaggregation |
| HiCache | Yes | Compatible with DSpark speculative decoding |
| Execution | Yes | Decode CUDA Graph, prefill CUDA Graph, and multi-stream overlap |
| Official Kernels | Yes | DeepGEMM, FlashMLA |
| Quantization | Yes | FP8 dense (MXFP8 on Blackwell), MXFP4 experts, and FP4 indexer cache |
| Vision | Yes | Image input, dynamic resolution, multi-image, and VL expert routing |
| API | Yes | Chat encoder, tool calling, reasoning parser, reasoning effort (low / high / xhigh / max or 1–100), thinking toggle, quick instruction tasks, structured output, and streaming |
| SWA Bounded Replay | Yes | Encoder SWA bounded replay and decoder SWA bounded replay |
| Engram Memory | Yes | On by default, with options to choose where it is stored (host or on device) |
Engram Memory
SGLang supports flexible placement of Engram tables in GPU or host memory. On NVIDIA GB-series systems, SGLang takes advantage of fast CPU–GPU connectivity to move these tables into host memory, freeing GPU HBM for KV cache. Shared host storage also eliminates cross-GPU collectives for Engram lookups.
We have evaluated host-memory placement on NVIDIA GB-series systems. The best placement strategy depends on each platform's memory and interconnect capabilities, and we are exploring additional platforms to broaden our recommendations.
For more details, read the technical deep dive.
Hardware
Day-0 support is available on NVIDIA Blackwell (B200, B300, and GB300), NVIDIA Hopper (H200), and AMD Instinct MI350X. All Day-0 recipes use preview images; support has not shipped in a stock SGLang release yet.
DeepSeek's updated FlashMLA and DeepGEMM kernels for V4.1 are integrated. We are still integrating the new DeepSelect to enable further performance improvements.
Launch recipes are in the SGLang Cookbook. For questions, join the SGLang Slack.
Miles Day-0 Support
Miles is an open-source RL and large-scale model post-training framework with native SGLang rollouts. For V4.1, its Megatron-Core plugin implements shared attention state, mHC, and Engram on the training path.
Miles is also actively developing support for long-horizon coding-agent training, including Terminal-Bench tasks.
| Area | Day-0 | Coverage |
|---|---|---|
| Backend | Yes | Megatron-Core plugin |
| Rollout | Yes | SGLang |
| Parallelism | Yes | DP / TP / SP / EP / PP / CP |
| Quantization | Yes | FP4 compressed state, FP8 window cache, and straight-through gradients |
| Routing Replay | Yes | Rollout expert assignments fed back to the trainer |
| Validated Run | Yes | DAPO-Math-17K, 16×GB300, TP4 / EP16, 128 samples/step, 120+ steps |
A DAPO-Math-17K run on 16×GB300 completed 120+ steps (TP4, EP16, 128 samples/step). Over steps 0–80, five-step mean reward rose from 0.51 to 0.78, and trainer–rollout per-token KL stayed between 0.0012 and 0.0017.
The training recipe is available in the Miles Cookbook.
For the implementation details behind SGLang serving and Miles post-training support, read the technical deep dive.