Model reference · open weights
DeepSeek-Flash-0731 is an open-weight language model from PrimeIntellect. DeepSeek-V4-Flash-0731-bf16 (BF16) weighs 569 GB; the smallest configuration that runs it is 8× A100 80 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the PrimeIntellect/DeepSeek-V4-Flash-0731-bf16 model card, 2026-10-21
What it is
| Released by | PrimeIntellect |
|---|---|
| Released | 2026-09-14 |
| Parameters | 284.3B |
| VRAM | 569 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB … 8× H100 80 GB 13 smaller cards | ||||
| 8× A100 80 GB tensor parallel | 7 | 1 | 62K | 78.2 GB a card |
| 4× B200 180 GB tensor parallel | 31 | 7 | 250K | 176 GB a card |
| 8× H200 141 GB tensor parallel | 81 | 20 | 655K | 138 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 571 GB | 573 GB |
| 5 | 574 GB | 585 GB |
| 8 | 576 GB | 593 GB |
| 16 | 582 GB | 616 GB |
| 32 | 593 GB | 663 GB |
| 64 | 616 GB | 755 GB |
One card, with vLLM's small-card settings.
From the model card
A bfloat16 re-export of deepseek-ai/DeepSeek-V4-Flash-0731. The weights are the published ones, dequantized: the dense linears from fp8 (e4m3 with ue8m0 scales over 128x128 blocks) and the routed experts from MXFP4, both to plain bfloat16. 43 layers, 114 safetensors shards.
The upstream config.json carries quantization_config and expert_dtype = "fp4". An inference
engine reading those fields allocates fp8 linear parameters and MXFP4 expert parameters, which
bfloat16 weights cannot fill. The routed experts at least fail loudly, in vLLM's _load_w13, where
the packed MXFP4 shape does not match. The fp8 linears fail silently: a bfloat16 tensor has the same
shape as the fp8 parameter it is loaded into, so copy_ casts it elementwise and no dequantization
scale is ever applied. Serving bfloat16 weights therefore requires a checkpoint whose config carries
neither field, which is what this repo is.
quantization_config and expert_dtype removed from config.json.num_nextn_predict_layers, dspark_block_size, dspark_markov_rank, dspark_noise_token_id and
dspark_target_layer_ids removed from config.json.mtp.* weights (upstream ships 4,705 of them; this export has none), so speculative decoding,
including DSpark, is not available here. Do not pass --speculative-config or
--speculative-algorithm DSPARK.There is no Jinja chat template, by upstream design. Use the bundled encoding/ directory, which is
copied verbatim from upstream and is the reference implementation for turning OpenAI-format messages
into an input string and for parsing completions back.
prime-rl users: MODEL_RENDERER_MAP is keyed on the upstream repo id
(deepseek-ai/DeepSeek-V4-Flash-0731), so this repo id does not resolve automatically and falls back
to the default renderer. Set the renderer explicitly:
[orchestrator.renderer]
name = "deepseek-v4"
Exported from the conversion cache that
prime-rl's trainer writes the first time it loads
the published checkpoint, which is where the fp8 and MXFP4 weights are dequantized to bfloat16. The
metadata.total_size in model.safetensors.index.json is byte-identical to that cache's
(568,753,728,604 bytes), so the cache-to-HF round trip drops nothing.
MIT, inherited from upstream. The upstream LICENSE is included verbatim.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.