Model reference · open weights
ESFT-token-summary-lite is an open-weight language model from DeepSeek. ESFT-token-summary-lite (BF16) weighs 2.3 GB; the smallest configuration that runs it is RTX 3060 12 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
What it is
| Released by | DeepSeek |
|---|---|
| Released | 2024-07-04 |
| Parameters | 1.2B |
| VRAM | 2.3 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory |
|---|---|---|---|
| 4K each | |||
| RTX 3060 12 GB | 63 | all 4K | 11.6 GB |
| RTX 4060 Ti 16 GB | 93 | all 4K | 15.4 GB |
| RTX 3090 24 GB | 156 | all 4K | 23.4 GB |
| RTX 4090 24 GB | 156 | all 4K | 23.4 GB |
| RTX 5090 32 GB | 216 | all 4K | 31.0 GB |
| L40S 48 GB | 317 | all 4K | 44.0 GB |
| A100 80 GB | 586 | all 4K | 78.2 GB |
| H100 80 GB | 545 | all 4K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 668 | all 4K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 774 | all 4K | 107 GB |
| H200 141 GB | 1000+ | all 4K | 138 GB |
| B200 180 GB | 1000+ | all 4K | 176 GB |
| Requests at once | 4K tokens each |
|---|---|
| 1 | 3.6 GB |
| 5 | 4.1 GB |
| 8 | 4.5 GB |
| 16 | 5.5 GB |
| 32 | 7.6 GB |
| 64 | 11.6 GB |
One card, with vLLM's small-card settings.
From the model card
This model is a specialized expert set of the base model at https://huggingface.co/deepseek-ai/ESFT-vanilla-lite .
If you want to download or use the model, please refer to the scripts at https://github.com/deepseek-ai/ESFT .
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.