Model reference · open weights
Qwen1.5-MoE is an open-weight language model from Qwen. Qwen1.5-MoE-A2.7B (BF16) weighs 28.6 GB; the smallest configuration that runs it is L40S 48 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the Qwen/Qwen1.5-MoE-A2.7B model card, 2026-10-01
What it is
| Released by | Qwen |
|---|---|
| Released | 2024-02-29 |
| Parameters | 14.3B |
| VRAM | 28.6 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory |
|---|---|---|---|
| 8K each | |||
| RTX 3060 12 GB … RTX 4090 24 GB 4 smaller cards | |||
| RTX 5090 32 GB | 6K | 31.0 GB | |
| L40S 48 GB | 8 | all 8K | 44.0 GB |
| A100 80 GB | 30 | all 8K | 78.2 GB |
| H100 80 GB | 26 | all 8K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 36 | all 8K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 45 | all 8K | 107 GB |
| H200 141 GB | 64 | all 8K | 138 GB |
| B200 180 GB | 86 | all 8K | 176 GB |
| 2× RTX 4090 24 GB tensor parallel | 9 | all 8K | 23.4 GB a card |
| 2× RTX 3090 24 GB tensor parallel | 10 | all 8K | 23.4 GB a card |
| 2× RTX 5090 32 GB tensor parallel | 19 | all 8K | 31.0 GB a card |
| Requests at once | 8K tokens each |
|---|---|
| 1 | 31.2 GB |
| 5 | 37.7 GB |
| 8 | 42.5 GB |
| 16 | 55.4 GB |
| 32 | 81.2 GB |
| 64 | 133 GB |
One card, with vLLM's small-card settings.
From the model card
Qwen1.5-MoE is a transformer-based MoE decoder-only language model pretrained on a large amount of data.
For more details, please refer to our blog post and GitHub repo.
Qwen1.5-MoE employs Mixture of Experts (MoE) architecture, where the models are upcycled from dense language models. For instance, Qwen1.5-MoE-A2.7B is upcycled from Qwen-1.8B. It has 14.3B parameters in total and 2.7B activated parameters during runtime, while achieving comparable performance to Qwen1.5-7B, it only requires 25% of the training resources. We also observed that the inference speed is 1.74 times that of Qwen1.5-7B.
The code of Qwen1.5-MoE has been in the latest Hugging face transformers and we advise you to build from source with command pip install git+https://github.com/huggingface/transformers, or you might encounter the following error:
KeyError: 'qwen2_moe'.
We do not advise you to use base language models for text generation. Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., on this model.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.