Model reference · open weights
Millie-11GB is an open-weight language model from llmsforall. Millie-35B-A3B-11GB (BF16) weighs 10.3 GB; the smallest configuration that runs it is RTX 3060 12 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the llmsforall/Millie-35B-A3B-11GB model card, 2026-10-01
What it is
| Released by | llmsforall |
|---|---|
| Released | 2026-08-30 |
| VRAM | 10.3 GB for the weights |
What it runs on
How much memory each request adds isn't estimated yet for this architecture. The weights need at least the cards below, plus room for the context.
| Card | Weights alone |
|---|---|
| RTX 3060 12 GB | tight |
| RTX 4060 Ti 16 GB | fits |
| RTX 3090 24 GB | fits |
| RTX 4090 24 GB | fits |
| RTX 5090 32 GB | fits |
| L40S 48 GB | fits |
| A100 80 GB | fits |
| H100 80 GB | fits |
| RTX PRO 6000 Blackwell 96 GB | fits |
| DGX Spark (GB10) 128 GB unified | fits |
| H200 141 GB | fits |
| B200 180 GB | fits |
| 2× RTX 3060 12 GB split by layers (llama.cpp) | fits |
From the model card
Millie is a 35B-parameter mixture-of-experts coding and agentic model (3B active per token, 262K context, image input) compressed to run on ordinary hardware. It is derived from Agents-A1, built on the Qwen3.5-35B-A3B architecture; expert weights are stored at ~2.2 bits per weight. A smaller 7 GB variant is also available.
| File | Size | Contents |
|---|---|---|
| Millie-35B-A3B-11GB.gguf | 10.3 GB | language model |
| Millie-35B-A3B-mmproj.gguf | 0.6 GB | vision tower (image input) |
The GGUF files require the llmsforall llama.cpp fork, which carries the kernels for their weight format (Metal, Vulkan, CPU); upstream llama.cpp cannot load them:
llama-server -m Millie-35B-A3B-11GB.gguf --mmproj Millie-35B-A3B-mmproj.gguf
Apache 2.0 — see LICENSE and NOTICE.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.
How it works