Model reference · open weights
Mistral Small is an open-weight language model from Mistral AI. this build (NVFP4) weighs 0 MB; the smallest configuration that runs it is RTX 3060 12 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
AxForge offerings: Mistral Small 3.2
What it is
| Released by | Mistral AI |
|---|---|
| Parameters | 24B dense |
| VRAM | 0 MB for the weights |
What it runs on
How much memory each request adds isn't estimated yet for this architecture. The weights need at least the cards below, plus room for the context.
| Card | Weights alone |
|---|---|
| RTX 3060 12 GB FP4 without its speed-up here | fits |
| RTX 4060 Ti 16 GB FP4 without its speed-up here | fits |
| RTX 3090 24 GB FP4 without its speed-up here | fits |
| RTX 4090 24 GB FP4 without its speed-up here | fits |
| RTX 5090 32 GB | fits |
| L40S 48 GB FP4 without its speed-up here | fits |
| A100 80 GB FP4 without its speed-up here | fits |
| H100 80 GB FP4 without its speed-up here | fits |
| RTX PRO 6000 Blackwell 96 GB | fits |
| DGX Spark (GB10) 128 GB unified | fits |
| H200 141 GB FP4 without its speed-up here | fits |
| B200 180 GB | fits |
Builds
| Build | Params | Precision | Weights | Smallest setup |
|---|---|---|---|---|
| build | 3.2 24B | BF16 | ≈ 0 MB | RTX 3060 12 GB |
| build | 3.2 24B | FP8 | ≈ 0 MB | RTX 3060 12 GB |
| build (above) packaged by RedHatAI |
3.2 24B | NVFP4 | ≈ 0 MB | RTX 3060 12 GB |
| build | 3.2 24B | GGUF (community) | ≈ 0 MB | RTX 3060 12 GB |
How it works