Model reference · open weights
airoboros-mistral2.2 is an open-weight language model from teknium. airoboros-mistral2.2-7b (BF16) weighs 14.5 GB; the smallest configuration that runs it is 2× RTX 3060 12 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the teknium/airoboros-mistral2.2-7b model card, 2026-10-01
What it is
| Released by | teknium |
|---|---|
| Released | 2023-10-03 |
| VRAM | 14.5 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB | 11.6 GB | |||
| RTX 4060 Ti 16 GB | 15.4 GB | |||
| RTX 3090 24 GB | 7 | 7 | all 32K | 23.4 GB |
| RTX 4090 24 GB | 7 | 7 | all 32K | 23.4 GB |
| RTX 5090 32 GB | 14 | 14 | all 32K | 31.0 GB |
| L40S 48 GB | 26 | 26 | all 32K | 44.0 GB |
| A100 80 GB | 58 | 58 | all 32K | 78.2 GB |
| H100 80 GB | 55 | 22 | all 32K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 70 | 28 | all 32K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 83 | 33 | all 32K | 107 GB |
| H200 141 GB | 111 | 44 | all 32K | 138 GB |
| B200 180 GB | 146 | 36 | all 32K | 176 GB |
| 2× RTX 3060 12 GB tensor parallel | 6 | 6 | all 32K | 11.6 GB a card |
| 2× RTX 4060 Ti 16 GB tensor parallel | 14 | 14 | all 32K | 15.4 GB a card |
| 2× RTX 4090 24 GB tensor parallel | 28 | 28 | all 32K | 23.4 GB a card |
| 2× RTX 3090 24 GB tensor parallel | 28 | 28 | all 32K | 23.4 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 16.2 GB | 16.2 GB |
| 5 | 20.5 GB | 20.5 GB |
| 8 | 23.7 GB | 23.7 GB |
| 16 | 32.3 GB | 32.3 GB |
| 32 | 49.5 GB | 49.5 GB |
| 64 | 83.8 GB | 83.8 GB |
One card, with vLLM's small-card settings.
From the model card
Mistral trained with the airoboros dataset!
Actual dataset is airoboros 2.2, but it seems to have been replaced on hf with 2.2.1.
Prompt Format:
USER:
ASSISTANT:
TruthfulQA:
hf-causal-experimental (pretrained=/home/teknium/dakota/lm-evaluation-harness/airoboros2.2-mistral/,dtype=float16), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8
| Task |Version|Metric|Value | |Stderr|
|-------------|------:|------|-----:|---|-----:|
|truthfulqa_mc| 1|mc1 |0.3562|± |0.0168|
| | |mc2 |0.5217|± |0.0156|
Wandb training charts: https://wandb.ai/teknium1/airoboros-mistral-7b/runs/airoboros-mistral-1?workspace=user-teknium1
More info to come
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.