AxForge
Sign in Get started

Model reference · open weights

gemma-2-bnb

LLMs unsloth Text gen 2 builds Open, with conditions 41k dl/mo

gemma-2-bnb is an open-weight language model from unsloth. gemma-2-27b-it-bnb-4bit (INT4) weighs 15.8 GB; the smallest configuration that runs it is 2× RTX 3060 12 GB.

Request Rent a GPU

For self-hosting, compare this build's requirements below with the GPU you plan to rent.

  • Gemma-2-bnb is a 2.7B parameter text-generation model created by unsloth and published as unsloth/gemma-2-2b-it-bnb-4bit.
  • It supports English with an 8192 token context length and is released under the gemma licence.
  • The model is directly quantized to 4bit using bitsandbytes.

Summary of the unsloth/gemma-2-2b-it-bnb-4bit model card, 2026-10-01. The estimate below is for another build of the family.

Hugging Face ↗

What it is

Released byunsloth
Released2024-07-31
Parameters2.7B
VRAM15.8 GB for the weights

What it runs on

Memory and cards for gemma-2-27b-it-bnb-4bit (INT4)

15.8 GBweights, file size
188 MBcache per 1K tokens
1.5 GBwindow cache per request
1.1 GBruntime overhead, at least
8,192 tokenscontext max
CardRequests at onceContext maxMemory
8K each
RTX 3060 12 GB … RTX 4060 Ti 16 GB
2 smaller cards
RTX 3090 24 GB2all 8K23.4 GB
RTX 4090 24 GB2all 8K23.4 GB
RTX 5090 32 GB4all 8K31.0 GB
L40S 48 GB8all 8K44.0 GB
A100 80 GB19all 8K78.2 GB
H100 80 GB17all 8K78.1 GB
RTX PRO 6000 Blackwell 96 GB22all 8K93.8 GB
DGX Spark (GB10) 128 GB unified27all 8K107 GB
H200 141 GB37all 8K138 GB
B200 180 GB49all 8K176 GB
2× RTX 3060 12 GB
tensor parallel		1all 8K11.6 GB a card
2× RTX 4060 Ti 16 GB
tensor parallel		4all 8K15.4 GB a card
2× RTX 4090 24 GB
tensor parallel		9all 8K23.4 GB a card
2× RTX 3090 24 GB
tensor parallel		9all 8K23.4 GB a card
2× RTX 5090 32 GB
tensor parallel		14all 8K31.0 GB a card
Memory needed at each load
Requests at once8K tokens each
120.0 GB
532.3 GB
841.6 GB
1666.3 GB
32116 GB
64214 GB

One card, with vLLM's small-card settings.

Builds

Sizes, precisions & builds

BuildParamsPrecisionWeightsSmallest setup
gemma-2-2b-it-bnb-4bit ↗ 2.7BINT4 2.2 GBRTX 3060 12 GB
gemma-2-27b-it-bnb-4bit (above) ↗ 28.0BINT4 15.8 GB2× RTX 3060 12 GB

From the model card

What unsloth says about gemma-2-bnb

Read the model card

Reminder to use the dev version Transformers:

pip install git+https://github.com/huggingface/transformers.git

Finetune Gemma, Llama 3, Mistral 2-5x faster with 70% less memory via Unsloth!

Directly quantized 4bit model with bitsandbytes.

All notebooks are beginner friendly! Add your dataset, click "Run All", and you'll get a 2x faster finetuned model which can be exported to GGUF, vLLM or uploaded to Hugging Face.

  • * Kaggle has 2x T4s, but we use 1. Due to overhead, 1x T4 is 5x faster.

Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.

© 2026 AxForge · EU-hosted AI infrastructure Pricing Docs Trust Privacy Terms