Model reference · open weights

gemma-2

LLMs unsloth Text gen 2 builds Open, with conditions 43k dl/mo

gemma-2 is an open-weight language model from unsloth. gemma-2-27b (BF16) weighs 54.5 GB; the smallest configuration that runs it is H100 80 GB.

Request Rent a GPU

For self-hosting, compare this build's requirements below with the GPU you plan to rent.

Gemma-2 is a 27.2B parameter text-generation model published by unsloth as unsloth/gemma-2-27b.
It supports English with a context length of 8192 tokens and is distributed under the gemma licence.
The model card describes it as a directly quantized 4bit version designed for efficient finetuning.

Summary of the unsloth/gemma-2-27b model card, 2026-10-01

Hugging Face ↗

What it is

Released by	unsloth
Released	2024-06-27
Parameters	27.2B
VRAM	54.5 GB for the weights

What it runs on

Memory and cards for gemma-2-27b (BF16)

54.5 GBweights, file size

188 MBcache per 1K tokens

1.5 GBwindow cache per request

1.1 GBruntime overhead, at least

8,192 tokenscontext max

Card	Requests at once	Context max	Memory
Card	8K each	Context max	Memory
RTX 3060 12 GB … L40S 48 GB 6 smaller cards
A100 80 GB	7	all 8K	78.2 GB
H100 80 GB	5	all 8K	78.1 GB
RTX PRO 6000 Blackwell 96 GB	10	all 8K	93.8 GB
DGX Spark (GB10) 128 GB unified	14	all 8K	107 GB
H200 141 GB	24	all 8K	138 GB
B200 180 GB	36	all 8K	176 GB
2× L40S 48 GB tensor parallel	10	all 8K	44.0 GB a card
4× RTX 4090 24 GB tensor parallel	11	all 8K	23.4 GB a card
4× RTX 3090 24 GB tensor parallel	11	all 8K	23.4 GB a card
4× RTX 5090 32 GB tensor parallel	21	all 8K	31.0 GB a card
2× H100 80 GB tensor parallel	28	all 8K	78.1 GB a card
2× A100 80 GB tensor parallel	32	all 8K	78.2 GB a card

Memory needed at each load

Requests at once	8K tokens each
1	58.6 GB
5	71.0 GB
8	80.2 GB
16	105 GB
32	154 GB
64	253 GB

One card, with vLLM's small-card settings.

Builds

Sizes, precisions & builds

Build	Params	Precision	Weights	Smallest setup
gemma-2-27b (above) ↗	27.2B	BF16	54.5 GB	H100 80 GB
gemma-2-2b-it ↗	2.6B	BF16	5.2 GB	RTX 3060 12 GB

From the model card

What unsloth says about gemma-2

Read the model card

Reminder to use the dev version Transformers:

pip install git+https://github.com/huggingface/transformers.git

Finetune Gemma, Llama 3, Mistral 2-5x faster with 70% less memory via Unsloth!

Directly quantized 4bit model with bitsandbytes.

All notebooks are beginner friendly! Add your dataset, click "Run All", and you'll get a 2x faster finetuned model which can be exported to GGUF, vLLM or uploaded to Hugging Face.

* Kaggle has 2x T4s, but we use 1. Due to overhead, 1x T4 is 5x faster.

Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.

Details

Type	Language models
Task	Text gen
Context	8,192 tokens
Runs with	transformers
Popularity	43k downloads / month
Weights	54.5 GB, gemma-2-27b (BF16, file size)

Languages