Vemana

Try it: on your device, in the browser · on Hugging Face Spaces · GGUF for llama.cpp · every checkpoint · code on GitHub · udgamlabs.com/models/vemana · technical report

Quick start

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("udgamlabs/Vemana-Base")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Vemana-Base")
inputs = tok("తెలుగు భాష", return_tensors="pt")   # the tokenizer adds
out = model.generate(**inputs, max_new_tokens=80, do_sample=True, temperature=0.8, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))

Needs transformers 5.19 or newer (Gemma 4 text layout). Keep prompt and output within 1,024 tokens, the length it was trained at (about 570 Telugu words). On a Mac mini's CPU it writes about 65 tokens per second.

What it writes (sampling as above; English translations are ours):

వర్షాకాలంలో సీజనల్ వ్యాధులు ఎక్కువగా వస్తాయి. అందువల్ల ఆరోగ్యంగా ఉండటం చాలా ముఖ్యం. మీరు వర్షాకాలంలో డెంగ్యూ, మలేరియా, టైఫాయిడ్, వైరల్ ఇన్ఫెక్షన్ల వంటి సీజనల్ వ్యాధుల బారిన పడే ప్రమాదం ఉంది. …

In the rainy season, seasonal illnesses are more common, so staying healthy matters a lot. In the rainy season you are at risk of seasonal illnesses such as dengue, malaria, typhoid and viral infections. …

కంప్యూటర్ అనేది మనకు అత్యంత అవసరమైన వస్తువులలో ఒకటి. కంప్యూటర్లు మనం చేసే చాలా పనులను చేస్తాయి, … ఈ సాధనాలలో ఒకటి CPU. …

A computer is one of the things we need most. Computers do many of the tasks we do, … One of these tools is the CPU. …

Results

Fresh Telugu text. Bits per byte (lower is better) on Telugu Wikipedia pages created after the compared models were released. One scoring script, the same pages and float32 for every model; window 1,024 tokens (Goldfish 512, its limit); measured by us on 27 October 2026.

• Vemana minus nanochat-telugu-560M (paired page bootstrap, 95% interval): −0.022 [−0.024, −0.021] on the Sep-Oct pages, −0.023 [−0.024, −0.022] on Jul-Aug, −0.018 [−0.021, −0.015] on 23-26 Oct. It holds at a 512-token window (−0.017) and without the editor who wrote a third of the Sep-Oct pages (−0.023).
• We fixed the test, the win margin (0.02 bits per byte) and the robustness checks before training, and chose the final checkpoint in advance.

Multiple-choice tests (lm-eval 0.4.13, 0-shot): Belebele-te 25.1%, MILU-te 25.3% (cloze form 30.2%), IndicXNLI-te 35.0%, IndicCOPA-te 58.4%. Chance is 25, 25, 33 and 50% (about 55% on IndicCOPA from answer length alone).

Where it falls short

• Larger models are better. Sarvam-1 (2.5B parameters, about 9× larger) is level with Vemana on the Sep-Oct pages (0.356 against 0.361) and better on the Jul-Aug pages (0.365 against 0.385).
• Knowledge and reasoning tests are at chance, as for most open models under 1B we tested; Qwen3.5-0.8B does better on the letter forms of Belebele (35.6%) and MILU (31.1%). They need a bigger model or more data.
• It makes things up. Asked to continue "తెలుగు భాష" (the Telugu language), it wrote fluently that Telugu Language Day is celebrated on 21 September; it is 29 August.
• It loses the thread. A short story prompt ("ఒక రోజు ఒక రైతు", one day a farmer) turns into the same sentence about a farmer and a cow, repeated with small changes.
• Base model only. It continues text; it does not answer questions or follow requests, and it has no safety tuning.
• 1,024-token context.

How it was trained

• Data: public Telugu web text from three open datasets: Sangraha (AI4Bharat, its human-verified Telugu part), FineWeb-2 (Hugging Face) and HPLT 3.0. Filtered for Telugu script and length, near-duplicates removed across the three, and the Sep-Oct test pages and the multiple-choice test sets removed from the training text. The Jul-Aug and late-October test pages were written in 2026, after the training text was collected (web crawls up to 2024). 5.1B unique tokens.
• Run: 12.0B token positions on one H100 in about 16 hours, about $69 of GPU time, live in public at udgamlabs.com/live/vemana.
• Architecture: Gemma 4 text layout, 18 layers, width 1,024, 8 query and 2 key-value heads, full attention, tied embeddings, our own 32,000-token Telugu tokenizer (byte fallback, so any t