Supra2.1-Ultra-001

Supra2.1-Ultra

Supra2.1-Ultra is the model that refers to the Supra2.1 family. It has 230M params, 40B tokens trained on, with LLaMa arch. It is a small base model that is probably going to generate incorrect or biased text, and it has neither been aligned nor evaluated for safety.

Config

Data used to train this model

HuggingFaceFW/fineweb-edu

mlfoundations/dclm-baseline-1.0

HuggingFaceTB/finemath

Hardware

This model was trained on a single A100 SXM.

Benchmarks

How to use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "SupraLabs/Supra2.1-Ultra"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16)

prompt = "The capital of France is"
inputs = tokenizer(prompt, return_tensors="pt")

output = model.generate(
    **inputs,
    max_new_tokens=50,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))

This code should download the model and generate a response.

License

This model is released under Apache-2.0 License.

Credits

thanks to:

Dedeprogames, for code to train this model.

SupraLabs Org © - 2026