Supra

Supra-50M is a compact 50M-parameter BASE causal language model built by SupraLabs, trained from scratch using a Llama-style architecture on 20 billion tokens of high-quality educational web text. Despite being significantly smaller than comparable open models, it achieves competitive or superior results on several key benchmarks. It's our first SupraLabs Scaling Up Plan model.

🏆 Benchmarks

🧠 Model Architecture & Hyperparameters

Supra-50M is based on the LlamaForCausalLM architecture with the following configuration:

📚 Training Data

🔤 Tokenizer

A custom Byte-Level BPE tokenizer was trained from scratch on 500,000 documents sampled from fineweb-edu (sample-10BT).

⚙️ Training Configuration

🚀 Inference

from transformers import pipeline
import torch

print("[*] Loading Supra-50M model from Hugging Face Hub...")
pipe = pipeline(
    "text-generation",
    model="SupraLabs/Supra-50M_BASE",
    device_map="auto",
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32
)

def generate_text(prompt, max_new_tokens=150):
    result = pipe(
        prompt,
        max_new_tokens=max_new_tokens,
        do_sample=True,
        temperature=0.5,
        top_k=25,
        top_p=0.9,
        repetition_penalty=1.2,
        pad_token_id=pipe.tokenizer.pad_token_id,
        eos_token_id=pipe.tokenizer.eos_token_id
    )
    return result[0]['generated_text']

# Example
prompt = "The importance of education is"
print(f"\nPrompt: {prompt}")
print("-" * 40)
print("\nOutput:\n" + generate_text(prompt))

💬 Sample Outputs

Prompt: "The main concept of physics is "

The main concept of physics is iffy, and the idea that we can make things behave in a certain way. The most important part of physics is called quantum mechanics which states that all particles are made up of energy (energy) and matter (matter). In physics, there are two types of particles: elementary particles and exotic ones. These particles have properties like mass, speed or momentum but they don’t interact with each other to form new objects. This is because these particles do not exist independently from one another. In this case, an exotic particle might be created by adding more energy into its structure than it would take for a normal particle. However, when you add additional energy to an exotic particle, the new object will become smaller and larger until it becomes too large to fit within the existing structure. If you think about how light travels through space, it takes around 20 billion years before the light reaches our eyes. Light waves travel faster than light at high speeds so if we could create some kind of light wave, then we wouldn’t need any special equipment. It just needs a few hundred millionths of a second to produce light rays. So even though the light is moving along the same path as the current, the speed of light is different depending on where the light hits the

Prompt: "Artificial intelligence is "

Artificial intelligence is iffy, it can be used to make intelligent machines that could take over the world. What does Artificial Intelligence mean? AI refers to ar