Tarka-Embedding

Introduction

Tarka-Embedding-150M-V1 is a 150M parameter embedding model designed to produce 768-dimensional dense text representations. It is optimized for a wide range of downstream applications such as semantic similarity, search, and retrieval-augmented generation (RAG). The model focuses on capturing deep contextual semantics to support general-purpose text understanding across diverse domains.

The model is trained using Data-Free Knowledge Distillation (DFKD). To prepare the training data, standard open-source datasets were used only as a source of raw textual content — all labels, annotations, and structural elements were stripped to create a plain, unlabeled text corpus. The resulting dataset contained approximately 2 billion tokens, all of which were used during model training.

Find more information about Tarka-Embedding-150M-V1 in our blog post.

[!NOTE] 🚀 Try our demo: https://huggingface.co/spaces/Tarka-AIR/Tarka-Embedding

Model Details

Tarka-Embedding-150M-V1 has the following features:

• Model Type: Text Embedding
• Supported Languages: English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish.
• Number of Paramaters: 150M
• Context Length: 2048
• Embedding Dimension: 768

Currently, only the MTEB (English) benchmark has been evaluated. Additional multilingual benchmark results will be released in future updates.

Training Details

• Initialization: Based on google/embeddinggemma-300m and LiquidAI/LFM2-350M
• Architecture Modifications: The tokenizer and embedding weights from the EmbeddingGemma model were replaced with the LFM2-350M tokenizer. The embedding layer and several hidden layers were reinitialized to align with the modified architecture, resulting in a dimensional change from [262,144, 768] → [64,400, 768], effectively reducing the total parameter count by approximately 50%.
• Training Data: 2 billion tokens curated from multiple open source datasets.
• Teacher Model: google/embeddinggemma-300m
• Compute Resources: 40 GPU hours on NVIDIA A100

Citation

@misc{tarka_ai_research_2025,
	author       = { Tarka AI Research },
	title        = { Tarka-Embedding-150M-V1 (Revision c5f4f43) },
	year         = 2025,
	url          = { https://huggingface.co/Tarka-AIR/Tarka-Embedding-150M-V1 },
	doi          = { 10.57967/hf/6875 },
	publisher    = { Hugging Face }
}

MTEB (Eng v2)

Sentence Transformers Usage

# Requires transformers>=4.51.0
# Requires sentence-transformers>=2.7.0

from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer("Tarka-AIR/Tarka-Embedding-150M-V1")

# The queries and documents to embed
queries = [
    "What is the capital of China?",
    "Explain gravity",
]
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun.",
]

query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# (2, 768) (2, 768)

# Compute similarities to determine a ranking
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7830, 0.2929],[0.3390, 0.6536]])

Prompt Instructions

EmbeddingGemma can generate optimized embeddings for various use cases—such as document retrieval, question answering, and fact verification—or for specific input types—either a query or a document—using prompts that are prepended to the input strings. Query prompts follow the form task: {task description} | query: where the task description varies by the use case, with the default task description being search result. Document-style prompts follow the form title: {title | "none"} | text: where the title is either none (the default) or the actual t