embeddingmagibu

embeddingmagibu-200m is a Turkish-focused, multilingual sentence embedding model developed through cross-lingual tokenizer surgery, teacher-model cloning, and offline embedding distillation.

This model is released as part of the artifacts for the paper:

Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation M. Ali Bayram, Banu Diri, Savaş Yıldırım arXiv: https://arxiv.org/abs/2605.29992

The model produces 768-dimensional L2-normalized sentence embeddings and supports an 8,192-token context window, making it suitable for Turkish semantic search, retrieval-augmented generation, long-document representation, clustering, classification, and semantic textual similarity tasks.

Although the model is optimized for Turkish, it was adapted from a multilingual teacher and trained with a multilingual distillation setup, making it useful for Turkish-centered multilingual NLP scenarios.

Key Features

• Turkish-focused sentence embedding model
• Multilingual support through a 40-language adaptation setup
• 8,192-token context length
• 768-dimensional normalized embeddings
• Approximately 200M parameters
• SentenceTransformers-compatible architecture
• Trained through offline embedding distillation
• Built with a Turkish-optimized multilingual tokenizer
• Designed for semantic search, RAG, clustering, classification, and STS

Associated Paper

This model is one of the released artifacts of the following paper:

Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation arXiv: https://arxiv.org/abs/2605.29992

The paper introduces an efficient adaptation pipeline for building Turkish-focused embedding models without expensive full pretraining. The pipeline consists of:

1. Constructing a Turkish-optimized multilingual tokenizer
2. Cloning the teacher embedding model while preserving transformer backbone weights
3. Distilling from precomputed teacher embeddings using an offline embedding dataset

Intended Use

This model is suitable for:

• Turkish semantic search
• Retrieval-augmented generation
• Long-context retrieval
• Semantic textual similarity
• Sentence and document embedding
• Clustering
• Classification using embeddings
• Multilingual retrieval with Turkish-centered applications
• Low-resource and morphologically rich language NLP research

Model Performance

1. Detailed TR-MTEB Results

The model was evaluated on Turkish embedding benchmark tasks. The table below summarizes the detailed task-level results.

2. Version Comparison: embeddingmagibu-200m vs embeddingmagibu-152m

The table below compares embeddingmagibu-200m with the previous embeddingmagibu-152m version on shared evaluation tasks.

Model Architecture

This model uses the SentenceTransformers format with the following pipeline. The max_seq_length value is set to 8192.

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: Gemma3TextModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Training and Adaptation Pipeline

The model was not trained from scratch. Instead, it was developed using an efficient three-stage adaptation pipeline.

1. Tokenizer Surgery

A Turkish-optimized multilingual tokenizer was constructed with a vocabulary size of 131,072 tokens.

The goal of this step was to improve Turkish tokenization efficiency while keeping multilingual capacity. This was done by pruning redundant tokens from the teacher voca