merged-mpnet

This is a Sentence Transformer model created by a 50/50 linear merge of all-mpnet-base-v2 and multi-qa-mpnet-base-cos-v1, using the SentenceTransformer.merge method from the sentence-transformers library. It maps sentences and paragraphs to 768-dimensional dense vectors for semantic search, clustering, and semantic similarity.

The merged model scores above the average of its two parents on 10 of the 13 NanoBEIR datasets, including 5 where it beats both. It achieves a mean nDCG@10 of 0.5823, compared with 0.5749 for all-mpnet-base-v2 and 0.5712 for multi-qa-mpnet-base-cos-v1, using the same inference settings for all three models.

On the nine datasets evaluated after candidate selection, the merge scores 0.5517, compared with 0.5422 and 0.5384 for its parents.

See Model Creation for the full generation script.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base models: all-mpnet-base-v2 and multi-qa-mpnet-base-cos-v1
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Language: English
• License: The all-mpnet-base-v2 model card specifies Apache-2.0. The multi-qa-mpnet-base-cos-v1 repository does not declare a license.

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Sentence Transformer Documentation
• Repository: Sentence Transformers on GitHub
• Parent model: all-mpnet-base-v2
• Parent model: multi-qa-mpnet-base-cos-v1

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'MPNetModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/merged-mpnet-base")
queries = ['Which planet is known as the Red Planet?']
documents = [
    "Venus is often called Earth's twin because of its similar size and proximity.",
    'Mars, known for its reddish appearance, is often referred to as the Red Planet.',
    'Saturn, famous for its rings, is sometimes mistaken for the Red Planet.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# (1, 768) (3, 768)

similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4703, 0.7748, 0.6875]])

Evaluation

Metrics

NanoBEIR

• Dataset: NanoBEIR-en
• Evaluated with NanoBEIREvaluator on all 13 datasets.
• Maximum input length: 512 tokens. Query and document prompts: empty.
• Similarity: cosine. FP32 inference with eager attention.
• Metric: nDCG@10. The mean is unweighted across datasets.

The parent models were evaluated with the same prompts and length limits.

Model Creation

The complete MPNet backbone was merged in fp32 with equal weights. Mean pooling and L2 normalization are shared by both parents. The maximum input length is 512 tokens, without any additional training.

This is all it took:

# pip install "sentence-transformers[merge]"
from sentence_transformers import SentenceTransformer

models = [
    SentenceTransformer("sentence-transformers/all-mpnet-base-v2"),
    SentenceTransformer("sentence-transformers/multi-qa-mpnet-base-cos-v1"),
]
for model in models:
    model.max_seq_length = 512
model = SentenceTransformer.merge(models, method="linear", parameters={"weight": [0.5, 0.5]}, dtype="float32")
model.similarity_fn_name = "cosine"
model.save_pretrained("merged-mpnet-base")

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 6.2.0
• Transformers: 5.4.0
• PyTorch: 2.11.0+cu128
• Mer