Echo-DSRN.3-Embed-Exp

[!WARNING] This repository contains experimental models designed strictly for academic evaluation and research purposes.

Critical Constraints:

• No Production Deployment: Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances.
• No Liability: Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.

This is a high-performance experimental sentence embedding model based on the recurrent-hybrid Echo-DSRN architecture.

It scales linearly ($O(N)$) with sequence length, offering extreme efficiency and sub-millisecond latency on both CPU and GPU.

🚀 Model Details

• Developer: ethicalabs
• Base Model: ethicalabs/Echo-DSRN-114M-v0.1.2
• Architecture: Echo-DSRN Recurrent-Hybrid
• Parameters: 98.26M (98,264,064)
• Embedding Dimension: 2048

📊 Evaluation Results (MTEB STS)

Spearman Rank Correlation scores on Semantic Textual Similarity (STS) benchmark tasks:

⚡ Efficiency and Systems Scaling Profile

Inference performance (latency and peak VRAM allocation) on GPU and CPU configurations across different sequence lengths:

GPU Latency & VRAM Benchmark

CPU Latency Benchmark

Note: 'N/A (OOR)' indicates sequence length exceeds model's maximum position embedding range.

🏗️ Architecture Details

📊 Parameter Breakdown

💻 Usage

You can load and use this model directly via sentence-transformers:

from sentence_transformers import SentenceTransformer

# Load model with auto-mapping enabled
model = SentenceTransformer("ethicalabs/Echo-DSRN-v0.1.3-Embed-Exp", trust_remote_code=True)

# Encode text to get 2048-dimensional embeddings
sentences = ["The recurrent slow state contains the aligned sequence representations.", "Echo-DSRN has linear complexity."]
embeddings = model.encode(sentences)
print(embeddings.shape) # (2, 2048)

🛠️ Training Procedure

The model was trained in three sequential phases:

1. Contrastive Pre-training: Representation space alignment using natural language inference datasets.
2. Fine-grained Similarity Tuning: Fine-tuning using semantic textual similarity benchmarks to calibrate similarity scores.
3. Multi-Task Generalization Tuning: Training on NLI retrieval and STS semantic similarity.

This model card was automatically generated by scripts/generate_model_card.py.

Note on tokenizer padding

The benchmark results on this card (STS table) were measured with left padding (padding_side: left), and this model version reproduces them under that convention. A right-padded training version is planned: right padding keeps padded-batch embeddings consistent with single-request embeddings (leading pad tokens do not pollute the recurrent state), so future checkpoints will be batch-composition independent.