agentlans/multilingual-MiniLM-L12-v2-harrier

Multilingual MiniLM-L12-v2 (Harrier-Distilled)

This model is a high-performance, lightweight sentence encoder aligned via embedding distillation to a top-performing cross-lingual teacher model from the MTEB leaderboard. Designed for high-throughput multilingual tasks, it delivers strong semantic retrieval capabilities while maintaining the speed and efficiency of a compact student architecture.

Model Details

Parameter	Value
Student Backbone	sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Teacher Model	microsoft/harrier-oss-v1-270m
Best Epoch	10
Validation Cosine Similarity	0.9377
Best Validation MSE	0.000195
Learning Rate	2e-05
Batch Size	16
Max Sequence Length	256
License	Apache 2.0

Training results

Validation History

Epoch	Train Loss	Val MSE	Val Cosine Similarity
1	0.000493	0.000277	0.9114
2	0.000249	0.000234	0.9250
3	0.000224	0.000220	0.9297
4	0.000212	0.000211	0.9325
5	0.000203	0.000204	0.9346
6	0.000197	0.000200	0.9361
7	0.000193	0.000197	0.9370
8	0.000190	0.000195	0.9374
9	0.000188	0.000195	0.9377
10	0.000187	0.000195	0.9377 (Best)

Installation

Because this model couples a transformer backbone with a trained projection layer, it requires the custom helper package:

pip install git+https://github.com/agentlans/proj-embed.git

Quick Start

from proj_embed import ProjEmbed

# Initialize the model (loads backbone and projection weights automatically)
model = ProjEmbed("agentlans/multilingual-MiniLM-L12-v2-harrier")

# Generate L2-normalized embeddings for a single text or a batch
embeddings = model.embed(["Hello world", "Greetings from ProjEmbed"])
print(embeddings.shape)  # torch.Size([2, 640])

# Compute cosine similarity between two texts
similarity_score = model.similarity("Machine learning is fascinating.", "AI and deep learning are amazing.")
print(f"Similarity: {similarity_score:.4f}") # Similarity: 0.7600

Intended Use

• Cross-Lingual Retrieval: Fast semantic search across multiple languages with reduced compute overhead.
• Dataset Filtering: Efficient processing, cleaning, and filtering of large-scale translation and parallel corpora.
• Semantic Similarity: Rapid text clustering, deduplication, and ranking.

Limitations

• Dimension & Architecture Mismatch: Does not share the same hidden dimensions or architecture as the larger teacher model.
• Multilingual Nuance: Performance may vary on low-resource dialects or highly domain-specific terminology compared to major-language benchmarks.

License

Apache 2.0 like the base model