Introducing the Ettin Reranker Family
AI Summary
Six new state-of-the-art Sentence Transformers CrossEncoder rerankers have been released, built on top of Ettin ModernBERT encoders and trained via distillation using mixedbread-ai/mxbai-rerank-large-v2 scores. Rerankers jointly encode query-document pairs to provide higher relevance scores than standalone embedding models, making them ideal for the second stage of retrieve-then-rerank pipelines. By leveraging a modular Transformer design that supports sequence unpadding and Flash Attention 2, these models achieve significant speedups and efficiency gains over traditional padded architectures.
Key Takeaways
- Built on Johns Hopkins University's Ettin ModernBERT encoders, supporting up to 8192 tokens of context.
- The 17M model achieves a throughput of 7517 pairs per second on an NVIDIA H100, nearly double the speed of traditional MiniLM models.
- Utilizes a modular Transformer design to unpad variable-length inputs natively, bypassing padding overhead during Flash Attention 2 execution.
Why it matters
For engineers building search and retrieval-augmented generation (RAG) pipelines, these models offer an immediate drop-in replacement that improves both latency and accuracy. The 17M parameter variant outperforms older MiniLM baselines while running significantly faster, and the long-context support (up to 8K tokens) handles extensive documents effectively.
Conversation 0
Be the first to share your perspective.