Manticore Revamps ONNX Path for 14x Faster Embeddings
embeddings
| Source: HN | Original article
Manticore's new ONNX path boosts embedding speed. Embeddings are now 14× faster on average.
Manticore Search has achieved a significant breakthrough in embedding speeds, with its new ONNX Runtime backend delivering results 14 times faster than the previous SentenceTransformers/Candle path. This improvement, released in Manticore Search 27.1.5, is substantial and holds steady across the same hardware, model, and weights.
The faster embedding capability matters because it enables more efficient and effective AI-powered search functionalities. With auto-embeddings now running at a much quicker pace, users can expect enhanced performance in search queries, which is crucial for applications relying on rapid and accurate information retrieval.
As Manticore continues to refine its search capabilities, the next steps to watch include how this enhanced embedding speed translates into real-world search performance and whether further optimizations are on the horizon. Given the recent focus on improving search functionalities, including the introduction of local auto-embeddings and updates to support better morphology for various languages, Manticore's developments are worth following for those interested in AI-driven search technologies.
Sources
Back to AIPULSEN