The Faiss library
Douze, Matthijs, Guzhva, Alexandr, Deng, Chengqi, Johnson, Jeff, Szilvasy, Gergely, Mazaré, Pierre-Emmanuel, Lomeli, Maria, Hosseini, Lucas, Jégou, Hervé
–arXiv.org Artificial Intelligence
Vector databases manage large collections of embedding vectors. As AI applications are growing rapidly, so are the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper first describes the tradeoff space of vector search, then the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of the library and discuss a few selected applications to highlight its broad applicability.
arXiv.org Artificial Intelligence
Jan-16-2024
- Genre:
- Research Report (0.40)
- Technology:
- Information Technology
- Artificial Intelligence
- Cognitive Science (0.68)
- Machine Learning
- Neural Networks (0.67)
- Statistical Learning > Nearest Neighbor Methods (0.46)
- Natural Language (1.00)
- Representation & Reasoning > Search (1.00)
- Data Science > Data Mining (1.00)
- Information Management > Search (0.68)
- Software > Programming Languages (0.92)
- Artificial Intelligence
- Information Technology