Classic relational databases are designed to find exact values: a customer number, a date, a name. AI applications often need something else, namely the entries whose content is most similar to a query. This is exactly what vector databases are built for. They play a central role when companies make their documents searchable for AI assistants.
How does a vector database work?
Each entry consists of a vector, the embedding, as well as additional information such as the original text, source, date, or permission group. During a search, the query is converted into a vector with the same model. The database then determines the vectors that lie closest to it.
With millions of entries, comparing against every single vector would be too slow. Vector databases therefore use special indexing methods for approximate nearest neighbor search. These deliver results in a very short time that come very close to the exact matches. A widespread method is HNSW, which organizes the vectors in a multilayer graph.
Many systems combine vector search with classic keyword search and with filters on the additional information. Such a hybrid search often delivers better results, for example when exact terms such as item numbers play a role.
Vector search is available as standalone database systems, as an extension of existing databases such as PostgreSQL, and as a feature in search and cloud services.
Typical examples
- Knowledge base for retrieval-augmented generation, for example with policies and manuals
- Searching for similar cases in a ticketing system
- Product search that also understands paraphrased queries
- Detecting similar or duplicate documents
What companies should keep in mind
- Check existing systems: Not every use case needs a dedicated database system. For smaller datasets, an extension of the existing database is often sufficient.
- Map permissions: Access rights of the original sources must be carried along as filters and applied with every search.
- Synchronize deletions: If a document is deleted or changed, the entry in the vector database must follow. This is also relevant for deletion obligations under the GDPR.
- Define the storage location: With cloud services, pay attention to data residency and contract terms.
- Plan for recovery: Although the index can be rebuilt from the sources, this can take time and incur costs for large datasets. A backup can therefore make sense.