A vector database is a specialized data system designed to store, index, and search high-dimensional vectors — dense numerical representations (embeddings) of data like text, images, audio, and code. Where a traditional database answers “find records where name = ‘Smith’”, a vector database answers “find the records most semantically similar to this query.” A search for “automobile maintenance” returns documents about “car repair,” “vehicle servicing,” and “engine diagnostics” — even though none of those results share keywords with the query. This semantic understanding is what makes vector databases the retrieval backbone for RAG systems, recommendation engines, and AI-powered search.
The vector database market has grown from a niche infrastructure category to a core component of the AI stack. Every RAG-based application — from enterprise knowledge bases to customer support chatbots to code search tools — relies on a vector database to bridge the gap between what users ask and what documents contain. As of 2026, the market includes over a dozen dedicated vector databases, and every major traditional database has added vector search capabilities.
How vector search works
Understanding vector databases requires grasping three concepts: embeddings, distance metrics, and approximate nearest neighbor (ANN) search.
Embeddings
An embedding model converts raw data — a sentence, a paragraph, an image — into a fixed-length array of floating-point numbers, typically 256 to 3,072 dimensions. These numbers encode semantic meaning: similar concepts produce vectors that are close together in high-dimensional space, while unrelated concepts produce distant vectors. The sentence “The cat sat on the mat” and “A kitten rested on the rug” would produce vectors that are very close to each other, despite sharing almost no exact words.
Modern embedding models have been trained on billions of text pairs to learn these semantic relationships. Leading models include OpenAI’s text-embedding-3-large (3,072 dimensions), Cohere’s embed-v4 (1,024 dimensions), Google’s text-embedding-005 (768 dimensions), and open-source models like BGE-M3 (1,024 dimensions, multilingual) and Nomic Embed (768 dimensions).
Distance metrics
Vector databases use mathematical distance functions to quantify how similar two vectors are:
| Metric | Formula concept | Best for | Range |
|---|---|---|---|
| Cosine similarity | Angle between vectors | Text similarity (most common) | -1 to 1 (1 = identical) |
| Euclidean (L2) distance | Straight-line distance | Image features, spatial data | 0 to infinity (0 = identical) |
| Dot product | Magnitude-weighted similarity | When vector magnitude carries meaning | -infinity to infinity |
| Manhattan (L1) distance | Grid-based distance | Sparse embeddings, certain ML features | 0 to infinity |
Cosine similarity is the default choice for text embeddings because it normalizes for vector magnitude, focusing purely on directional similarity. Most embedding models produce normalized vectors, making cosine similarity and dot product equivalent.
Approximate nearest neighbor (ANN) algorithms
The naive approach to finding the nearest vectors — computing the distance from the query to every stored vector — is computationally prohibitive at scale. Comparing a query vector to 100 million 1,024-dimensional vectors requires ~100 billion floating-point operations. ANN algorithms sacrifice a small amount of recall accuracy (typically retrieving 95-99% of the true nearest neighbors) for orders-of-magnitude speedups:
HNSW (Hierarchical Navigable Small World). The most widely adopted ANN algorithm. HNSW builds a multi-layer graph where each node is a vector and edges connect nearby vectors. Search starts at the top layer (sparse, long-range connections) and descends through denser layers, greedily following edges toward the nearest neighbor. HNSW achieves sub-millisecond query times on collections of tens of millions of vectors with 95-99% recall. Used by Pinecone, Weaviate, Qdrant, pgvector, and most other vector databases.
IVF (Inverted File Index). Clusters vectors into partitions (typically hundreds to thousands of clusters) using k-means. At query time, only the nearest clusters are searched, reducing the search space by 10-100x. IVF is often combined with Product Quantization (IVF-PQ) to compress vectors for faster distance computation. Used by FAISS (Meta), Milvus, and some cloud services.
ScaNN (Scalable Nearest Neighbors). Google’s algorithm that uses learned quantization and asymmetric hashing. Designed for billion-scale datasets with strong throughput-recall tradeoffs. Used in Google’s Vertex AI Vector Search.
DiskANN. Microsoft’s algorithm designed for datasets too large to fit in memory, using SSDs for storage with an in-memory navigation graph. Achieves competitive recall with a fraction of the memory requirements of HNSW, making it cost-effective at billion-vector scale. Used in Azure AI Search.
The vector database landscape
The market divides into three categories: dedicated vector databases, vector extensions to existing databases, and cloud-integrated vector services.
Dedicated vector databases
| Database | Architecture | Scale | Key differentiator |
|---|---|---|---|
| Pinecone | Managed cloud-native | Billions of vectors | Fully managed, serverless option, zero ops |
| Weaviate | Open-source, self-hosted or cloud | Hundreds of millions | Hybrid search (vector + BM25), built-in vectorizers |
| Qdrant | Open-source, Rust-based | Hundreds of millions | Performance, payload filtering, Rust memory safety |
| Chroma | Open-source, Python-native | Millions | Developer simplicity, in-process embedding |
| Milvus / Zilliz | Open-source / managed cloud | Billions of vectors | Scale, GPU-accelerated indexing |
| LanceDB | Open-source, serverless | Hundreds of millions | Zero-copy storage, Lance columnar format |
Vector extensions to existing databases
| Database | Vector feature | Max dimensions | Production readiness |
|---|---|---|---|
| PostgreSQL + pgvector | HNSW and IVF indexes | 2,000 | Production-ready for < 10M vectors |
| MongoDB Atlas Vector Search | Dedicated vector index | 4,096 | Production-ready, integrated with MongoDB |
| Elasticsearch | Dense vector fields, kNN | 4,096 | Mature, strong hybrid search |
| Redis + RediSearch | HNSW and FLAT indexes | Unlimited | Very fast, in-memory |
| SingleStore | Vector index type | 32,768 | SQL-native, combined analytics |
| Supabase + pgvector | Managed PostgreSQL | 2,000 | Developer-friendly, open-source |
Cloud-integrated vector services
| Service | Provider | Backend | Best for |
|---|---|---|---|
| Vertex AI Vector Search | Google Cloud | ScaNN | GCP-native workloads |
| Azure AI Search | Microsoft | DiskANN | Azure-native, enterprise |
| Amazon OpenSearch Serverless | AWS | FAISS-derived | AWS-native workloads |
| Bedrock Knowledge Bases | AWS | OpenSearch | RAG with AWS Bedrock models |
Choosing the right vector database
The choice depends on scale, existing infrastructure, and operational preferences. A practical decision framework:
Under 1 million vectors, using PostgreSQL already: Use pgvector. It avoids adding a new database to your stack, supports standard SQL queries, and handles this scale well. Build HNSW indexes for query performance. Supabase offers a managed PostgreSQL with pgvector pre-configured.
1-50 million vectors, want minimal ops: Use Pinecone (managed) or Qdrant Cloud. Both offer serverless options where you pay per query rather than managing infrastructure. Pinecone has the most mature managed offering; Qdrant offers better price-performance and the option to self-host.
50-500 million vectors, need hybrid search: Use Weaviate or Elasticsearch. Both excel at combining vector similarity with keyword matching and metadata filtering — critical for production search applications where pure semantic search produces too many irrelevant results.
Billion-scale vectors: Use Milvus/Zilliz or Google Vertex AI Vector Search. These systems are designed for billion-vector collections with GPU-accelerated indexing and distributed architectures. Expect higher operational complexity.
Tight integration with existing database: Use the vector extension for your current database. If you run MongoDB, use Atlas Vector Search. If you run Elasticsearch, use its kNN features. The quality-of-life benefits of not managing a separate system often outweigh the performance advantages of dedicated vector databases at moderate scale.
Vector databases in RAG architectures
Retrieval-augmented generation (RAG) is the primary use case driving vector database adoption. The architecture works as follows:
Indexing pipeline. Documents are split into chunks (typically 200-1,000 tokens each), each chunk is converted to an embedding vector by an embedding model, and the vectors are stored in the vector database alongside the original text and metadata (source, date, permissions, section headers).
Query pipeline. A user query is converted to an embedding using the same model, the vector database returns the k most similar chunks (typically k=5-20), and these chunks are injected into the LLM’s prompt as context for generating the answer.
The chunking problem. How documents are split into chunks has an outsized impact on retrieval quality. Too small, and individual chunks lack context. Too large, and relevant information is diluted by irrelevant text. Advanced chunking strategies include:
- Semantic chunking: Split at natural topic boundaries rather than fixed token counts, using embedding similarity between adjacent sentences to detect topic shifts.
- Hierarchical chunking: Store both large parent chunks (for context) and small child chunks (for precision), retrieving child chunks but providing parent chunks to the LLM.
- Sliding window: Overlapping chunks (e.g., 512 tokens with 128-token overlap) ensure that information spanning chunk boundaries is not lost.
Hybrid search for production RAG
Pure vector search has a well-documented failure mode: it struggles with queries containing specific identifiers — product codes, error messages, proper nouns, or technical terms — where exact keyword matching is essential. The solution is hybrid search, which combines:
- Vector similarity for semantic understanding (“how do I fix the database connection issue?”)
- BM25/TF-IDF keyword matching for precise term matching (“ERROR_CODE_4012”)
- Reciprocal Rank Fusion (RRF) or learned re-ranking to merge results from both systems
Production RAG systems from companies like Perplexity, Glean, and Datastax use hybrid search as the default retrieval strategy, typically seeing 10-20% recall improvements over pure vector search.
Advanced vector database features
Modern vector databases have evolved well beyond simple nearest-neighbor lookup:
Metadata filtering. Narrowing vector search by metadata fields (date ranges, document types, user permissions) before or during similarity computation. This is essential for multi-tenant applications where users should only see results from documents they have access to. Pre-filtering reduces the search space; post-filtering applies after vector search but can miss relevant results if the filter is restrictive.
Multi-tenancy. Serving multiple customers from a single database deployment while keeping data isolated. Approaches include namespace-based isolation (Pinecone), partition-based isolation (Milvus), and collection-based isolation (Qdrant). The choice affects cost, performance, and security guarantees.
Sparse-dense hybrid indexes. Storing both dense embeddings (from neural models) and sparse vectors (from BM25 or SPLADE) in the same index, enabling hybrid search without maintaining separate systems. Weaviate and Qdrant support this natively.
Re-ranking. A two-stage retrieval pipeline: the vector database returns a broad set of candidates (e.g., top 100), then a cross-encoder re-ranking model scores each candidate against the query with full attention, re-ordering the results for higher precision. Cohere Rerank and cross-encoder models from Sentence Transformers are widely used.
Real-time updates. Production applications need to insert, update, and delete vectors without rebuilding indexes. HNSW supports incremental insertions well, but deletions and updates require careful handling to maintain index quality. Some databases (Qdrant, Weaviate) support soft deletes with periodic compaction; others require periodic re-indexing.
Performance benchmarks and scaling characteristics
Vector database performance varies significantly by workload. Key metrics:
| Metric | Small scale (1M vectors) | Medium scale (100M vectors) | Large scale (1B vectors) |
|---|---|---|---|
| Query latency (p99) | 1-5 ms | 5-20 ms | 20-100 ms |
| Recall@10 | 98-99% | 95-98% | 92-97% |
| Indexing throughput | 10K-50K vectors/sec | 5K-20K vectors/sec | 1K-10K vectors/sec |
| Memory per vector (1024d) | ~4 KB (FP32) | ~4 KB (FP32) or ~1 KB (INT8) | ~0.5 KB (PQ compressed) |
| Total memory (FP32) | ~4 GB | ~400 GB | ~4 TB (requires compression) |
At billion-scale, compression is mandatory. Product Quantization (PQ) can reduce memory per vector from 4 KB to 64-256 bytes with 2-5% recall loss. Scalar quantization (INT8) offers a middle ground — 4x compression with under 1% recall loss. Binary quantization (Hamming distance) provides 32x compression but with 5-15% recall loss, useful for first-stage candidate retrieval before re-scoring with full-precision vectors.
The future of vector databases
Several trends are reshaping the vector database landscape:
Convergence with traditional databases. The line between vector databases and general-purpose databases is blurring. PostgreSQL with pgvector, SingleStore, and ClickHouse with vector indexes offer “good enough” vector search alongside full SQL capabilities. For many applications, a separate vector database may become unnecessary.
Agentic retrieval. Rather than simple top-k similarity search, AI agents are performing multi-step retrieval — querying the vector database, analyzing results, refining the query, and re-querying. This requires vector databases to support session-aware querying and iterative refinement.
Multimodal vector search. As embedding models become multimodal (encoding text, images, and audio into the same vector space), vector databases must handle heterogeneous data types. Searching a product catalog with an image query (visual similarity) combined with text filters (“red, under $50”) is becoming a standard requirement.
Learned indexes. Replacing hand-tuned ANN algorithms with neural networks that learn the data distribution and optimize search paths. Early research shows potential for higher recall at lower latency, particularly on skewed data distributions.
Frequently asked questions
Do I need a dedicated vector database, or can I use PostgreSQL with pgvector? For most applications with under 5-10 million vectors, pgvector is sufficient and avoids the operational complexity of a separate database. It supports HNSW indexes, achieves sub-10ms query latency at this scale, and integrates with standard SQL workflows. Choose a dedicated vector database when you need billion-scale collections, advanced features like multi-tenancy or hybrid search built in, or when vector search is your primary workload and you need maximum throughput.
What embedding model should I use with a vector database? For English text, OpenAI’s text-embedding-3-large (3,072 dimensions) and Cohere’s embed-v4 (1,024 dimensions) lead quality benchmarks. For multilingual text, BGE-M3 is the strongest open-source option. For cost-optimized applications, OpenAI’s text-embedding-3-small (1,536 dimensions) or Nomic Embed (768 dimensions) offer strong quality at lower dimensionality (fewer dimensions = lower storage and faster search). The embedding model matters more than the vector database choice for retrieval quality — a great embedding model with pgvector will outperform a weak embedding model with any dedicated vector database.
How should I chunk documents for a vector database? There is no universal answer, but a strong starting point is 400-800 token chunks with 100-200 token overlaps, using semantic boundaries (paragraph breaks, section headers) as preferred split points. Test retrieval quality empirically with your actual queries and documents. Hierarchical chunking — storing both small (256-token) and large (1,024-token) chunks with parent-child relationships — consistently outperforms fixed-size chunking in benchmarks and production systems.
What is the difference between a vector database and a traditional search engine? Traditional search engines (Elasticsearch, Solr) use inverted indexes to match keywords — they find documents containing the exact terms in your query. Vector databases use embeddings to match meaning — they find documents semantically similar to your query regardless of shared keywords. In practice, the best retrieval systems combine both approaches (hybrid search), using keyword matching for precision on specific terms and vector search for semantic understanding. Most modern search engines now support both modes.
How much does running a vector database cost? Costs vary enormously by scale. Pinecone’s serverless tier starts free for up to 2 GB. A dedicated Pinecone pod for ~1 million vectors costs $70-100/month. Self-hosted Qdrant or Weaviate on a modest cloud instance costs $50-200/month for similar scale. At 100M+ vectors, costs scale to $1,000-10,000+/month depending on performance requirements and redundancy. The largest cost driver is memory — vectors must be in RAM (or fast SSD) for low-latency queries, and high-dimensional vectors consume significant memory per record.