Embeddings are dense numerical vectors — typically arrays of 256 to 3,072 floating-point numbers — that encode the semantic meaning of text, images, audio, or other data. The key property is that similar meanings produce similar vectors: the embeddings for “how to fix a leaking faucet” and “repair a dripping tap” will be close together in vector space, while both will be far from “quarterly earnings report.” This property transforms the fuzzy concept of “meaning” into a precise mathematical operation — measuring similarity becomes computing a distance between two points.
Embeddings are the invisible infrastructure behind semantic search, retrieval-augmented generation (RAG), recommendation systems, clustering, anomaly detection, and classification. Every time a search engine returns results based on meaning rather than keywords, every time a chatbot retrieves relevant documents to ground its answer, and every time a recommendation system surfaces “similar” items — embeddings are doing the work.
How embeddings work
An embedding model is a neural network — almost always based on the Transformer architecture — that takes an input (a sentence, paragraph, image, or audio clip) and produces a fixed-length vector. The model is trained so that inputs with similar meanings produce vectors that are close together, and inputs with different meanings produce vectors that are far apart.
The training process
Modern text embedding models are trained in two or three stages:
Stage 1: Pre-training. The base model (typically a BERT-style encoder or a decoder model) is pre-trained on a massive text corpus using standard language modeling objectives. This gives the model general language understanding.
Stage 2: Contrastive learning. The model is fine-tuned on pairs of texts labeled as similar or dissimilar. The training objective pushes similar pairs closer together in vector space and dissimilar pairs further apart. This is typically done with a contrastive loss function like InfoNCE. Large-scale training datasets include MS MARCO (passage retrieval pairs), Natural Questions (question-passage pairs), and synthetically generated pairs from large language models.
Stage 3: Task-specific fine-tuning (optional). For specialized domains (legal, medical, financial), the model can be further trained on domain-specific similarity data. This is particularly effective when general-purpose embeddings underperform on specialized vocabulary and concepts.
Measuring similarity
Two vectors can be compared using several distance metrics:
| Metric | Formula intuition | When to use |
|---|---|---|
| Cosine similarity | Angle between vectors (ignoring magnitude) | Most common default; works well for normalized embeddings |
| Dot product | Magnitude-weighted similarity | When vector length encodes importance or confidence |
| Euclidean distance | Straight-line distance between points | When absolute positioning matters; less common for text |
| Manhattan distance | Sum of absolute differences per dimension | Faster to compute; used in some approximate search implementations |
Cosine similarity is the industry default for text embeddings. A cosine similarity of 1.0 means the vectors point in exactly the same direction (identical meaning), 0 means they are orthogonal (unrelated), and -1 means they point in opposite directions (opposite meaning, though this is rare in practice).
What the dimensions represent
Each dimension in an embedding vector does not correspond to a human-interpretable concept like “formality” or “topic.” Instead, the model learns abstract features distributed across all dimensions. However, research has shown that embedding spaces have geometric structure: directions in the space correspond to semantic relationships. The classic example is that the vector from “king” to “queen” is approximately parallel to the vector from “man” to “woman” — the model has learned a “gender” direction without being explicitly taught one.
The embedding model landscape (2026)
The embedding model market has matured into a competitive ecosystem with clear tiers:
Proprietary embedding APIs
| Model | Provider | Dimensions | Max tokens | MTEB score (avg) | Price per 1M tokens |
|---|---|---|---|---|---|
| text-embedding-3-large | OpenAI | 3,072 | 8,191 | ~64.6 | $0.13 |
| text-embedding-3-small | OpenAI | 1,536 | 8,191 | ~62.3 | $0.02 |
| text-embedding-005 | 768 | 2,048 | ~66.0 | $0.00005 (batch) | |
| Embed v4 | Cohere | 1,024 | 512 | ~65.0 | $0.10 |
| Voyage 3 Large | Voyage AI | 1,024 | 32,000 | ~67.0 | $0.18 |
Open-source embedding models
| Model | Developer | Dimensions | MTEB score (avg) | Key strength |
|---|---|---|---|---|
| BGE-M3 | BAAI (Beijing) | 1,024 | ~65.0 | Multilingual, multi-granularity |
| E5-Mistral-7B | Microsoft | 4,096 | ~66.6 | Large model, high quality |
| GTE-Qwen2 | Alibaba | 1,536 | ~65.4 | Strong multilingual performance |
| NomicEmbed v2 | Nomic AI | 768 | ~63.5 | Long context (8,192 tokens), open-source |
| Snowflake Arctic Embed L | Snowflake | 1,024 | ~64.8 | Retrieval-focused optimization |
The MTEB (Massive Text Embedding Benchmark) leaderboard, hosted on Hugging Face, is the standard evaluation for embedding models. It evaluates across retrieval, classification, clustering, pair classification, reranking, semantic textual similarity, and summarization tasks. Scores range from roughly 55 (basic models) to 68+ (frontier models).
Embeddings in production: the RAG pipeline
The single largest use case for embeddings is powering the retrieval step in RAG systems. The pipeline works as follows:
Indexing phase (offline):
- Split source documents into chunks (typically 256-1,024 tokens per chunk)
- Generate an embedding vector for each chunk using an embedding model
- Store the vectors in a vector database alongside the original text
Query phase (online):
- Embed the user’s query using the same embedding model
- Search the vector database for the K most similar chunk embeddings
- Pass the retrieved chunks as context to the language model
- The model generates an answer grounded in the retrieved content
The quality of embeddings directly determines the quality of retrieval, which in turn determines the quality of the generated answer. A RAG system with excellent embeddings and a mediocre language model will often outperform a system with poor embeddings and a frontier language model — retrieval is the bottleneck, not generation.
Chunking strategies
How documents are split into chunks significantly affects embedding quality:
| Strategy | Description | Best for |
|---|---|---|
| Fixed-size | Split every N tokens | Simple, predictable; works for uniform documents |
| Sentence-based | Split on sentence boundaries | General-purpose text retrieval |
| Paragraph-based | Split on paragraph boundaries | Documents with clear structural hierarchy |
| Semantic chunking | Use embedding similarity to detect topic boundaries | Long documents with flowing content |
| Hierarchical | Store chunks at multiple granularities (sentence, paragraph, section) | Complex documents requiring both detail and context |
| Document-aware | Respect document structure (headings, sections, tables) | Structured documents like legal filings, technical docs |
In practice, overlap between chunks (typically 10-20% of chunk size) improves retrieval by ensuring that information split across a boundary is captured in at least one chunk.
Multimodal embeddings
Embedding models are not limited to text. Multimodal embeddings map different data types into the same vector space, enabling cross-modal retrieval:
CLIP and SigLIP (OpenAI, Google) map text and images into a shared embedding space. A text query like “a golden retriever playing in snow” can retrieve relevant images from a database of image embeddings, and vice versa. CLIP is the foundation of image search in products like Google Photos, Pinterest, and Shutterstock.
ImageBind (Meta) extends the shared embedding space to six modalities: text, images, audio, video, depth, and thermal data. A query in any modality can retrieve results in any other modality.
CLAP (Microsoft) maps text and audio into a shared space, enabling text-based audio search: “sound of rain on a tin roof” retrieves the corresponding audio clip.
These models are trained on paired data (text-image pairs, text-audio pairs) using contrastive learning — the same fundamental approach as text embedding models, applied across modalities.
Advanced embedding techniques
Several techniques extend the basic embedding paradigm for production use:
Matryoshka Representation Learning (MRL)
Introduced by Microsoft Research in 2022, MRL trains embedding models so that the first N dimensions of the full embedding are themselves a valid, lower-dimensional embedding. A 1,024-dimensional Matryoshka embedding can be truncated to 512, 256, or even 64 dimensions with graceful quality degradation rather than catastrophic failure. This gives developers fine-grained control over the storage-quality tradeoff:
| Dimensions | Relative quality (retrieval) | Storage per vector |
|---|---|---|
| 1,024 (full) | 100% | 4 KB |
| 512 | ~98% | 2 KB |
| 256 | ~95% | 1 KB |
| 64 | ~85% | 256 bytes |
OpenAI’s text-embedding-3 models support Matryoshka truncation natively. For applications with millions or billions of vectors, the storage and compute savings at 256 dimensions are substantial.
Late interaction (ColBERT)
Rather than compressing an entire document into a single vector, ColBERT produces one embedding per token and computes similarity by matching query tokens against document tokens at retrieval time. This preserves more fine-grained information and improves retrieval quality by 5-15% on benchmarks, but requires more storage and computation. RAGatouille provides a popular open-source ColBERT implementation for RAG applications.
Hybrid search
Combining embedding-based semantic search with traditional keyword search (BM25) using reciprocal rank fusion (RRF) outperforms either approach alone by 5-20% on retrieval benchmarks. Keyword search catches exact matches that semantic search may miss (product codes, abbreviations, proper nouns), while semantic search captures meaning that keyword search misses. Most production RAG systems use hybrid search.
Reranking
A two-stage retrieval pipeline first retrieves a large candidate set (50-200 documents) using fast embedding search, then reranks the candidates using a more expensive cross-encoder model that jointly processes the query and each candidate. Reranking models like Cohere Rerank, Jina Reranker, and BGE-Reranker improve retrieval precision by 10-25% over embedding-only retrieval.
The economics of embeddings at scale
Embedding costs are dominated by three factors: generation, storage, and search.
Generation costs. Embedding a million tokens costs $0.02-$0.18 via commercial APIs, or effectively free for self-hosted open-source models (just GPU time). A typical enterprise knowledge base of 10 million chunks (~5 billion tokens) costs $100-$900 to embed using commercial APIs — a one-time cost since embeddings are generated once and stored.
Storage costs. Each 1,024-dimensional float32 embedding requires 4 KB. At scale:
| Documents | Chunks (est.) | Storage (1,024-dim) | Storage (256-dim, MRL) |
|---|---|---|---|
| 10,000 | 100,000 | 400 MB | 100 MB |
| 1,000,000 | 10,000,000 | 40 GB | 10 GB |
| 100,000,000 | 1,000,000,000 | 4 TB | 1 TB |
Search costs. Exact nearest-neighbor search over millions of vectors is computationally expensive. Approximate nearest-neighbor (ANN) algorithms — HNSW, IVF, and ScaNN — trade a small amount of recall (typically 95-99% of exact search quality) for orders-of-magnitude speedups. A well-tuned HNSW index can search 100 million vectors in under 10 milliseconds.
Vector databases like Pinecone, Weaviate, Qdrant, Milvus, and pgvector handle this infrastructure, providing managed indexing, search, and scaling. Pinecone is the most popular managed option; pgvector is the most popular self-hosted option, running as an extension to PostgreSQL.
Frequently asked questions
What is the difference between embeddings and tokens? Tokens are the subword units that language models process — the raw pieces of text. Embeddings are the numerical vectors that represent those tokens (or groups of tokens) in a way that captures semantic meaning. A token is a symbol; an embedding is its mathematical representation. Token embeddings exist inside language models as the first layer, but the term “embeddings” in production contexts usually refers to the output of a dedicated embedding model that encodes an entire sentence or passage into a single vector.
How do I choose the right embedding model? Start with the MTEB leaderboard to compare models on tasks similar to yours. Key decision factors: retrieval quality on your domain, dimensionality (affects storage and speed), max input length, multilingual requirements, and cost. For most English-language RAG applications, OpenAI’s text-embedding-3-large or an open-source model like BGE-M3 are strong defaults. For specialized domains, fine-tuning an open-source embedding model on domain-specific data often outperforms the best general-purpose models.
Do I need a vector database, or can I use a regular database? For fewer than 100,000 vectors, you can compute similarities in memory or use pgvector as a PostgreSQL extension — no dedicated vector database needed. Above 1 million vectors, a dedicated vector database (Pinecone, Weaviate, Qdrant) or a purpose-built ANN index provides dramatically better search performance. The crossover point depends on your latency requirements and query volume.
How often should I re-embed my documents? Re-embed when: (1) you switch to a better embedding model, (2) your source documents are updated, or (3) you fine-tune your embedding model on domain-specific data. Embeddings from unchanged documents using the same model never go stale — the vector is deterministic. Most organizations re-embed their full corpus every 3-6 months as embedding models improve, and incrementally embed new or updated documents as they arrive.
What is the difference between semantic search and keyword search? Keyword search (BM25, TF-IDF) matches exact words and treats each word independently. Semantic search uses embeddings to match meaning — it understands that “affordable housing” and “low-cost apartments” are related even though they share no words. Semantic search handles synonyms, paraphrasing, and conceptual queries far better. Keyword search is better for exact matches (product IDs, technical terms, proper nouns). The best production systems use hybrid search that combines both approaches.