Principle:FlagOpen FlagEmbedding Embedding Similarity Computation
| Field | Value |
|---|---|
| Sources | Paper: BGE M3 |
| Domains | Information_Retrieval, NLP |
Overview
A set of methods for measuring relevance between query and passage embeddings using dot product, sparse lexical matching, and ColBERT token-level interaction.
Description
After encoding texts to vectors, similarity must be computed. Three methods exist:
(1) Dense similarity via matrix multiplication (embeddings_q @ embeddings_p.T), producing cosine similarity when embeddings are normalized.
(2) Sparse lexical matching via compute_lexical_matching_score() which sums shared token weights. For each token present in both the query and passage lexical weight dictionaries, the product of their weights is accumulated into a total score.
(3) ColBERT late interaction via MaxSim between query and passage token embeddings. Each query token embedding is compared against all passage token embeddings, the maximum similarity is taken, and these maxima are averaged over query tokens.
M3 models combine all three scoring methods with configurable weights, enabling flexible relevance scoring that leverages dense semantic matching, sparse lexical precision, and fine-grained token interaction simultaneously.
Usage
After encoding queries and passages, to compute relevance scores for ranking or filtering. The three methods can be used independently or combined via weighted summation depending on the use case and accuracy requirements.
Theoretical Basis
Dense: cosine similarity = dot product of L2-normalized vectors. When embeddings are normalized to unit length, the dot product equals cosine similarity, bounded in [-1, 1].
Sparse: BM25-like lexical matching using learned token weights. The score is computed as:
sparse_score = sum(w_q[token] * w_p[token] for token in shared_tokens)
ColBERT: MaxSim(q_i, p_j) aggregated over query tokens. For each query token i, find the maximum dot product with any passage token j, then average:
colbert_score = (1 / |Q|) * sum_i(max_j(q_i . p_j))
Combined: weighted score combining all three methods:
weighted_score = (w_dense * dense + w_sparse * sparse + w_colbert * colbert) / (w_dense + w_sparse + w_colbert)
Default weights are [1.0, 1.0, 1.0] when not specified.