Implementation:Lance format Lance VectorThroughputBench

Knowledge Sources	Lance
Domains	Benchmarking, Performance
Last Updated	2026-02-08 19:33 GMT

Overview

Description

VectorThroughputBench is a Criterion-based benchmark that measures concurrent IVF_PQ vector search throughput on Lance datasets. It is the Rust equivalent of the Python test_ivf_pq_throughput benchmark and evaluates how well Lance handles multiple simultaneous KNN queries against a large indexed dataset.

The benchmark:

1. Creates (or reuses) a dataset with 1,000,000 rows of 768-dimensional random float vectors.
2. Builds an IVF_PQ index with 256 partitions, 48 sub-vectors, and 8-bit quantization.
3. Runs 100 concurrent vector search queries with K=50, nprobes=20, and refine_factor=10.
4. Measures throughput across different concurrency levels (1, 2, 4, 8, 16 threads).

The benchmark tests both cached and uncached scenarios. In the uncached case (Linux only), it uses posix_fadvise(POSIX_FADV_DONTNEED) to drop dataset files from the OS page cache before each iteration, simulating cold-start conditions. Both V2_0 and V2_1 Lance file format versions are tested.

Datasets are persisted to disk at /tmp/lance_bench_throughput_* and reused across benchmark runs to avoid costly recreation.

Usage

This benchmark is used to evaluate vector search scalability under concurrent load, which is critical for serving applications that perform many simultaneous similarity queries. It helps developers optimize the IVF_PQ search pipeline, I/O scheduling, and caching behavior.

Code Reference

Source Location

rust/lance/benches/vector_throughput.rs (355 lines)

Signature

The benchmark defines a single benchmark target:

fn bench_ivf_pq_throughput(c: &mut Criterion)

Supporting structures and functions:

struct CachedDataset {
    dataset: Arc<Dataset>,
    query_vectors: Vec<Arc<Float32Array>>,
}

fn get_or_create_dataset(rt: &Runtime, version: LanceFileVersion) -> Arc<CachedDataset>
async fn create_dataset(uri: &str)
async fn create_ivf_pq_index(dataset: &mut Dataset)
fn generate_query_vectors() -> Vec<Arc<Float32Array>>
async fn run_queries(dataset: Arc<Dataset>, queries: &[Arc<Float32Array>], concurrent: usize)

Import

use lance::dataset::{Dataset, WriteMode, WriteParams};
use lance::index::vector::VectorIndexParams;
use lance_index::vector::{ivf::IvfBuildParams, pq::PQBuildParams};
use lance_index::{DatasetIndexExt, IndexType};
use lance_linalg::distance::MetricType;
use lance_testing::datagen::generate_random_array;
use lance_file::version::LanceFileVersion;
use criterion::{criterion_group, criterion_main, BatchSize, Criterion, Throughput};

I/O Contract

Inputs

Parameter	Type	Value	Description
NUM_ROWS	Constant	1,000,000	Total rows in the dataset
DIM	Constant	768	Vector dimension
NUM_QUERIES	Constant	100	Number of query vectors per iteration
K	Constant	50	Number of nearest neighbors to return
NPROBES	Constant	20	Number of IVF partitions to probe
REFINE_FACTOR	Constant	10	Factor for refining results after PQ approximation
IVF_PARTITIONS	Constant	256	Number of IVF partitions in the index
PQ_SUB_VECTORS	Constant	48	Number of PQ sub-vectors (DIM / 16)
Concurrency levels	Variable	1, 2, 4, 8, 16	Number of concurrent query threads
File versions	LanceFileVersion	V2_0, V2_1	Lance file format versions

Outputs

Output	Type	Description
Criterion report	HTML/JSON	Throughput statistics (queries/sec) for each combination of file version, concurrency level, and cache state
Dataset files	Disk	Persisted dataset at /tmp/lance_bench_throughput_* for reuse across runs
Flamegraph	SVG (Linux only)	CPU profiling flamegraphs via pprof

Usage Examples

Run the full vector throughput benchmark:

cargo bench -p lance --bench vector_throughput

Note: The first run will create the dataset and IVF_PQ index, which can take several minutes for 1M rows at 768 dimensions. Subsequent runs reuse the persisted dataset.

Run with a filter for a specific concurrency level:

cargo bench -p lance --bench vector_throughput -- "8threads"

Run only the cached variant:

cargo bench -p lance --bench vector_throughput -- "cached"

The benchmark uses a sample size of 10 iterations and a significance level of 0.1, with flamegraph profiling enabled on Linux via pprof.

Related Pages

• Lance_format_Lance_TakeBench — Benchmark for random row access (take) operations
• Lance_format_Lance_DecoderBench — Benchmark for Lance encoding/decoding throughput
• Lance_format_Lance_MemWalReadBench — Benchmark for LSM scanner read performance including vector search