Milvus vs LanceDB: an open-source vector database comparison
Milvus vs LanceDB is a comparison of two different kinds of software: both are Apache-2.0, but Milvus is a distributed vector database service with storage-compute separation, while LanceDB OSS is an embedded library that runs inside your application process over the Lance format. Milvus serves many clients, replicas and writers from one cluster; LanceDB gives one process a serverless vector database with fast local search and nothing to run. Choose Milvus when several services, writers or tenants share the data or it will grow toward billions of vectors; choose LanceDB for single-process, local-first and Lance-native work.
- Apache-2.0, 40K+ GitHub stars
- Scales to tens of billions of vectors
- Storage and compute separated
- Searches Lance datasets in place with external collections
- Salesforce
- Walmart
- NVIDIA
- eBay
- IBM
- Cisco
- Roblox
- Exa.ai
- OpenEvidence
- Read.ai
- Plaud
- Filevine
Milvus vs LanceDB scalability: service or library, and how far does each grow?
Milvus is built on storage-compute separation. Segments and the write-ahead log live in object storage; streaming nodes take writes, query nodes serve search with in-memory replicas for throughput and failover, and data nodes build indexes. Each node type scales on its own; Milvus Distributed is documented for tens of billions of vectors.
LanceDB OSS is an embedded library over Lance tables, using the serving application process for query resources and local disk or object storage for data. Capacity and concurrency depend on the index, storage and host resources. LanceDB Cloud and Enterprise are separate service deployments; their architecture should not be conflated with the OSS library.
| Capability | Milvus | LanceDB (OSS) |
|---|---|---|
| Architecture | Distributed service: proxies, streaming, query and data nodes | Embedded library inside the application process; no server |
| Where data lives | Object storage (S3, MinIO, Azure Blob, GCS), including the WAL | Lance tables on local NVMe, EBS, EFS or S3 / GCS / Azure Blob |
| Documented scale | 100 million to tens of billions of vectors (Distributed) | Evaluate the index, storage and host; assess Cloud/Enterprise separately |
| Concurrent writers | Multiple clients write through proxies and the WAL, subject to cluster resources and limits | Supported; each add() commits a version, and writers retry commits a limited number of times |
| Replicas and failover | In-memory replicas: more QPS, and requests move to another query node on failure | The host process; Enterprise plan executors run with 2+ replicas |
| Adding capacity | Add a query node; it loads segments from object storage | Scale application serving resources or choose a service deployment |
| Consistency | Strong, Bounded (default), Session or Eventually, per request | read_consistency_interval; by default no cross-process refresh check |
| Multi-tenant vector database | Database, collection, partition, or partition key for millions of tenants | Namespaces that organize many Lance tables |
Milvus vs LanceDB performance: what does VectorDBBench show?
These charts show the verified Milvus and Zilliz Cloud VectorDBBench configurations. They do not include a same-condition result for the other product, so they cannot establish a speed advantage. Benchmark with matching data, filters, recall targets and hardware before comparing.
- Zilliz CloudZillizCloud-8cu-perf13,316.23 QPSP99 2 ms · recall 0.9383
- MilvusMilvus-16c64g-sq4u-fp16-force_merge9,575.69 QPSP99 2.3 ms · recall 0.9189
- Zilliz CloudZillizCloud-8cu-perf-partition_key10,020.53 QPSP99 2.6 ms · recall 0.9788
- MilvusMilvus-16c64g-sq8-partition_key11,280.08 QPSP99 1.6 ms · recall 0.9507
Zilliz Cloud is fully managed Milvus: the same engine and top-tier throughput, with no clusters to size, upgrade or scale.
Try Zilliz Cloud freeLanceDB vs Milvus features: a vector database comparison table
Both cover vector search, BM25 full-text search, hybrid search and filtering. Milvus has more index families, server-side hybrid ranking and automatic indexing of new data; LanceDB keeps everything in one versioned Lance table. The two also meet: if your embeddings already sit in Lance datasets, Milvus external collections (Milvus 3.0+) index them in place and serve search without copying a row (see vector search on the data lake).
| Feature | Milvus | LanceDB |
|---|---|---|
| Dense vector index types | HNSW, IVF (FLAT, SQ8, PQ, RaBitQ), SCANN, DiskANN, GPU (CAGRA, IVF) | IVF_PQ, IVF_RQ, IVF_HNSW_SQ, IVF_HNSW_PQ, IVF_HNSW_FLAT; brute-force scan below about 100K vectors |
| Quantization | SQ8, PQ, PRQ, RaBitQ; HNSW_SQ, HNSW_PQ, HNSW_PRQ | PQ, SQ, RaBitQ (1 to 8 bits per dimension) |
| GPU | GPU index build and search (CAGRA, GPU IVF) | GPU index build: manual IVF_PQ via PyTorch in OSS; automatic in Enterprise |
| BM25 and full-text search | Built-in BM25 function turns raw text into sparse vectors on the server | Lance-native BM25 FTS index; tokenizers include jieba, lindera and ICU |
| Hybrid search | Multiple vector fields per query, fused with RRF or weighted ranker on the server | Vector plus FTS query with a reranker: RRF by default, Cohere, cross-encoder, ColBERT |
| Multi-vector | Up to 10 vector fields per collection; StructArray EmbeddingList fields searched with MAX_SIM for ColBERT and ColPali | Multivector columns scored with MaxSim for ColBERT-style models |
| Filtering and scalar indexes | Filter expressions, JSON field with JSON path index; INVERTED, BITMAP, STL_SORT | SQL WHERE via DataFusion, pre-filtering by default; BTREE, BITMAP, LABEL_LIST |
| Index freshness | Data nodes index sealed segments automatically; new data visibility follows the selected consistency level | OSS: call optimize() to compact and index new rows, which are brute-forced until then; Enterprise indexes in the background |
| Lance format and versioning | External collections search Lance datasets in place, zero-copy | Native storage format; every write is a new version with time travel, tags and branches |
| Multimodal data | Vectors with scalar, JSON, array, text and StructArray fields; raw media stays in your object store | Images, audio, video, text and embeddings in one Lance table |
| Data-frame and lakehouse interop | Bulk import from Parquet, JSON or CSV; external collections over Lance, Iceberg, Parquet and Vortex | pandas, PyArrow, Polars, DuckDB, Pydantic and dlt; PyTorch loaders on the same table |
| Deployment options | Milvus Lite, Standalone, Distributed, Zilliz Cloud | Embedded in your binary, container or serverless function; LanceDB Cloud and Enterprise (managed or BYOC) |
| RAG framework integrations | LangChain, LlamaIndex, Haystack, Dify, DSPy, n8n and more | LangChain, LlamaIndex and Hugging Face |
| License and language | Apache-2.0; Go and C++ | Apache-2.0; Rust |
| SDK languages | Python, Java, Go, Node.js, C#, C++, Rust, RESTful | Python, TypeScript/JavaScript, Rust, Java; REST API for Cloud and Enterprise |
| Community and governance | LF AI & Data graduated project; 40K+ GitHub stars | LanceDB Inc.; 10K+ GitHub stars |
Milvus vs LanceDB cost and operations: what does each take to run?
LanceDB vs Milvus cost starts with what you run. LanceDB OSS is zero-ops: pip install, then a directory or bucket. Milvus Lite is also a pip install and Milvus Standalone one Docker image, so Kubernetes only enters with Milvus Distributed. After that, cost is RAM on whichever process serves the index.
| Topic | Milvus | LanceDB (OSS) |
|---|---|---|
| Production without Kubernetes | Milvus Standalone: a single Docker image with the Woodpecker WAL embedded, up to 100 million vectors | The library inside your service; LanceDB Enterprise for a cluster |
| Services you operate | Distributed: etcd and object storage (MinIO or S3) via Helm or the Milvus Operator; Woodpecker replaces Kafka or Pulsar | None: a Lance directory on local NVMe, EBS, EFS or an S3, GCS or Azure bucket |
| What drives cost | RAM on query nodes; quantization, DiskANN and tiered storage trade it for disk and object storage | RAM and CPU of the host; IVF_PQ keeps indexes on disk, and its docs favour local NVMe over object storage for low latency |
| Versioning and backup | milvus-backup to S3, GCS, Azure or local disk while the cluster stays online; RBAC and TLS | Every write is a new Lance version: time travel, tags, branches; optimize() cleans old versions |
When is LanceDB the better choice?
What is LanceDB and when should I use it? LanceDB is a well-made embedded retrieval library, and for a single process it removes a whole tier of infrastructure. Teams looking for LanceDB alternatives usually need several services, writers or tenants on the same data; if one application owns the table, LanceDB is a sound choice.
- A single-process app, notebook, CLI or edge function that wants search with no server.
- Local-first development where the dataset fits one machine and ships with the code.
- Lance-native pipelines: PyTorch, Ray or Spark jobs that train on and query the same versioned table.
- Small teams without Kubernetes who prefer a library and a bucket over a cluster.
- Multimodal datasets where images, audio and embeddings belong in one table.
Which AI companies and enterprises run Milvus in production?
Milvus and Zilliz Cloud (managed Milvus) run in AI-native companies and large enterprises alike. The milvus.io adopter wall includes Salesforce, Walmart, NVIDIA, eBay, IBM, Cisco, Roblox, and Reddit, alongside Exa.ai, OpenEvidence, Read.ai, Notta.ai, Plaud, and Filevine. Published results:
- OpenEvidence: under 10 ms search latency and 4,000 QPS at peak on Zilliz Cloud.
- Plaud: 10B+ vector chunks per region with under 200 ms average recall latency.
- Exa: neural search latency cut from seconds to under 200 ms with hybrid search.
- Airtable: self-hosts Milvus on Kubernetes with the Milvus Operator for semantic search.
Customer stories
Filevine
30 billion vectors managed across millions of legal documents on Zilliz Cloud.
Read the storyMeituan
10B+ vectors in production on self-run Milvus, with 2B+ in a single collection.
Read the storyRead AI
20–50 ms retrieval latency on Milvus for millions of monthly users and million-scale active tenants.
Read the storyExplore the concepts
- What determines vector-database scalability?
- How do sharding and replication differ in a vector database?
- How do you choose a vector index for your workload?
- How can Milvus search data stored in Lance?
- How do Lance, Iceberg, and Parquet play different roles in a vector lake?
- What is hybrid search, and when should you use it?
- How should you design multi-tenancy for vector search?
- How should you benchmark vector search performance?
Frequently asked questions
What is LanceDB and when should I use it?
LanceDB OSS calls itself an open-source embedded retrieval library for multimodal AI: it runs inside your Python, TypeScript or Rust process and reads Lance tables from disk or object storage. Use it when one application owns the data; use Milvus when many clients, writers or tenants share it.
What is the difference between Lance and LanceDB?
Lance is the Apache-2.0 columnar format; Lance format vs Parquet comes down to versioning and random row access for embeddings and media. LanceDB is the library and service that indexes and queries Lance tables. Milvus reads Lance datasets too, through external collections.
How large of a dataset can LanceDB handle?
LanceDB OSS is an embedded library over Lance tables, using the serving application process for query resources and local disk or object storage for data. Capacity and concurrency depend on the index, storage and host resources. LanceDB Cloud and Enterprise are separate service deployments; their architecture should not be conflated with the OSS library.
Does LanceDB support concurrent operations?
Reads scale with your processes. Concurrent writes work with limits: every add() commits a new table version, and the FAQ notes that too many concurrent writers can lead to failing writes because commit retries are bounded. Milvus accepts writes from any number of clients through its streaming WAL.
Which is faster, Milvus or LanceDB?
These charts show the verified Milvus and Zilliz Cloud VectorDBBench configurations. They do not include a same-condition result for the other product, so they cannot establish a speed advantage. Benchmark with matching data, filters, recall targets and hardware before comparing.
Can Milvus search Lance data without importing it?
Yes. Milvus 3.0+ external collections point at a Lance dataset directory with format lance-table, map columns to fields, then refresh, index and search. Rows stay in Lance, read-only, with no second copy.
Do I need Kubernetes to run Milvus in production?
No. Milvus Standalone packs every component into a single Docker image with the Woodpecker WAL embedded and is documented up to 100 million vectors. Kubernetes, via Helm or the Milvus Operator, is for Milvus Distributed.
What is the difference between Milvus and Zilliz Cloud?
Zilliz Cloud is fully managed Milvus from the team that created it. It uses the same client APIs, so code written against Milvus Lite or Standalone moves to Zilliz Cloud without a rewrite.
Prefer not to run a cluster?
Zilliz Cloud is fully managed Milvus from the team that created it, with the same MilvusClient API you use locally. Start on open-source Milvus and move to the managed service without rewriting your application.
Sources and technical documentation (64)
- https://milvus.io
- https://github.com/milvus-io/milvus
- https://milvus.io/docs/architecture_overview.md
- https://milvus.io/docs/install-overview.md
- https://milvus.io/docs/replica.md
- https://milvus.io/docs/consistency.md
- https://milvus.io/docs/scaleout.md
- https://milvus.io/docs/index-explained.md
- https://milvus.io/docs/full-text-search.md
- https://milvus.io/docs/multi-vector-search.md
- https://milvus.io/docs/json-indexing.md
- https://milvus.io/docs/multi_tenancy.md
- https://milvus.io/docs/create-an-external-collection.md
- https://github.com/lancedb/lancedb
- https://docs.lancedb.com/
- https://docs.lancedb.com/faq/faq-oss
- https://docs.lancedb.com/faq/faq-enterprise
- https://docs.lancedb.com/enterprise
- https://docs.lancedb.com/enterprise/architecture
- https://docs.lancedb.com/enterprise/benchmarks
- https://docs.lancedb.com/storage
- https://docs.lancedb.com/tables/consistency
- https://docs.lancedb.com/tables/versioning
- https://docs.lancedb.com/indexing/vector-index
- https://docs.lancedb.com/indexing/quantization
- https://docs.lancedb.com/indexing/gpu-indexing
- https://docs.lancedb.com/indexing/fts-index
- https://docs.lancedb.com/indexing/scalar-index
- https://docs.lancedb.com/indexing/reindexing
- https://docs.lancedb.com/search/hybrid-search
- https://docs.lancedb.com/search/multivector-search
- https://docs.lancedb.com/search/filtering
- https://docs.lancedb.com/namespaces
- https://docs.lancedb.com/performance
- https://zilliz.com/vdbbench-leaderboard
- https://github.com/zilliztech/VectorDBBench
- https://github.com/zilliztech/VectorDBBench/blob/main/vectordb_bench/results/leaderboard_v2.json
- https://github.com/zilliztech/VectorDBBench/tree/main/vectordb_bench/backend/clients/lancedb
- https://zilliz.com/customers/filevine
- https://zilliz.com/customers/openevidence
- https://zilliz.com/customers/exa
- https://zilliz.com/customers/airtable
- https://zilliz.com/customers/plaud
- https://zilliz.com/customers/read-ai
- https://zilliz.com/customers/meituan
- https://milvus.io/docs/quickstart.md
- https://milvus.io/docs/rbac.md
- https://milvus.io/docs/integrations_overview.md
- https://milvus.io/docs/woodpecker_architecture.md
- https://milvus.io/docs/install_cluster-helm.md
- https://milvus.io/docs/install_cluster-milvusoperator.md
- https://milvus.io/docs/prepare-source-data.md
- https://milvus.io/docs/array-of-structs.md
- https://milvus.io/docs/search-with-embedding-lists.md
- https://github.com/zilliztech/milvus-backup
- https://lfaidata.foundation/projects/
- https://github.com/lancedb/lance
- https://docs.lancedb.com/tables/multimodal
- https://docs.lancedb.com/integrations/ai/langchain
- https://docs.lancedb.com/integrations/ai/llamaIndex
- https://docs.lancedb.com/integrations/data/duckdb
- https://docs.lancedb.com/integrations/data/polars_arrow
- https://docs.lancedb.com/integrations/data/pandas_and_pyarrow
- https://docs.lancedb.com/training/torch