Build an Index on Vectors
This guide describes how to build an index on vectors in Milvus.
Vector indexes are an organizational unit of metadata used to accelerate vector similarity search . Without the index built on vectors, Milvus will perform a brute-force search by default.
See Vector Index for more information about the mechanism and varieties of vector indexes.
The following example builds a 1024-cluster IVF_FLAT index with Euclidean distance (L2) as the similarity metric. You can choose the index and metrics that suit your scenario. See Similarity Metrics for more information.
Prepare index parameter
Prepare the index parameters as follows:
index_params = {
"metric_type":"L2",
"index_type":"IVF_FLAT",
"params":{"nlist":1024}
}
const index_params = {
metric_type: "L2",
index_type: "IVF_FLAT",
params: JSON.stringify({ nlist: 1024 }),
};
idx, err := entity.NewIndexIvfFlat( // NewIndex func
entity.L2, // metricType
1024, // ConstructParams
)
if err != nil {
log.Fatal("fail to create ivf flat index parameter:", err.Error())
}
final IndexType INDEX_TYPE = IndexType.IVF_FLAT; // IndexType
final String INDEX_PARAM = "{\"nlist\":1024}"; // ExtraParam
create index
Collection name (book): book
The name of the field to create an index for (book_intro): book_intro
Index type (FLAT, IVF_FLAT, IVF_SQ8, IVF_PQ, RNSG, HNSW, ANNOY): IVF_FLAT
Index metric type (L2, IP, HAMMING, TANIMOTO): L2
Index params nlist: 1024
Timeout []:
curl -X 'POST' \
'http://localhost:9091/api/v1/index' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"collection_name": "book",
"field_name": "book_intro",
"extra_params":[
{"key": "metric_type", "value": "L2"},
{"key": "index_type", "value": "IVF_FLAT"},
{"key": "params", "value": "{\"nlist\":1024}"}
]
}'
Parameter
Description
Options
metric_type
Type of metrics used to measure the similarity of vectors.
For floating point vectors:
L2
(Euclidean distance)
IP
(Inner product)
For binary vectors:
JACCARD
(Jaccard distance)
TANIMOTO
(Tanimoto distance)
HAMMING
(Hamming distance)
SUPERSTRUCTURE
(Superstructure)
SUBSTRUCTURE
(Substructure)
index_type
Type of index used to accelerate the vector search.
For floating point vectors:
FLAT
(FLAT)
IVF_FLAT
(IVF_FLAT)
IVF_SQ8
(IVF_SQ8)
IVF_PQ
(IVF_PQ)
HNSW
(HNSW)
ANNOY
(ANNOY)
DISKANN*
(DISK_ANN)
For binary vectors:
BIN_FLAT
(BIN_FLAT)
BIN_IVF_FLAT
(BIN_IVF_FLAT)
params
Building parameter(s) specific to the index.
See In-memory Index and On-disk Index for more information.
* DISKANN has certain prerequisites to meet. For details, see On-disk Index .
Parameter
Description
Option
metric_type
Type of metrics used to measure the similarity of vectors.
For floating point vectors:
L2
(Euclidean distance)
IP
(Inner product)
For binary vectors:
JACCARD
(Jaccard distance)
TANIMOTO
(Tanimoto distance)
HAMMING
(Hamming distance)
SUPERSTRUCTURE
(Superstructure)
SUBSTRUCTURE
(Substructure)
index_type
Type of index used to accelerate the vector search.
For floating point vectors:
FLAT
(FLAT)
IVF_FLAT
(IVF_FLAT)
IVF_SQ8
(IVF_SQ8)
IVF_PQ
(IVF_PQ)
HNSW
(HNSW)
ANNOY
(ANNOY)
For binary vectors:
BIN_FLAT
(BIN_FLAT)
BIN_IVF_FLAT
(BIN_IVF_FLAT)
params
Building parameter(s) specific to the index.
See In-memory Index and On-disk Index for more information.
Parameter
Description
Options
NewIndex func
Function to create entity. Index according to different index types.
For floating point vectors:
NewIndexFlat
(FLAT)
NewIndexIvfFlat
(IVF_FLAT)
NewIndexIvfSQ8
(IVF_SQ8)
NewIndexIvfPQ
(IVF_PQ)
NewIndexHNSW
(HNSW)
NewIndexANNOY
(ANNOY)
NewIndexDISKANN*
(DISK_ANN)
For binary vectors:
NewIndexBinFlat
(BIN_FLAT)
NewIndexBinIvfFlat
(BIN_IVF_FLAT)
metricType
Type of metrics used to measure the similarity of vectors.
For floating point vectors:
L2
(Euclidean distance)
IP
(Inner product)
For binary vectors:
JACCARD
(Jaccard distance)
TANIMOTO
(Tanimoto distance)
HAMMING
(Hamming distance)
SUPERSTRUCTURE
(Superstructure)
SUBSTRUCTURE
(Substructure)
ConstructParams
Building parameter(s) specific to the index.
See In-memory Index and On-disk Index for more information.
* DISKANN has certain prerequisites to meet. For details, see On-disk Index .
Parameter
Description
Options
IndexType
Type of index used to accelerate the vector search.
For floating point vectors:
FLAT
(FLAT)
IVF_FLAT
(IVF_FLAT)
IVF_SQ8
(IVF_SQ8)
IVF_PQ
(IVF_PQ)
HNSW
(HNSW)
ANNOY
(ANNOY)
DISKANN*
(DISK_ANN)
For binary vectors:
BIN_FLAT
(BIN_FLAT)
BIN_IVF_FLAT
(BIN_IVF_FLAT)
ExtraParam
Building parameter(s) specific to the index.
See In-memory Index and On-disk Index for more information.
* DISKANN has certain prerequisites to meet. For details, see On-disk Index .
Option
Description
--help
Displays help for using the command.
Parameter
Description
Options
collection_name
Name of the collection to build the index on.
field_name
Name of the vector field to build the index on.
metric_type
Type of metrics used to measure the similarity of vectors.
For floating point vectors:
L2
(Euclidean distance)
IP
(Inner product)
For binary vectors:
JACCARD
(Jaccard distance)
TANIMOTO
(Tanimoto distance)
HAMMING
(Hamming distance)
SUPERSTRUCTURE
(Superstructure)
SUBSTRUCTURE
(Substructure)
index_type
Type of index used to accelerate the vector search.
For floating point vectors:
FLAT
(FLAT)
IVF_FLAT
(IVF_FLAT)
IVF_SQ8
(IVF_SQ8)
IVF_PQ
(IVF_PQ)
HNSW
(HNSW)
ANNOY
(ANNOY)
For binary vectors:
BIN_FLAT
(BIN_FLAT)
BIN_IVF_FLAT
(BIN_IVF_FLAT)
params
Building parameter(s) specific to the index.
See In-memory Index for more information.
Build index
Build the index by specifying the vector field name and index parameters.
from pymilvus import Collection, utility
# Get an existing collection.
collection = Collection("book")
collection.create_index(
field_name="book_intro",
index_params=index_params
)
utility.index_building_progress("book")
# Output: {'total_rows': 0, 'indexed_rows': 0}
await milvusClient.createIndex({
collection_name: "book",
field_name: "book_intro",
extra_params: index_params,
});
err := milvusClient.CreateIndex(
context.Background(), // ctx
"book", // CollectionName
"book_intro", // fieldName
idx, // entity.Index
false, // async
)
if err != nil {
log.Fatal("fail to create index:", err.Error())
}
milvusClient.createIndex(
CreateIndexParam.newBuilder()
.withCollectionName("book")
.withFieldName("book_intro")
.withIndexType(INDEX_TYPE)
.withMetricType(MetricType.L2)
.withExtraParam(INDEX_PARAM)
.withSyncMode(Boolean.FALSE)
.build()
);
# Follow the previous step.
# Follow the previous step.
Parameter
Description
field_name
Name of the vector field to build index on.
index_params
Parameters of the index to build.
Parameter
Description
collection_name
Name of the collection to build index in.
field_name
Name of the vector field to build index on.
extra_params
Parameters of the index to build.
Parameter
Description
ctx
Context to control API invocation process.
CollectionName
Name of the collection to build index on.
fieldName
Name of the vector field to build index on.
entity.Index
Parameters of the index to build.
async
Switch to control sync/async behavior. The deadline of context is not applied in sync building process.
What's next
Learn more basic operations of Milvus: