Index Scalar Fields

In Milvus, a scalar index is used to speed up metafiltering by a specific non-vector field value, similar to a traditional database index. This guide will walk you through creating and configuring scalar indexes for fields such as integers, strings, etc.

Types of scalar indexing

• Auto indexing: Milvus automatically decides the index type based on the data type of the scalar field. This is suitable when you do not need to control the specific index type.

• Custom indexing: You specify the exact index type, such as an inverted index or bitmap index. This provides more control over the index type selection.

Auto indexing

For mappings between scalar data types and default indexing algorithms, refer to Scalar field indexing algorithms.

# Auto indexing
client = MilvusClient(
    uri="http://localhost:19530"
)

index_params = MilvusClient.prepare_index_params() # Prepare an empty IndexParams object, without having to specify any index parameters

index_params.add_index(
    field_name="scalar_1", # Name of the scalar field to be indexed
    index_type="", # Type of index to be created. For auto indexing, leave it empty or omit this parameter.
    index_name="default_index" # Name of the index to be created
)

client.create_index(
  collection_name="test_scalar_index", # Specify the collection name
  index_params=index_params
)

import io.milvus.v2.common.IndexParam;
import io.milvus.v2.service.index.request.CreateIndexReq;

IndexParam indexParamForScalarField = IndexParam.builder()
    .fieldName("scalar_1") // Name of the scalar field to be indexed
    .indexName("default_index") // Name of the index to be created
    .indexType("") // Type of index to be created. For auto indexing, leave it empty or omit this parameter.
    .build();

List<IndexParam> indexParams = new ArrayList<>();
indexParams.add(indexParamForVectorField);

CreateIndexReq createIndexReq = CreateIndexReq.builder()
    .collectionName("test_scalar_index") // Specify the collection name
    .indexParams(indexParams)
    .build();

client.createIndex(createIndexReq);

await client.createIndex({
    collection_name: "test_scalar_index", // Specify the collection name
    field_name: "scalar_1", // Name of the scalar field to be indexed
    index_name: "default_index", // Name of the index to be created
    index_type: "" // Type of index to be created. For auto indexing, leave it empty or omit this parameter.
})

Custom indexing

The example below creates an inverted index for the scalar field scalar_2.

index_params = MilvusClient.prepare_index_params() #  Prepare an IndexParams object

index_params.add_index(
    field_name="scalar_2", # Name of the scalar field to be indexed
    index_type="INVERTED", # Type of index to be created
    index_name="inverted_index" # Name of the index to be created
)

client.create_index(
  collection_name="test_scalar_index", # Specify the collection name
  index_params=index_params
)

import io.milvus.v2.common.IndexParam;
import io.milvus.v2.service.index.request.CreateIndexReq;

IndexParam indexParamForScalarField = IndexParam.builder()
    .fieldName("scalar_1") // Name of the scalar field to be indexed
    .indexName("inverted_index") // Name of the index to be created
    .indexType("INVERTED") // Type of index to be created
    .build();

List<IndexParam> indexParams = new ArrayList<>();
indexParams.add(indexParamForVectorField);

CreateIndexReq createIndexReq = CreateIndexReq.builder()
    .collectionName("test_scalar_index") // Specify the collection name
    .indexParams(indexParams)
    .build();

client.createIndex(createIndexReq);

await client.createIndex({
    collection_name: "test_scalar_index", // Specify the collection name
    field_name: "scalar_1", // Name of the scalar field to be indexed
    index_name: "inverted_index", // Name of the index to be created
    index_type: "INVERTED" // Type of index to be created
})

Verifying the result

client.list_indexes(
    collection_name="test_scalar_index"  # Specify the collection name
)

# Output:
# ['default_index','inverted_index']

import java.util.List;
import io.milvus.v2.service.index.request.ListIndexesReq;

ListIndexesReq listIndexesReq = ListIndexesReq.builder()
    .collectionName("test_scalar_index")  // Specify the collection name
    .build();

List<String> indexNames = client.listIndexes(listIndexesReq);

System.out.println(indexNames);

// Output:
// [
//     "default_index",
//     "inverted_index"
// ]

res = await client.listIndexes({
    collection_name: 'test_scalar_index'
})

console.log(res.indexes)

// Output:
// [
//     "default_index",
//     "inverted_index"
// ]   

Limits

• Currently, scalar indexing supports INT8, INT16, INT32, INT64, FLOAT, DOUBLE, BOOL, VARCHAR, and ARRAY data types, but not the JSON data type.