Alter Collection Schema

As a collection moves from development to production, its schema often changes. You might add scalar fields such as source_uri or review_status for filtering and application logic, add a new vector field for embeddings generated by your application, add a BM25 Function and its generated sparse vector field for lexical search over existing text, or remove fields and Functions that are no longer used. Alter Collection Schema lets you make supported field and Function changes in place instead of recreating the collection.

Limits

Add user-defined fields

• Added user-defined fields must be nullable. Set nullable=True when calling add_collection_field(). For existing entities, the added field is NULL unless you add a scalar field with a default_value.

• Adding user-defined scalar fields is supported in Milvus 2.6.x and later. Adding user-defined vector fields is supported in Milvus 2.6.18 and later.

• Adding StructArray fields is supported in Milvus 3.0.0 and later. Added StructArray fields must be nullable.

• Field names must be unique among fields in the collection.

Add a Function and its generated vector field

• Each schema update can add only one Function and one generated vector field.

• The supported Function determines the generated vector field type: BM25 generates a SPARSE_FLOAT_VECTOR field, and MINHASH generates a BINARY_VECTOR field.

• The generated vector field must be a new field. It cannot point to a field that already exists in the collection schema.

• The generated vector field cannot be nullable.

• The input fields used by the Function must already exist in the collection.

• When adding a BM25 or MinHash Function to an existing collection, the Function input must be a VARCHAR field. A TEXT input is not supported in this workflow because Milvus cannot backfill the generated output for existing entities from that input type.

Drop user-defined fields

• You cannot drop the primary key field, partition key field, clustering key field, or the last vector field in a collection.

• You can drop a whole ARRAY<STRUCT> field, but you cannot drop an individual sub-field inside an ARRAY<STRUCT> field.

• You cannot directly drop a field that is used as a Function input field or generated as a Function output field. To remove a Function output field, drop the Function that generates it.

Drop a Function and its generated vector field

• In this schema-change workflow, dropping a Function removes the Function, its generated vector field, and the associated index. The Function input fields remain in the collection schema.

• Dropping a Function is rejected if removing its generated vector field would leave the collection without any vector field.

Add fields and Functions to an existing collection

Choose the workflow based on whether you are adding a user-defined field or a Function that generates a vector field:

• Add user-defined scalar fields when you need new metadata for filtering, query output, or application logic.

• Add StructArray fields when you need an array field whose elements share the same Struct schema.

• Add user-defined vector fields when your application generates embeddings and writes vector values to Milvus.

• Add a Function and its generated vector field when Milvus should generate vector values from existing fields, such as BM25 sparse vectors or MinHash signatures from text.

In all cases, the new field name must not already exist in the collection, and the total number of fields cannot exceed the Milvus field-count limit. For details, refer to Milvus Limits.

Add user-defined scalar fieldsCompatible with Milvus 2.6.x

Use add_collection_field() to add a user-defined scalar field to an existing collection.

This differs from storing arbitrary keys in the dynamic field: after the schema update is available, the new scalar field becomes a regular part of the collection schema. You can insert or upsert values into it, create indexes on it where supported, use it in queries and search filters, and return it in query or search output.

Because existing entities were inserted before the new field existed, every added user-defined scalar field must be nullable:

• If you add a scalar field with nullable=True and no default_value, existing entities return NULL for the new field.

• If you add a scalar field with nullable=True and default_value, existing entities return the default value instead of NULL.

Scalar filter expressions do not match NULL scalar values. For details, refer to Nullable Fields.

Example: Add a nullable scalar field

The following example adds a nullable source field to an existing collection named product_catalog.

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.add_collection_field(
    collection_name="product_catalog",
    field_name="source",
    data_type=DataType.VARCHAR,
    max_length=128,
    nullable=True,
)

After the field is added, entities that already existed in the collection return NULL for source. New entities can set source during insert or upsert.

Example: Add a scalar field with a default value

If existing entities should return a concrete value instead of NULL, specify default_value when adding a scalar field. The following example adds a review_status field and uses "unreviewed" as the default value.

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.add_collection_field(
    collection_name="product_catalog",
    field_name="review_status",
    data_type=DataType.VARCHAR,
    max_length=32,
    nullable=True,
    default_value="unreviewed",
)

After the field is added, entities that already existed in the collection return "unreviewed" for review_status. New entities can set a different value or use the default value when no value is provided.

Add StructArray fieldsCompatible with Milvus 3.0.0

Use add_collection_struct_field() to add a StructArray field that accepts arrays of Struct elements. To add a StructArray field, do as follows:

1. Create a Struct schema that contains the necessary sub-fields of supported data types. For applicable data types, refer to StructArray Limits.

2. Reference the Struct schema created above and set the maximum capacity of the field in add_collection_struct_field().

3. Set nullable=True in the request.

Example: Add a nullable StructArray field

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="http://localhost:19530")

# Create a Struct schema.
struct_schema = client.create_struct_field_schema()

# Add scalar fields to the Struct.
struct_schema.add_field("text", DataType.VARCHAR, max_length=65535)
struct_schema.add_field("chapter", DataType.VARCHAR, max_length=512)

# Add vector fields to the Struct with mmap enabled.
struct_schema.add_field("text_vector", DataType.FLOAT_VECTOR, mmap_enabled=True, dim=5)
struct_schema.add_field("chapter_vector", DataType.FLOAT_VECTOR, mmap_enabled=True, dim=5)

client.add_collection_struct_field(
    collection_name="books",
    field_name="chunks",
    struct_schema=struct_schema,
    max_capacity=1024,
    nullable=True,
)

After the StructArray field is added, entities that already exist in the collection return NULL for chunks across all its sub-fields. When you insert a new entity, ensure that all sub-fields are either NULL or have valid values. Inserting an entity with some sub-fields set to NULL and others set to valid values results in errors.

Add user-defined vector fieldsCompatible with Milvus 2.6.18+

Use add_collection_field() to add a user-defined vector field when your application generates embeddings and writes vector values to Milvus.

Every added user-defined vector field must be nullable. Existing entities have NULL for the new vector field until you write vector values through upsert or a backfill workflow. New entities can include the vector field during insert. Vector search skips entities whose vector value is NULL. For details, refer to Nullable Fields.

Example: Add a nullable vector field

The following example adds a nullable dense vector field named embedding_v2 to an existing collection. Set dim to the dimensionality of the embeddings generated by your application.

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.add_collection_field(
    collection_name="product_catalog",
    field_name="embedding_v2",
    data_type=DataType.FLOAT_VECTOR,
    dim=768,
    nullable=True,
)

After the field is added, create an index on the new vector field before searching it:

index_params = client.prepare_index_params()

index_params.add_index(
    field_name="embedding_v2",
    index_type="AUTOINDEX",
    metric_type="COSINE",
)

client.create_index(
    collection_name="product_catalog",
    index_params=index_params,
)

Existing entities have NULL for embedding_v2 and are skipped when you search on this field. To make existing entities searchable through embedding_v2, write non-NULL vector values through upsert or a backfill workflow. New entities can include embedding_v2 during insert.

Add a Function and its generated vector fieldCompatible with Milvus 3.0.x

Use this workflow when Milvus should generate a new vector field from data already stored in an existing collection. The operation adds three related schema elements:

• A Function definition that reads from one or more existing input fields.

• A new vector field that stores the Function output.

• An index definition bound to the new vector field.

For example, a BM25 Function reads an existing VARCHAR field and generates a SPARSE_FLOAT_VECTOR field for lexical search. A MinHash Function generates a BINARY_VECTOR field for near-duplicate detection. This workflow does not add or replace the Function input field.

Adding a Function and its generated vector field to an existing collection also requires schema version compaction and storage version compaction. Milvus rejects the request if either setting is disabled. These additional prerequisites apply only when modifying an existing collection; defining the Function in the initial collection schema does not use this existing-data backfill workflow.

The supported Function determines the generated vector field type:

Function	Generated vector field type	Typical input field	Typical use case
BM25	SPARSE_FLOAT_VECTOR	A VARCHAR field with analyzer enabled	Lexical search and keyword relevance
MINHASH	BINARY_VECTOR	A VARCHAR field	Near-duplicate detection

For details about how each Function works, refer to BM25 Function and MinHash Function.

The generated vector field must not already exist in the collection, and it cannot be nullable. The Function input field must already exist.

Example: Add a BM25 Function and its generated sparse vector field

The following example adds a BM25 Function named text_bm25 and its generated sparse vector field named text_sparse to an existing collection. The collection must already have a VARCHAR field named text with analyzer enabled.

from pymilvus import DataType, Function, FunctionType, MilvusClient

client = MilvusClient(uri="http://localhost:19530")

sparse_field = client.create_field_schema(
    name="text_sparse",
    data_type=DataType.SPARSE_FLOAT_VECTOR,
    desc="BM25-generated sparse vector field",
)

bm25_function = Function(
    name="text_bm25",
    input_field_names=["text"],
    output_field_names=["text_sparse"],
    function_type=FunctionType.BM25,
)

index_params = client.prepare_index_params()

index_params.add_index(
    field_name="text_sparse",
    index_type="SPARSE_INVERTED_INDEX",
    metric_type="BM25",
    params={
        "inverted_index_algo": "DAAT_MAXSCORE",
        "bm25_k1": 1.2,
        "bm25_b": 0.75,
    },
)

client.add_function_field(
    collection_name="product_catalog",
    field_schema=sparse_field,
    func=bm25_function,
    index_params=index_params,
)

The index_params object must contain exactly one index definition for the new Function output field. Milvus adds the Function, its generated vector field, and the bound index definition in the same schema change. Do not call create_index() separately after add_function_field().

Conceptually, this operation adds the following Function, generated output field, and bound index definitions:

New Function:
  name: "text_bm25"
  type: BM25
  input_field_names: ["text"]
  output_field_names: ["text_sparse"]

New generated output field:
  name: "text_sparse"
  data_type: SPARSE_FLOAT_VECTOR
  nullable: false

Bound index:
  field_name: "text_sparse"
  index_type: SPARSE_INVERTED_INDEX
  metric_type: BM25

After the request succeeds, describe_collection() returns both the new text_bm25 Function and its generated text_sparse vector field in the collection schema. Milvus generates the Function output for new entities as they are written. For existing entities, Milvus populates the generated vector field asynchronously through background compaction. Schema visibility confirms that the schema update succeeded, but does not indicate that backfill has completed for every existing entity. For the complete BM25 search workflow, refer to Full Text Search.

Milvus also supports MinHash Functions and their generated binary vector fields for near-duplicate detection. A MinHash Function uses FunctionType.MINHASH and writes to a new BINARY_VECTOR output field. For configuration details, refer to MinHash Function.

Drop fields and Functions from an existing collection

You can remove user-defined fields directly when they are no longer part of your collection model. To remove a Function and its generated vector field, drop the Function; Milvus removes the generated field and its index in the same schema change.

Drop user-defined fieldsCompatible with Milvus 3.0.x

Use drop_collection_field() to remove a user-defined scalar, vector, or StructArray field that is no longer part of your collection model.

Dropping a field first changes the collection schema and field visibility:

• After drop_collection_field() succeeds, the collection schema is updated: describe_collection() no longer returns the dropped field, and queries or searches can no longer return the field in output_fields or use it in expressions.

• Indexes built on the dropped field are cleaned up as part of the schema update.

Storage cleanup is handled separately from schema cleanup. For details, refer to When is storage space reclaimed after dropping a field?.

Example: Drop a user-defined scalar field

The following example assumes that experiment_tag is a user-defined scalar field in product_catalog, and drops it from the collection.

from pymilvus import MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.drop_collection_field(
    collection_name="product_catalog",
    field_name="experiment_tag",
)

After dropping a field, you can call describe_collection() to verify that the field is no longer part of the schema.

Example: Drop a StructArray field

The following example assumes that chunks is a StructArray field in my_collection, and drops it from the collection.

from pymilvus import MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.drop_collection_field(
    collection_name="my_collection",
    field_name="chunks",
)

Example: Drop a user-defined vector field

You can drop a vector field with the same drop_collection_field() method, but the collection must still contain at least one vector field after the drop. This is useful for collections that temporarily carry multiple vector representations and later standardize on one of them.

The following example assumes that image_vector is a user-defined vector field in hybrid_catalog, and that the collection still retains another vector field, such as text_vector.

from pymilvus import MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.drop_collection_field(
    collection_name="hybrid_catalog",
    field_name="image_vector",
)

If image_vector is the last vector field in the collection, the drop operation is rejected.

Drop a Function and its generated vector fieldCompatible with Milvus 3.0.x

Use this operation when you no longer need a Function or its generated vector field, such as a BM25 Function and its generated sparse vector field.

Call drop_function_field() with the Function name. Milvus removes the Function, its generated vector field, and the associated index while preserving the Function input fields.

Example: Drop a BM25 Function and its generated sparse vector field

The following example assumes that text_bm25 is a BM25 Function in product_catalog and generates a sparse vector output field named text_sparse.

from pymilvus import MilvusClient

client = MilvusClient(uri="http://localhost:19530")

client.drop_function_field(
    collection_name="product_catalog",
    function_name="text_bm25",
)

After the operation succeeds, describe_collection() no longer returns the dropped Function or its generated vector field. The Function input fields remain in the schema.

If removing the Function output field would leave the collection without any vector field, the operation is rejected.

FAQ

Which method should I use to add a field or Function?

Use add_collection_field() to add a user-defined scalar or vector field.

Use add_collection_struct_field() to add a StructArray field when you need an array field whose elements share the same Struct schema.

Use add_function_field() to add a Function, its generated vector field, and the bound index definition in the same schema change.

Why must added user-defined fields be nullable?

Existing entities were inserted before the new field existed, so they do not have values for that field. Setting nullable=True lets Milvus represent the missing value as NULL until your application writes a value or, for scalar fields, until a default value applies.

This rule applies to user-defined scalar fields and user-defined vector fields added with add_collection_field(), and to StructArray fields added with add_collection_struct_field(). It does not apply to a Function’s generated vector field, which cannot be nullable.

What happens to existing entities after I add a user-defined field?

For a user-defined scalar field, existing entities return NULL unless you set a default_value. If you set a default_value, existing entities return that default value.

For a user-defined vector field, existing entities have NULL for the new vector field. Vector search on the added field skips entities whose vector value is NULL. To make existing entities searchable through the new vector field, write non-NULL vector values through upsert or a backfill workflow. New entities can include the new vector field during insert.

For a StructArray field, existing entities return NULL for the new StructArray field across all its sub-fields. New entities must provide either NULL for all sub-fields or valid values for all sub-fields.

Can I add BM25 lexical search to an existing collection?

Yes. If the collection already has a VARCHAR field with analyzer enabled, you can add a BM25 Function and its generated sparse vector field for lexical search. In this workflow, Milvus adds the Function, the new SPARSE_FLOAT_VECTOR output field, and the bound index definition in the same schema change. You cannot use an existing TEXT field as the BM25 input in this schema-change workflow. To use a TEXT input, define the field and BM25 Function when you create the collection.

When calling add_function_field(), provide an index_params object that contains one SPARSE_INVERTED_INDEX index with metric_type="BM25" for the new output field. Milvus binds the index definition to the generated field as part of the same schema change.

How do I drop a Function and its generated vector field?

Call drop_function_field() with the Function name. In this schema-change workflow, Milvus removes the Function, its generated vector field, and the associated index together while preserving the Function input fields.

Do I need to wait after altering a collection schema?

Usually, no manual wait is required. If your next operation depends on the updated schema, you can call describe_collection() first to confirm the schema that Milvus currently returns.

In a distributed deployment, there can be a short propagation window while Milvus components refresh collection metadata. If an operation immediately after the schema change fails with a schema-related error, refresh the schema and retry the operation.

When is storage space reclaimed after dropping a field?

Dropping a field removes it from the current schema and normal query/search visibility, but historical data for that field is not physically deleted from object storage immediately.

Storage space can be reclaimed later during compaction. Compaction is a background process that reorganizes existing data files into new, more compact files. After a field is dropped, newly compacted files follow the current schema and omit the dropped field. Milvus does not guarantee an immediate or fixed-time storage-space reduction after dropping a field.

What happens if I add a scalar field with the same name as a dynamic field key?

If dynamic field is enabled, you can add a scalar field with the same name as an existing dynamic field key. The new scalar field masks the dynamic field key in normal query output, but the original dynamic data is preserved in $meta.

For example, if existing entities store a dynamic key named source, and you later add a scalar field named source, normal output for source refers to the scalar field. To access the original dynamic value, use the $meta path syntax, such as $meta["source"].