Compact Data

This topic describes how to compact data in Milvus.

Milvus supports automatic data compaction by default. You can configure your Milvus to enable or disable compaction and automatic compaction.

If automatic compaction is disabled, you can still compact data manually.

Compact data manually

Compaction requests are processed asynchronously because they are usually time-consuming.

from pymilvus import Collection
collection = Collection("book")      # Get an existing collection.
collection.compact()

const res = await milvusClient.compact({
  collection_name: "book",
});
const compactionID = res.compactionID;

compactionID, err := milvusClient.ManualCompaction(
    context.Background(), // ctx
    "book",               // collection name
    0,                    // tolerance duration
)
if err != nil {
    log.Fatal("failed to manual compaction:", err.Error())
}

R<ManualCompactionResponse> response = milvusClient.manualCompaction(
  ManualCompactionParam.newBuilder()
    .withCollectionName("book")
    .build()
);
long compactionID = response.getData().getCompactionID();

compact -c book

curl -X 'POST' \
  'http://localhost:9091/api/v1/compaction' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "collectionID": 434262071120432449
  }'

{"status":{},"compactionID":434262132129005569}

Check compaction status

You can check the compaction status with the compaction ID returned when the manual compaction is triggered.

collection.get_compaction_state()

const state = await milvusClient.getCompactionState({
    compactionID
});

compactionState, err := milvusClient.GetCompactionState(
    context.Background(), // ctx
    compactionID,         // compaction id
)
if err != nil {
    log.Fatal("failed to get compaction state:", err.Error())
}

milvusClient.getCompactionState(GetCompactionStateParam.newBuilder()
  .withCompactionID(compactionID)
  .build()
);

show compaction_state -c book

curl -X 'GET' \
  'http://localhost:9091/api/v1/compaction/state' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "compactionID": 434262132129005569
  }'

{"status":{},"state":2}

What’s next

• Learn more basic operations of Milvus:
  • Insert data into Milvus
  • Create a partition
  • Build an index for vectors
  • Conduct a vector search
  • Conduct a hybrid search