Load a Partition

This topic describes how to load a partition to memory. Loading partitions instead of the whole collection to memory can significantly reduce the memory usage. All search and query operations within Milvus are executed in memory.

Milvus 2.1 allows users to load a partition as multiple replicas to utilize the CPU and memory resources of extra query nodes. This feature boost the overall QPS and throughput with extra hardware. It is supported on PyMilvus in current release.

from pymilvus import Collection
collection = Collection("book")      # Get an existing collection.
collection.load(["novel"], replica_number=2)

# Or you can load a partition with the partition as an object
from pymilvus import Partition
partition = Partition("novel")       # Get an existing partition.
partition.load(replica_number=2)

await milvusClient.partitionManager.loadPartitions({
  collection_name: "book",
  partition_names: ["novel"],
});

err := milvusClient.LoadPartitions(
  context.Background(),   // ctx
  "book",                 // CollectionName
  []string{"novel"},      // partitionNames
  false                   // async
)
if err != nil {
  log.Fatal("failed to load partitions:", err.Error())
}

milvusClient.loadPartitions(
  LoadPartitionsParam.newBuilder()
          .withCollectionName("book")
          .withPartitionNames(["novel"])
          .build()
);

load -c book -p novel

curl -X 'POST' \
  'http://localhost:9091/api/v1/partitions/load' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "collection_name": "book",
    "partition_names": ["novel"],
    "replica_number": 1
  }'

Get replica information

You can check the information of the loaded replicas.

from pymilvus import Partition
partition = Partition("novel")       # Get an existing partition.
partition.load(replica_number=2)     # Load partition as 2 replicas
result = partition.get_replicas()
print(result)

Constraints

• Error will be returned at the attempt to load partition(s) when the parent collection is already loaded. Future releases will support releasing partitions from a loaded collection, and (if needed) then loading some other partition(s).
• “Load successfully” will be returned at the attempt to load the collection that is already loaded.
• Error will be returned at the attempt to load the collection when the child partition(s) is/are already loaded. Future releases will support loading the collection when some of its partitions are already loaded.
• Loading different partitions in a same collection via separate RPCs is not allowed.

What’s next

• Learn more basic operations of Milvus:
  • Insert data into Milvus
  • Build an index for vectors
  • Conduct a vector search
  • Conduct a hybrid search