Scale Milvus Dependencies

Milvus relies on various dependencies such as MinIO, Kafka, Pulsar, and etcd. Scaling these components can enhance Milvus's adaptability to different requirements.

For Milvus Operator users, please also refer to Manage Dependencies For Milvus Operator

Scale MinIO

Increase resources per MinIO pod

MinIO, an object storage system used by Milvus, can have its CPU and memory resources increased for each pod.

# new-values.yaml
minio:
  resources:
     limits:
       cpu: 2
       memory: 8Gi

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

You can also increase the disk capacity for the MioIO cluster by manually changing the value of spec.resources.requests.storage for each MioIO Persistent Volume Claim (PVC). Note that your default storage class should allow volume expansion.

Add an extra MinIO server pool (Recommended)

You are advised to add an extra MioIO server pool for your Milvus instance.

# new-values.yam;
minio:
  zones: 2

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

This adds an additional server pool to your MinIO cluster, allowing Milvus to write to the MinIO server pool based on the free disk capacity of each server pool. For example, if a group of three pools has a total of 10 TiB free space distributed across the pools as follows:

	Free space	Write possiblity
Pool A	3 TiB	30% (3/10)
Pool B	2 TiB	20% (2/10)
Pool C	5 TiB	50% (5/10)

MinIO does not automatically rebalance objects across new server pools. You can manually initiate a rebalance procedure with mc admin rebalance if needed.

Kafka

Increase resource per Kafka broker pod

Enhance the Kafka broker capacity by adjusting the CPU and memory resources for each broker pod.

# new-values.yaml
kafka:
  resources:
     limits:
        cpu: 2
        memory: 12Gi

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

You can also increase the disk capacity for the Kafka cluster by manually changing the value of spec.resources.requests.storage for each Kafka Persistent Volume Claim (PVC). Ensure your default storage class allows volume expansion.

Add an extra Kafka broker pool (Recommended)

You are advised to add an extra Kafka server pool for your Milvus instance.

# new-values.yaml
kafka:
  replicaCount: 4

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

This will add an extra broker to your Kafka cluster.

Kafka does not automatically rebalance topics across all brokers. Manually rebalance topics/partitions across all Kafka brokers using bin/kafka-reassign-partitions.sh after logging into each Kafka broker pod if needed.

Pulsar

Pulsar separates computation and storage. You can independently increase the capacity of Pulsar brokers (computation) and Pulsar bookies (storage).

Increase resources per Pulsar broker pod

# new-values.yaml
pulsar:
  broker:
    resources:
       limits:
         cpu: 4
         memory: 16Gi

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

Increase resources per Pulsar bookie pod

# new-values.yaml
pulsar:
  bookkeeper:
    resources:
       limits:
         cpu: 4
         memory: 16Gi

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

You can also increase the disk capacity for the Pulsar cluster by manually changing the value of spec.resources.requests.storage for each Pulsar bookie's Persistent Volume Claim (PVC). Note that your default storage class should allow volume expansion.

A Pulsar bookie pod has two types of storage: journal and legers. For the journal type of storage, consider using ssd or gp3 as the storage class.

Add an extra Pulsar broker pod

# new-values.yaml
pulsar:
  broker:
    replicaCount: 3

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

Add an extra Pulsar bookie pod (Recommended)

# new-values.yaml
pulsar:
  bookkeeper:
    replicaCount: 3

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

etcd

Increase resources per etcd pod (recommended)

# new-values.yaml
etcd:
  resources:
     limits:
       cpu: 2
       memory: 8Gi

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus

Add extra etcd pods

The total number of etcd pods should be in odd numbers.

# new-values.yaml
etcd:
  replicaCount: 5

After saving the file, apply the changes with the following command:

helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus