Configure Milvus with Docker Compose

This topic describes how to configure Milvus components and its third-party dependencies with Docker Compose.

In current release, all parameters take effect only after Milvus restarts.

Download a configuration file

Download milvus.yaml directly or with the following command.

$ wget https://raw.githubusercontent.com/milvus-io/milvus/v2.3.0-beta/configs/milvus.yaml

Modify the configuration file

Configure your Milvus instance to suit your application scenarios by adjusting corresponding parameters in milvus.yaml.

Check the following links for more information about each parameter.

Sorted by:

Components or dependencies Configuration purposes

Dependencies	Components
etcd MinIO or S3 Pulsar RocksMQ	Root coord Proxy Query coord Query node Index coord Index node Data coord Data node Local storage Log Message channel Common Knowhere Quota and Limits

Purpose	Parameters
Performance tuning	`queryNode.gracefulTime` `rootCoord.minSegmentSizeToEnableIndex` `dataCoord.segment.maxSize` `dataCoord.segment.sealProportion` `dataNode.flush.insertBufSize` `queryCoord.autoHandoff` `queryCoord.autoBalance` `localStorage.enabled`
Data and meta	`common.retentionDuration` `rocksmq.retentionTimeInMinutes` `dataCoord.enableCompaction` `dataCoord.enableGarbageCollection` `dataCoord.gc.dropTolerance`
Administration	`log.level` `log.file.rootPath` `log.file.maxAge` `minio.accessKeyID` `minio.secretAccessKey`
Quota and Limits	`quotaAndLimits.ddl.enabled` `quotaAndLimits.ddl.collectionRate` `quotaAndLimits.ddl.partitionRate` `quotaAndLimits.indexRate.enabled` `quotaAndLimits.indexRate.max` `quotaAndLimits.flushRate.enabled` `quotaAndLimits.flush.max` `quotaAndLimits.compation.enabled` `quotaAndLimits.compaction.max` `quotaAndLimits.dml.enabled` `quotaAndLimits.dml.insertRate.max` `quotaAndLimits.dml.deleteRate.max` `quotaAndLimits.dql.enabled` `quotaAndLimits.dql.searchRate.max` `quotaAndLimits.dql.queryRate.max` `quotaAndLimits.limitWriting.ttProtection.enabled` `quotaAndLimits.limitWriting.ttProtection.maxTimeTickDelay` `quotaAndLimits.limitWriting.memProtection.enabled` `quotaAndLimits.limitWriting.memProtection.dataNodeMemoryLowWaterLevel` `quotaAndLimits.limitWriting.memProtection.queryNodeMemoryLowWaterLevel` `quotaAndLimits.limitWriting.memProtection.dataNodeMemoryHighWaterLevel` `quotaAndLimits.limitWriting.memProtection.queryNodeMemoryHighWaterLevel` `quotaAndLimits.limitWriting.diskProtection.enabled` `quotaAndLimits.limitWriting.diskProtection.diskQuota` `quotaAndLimits.limitWriting.forceDeny` `quotaAndLimits.limitReading.queueProtection.enabled` `quotaAndLimits.limitReading.queueProtection.nqInQueueThreshold` `quotaAndLimits.limitReading.queueProtection.queueLatencyThreshold` `quotaAndLimits.limitReading.resultProtection.enabled` `quotaAndLimits.limitReading.resultProtection.maxReadResultRate` `quotaAndLimits.limitReading.forceDeny`

Download an installation file

Download the installation file for Milvus standalone or cluster, and save it as docker-compose.yml.

If you want to install Milvus with GPU support, download the installation file for Milvus standalone or cluster, and save it as docker-compose.yml.

You can also simply run the following command.

# For Milvus standalone

# With CPU-only support
$ wget https://github.com/milvus-io/milvus/releases/download/v2.3.0-beta/milvus-standalone-docker-compose.yml -O docker-compose.yml

# With GPU support
$ wget https://github.com/milvus-io/milvus/releases/download/v2.3.0-beta/milvus-standalone-docker-compose-gpu.yml -O docker-compose.yml

# For Milvus cluster

# With CPU-only support
$ wget https://github.com/milvus-io/milvus/releases/download/v2.3.0-beta/milvus-cluster-docker-compose.yml -O docker-compose.yml

# With GPU support
wget https://github.com/milvus-io/milvus/releases/download/v2.3.0-beta/milvus-cluster-docker-compose-gpu.yml -O docker-compose.yml

To use Milvus with GPU support, ensure that

Docker Compose version is v1.28.0 or later.
NVIDIA Tesla driver version is 450.80.02 or later.
NVIDIA GTX driver version is 510.47.03 or later.
NVIDIA Container Toolkit has been installed. For details, refer to Setting up NVIDIA Container Toolkit.

Modify the installation file

In docker-compose.yml, add a volumes section under each Milvus component, i.e. RootCoord, DataCoord, DataNode, QueryCoord, QueryNode, IndexCoord, IndexNode, and Proxy.

Map the local path to your milvus.yaml file onto the corresponding docker container paths to the configuration files /milvus/configs/milvus.yaml under all volumes sections.

...
proxy:
    container_name: milvus-proxy
    # Change this to "milvusdb/milvus:v2.3.0-beta 
    # if you prefer to use Milvus without GPU support"
    image: milvusdb/milvus:v2.3.0-gpu-beta 
    command: ["milvus", "run", "proxy"]
    volumes:       # Add a volumes section.
      - /local/path/to/your/milvus.yaml:/milvus/configs/milvus.yaml   # Map the local path to the container path
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
      PULSAR_ADDRESS: pulsar://pulsar:6650
    ports:
      - "19530:19530"
...

Data are stored in the /volumes folder according to the default configuration in docker-compose.yml. To change the folder to store data, edit docker-compose.yml or run $ export DOCKER_VOLUME_DIRECTORY=.

Start Milvus

Having finished modifying the configuration file and installation file, you can then start Milvus.

$ sudo docker-compose up -d

What's next

Learn how to manage the following Milvus dependencies with Docker Compose or Helm: