Run Milvus with GPU Support Using Docker Compose

This page illustrates how to start a Milvus instance with GPU support using Docker Compose.

Prerequisites

• Install Docker.
• Check the requirements for hardware and software prior to your installation.

Install Milvus

To install Milvus with GPU support using Docker Compose, follow these steps.

1. Download and configure the YAML file

Download milvus-standalone-docker-compose-gpu.yml and save it as docker-compose.yml manually, or with the following command.

$ wget https://github.com/milvus-io/milvus/releases/download/v2.4.13/milvus-standalone-docker-compose-gpu.yml -O docker-compose.yml

You need to make some changes to the environment variables of the standalone service in the YAML file as follows:

• To assign a specific GPU device to Milvus, locate the deploy.resources.reservations.devices[0].devices_ids field in the definition of the standalone service and replace its value with the ID of the desired GPU. You can use the nvidia-smi tool, included with NVIDIA GPU display drivers, to determine the ID of a GPU device. Milvus supports multiple GPU devices.

Assign a single GPU device to Milvus:

...
standalone:
  ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            capabilities: ["gpu"]
            device_ids: ["0"]
...

Assign multiple GPU devices to Milvus:

...
standalone:
  ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            capabilities: ["gpu"]
            device_ids: ['0', '1']
...

2. Start Milvus

In the directory that holds docker-compose.yml, start Milvus by running:

$ sudo docker compose up -d

Creating milvus-etcd  ... done
Creating milvus-minio ... done
Creating milvus-standalone ... done

After starting up Milvus,

• Containers named milvus-standalone, milvus-minio, and milvus-etcd are up.
  • The milvus-etcd container does not expose any ports to the host and maps its data to volumes/etcd in the current folder.
  • The milvus-minio container serves ports 9090 and 9091 locally with the default authentication credentials and maps its data to volumes/minio in the current folder.
  • The milvus-standalone container serves ports 19530 locally with the default settings and maps its data to volumes/milvus in the current folder.

You can check if the containers are up and running using the following command:

$ sudo docker compose ps

      Name                     Command                  State                            Ports
--------------------------------------------------------------------------------------------------------------------
milvus-etcd         etcd -advertise-client-url ...   Up             2379/tcp, 2380/tcp
milvus-minio        /usr/bin/docker-entrypoint ...   Up (healthy)   9000/tcp
milvus-standalone   /tini -- milvus run standalone   Up             0.0.0.0:19530->19530/tcp, 0.0.0.0:9091->9091/tcp

If you have assigned multiple GPU devices to Milvus in docker-compose.yml, you can specify which GPU device is visible or available for use.

Make GPU device 0 visible to Milvus:

$ CUDA_VISIBLE_DEVICES=0 ./milvus run standalone

Make GPU devices 0 and 1 visible to Milvus:

$ CUDA_VISIBLE_DEVICES=0,1 ./milvus run standalone

You can stop and delete this container as follows.

# Stop Milvus
$ sudo docker compose down

# Delete service data
$ sudo rm -rf volumes

Configure memory pool

After Milvus is up and running, you can customize the memory pool by modifying the initMemSize and maxMemSize settings in the milvus.yaml file.

To confgiure the memory pool, modify the initMemSize and maxMemSize settings in the milvus.yaml file as follows.

1. Use the following command to copy milvus.yaml from the Milvus container to your local machine. Replace <milvus_container_id> with your actual Milvus container ID.

   docker cp <milvus_container_id>:/milvus/configs/milvus.yaml milvus.yaml
   

2. Open the copied milvus.yaml file with your preferred text editor. For example, using vim:

   vim milvus.yaml
   

3. Edit the initMemSize and maxMemSize settings as needed and save your changes:

   ...
   gpu:
     initMemSize: 0
     maxMemSize: 0
   ...
   

   • initMemSize: Initial size of the memory pool. Defaults to 1024.
   • maxMemSize: Maximum size of the memory pool. Defaults to 2048.

4. Use the following command to copy the modified milvus.yaml file back to the Milvus container. Replace <milvus_container_id> with your actual Milvus container ID.

   docker cp milvus.yaml <milvus_container_id>:/milvus/configs/milvus.yaml
   

5. Restart the Milvus container to apply the changes:

   docker stop <milvus_container_id>
   docker start <milvus_container_id>
   

What’s next

Having installed Milvus in Docker, you can:

• Check Quickstart to see what Milvus can do.

• Learn the basic operations of Milvus:

  • Manage Databases
  • Manage Collections
  • Manage Partitions
  • Insert, Upsert & Delete
  • Single-Vector Search
  • Hybrid Search

• Upgrade Milvus Using Helm Chart.

• Scale your Milvus cluster.

• Deploy your Milvu cluster on clouds:

  • Amazon EKS
  • Google Cloud
  • Microsoft Azure

• Explore Milvus Backup, an open-source tool for Milvus data backups.

• Explore Birdwatcher, an open-source tool for debugging Milvus and dynamic configuration updates.

• Explore Attu, an open-source GUI tool for intuitive Milvus management.

• Monitor Milvus with Prometheus.