Install Milvus Cluster with Docker Compose

This topic describes how to install a Milvus cluster with GPU support using Docker Compose.

Prerequisites

Before installing Milvus with GPU support, make sure you have the following prerequisites:

• The compute capability of your GPU device is 7.0、7.5、8.0、8.6、8.9、9.0. To check whether your GPU device suffices the requirement, check Your GPU Compute Capability on the NVIDIA developer website.

• You have installed the NVIDIA driver for your GPU device on one of the supported Linux distributions and then the NVIDIA Container Toolkit following this guide.

  For Ubuntu 22.04 users, you can install the driver and the container toolkit with the following commands:

  $ sudo apt install --no-install-recommends nvidia-headless-545 nvidia-utils-545
  

  For other OS users, please refer to the official installation guide.

  You can check whether the driver has been installed correctly by running the following command:

  $ modinfo nvidia | grep "^version"
  version:        535.161.07
  

  You are recommended to use the drivers of version 535 and above.

• You have installed a Kubernetes cluster, and the kubectl command-line tool has been configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts.

• You have installed Docker and Docker Compose on your local machine.

• Check the requirements for hardware and software requirements before installing Milvus.

  • For the users using MacOS 10.14 or later, set the Docker virtual machine (VM) to use a minimum of 2 virtual CPUs (vCPUs) and 8 GB of initial memory. Otherwise, the installation might fail.

Install Milvus Standalone with Docker Compose

To install Milvus standalone with Docker Compose, follow these steps:

Download and configure the YAML file

Download milvus-standalone-docker-compose-gpu.yml and save it as docker-compose.yml manually, or with the following command.

$ wget https://github.com/milvus-io/milvus/releases/download/v2.3.21/milvus-standalone-docker-compose-gpu.yml -O docker-compose.yml

You need to make some changes to the environment variables of the standalone service in the YAML file as follows:

• To assign a specific GPU device to Milvus, locate the deploy.resources.reservations.devices[0].devices_ids field in the definition of the standalone service and replace its value with the ID of the desired GPU. You can use the nvidia-smi tool, included with NVIDIA GPU display drivers, to determine the ID of a GPU device. Milvus supports multiple GPU devices.

Assign a single GPU device to Milvus:

...
standalone:
  ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            capabilities: ["gpu"]
            device_ids: ["0"]
...

Assign multiple GPU devices to Milvus:

...
standalone:
  ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            capabilities: ["gpu"]
            device_ids: ['0', '1']
...

Start Milvus

In the directory that holds docker-compose.yml, start Milvus by running:

$ sudo docker compose up -d

Creating milvus-etcd  ... done
Creating milvus-minio ... done
Creating milvus-standalone ... done

Now check if the containers are up and running.

$ sudo docker compose ps

Verify the Installation

After Milvus standalone starts, there will be three docker containers running, including the Milvus standalone service and its two dependencies.

      Name                     Command                  State                            Ports
--------------------------------------------------------------------------------------------------------------------
milvus-etcd         etcd -advertise-client-url ...   Up             2379/tcp, 2380/tcp
milvus-minio        /usr/bin/docker-entrypoint ...   Up (healthy)   9000/tcp
milvus-standalone   /tini -- milvus run standalone   Up             0.0.0.0:19530->19530/tcp, 0.0.0.0:9091->9091/tcp

If you have assigned multiple GPU devices to Milvus in docker-compose.yml, you can specify which GPU device is visible or available for use.

Make GPU device 0 visible to Milvus:

CUDA_VISIBLE_DEVICES=0 ./milvus run standalone

Make GPU devices 0 and 1 visible to Milvus:

CUDA_VISIBLE_DEVICES=0,1 ./milvus run standalone

Connect to Milvus

Verify which local port the Milvus server is listening on. Replace the container name with your own.

$ docker port milvus-standalone 19530/tcp

Please refer to Hello Milvus, then run the example code.

Configure memory pool

After Milvus is up and running, you can customize the memory pool by modifying the initMemSize and maxMemSize settings in the milvus.yaml file.

To confgiure the memory pool, modify the initMemSize and maxMemSize settings in the milvus.yaml file as follows.

1. Use the following command to copy milvus.yaml from the Milvus container to your local machine. Replace <milvus_container_id> with your actual Milvus container ID.

   docker cp <milvus_container_id>:/milvus/configs/milvus.yaml milvus.yaml
   

2. Open the copied milvus.yaml file with your preferred text editor. For example, using vim:

   vim milvus.yaml
   

3. Edit the initMemSize and maxMemSize settings as needed and save your changes:

   ...
   gpu:
     initMemSize: 0
     maxMemSize: 0
   ...
   

   • initMemSize: Initial size of the memory pool. Defaults to 1024.
   • maxMemSize: Maximum size of the memory pool. Defaults to 2048.

4. Use the following command to copy the modified milvus.yaml file back to the Milvus container. Replace <milvus_container_id> with your actual Milvus container ID.

   docker cp milvus.yaml <milvus_container_id>:/milvus/configs/milvus.yaml
   

5. Restart the Milvus container to apply the changes:

   docker stop <milvus_container_id>
   docker start <milvus_container_id>
   

Stop Milvus

To stop Milvus standalone, run:

sudo docker compose down

To delete data after stopping Milvus, run:

sudo rm -rf  volumes

What’s next

Having installed Milvus, you can:

• Check Hello Milvus to run an example code with different SDKs to see what Milvus can do.
• Check In-memory Index for more about CPU-compatible index types.
• Learn the basic operations of Milvus:
  • Connect to Milvus server
  • Manage Databases
  • Create a collection
  • Create a partition
  • Insert data
  • Conduct a vector search
• Explore Milvus Backup, an open-source tool for Milvus data backups.
• Explore Birdwatcher, an open-source tool for debugging Milvus and dynamic configuration updates.
• Explore Attu, an open-source GUI tool for intuitive Milvus management.
• Monitor Milvus with Prometheus