Install Milvus Standalone with Docker Compose (GPU)

This guide describes the hardware/software preparations and the procedure to install Milvus with GPU support enabled using Docker Compose.

Milvus now can use GPU devices to build indexes and perform ANN searches thanks to the contributions from NVIDIA.

Prerequisites

Ensure that

• The compute capability of your GPU device is 6.1, 7.0, 7.5, or 8.0. To check whether your GPU device suffices the requirement, check Your GPU Compute Capability on the NVIDIA developer website.

• You have installed the NVIDIA driver for your GPU device on one of the supported Linux distributions and then the NVIDIA Container Toolkit following this guide.

• Your hardware and software meet the specification requirements listed on this page.

Download the YAML file

Download milvus-standalone-docker-compose-gpu.yml and save it as docker-compose.yml manually, or with the following command.

$ wget https://github.com/milvus-io/milvus/releases/download/v2.3.1/milvus-standalone-docker-compose-gpu.yml -O docker-compose.yml

You need to make some changes to the environment variables of the standalone service in the YAML file as follows:

• To assign a specific GPU device to Milvus, locate the deploy.resources.reservations.devices[0].devices_ids field in the definition of the standalone service and replace its value with the ID of the desired GPU. You can use the nvidia-smi tool, included with NVIDIA GPU display drivers, to determine the ID of a GPU device. Please note that only one GPU device can be set at this time.

• Add KNOWHERE_GPU_MEM_POOL_SIZE in the environment section of the standalone service and set its value to reflect the size of a shared display memory pool. The format for the size is initialSize;maximumSize, where initialSize represents the initial size of the memory pool and maximumSize represents its maximum size. Both values should be set in MB. Milvus uses this field to allocate display memory to each process.

...
standalone:
  environment:
    ...
    KNOWHERE_GPU_MEM_POOL_SIZE: 2048:4096
  ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            capabilities: ["gpu"]
            device_ids: ["0"]
...

Start Milvus

In the directory that holds docker-compose.yml, start Milvus by running:

$ sudo docker compose up -d

Creating milvus-etcd  ... done
Creating milvus-minio ... done
Creating milvus-standalone ... done

Now check if the containers are up and running.

$ sudo docker compose ps

After Milvus standalone starts, there will be three docker containers running, including the Milvus standalone service and its two dependencies.

      Name                     Command                  State                            Ports
--------------------------------------------------------------------------------------------------------------------
milvus-etcd         etcd -advertise-client-url ...   Up             2379/tcp, 2380/tcp
milvus-minio        /usr/bin/docker-entrypoint ...   Up (healthy)   9000/tcp
milvus-standalone   /tini -- milvus run standalone   Up             0.0.0.0:19530->19530/tcp, 0.0.0.0:9091->9091/tcp

Connect to Milvus

Verify which local port the Milvus server is listening on. Replace the container name with your own.

$ docker port milvus-standalone 19530/tcp

You can connect to Milvus using the local IP address and port number returned by this command.

Stop Milvus

To stop Milvus standalone, run:

sudo docker-compose down

To delete data after stopping Milvus, run:

sudo rm -rf  volumes

What's next

Having installed Milvus, you can:

• Check Hello Milvus to run an example code with different SDKs to see what Milvus can do.

• Check In-memory Index for more about GPU-compatible index types.

• Learn the basic operations of Milvus:

  • Connect to Milvus server
  • Manage Databases
  • Create a collection
  • Create a partition
  • Insert data
  • Conduct a vector search

• Explore Milvus Backup, an open-source tool for Milvus data backups.

• Explore Birdwatcher, an open-source tool for debugging Milvus and dynamic configuration updates.

• Explore Attu, an open-source GUI tool for intuitive Milvus management.

• Monitor Milvus with Prometheus