Install and Start GPU-enabled Milvus

Prerequisites

System Requirements

Operating system	Supported versions
CentOS	7.5 or higher
Ubuntu LTS	18.04 or higher

Hardware Requirements

Component	Recommended configuration
CPU	Intel CPU Sandy Bridge or higher.
CPU Instruction Set	SSE42 AVX AVX2 AVX512
GPU	NVIDIA Pascal or higher
RAM	8 GB or more (depends on data volume)
Hard Drive	SATA 3.0 SSD or higher

Software Requirements

Software	Version
Docker	19.03 or higher
NVIDIA Driver	418 or higher
NVIDIA Container Toolkit	NVIDIA-Container-Toolkit

Confirm Docker Status

Confirm that the Docker daemon is running in the background:

$ sudo docker info

If you do not see the server listed, start the Docker daemon.
On Linux, Docker needs sudo privileges. To run Docker commands without sudo privileges, create a docker group and add your users (see Post-installation Steps for Linux for details).

Pull Milvus Image

Pull the GPU-enabled image:

$ sudo docker pull milvusdb/milvus:1.1.0-gpu-d050721-5e559c

If you cannot use your host to acquire Docker images and configuration files online because of network restrictions, please acquire them online from another available host, save them as a TAR file, pass it on to your local machine, and then load the TAR file as a Docker image:
Click here to view the sample code.
1. Save the Docker image as a TAR file, and pass it on to your local machine:
  $ docker save milvusdb/milvus > milvus_image.tar
2. Load the TAR file as a Docker image:
  $ docker load < milvus_image.tar
If pulling the docker image is too slow or keeps failing, see Operational FAQ for solutions.

Download Configuration Files

$ mkdir -p /home/$USER/milvus/conf
$ cd /home/$USER/milvus/conf
$ wget https://raw.githubusercontent.com/milvus-io/milvus/v1.1.0/core/conf/demo/server_config.yaml

If you cannot download configuration files via the wget command, you can create a server_config.yaml file under /home/$USER/milvus/conf, and then copy the content from server config to it.

After you downloaded the configuration file, you must set enable to true in gpu section of server_config.yaml.

Start Docker Container

Before starting Docker container, you must set enable to true in gpu section of server_config.yaml.

Start Docker container and map the paths to the local files to the container:

$ sudo docker run -d --name milvus_gpu_1.1.0 --gpus all \
-p 19530:19530 \
-p 19121:19121 \
-v /home/$USER/milvus/db:/var/lib/milvus/db \
-v /home/$USER/milvus/conf:/var/lib/milvus/conf \
-v /home/$USER/milvus/logs:/var/lib/milvus/logs \
-v /home/$USER/milvus/wal:/var/lib/milvus/wal \
milvusdb/milvus:1.1.0-gpu-d050721-5e559c

The docker run options used in the above command are defined as follows:

-d: Runs container in the background and prints container ID.
--name: Assigns a name to the container.
--gpus: Assigns GPU devices to the container (all represents all GPUs).
-p: Publishes a container’s port(s) to the host.
-v: Mounts the directory into the container.

Confirm the running state of Milvus:

$ sudo docker ps

If the Milvus server does not start up properly, check the error logs:

$ sudo docker logs milvus_gpu_1.1.0

FAQ

Can I install Milvus on Windows?

Yes, so long as you have set up a Docker environment on your operating system.

Why does Milvus return Illegal instruction during startup?

If your CPU does not support SSE42, AVX, AVX2, or AVX512, Milvus cannot start properly. You can use cat /proc/cpuinfo to check the supported instruction sets.

How to migrate data in Milvus?

For details, see data migration.

Data formats of different versions may not be compatible with each other. The current data format is backward compatible with Milvus v0.7.0.

Is Docker the only way to install and run Milvus?

No. You can also build Milvus from source code in Linux. See Build Milvus from source code for more information.

How to set nlist and nprobe for IVF indexes?

In general terms, the recommended value of nlist is 4 × sqrt(n), where n is the total number of entities in a segment.

Determining nprobe is a trade-off between search performance and accuracy, and based on your dataset and scenario. It is recommended to run several rounds of tests to determine the value of nprobe.

The following charts are from a test running on the sift50m dataset and IVF_SQ8 index. The test compares search performance and recall rate between different nlist/nprobe pairs.

We only show the results of GPU-enabled Milvus here, because the two distributions of Milvus show similar results.

Key takeaways: This test shows that the recall rate increases with the nlist/nprobe pair.

Key takeaways: When nlist is 4096 and nprobe 128, Milvus shows the best search performance.

How to assign GPU devices to index/search?

In server_config.yaml file under /home/$USER/milvus/conf, you can assign GPU devices to index/search. See example:

gpu:
  enable: true
  cache_size: 10GB
  gpu_search_threshold: 0
  search_devices:
    - gpu0
    - gpu1
  build_index_devices:
    - gpu2
    - gpu3

GPU 0,1,2,3 are first 4 GPUs from the list of GPU devices assigned to the docker container.

What's next

If you're just getting started with Milvus:
If you're ready to run Milvus in production:
- Build a monitoring and alerting system to check real-time application performance.
- Tune Milvus performance through configuration.
If you want to run Milvus on machines without an Nvidia GPU:
- Install CPU-only Milvus