Install Milvus Cluster with Helm (GPU)

This topic introduces how to deploy a Milvus cluster with GPU support using Helm on Kubernetes.

Prerequisites

• The compute capability of your GPU device is 6.1, 7.0, 7.5, or 8.0. To check whether your GPU device suffices the requirement, check Your GPU Compute Capability on the NVIDIA developer website.

• You have installed the NVIDIA driver for your GPU device on one of the supported Linux distributions and then the NVIDIA Container Toolkit following this guide.

• You have installed a Kubernetes cluster, and the kubectl command-line tool has been configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts.

• Check the requirements for hardware and software prior to your installation.

Start a Kubernetes cluster with GPU worker nodes

We recommend installing Milvus on a Kubernetes cluster with GPU worker nodes and using the default storage class provisioned.

1. Prepare GPU worker nodes

See Prepare GPU worker nodes for more information.

2. Enable GPU support on Kubernetes

See install nvidia-device-plugin with helm for more information.

After setting up, run kubectl describe node <gpu-worker-node> to view the GPU resources. The command output should be similar to the following:

Capacity:
  ...
  nvidia.com/gpu:     4
  ...
Allocatable:
  ...
  nvidia.com/gpu:     4
  ...

Note: In this example, we have set up a GPU worker node with 4 GPU cards.

3. Check the default storage class

Milvus relies on the default storage class to automatically provision volumes for data persistence. Run the following command to check storage classes:

$ kubectl get sc

The command output should be similar to the following:

NAME                   PROVISIONER                                     RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-path (default)   rancher.io/local-path                           Delete          WaitForFirstConsumer   false                  461d

Install Helm Chart for Milvus

Helm is a Kubernetes package manager that can help you deploy Milvus quickly.

1. Add Milvus to Helm's repository.

$ helm repo add milvus https://zilliztech.github.io/milvus-helm/

helm repo add zilliztech https://zilliztech.github.io/milvus-helm
helm repo update
# upgrade existing helm release
helm upgrade my-release zilliztech/milvus

2. Update your local chart repository.

$ helm repo update

Start Milvus

Start Milvus with Helm by specifying the release name, the chart, and parameters you expect to change. This topic uses my-release as the release name. To use a different release name, replace my-release in the command.

Milvus allows you to assign one or more GPU devices to Milvus.

Assign a single GPU device

Run the following commands to assign a single GPU device to Milvus:

cat <<EOF > custom-values.yaml
indexNode:
  resources:
    requests:
      nvidia.com/gpu: "1"
    limits:
      nvidia.com/gpu: "1"
queryNode:
  resources:
    requests:
      nvidia.com/gpu: "1"
    limits:
      nvidia.com/gpu: "1"
EOF

$ helm install my-release milvus/milvus -f custom-values.yaml

Assign multiple GPU devices

Run the following commands to assign multiple GPU devices to Milvus:

cat <<EOF > custom-values.yaml
indexNode:
  resources:
    requests:
      nvidia.com/gpu: "2"
    limits:
      nvidia.com/gpu: "2"
  extraEnv:
  - name: CUDA_VISIBLE_DEVICES
    value: "0, 1"
queryNode:
  resources:
    requests:
      nvidia.com/gpu: "2"
    limits:
      nvidia.com/gpu: "2"
  extraEnv:
  - name: CUDA_VISIBLE_DEVICES
    value: "0, 1"
EOF

In the configuration above, a total of four GPU cards are assigned, with two cards designated for the index node and the other two for the query node.

$ helm install my-release milvus/milvus -f custom-values.yaml

Check the status of the running pods.

$ kubectl get pods

After Milvus starts, the READY column displays 1/1 for all pods.

NAME                                             READY  STATUS   RESTARTS  AGE
my-release-etcd-0                                1/1    Running   0        3m23s
my-release-etcd-1                                1/1    Running   0        3m23s
my-release-etcd-2                                1/1    Running   0        3m23s
my-release-milvus-datacoord-6fd4bd885c-gkzwx     1/1    Running   0        3m23s
my-release-milvus-datanode-68cb87dcbd-4khpm      1/1    Running   0        3m23s
my-release-milvus-indexcoord-5bfcf6bdd8-nmh5l    1/1    Running   0        3m23s
my-release-milvus-indexnode-5c5f7b5bd9-l8hjg     1/1    Running   0        3m24s
my-release-milvus-proxy-6bd7f5587-ds2xv          1/1    Running   0        3m24s
my-release-milvus-querycoord-579cd79455-xht5n    1/1    Running   0        3m24s
my-release-milvus-querynode-5cd8fff495-k6gtg     1/1    Running   0        3m24s
my-release-milvus-rootcoord-7fb9488465-dmbbj     1/1    Running   0        3m23s
my-release-minio-0                               1/1    Running   0        3m23s
my-release-minio-1                               1/1    Running   0        3m23s
my-release-minio-2                               1/1    Running   0        3m23s
my-release-minio-3                               1/1    Running   0        3m23s
my-release-pulsar-autorecovery-86f5dbdf77-lchpc  1/1    Running   0        3m24s
my-release-pulsar-bookkeeper-0                   1/1    Running   0        3m23s
my-release-pulsar-bookkeeper-1                   1/1    Running   0        98s
my-release-pulsar-broker-556ff89d4c-2m29m        1/1    Running   0        3m23s
my-release-pulsar-proxy-6fbd75db75-nhg4v         1/1    Running   0        3m23s
my-release-pulsar-zookeeper-0                    1/1    Running   0        3m23s
my-release-pulsar-zookeeper-metadata-98zbr       0/1   Completed  0        3m24s

Connect to Milvus

Verify which local port the Milvus server is listening on. Replace the pod name with your own.

$ kubectl get pod my-release-milvus-proxy-6bd7f5587-ds2xv --template
='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}'
19530

Open a new terminal and run the following command to forward a local port to the port that Milvus uses. Optionally, omit the designated port and use :19530 to let kubectl allocate a local port for you so that you don't have to manage port conflicts.

$ kubectl port-forward service/my-release-milvus 27017:19530
Forwarding from 127.0.0.1:27017 -> 19530

By default, ports forward by kubectl only listen on localhost. Use flag address if you want Milvus server to listen on selected IP or all addresses.

$ kubectl port-forward --address 0.0.0.0 service/my-release-milvus 27017:19530
Forwarding from 0.0.0.0:27017 -> 19530

Uninstall Milvus

Run the following command to uninstall Milvus.

$ helm uninstall my-release

What's next

Having installed Milvus, you can:

• Check Hello Milvus to run an example code with different SDKs to see what Milvus can do.

• Learn the basic operations of Milvus:

  • Connect to Milvus server
  • Manage Databases
  • Create a collection
  • Create a partition
  • Insert data
  • Conduct a vector search

• Upgrade Milvus Using Helm Chart.

• Scale your Milvus cluster.

• Deploy your Milvu cluster on clouds:

  • Amazon EC2
  • Amazon EKS

• Explore Milvus Backup, an open-source tool for Milvus data backups.

• Explore Birdwatcher, an open-source tool for debugging Milvus and dynamic configuration updates.

• Explore Attu, an open-source GUI tool for intuitive Milvus management.

• Monitor Milvus with Prometheus.