Requirements for Installing Milvus with GPU

This page lists the hardware and software requirements to set up Milvus with GPU support.

Compute capability

The compute capability of your GPU device must be one of the following: 6.0, 7.0, 7.5, 8.0, 8.6, 9.0.

To check whether your GPU device suffices the requirement, check Your GPU Compute Capability on the NVIDIA developer website.

NVIDIA driver

The NVIDIA driver for your GPU device must be on one of the supported Linux distributions, and the NVIDIA Container Toolkit has been installed by following this guide.

For Ubuntu 22.04 users, you can install the driver and the container toolkit with the following commands:

$ sudo apt install --no-install-recommends nvidia-headless-545 nvidia-utils-545

For other OS users, refer to the official installation guide.

You can check whether the driver has been installed correctly by running the following command:

$ modinfo nvidia | grep "^version"
version:        545.29.06

You are recommended to use the drivers of version 545 and above.

Software requirements

It is recommended that you run the Kubernetes cluster on Linux platforms.

• kubectl is the command-line tool for Kubernetes. Use a kubectl version that is within one minor version difference of your cluster. Using the latest version of kubectl helps avoid unforeseen issues.
• minikube is required when running Kubernetes cluster locally. minikube requires Docker as a dependency. Ensure that you install Docker before installing Milvus using Helm. See Get Docker for more information.

Operating system	Software	Note
Linux platforms	Kubernetes 1.16 or later kubectl Helm 3.0.0 or later minikube (for Milvus standalone) Docker 19.03 or later (for Milvus standalone)	See Helm Docs for more information.

FAQs

How can I start a K8s cluster locally for test purposes?

You can use tools like minikube, kind, and Kubeadm, to quickly set up a Kubernetes cluster locally. The following procedure uses minikube as an example.

1. Download minikube

Go to the Get Started page, check whether you have met the conditions listed in the What you’ll need section, click on the buttons that describe your target platform, and copy the commands to download and install the binary.

2. Start a K8s cluster using minikube

$ minikube start

3. Check the status of the K8s cluster

You can check the status of the K8s cluster installed using the following command.

$ kubectl cluster-info

How can I start a K8s cluster with GPU worker nodes?

If you prefer to use GPU-enabled worker nodes, you can follow the steps below to create a K8s cluster with GPU worker nodes. We recommend installing Milvus on a K8s cluster with GPU worker nodes and using the default storage class provisioned.

1. Prepare GPU worker nodes

To use GPU-enabled worker nodes, follow steps in Prepare your GPU nodes.

2. Enable GPU support on K8s

Deploy the nvidia-device-plugin with Helm by following these steps.

After setting up, view the GPU resources with the following command. Replace <gpu-worker-node> with the actual node name.

  $ kubectl describe node <gpu-worker-node>

  Capacity:
  ...
  nvidia.com/gpu:     4
  ...
  Allocatable:
  ...
  nvidia.com/gpu:     4
  ...
  ```