Deploying Milvus on Kubernetes: A Step-by-Step Guide for Kubernetes Users

Milvus is an open-source vector database designed to store, index, and search massive amounts of unstructured data through vector representations, making it perfect for AI-driven applications, such as similarity search, semantic search, retrieval augmented generation (RAG), recommendation engines, and other machine learning tasks.

But what makes Milvus even more powerful is its seamless integration with Kubernetes. If you’re a Kubernetes aficionado, you know the platform is perfect for orchestrating scalable, distributed systems. Milvus takes full advantage of Kubernetes’ capabilities, allowing you to easily deploy, scale, and manage distributed Milvus clusters. This guide will provide a clear, step-by-step walkthrough for setting up Milvus on Kubernetes using the Milvus Operator.

Prerequisites

Before we begin, ensure you have the following prerequisites in place:

• A Kubernetes cluster up and running. If you’re testing locally, minikube is a great choice.

• kubectl installed and configured to interact with your Kubernetes cluster.

• Familiarity with basic Kubernetes concepts like pods, services, and deployments.

Step 1: Installing Minikube (For Local Testing)

If you need to set up a local Kubernetes environment, minikube is the tool for you. Official installation instructions are on the minikube getting started page.

1. Install Minikube

Visit the minikube releases page and download the appropriate version for your operating system. For macOS/Linux, you can use the following command:

$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
$ sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64

2. Start Minikube

$ minikube start

3. Interact with Cluster

Now, you can interact with your clusters with the kubectl inside minikube. If you haven’t installed kubectl, minikube will download the appropriate version by default.

$ minikube kubectl cluster-info

Alternatively, you can create a symbolic link to minikube’s binary named kubectl for easier usage.

$ sudo ln -s $(which minikube) /usr/local/bin/kubectl
$ kubectl cluster-info

Step 2: Configuring the StorageClass

In Kubernetes, a StorageClass defines the types of storage available for your workloads, providing flexibility in managing different storage configurations. Before proceeding, you must ensure a default StorageClass is available in your cluster. Here’s how to check and configure one if necessary.

1. Check Installed StorageClasses

To see the available StorageClasses in your Kubernetes cluster, run the following command:

$ kubectl get sc

This will display the list of storage classes installed in your cluster. If a default StorageClass is already configured, it will be marked with (default).

2. Configure a Default StorageClass (if necessary)

If no default StorageClass is set, you can create one by defining it in a YAML file. Use the following example to create a default StorageClass:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: default-storageclass
provisioner: k8s.io/minikube-hostpath

This YAML configuration defines a StorageClass called default-storageclass that uses the minikube-hostpath provisioner, commonly used in local development environments.

3. Apply the StorageClass

Once the default-storageclass.yaml file is created, apply it to your cluster using the following command:

$ kubectl apply -f default-storageclass.yaml

This will set up the default StorageClass for your cluster, ensuring that your storage needs are properly managed in the future.

Step 3: Installing Milvus Using the Milvus Operator

The Milvus Operator simplifies deploying Milvus on Kubernetes, managing the deployment, scaling, and updates. Before installing the Milvus Operator, you’ll need to install the cert-manager, which provides certificates for the webhook server used by the Milvus Operator.

1. Install cert-manager

Milvus Operator requires a cert-manager to manage certificates for secure communication. Make sure you install cert-manager version 1.1.3 or later. To install it, run the following command:

$ kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.3/cert-manager.yaml

After the installation, verify that the cert-manager pods are running by executing:

$ kubectl get pods -n cert-manager

2. Install the Milvus Operator

Once the cert-manager is up and running, you can install the Milvus Operator. Run the following command to deploy it using kubectl:

$ kubectl apply -f https://raw.githubusercontent.com/zilliztech/milvus-operator/main/deploy/manifests/deployment.yaml

You can check if the Milvus Operator pod is running using the following command:

$ kubectl get pods -n milvus-operator

3. Deploy Milvus Cluster

Once the Milvus Operator pod is running, you can deploy a Milvus cluster with the operator. The following command deploys a Milvus cluster with its components and dependencies in separate pods using default configurations:

$ kubectl apply -f https://raw.githubusercontent.com/zilliztech/milvus-operator/main/config/samples/milvus_cluster_default.yaml

To customize the Milvus settings, you will need to replace the YAML file with your own configuration YAML file. In addition to manually editing or creating the file, you can use the Milvus Sizing Tool to adjust the configurations and then download the corresponding YAML file.

To customize Milvus settings, you must replace the default YAML file with your own configuration. You can either manually edit or create this file, tailoring it to your specific requirements.

Alternatively, you can use the Milvus Sizing Tool for a more streamlined approach. This tool allows you to adjust various settings, such as resource allocation and storage options, and then download the corresponding YAML file with your desired configurations. This ensures that your Milvus deployment is optimized for your specific use case.

Figure: Milvus sizing tool

It may take some time to finish the deployment. You can check the status of your Milvus cluster via the command:

$ kubectl get milvus my-release

Once your Milvus cluster is ready, all pods in the Milvus cluster should be running or completed:

$ kubectl get pods

Step 4: Accessing Your Milvus Cluster

Once your Milvus cluster is deployed, you need to access it by forwarding a local port to the Milvus service port. Follow these steps to retrieve the service port and set up port forwarding.

1. Get the Service Port

First, identify the service port by using the following command. Replace <YOUR_MILVUS_PROXY_POD> with the name of your Milvus proxy pod, which typically starts with my-release-milvus-proxy-:

$ kubectl get pod <YOUR_MILVUS_PROXY_POD> --template ='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}'

This command will return the port number that your Milvus service is using.

2. Forward the Port

To access your Milvus cluster locally, forward a local port to the service port using the following command. Replace <YOUR_LOCAL_PORT> with the local port you want to use and <YOUR_SERVICE_PORT> with the service port retrieved in the previous step:

$ kubectl port-forward --address 0.0.0.0 service/my-release-milvus <YOUR_LOCAL_PORT>:<YOUR_SERVICE_PORT>

This command allows port-forwarding to listen on all IP addresses of the host machine. If you only need the service to listen on localhost, you can omit the --address 0.0.0.0 option.

Once the port-forwarding is set up, you can access your Milvus cluster via the specified local port for further operations or integrations.

Step 5: Connecting to Milvus Using Python SDK

With your Milvus cluster up and running, you can now interact with it using any Milvus SDK. In this example, we’ll use PyMilvus, Milvus’s Python SDK, to connect to the cluster and perform basic operations.

1. Install PyMilvus

To interact with Milvus via Python, you need to install the pymilvus package:

$ pip install pymilvus

2. Connect to Milvus

The following is a sample Python script that connects to your Milvus cluster and demonstrates how to perform basic operations such as creating a collection.

from pymilvus import MilvusClient

# Connect to the Milvus server
client = MilvusClient(uri="http://localhost:<YOUR_LOCAL_PORT>")

# Create a collection
collection_name = "example_collection"
if client.has_collection(collection_name):
   client.drop_collection(collection_name)
client.create_collection(
   collection_name=collection_name,
   dimension=768,  # The vectors we will use in this demo has 768 dimensions
)

Explanation:

• Connect to Milvus: The script connects to the Milvus server running on localhost using the local port you set up in Step 4.

• Create a Collection: It checks if a collection named example_collection already exists, drops it if so, and then creates a new collection with vectors of 768 dimensions.

This script establishes a connection to the Milvus cluster and creates a collection, serving as a starting point for more complex operations like inserting vectors and performing similarity searches.

Conclusion

Deploying Milvus in a distributed setup on Kubernetes unlocks powerful capabilities for managing large-scale vector data, enabling seamless scalability and high-performance AI-driven applications. Following this guide, you’ve learned how to set up Milvus using the Milvus Operator, making the process streamlined and efficient.

As you continue to explore Milvus, consider scaling your cluster to meet growing demands or deploying it on cloud platforms such as Amazon EKS, Google Cloud, or Microsoft Azure. For enhanced management and monitoring, tools like Milvus Backup, Birdwatcher, and Attu offer valuable support for maintaining the health and performance of your deployments.

You’re now ready to harness the full potential of Milvus on Kubernetes—happy deploying! 🚀

Further Resources

• Milvus Documentation

• Milvus Lite vs. Standalone vs. Distributed: Which Mode is Right for You?

• Supercharging Vector Search: Milvus on GPUs with NVIDIA RAPIDS cuVS

• What is RAG?

• Generative AI Resource Hub | Zilliz

• Top Performing AI Models for Your GenAI Apps | Zilliz