milvus-logo
LFAI
Home
  • Administration Guide

Deploy a Milvus Cluster on EKS

This topic describes how to deploy a Milvus cluster on Amazon EKS.

Prerequisites

  • You have AWS CLI installed on your local PC or an Amazon EC2, which will serve as your endpoint to do the operations covered in this document. For an Amazon Linux 2 or Amazon Linux 2023, the AWS CLI tools are already installed. To install AWS CLi on your local PC. Refer to How to install AWS CLI.
  • You have installed Kubernetes and EKS tools installed on the preferred endpoint device, including:
  • AWS IAM permissions have been granted properly. The IAM security principal you are using must have permission to use Amazon EKS IAM roles, service-related roles, AWS CloudFormation, VPCs, and other related resources. You can follow either of the following ways to grant your principal proper permissions.
    • (Not recommended) Simply set the association policy of the user/role that you used to AWS managed policy AdministratorAccess.
    • (Strongly recommended) To implement the principle of least privilege, do as follows:
      • To set up permission for eksctl, refer to Minimum permission for eksctl.

      • To set up permission for creating/deleting AWS S3 buckets, refer to the following permission settings:

        {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "S3BucketManagement",
              "Effect": "Allow",
              "Action": [
                "s3:CreateBucket",
                "s3:PutBucketAcl",
                "s3:PutBucketOwnershipControls",
                "s3:DeleteBucket"
              ],
              "Resource": [
                "arn:aws:s3:::milvus-bucket-*"
              ]
            }
          ]
        }
        
      • To set up permissions for creating/deleting IAM policies, refer to the following permission settings. Do replace YOUR_ACCOUNT_ID with your own.

        {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "IAMPolicyManagement",
              "Effect": "Allow",
              "Action": [
                "iam:CreatePolicy",
                "iam:DeletePolicy"
              ],
              "Resource": "arn:aws:iam::YOUR_ACCOUNT_ID:policy/MilvusS3ReadWrite"
            }
          ]
        }    
        

Set up AWS Resources

You can set up the required AWS resources, including an AWS S3 bucket and an EKS cluster, using either AWS Management Console, AWS CLI, or IaC tools, such as Terraform. In this document, the AWS CLI is preferred to demonstrate how to set up the AWS resources.

Create an Amazon S3 Bucket

  1. Create an AWS S3 bucket.

    Read Bucket Naming Rules and observe the naming rules when naming your AWS S3 bucket.

    milvus_bucket_name="milvus-bucket-$(openssl rand -hex 12)"
    
    aws s3api create-bucket --bucket "$milvus_bucket_name" --region 'us-east-2' --acl private  --object-ownership ObjectWriter --create-bucket-configuration LocationConstraint='us-east-2'
    
    
    # Output
    #
    # "Location": "http://milvus-bucket-039dd013c0712f085d60e21f.s3.amazonaws.com/"
    
  2. Creat an IAM policy for reading and writing objects within the bucket created above. Do replace the bucket name with your own.

    echo '{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:ListBucket",
            "s3:DeleteObject"
          ],
          "Resource": [
            "arn:aws:s3:::your-milvus-bucket-name",
            "arn:aws:s3:::your-milvus-bucket-name/*"
          ]
        }
      ]
    }' > milvus-s3-policy.json
    
    aws iam create-policy --policy-name MilvusS3ReadWrite --policy-document file://milvus-s3-policy.json
    
    
    # Get the ARN from the command output as follows:
    # {
    #     "Policy": {
    #         "PolicyName": "MilvusS3ReadWrite",
    #         "PolicyId": "AN5QQVVPM1BVTFlBNkdZT",
    #         "Arn": "arn:aws:iam::12345678901:policy/MilvusS3ReadWrite",
    #         "Path": "/",
    #         "DefaultVersionId": "v1",
    #         "AttachmentCount": 0,
    #         "PermissionsBoundaryUsageCount": 0,
    #         "IsAttachable": true,
    #         "CreateDate": "2023-11-16T06:00:01+00:00",
    #        "UpdateDate": "2023-11-16T06:00:01+00:00"
    #     }
    # }    
    
  3. (Optional) Attach the policy to your AWS User/Role if you want to accesskey instead of IAM AssumeRole.

    aws iam attach-user-policy --user-name <your-user-name> --policy-arn "arn:aws:iam::<your-iam-account-id>:policy/MilvusS3ReadWrite"
    

Create an Amazon EKS Cluster

  1. Prepare a cluster configuration file as follows and name it eks_cluster.yaml. Do replace MilvusS3ReadWrite_Policy_ARN with the one listed in the command output above.

    apiVersion: eksctl.io/v1alpha5
    kind: ClusterConfig
    
    metadata:
      name: 'milvus-eks-cluster'
      region: 'us-east-2'
      version: "1.27"
    
    iam:
      withOIDC: true
    
      serviceAccounts:
      - metadata:
          name: aws-load-balancer-controller
          namespace: kube-system
        wellKnownPolicies:
          awsLoadBalancerController: true
      - metadata:
          name: milvus-s3-access-sa
          namespace: milvus
          labels: {aws-usage: "milvus"}
        attachPolicyARNs:
        - "MilvusS3ReadWrite_Policy_ARN"
    
    managedNodeGroups:
      - name: milvus-node-group
        labels: { role: milvus }
        instanceType: m6i.4xlarge
        desiredCapacity: 3
        privateNetworking: true
        
    addons:
    - name: vpc-cni
      version: latest
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
    - name: coredns
      version: latest
    - name: kube-proxy
      version: latest
    - name: aws-ebs-csi-driver
      version: latest
      wellKnownPolicies:
        ebsCSIController: true
    
  2. Run the following command to create an EKS cluster.

    eksctl create cluster -f eks_cluster.yaml
    
  3. Get the kubeconfig file.

    aws eks update-kubeconfig --region 'us-east-2' --name 'milvus-eks-cluster'
    
  4. Verify the EKS cluster.

    kubectl cluster-info
    
    kubectl get nodes -A -o wide
    

Create a StorageClass

Milvus uses etcd as meta storage and needs to rely on the gp3 StorageClass to create and manage PVC.

cat <<EOF | kubectl apply -f -
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-gp3-sc
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
  type: gp3
EOF

Set the original gp2 StorageClass to non-default.

kubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

Install AWS LoadBalancer Controller

  1. Add Helm chars repo.

    helm repo add eks https://aws.github.io/eks-charts
    helm repo update
    
  2. Install the AWS Load Balancer Controller.

    helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
      -n kube-system \
      --set clusterName='milvus-eks-cluster' \
      --set serviceAccount.create=false \
      --set serviceAccount.name=aws-load-balancer-controller 
    
  3. Verify the installation

    kubectl get deployment -n kube-system aws-load-balancer-controller
    

Deploy Milvus

In this guide, we will use Milvus Helm Charts to deploy a Milvus cluster. You can find the charts here.

  1. Add Milvus Helm Chart repo.

    helm repo add milvus https://zilliztech.github.io/milvus-helm/
    helm repo update
    
  2. Prepare the Milvus configuration file milvus.yaml, and replace <bucket-name> with the name of the bucket created above.

    • To configure HA for your Milvus, refer to this calculator for more information. You can download the related configurations directly from the calculator, and you should remove MinIO-related configurations.
    • To implement multi-replica deployments of coordinators, set xxCoordinator.activeStandby.enabled to true.
    • To access your Milvus from the Internet, change service.beta.kubernetes.io/aws-load-balancer-scheme from internal to internet-facing.

    cluster:
      enabled: true
    
    service:
      type: LoadBalancer
      port: 19530
      annotations: 
        service.beta.kubernetes.io/aws-load-balancer-type: external
        service.beta.kubernetes.io/aws-load-balancer-name: milvus-service
        service.beta.kubernetes.io/aws-load-balancer-scheme: internal
        service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    
    serviceAccount:
      create: false
      name: milvus-s3-access-sa
    
    minio:
      enabled: false
    
    # Use the milvus-s3-access-sa to access milvus buckets instead of using ak/sk.  
    # Details see https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
    externalS3:
      enabled: true
      host: "s3.us-east-2.amazonaws.com"
      port: "443"
      useSSL: true
      bucketName: "<bucket-name>"
      useIAM: true
      cloudProvider: "aws"
      iamEndpoint: ""
    
    # HA Configurations
    rootCoordinator:
      replicas: 2
      activeStandby:
        enabled: true
      resources: 
        limits:
          cpu: 1
          memory: 2Gi
    
    indexCoordinator:
      replicas: 2
      activeStandby:
        enabled: true
      resources: 
        limits:
          cpu: "0.5"
          memory: 0.5Gi
    
    queryCoordinator:
      replicas: 2
      activeStandby:
        enabled: true
      resources: 
        limits:
          cpu: "0.5"
          memory: 0.5Gi
    
    dataCoordinator:
      replicas: 2
      activeStandby:
        enabled: true
      resources: 
        limits:
          cpu: "0.5"
          memory: 0.5Gi
    
    proxy:
      replicas: 2
      resources: 
        limits:
          cpu: 1
          memory: 2Gi  
    
  3. Install Milvus.

    helm install milvus-demo milvus/milvus -n milvus -f milvus.yaml
    
  4. Wait until all pods are Running.

    kubectl get pods -n milvus
    

    Helm does not support scheduling the order of service creation. It is normal that business pods to restart for one or two times before etcd and pulsar are up in the early stage.

  5. Get Milvus service address.

    kubectl get svc -n milvus
    

Verify the installation

You can follow the simple guide below to verify the installation. For more details, refer to this example.

  1. Download the example code.

    wget https://raw.githubusercontent.com/milvus-io/pymilvus/master/examples/hello_milvus.py
    
  2. Change the host argument in the example code to the Milvus service address above.

    If You have set service.beta.kubernetes.io/aws-load-balancer-scheme to internal in milvus.yaml. You should run the example code within the EKS VPC.

    ...
    connections.connect("default", host="milvus-service-06b515b1ce9ad10.elb.us-east-2.amazonaws.com", port="19530")
    ...
    
  3. Run the example code.

    python3 hello_milvus.py
    

    The output should be similar to the following:

    === start connecting to Milvus     ===
    
    Does collection hello_milvus exist in Milvus: False
    
    === Create collection `hello_milvus` ===
    
    
    === Start inserting entities       ===
    
    Number of entities in Milvus: 3000
    
    === Start Creating index IVF_FLAT  ===
    
    
    === Start loading                  ===
    
    
    === Start searching based on vector similarity ===
    
    hit: id: 2998, distance: 0.0, entity: {'random': 0.9728033590489911}, random field: 0.9728033590489911
    hit: id: 1262, distance: 0.08883658051490784, entity: {'random': 0.2978858685751561}, random field: 0.2978858685751561
    hit: id: 1265, distance: 0.09590047597885132, entity: {'random': 0.3042039939240304}, random field: 0.3042039939240304
    hit: id: 2999, distance: 0.0, entity: {'random': 0.02316334456872482}, random field: 0.02316334456872482
    hit: id: 1580, distance: 0.05628091096878052, entity: {'random': 0.3855988746044062}, random field: 0.3855988746044062
    hit: id: 2377, distance: 0.08096685260534286, entity: {'random': 0.8745922204004368}, random field: 0.8745922204004368
    search latency = 0.4693s
    
    === Start querying with `random > 0.5` ===
    
    query result:
    -{'embeddings': [0.20963514, 0.39746657, 0.12019053, 0.6947492, 0.9535575, 0.5454552, 0.82360446, 0.21096309], 'pk': '0', 'random': 0.6378742006852851}
    search latency = 0.9407s
    query pagination(limit=4):
            [{'random': 0.6378742006852851, 'pk': '0'}, {'random': 0.5763523024650556, 'pk': '100'}, {'random': 0.9425935891639464, 'pk': '1000'}, {'random': 0.7893211256191387, 'pk': '1001'}]
    query pagination(offset=1, limit=3):
            [{'random': 0.5763523024650556, 'pk': '100'}, {'random': 0.9425935891639464, 'pk': '1000'}, {'random': 0.7893211256191387, 'pk': '1001'}]
    
    === Start hybrid searching with `random > 0.5` ===
    
    hit: id: 2998, distance: 0.0, entity: {'random': 0.9728033590489911}, random field: 0.9728033590489911
    hit: id: 747, distance: 0.14606499671936035, entity: {'random': 0.5648774800635661}, random field: 0.5648774800635661
    hit: id: 2527, distance: 0.1530652642250061, entity: {'random': 0.8928974315571507}, random field: 0.8928974315571507
    hit: id: 2377, distance: 0.08096685260534286, entity: {'random': 0.8745922204004368}, random field: 0.8745922204004368
    hit: id: 2034, distance: 0.20354536175727844, entity: {'random': 0.5526117606328499}, random field: 0.5526117606328499
    hit: id: 958, distance: 0.21908017992973328, entity: {'random': 0.6647383716417955}, random field: 0.6647383716417955
    search latency = 0.4652s
    
    === Start deleting with expr `pk in ["0" , "1"]` ===
    
    query before delete by expr=`pk in ["0" , "1"]` -> result:
    -{'random': 0.6378742006852851, 'embeddings': [0.20963514, 0.39746657, 0.12019053, 0.6947492, 0.9535575, 0.5454552, 0.82360446, 0.21096309], 'pk': '0'}
    -{'random': 0.43925103574669633, 'embeddings': [0.52323616, 0.8035404, 0.77824664, 0.80369574, 0.4914803, 0.8265614, 0.6145269, 0.80234545], 'pk': '1'}
    
    query after delete by expr=`pk in ["0" , "1"]` -> result: []
    
    
    === Drop collection `hello_milvus` ===
    

Clean-up works

In case you need to restore the environment by uninstalling Milvus, destroying the EKS cluster, and deleting the AWS S3 buckets and related IAM policies.

  1. Uninstall Milvus.

    helm uninstall milvus-demo -n milvus
    
  2. Destroy the EKS cluster.

    eksctl delete cluster --name milvus-eks-cluster
    
  3. Delete the AWS S3 bucket and related IAM policies.

    You should replace the bucket name and policy ARN with your own.

    aws s3api delete-bucket --bucket milvus-bucket-039dd013c0712f085d60e21f --region us-east-2
    
    aws iam delete-policy --policy-arn 'arn:aws:iam::12345678901:policy/MilvusS3ReadWrite'
    

What's next

If you want to learn how to deploy Milvus on other clouds: