Configure S3 Access by IAM Role
This topic introduces how to configure s3 access by IAM Role when you install Milvus with helm. For more details, refer to IAM roles.
Before you start
Please enable OIDC when creating an EKS cluster using eksctl. For more details, refer to IAM OIDC.
This guide assumes that you have already deployed a Milvus Cluster on AWS with Kubernetes.
Associate an IAM role with a Kubernetes service account
Create an AWS S3 bucket.
Read Bucket Naming Rules and observe the naming rules when naming your AWS S3 bucket.
milvus_bucket_name="milvus-bucket-$(openssl rand -hex 12)" aws s3api create-bucket --bucket "$milvus_bucket_name" --region 'us-east-2' --acl private --object-ownership ObjectWriter --create-bucket-configuration LocationConstraint='us-east-2' # Output # # "Location": "http://milvus-bucket-039dd013c0712f085d60e21f.s3.amazonaws.com/"
Create an IAM policy for reading and writing objects within the bucket created above. Do replace the bucket name with your own.
echo '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:ListBucket", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::<bucket-name>", "arn:aws:s3:::<bucket-name>/*" ] } ] }' > milvus-s3-policy.json aws iam create-policy --policy-name MilvusS3ReadWrite --policy-document file://milvus-s3-policy.json # Get the ARN from the command output as follows: # { # "Policy": { # "PolicyName": "MilvusS3ReadWrite", # "PolicyId": "AN5QQVVPM1BVTFlBNkdZT", # "Arn": "arn:aws:iam::12345678901:policy/MilvusS3ReadWrite", # "Path": "/", # "DefaultVersionId": "v1", # "AttachmentCount": 0, # "PermissionsBoundaryUsageCount": 0, # "IsAttachable": true, # "CreateDate": "2023-11-16T06:00:01+00:00", # "UpdateDate": "2023-11-16T06:00:01+00:00" # } # }
Create an IAM role and associate it with a Kubernetes service account. Replace
your-account-id
with you account ID.
eksctl create iamserviceaccount --name milvus-s3-access-sa --namespace milvus --cluster milvus-eks-cluster --role-name milvus-s3-access-sa \
--attach-policy-arn arn:aws:iam::<your-account-id>:policy/MilvusS3ReadWrite --approve
Verify the role and service account setup
Please refer to IAM roles.
- Confirm that the IAM role’s trust policy is configured correctly.
aws iam get-role --role-name milvus-s3-access-sa --query Role.AssumeRolePolicyDocument
# An example output is as follows
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:default:my-service-account",
"oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
}
}
}
]
}
- Confirm that the policy that you attached to your role in a previous step is attached to the role.
aws iam list-attached-role-policies --role-name milvus-s3-access-sa --query 'AttachedPolicies[].PolicyArn' --output text
# An example output is as follows
arn:aws:iam::12345678901:policy/MilvusS3ReadWrite
- View the default version of the policy.
export policy_arn='arn:aws:iam::12345678901:policy/MilvusS3ReadWrite'
aws iam get-policy --policy-arn $policy_arn
# An example output is as follows
{
"Policy": {
"PolicyName": "MilvusS3ReadWrite",
"PolicyId": "EXAMPLEBIOWGLDEXAMPLE",
"Arn": "arn:aws:iam::12345678901:policy/MilvusS3ReadWrite",
"Path": "/",
"DefaultVersionId": "v2",
[...]
}
}
- View the policy contents to make sure that the policy includes all the permissions that your Pod needs. If necessary, replace 1 in the following command with the version that’s returned in the previous output.
aws iam get-policy-version --policy-arn $policy_arn --version-id v2
# An example output is as follows
{
"PolicyVersion": {
"Document": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::<bucket-name>",
"arn:aws:s3:::<bucket-name>/*"
]
}
]
},
[...]
}
}
- Confirm that the Kubernetes service account is annotated with the role.
kubectl describe serviceaccount milvus-s3-access-sa -n milvus
# An example output is as follows
Name: milvus-s3-access-sa
Namespace: milvus
Labels: app.kubernetes.io/managed-by=eksctl
Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::12345678901:role/milvus-s3-access-sa
[...]
Deploy Milvus
In this guide, we will use Milvus Helm Charts to deploy a Milvus cluster. You can find the charts here.
- Add Milvus Helm Chart repo.
helm repo add milvus https://zilliztech.github.io/milvus-helm/
helm repo update
- Prepare the Milvus configuration file
milvus.yaml
, and replace<bucket-name>
with the name of the bucket created above.
cluster:
enabled: true
service:
type: LoadBalancer
port: 19530
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-name: milvus-service
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
serviceAccount:
create: false
name: milvus-s3-access-sa
minio:
enabled: false
externalS3:
enabled: true
host: "s3.us-east-2.amazonaws.com"
port: "443"
useSSL: true
bucketName: "<bucket-name>"
useIAM: true
cloudProvider: "aws"
iamEndpoint: ""
rootCoordinator:
replicas: 2
activeStandby:
enabled: true
resources:
limits:
cpu: 1
memory: 2Gi
indexCoordinator:
replicas: 2
activeStandby:
enabled: true
resources:
limits:
cpu: "0.5"
memory: 0.5Gi
queryCoordinator:
replicas: 2
activeStandby:
enabled: true
resources:
limits:
cpu: "0.5"
memory: 0.5Gi
dataCoordinator:
replicas: 2
activeStandby:
enabled: true
resources:
limits:
cpu: "0.5"
memory: 0.5Gi
proxy:
replicas: 2
resources:
limits:
cpu: 1
memory: 2Gi
- Install Milvus.
helm upgrade --install milvus-demo milvus/milvus -n milvus -f milvus.yaml
Verify the installation
Please refer to Verify-the-installation.