Configure Object Storage with Docker Compose or Helm
Milvus uses MinIO for object storage by default, but it also supports using Amazon Simple Storage Service (S3) as persistent object storage for log and index files. This topic describes how to configure S3 for Milvus. You can skip this topic if you are satisfied with MinIO.
You can configure S3 with Docker Compose or on K8s.
Configure S3 with Docker Compose
1. Configure S3
MinIO is compatible with S3. To configure S3 with Docker Compose, provide your values for the minio
section in the milvus.yaml
file on the milvus/configs path.
minio:
address: <your_s3_endpoint>
port: <your_s3_port>
accessKeyID: <your_s3_access_key_id>
secretAccessKey: <your_s3_secret_access_key>
useSSL: <true/false>
bucketName: "<your_bucket_name>"
See MinIO/S3 Configurations for more information.
2. Refine docker-compose.yaml
You’d also remove the MINIO_ADDRESS
environment variable for milvus service at docker-compose.yaml
. By default milvus will use local minio instead of external S3.
3. Run Milvus
Run the following command to start Milvus that uses the S3 configurations.
docker compose up
Configure S3 on K8s
For Milvus clusters on K8s, you can configure S3 in the same command that starts Milvus. Alternatively, you can configure S3 using the values.yml
file on the /charts/milvus path in the milvus-helm repository before you start Milvus.
The following table lists the keys for configuring S3 in the YAML file.
Key | Description | Value |
---|---|---|
minio.enabled | Enables or disables MinIO. | true /false |
externalS3.enabled | Enables or disables S3. | true /false |
externalS3.host | The endpoint to access S3. | |
externalS3.port | The port to access S3. | |
externalS3.rootPath | The root path of the S3 storage. | An emtpy string by default. |
externalS3.accessKey | The access key ID for S3. | |
externalS3.secretKey | The secret access key for S3. | |
externalS3.bucketName | The name of the S3 bucket. | |
externalS3.useSSL | Whether to use SSL when connecting | The values defaults to false |
Using the YAML file
- Configure the
minio
section in thevalues.yaml
file.
minio:
enabled: false
- Configure the
externalS3
section using your values in thevalues.yaml
file.
externalS3:
enabled: true
host: "<your_s3_endpoint>"
port: "<your_s3_port>"
accessKey: "<your_s3_access_key_id>"
secretKey: "<your_s3_secret_key>"
useSSL: <true/false>
bucketName: "<your_bucket_name>"
- After configuring the preceding sections and saving the
values.yaml
file, run the following command to install Milvus that uses the S3 configurations.
helm install <your_release_name> milvus/milvus -f values.yaml
Using a command
To install Milvus and configure S3, run the following command using your values.
helm install <your_release_name> milvus/milvus --set cluster.enabled=true --set minio.enabled=false --set externalS3.enabled=true --set externalS3.host=<your_s3_endpoint> --set externalS3.port=<your_s3_port> --set externalS3.accessKey=<your_s3_access_key_id> --set externalS3.secretKey=<your_s3_secret_key> --set externalS3.bucketName=<your_bucket_name>
What’s next
Learn how to configure other Milvus dependencies with Docker Compose or Helm: