About Milvus
Get Started
User Guide
Administration Guide
Tools
Reference
Example Applications
FAQs
API Reference

Configure Message Storage with Docker Compose or Helm

Milvus uses Pulsar or Kafka for managing logs of recent changes, outputting stream logs, and providing log subscriptions. Pulsar is the default message storage system. This topic introduces how to configure message storage with Docker Compose or Helm.

You can configure Pulsar with Docker Compose or on K8s and configure Kafka on K8s.

Configure Pulsar with Docker Compose

1. Configure Pulsar

To configure Pulsar with Docker Compose, provide your values for the pulsar section in the milvus.yaml file on the milvus/configs path.

pulsar:
  address: localhost # Address of pulsar
  port: 6650 # Port of pulsar
  maxMessageSize: 5242880 # 5 * 1024 * 1024 Bytes, Maximum size of each message in pulsar.

See Pulsar-related configurations for more information.

2. Run Milvus

Run the following command to start Milvus that uses the Pulsar configurations.

docker-compose up

Configurations only take effect after Milvus starts. See Start Milvus for more information.

Configure Pulsar on K8s

For Milvus clusters on K8s, you can configure Pulsar in the same command that starts Milvus. Alternatively, you can configure Pulsar using the values.yml file on the /charts/milvus path in the milvus-helm repository before you start Milvus.

The following table lists the keys for configuring Pulsar in the YAML file.

Key	Description	Value
`externalpulsar.enabled`	Enables or disables Pulsar.	`true`/`false`
`externalpulsar.host`	The endpoint to access Pulsar.
`externalpulsar.port`	The port to access Pulsar.

Using the YAML file

Configure the pulsar section in the values.yaml file.

pulsar:
  enabled: false

Configure the externalPulsar section using your values in the values.yaml file.

externalPulsar:
  enabled: true
  host: <your_pulsar_localhost>
  port: <your_pulsar_port>

After configuring the preceding sections and saving the values.yaml file, run the following command to install Milvus that uses the Pulsar configurations.

helm install <your_release_name> milvus/milvus -f values.yaml

Using a command

To install Milvus and configure Pulsar, run the following command using your values.

helm install <your_release_name> milvus/milvus --set cluster.enabled=true --set externalPulsar.enabled=true --set externalPulsar.host='<your_pulsar_endpoint>' --set externalPulsar.port=<your_pulsar_port>

Configure Kafka on K8s

For Milvus clusters on K8s, you can configure Kafka in the same command that starts Milvus. Alternatively, you can configure Kafka using the values.yml file on the /charts/milvus path in the milvus-helm repository before you start Milvus.

The following table lists the keys for configuring Kafka in the YAML file.

Key	Description	Value
`externalkafka.enabled`	Enables or disables Kafka.	`true`/`false`
`externalpulsar.brokerlist`	The brokerlist to access Pulsar.

Using the YAML file

Configure the kafka section in the values.yaml file.

kafka:
  enabled: false
  name: kafka
  replicaCount: 3
  image:
    repository: bitnami/kafka
    tag: 3.1.0-debian-10-r52

Configure the externalKafka section using your values in the values.yaml file.

externalKafka:
  enabled: false
  brokerList: <your_kafka_brokerlist>

After configuring the preceding sections and saving the values.yaml file, run the following command to install Milvus that uses the Kafka configurations.

helm install <your_release_name> milvus/milvus -f values.yaml

Using a command

To install Milvus and configure Kafka, run the following command using your values.

helm install <your_release_name> milvus/milvus --set cluster.enabled=true --set externalKafka.enabled=true --set externalKafka.brokerlist='<your_kafka_brokerlist>'