MMap-enabled Data Storage
In Milvus, memory-mapped files allow for direct mapping of file contents into memory. This feature enhances memory efficiency, particularly in situations where available memory is scarce but complete data loading is infeasible. This optimization mechanism can increase data capacity while ensuring performance up to a certain limit; however, when the amount of data exceeds memory by too much, search and query performance may suffer serious degradation, so please choose to turn this feature on or off as appropriate.
Configure memory mapping
Starting with Milvus 2.4, you have the flexibility to adjust the static configuration file to configure default memory mapping settings for the entire cluster before deployment. Additionally, there’s the option for you to dynamically alter parameters to fine-tune memory mapping settings at both the cluster and index levels. Looking ahead, future updates will extend memory mapping capabilities to include field-level configurations.
Before cluster deployment: global configuration
Before you deploy a cluster, cluster-level settings apply memory mapping across your entire cluster. This ensures all new objects will automatically adhere to these configurations. It’s important to note that modifying these settings requires a restart of the cluster to become effective.
To adjust your cluster’s memory mapping settings, edit the configs/milvus.yaml file. Within this file, you can specify whether to enable memory mapping by default and determine the directory path for storing memory-mapped files. If the path (mmapDirPath) is left unspecified, the system defaults to storing memory-mapped files in {localStorage.path}/mmap. For more information, refer to Local Storage-related Configurations.
# This parameter was set in configs/milvus.yaml
...
queryNode:
mmap:
# Set memory mapping property for whole cluster
mmapEnabled: false | true
# Set memory-mapped directory path, if you leave mmapDirPath unspecified, the memory-mapped files will be stored in {localStorage.path}/ mmap by default.
mmapDirPath: any/valid/path
....
After 2.4.10, the configuration queryNode.mmap.mmapEnabled splits into below four seperate fields, and all defaults are false:
queryNode.mmap.vectorField, controls whether vector data is mmap;queryNode.mmap.vectorIndex, controls whether vector index is mmap;queryNode.mmap.scalarField, controls whether scalar data is mmap;queryNode.mmap.scalarIndex, controls whether scalar index is mmap;
# This parameter was set in configs/milvus.yaml
...
queryNode:
mmap:
vectorField: false # Enable mmap for loading vector data
vectorIndex: false # Enable mmap for loading vector index
scalarField: false # Enable mmap for loading scalar data
scalarIndex: false # Enable mmap for loading scalar index
....
In addition, only vector index and vector data mmap can be turned on and off for a collection individually, but not for others.
Compatibility: If the original configuration queryNode.mmap.mmapEnabled is set to true, the newly added configuration will be set to true at this time. If queryNode.mmap.mmapEnabled is set to false, if the new configuration is set to true, the final value will be true.
During cluster operation: dynamic configuration
During cluster runtime, you can dynamically adjust memory mapping settings at either the collection or index level.
At the collection level, memory mapping is applied to all unindexed raw data within a collection, excluding primary keys, timestamps, and row IDs. This approach is particularly suited for comprehensive management of large datasets.
For dynamic adjustments to memory mapping settings within a collection, utilize the set_properties() method. Here, you can toggle mmap.enabled between True or False as needed.
# Get existing collection
collection = Collection("test_collection") # Replace with your collection name
# Set memory mapping property to True or Flase
collection.set_properties({'mmap.enabled': True})
After 2.4.10, the memory mapping settings within a collection, utilize the add_field method. Here, you can toggle mmap_enabled between True or False as needed.
schema = MilvusClient.create_schema()
schema.add_field(field_name="embedding", datatype=DataType.FLOAT_VECTOR, dim=768, mmap_enabled=True)
For index-level settings, memory mapping can be specifically applied to vector indexes without affecting other data types. This feature is invaluable for collections that require optimized performance for vector searches.
To enable or disable memory mapping for an index within a collection, call the alter_index() method, specifying the target index name in index_name and setting mmap.enabled to True or False.
collection.alter_index(
index_name="vector_index", # Replace with your vector index name
extra_params={"mmap.enabled": True} # Enable memory mapping for index
)
Customize storage path in different deployments
Memory-mapped files default to the /mmap directory within localStorage.path. Here’s how to customize this setting across various deployment methods:
- For Milvus installed using Helm Chart:
# new-values.yaml
extraConfigFiles:
user.yaml: |+
queryNode:
mmap:
mmapEnabled: true
mmapDirPath: any/valid/path
helm upgrade <milvus-release> --reuse-values -f new-values.yaml milvus/milvus
- For Milvus installed using Milvus Operator:
# patch.yaml
spec:
config:
queryNode:
mmap:
mmapEnabled: true
mmapDirPath: any/valid/path
kubectl patch milvus <milvus-name> --patch-file patch.yaml
- For Milvus installed using Docker:
# A new installation script is provided to enable mmap-related settings.
Limits
Memory mapping cannot be enabled for a loaded collection, ensure the collection has been released before enabling memory mapping.
Memory mapping is not supported for DiskANN or GPU-class indexes.
FAQ
In which scenarios is it recommended to enable memory mapping? What are the trade-offs after enabling this feature?
Memory mapping is recommended when memory is limited or when performance requirements are moderate. Enabling this feature increases the capacity for data loading. For example, with a configuration of 2 CPUs and 8 GB of memory, enabling memory mapping can allow for up to 4 times more data to be loaded compared to not enabling it. The impact on performance varies:
With sufficient memory, the expected performance is similar to that of using only memory.
With insufficient memory, the expected performance may degrade.
What is the relationship between collection-level and index-level configurations?
Collection-level and index-level are not inclusive relationships, collection-level controls whether the original data is mmap-enabled or not, whereas index-level is for vector indexes only.
Is there any recommended index type for memory mapping?
Yes, HNSW is recommended for enable mmap. We have tested HNSW, IVF_FLAT, IVF_PQ/SQ series indexes before, the performance of IVF series indexes dropped seriously, while the performance drop of turning on mmap for HNSW indexes is still within expectation.
What kind of local storage is required for memory mapping?
A high-quality disk enhances performance, with NVMe drives being the preferred option.
Can scalar data be memory-mapped?
Memory mapping can be applied to scalar data, but it is not applicable to indexes built on scalar fields.
How is the priority determined for memory mapping configurations across different levels?
In Milvus, when memory mapping configurations are explicitly defined across multiple levels, index-level and collection-level configurations share the highest priority, which is then followed by cluster-level configurations.
If I upgrade from Milvus 2.3 and have configured the memory mapping directory path, what will happen?
If you upgrade from Milvus 2.3 and have configured the memory mapping directory path (
mmapDirPath), your configuration will be retained, and the default setting for memory mapping enabled (mmapEnabled) will betrue. It’s important to migrate the metadata to synchronize the configuration of your existing memory-mapped files. For more details, refer to Migrate the metadata.