milvus-logo
LFAI
Home

Version Migration

This topic describes how to migrate data from Milvus 1.x to Milvus 2.0 using MilvusDM, an open-source tool specifically designed for Milvus data migration.

MilvusDM does not support migrating data from Milvus 2.0 standalone to Milvus 2.0 cluster.

Prerequisites

You need to install MilvusDM before migrating Milvus data.

1. Download YAML file

Download the M2M.yaml file.

$ wget https://raw.githubusercontent.com/milvus-io/milvus-tools/main/yamls/M2M.yaml

2. Set the parameters

Configuration parameters include:

ParameterDescriptionExample
milvus_versionVersion of Milvus.2.0.0
data_pathPath to the HDF5 files. Set either data_path or data_dir.- /Users/zilliz/float_1.h5
- /Users/zilliz/float_2.h5
data_dirDirectory of the HDF5 files. Set either data_path or data_dir.'/Users/zilliz/Desktop/HDF5_data'
dest_hostMilvus server address.'127.0.0.1'
dest_portMilvus server port.19530
modeMode of migration, including skip, append, and overwrite. This parameter works only when the specified collection name exists in the Milvus library.
  • skip refers to skipping data migration if the specified collection or partition already exists.
  • append refers to appending data if the specified collection or partition already exists.
  • overwrite refers to deleting existing data before insertion if the specified collection or partition already exists.
  • 'append'
    dest_collection_nameName of the collection to import data to.'test_float'
    dest_partition_name (optional)Name of the partition to import data to.'partition_1'
    collection_parameterCollection-specific information including vector dimension, index file size, and similarity metric."dimension: 512
    index_file_size: 1024
    metric_type: 'HAMMING'"

    The following two examples of configuration are for your reference. The first example involves setting mysql_parameter. If you do not use MySQL for managing vector IDs in Milvus 1.x, refer to the second example.

    Example 1

    M2M:
      milvus_version: 2.0.0
      source_milvus_path: '/home/user/milvus'
      mysql_parameter:
        host: '127.0.0.1'
        user: 'root'
        port: 3306
        password: '123456'
        database: 'milvus'
      source_collection: # specify the 'partition_1' and 'partition_2' partitions of the 'test' collection.
        test:
          - 'partition_1'
          - 'partition_2'
      dest_host: '127.0.0.1'
      dest_port: 19530
      mode: 'skip' # 'skip/append/overwrite'
    

    Example 2

    M2M:
      milvus_version: 2.0.0
      source_milvus_path: '/home/user/milvus'
      mysql_parameter:
      source_collection: # specify the collection named 'test'
        test:
      dest_host: '127.0.0.1'
      dest_port: 19530
      mode: 'skip' # 'skip/append/overwrite'
    

    3. Migrate data from Milvus to Milvus

    Run MilvusDM to import data from Milvus 1.x to Milvus 2.0 with the following command.

    $ milvusdm --yaml M2M.yaml
    

    What's next