milvus-logo
Star
0
Forks
0
Get Started

Version Migration

This topic describes how to migrate data from Milvus 1.x to Milvus 2.0 using MilvusDM, an open-source tool specifically designed for Milvus data migration.

MilvusDM does not support migrating data from Milvus 2.0 standalone to Milvus 2.0 cluster.

Prerequisites

You need to install MilvusDM before migrating Milvus data.

1. Download YAML file

Download the M2M.yaml file.

$ wget https://raw.githubusercontent.com/milvus-io/milvus-tools/main/yamls/M2M.yaml

2. Set the parameters

Configuration parameters include:

Parameter Description Example
milvus_version Version of Milvus. 2.0.0
data_path Path to the HDF5 files. Set either data_path or data_dir. - /Users/zilliz/float_1.h5
- /Users/zilliz/float_2.h5
data_dir Directory of the HDF5 files. Set either data_path or data_dir. '/Users/zilliz/Desktop/HDF5_data'
dest_host Milvus server address. '127.0.0.1'
dest_port Milvus server port. 19530
mode Mode of migration, including skip, append, and overwrite. This parameter works only when the specified collection name exists in the Milvus library.
  • skip refers to skipping data migration if the specified collection or partition already exists.
  • append refers to appending data if the specified collection or partition already exists.
  • overwrite refers to deleting existing data before insertion if the specified collection or partition already exists.
  • 'append'
    dest_collection_name Name of the collection to import data to. 'test_float'
    dest_partition_name (optional) Name of the partition to import data to. 'partition_1'
    collection_parameter Collection-specific information including vector dimension, index file size, and similarity metric. "dimension: 512
    index_file_size: 1024
    metric_type: 'HAMMING'"

    The following two examples of configuration are for your reference. The first example involves setting mysql_parameter. If you do not use MySQL for managing vector IDs in Milvus 1.x, refer to the second example.

    Example 1

    M2M:
      milvus_version: 2.0.0
      source_milvus_path: '/home/user/milvus'
      mysql_parameter:
        host: '127.0.0.1'
        user: 'root'
        port: 3306
        password: '123456'
        database: 'milvus'
      source_collection: # specify the 'partition_1' and 'partition_2' partitions of the 'test' collection.
        test:
          - 'partition_1'
          - 'partition_2'
      dest_host: '127.0.0.1'
      dest_port: 19530
      mode: 'skip' # 'skip/append/overwrite'
    

    Example 2

    M2M:
      milvus_version: 2.0.0
      source_milvus_path: '/home/user/milvus'
      mysql_parameter:
      source_collection: # specify the collection named 'test'
        test:
      dest_host: '127.0.0.1'
      dest_port: 19530
      mode: 'skip' # 'skip/append/overwrite'
    

    3. Migrate data from Milvus to Milvus

    Run MilvusDM to import data from Milvus 1.x to Milvus 2.0 with the following command.

    $ milvusdm --yaml M2M.yaml
    

    What's next

    Is this page helpful?
    On this page