milvus-logo
LFAI
Home

Migrate Data from Faiss to Milvus

This topic describes how to import data from Faiss to Milvus using MilvusDM, an open-source tool specifically designed for Milvus data migration.

Prerequisites

You need to install MilvusDM before migrating Milvus data.

1. Download YAML file

Download the F2M.yaml file.

$ wget https://raw.githubusercontent.com/milvus-io/milvus-tools/main/yamls/F2M.yaml

2. Set the parameters

Configuration parameters include:

ParameterDescriptionExample
milvus_versionVersion of Milvus.2.0.0
data_pathPath to the data in Faiss.‘/home/user/data/faiss.index’
data_dirDirectory of the HDF5 files. Set either data_path or data_dir.‘/Users/zilliz/Desktop/HDF5_data’
dest_hostMilvus server address.‘127.0.0.1’
dest_portMilvus server port.19530
modeMode of migration, including skip, append, and overwrite. This parameter works only when the specified collection name exists in the Milvus library.
  • skip refers to skipping data migration if the specified collection or partition already exists.
  • append refers to appending data if the specified collection or partition already exists.
  • overwrite refers to deleting existing data before insertion if the specified collection or partition already exists.
  • ‘append’
    dest_collection_nameName of the collection to import data to.‘test’
    dest_partition_name (optional)Name of the partition to import data to.‘partition’
    collection_parameterCollection-specific information including vector dimension, index file size, and similarity metric.“dimension: 512
    index_file_size: 1024
    metric_type: 'HAMMING’”

    Example

    The following example of configuration is for your reference.

    F2M:
      milvus_version: 2.0.0
      data_path: '/home/data/faiss1.index'
      dest_host: '127.0.0.1'
      dest_port: 19530
      mode: 'append'
      dest_collection_name: 'test'
      dest_partition_name: ''
      collection_parameter:
        dimension: 256
        index_file_size: 1024
        metric_type: 'L2'
    

    3. Migrate data from Faiss to Milvus

    Run MilvusDM to import data from Faiss to Milvus with the following command.

    $ milvusdm --yaml F2M.yaml
    

    What’s next

    • If you are interested in migrating data in other forms into Milvus,
    • If you are looking for information about how to migrate data from Milvus 1.x to Milvus 2.0,
    • If you are interested in learning more about the data migration tool,