milvus-logo
Star
0
Forks
0
Get Started

Migrate Data from Faiss to Milvus

This topic describes how to import data from Faiss to Milvus using MilvusDM, an open-source tool specifically designed for Milvus data migration.

Prerequisites

You need to install MilvusDM before migrating Milvus data.

1. Download YAML file

Download the F2M.yaml file.

$ wget https://raw.githubusercontent.com/milvus-io/milvus-tools/main/yamls/F2M.yaml

2. Set the parameters

Configuration parameters include:

Parameter Description Example
milvus_version Version of Milvus. 2.0.0
data_path Path to the data in Faiss. '/home/user/data/faiss.index'
data_dir Directory of the HDF5 files. Set either data_path or data_dir. '/Users/zilliz/Desktop/HDF5_data'
dest_host Milvus server address. '127.0.0.1'
dest_port Milvus server port. 19530
mode Mode of migration, including skip, append, and overwrite. This parameter works only when the specified collection name exists in the Milvus library.
  • skip refers to skipping data migration if the specified collection or partition already exists.
  • append refers to appending data if the specified collection or partition already exists.
  • overwrite refers to deleting existing data before insertion if the specified collection or partition already exists.
  • 'append'
    dest_collection_name Name of the collection to import data to. 'test'
    dest_partition_name (optional) Name of the partition to import data to. 'partition'
    collection_parameter Collection-specific information including vector dimension, index file size, and similarity metric. "dimension: 512
    index_file_size: 1024
    metric_type: 'HAMMING'"

    Example

    The following example of configuration is for your reference.

    F2M:
      milvus_version: 2.0.0
      data_path: '/home/data/faiss1.index'
      dest_host: '127.0.0.1'
      dest_port: 19530
      mode: 'append'
      dest_collection_name: 'test'
      dest_partition_name: ''
      collection_parameter:
        dimension: 256
        index_file_size: 1024
        metric_type: 'L2'
    

    3. Migrate data from Faiss to Milvus

    Run MilvusDM to import data from Faiss to Milvus with the following command.

    $ milvusdm --yaml F2M.yaml
    

    What's next

    • If you are interested in migrating data in other forms into Milvus,
    • If you are looking for information about how to migrate data from Milvus 1.x to Milvus 2.0,
    • If you are interested in learning more about the data migration tool,
    Is this page helpful?
    On this page