跨存储桶在实例间迁移

本主题详细介绍从一个 Milvus 实例备份一个 Collections 并将其还原到另一个实例的过程，每个实例在同一对象存储中使用不同的存储桶。

概述

下图说明了在同一对象存储中使用不同存储桶进行备份和恢复的过程。

跨存储桶备份和恢复.png

假设我们有两个 Milvus 实例，milvus_A 和milvus_B ，它们都使用默认的 MinIO 存储引擎进行对象存储。这些实例在同一对象存储中使用不同的存储桶 bucket_A 和bucket_B 。在这个示例中，我们的目标是完成以下任务：

1. 在bucket_A 中为 Collectionscoll 创建备份 (my_backup) 并将备份存储在bucket_B 中。

2. 在bucket_B 中，从备份中还原，并将还原后的 Collections 命名为coll_bak 。

前提条件**

• 确保已安装Milvus-backup工具。

• 熟悉配置 Milvus 对象存储设置，详情请参阅对象存储。

从 milvus_A 备份 Collections

步骤 1：准备配置

进入 milvus-backup 项目目录，创建名为 configs 的目录：

mkdir configs
cd configs

下载备份配置文件backup.yaml ：

wget https://raw.githubusercontent.com/zilliztech/milvus-backup/main/configs/backup.yaml

文件结构如下：

├── configs
│   └── backup.yaml
├── milvus-backup
└── README.md

第 2 步：编辑配置文件

修改 backup.yaml 文件，为milvus_A 设置适当的配置：

• 连接配置

  # milvus proxy address, compatible to milvus.yaml
  milvus:
    address: milvus_A
    port: 19530
    authorizationEnabled: false
    # tls mode values [0, 1, 2]
    # 0 is close, 1 is one-way authentication, 2 is two-way authentication.
    tlsMode: 0
    user: "root"
    password: "Milvus"
  

  • milvus.address:milvus_A 服务器的 IP 地址或主机名。

  • milvus.port:Milvus 服务器监听的 TCP 端口（默认 19530）。

• 存储配置（MinIO/S3 设置）

  # Related configuration of minio, which is responsible for data persistence for Milvus.
  minio:
    # cloudProvider: "minio" # deprecated use storageType instead
    storageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
    
    address: localhost # Address of MinIO/S3
    port: 9000   # Port of MinIO/S3
    accessKeyID: minioadmin  # accessKeyID of MinIO/S3
    secretAccessKey: minioadmin # MinIO/S3 encryption string
    useSSL: false # Access to MinIO/S3 with SSL
    useIAM: false
    iamEndpoint: ""
    
    bucketName: "bucket_A" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
    rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance
  
    # only for azure
    backupAccessKeyID: minioadmin  # accessKeyID of MinIO/S3
    backupSecretAccessKey: minioadmin # MinIO/S3 encryption string
    
    backupBucketName: "bucket_B" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRootPath
    backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath
  

• minio.bucketName:milvus_A 中数据存储桶的名称。在本例中，设置为bucket_A 。

• minio.rootPath:存储milvus_A 数据的存储桶根路径。在本例中，设置为files 。

• minio.backupBucketName:milvus_B 中用于备份存储的存储桶名称。在本例中，milvus_A 和milvus_B 使用不同的存储桶。因此，设置为bucket_B.

• minio.backupRootPath:指定用于在milvus_B 中存储备份文件的存储桶内的根路径。在本例中，设置为backup 。

第 3 步：创建备份

保存 backup.yaml 后，创建名为my_backup 的备份：

./milvus-backup create -c coll -n my_backup

此命令在对象存储中为 Collections coll 创建备份bucket_B/backup/my_backup 。

将备份还原到 milvus_B

步骤 1：配置恢复设置

重复步骤 2，修改配置以还原到milvus_B ，确保minio.bucketName 设置为bucket_B 。

下面是一个配置示例：

...
# milvus proxy address, compatible to milvus.yaml
milvus:
  address: milvus_B
  port: 19530
  authorizationEnabled: false
  # tls mode values [0, 1, 2]
  # 0 is close, 1 is one-way authentication, 2 is two-way authentication.
  tlsMode: 0
  user: "root"
  password: "Milvus"
  
# Related configuration of minio, which is responsible for data persistence for Milvus.
minio:
  # cloudProvider: "minio" # deprecated use storageType instead
  storageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
  
  address: localhost # Address of MinIO/S3
  port: 9000   # Port of MinIO/S3
  accessKeyID: minioadmin  # accessKeyID of MinIO/S3
  secretAccessKey: minioadmin # MinIO/S3 encryption string
  useSSL: false # Access to MinIO/S3 with SSL
  useIAM: false
  iamEndpoint: ""
  
  bucketName: "bucket_B" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
  rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance

  # only for azure
  backupAccessKeyID: minioadmin  # accessKeyID of MinIO/S3
  backupSecretAccessKey: minioadmin # MinIO/S3 encryption string
  
  backupBucketName: "bucket_B" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRootPath
  backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath
...

步骤 2：恢复备份

将备份还原到milvus_B ：

./milvus-backup restore -c coll -n my_backup -s _bak

此命令将备份还原到milvus_B 中名为coll_bak 的新 Collections 中，数据存储在bucket_B/files/insert_log/[ID of new collection] 中。