更新實體
upsert 操作提供了在集合中插入或更新實體的便捷方法。
概述
您可以使用upsert 來插入新的實體或更新現有的實體，這取決於在 upsert 請求中提供的主索引鍵是否存在於集合中。如果找不到主索引鍵，則會執行插入操作。否則，將執行更新操作。
在 Milvus 中，upsert 在覆寫 或合併模式下工作。
在覆蓋模式下的upsert
在覆寫模式下工作的 upsert 請求結合了插入和刪除。當收到一個現有實體的upsert 請求時，Milvus 會插入請求有效載荷中攜帶的資料，並同時刪除資料中指定原始主鍵的現有實體。
在覆寫模式下的 Upsert
如果目標集合的主欄位啟用了autoid ，Milvus 會在插入前為請求付載中攜帶的資料產生一個新的主索引鍵。
對於啟用nullable 的欄位，如果它們不需要任何更新，您可以在upsert 請求中省略它們。
在合併模式中倒插Compatible with Milvus v2.6.2+
您也可以使用partial_update 標誌，使 upsert 請求以合併模式運作。這可讓您在請求有效負載中只包含需要更新的欄位。
合併模式下的 UPSERT
要執行合併，在upsert 請求中將partial_update 設為True ，同時設置主索引鍵和要更新的字段及其新值。
接收到這樣的請求後，Milvus 會執行強一致性查詢來擷取實體，根據請求中的資料更新欄位值，插入修改後的資料，然後刪除請求中帶有原始主索引鍵的現有實體。
Upsert 行為：特別注意事項
在使用合併功能之前，您應該考慮幾個特別注意事項。以下情況假設您有一個集合，其中有兩個標量欄位，分別命名為title 和issue ，以及一個主索引鍵id 和一個向量欄位，命名為vector 。
啟用
nullable的倒置欄位 。假設
issue欄位可以為空。當您倒插這些欄位時，請注意：如果在
upsert請求中省略issue欄位，並且停用partial_update，issue欄位將會更新為null，而不是保留其原始值。若要保留
issue欄位的原始值，您需要啟用partial_update並省略issue欄位，或在upsert請求中包含issue欄位及其原始值。
在動態欄位中倒插鍵。
假設您在範例集合中啟用了動態鍵，且實體的動態欄位中的鍵值對與
{"author": "John", "year": 2020, "tags": ["fiction"]}相似。當您向上插入實體的鍵，例如
author,year, 或tags, 或新增其他鍵時，請注意：如果您在
partial_update禁用的情況下 upsert，預設行為是覆蓋。這表示動態欄位的值將被所有包含在請求中的非結構描述定義的欄位及其值覆蓋。例如，如果請求中包含的資料是
{"author": "Jane", "genre": "fantasy"}，則目標實體的動態欄位中的鍵值對將更新為該資料。如果啟用
partial_update的 upsert，預設行為是合併。這表示動態欄位的值會與所有包含在請求中的非結構描述定義的欄位及其值合併。例如，如果請求中包含的資料是
{"author": "John", "year": 2020, "tags": ["fiction"]}，則在 upsert 之後，目標實體的動態欄位中的鍵值對會變成{"author": "John", "year": 2020, "tags": ["fiction"], "genre": "fantasy"}。
倒插一個 JSON 欄位。
假設範例集合有一個模式定義的 JSON 欄位，名稱為
extras，且此實體的 JSON 欄位中的關鍵值對與{"author": "John", "year": 2020, "tags": ["fiction"]}相似。當您使用修改過的 JSON 資料倒插實體的
extras欄位時，請注意 JSON 欄位會被視為一個整體，您無法選擇性地更新個別鍵。換句話說，JSON 欄位在合併模式下不支援倒插。
限制與約束
基於上述內容，有幾個限制與約束需要遵循：
upsert請求必須始終包含目標實體的主鍵。目標集合必須已載入並可供查詢。
請求中指定的所有欄位必須存在於目標集合的模式中。
請求中指定的所有欄位的值必須符合模式中定義的資料類型。
對於使用函數從另一個欄位衍生出來的任何欄位，Milvus 會在倒插過程中移除衍生欄位，以允許重新計算。
在集合中倒插實體
在本節中，我們將倒插實體到一個名為my_collection 的集合中。這個集合只有兩個欄位，名為id,vector,title, 和issue 。id 欄位是主要欄位，而title 和issue 欄位是標量欄位。
如果集合中存在這三個實體，它們會被包含 upsert 請求的實體覆蓋。
from pymilvus import MilvusClient
client = MilvusClient(
uri="http://localhost:19530",
token="root:Milvus"
)
data=[
{
"id": 0,
"vector": [-0.619954382375778, 0.4479436794798608, -0.17493894838751745, -0.4248030059917294, -0.8648452746018911],
"title": "Artificial Intelligence in Real Life",
"issue": "vol.12"
}, {
"id": 1,
"vector": [0.4762662251462588, -0.6942502138717026, -0.4490002642657902, -0.628696575798281, 0.9660395877041965],
"title": "Hollow Man",
"issue": "vol.19"
}, {
"id": 2,
"vector": [-0.8864122635045097, 0.9260170474445351, 0.801326976181461, 0.6383943392381306, 0.7563037341572827],
"title": "Treasure Hunt in Missouri",
"issue": "vol.12"
}
]
res = client.upsert(
collection_name='my_collection',
data=data
)
print(res)
# Output
# {'upsert_count': 3}
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import io.milvus.v2.client.ConnectConfig;
import io.milvus.v2.client.MilvusClientV2;
import io.milvus.v2.service.vector.request.UpsertReq;
import io.milvus.v2.service.vector.response.UpsertResp;
import java.util.*;
MilvusClientV2 client = new MilvusClientV2(ConnectConfig.builder()
.uri("http://localhost:19530")
.token("root:Milvus")
.build());
Gson gson = new Gson();
List<JsonObject> data = Arrays.asList(
gson.fromJson("{\"id\": 0, \"vector\": [-0.619954382375778, 0.4479436794798608, -0.17493894838751745, -0.4248030059917294, -0.8648452746018911], \"title\": \"Artificial Intelligence in Real Life\", \"issue\": \"\vol.12\"}", JsonObject.class),
gson.fromJson("{\"id\": 1, \"vector\": [0.4762662251462588, -0.6942502138717026, -0.4490002642657902, -0.628696575798281, 0.9660395877041965], \"title\": \"Hollow Man\", \"issue\": \"vol.19\"}", JsonObject.class),
gson.fromJson("{\"id\": 2, \"vector\": [-0.8864122635045097, 0.9260170474445351, 0.801326976181461, 0.6383943392381306, 0.7563037341572827], \"title\": \"Treasure Hunt in Missouri\", \"issue\": \"vol.12\"}", JsonObject.class),
);
UpsertReq upsertReq = UpsertReq.builder()
.collectionName("my_collection")
.data(data)
.build();
UpsertResp upsertResp = client.upsert(upsertReq);
System.out.println(upsertResp);
// Output:
//
// UpsertResp(upsertCnt=3)
const { MilvusClient, DataType } = require("@zilliz/milvus2-sdk-node")
const address = "http://localhost:19530";
const token = "root:Milvus";
const client = new MilvusClient({address, token});
data = [
{id: 0, vector: [-0.619954382375778, 0.4479436794798608, -0.17493894838751745, -0.4248030059917294, -0.8648452746018911], title: "Artificial Intelligence in Real Life", issue: "vol.12"},
{id: 1, vector: [0.4762662251462588, -0.6942502138717026, -0.4490002642657902, -0.628696575798281, 0.9660395877041965], title: "Hollow Man", issue: "vol.19"},
{id: 2, vector: [-0.8864122635045097, 0.9260170474445351, 0.801326976181461, 0.6383943392381306, 0.7563037341572827], title: "Treasure Hunt in Missouri", issue: "vol.12"},
]
res = await client.upsert({
collection_name: "my_collection",
data: data,
})
console.log(res.upsert_cnt)
// Output
//
// 3
//
import (
"context"
"fmt"
"github.com/milvus-io/milvus/client/v2/column"
"github.com/milvus-io/milvus/client/v2/milvusclient"
)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
milvusAddr := "localhost:19530"
client, err := milvusclient.New(ctx, &milvusclient.ClientConfig{
Address: milvusAddr,
})
if err != nil {
fmt.Println(err.Error())
// handle error
}
defer client.Close(ctx)
titleColumn := column.NewColumnString("title", []string{
"Artificial Intelligence in Real Life", "Hollow Man", "Treasure Hunt in Missouri",
})
issueColumn := column.NewColumnString("issue", []string{
"vol.12", "vol.19", "vol.12"
})
_, err = client.Upsert(ctx, milvusclient.NewColumnBasedInsertOption("my_collection").
WithInt64Column("id", []int64{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}).
WithFloatVectorColumn("vector", 5, [][]float32{
{0.3580376395471989, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592},
{0.19886812562848388, 0.06023560599112088, 0.6976963061752597, 0.2614474506242501, 0.838729485096104},
{0.43742130801983836, -0.5597502546264526, 0.6457887650909682, 0.7894058910881185, 0.20785793220625592},
}).
WithColumns(titleColumn, issueColumn),
)
if err != nil {
fmt.Println(err.Error())
// handle err
}
export CLUSTER_ENDPOINT="http://localhost:19530"
export TOKEN="root:Milvus"
curl --request POST \
--url "${CLUSTER_ENDPOINT}/v2/vectordb/entities/upsert" \
--header "Authorization: Bearer ${TOKEN}" \
--header "Content-Type: application/json" \
--header "Request-Timeout: 10" \
-d '{
"data": [
{"id": 0, "vector": [0.3580376395471989, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592], "title": "Artificial Intelligence in Real Life", "issue": "vol.12"},
{"id": 1, "vector": [0.19886812562848388, 0.06023560599112088, 0.6976963061752597, 0.2614474506242501, 0.838729485096104], "title": "Hollow Man", "issue": "vol.19"},
{"id": 2, "vector": [0.43742130801983836, -0.5597502546264526, 0.6457887650909682, 0.7894058910881185, 0.20785793220625592], "title": "Treasure Hunt in Missouri", "issue": "vol.12"},
],
"collectionName": "my_collection"
}'
# {
# "code": 0,
# "data": {
# "upsertCount": 3,
# "upsertIds": [
# 0,
# 1,
# 2,
# ]
# }
# }
在分區中倒插實體
您也可以向上插入實體到指定的分割區。以下程式碼片段假設您的集合中有一個名為PartitionA的分割區。
這三個實體，如果存在於該分割區中，將會被包含在請求中的實體所覆蓋。
data=[
{
"id": 10,
"vector": [0.06998888224297328, 0.8582816610326578, -0.9657938677934292, 0.6527905683627726, -0.8668460657158576],
"title": "Layour Design Reference",
"issue": "vol.34"
},
{
"id": 11,
"vector": [0.6060703043917468, -0.3765080534566074, -0.7710758854987239, 0.36993888322346136, 0.5507513364206531],
"title": "Doraemon and His Friends",
"issue": "vol.2"
},
{
"id": 12,
"vector": [-0.9041813104515337, -0.9610546012461163, 0.20033003106083358, 0.11842506351635174, 0.8327356724591011],
"title": "Pikkachu and Pokemon",
"issue": "vol.12"
},
]
res = client.upsert(
collection_name="my_collection",
data=data,
partition_name="partitionA"
)
print(res)
# Output
# {'upsert_count': 3}
import io.milvus.v2.service.vector.request.UpsertReq;
import io.milvus.v2.service.vector.response.UpsertResp;
Gson gson = new Gson();
List<JsonObject> data = Arrays.asList(
gson.fromJson("{\"id\": 10, \"vector\": [0.06998888224297328, 0.8582816610326578, -0.9657938677934292, 0.6527905683627726, -0.8668460657158576], \"title\": \"Layour Design Reference\", \"issue\": \"vol.34\"}", JsonObject.class),
gson.fromJson("{\"id\": 11, \"vector\": [0.6060703043917468, -0.3765080534566074, -0.7710758854987239, 0.36993888322346136, 0.5507513364206531], \"title\": \"Doraemon and His Friends\", \"issue\": \"vol.2\"}", JsonObject.class),
gson.fromJson("{\"id\": 12, \"vector\": [-0.9041813104515337, -0.9610546012461163, 0.20033003106083358, 0.11842506351635174, 0.8327356724591011], \"title\": \"Pikkachu and Pokemon\", \"issue\": \"vol.12\"}", JsonObject.class),
);
UpsertReq upsertReq = UpsertReq.builder()
.collectionName("my_collection")
.partitionName("partitionA")
.data(data)
.build();
UpsertResp upsertResp = client.upsert(upsertReq);
System.out.println(upsertResp);
// Output:
//
// UpsertResp(upsertCnt=3)
const { MilvusClient, DataType } = require("@zilliz/milvus2-sdk-node")
// 6. Upsert data in partitions
data = [
{id: 10, vector: [0.06998888224297328, 0.8582816610326578, -0.9657938677934292, 0.6527905683627726, -0.8668460657158576], title: "Layour Design Reference", issue: "vol.34"},
{id: 11, vector: [0.6060703043917468, -0.3765080534566074, -0.7710758854987239, 0.36993888322346136, 0.5507513364206531], title: "Doraemon and His Friends", issue: "vol.2"},
{id: 12, vector: [-0.9041813104515337, -0.9610546012461163, 0.20033003106083358, 0.11842506351635174, 0.8327356724591011], title: "Pikkachu and Pokemon", issue: "vol.12"},
]
res = await client.upsert({
collection_name: "my_collection",
data: data,
partition_name: "partitionA"
})
console.log(res.upsert_cnt)
// Output
//
// 3
//
titleColumn = column.NewColumnString("title", []string{
"Layour Design Reference", "Doraemon and His Friends", "Pikkachu and Pokemon",
})
issueColumn = column.NewColumnString("issue", []string{
"vol.34", "vol.2", "vol.12",
})
_, err = client.Upsert(ctx, milvusclient.NewColumnBasedInsertOption("my_collection").
WithPartition("partitionA").
WithInt64Column("id", []int64{10, 11, 12, 13, 14, 15, 16, 17, 18, 19}).
WithFloatVectorColumn("vector", 5, [][]float32{
{0.3580376395471989, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592},
{0.19886812562848388, 0.06023560599112088, 0.6976963061752597, 0.2614474506242501, 0.838729485096104},
{0.43742130801983836, -0.5597502546264526, 0.6457887650909682, 0.7894058910881185, 0.20785793220625592},
}).
WithColumns(titleColumn, issueColumn),
)
if err != nil {
fmt.Println(err.Error())
// handle err
}
export CLUSTER_ENDPOINT="http://localhost:19530"
export TOKEN="root:Milvus"
curl --request POST \
--url "${CLUSTER_ENDPOINT}/v2/vectordb/entities/upsert" \
--header "Authorization: Bearer ${TOKEN}" \
--header "Content-Type: application/json" \
--header "Request-Timeout: 10" \
-d '{
"data": [
{"id": 10, "vector": [0.06998888224297328, 0.8582816610326578, -0.9657938677934292, 0.6527905683627726, -0.8668460657158576], "title": "Layour Design Reference", "issue": "vol.34"},
{"id": 11, "vector": [0.6060703043917468, -0.3765080534566074, -0.7710758854987239, 0.36993888322346136, 0.5507513364206531], "title": "Doraemon and His Friends", "issue": "vol.2"},
{"id": 12, "vector": [-0.9041813104515337, -0.9610546012461163, 0.20033003106083358, 0.11842506351635174, 0.8327356724591011], "title": "Pikkachu and Pokemon", "issue": "vol.12"},
],
"collectionName": "my_collection",
"partitionName": "partitionA"
}'
# {
# "code": 0,
# "data": {
# "upsertCount": 3,
# "upsertIds": [
# 10,
# 11,
# 12,
# ]
# }
# }
在合併模式中倒插實體Compatible with Milvus v2.6.2+
以下程式碼範例示範如何以部分更新的方式倒插實體。只提供需要更新的欄位及其新值，以及明確的部分更新標誌。
在下面的示例中，upsert 請求中指定的實體的issue 欄位將更新為請求中包含的值。
在合併模式下執行 upsert 時，請確保請求中涉及的實體具有相同的欄位集。假設有兩個或兩個以上的實體要進行 upsert，如以下程式碼片段所示，它們必須包含相同的欄位，以防止錯誤並維護資料完整性。
data=[
{
"id": 1,
"issue": "vol.14"
},
{
"id": 2,
"issue": "vol.7"
}
]
res = client.upsert(
collection_name="my_collection",
data=data,
partial_update=True
)
print(res)
# Output
# {'upsert_count': 2}
JsonObject row1 = new JsonObject();
row1.addProperty("id", 1);
row1.addProperty("issue", "vol.14");
JsonObject row2 = new JsonObject();
row2.addProperty("id", 2);
row2.addProperty("issue", "vol.7");
UpsertReq upsertReq = UpsertReq.builder()
.collectionName("my_collection")
.data(Arrays.asList(row1, row2))
.partialUpdate(true)
.build();
UpsertResp upsertResp = client.upsert(upsertReq);
System.out.println(upsertResp);
// Output:
//
// UpsertResp(upsertCnt=2)
pkColumn := column.NewColumnInt64("id", []int64{1, 2})
issueColumn = column.NewColumnString("issue", []string{
"vol.17", "vol.7",
})
_, err = client.Upsert(ctx, milvusclient.NewColumnBasedInsertOption("my_collection").
WithColumns(pkColumn, issueColumn).
WithPartialUpdate(true),
)
if err != nil {
fmt.Println(err.Error())
// handle err
}
const data=[
{
"id": 1,
"issue": "vol.14"
},
{
"id": 2,
"issue": "vol.7"
}
];
const res = await client.upsert({
collection_name: "my_collection",
data,
partial_update: true
});
console.log(res)
// Output
//
// 2
//
export CLUSTER_ENDPOINT="http://localhost:19530"
export TOKEN="root:Milvus"
export COLLECTION_NAME="my_collection"
export UPSERT_DATA='[
{
"id": 1,
"issue": "vol.14"
},
{
"id": 2,
"issue": "vol.7"
}
]'
curl -X POST "http://localhost:19530/v2/vectordb/entities/upsert" \
-H "Content-Type: application/json" \
-H "Request-Timeout: 10" \
-H "Authorization: Bearer ${TOKEN}" \
-d "{
\"collectionName\": \"${COLLECTION_NAME}\",
\"data\": ${UPSERT_DATA},
\"partialUpdate\": true
}"
# {
# "code": 0,
# "data": {
# "upsertCount": 2,
# "upsertIds": [
# 3,
# 12,
# ]
# }
# }