使用 JSON 字段
本指南介绍如何使用 JSON 字段,例如插入 JSON 值以及使用基本和高级操作符在 JSON 字段中搜索和查询。
概述
JSON 是 Javascript Object Notation 的缩写,是一种基于文本的轻量级简单数据格式。JSON 中的数据采用键值对结构,其中每个键都是一个字符串,可映射到数字、字符串、布尔、列表或数组的值。使用 Milvus 群集,可以将字典作为字段值存储在 Collections 中。
例如,以下代码会随机生成键值对,每个键值对都包含一个键值为颜色的 JSON 字段。
# 3. Insert randomly generated vectors
colors = ["green", "blue", "yellow", "red", "black", "white", "purple", "pink", "orange", "brown", "grey"]
data = []
for i in range(1000):
current_color = random.choice(colors)
current_tag = random.randint(1000, 9999)
current_coord = [ random.randint(0, 40) for _ in range(3) ]
current_ref = [ [ random.choice(colors) for _ in range(3) ] for _ in range(3) ]
data.append({
"id": i,
"vector": [ random.uniform(-1, 1) for _ in range(5) ],
"color": {
"label": current_color,
"tag": current_tag,
"coord": current_coord,
"ref": current_ref
}
})
print(data[0])
import java.util.*;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
// 3. Insert randomly generated vectors and JSON data into the collection
List<String> colors = Arrays.asList("green", "blue", "yellow", "red", "black", "white", "purple", "pink", "orange", "brown", "grey");
List<JsonObject> data = new ArrayList<>();
Gson gson = new Gson();
Random rand = new Random();
for (int i=0; i<1000; i++) {
String current_color = colors.get(rand.nextInt(colors.size()-1));
Integer current_tag = rand.nextInt(8999) + 1000;
List<Integer> current_coord = Arrays.asList(rand.nextInt(40), rand.nextInt(40), rand.nextInt(40));
List<List<String>> current_ref = Arrays.asList(
Arrays.asList(colors.get(rand.nextInt(colors.size()-1)), colors.get(rand.nextInt(colors.size()-1)), colors.get(rand.nextInt(colors.size()-1))),
Arrays.asList(colors.get(rand.nextInt(colors.size()-1)), colors.get(rand.nextInt(colors.size()-1)), colors.get(rand.nextInt(colors.size()-1))),
Arrays.asList(colors.get(rand.nextInt(colors.size()-1)), colors.get(rand.nextInt(colors.size()-1)), colors.get(rand.nextInt(colors.size()-1)))
);
JsonObject row = new JsonObject();
row.addProperty("id", (long) i);
row.add("vector", gson.toJsonTree(Arrays.asList(rand.nextFloat(), rand.nextFloat(), rand.nextFloat(), rand.nextFloat(), rand.nextFloat())));
JsonObject color = new JsonObject();
color.addProperty("label", current_color);
color.addProperty("tag", current_tag);
color.add("coord", gson.toJsonTree(current_coord));
color.add("ref", gson.toJsonTree(current_ref));
row.add("color", color);
data.add(row);
}
System.out.println(data.get(0));
// 3. Insert randomly generated vectors
const colors = ["green", "blue", "yellow", "red", "black", "white", "purple", "pink", "orange", "brown", "grey"]
var data = []
for (let i = 0; i < 1000; i++) {
const current_color = colors[Math.floor(Math.random() * colors.length)]
const current_tag = Math.floor(Math.random() * 8999 + 1000)
const current_coord = Array(3).fill(0).map(() => Math.floor(Math.random() * 40))
const current_ref = [ Array(3).fill(0).map(() => colors[Math.floor(Math.random() * colors.length)]) ]
data.push({
id: i,
vector: [Math.random(), Math.random(), Math.random(), Math.random(), Math.random()],
color: {
label: current_color,
tag: current_tag,
coord: current_coord,
ref: current_ref
}
})
}
console.log(data[0])
您可以通过查看第一个条目来查看生成数据的结构。
{
"id": 0,
"vector": [
-0.8017921296923975,
0.550046715206634,
0.764922589768134,
0.6371433836123146,
0.2705233937454232
],
"color": {
"label": "blue",
"tag": 9927,
"coord": [
22,
36,
6
],
"ref": [
[
"blue",
"green",
"white"
],
[
"black",
"green",
"pink"
],
[
"grey",
"black",
"brown"
]
]
}
}
注释
确保列表或数组中的所有值都是相同的数据类型。
JSON 字段值中的任何嵌套字典都将被视为字符串。
仅使用字母数字字符和下划线来命名 JSON 键,因为其他字符可能会导致过滤或搜索出现问题。
- 目前,还不能为 JSON 字段编制索引,这可能会导致过滤耗时。不过,这一限制将在即将发布的版本中得到解决。
定义 JSON 字段
要定义 JSON 字段,只需遵循与定义其他类型字段相同的步骤即可。
有关参数的更多信息,请参阅 MilvusClient
, create_schema()
, add_field()
, add_index()
, create_collection()
和 get_load_state()
在 SDK 参考资料中。
有关参数的更多信息,请参阅 MilvusClientV2
, createSchema()
, addField()
, IndexParam
, createCollection()
和 getLoadState()
在 SDK 参考资料中。
有关参数的更多信息,请参阅 MilvusClient
和 createCollection()
和 createCollection()
SDK 参考资料中的参数。
import random, time
from pymilvus import connections, MilvusClient, DataType
CLUSTER_ENDPOINT = "http://localhost:19530"
# 1. Set up a Milvus client
client = MilvusClient(
uri=CLUSTER_ENDPOINT
)
# 2. Create a collection
schema = MilvusClient.create_schema(
auto_id=False,
enable_dynamic_field=False,
)
schema.add_field(field_name="id", datatype=DataType.INT64, is_primary=True)
schema.add_field(field_name="vector", datatype=DataType.FLOAT_VECTOR, dim=5)
schema.add_field(field_name="color", datatype=DataType.JSON)
index_params = MilvusClient.prepare_index_params()
index_params.add_index(
field_name="id",
index_type="STL_SORT"
)
index_params.add_index(
field_name="vector",
index_type="IVF_FLAT",
metric_type="L2",
params={"nlist": 1024}
)
client.create_collection(
collection_name="test_collection",
schema=schema,
index_params=index_params
)
res = client.get_load_state(
collection_name="test_collection"
)
print(res)
# Output
#
# {
# "state": "<LoadState: Loaded>"
# }
import io.milvus.v2.client.MilvusClientV2;
import io.milvus.v2.client.ConnectConfig;
import io.milvus.v2.common.DataType;
import io.milvus.v2.common.IndexParam;
import io.milvus.v2.service.collection.request.*;
import io.milvus.v2.service.vector.request.*;
import io.milvus.v2.service.vector.request.data.*;
import io.milvus.v2.service.vector.response.*;
String CLUSTER_ENDPOINT = "http://localhost:19530";
// 1. Connect to Milvus server
ConnectConfig connectConfig = ConnectConfig.builder()
.uri(CLUSTER_ENDPOINT)
.build();
MilvusClientV2 client = new MilvusClientV2(connectConfig);
// 2. Create a collection in customized setup mode
// 2.1 Create schema
CreateCollectionReq.CollectionSchema schema = client.createSchema();
// 2.2 Add fields to schema
schema.addField(AddFieldReq.builder()
.fieldName("id")
.dataType(DataType.Int64)
.isPrimaryKey(true)
.autoID(false)
.build());
schema.addField(AddFieldReq.builder()
.fieldName("vector")
.dataType(DataType.FloatVector)
.dimension(5)
.build());
schema.addField(AddFieldReq.builder()
.fieldName("color")
.dataType(DataType.JSON)
.build());
// 2.3 Prepare index parameters
IndexParam indexParamForIdField = IndexParam.builder()
.fieldName("id")
.indexType(IndexParam.IndexType.STL_SORT)
.build();
Map<String, Object> params = new HashMap<>();
params.put("nlist", 1024);
IndexParam indexParamForVectorField = IndexParam.builder()
.fieldName("vector")
.indexType(IndexParam.IndexType.IVF_FLAT)
.metricType(IndexParam.MetricType.IP)
.extraParams(params)
.build();
List<IndexParam> indexParams = new ArrayList<>();
indexParams.add(indexParamForIdField);
indexParams.add(indexParamForVectorField);
// 2.4 Create a collection with schema and index parameters
CreateCollectionReq customizedSetupReq = CreateCollectionReq.builder()
.collectionName("test_collection")
.collectionSchema(schema)
.indexParams(indexParams)
.build();
client.createCollection(customizedSetupReq);
// 2.5 Check if the collection is loaded
GetLoadStateReq getLoadStateReq = GetLoadStateReq.builder()
.collectionName("test_collection")
.build();
Boolean isLoaded = client.getLoadState(getLoadStateReq);
System.out.println(isLoaded);
// Output:
// true
const { MilvusClient, DataType, sleep } = require("@zilliz/milvus2-sdk-node")
const address = "http://localhost:19530"
async function main() {
// 1. Set up a Milvus Client
client = new MilvusClient({address});
// 2. Create a collection
// 2.1 Define fields
const fields = [
{
name: "id",
data_type: DataType.Int64,
is_primary_key: true,
auto_id: false
},
{
name: "vector",
data_type: DataType.FloatVector,
dim: 5
},
{
name: "color",
data_type: DataType.JSON,
}
]
// 2.2 Prepare index parameters
const index_params = [{
field_name: "vector",
index_type: "IVF_FLAT",
metric_type: "IP",
params: { nlist: 1024}
}]
// 2.3 Create a collection with fields and index parameters
res = await client.createCollection({
collection_name: "test_collection",
fields: fields,
index_params: index_params
})
console.log(res.error_code)
// Output
//
// Success
//
res = await client.getLoadState({
collection_name: "test_collection",
})
console.log(res.state)
// Output
//
// LoadStateLoaded
//
有关参数的更多信息,请参阅 MilvusClient
, create_schema()
, add_field()
, add_index()
, create_collection()
和 get_load_state()
在 SDK 参考资料中。
有关参数的更多信息,请参阅 MilvusClientV2
, createSchema()
, addField()
, IndexParam
, createCollection()
和 getLoadState()
在 SDK 参考资料中。
有关参数的更多信息,请参阅 MilvusClient
, createCollection()
和 getLoadState()
的更多信息。
插入字段值
从CollectionSchema
对象创建一个 Collections 后,就可以向其中插入字典,如上面的字典。
使用 insert()
方法将数据插入 Collections。
使用 insert()
方法将数据插入 Collections。
使用 insert()
方法将数据插入 Collections。
res = client.insert(
collection_name="test_collection",
data=data
)
print(res)
# Output
#
# {
# "insert_count": 1000,
# "ids": [
# 0,
# 1,
# 2,
# 3,
# 4,
# 5,
# 6,
# 7,
# 8,
# 9,
# "(990 more items hidden)"
# ]
# }
// 3.1 Insert data into the collection
InsertReq insertReq = InsertReq.builder()
.collectionName("test_collection")
.data(data)
.build();
InsertResp insertResp = client.insert(insertReq);
System.out.println(insertResp.getInsertCnt());
// Output:
// 1000
// 3. Insert randomly generated vectors
const colors = ["green", "blue", "yellow", "red", "black", "white", "purple", "pink", "orange", "brown", "grey"]
var data = []
for (let i = 0; i < 1000; i++) {
const current_color = colors[Math.floor(Math.random() * colors.length)]
const current_tag = Math.floor(Math.random() * 8999 + 1000)
const current_coord = Array(3).fill(0).map(() => Math.floor(Math.random() * 40))
const current_ref = [ Array(3).fill(0).map(() => colors[Math.floor(Math.random() * colors.length)]) ]
data.push({
id: i,
vector: [Math.random(), Math.random(), Math.random(), Math.random(), Math.random()],
color: {
label: current_color,
tag: current_tag,
coord: current_coord,
ref: current_ref
}
})
}
console.log(data[0])
// Output
//
// {
// id: 0,
// vector: [
// 0.11455530974226114,
// 0.21704086958595314,
// 0.9430119822312437,
// 0.7802712923612023,
// 0.9106927960926137
// ],
// color: { label: 'grey', tag: 7393, coord: [ 22, 1, 22 ], ref: [ [Array] ] }
// }
//
res = await client.insert({
collection_name: "test_collection",
data: data,
})
console.log(res.insert_cnt)
// Output
//
// 1000
//
基本标量过滤
一旦添加了所有数据,您就可以使用 JSON 字段中的键进行搜索和查询,方式与使用标准标量字段相同。
有关参数的更多信息,请参阅 search()
有关参数的更多信息,请参阅 SDK 参考资料中的
有关参数的更多信息,请参阅 search()
有关参数的更多信息,请参阅 SDK 参考资料中的
有关参数的更多信息,请参阅 search()
有关参数的更多信息,请参阅 SDK 参考资料中的
# 4. Basic search with a JSON field
query_vectors = [ [ random.uniform(-1, 1) for _ in range(5) ]]
res = client.search(
collection_name="test_collection",
data=query_vectors,
filter='color["label"] in ["red"]',
search_params={
"metric_type": "L2",
"params": {"nprobe": 16}
},
output_fields=["id", "color"],
limit=3
)
print(res)
# Output
#
# [
# [
# {
# "id": 460,
# "distance": 0.4016231596469879,
# "entity": {
# "id": 460,
# "color": {
# "label": "red",
# "tag": 5030,
# "coord": [14, 32, 40],
# "ref": [
# [ "pink", "green", "brown" ],
# [ "red", "grey", "black"],
# [ "red", "yellow", "orange"]
# ]
# }
# }
# },
# {
# "id": 785,
# "distance": 0.451080858707428,
# "entity": {
# "id": 785,
# "color": {
# "label": "red",
# "tag": 5290,
# "coord": [31, 13, 23],
# "ref": [
# ["yellow", "pink", "pink"],
# ["purple", "grey", "orange"],
# ["grey", "purple", "pink"]
# ]
# }
# }
# },
# {
# "id": 355,
# "distance": 0.5839247703552246,
# "entity": {
# "id": 355,
# "color": {
# "label": "red",
# "tag": 8725,
# "coord": [5, 10, 22],
# "ref": [
# ["white", "purple", "yellow"],
# ["white", "purple", "white"],
# ["orange", "white", "pink"]
# ]
# }
# }
# }
# ]
# ]
// 4. Search with partition key
List<BaseVector> query_vectors = Collections.singletonList(new FloatVec(new float[]{0.3580376395471989f, -0.6023495712049978f, 0.18414012509913835f, -0.26286205330961354f, 0.9029438446296592f}));
SearchReq searchReq = SearchReq.builder()
.collectionName("test_collection")
.data(query_vectors)
.filter("color[\"label\"] in [\"red\"]")
.outputFields(Arrays.asList("id", "color"))
.topK(3)
.build();
SearchResp searchResp = client.search(searchReq);
List<List<SearchResp.SearchResult>> searchResults = searchResp.getSearchResults();
for (List<SearchResp.SearchResult> results : searchResults) {
System.out.println("TopK results:");
for (SearchResp.SearchResult result : results) {
System.out.println(result);
}
}
// Output:
// SearchResp.SearchResult(entity=\{color=\{"label":"red","tag":1018,"coord":[3,30,1],"ref":[["yellow","brown","orange"],["yellow","purple","blue"],["green","purple","purple"]]}, id=295}, score=1.1190735, id=295)
// SearchResp.SearchResult(entity=\{color=\{"label":"red","tag":8141,"coord":[38,31,29],"ref":[["blue","white","white"],["green","orange","green"],["yellow","green","black"]]}, id=667}, score=1.0679582, id=667)
// SearchResp.SearchResult(entity=\{color=\{"label":"red","tag":6837,"coord":[29,9,8],"ref":[["green","black","blue"],["purple","white","green"],["red","blue","black"]]}, id=927}, score=1.0029297, id=927)
// 4. Basic search with a JSON field
query_vectors = [[0.6765405125697714, 0.759217474274025, 0.4122471841491111, 0.3346805565394215, 0.09679748345514638]]
res = await client.search({
collection_name: "test_collection",
data: query_vectors,
filter: 'color["label"] in ["red"]',
output_fields: ["color", "id"],
limit: 3
})
console.log(JSON.stringify(res.results, null, 4))
// Output
//
// [
// {
// "score": 1.777988076210022,
// "id": "595",
// "color": {
// "label": "red",
// "tag": 7393,
// "coord": [31,34,18],
// "ref": [
// ["grey", "white", "orange"]
// ]
// }
// },
// {
// "score": 1.7542595863342285,
// "id": "82",
// "color": {
// "label": "red",
// "tag": 8636,
// "coord": [4,37,29],
// "ref": [
// ["brown", "brown", "pink"]
// ]
// }
// },
// {
// "score": 1.7537562847137451,
// "id": "748",
// "color": {
// "label": "red",
// "tag": 1626,
// "coord": [31,4,25
// ],
// "ref": [
// ["grey", "green", "blue"]
// ]
// }
// }
// ]
//
高级标量过滤
Milvus 为 JSON 字段中的标量过滤提供了一组高级过滤器。这些过滤器是JSON_CONTAINS
、JSON_CONTAINS_ALL
和JSON_CONTAINS_ANY
。
过滤以
["blue", "brown", "grey"]
作为参考颜色集的所有实体。# 5. Advanced search within a JSON field res = client.query( collection_name="test_collection", data=query_vectors, filter='JSON_CONTAINS(color["ref"], ["blue", "brown", "grey"])', output_fields=["id", "color"], limit=3 ) print(res) # Output # # [ # { # "id": 79, # "color": { # "label": "orange", # "tag": 8857, # "coord": [ # 10, # 14, # 5 # ], # "ref": [ # [ # "yellow", # "white", # "green" # ], # [ # "blue", # "purple", # "purple" # ], # [ # "blue", # "brown", # "grey" # ] # ] # } # }, # { # "id": 371, # "color": { # "label": "black", # "tag": 1324, # "coord": [ # 2, # 18, # 32 # ], # "ref": [ # [ # "purple", # "orange", # "brown" # ], # [ # "blue", # "brown", # "grey" # ], # [ # "purple", # "blue", # "blue" # ] # ] # } # }, # { # "id": 590, # "color": { # "label": "red", # "tag": 3340, # "coord": [ # 13, # 21, # 13 # ], # "ref": [ # [ # "yellow", # "yellow", # "red" # ], # [ # "blue", # "brown", # "grey" # ], # [ # "pink", # "yellow", # "purple" # ] # ] # } # } # ]
// 5. Advanced search within a JSON field searchReq = SearchReq.builder() .collectionName("test_collection") .data(query_vectors) .filter("JSON_CONTAINS(color[\"ref\"], [\"purple\", \"pink\", \"orange\"])") .outputFields(Arrays.asList("id", "color")) .topK(3) .build(); searchResp = client.search(searchReq); searchResults = searchResp.getSearchResults(); for (List<SearchResp.SearchResult> results : searchResults) { System.out.println("TopK results:"); for (SearchResp.SearchResult result : results) { System.out.println(result); } } // Output: // SearchResp.SearchResult(entity={color={"label":"pink","tag":2963,"coord":[15,33,30],"ref":[["green","white","white"],["purple","pink","orange"],["yellow","black","pink"]]}, id=273}, score=0.46558747, id=273) // SearchResp.SearchResult(entity={color={"label":"pink","tag":4027,"coord":[32,34,19],"ref":[["red","white","blue"],["white","pink","yellow"],["purple","pink","orange"]]}, id=344}, score=0.2637315, id=344) // SearchResp.SearchResult(entity={color={"label":"black","tag":1603,"coord":[33,12,23],"ref":[["pink","brown","black"],["black","purple","black"],["purple","pink","orange"]]}, id=205}, score=0.26133868, id=205)
// 5. Advanced search within a JSON field res = await client.search({ collection_name: "test_collection", data: query_vectors, filter: 'JSON_CONTAINS(color["ref"], ["blue", "brown", "grey"])', output_fields: ["color", "id"], limit: 3 }) console.log(JSON.stringify(res.results, null, 4)) // Output // // [ // { // "id": 79, // "color": { // "label": "orange", // "tag": 8857, // "coord": [ // 10, // 14, // 5 // ], // "ref": [ // [ // "yellow", // "white", // "green" // ], // [ // "blue", // "purple", // "purple" // ], // [ // "blue", // "brown", // "grey" // ] // ] // } // }, // { // "id": 371, // "color": { // "label": "black", // "tag": 1324, // "coord": [ // 2, // 18, // 32 // ], // "ref": [ // [ // "purple", // "orange", // "brown" // ], // [ // "blue", // "brown", // "grey" // ], // [ // "purple", // "blue", // "blue" // ] // ] // } // }, // { // "id": 590, // "color": { // "label": "red", // "tag": 3340, // "coord": [ // 13, // 21, // 13 // ], // "ref": [ // [ // "yellow", // "yellow", // "red" // ], // [ // "blue", // "brown", // "grey" // ], // [ // "pink", // "yellow", // "purple" // ] // ] // } // } // ] //
过滤协调器为
[4, 5]
的实体。res = client.query( collection_name="test_collection", data=query_vectors, filter='JSON_CONTAINS_ALL(color["coord"], [4, 5])', output_fields=["id", "color"], limit=3 ) print(res) # Output # # [ # { # "id": 281, # "color": { # "label": "red", # "tag": 3645, # "coord": [ # 5, # 33, # 4 # ], # "ref": [ # [ # "orange", # "blue", # "pink" # ], # [ # "purple", # "blue", # "purple" # ], # [ # "black", # "brown", # "yellow" # ] # ] # } # }, # { # "id": 464, # "color": { # "label": "brown", # "tag": 6261, # "coord": [ # 5, # 9, # 4 # ], # "ref": [ # [ # "purple", # "purple", # "brown" # ], # [ # "black", # "pink", # "white" # ], # [ # "brown", # "grey", # "brown" # ] # ] # } # }, # { # "id": 567, # "color": { # "label": "green", # "tag": 4589, # "coord": [ # 5, # 39, # 4 # ], # "ref": [ # [ # "purple", # "yellow", # "white" # ], # [ # "yellow", # "yellow", # "brown" # ], # [ # "blue", # "red", # "yellow" # ] # ] # } # } # ]
searchReq = SearchReq.builder() .collectionName("test_collection") .data(query_vectors) .filter("JSON_CONTAINS_ALL(color[\"coord\"], [4, 5])") .outputFields(Arrays.asList("id", "color")) .topK(3) .build(); searchResp = client.search(searchReq); searchResults = searchResp.getSearchResults(); for (List<SearchResp.SearchResult> results : searchResults) { System.out.println("TopK results:"); for (SearchResp.SearchResult result : results) { System.out.println(result); } } // Output: // SearchResp.SearchResult(entity={color={"label":"green","tag":9899,"coord":[5,4,25],"ref":[["purple","black","yellow"],["orange","green","purple"],["red","purple","pink"]]}, id=708}, score=0.56576324, id=708) // SearchResp.SearchResult(entity={color={"label":"red","tag":2176,"coord":[4,5,23],"ref":[["red","black","green"],["brown","orange","brown"],["brown","orange","yellow"]]}, id=981}, score=0.5656834, id=981) // SearchResp.SearchResult(entity={color={"label":"pink","tag":3085,"coord":[5,3,4],"ref":[["yellow","orange","green"],["black","pink","red"],["orange","blue","blue"]]}, id=221}, score=0.3708634, id=221)
res = await client.search({ collection_name: "test_collection", data: query_vectors, filter: 'JSON_CONTAINS_ALL(color["coord"], [4, 5])', output_fields: ["color", "id"], limit: 3 }) console.log(JSON.stringify(res.results, null, 4)) // Output // // [ // { // "score": 1.8944344520568848, // "id": "792", // "color": { // "label": "purple", // "tag": 8161, // "coord": [ // 4, // 38, // 5 // ], // "ref": [ // [ // "red", // "white", // "grey" // ] // ] // } // }, // { // "score": 1.2801706790924072, // "id": "489", // "color": { // "label": "red", // "tag": 4358, // "coord": [ // 5, // 4, // 1 // ], // "ref": [ // [ // "blue", // "orange", // "orange" // ] // ] // } // }, // { // "score": 1.2097992897033691, // "id": "656", // "color": { // "label": "red", // "tag": 7856, // "coord": [ // 5, // 20, // 4 // ], // "ref": [ // [ // "black", // "orange", // "white" // ] // ] // } // } // ] //
过滤协调器包含
4
或5
的实体。res = client.query( collection_name="test_collection", data=query_vectors, filter='JSON_CONTAINS_ANY(color["coord"], [4, 5])', output_fields=["id", "color"], limit=3 ) print(res) # Output # # [ # { # "id": 0, # "color": { # "label": "yellow", # "tag": 6340, # "coord": [ # 40, # 4, # 40 # ], # "ref": [ # [ # "purple", # "yellow", # "orange" # ], # [ # "green", # "grey", # "purple" # ], # [ # "black", # "white", # "yellow" # ] # ] # } # }, # { # "id": 2, # "color": { # "label": "brown", # "tag": 9359, # "coord": [ # 38, # 21, # 5 # ], # "ref": [ # [ # "red", # "brown", # "white" # ], # [ # "purple", # "red", # "brown" # ], # [ # "pink", # "grey", # "black" # ] # ] # } # }, # { # "id": 7, # "color": { # "label": "green", # "tag": 3560, # "coord": [ # 5, # 9, # 5 # ], # "ref": [ # [ # "blue", # "orange", # "green" # ], # [ # "blue", # "blue", # "black" # ], # [ # "green", # "purple", # "green" # ] # ] # } # } # ]
searchReq = SearchReq.builder() .collectionName("test_collection") .data(query_vectors) .filter("JSON_CONTAINS_ANY(color[\"coord\"], [4, 5])") .outputFields(Arrays.asList("id", "color")) .topK(3) .build(); searchResp = client.search(searchReq); searchResults = searchResp.getSearchResults(); for (List<SearchResp.SearchResult> results : searchResults) { System.out.println("TopK results:"); for (SearchResp.SearchResult result : results) { System.out.println(result); } } // Output: // SearchResp.SearchResult(entity={color={"label":"brown","tag":8414,"coord":[3,4,15],"ref":[["blue","green","pink"],["red","orange","pink"],["yellow","pink","green"]]}, id=11}, score=1.18235, id=11) // SearchResp.SearchResult(entity={color={"label":"yellow","tag":2846,"coord":[20,4,15],"ref":[["white","black","purple"],["green","black","yellow"],["red","purple","brown"]]}, id=589}, score=1.1414992, id=589) // SearchResp.SearchResult(entity={color={"label":"pink","tag":6744,"coord":[25,33,5],"ref":[["orange","purple","white"],["white","pink","brown"],["red","pink","red"]]}, id=567}, score=1.1087029, id=567)
res = await client.search({ collection_name: "test_collection", data: query_vectors, filter: 'JSON_CONTAINS_ANY(color["coord"], [4, 5])', output_fields: ["color", "id"], limit: 3 }) console.log(JSON.stringify(res.results, null, 4)) // Output // // [ // { // "score": 1.9083369970321655, // "id": "453", // "color": { // "label": "brown", // "tag": 8788, // "coord": [ // 21, // 18, // 5 // ], // "ref": [ // [ // "pink", // "black", // "brown" // ] // ] // } // }, // { // "score": 1.8944344520568848, // "id": "792", // "color": { // "label": "purple", // "tag": 8161, // "coord": [ // 4, // 38, // 5 // ], // "ref": [ // [ // "red", // "white", // "grey" // ] // ] // } // }, // { // "score": 1.8615753650665283, // "id": "272", // "color": { // "label": "grey", // "tag": 3400, // "coord": [ // 5, // 1, // 32 // ], // "ref": [ // [ // "purple", // "green", // "white" // ] // ] // } // } // ] //
JSON 过滤器参考
在处理 JSON 字段时,您可以使用 JSON 字段作为过滤器,也可以使用它的某些特定键。
注释
- Milvus 将字符串值原封不动地存储在 JSON 字段中,而不执行语义转义或转换。
例如,'a"b'
、"a'b"
、'a\\\\'b'
和"a\\\\"b"
将按原样保存,而'a'b'
和"a"b"
将被视为无效值。
要使用 JSON 字段构建过滤表达式,可以利用字段中的键。
如果键值是整数或浮点数,则可以将其与另一个整数或浮点数键值或 INT32/64 或 FLOAT32/64 字段进行比较。
如果键值是字符串,则只能与另一个字符串键或 VARCHAR 字段进行比较。
JSON 字段中的基本操作符
下表假定名为json_key
的 JSON 字段的值有一个名为A
的键。在使用 JSON 字段键构建布尔表达式时,请将其作为参考。
操作符 | 示例 | 备注 |
---|---|---|
< | 'json_field["A"] < 3' | 如果json_field["A"] 的值小于3 ,则该表达式的值为 true。 |
> | 'json_field["A"] > 1' | 如果json_field["A"] 的值大于1 ,则此表达式的值为 true。 |
== | 'json_field["A"] == 1' | 如果json_field["A"] 的值等于1 ,则此表达式的值为真。 |
!= | 'json_field["A"][0]' != "abc"' | 如果 - json_field 没有名为A 的键,则此表达式的值为真。- json_field 有一个名为A 的键,但json_field["A"] 不是数组。- json_field["A"] 是一个空数组。- json_field["A"] 是一个数组,但第一个元素不是abc 。 |
<= | 'json_field["A"] <= 5' | 如果json_field["A"] 的值小于或等于5 ,则此表达式的值为 true。 |
>= | 'json_field["A"] >= 1' | 如果json_field["A"] 的值大于或等于1 ,则此表达式的值为真。 |
不 | 'not json_field["A"] == 1' | 如果 - json_field 没有名为A 的键,则此表达式的值为真。- json_field["A"] 不等于1 . |
在 | 'json_field["A"] in [1, 2, 3]' | 如果json_field["A"] 的值是1 ,2 或3 ,则此表达式的值为 true。 |
和 (&&) | 'json_field["A"] > 1 && json_field["A"] < 3' | 如果json_field["A"] 的值大于 1 且小于3 ,则此表达式的值为 true。 |
或 (||) | ‘json_field[“A”] > 1 || json_field[“A”] < 3’ | 如果json_field["A"] 的值大于1 或小于3 ,则此表达式的值为 true。 |
存在 | 'exists json_field["A"]' | 如果json_field 有一个名为A 的键,则此表达式的值为真。 |
高级操作符
以下操作符专门针对 JSON 字段:
json_contains(identifier, jsonExpr)
此操作符可过滤标识符包含指定 JSON 表达式的实体。
例 1:
{"x": [1,2,3]}
json_contains(x, 1) # => True (x contains 1.) json_contains(x, "a") # => False (x does not contain a member "a".)
例 2:
{"x", [[1,2,3], [4,5,6], [7,8,9]]}
json_contains(x, [1,2,3]) # => True (x contains [1,2,3].) json_contains(x, [3,2,1]) # => False (x does contain a member [3,2,1].)
json_contains_all(identifier, jsonExpr)
此操作符可筛选标识符包含 JSON 表达式所有成员的实体。
示例 3
{"x": [1,2,3,4,5,7,8]}
json_contains_all(x, [1,2,8]) # => True (x contains 1, 2, and 8.) json_contains_all(x, [4,5,6]) # => False (x does not has a member 6.)
json_contains_any(identifier, jsonExpr)
此操作符可筛选标识符包含 JSON 表达式任意成员的实体。
示例 1: 示例 2: 此操作符可筛选标识符包含所有 JSON 表达式成员的实体:
{"x": [1,2,3,4,5,7,8]}
json_contains_any(x, [1,2,8]) # => True (x contains 1, 2, and 8.) json_contains_any(x, [4,5,6]) # => True (x contains 4 and 5.) json_contains_any(x, [6,9]) # => False (x contains none of 6 and 9.)