About Milvus-sdk-node

Milvus-sdk-node is a node.js SDK for Milvus, an open-source vector database. Its source code is available on GitHub.

Compatibility

The following collection shows Milvus versions and recommended @zilliz/milvus2-sdk-node versions:

Milvus version	Node sdk version	Installation
v2.2.0+	latest	yarn add @zilliz/milvus2-sdk-node

Dependencies

• Milvus
• Zilliz Cloud
• Node: v14+

Installation

The recommended way to get started using the Milvus node.js client is by using npm (Node package manager) to install the dependency in your project.

npm install @zilliz/milvus2-sdk-node
# or ...
yarn add @zilliz/milvus2-sdk-node

This will download the Milvus node.js client and add a dependency entry in your package.json file.

Quick Start

This guide will show you how to set up a simple application using Node.js and Milvus. Its scope is only how to set up the node.js client and perform the simple CRUD operations. For more in-depth coverage, see the Milvus official website.

Create the package.json file

First, create a directory where your application will live.

mkdir myProject
cd myProject

Enter the following command and answer the questions to create the initial structure for your new project:

npm init -y

Next, install this client as a dependency.

npm install @zilliz/milvus2-sdk-node

Start a milvus server

# Download the milvus standalone yaml file
$ wget https://github.com/milvus-io/milvus/releases/download/v2.2.8/milvus-standalone-docker-compose.yml -O docker-compose.yml

# start the milvus server
sudo docker-compose up -d

Connect to Milvus

Create a new app.js file and add the following code to try out some basic vector operations using the Milvus node.js client.

import { MilvusClient, DataType } from "@zilliz/milvus2-sdk-node";

const address = "your-milvus-ip";
const username = "your-milvus-username"; // optional username
const password = "your-milvus-password"; // optional password
const ssl = false; // secure or not

// connect to milvus
const client = new MilvusClient({ address, ssl, username, password });

Parameters	Description	Type	Example
address	The Milvus IP address	String	'192.168.0.1:19530'
ssl?	SSL connection. It is false by default.	Boolean	false
username?	The username used to connect to Milvus	String	milvus
address?	The password used to connect to Milvus	String	milvus
maxRetries?	The number of retries for the grpc method, by default: 3	Number	3
retryDelay?	The delay between attempts at retrying a failed grpc method in ms, by default: 30	Number	30
channelOptions?	an optional configuration object that can be passed to a gRPC client when creating a channel to connect to a gRPC server	Number	30

define schema for collection

The code shows an example of how to define a schema for a collection in Milvus using the Milvus Node SDK. A schema defines the properties of a collection, such as the names and data types of the fields that make up the vectors.

// define schema
const collection_name = `book`;
const dim = 128;
const schema = [
  {
    name: `book_id`,
    description: `customized primary id`,
    data_type: DataType.Int64,
    is_primary_key: true,
    autoID: false,
  },
  {
    name: `word_count`,
    description: `word count`,
    data_type: DataType.Int64,
  },
  {
    name: `book_intro`,
    description: `word count`,
    data_type: DataType.FloatVector,
    dim: dim,
  },
];

In the code, the schema variable is defined as an array of objects, where each object represents a field in the collection. The fields are defined using the following properties:

name: The name of the field, which must be unique within the collection.

description: A description of the field, which can be used to provide additional information about the field.

data_type: The data type of the field, which can be one of several predefined data types such as Int64 or FloatVector.

is_primary_key: A boolean flag that indicates whether the field is a primary key. Primary keys are unique identifiers for each vector in the collection, and can be used to efficiently retrieve specific vectors.

autoID: A boolean flag that indicates whether the primary key is automatically generated by Milvus.

dim: The dimensionality of the vector field. For fields of type FloatVector, this specifies the number of dimensions in each vector.

create collection

await client.createCollection({
  collection_name,
  description: `my first collection`,
  fields: schema,
});

prepare data

When using the Milvus Node SDK to insert data into a collection, it's important to ensure that the data format of the input matches the schema defined for the collection. The data format used by the Milvus Node SDK consists of an array of objects, where each object represents an entity. Typically, each object contains a unique identifier for the entity, which can be an integer or string, and a vector field that stores the feature values as an array of floating-point numbers.

// generate mock data
const fields_data = [];

// generate mock data
for (let i = 0; i < 1000; i++) {
  // create a new object with random values for each field
  const r = {
    book_id: Math.floor(Math.random() * 100000), // generate a random book ID
    word_count: Math.floor(Math.random() * 1000), // generate a random word count
    book_intro: [...Array(dim)].map(() => Math.random()), // generate a random vector for book_intro
  };
  // add the new object to the fields_data array
  fields_data.push(r);
}

insert data into collection

Once we have the data, you can insert data into the collection.

await client.insert({
  collection_name,
  fields_data,
});

create index

By creating an index and loading the collection into memory, you can improve the performance of search and retrieval operations in Milvus, making it faster and more efficient to work with large-scale datasets.

// create index
await client.createIndex({
  collection_name,
  field_name: "book_intro",
  index_name: "myindex",
  index_type: "HNSW",
  params: { efConstruction: 10, M: 4 },
  metric_type: "L2",
});

Milvus supports several different types of indexes, each of which is optimized for different use cases and data distributions. Some of the most commonly used index types in Milvus include IVF_FLAT, IVF_SQ8, IVF_PQ, and HNSW. When creating an index in Milvus, you must choose an appropriate index type based on your specific use case and data distribution.

collection_name: The name of the collection to create the index for.

field_name: The name of the field to create the index on.

index_name: The name of the index to create.

index_type: The type of index to create.

param: The index build parameters, please refer to the index docs.

metric_type: The distance metric to use when computing the similarity between vectors. In this case, the metric type is set to L2, which is a commonly used metric for computing the Euclidean distance between vectors.

load collection

When you create a collection in Milvus, the collection data is initially stored on disk, and it is not immediately available for search and retrieval. In order to search or retrieve data from the collection, you must first load the collection into memory using the loadCollectionSync method.

// load collection
await client.loadCollectionSync({
  collection_name,
});

vector search

Now you can perform vector search on your collection.

// Generate a random search vector
const searchVector = [...Array(dim)].map(() => Math.random());

// Perform a vector search on the collection
const res = await client.search({
  collection_name, // required, the collection name
  vector: [0.1, 0.2, 0.3, 0.4], // required, vector used to compare other vectors in milvus
  filter: "word_count > 0", // optional, filter
  params: { nprobe: 64 }, // optional, specify the search parameters
  limit: 1, // specify the number of nearest neighbors to return
  metric_type: "L2", // optional, metric to calculate similarity of two vectors
  output_fields: ["book_id", "word_count"], // optional, specify the fields to return in the search results
});

The code snippet performs a vector search on the collection.

First, a random search vector is generated. Then, the client.search() method is called with several parameters: collection_name: Required. Specifies the name of the Milvus collection to search.

vectors: Required. Specifies the vector used to compare other vectors in Milvus. The example code passes an array of floating-point numbers as the vector.

filter or expr: Optional. Specifies a filter to apply to the search query. The example code passes a filter that only includes vectors where the word_count field is greater than 0.

params: Optional. Specifies additional search parameters. The example code sets nprobe to 64, which controls the number of clusters to search during the query. By default, nothing will be set.

limit or topk: Optional. Specifies the number of nearest neighbors to return. The example code sets this to 1. By default, it will be 100

metric_type: Optional. Specifies the metric used to calculate the similarity of two vectors. The example code sets this to "L2". By default, it will be "L2".

output_fields: Optional. Specifies which fields to include in the search results. The example code specifies the book_id and word_count fields, by default the node sdk will output all fields.