RunAnalyzer()

This operation processes the input data and generates tokenized output.

func (c *Client) RunAnalyzer(ctx context.Context, option RunAnalyzerOption, callOptions ...grpc.CallOption) ([]*entity.AnalyzerResult, error)

Request Parameters

Parameter	Description	Type
ctx	Context for the current call to work.	context.Context
option	Optional parameters of the methods.	RunAnalyzerOption
callOptions	Optional parameters for calling the methods.	grpc.CallOption

RunAnalyzerOption

This is an interface type. The runAnalyzerOption struct type implements this interface. You can use NewRunAnalyzerOption() to get its concrete implementation.

func NewRunAnalyzerOption(text []string) *runAnalyzerOption

Parameter	Description	Type
text	The input text or a list of texts to be analyzed.	[]string

You can chain the following methods to append extra settings to the current RunAnalyzerOption struct.

• WithAnalyzerParamsStr

• WithAnalyzerParams

• WithDetail

• WithHash

• WithField

• WithAnalyzerName

WithAnalyzerParamsStr

This method appends the parameter settings for the current RunAnalyzerOption struct. The signature is as follows:

func (opt *runAnalyzerOption) WithAnalyzerParamsStr(params string) *runAnalyzerOption

Parameter	Description	Type
params	The analyzer parameter settings in plain-text.	string

WithAnalyzerParams

This method appends the parameter settings for the current RunAnalyzerOption struct.

func (opt *runAnalyzerOption) WithAnalyzerParams(params map[string]any) *runAnalyzerOption

Parameter	Description	Type
params	The analyzer parameter settings.	map[string]any

WithDetail

This method appends the parameter setting regarding whether to include hash-based processing.

func (opt *runAnalyzerOption) WithDetail() *runAnalyzerOption

This method has no parameters. Using this parameter indicates that hash-based processing will be included.

WithHash

This method appends the parameter setting regarding whether to return detailed analysis output.

func (opt *runAnalyzerOption) WithHash() *runAnalyzerOption

This method has no parameters. Using this parameter indicates that detailed analysis output will be returned.

WithField

This method specifies a scalar field for the current RunAnalyzerOption struct.

func (opt *runAnalyzerOption) WithField(collectionName, fieldName string) *runAnalyzerOption

Parameter	Description	Type
collectionName	The name of the target collection.	string
fieldName	The name of the target field in the above-specified collection.	string

WithAnalyzerName

This method appends the parameter settings regarding the analyzer names.

func (opt *runAnalyzerOption) WithAnalyzerName(names ...string) *runAnalyzerOption

Parameter	Description	Type
names	The analyzer names to append.	...string

entity.AnalyzerResult

The entity.AnalzyerResult struct type is as follows:

type AnalyzerResult struct {
    Tokens []*Token
}

The struct type that appears above is as follows:

• entity.Token

entity.Token

The entity.Token struct type is as follows:

type Token struct {
    Text           string
    StartOffset    int64
    EndOffset      int64
    Position       int64
    PositionLength int64
    Hash           uint32
}

Return

*entity.AnalyzerResult

Example

package main

import (
    "context"
    "log"

    "github.com/milvus-io/milvus/client/v2/entity"
    "github.com/milvus-io/milvus/client/v2/milvusclient"
)

func main() {
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    
    milvusAddr := "127.0.0.1:19530"
    collectionName := "test_run_analyzer"
    
    cli, err := milvusclient.New(ctx, &milvusclient.ClientConfig{
        Address: milvusAddr,
    })
    if err != nil {
        log.Fatal("failed to connect to milvus server: ", err.Error())
    }
    defer cli.Close(ctx)
    
    schema := entity.NewSchema().
        WithField(entity.NewField().WithName("pk").WithIsPrimaryKey(true).WithIsAutoID(true).WithDataType(entity.FieldTypeInt64)).
        WithField(entity.NewField().WithName("text").WithDataType(entity.FieldTypeVarChar).WithMaxLength(255).WithEnableAnalyzer(true).WithAnalyzerParams(map[string]any{"tokenizer": "standard"})).
        WithField(entity.NewField().WithName("sparse").WithDataType(entity.FieldTypeSparseVector)).
        WithFunction(entity.NewFunction().WithInputFields("text").WithOutputFields("sparse").WithType(entity.FunctionTypeBM25).WithName("bm25")).
        WithAutoID(true)
    
    err = cli.CreateCollection(ctx, milvusclient.NewCreateCollectionOption(collectionName, schema))
    if err != nil {
        log.Fatal("failed to connect to create test collection: ", err.Error())
    }
    
    cli.CreateIndex(ctx, milvusclient.NewCreateIndexOption(collectionName, "sparse", index.NewAutoIndex(entity.BM25)).WithIndexName("bm25"))
    cli.LoadCollection(ctx, milvusclient.NewLoadCollectionOption(collectionName))
    
    // Run analyzer with loaded collection field (Must be bm25 function input)
    result, err := cli.RunAnalyzer(ctx, milvusclient.NewRunAnalyzerOption("test milvus").WithField(collectionName, "text"))
    if err != nil {
        log.Fatal("failed to run analyzer with loaded collection field: ", err)
    }
    
    println("Run analyzer result with loaded collection field")
    for _, token := range result[0].Tokens {
        println(token.Text)
    }
    
    params := map[string]any{
        "tokenizer": "standard",
        "filter": []any{map[string]any{
            "type":       "stop",
            "stop_words": []string{"test"}, // remove word "test"
        }},
    }
    // Run analyzer with new analyzer params
    result, err = cli.RunAnalyzer(ctx, milvusclient.NewRunAnalyzerOption("test milvus").WithAnalyzerParams(params))
    if err != nil {
        log.Fatal("failed to run analyzer with new analyzer params: ", err)
    }
    
    println("Run analyzer with new analyzer params")
    for _, token := range result[0].Tokens {
        println(token.Text)
    }
}