Release notes

v1.1.0

Release date：2021-05-07

Compatibility

New Features

• #4564 Supports specifying partition in a get_entity_by_id() method call.
• #4806 Supports specifying partition in a delete_entity_by_id() method call.
• #4905 Adds the release_collection() method, which unloads a specific collection from cache.

Improvements

• #4756 Improves the performance of the get_entity_by_id() method call.
• #4856 Upgrades hnswlib to v0.5.0.
• #4958 Improves the performance of IVF index training.

Fixed issues

• #4778 Fails to access vector index in Mishards.
• #4797 The system returns false results after merging search requests with different topK parameters.
• #4838 The server does not respond immediately to an index building request on an empty collection.
• #4858 For GPU-enabled Milvus, the system crashes on a search request with a large topK (> 2048).
• #4862 A read-only node merges segments during startup.
• #4894 The capacity of a Bloom filter does not equal to the row count of the segment it belongs to.
• #4908 The GPU cache is not cleaned up after a collection is dropped.
• #4933 It takes a long while for the system to build index for a small segment.
• #4952 Fails to set timezone as "UTC + 5:30".
• #5008 The system crashes randomly during continuous, concurrent delete, insert, and search operations.
• #5010 For GPU-enabled Milvus, query fails on IVF_PQ if nbits ≠ 8.
• #5050 get_collection_stats() returns false index type for segments still in the process of index building.
• #5063 The system crashes when an empty segment is flushed.
• #5078 For GPU-enabled Milvus, the system crashes when creating an IVF index on vectors of 2048, 4096, or 8192 dimensions.

v1.0.0

Release date：2021-03-09

Compatibility

New Features

• Supports writing log to stdout. #3977

Improvements

• Reduces the package size of grpc-milvus for the C++ SDK. #4754

Fixed issues

• Memory leaks during indexing or querying operations. #4749, #4757, #4765, #4766

• For more information about the features of Milvus 1.0, go to What's Inside Milvus 1.0?.
• For more information about its roadmap, see Milvus 1.0: The World's Most Popular Open-Source Vector Database Just Got Better.

v0.10.6

Release date：2021-02-23

Compatibility

Compatibility changes

• Adds an optional argument nbits to the create_index() method for the IVF_PQ index. #3920

Improvements

• Improves the FLAT search performance on binary vectors using the AVX2 instruction set. #1970
• Adds an optional parameter nbits to the create_index() method for the IVF_PQ index. #3920
• Supports configuring Prometheus labels cluster_label and instance_label under metric. #4614

Fixed issues

• The system returns a -0 distance, if metric type is tanimoto. #4683
• A FLAT search on binary vectors causes the server to crash, if the dimension of the vectors is not multiple of 2. #4678
• The GPU cache holds more data than specified. #4719

See CHANGELOG for more information.

v0.10.5

Release date：2021-01-07

Compatibility

Compatibility changes

• The load_collection() method supports preloading specified partitions. #4307

Improvements

• Optimizes the process of indexing and querying. #4454

Fixed issues

• Calling load_collection() and search() in multiple threads causes Milvus to hang. #4378
• Milvus only searches the default partition if the search parameter partition_tags contains _default. #4484

See CHANGELOG for more information.

v0.10.4

Release date：2020-12-03

Compatibility

Improvements

• Improves the query performance in situations where no entities are deleted. #3775
• Reduces the time to create a large number of partitions. #4318

Fixed issues

• The server crashes when searching with IVF_PQ on GPU. #3626
• The performance of query on IVF_SQ8H worsens. #3903
• The server crashes when deleting entities. #3906
• The server occasionally hangs when creating and dropping partitions continuously. #4012
• An out-of-memory error occurs when searching on GPU. #4174
• Memory leaks when building an IVF index. #4318

See CHANGELOG for more information.

v0.10.3

Release date：2020-9-21

Compatibility

Improvements

• Allows the user to specify metric_type when searching in a collection with FLAT index. #3213
• Adds more valid values to the indexing parameter m of IVF_PQ: An m is valid so long as it can be divided by dim, which represents the number of vector dimensions. #3254
• Expands the range of nprobe to [1, 16384] for all IVF indexes. #3606
• Expands the range of top_k in an embeddings search to [1, 16384]. #3639
• Optimizes the memory release strategy to avoid memory overuse. #3536

Fixed issues

• Milvus running on multiple GPU devices and using IVF_SQ8H crashes if GPU cache is set too low. #3742
• Milvus reports a CUDA error when quitting, if the index is IVF_SQ8H. #3760

See CHANGELOG for more information.

v0.10.2

Release date：2020-8-15

Compatibility

Improvements

• Improved IVF search performance when both nq and nprobe are large. #2653

Fixed issues

• Inaccurate calculation of the memory allocated for index files. #2890
• Search result sets based on IVF_PQ index with IP metric are incorrectly merged in situations where multiple segments are produced for the inserted data. #2952
• The server crashes if you set cache.cache_size smaller than the size of an index file in a multi-GPU scenario. #3012
• Search result sets based on IP metric are incorrectly merged, if you insert vectors to multiple partitions in Mishards. #3133

See CHANGELOG for more information.

v0.10.1

Release date：2020-7-20

Compatibility

Fixed issues

• Occasionally, the result count does not match the query count. #2578
• IVF_PQ does not support IP on GPU-enabled Milvus. #2585
• Milvus reports "illegal instruction" when running on some legacy CPUs. #2598
• Updated the range of HNSW settings. #2637
• Milvus processes crash when building an index. #2642
• The default Annoy settings do not match the settings specified in the document. #2649
• Milvus does not respond under a stress test. #2692
• The precision of the returned vectors is incorrect in an HTTP interface. #2752
• The upper limit of nprobe is incorrect on the GPU-enabled Milvus. #2767
• The number of the vectors changes occasionally after building an index. #2768

See CHANGELOG for more information.

v0.10.0

Release date：2020-6-15

Compatibility

Compatibility changes

• Updates the Milvus configuration file. #2510

Improvements

• Optimizes the index building performance when many small segments exist. #2373
• FAISS is upgraded to 1.6.3. #2381
• Optimizes dropping collection performance when too many partitions exist. #2394
• Optimizes the implementation of k-selection algorithm for GPU-enabled Milvus. #2466

Fixed issues

• The search performance degrades on Milvus 0.9.1. #2429

See CHANGELOG for more information.

v0.9.1

Release date：2020-5-29

Compatibility

Fixed issues

• In a multi-partition situation, data is inserted twice after the server reboots. #2378
• A cudaMalloc failure occurs with GPU IVF index when nq (number of queries) is high. #2395.
• Deleted vectors are still found in GPU-enabled Milvus. #2450

See CHANGELOG for more information.

v0.9.0

Release date：2020-5-15

Compatibility

New features

• Checks the CPU instruction set, GPU driver version, and CUDA version, when Milvus starts up. #2054 #2111
• Prevents multiple Milvus instances from accessing the same Milvus database at the same time. #2059
• Supports log file rotating. #2206
• Suspends index building when a search request comes in. #2283

Improvements

• Refactors log output. #221
• Upgrades OpenBLAS to improve Milvus' performance. #1796
• Unifies the vector distance calculation algorithms among FAISS, NSG, HNSW, and Annoy. #1965
• Supports SSE4.2 instruction set. #2039
• Refactors the configuration files. #2149 #2167
• Uses Elkan K-means algorithm to improve the IVF index performance. #2178

Fixed issues

See CHANGELOG for more information.

API changes

Added methods

Changed methods

v0.8.0

Release date：2020-4-15

Compatibility

New features

• Annoy index support

  Added support for Annoy index type. See our documentation for more information. #261

• Vector deletion

  Added support to delete one or multiple vectors for more index types. #1655 #1660 #1661 #1849

  Including: Flat/IVFlat/IVFPQ/IVFSQ8/IVFSQ8H/NSG/HNSW/Annoy

Improvements

• Added new metric SuperStructure and SubStructure in HTTP module. #1784

Fixed issues

• Limited the maximum number of partitions to 4096. #1276
• Forbidden to create partition with name _default. #1762
• Resolved the issue that concurrent operations from multiple clients cause system crash. #1789
• Resolved the issue that some raw vectors are missed when the raw data file size is larger than 2GB. #1883

v0.7.1

Release date：2020-3-30

Compatibility

New features

• Added new distance metrics, including substructure and superstructure, for the FLAT index type. These metrics are used for substructure and superstructure search of chemical structures.#1603.

Improvements

• Improved the performance of the compact operation. #1619
• Improved search performance using CPU, especially for scenarios with multiple, concurrent connections. #267
• Improved the search performance when nq is less than the number of threads in the CPU. #1690
• Milvus performs a combined search for the same search requests from multiple clients, thus significantly improving search speed. #1728
• Upgraded Mishards to 0.7.1. #1698

Fixed issues

Refer to CHANGELOG for details.

v0.7.0

Release date：2020-3-10

Compatibility

New features

• Vector deletion

  Added support to delete one or multiple vectors. If you performed vector deletion on a collection, later search operations for this collection are limited to part of the index types, including FLAT, IVFLAT, IVFSQ8, etc. Milvus is planned to support other index types in the upcoming versions.#861

• Get vector by ID

  Added support to get vector data by ID. #861

• Flush and compact

  Added support to flushing and compaction. You can configure flushing at an interval or manual flushing to avoid data loss. If some vectors are deleted from a segment, the space taken by the deleted vectors cannot be released automatically. You can compact segments in a collection to release space. #861 #1426

• Change Milvus server configurations during runtime

  Added support to update Milvus server configurations during runtime. You can use Milvus clients to update the parameters. Changes to some parameters take effect immediately without restarting Milvus. #665

• Write-Ahead logging (WAL)

  Added support for WAL, which significantly improves the reliability of data operations. You can configure WAL settings in the Milvus server configuration file (server_config.yaml). #830

• RESTful API

  Added RESTful API. Refer to RESTful API Readme for more information.

• Go SDK

  Added Go SDK. Refer to https://github.com/milvus-io/milvus-sdk-go for more information.

• HNSW index support

  Added support for HNSW index type. Refer to Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs for more information about HNSW. #853

• Jaccard/Hamming/Tanimoto distance support

  Added support for Jaccard distance, Hamming distance, and Tanimoto distance. #823

• Pushgateway support in Prometheus

  Added support for Pushgateway in Prometheus. Pushgateway makes it possible for short-lived, batch metrics to be acquired by Prometheus. #813

• AVX 512

  Added support for AVX 512 instruction set. Milvus theoretically supports all CPUs with AVX 512. #1122

Changes

• Interface updates for index creation and vector search

  Starting from Milvus 0.7.0, part of the Milvus client parameters for index creation and vector search use JSON strings as values.

• Milvus server configuration file updates

  Starting from Milvus 0.7.0, the Milvus server configuration file (server_config.yaml) is updated to 0.2 with parameter changes.

• Term updates

  As of Milvus 0.7.0, Table is named as Collection in Milvus.

Fixed issues

• Resolved the issue that duplicate IDs may be generated when inserting vector data using auto-generated IDs. #1508

v0.6.0

Release date: 2019-12-07

Version Compatibility

New features

• CPU-only Milvus

  Milvus v0.6.0 provides Docker images for both CPU-only and GPU support Milvus. Milvus compilation on Docker is also supported on machines with or without GPU. #192

• Table partitioning

  Add table partitioning funtion to secure fast query performance for incremental data. Partitioning APIs are added to Python, Java and C++ SDK to support partition creation, vector insertion into a specified partition and query against a specified partition, etc. #245

• Experimental features

  The experimental features in Milvus are still under development and subject to change. They may contain unknown errors, and are intended for testing and user feedback gathering.

  • Mishards

    Propose Mishards, a Milvus sharding middleware, as the distributed deployment solution. Mishards provides unlimited extension of memory and computation capacity through request forwarding, read/write splitting, horizontal scalability and dynamic extension. #232

  • New index types

    Start supporting new experimental index types such as SPTAG-KDT, SPTAG-BKT, RNSG and IVFPQ. SPTAG#438 RNSG#554 IVFPQ#324

• Index test reports

  Provide performance test reports for IVFFLAT, IVFSQ8 and IVFSQ8H indexes.

Improvements

• Milvus internal FAISS

  In addition to original FAISS, Milvus has made deep optimizations to increase query performance and support more index types such as IVFSQ8H. Now this part of internal FAISS is open sourced. #585

• Multiple GPUs for index building

  Support index building by multiple GPUs to reduce index building and overall query time. You can specify multiple GPUs for index building process through Milvus configuration parameter build_index_resources. #414

Fixed issues

• Solved the issue of increased memory usage during vector queries. #335

v0.5.3

Release date: 2019-11-14

Version Compatibility

Improvements

• Double the transmission speed of search results to the client application through the following updates to gRPC:

  • Optimize messages.
  • Change the API of generated code.
  • Remove compression.

• Python SDK

  • Divide the storage of search result ids and distances into separate arrays, which reduces the API response time.

  • Add a new option to retrieve a specific target vector in search results: id = results.id_array[i][j], distance = results.distance_array[i][j].

  • Add a new option for looping over arrays, which takes much less time if nq and top_k is large.

    >> for id_list, dis_list in zip(results.id_array, results.distance_array):
    >>     for id, dis in zip(id_list, dis_list):
    >>        print("id={}, distance={}".format(id, dis))
    

• Java SDK

  • Add keepalive and idleTimeout settings when connecting to Milvus server.
  • Now users can retrieve search result ids and distances separately through getResultIdsList and getResultDistancesList with better performance, or they can retrieve them together as a list of QueryResult objects through getQueryResultsList.

• C++ SDK

  • Now C++ SDK uses shared library.
  • Add README file.

• Enhance the search performance of IVF_SQ8H.

v0.5.2

Release date: 2019-11-07

Fixed issues

Add a system lock to avoid the generation of files with duplicated data file names, which fixes the bug of search failure due to false deletion of files that have duplicated file names.

Improvements

Add a Japanese version of README file. (from an external contributor)

v0.5.1

Release date: 2019-11-04

Features

• Start supporting GPU-only mode for IVF_SQ8 and IVFFLAT index types.
• Add configuration parameter gpu_search_threshold to control GPU-only execution trigger point.

Improvements

• Reduce memory footprint of queries.
• Optimize query performance to achieve unfluctuating search speed.

v0.5.0

Release date: 2019-10-15

Features

• Start supporting a new index type IVF_SQ8H.

• Add Java SDK.

• Add preload table into memory at Milvus startup.

v0.4.0

Release date: 2019-09-11

Features

• Milvus now supports adding multiple GPU scheduler for resource management.

• Start supporting a new index type IVF_SQ8.

• Add new API about index creation, user-defined vector ids, and vector deletion by date range, etc.

Improvements

• Use gRPC as the communication facility.

v0.3.1

Release date: 2019-08-08

Features

• Added a new type of index IVFSQ which could significantly improve the overall throughput of vector processing.
• Added a new metric of vector distance calculation IP (Inner Product), in addition to L2 (Euclidean Distance).
• Added multiple parameters which optimizes index building, search precision and search speed.

Improvements

• When the data size is huge and cannot fit in the data file on one disk, you can add multiple secondary data storage directories on other disks.
• You can choose if to enable parallel computing of vectors by multiple threads, by configuring parameter parallel_reduce.
• You can designate a portion of the memory for buffer usage of data insertion, by configuring parameter insert_buffer_size.
• In regard to cache management, by configuring cache_free_percent, you can now decide, when the cache reaches its capacity, how much data should be kept instead of being erased.
• You can enable simultaneous inserting and searching of vectors by setting insert_cache_immediately to True.
• Search results are evaluated based on the distances between search results and the target vectors, rather than the score.

v0.3.0

Release date: 2019-06-30

Features

• Distributed architecture based on Celery
• MinIO based storage separation solution
• You can now delete a table
• ARM64 architecture is now supported

Improvements

• File life cycle management
• More interface on C++/Python SDK
• Lots of update on Milvus configure
• Mem table serialization and SSTable consolidation strategy improved
• Improved the Meta management implementation
• 90%+ unit test code coverage
• CMake makefile refactoring
• Improved the time range query

v0.2.1

Release date: 2019-06-14

Features

Added data loading and computation pipeline.

Improvements

You can now search data within a specific date range.

v0.2.0

Release date: 2019-05-31

Features

• Added C++/Python SDK.
• Added monitoring items on Prometheus-based monitoring dashboard.
• Added vector indexing built on Inverted File.
• Single node Milvus realized.