Home
Blog
Spatial Meets Semantic: Unlock Geo-Vector Search with Geometry Fields and RTREE Index in Milvus

Spatial Meets Semantic: Unlock Geo-Vector Search with Geometry Fields and RTREE Index in Milvus

Engineering

December 08, 2025

Cai Zhang

As modern systems grow more intelligent, geolocation data has become essential to applications such as AI-driven recommendations, smart dispatching, and autonomous driving.

For example, when you order food on platforms like DoorDash or Uber Eats, the system considers much more than the distance between you and the restaurant. It also weighs restaurant ratings, courier locations, traffic conditions, and even your personal preference embeddings. In autonomous driving, vehicles must perform path planning, obstacle detection, and scene-level semantic understanding, often within just a few milliseconds.

All of this depends on the ability to efficiently index and retrieve geospatial data.

Traditionally, geographic data and vector data lived in two separate systems:

Geospatial systems store coordinates and spatial relationships (latitude, longitude, polygon regions, etc.).
Vector databases handle semantic embeddings and similarity search generated by AI models.

This separation complicates architecture, slows queries, and makes it difficult for applications to perform spatial and semantic reasoning at the same time.

Milvus 2.6 addresses this problem by introducing the Geometry Field, which allows vector similarity search to be combined directly with spatial constraints. This enables use cases such as:

Location-Base Service (LBS): “find similar POIs within this city block”
Multi‑modal search: “retrieve similar photos within 1km of this point”
Maps & logistics: “assets inside a region” or “routes intersecting a path”

Paired with the new RTREE index—a tree-based structure optimized for spatial filtering—Milvus now supports efficient geospatial operators like st_contains, st_within, and st_dwithin alongside high-dimensional vector search. Together, they make spatially aware intelligent retrieval not just possible, but practical.

In this post, we’ll walk through how the Geometry Field and RTREE index work, and how they combine with vector similarity search to enable real-world, spatial-semantic applications.

What Is a Geometry Field?

A Geometry field is a schema-defined data type (DataType.GEOMETRY) in Milvus used to store geometric data. Unlike systems that handle only raw coordinates, Milvus supports a range of spatial structures—including Point, LineString, and Polygon.

This makes it possible to represent real-world concepts such as restaurant locations (Point), delivery zones (Polygon), or autonomous-vehicle trajectories (LineString), all within the same database that stores semantic vectors. In other words, Milvus becomes a unified system for both where something is and what it means.

Geometry values are stored using the Well-Known Text (WKT) format, a human-readable standard for inserting and querying geometric data. This simplifies data ingestion and querying because WKT strings can be inserted directly into a Milvus record. For example:

data = [
    { 
        "id": 1,
        "geo": "POINT(116.4074 39.9042)",
        "vector": vector,
    }
]

What Is the RTREE Index and How Does It Work?

Once Milvus introduces the Geometry data type, it also needs an efficient way to filter spatial objects. Milvus handles this using a two-stage spatial filtering pipeline:

Coarse filtering: Quickly narrows down candidates using spatial indexes such as RTREE.
Fine filtering: Applies exact geometry checks on the candidates that remain, ensuring correctness at boundaries.

At the core of this process is RTREE (Rectangle Tree), a spatial indexing structure designed for multidimensional geometric data. RTREE accelerates spatial queries by organizing geometric objects hierarchically.

Phase 1: Build the index

1. Create leaf nodes: For each geometry object, calculate its Minimum Bounding Rectangle (MBR)—the smallest rectangle that fully contains the object—and store it as a leaf node.

2. Group into larger boxs: Cluster nearby leaf nodes and wrap each group inside a new MBR, producing internal nodes.

3. Add the root node: Create a root node whose MBR covers all internal groups, forming a height-balanced tree structure.

Phase 2: Accelerate queries

1. Form the query MBR: Calculate the MBR for the geometry used in your query.

2. Prune branches: Starting from the root, compare the query MBR with each internal node. Skip any branch whose MBR does not intersect with the query MBR.

3. Collect candidates: Descend into intersecting branches and gather the candidate leaf nodes.

4. Perform exact matching: For each candidate, run the spatial predicate to get precise results.

Why RTREE Is Fast

RTREE delivers strong performance in spatial filtering because of several key design features:

Every node stores an MBR: Each node approximates the area of all geometries in its subtree. This makes it easy to decide whether a branch should be explored during a query.
Fast pruning: Only subtrees whose MBR intersects the query region are explored. Irrelevant areas are ignored entirely.
Scales with data size: RTREE supports spatial searches in O(log N) time, enabling fast queries even as the dataset expands.
Boost.Geometry implementation: Milvus builds its RTREE index using Boost.Geometry, a widely used C++ library that provides optimized geometry algorithms and a thread-safe RTREE implementation suitable for concurrent workloads.

Supported geometry operators

Milvus provides a set of spatial operators that allow you to filter and retrieve entities based on geometric relationships. These operators are essential for workloads that need to understand how objects relate to one another in space.

The following table lists the geometry operators currently available in Milvus.

Operator	Description
st_intersects(A, B)	Returns TRUE if geometries A and B share at least one common point.
st_contains(A, B)	Returns TRUE if geometry A completely contains geometry B (excluding the boundary).
st_within(A, B)	Returns TRUE if geometry A is completely contained within geometry B. This is the inverse of st_contains(A, B).
st_covers(A, B)	Returns TRUE if geometry A covers geometry B (including the boundary).
st_touches(A, B)	Returns TRUE if geometries A and B touch at their boundaries but do not intersect internally.
st_equals(A, B)	Returns TRUE if geometries A and B are spatially identical.
st_overlaps(A, B)	Returns TRUE if geometries A and B partially overlap and neither fully contains the other.
st_dwithin(A, B, d)	Returns TRUE if the distance between A and B is less than d.

How to Combine Geolocation Index and Vector Index

With Geometry support and the RTREE index, Milvus can combine geospatial filtering with vector similarity search in a single workflow. The process works in two steps:

1. Filter by location using RTREE: Milvus first uses the RTREE index to narrow the search to entities within the specified geographic range (e.g., “within 2 km”).

2. Rank by semantics using vector search: From the remaining candidates, the vector index selects the Top-N most similar results based on embedding similarity.

Real-World Applications of Geo-Vector Retrieval

1. Delivery Services: Smarter, Location-Aware Recommendations

Platforms such as DoorDash or Uber Eats handle hundreds of millions of requests each day. The moment a user opens the app, the system must determine—based on the user’s location, time of day, taste preferences, estimated delivery times, real-time traffic, and courier availability—which restaurants or couriers are the best match right now.

Traditionally, this requires querying a geospatial database and a separate recommendation engine, followed by multiple rounds of filtering and re-ranking. With the Geolocation Index, Milvus greatly simplifies this workflow:

Unified storage — Restaurant coordinates, courier locations, and user preference embeddings all live in one system.
Joint retrieval — First apply a spatial filter (e.g., restaurants within 3 km), then use vector search to rank by similarity, taste preference, or quality.
Dynamic decision-making — Combine real-time courier distribution and traffic signals to quickly assign the nearest, most suitable courier.

This unified approach allows the platform to perform spatial and semantic reasoning in a single query. For example, when a user searches “curry rice,” Milvus retrieves restaurants that are semantically relevant and prioritizes those that are nearby, deliver quickly, and match the user’s historical taste profile.

2. Autonomous Driving: More Intelligent Decisions

In autonomous driving, geospatial indexing is fundamental to perception, localization, and decision-making. Vehicles must continuously align themselves to high-definition maps, detect obstacles, and plan safe trajectories—all within just a few milliseconds.

With Milvus, the Geometry type and RTREE index can store and query rich spatial structures such as:

Road boundaries (LineString)
Traffic regulation zones (Polygon)
Detected obstacles (Point)

These structures can be indexed efficiently, allowing geospatial data to take part directly in the AI decision loop. For example, an autonomous vehicle can quickly determine whether its current coordinates fall within a specific lane or intersect with a restricted area, simply through an RTREE spatial predicate.

When combined with vector embeddings generated by the perception system—such as scene embeddings that capture the current driving environment—Milvus can support more advanced queries, like retrieving historical driving scenarios similar to the current one within a 50-meter radius. This helps models interpret the environment faster and make better decisions.

Conclusion

Geolocation is more than latitude and longitude—it is a valuable source of semantic information that tells us where things happen, how they relate to their surroundings, and what context they belong to.

In Zilliz’s next-generation database, vector data and geospatial information are gradually coming together as a unified foundation. This enables:

Joint retrieval across vectors, geospatial data, and time
Spatially aware recommendation systems
Multimodal, location-based search (LBS)

In the future, AI will not only understand what content means, but also where it applies and when it matters most.

For more information about the Geometry Field and the RTREE index, check the documentation below:

Have questions or want a deep dive on any feature of the latest Milvus? Join our Discord channel or file issues on GitHub. You can also book a 20-minute one-on-one session to get insights, guidance, and answers to your questions through Milvus Office Hours.