How do benchmarks handle multi-model databases?

Benchmarks for multi-model databases evaluate performance across different data models (like documents, graphs, or key-value stores) within a single database, while also testing how these models interact. Unlike single-model benchmarks that focus on one data type, multi-model benchmarks simulate mixed workloads to reflect real-world scenarios where applications use multiple data models simultaneously. For example, a benchmark might test a database’s ability to handle JSON document writes, graph traversals, and key-value lookups in parallel. Tools like YCSB (Yahoo! Cloud Serving Benchmark) and LDBC (Linked Data Benchmark Council) are often adapted or extended to cover these hybrid use cases, measuring throughput, latency, and consistency under varying loads.

A key focus is testing cross-model operations, such as querying data stored in one model (e.g., a document) and combining it with data from another (e.g., a graph). Benchmarks might simulate an e-commerce application that stores product details as documents, user relationships as graphs, and inventory counts as key-value pairs. A query like “recommend products based on a user’s social network” would require traversing a graph and joining results with document data. Benchmarks assess how efficiently the database handles these operations, including indexing strategies, caching, and transaction management. They also measure resource usage (CPU, memory) to identify bottlenecks when models compete for shared infrastructure.

Challenges include balancing fairness and relevance. For example, a graph-heavy workload might overshadow document operations if the benchmark doesn’t weight scenarios appropriately. Some benchmarks use modular designs, allowing users to customize the mix of workloads (e.g., 70% document reads, 30% graph updates). Real-world benchmarks like ArangoDB’s “Multi-Bench” test hybrid queries across its document, graph, and search engines. These tests help developers understand trade-offs, such as whether native multi-model support outperforms stitching together single-model databases. Ultimately, the goal is to provide actionable insights into how well a database handles diverse, interconnected data at scale.