Benchmarks for hybrid transactional/analytical processing (HTAP) focus on evaluating systems that handle both real-time transactional workloads and complex analytical queries simultaneously. These benchmarks simulate mixed workloads to measure how well a database or platform maintains performance for transactions (like updates or row-level reads) while also executing analytical scans or aggregations. Key metrics include transaction throughput, query latency, data freshness (how quickly analytical queries reflect recent transactions), and resource utilization. Unlike traditional benchmarks that test OLTP or OLAP in isolation, HTAP benchmarks stress concurrent access and ensure minimal interference between the two workload types.
A common approach involves using a shared dataset for both transactional and analytical operations. For example, the CH-benCHmark extends the TPC-C (transactional) and TPC-H (analytical) benchmarks by integrating their schemas and workloads. It simulates an environment where orders are placed and modified (transactions) while analytics like sales trends are queried in real time. Another example is the HTAP Benchmark by the Transaction Processing Performance Council (TPC), which defines strict rules for concurrent workload execution. These benchmarks often include mechanisms to ensure analytical queries access the most recent data—such as timestamp checks—to validate consistency. Tools like HammerDB or custom scripts may inject both workload types simultaneously, scaling complexity to test system limits.
Developers should note that HTAP benchmarks prioritize balance. For instance, a system might handle 10,000 transactions per second (TPS) while maintaining sub-second analytical query response times. The benchmarks also measure how systems isolate resources (e.g., using in-memory processing for transactions and columnar storage for analytics) or optimize data replication between transactional and analytical engines. Some systems, like Apache IoTDB or ClickHouse, are tested for hybrid capabilities by running time-series insertions alongside aggregate queries. Results often highlight trade-offs: aggressive indexing for analytics might slow transactions, while prioritizing transaction speed could stale analytical data. Effective HTAP benchmarks reveal whether a system’s architecture—such as row-column storage hybrids or real-time replication—achieves the required balance without compromising either workload.