How do benchmarks handle schema design?

Benchmarks handle schema design by providing standardized database structures that reflect real-world scenarios. These schemas are carefully crafted to represent common use cases, ensuring consistent testing conditions across different systems. For example, the TPC-C benchmark for transactional systems uses a warehouse-centric schema with tables like orders, stock, and customer, mimicking an inventory management system. This predefined structure allows fair comparisons between databases by eliminating variations in table design or indexing that could skew results. The schema’s complexity—such as foreign key relationships, data types, and normalization levels—is intentionally designed to stress-test features like join performance, transaction throughput, and concurrency control.

Benchmarks also define how schemas scale with data volume. TPC-H, which focuses on analytical workloads, uses a star schema with a central lineitem fact table linked to dimension tables like part and supplier. The schema includes scaling factors (e.g., 1GB, 100GB) to simulate different dataset sizes, ensuring the benchmark evaluates how systems handle large-scale data. For instance, a 10TB TPC-H schema tests partitioning strategies, query optimization for star joins, and storage efficiency. These scaling mechanisms ensure the schema remains relevant as hardware and data requirements evolve, while maintaining reproducibility across test runs.

Some benchmarks allow limited schema customization to assess flexibility. For example, YCSB (Yahoo! Cloud Serving Benchmark) for NoSQL databases lets users define their own schemas but provides default structures like key-value pairs with variable columns. This approach tests how different data models (e.g., document vs. wide-column) perform under specific workloads. However, most benchmarks enforce strict schema rules to isolate performance metrics. Deviations, such as adding unnecessary indexes or denormalizing tables, are prohibited to prevent unfair optimizations. By balancing standardization with controlled flexibility, benchmarks ensure results reflect real-world trade-offs in schema design without compromising comparability.