Realistic benchmark for semi-sorted workload: case for RocksDB in Ceph Bluestore

Abstract

In this study, we propose a new benchmark of a key-value store with a semi-sorted workload to address the limitations of the existing benchmarks. We observe the write amplification and latency spike issues, which are mainly from background running compaction operations. We analyze the real workload of RocksDB in Ceph Bluestore by collecting a query trace of up-and-running geo-scale commercial service platform. Based upon the observed workload characteristics, we develop a semi-sorted key generator and embed it with db_bench. This benchmark is tailored towards the workload characteristics of the metadata management of the Ceph Bluestore. The proposed benchmark can effectively reproduce the write stall phenomenon that observed in the commercial settings. The proposed benchmark are designed to efficiently adjust the spatial locality of the semi-sorted workload. It can greatly simplify the process of evaluating a given column family separation strategy and determining the optimal write buffer size.