A Configuration-Independent Score-Based Benchmark for Distributed Databases

The business potential of big data is leading to a data-driven economy, where low-cost and low-latency data analysis represents a major competitive advantage. The research community has proposed many technological solutions for big data, such as NoSQL databases, which are difficult to evaluate and compare via standard IT procurement procedures. In addition, lack of competences in big data domains make procurement of big data solutions a tedious and uncertain process, which might impair the success of a business. In this paper, we present a score-based benchmark for distributed databases, which supports adopters in selecting a solution that fits their needs. The proposed benchmark is independent from the configurations of the specific database and deployment environment, requires low effort on the part of end users, is extensible and can be applied to both SQL and NoSQL databases, can be used to evaluate databases according to different properties (e.g., performance, consistency), and can be integrated with existing benchmarks to reduce the burden of their execution. We experimentally evaluate our methodology to validate its effectiveness.