TPC Benchmark Status November 2014 |
| Overview |
| Published about 3 times per year, the TPC Benchmark Status Report is a digest of the activities of the TPC and its technical subcommittees.
The TPC held meetings in September and November in Hangzhou, China, and Palo Alto, CA, respectively. A new class of benchmarks was launched with the TPCx-HS "Express" benchmark. The TPC-E and TPC-H specifications were updated following enhancements and minor corrections and clarifications. The TPC’s sixth Technology Conference, TPCTC 2014, will be held in conjunction with the annual conference on Very Large Databases (VLDB), in Hangzhou, China.
|
Current Benchmarks |
| TPC-C |
|
The TPC-C Maintenance Subcommittee had no outstanding issues to discuss. The subcommittee continues to monitor the benchmark, along with any TAB interpretations and rulings, for discussion and possible inclusion in future versions of the TPC-C specification.
|
| TPC-DS |
|
The TPC-DS Maintenance Subcommittee is discussing changes necessary to create a version of TPC-DS that can be run on Hive/Hadoop platforms. That new version will not require the database product to enforce Atomicity, Consistency and Isolation. But it will be required to enforce Durability. The new version will also not require the database product to run the DS workload using the SQL text as provided. The SQL text may be modified as long as the modification is functionally equivalent to the SQL text.
|
| TPC-E |
|
The TPC-E Maintenance Subcommittee released a new revision of the TPC-E benchmark standard (v1.13.0) that adds many clarifications to the Response Time, Test Run, Required Reporting, Durability, Data Accessibility, and Full Disclosure Requirements clauses of the specification, and further extends the unit testing framework and resolves numerous portability issues, minor bugs and code cleanups in EGen.
This revision of the standard was approved and is available immediately, and v1.12.0 becomes obsolete at 11:59 PM PT on June 23, 2014. Prior results and Version 1.13.0 results are comparable. Prior results do not need to be upgraded to Version 1.13.0 results.
|
| TPC-H |
|
The TPC-H Maintenance Subcommittee released Version 2.17.0 of the TPC-H specification, which corrects issues with the answer set and updates correspondingly dbgen and qgen.
|
| TPC-VMS |
|
The TPC-VMS Maintenance Subcommittee had no outstanding issues to discuss. TPC-VMS specifies the rules and methodology for measuring and reporting TPC-C, TPC-E, TPC-H and TPC-DS metrics running in a Virtualized Environment.
|
| TPC-DI |
|
Data Integration (DI), also known as ‘ETL’, is the analysis, combination, and transformation of data from a variety of sources and formats into a unified data model representation. Data Integration is a key element of data warehousing, application integration, and business analytics solutions. The variety and volume of data is always increasing and performance of data integration systems is critical. However, there has been no industry standard for measuring and comparing the performance of DI systems.
The TPC-DI benchmark subcommittee is continuing refinement of the specification.
|
| TPCx-HS |
|
TPCx-HS is the TPC’s first Express Benchmark™ – developed to provide verifiable performance, price / performance, availability, and optional energy consumption metrics of big data systems. Express-class benchmarks are being developed in response to overwhelming demand for a turnkey alternative to enterprise-class benchmarks, which have distinct advantages but are also substantially more time-intensive and costly to run.
TPCx-HS is the industry’s first objective specification enabling measurement of hardware, OS (operating system) and commercial Apache Hadoop File System API compatible software distributions
|
Benchmarks Under Development |
| TPC-V |
|
The TPC-V development subcommittee has been making progress in developing an end-to-end reference benchmarking kit, which can now run transactions on multiple databases in multiple VMs on a virtualized server. A beta version of the end-to-end, multi-VM express benchmarking kit is available to TPC members. The subcommittee is working on completing the specification for an Express version of the benchmark.
|
Other TPC Activities |
| TPC Technology Conference |
|
The TPC remains committed to developing new benchmark standards to keep pace, and one vehicle for achieving this objective is the sponsorship of the Technology Conference on Performance Evaluation and Benchmarking (TPCTC). Over the last five years we have held TPCTC successfully in conjunction with VLDB.
With the TPCTC 2014 conference, in conjuction with with VLDB 2014, the TPC encourages researchers and industry experts to submit novel ideas and methodologies in performance evaluation, measurement, and characterization. The conference will be held on September 5, 2014 in Hangzhou, China.
Topics will include:
- Big Data Appliances
- Cloud Computing
- In-memory databases
- Social media infrastructure
- Business intelligence
- Complex event processing
- Database optimizations
- Green computing
- Disaster tolerance and recovery
- Energy and space efficiency
- Hardware innovations
- Data Integration
- Hybrid workloads
- Virtualization
- Lessons learned in practice using TPC workloads
- Enhancements to TPC workloads
|
| All Benchmark Status Reports |
|
|
