TPC Benchmark Status May 2015 |
| Overview |
| Published about 3 times per year, the TPC Benchmark Status Report is a digest of the activities of the TPC and its technical subcommittees.
The TPC held a meetings in February in Houston, TX, and in April in Raleigh, NC, and teleconferences in January and March. The TPC-DS , TPC-DI , TPC-E, and TPCx-HS specifications were updated following enhancements and minor corrections and clarifications. The TPC’s seventh Technology Conference, TPCTC 2015, will be held in conjunction with the annual conference on Very Large Databases (VLDB), in Waikoloa, Hawaii.
|
Current Benchmarks |
| TPC-C |
|
The TPC-C Maintenance Subcommittee had no outstanding issues to discuss. The subcommittee continues to monitor the benchmark, along with any TAB interpretations and rulings, for discussion and possible inclusion in future versions of the TPC-C specification.
|
| TPC-DS |
|
The maintenance subcommittee released Version 1.3.1 of TPC-DS which includes wording clarifications and editorial changes.
The maintenance subcommittee also continued discussing changes necessary to create a version of TPC-DS that can be run on Hive/Hadoop platforms. That new version will not require the database product to enforce Atomicity, Consistency, Isolation and Durability. But it will be required to enforce Data Accessibility. All 99 queries will be part of this benchmark. There will be minimal changes to the data maintenance portion of the workload
|
| TPC-E |
|
The TPC-E Subcommittee has released version 1.14.0 of the TPC-E benchmark standard that updates the EULA and resolves an issue that allows for greater flexibility in driver implementations.
This revision of the standard was approved and is available immediately, and v1.13.0 becomes obsolete at 11:59 PM PT on June 22, 2015. Prior results and Version 1.14.0 results are comparable. Prior results do not need to be upgraded to Version 1.14.0 results.
|
| TPC-H |
|
The TPC-H Subcommittee continues to monitor the benchmark, along with any TAB interpretations and rulings, for discussion and possible inclusion in future versions of the TPC-H specification.
|
| TPC-VMS |
|
The TPC-VMS Maintenance Subcommittee had no outstanding issues to discuss. TPC-VMS specifies the rules and methodology for measuring and reporting TPC-C, TPC-E, TPC-H and TPC-DS metrics running in a Virtualized Environment.
|
| TPC-DI |
|
Data Integration (DI), also known as ‘ETL’, is the analysis, combination, and transformation of data from a variety of sources and formats into a unified data model representation. Data Integration is a key element of data warehousing, application integration, and business analytics solutions. The variety and volume of data is always increasing and performance of data integration systems is critical. However, there has been no industry standard for measuring and comparing the performance of DI systems.
The TPC-DI benchmark subcommittee released version 1.1.0 of the specification which includes changes in the visibility query and adjustments to PDGF and DIgen.
|
| TPCx-HS |
|
TPCx-HS is the TPC’s first Express Benchmark™ – developed to provide verifiable performance, price / performance, availability, and optional energy consumption metrics of big data systems. Express-class benchmarks are being developed in response to overwhelming demand for a turnkey alternative to enterprise-class benchmarks, which have distinct advantages but are also substantially more time-intensive and costly to run.
The TPCx-HS Subcommittee released version 1.3.0 of the TPCx-HS specification which clarifies some of the
|
| Pricing |
|
The TPC-Pricing Maintenance Subcommittee has continued discussing and developing wording to address the pricing of TPC workloads on cloud environments. The subcommittee is targeting August 2015 for the release of the revised specification.
|
Benchmarks Under Development |
| TPC-V |
|
The TPC-V development subcommittee has developed a beta version of the end-to-end, multi-VM express benchmark kit which is now available to members, and members are prtotyping with the beta kit. The subcommittee is working on completing the specification for an Express version of the benchmark.
|
| BigBench |
|
The TPC-BigBench subcommittee was created in November 2014 to investigate the development of a benchmark based on the Intel BigBench proposal. The Working Group is expected to report back to the TPC at the June 2015 meeting.
|
Other TPC Activities |
| TPC Technology Conference |
|
The TPC remains committed to developing new benchmark standards to keep pace, and one vehicle for achieving this objective is the sponsorship of the Technology Conference on Performance Evaluation and Benchmarking (TPCTC). Over the last five years we have held TPCTC successfully in conjunction with VLDB.
With the TPCTC 2015 conference, in conjuction with with VLDB 2015, the TPC encourages researchers and industry experts to submit novel ideas and methodologies in performance evaluation, measurement, and characterization. The conference will be held on August 31, 2015 in Waikoloa, Hawaii.
Topics will include:
- Big Data
- Data Analytics
- Cloud Computing
- In-memory databases
- Social media infrastructure
- Security
|
- Complex event processing
- Internet of Things
- Database Optimizations
- Disaster tolerance and recovery
- Energy and space efficiency
- Hardware innovations
|
- Hybrid workloads
- Virtualization
- Lessons learned in practice using TPC workloads
- Enhancements to TPC workloads
- Data Integration
|
Authors are invited to submit original, unpublished papers that are not currently under review for any other conference or journal. We also encourage the submission of extended abstracts, position statement papers and lessons learned in practice. The length of a paper should not exceed 16 pages. Papers should follow the LNCS format. The title page must contain a short abstract. All papers should be submitted electronically in PDF format to the review web site at: https://www.easychair.org/conferences/?conf=tpctc2015
Important Dates
| Abstracts due |
June 8, 2015 |
| Papers due |
June 15, 2015 |
| Notification of acceptance |
July 1, 2015 |
| Camera-ready copies |
July 15, 2015 |
| Conference Day |
August 31, 2015 |
|
| Website Changes |
|
The TPC has updated its web site to accommodate mobile platforms, thus obviating the need for a dedicated mobile site. As a result, the mobile web site has been retired.
Also, a revised member application has been implemented to make it easier to fill out the required information.
The TPC has released several new benchmarks. To receive notifications when results are published, please add yourself to the mailing list at http://www.tpc.org/information/about/mailinglist_signup5.asp. There you will be able to specify which benchmarks you wish to be notified about.
|
| All Benchmark Status Reports |
|
|
