TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
February 2006
Overview

The TPC held a General Council meeting on December 8th, in Dana Point, California. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, work continues on the latest draft of the TPC-E specification. Prototype data was reviewed and runtime workload characteristics were studied. Version 5.6 of the TPC-C specification was completed and approved. With respect to decision support, the TPC-H subcommittee corrected a bug in dbgen and developed a strategy to ensure the quality of future releases. Development of the DS benchmark continues with closure on most of the open issues. A draft of the specification was approved for public comment. Finally, the newly renamed TPC-App subcommittee (formerly TPC-W) updated the TPC-App specification to version 1.1.2 to clarify some definitions and implementations.
 

Current Benchmarks
TPC-C

The TPC-C Maintenance Subcommittee completed work on Version 5.6 of the TPC-C Specification.  This version is effective immediately and will be required as of February 6, 2006.  The changes in Version 5.6 are:

  • Wording in Clause 5.5.1.2 to clarify the 8 hours of log space requirement
  • Replacement of Clause 6.6.6 to address the limits of operator intervention during the benchmark
  • Wording in Clause 8.1.1.2 to clarify the requirements of the diagram included in the Executive Summary
Results obtained with TPC-C Version 5.6 are comparable with those obtained with TPC-C Version 5.5.
 
TPC-H

The dbgen bugs that were reported at the October TPC meetings were resolved.  The subcommittee also passed a motion during the first conference call following the October meeting to allow the use of dbgen 1.3.0 and dbgen 2.3.0 for publications before December 1st, 2005.  Further, the subcommittee developed a strategy to assure quality of TPC-H’s future data and query generators.  This strategy includes reference data sets for each table at all scale factors, for update data at all scale factors and for qgen query parameters using different seeds.  New rules will be included in a new version of the TPC-H specification at the February, 2006 TPC meeting.  It is intended to release a new minor version of the TPC-H specification and TPC-H tools (dbgen/qgen).
 
TPC-App (TPC-W)

The TPC-App subcommittee worked on TPC-App Version 1.1.1 definition clarifications, implementation clarifications, as well as review of Java and C# based reference implementations.  The changes are effective immediately and are incorporated in the latest version of the specification, TPC-App Version 1.1.2 and are available on the web site.

TPC-App is a web service benchmark consisting of transactions involving database interactions, durable message queue operations, and displaying ACID properties running against a commercial application server. The primary metrics are Total SIPS (Service Interactions Per Second), SIPS per Application Server, Price Performance (for example $USD/SIPS), and the System Availability Date.

The majority of the changes were related to the recent adoption of the common TPC usage of the terms processors/cores/threads.  In order to encourage TPC-App results, the subcommittee is working towards providing reference implementation source code.  Currently, work has begun on a complete open source java implementation.  Additionally, a C# based RBE driver application and post-processing tool for performing and analyzing TPC-App measurements is targeted to be released as open source and made available on the TPC web site.
 

New Benchmarks
TPC-E

Over the past two months, the TPC-E subcommittee has continued its efforts in the development of a new OLTP benchmark. Time has been spent reviewing prototype data and studying various runtime workload characteristics. In addition, a new version of the TPC provided code has been released to subcommittee members.

Moving forward, the subcommittee will continue to study and modify various runtime workload characteristics. Any feedback from the release of the public draft specification will be reviewed and addressed.
 
TPC-DS

The focus of this meeting was to review query sensitivity analysis data and qualification output.  The TPC-DS subcommittee also reviewed the specification and closed open issues.  There are some unresolved bugs with dbgen2 and qgen2 for TPC-DS (RI).  Despite these bugs queries and tools are ready for running the query part of the benchmark.  The Data Maintenance process is now defined but implementation by database vendors needs to be finished.  All member companies are highly encouraged to run the benchmark from the beginning to end before going to company review. 

A draft version of the TPC-DS specification was approved for public review and comment.
 

Other TPC Activities
Pricing Specification

The TPC approved the TPC Pricing Specification Version 1.0.1 in June 2005. No significant changes have been requested to the current version of the specification. The Pricing Maintenance Committee has developed minor clarifications that will be rolled into a future editorial revision of the specification.

The use of the Pricing Specification is currently required for all active TPC benchmarks.
 
Public Relations Committee

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. The prototype of a new customized information system for subscribers of TPC lists and reports continues its tests. The PR Committee is also implementing the new methodology for how CPUs are counted on the TPC web site (see the August 2005 BSR for further information).
 
All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!

©2001-2019 TPC. All rights reserved. Web-Design and Maintenance by Hotea Solutions