TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
February 2005

Overview

The TPC held a General Council meeting on December 16th in Dana Point, California. The main focus of the work again was on refining existing benchmarks and laying the foundation for new benchmarks. Several of the committees (TPC-C, TPC-H and R) reviewed the potential impact of the proposed Pricing Specification on their benchmarks. On the OLTP front, work continues on the TPC-E specification, with the release of a new draft of the specification and a new version of the TPC provided code. With respect to decision support, development of the DS benchmark continues in the areas of queries, specification, and data maintenance. As for the transactional web e-Commerce benchmark, work continues on the freshly renamed TPC-App specification with substantial time and resources spent on testing and reviewing modifications to the interaction parameters, mix, and workload in order to increase the application server to DBMS processing ratio. Finally, the Pricing Subcommittee focused on completing the Pricing specification and working with the TPC benchmark subcommittees to coordinate changes that will be compatible with the specification.

 

Current Benchmarks

TPC-C

The TPC-C Maintenance subcommittee met with the Pricing subcommittee to discuss wording changes that will be required by the proposed TPC Pricing Specification.  The subcommittee will be reviewing the changes and will continue to work on the coordination and integration of the TPC-C specification and the proposed pricing specification.

 

TPC-H and R

In anticipation of the acceptance of the pricing specification the group met with a representative of the pricing subcommittee to discuss the impact of the pricing specification on TPC-H.

 

New Benchmarks

TPC-E (OLTP)

Over the past month, the TPC-E subcommittee has continued its efforts in the development of a new OLTP benchmark. During this time a new draft of the specification has been approved. In addition, a new version of the TPC provided code has been released to subcommittee members.

Prototyping work continues. Initial database sizing is being evaluated; criteria for establishing the scale factor are being explored; runtime I/O characteristics are being studied; and isolation levels for the various transactions have been preliminarily set.

Moving forward, prototyping will continue in the areas mentioned above. Additionally, work has already begun on the next draft of the specification.

 

TPC-DS

The subcommittee has continued its work on developing TPC-DS in the areas of queries, specification and data maintenance.  For 34 out of the current 50 base templates the group reviewed sensitivity analysis data to assure metric comparability of results obtained from the same scale factor.  In order to assure query soundness, including semantic correctness, number of rows returned, elapsed time, the subcommittee reviewed prototype data.  A deadline of January 31st was set until query templates could be submitted for consideration in TPC-DS.  After sensitivity analysis of all queries has been performed the group will define the final set at the February 2005 meeting.  In preparation for this step the group defined query acceptance criteria.  The group also reviewed the current specification to synchronize it with current development in the areas of schema and data population as well as to identify areas that need to be refined. 

 

TPC-App (TPC-W)

The TPC-W subcommittee has finalized all modifications and clarifications from the review phase of TPC-App. TPC-App is a web service benchmark consisting of transactions involving database interactions, durable message queue operations, and displaying ACID properties running against a commercial application server. The primary metrics are Total SIPS (Service Interactions Per Second), SIPS per Application Server, Price Performance (for example $/SIPS), and the System Availability Date.

The subcommittee has spent the last two months determining the final interaction mix and parameters to attain the desired application server to database server processing ratio.  The current ratio is now approximately 5 : 1.    The complete suite of application verification test, ACID tests, and reporting requirements has been finalized. Additionally, requirements regarding various application logs were clarified.  SUT restrictions, in particular permissible caching techniques were further clarified. The format of the Executive Summary was completed, and includes a sample document in the appendices. Also added to the appendices is a rationale document which answers some of the common questions regarding the benchmark and business model.

The TPC-App specification has been sent to Mail Ballot.  The Mail Ballot phase extends until February 16th. If approved, the specification will be available immediately for publication of results.

 

Other TPC Activities

Public Relations Committee

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. Presentations for TPC speakers have been prepared and are in review. The prototype of a new customized information system for subscribers of TPC lists and reports is now being tested.

 

Pricing

The TPC Price Subcommittee drafted a pricing specification that can be applied to all TPC benchmarks. The specification does not substantially change what is priced, but the requirements for reporting and ordering are enhanced and strengthened, allowing improved verifiability of pricing. If approved, the pricing specification will be referenced by all existing and future TPC specifications for a consistent methodology.

The draft TPC pricing specification is currently in the Mail Ballot process. If approved, it becomes effective on August 31, 2005 for all benchmarks, and earlier for any given benchmark that passes a minor revision that recognizes the TPC-Price specification.


All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!