TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
August 2004
Overview

The TPC held a General Council meeting on June 25th in Paderborn, Germany. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, work continues on the new TPC-E specification with progress in the areas of the partitionability of schema, initial database load, corrections to ACID tests, transaction isolation requirements, full disclosure requirements, and transaction output parameters. With respect to decision support, progress continues on the new DS benchmark queries, data generator,  and schema. As for the transactional web e-Commerce benchmark, nearly all the comments that were received on the TPC-W Version 2 draft specification have been addressed. The committee is now looking at the feasibility of including clustered vs. non-clustered results. Similarly, the Pricing Subcommittee completed the review of comments submitted on the draft Pricing Specification and has addressed them in an updated draft.

 

Current Benchmarks
TPC-C

The TPC-C Maintenance Subcommittee did not convene at this meeting.

 
TPC-H and R

The TPC-H/R Maintenance Subcommittees did not convene at this meeting.

 

New Benchmarks
TPC-E (OLTP)

The TPC-E development effort continued to make robust progress in the last two months.  The subcommittee accepted version 0.24 as the current draft specification.  Specific areas which were addressed are:

  • Partitionability of Schema. To facilitate parallel load and incremental growth of the database, it is highly desirable that the database can be loaded in slices which are independent of other slices.  This must be carefully balanced against runtime behavior so that the workload is not prone to trivial scale-out optimizations.
  • Initial Database Load. Generation of the initial database must be able to run in parallel to take advantage of large multi-CPU systems and reduce load time.  In addition, the initial database must match the essential characteristics of the database which is produced by the steady-state runtime.
  • Corrections to ACID Tests. A review of the ACID tests lead to minor revisions and corrections.
  • Transaction Isolation Requirements. Progress was made in determining the isolation level which will be required for each of the transactions in the benchmark.
  • Full Disclosure Requirements Clause.  The FDR requirements clause was written and added to the specification.
  • Transaction Output Parameters. The spec was updated to define precisely which transaction output parameters must be returned to the driver and recorded for post-run processing.

Moving forward, work continues with prototyping and analysis of the overall workload.

 
TPC-DS

The subcommittee has continued its work on developing TPC-DS in the areas of queries, data generator and schema. 

The data generator was advanced to generate NULL values.  Bugs found by members were fixed.  Due to the increased complexity of TPC-DS’ data generator compared to TPC-H’s data generator (it needs many more random numbers), it takes hours to generate the data set of a 100GB database (even if generated in parallel).  This will be addressed.  Four members companies built five 100 GB prototype TPC-DS databases.  Queries were executed against these databases.  Due to the choice of query predicates some of the queries did not return results.  These queries are being investigated.

The schema has been reviewed and cleaned up so that columns are defined consistently across sales channels: column names, column data types and constraints.  Most of the changes do not require changes in the data generator.  However, the ones that do affect the data generator will be integrated as soon as possible.

 
TPC-W

The TPC-W subcommittee has completed reviewing the comments received during the public review phase. TPC-W Version 2 is a web service benchmark consisting of transactions involving database interactions, durable message queue operations, and displaying ACID properties running against a commercial application server. The primary metrics are SIPS (Service Interactions Per Second), Price Performance which is, for example $/SIPS, and the system availability date.

The subcommittee received numerous comments on the TPC-W V2 draft specification.  Almost all comments have been addressed with either revisions or clarifications to the specification. The largest revision is the permission and inclusion of a Clustered configuration.  This inclusion, however, has raised a few additional comments concerning the marketability of Clustered results compared to Non-Clustered results.  The subcommittee is reviewing proposals to ensure the focus of the benchmark remains on the application server system.  

The subcommittee will address the final remaining comments and marketing issues. 

 

Other TPC Activities
Public Relations Committee

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. The TPC is in the process of renewing its contract with its PR firm as well as beginning work on a project that will enable the TPC to deliver customized information to its subscribers.

 
Pricing

The TPC Price Subcommittee drafted a pricing specification that can be applied to all TPC benchmarks. The specification does not substantially change what is priced, but the requirements for reporting and ordering are enhanced and strengthened, allowing improved verifiability of pricing. If approved, the pricing specification will be referenced by all existing and future TPC specifications for a consistent methodology.

The pricing specification is currently scheduled for mail ballot in October.

 
All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!

©2001-2019 TPC. All rights reserved. Web-Design and Maintenance by Hotea Solutions