TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
August 2003
Overview
 

The TPC held a General Council meeting on June 19 in San Francisco, California. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, work has ended on a modification of the TPC-C benchmark but the specification of a brand new OLTP benchmark is making progress. With respect to decision support, progress continues on the new DS benchmark with the completion of the parameter substitution model. As for the transactional web e-Commerce benchmark, development continues on the next version of TPC-W.
 
TPC-C and OLTP
 

The TPC-C Maintenance Subcommittee finished the exploration of a modification to the current version of the TPC-C benchmark. After determining the areas to study, prototyping the implementation, and studying the results, the subcommittee came to the conclusion that the proposed modifications would not meet all of the TPC's objectives. In particular, the proposed new version of the TPC-C benchmark would not provide a fair comparison with the existing set of TPC-C results. Based on the knowledge of the objectives and the prototype results, the development effort on a new version of the TPC-C benchmark specification has been terminated.

Work on a new OLTP benchmark continues and the OLTP Technical Development Subcommittee passed two important motions. The first motion established June 1, 2003 as the deadline for identifying all new features for the benchmark. The second motion established August 1, 2003 as the deadline for submitting position papers on all identified new features.

A total of 7 areas were identified for consideration for new features in the benchmark.

  • Security
  • Middle Tier
  • Non-linear Database Scaling vs. Performance
  • Interactions between Transaction Harness and Frames
  • Data Types Used in Schema
  • Isolation Levels for All Transactions
  • Trade Limit / Market Feed alternate implementation

The source code included in the initial proposal has been ported to a variety of member company platforms. In addition, the ability to generate flat files for loading into the database has also been added.
 
TPC-H, R and DS
 

The TPC-H and TPC-R subcommittees have been working on extending the allowable scale factors to 30,000 and 100,000 terabytes. This process entails both changes in the specification and the data generator. Changes in the specification are complete while the subcommittee is currently implementing the necessary changes in dbgen. Vote on additional scale factors is expected at the August meeting.

Progress continurs on a new Decision Support (DS) benchmark. In the last two months the TPC-DS development subcommittee has completed the parameter substitution model. It now guarantees repeatable query runs with different substitution parameters. Furthermore, the query set is being finalized. Until the next meeting the aim is to make sure that all queries parse on all database systems and to analyze query run time behavior to ensure they meet expectations. Furthermore the data maintenance rules will be reviewed and members will be encouraged to build prototypes.
 
TPC-W
 

The TPC-W subcommittee continued its development of the next version of TPC-W. The benchmark is a transactional web service benchmark consisting of database interactions displaying ACID properties running against a commercial application server.

The TPC-W subcommittee is in the prototyping phase of the web services version of the benchmark. The benchmark will provide for the measurement of the performance and throughput of a commercially available web/application server that provides web services to clients. The primary metrics are SIPS (Service Interactions Per Second), Price Performance which is $/SIPS, and the system availability date. An industry standard database product running on a second system will be used to provide data storage and integrity of business information. The business model addressed by this benchmark is focused on the business to business scenario. The SUT will be driven by web service requests made through the industry standard and vendor neutral protocols of SOAP and XML. The business logic of the web services must be implemented such that it runs in the context of a managed environment. The benchmark incorporates common B2B functionality such as distributed transactions and message queuing utilizing durable queues.

Currently the benchmark defines the following B2B interactions:

  • New Customer
  • Create Order
  • Shipping Process
  • Stock Management
  • Order Status
  • New Products List
  • Product Detail
  • Change Payment Method
  • Product Update

All processing requirements for these operations have been defined. All of the implementation details, including SUT architecture restrictions, have been reviewed and accepted by the subcommittee. The subcommittee has also defined the majority of the reporting requirements for the Test Run.

The subcommittee has defined the requirements for the environment in which the benchmark is measured. The environments must be either compliant with the Java Runtime Environment (JRE) 1.3 or greater as published by Sun Microsystems, or ECMA-335 as published by the European Computer Manufacturers Association. Requirements for random number generators and seeding methods have been defined.

The subcommittee is adopting additional requirements such as adhering to the WS-I basic profile (BP1.0) for web service interoperability and requiring a standard random number generation algorithm. The schedule for submitting the benchmark for TPC General Council vote is set for December of 2003.
 
Public Relations Committee
 

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. The committee has reviewed the responses to the RFP for PR activities and is finalizing the selection of the winning proposal.
 
All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!

©2001-2019 TPC. All rights reserved. Web-Design and Maintenance by Hotea Solutions