TPC Benchmark Status
November 1999 |
By Kim Shanley, TPC's Chief Operating Officer
|
TPC Benchmark Status is published about every two months.
The first and primary purpose of the newsletter is to keep interested parties informed about the content, issues, and schedule of the TPC's benchmark development efforts. The second purpose is to invite new members to join these important development efforts.
We've already outlined most of the reasons for joining the TPC in another article, Why Join.
To receive the status report by email, please click here.
|
| Last Meeting |
The TPC held a General Council meeting on October 21st in Atlantic City, New Jersey. There were two significant developments at this meeting:
|
- The TPC-W Subcommittee postponed asking the Council to proceed to final approval of TPC-W until the December 1999 meeting (it was scheduled to go to mail ballot approval at this meeting);
- the TPC-C Subcommittee requested to proceed to the public and company review of Version 4 of TPC-C.
|
| TPC-W Subcommittee |
Jerrold Buggert of Unisys, the TPC-W Subcommittee Chair, provided another overview of the comments the Subcommittee had received during the company/member review period, which concluded in September.
|
- The Subcommittee received 199 comments from over 20 sources. This included input from both within and outside the TPC membership. In summary, there were no “show-stopping” criticisms or holes found in the benchmark workload.
- Since receiving the first wave of comments, and based on additional prototype results which indicated too heavy of a load on the database server, the Subcommittee had made some significant changes in the benchmark workload, including:
- Allowing caching in the application server.
- Allowing the use of commercially available text search engines.
- Increasing the level of encryption to128 bit RC4/1024 bit key.
- Requiring redundancy for all durable storage.
- Added some commercially available functionality requirements, including:
- Application level caching.
- Load balancing.
- Routing.
- Added application transparency requirement to the use of commercially available functions.
- Further clarified the Ecommerce functionality of the benchmark to include:
- SSL.
- Shopping Cart.
- Credit Card
Verification.
- Secure Online Payment Authorization.
- TPC-W Schedule
- 12/99: TPC-W submitted for TPC mail ballot approval.
- 1-2/2000: TPC-W approved.
- 3/-4/2000results can be published.
|
| TPC-C Subcommittee |
John Fowler of Sun, Chair of TPC-C Subcommittee, reviewed the latest changes to Version 4 of TPC-C, the newest major version of the TPC’s OLTP benchmark.
|
- The primary focus of the Subcommittee was preparing the Version 4 specification for public and company review. To that end, the Subcommittee had:
- Deleted the requirements for ACID properties on the message queue.
- Removed the requirement for message queue function to be provided by a commercial application.
- Changed "deferred" to "asynchronous" throughput.
- Some TVR and wording changes.
- Reviewed Version 4 goals:
- John indicted the Subcommittee had met the general goals of the V4 development although admitted that the number of users and tpmC rate was not lowered as much as they would have liked.
- TPC-C Version 4.0 Schedule
- 10/1999: TPC-C Version 4 (V.4) submitted TPC company review.
- 12/1999: TPC-C V. 4 submitted for TPC mail ballot approval.
- 2/2000: TPC-C V. 4 approved as an official TPC benchmark.
- 4/2000: First TPC-C V. 4 results can be published.
|
| TPC-R/H Subcommittees |
Roger MacNicol of Sybase, Chair of the TPC-R and TPC-H Subcommittees, highlighted a few minor changes made to each benchmark specification:
|
- In TPC-R (TPC’s business reporting decision support benchmark), the Subcommittee clarified three issues: 1) Rewrite Clause 1.5.5 to permit vertical partitioning; 2) clarify streams/sessions (Clause 5.3.3.1) in the power test, which had been interpreted differently by different TPC auditors; 3) re-write Clause 7.1.3 in line with TAB ruling and drop requirement for pricing 8 hours log disk. In this re-write recovery data was defined to be the “data required to recover,” and circular logs were permitted. As a result of these changes, the Council approved the Subcommittee’s request to create TPC-R, Version 1.2, a minor version change.
- In TPC-H (TPC’s ad-hoc decision support benchmark), the Subcommittee had re-written clause 7.1.3 in line with TAB ruling and drop requirement for pricing 8 hours log disk. As a result, the Council approved the Subcommittee’s request to create TPC-H, Version 1.2.1, an insignificant version change.
|
| New Decision Support Benchmark Results |
As those familiar with the TPC or these reports will recall, in
April of this year, the TPC split its TPC-D decision support benchmark into two
different benchmarks, TPC-R and TPC-H. TPC-R, the business reporting benchmark,
represents a decision support environment in which users know the queries very well
and can optimize their DBMS to execute these queries very rapidly. TPC-H, the ad
hoc benchmark, represents the original TPC-D ad-hoc decision support environment
in which users don’t know the queries in advance and the execution times can
be very long. After the creation of TPC-R and TPC-H in April, it took a few months
for the companies to digest the requirements of these new benchmarks. But now we
see seven new TPC-H results, and one TPC-R result has been published. We would
certainly like to see more results on the books, so to speak, but even these few
results are positive for the industry. The competitive race to produce better
decision support results has re-ignited, and this in turn will generate better and
more powerful decision support systems for the industry’s users.
|
| All Benchmark Status Reports |
|
|
