News

TPC-C Benchmark Heats Up, Windows Back on Top

• By Scott Bekker
• 05/20/2003

The flurry of activity marks a renewed interest in the Transaction Processing Performance Council's TPC-C benchmark after several years with little meaningful activity at the top performance tier of the benchmark.

By matching HP's Superdome result on May 9, IBM showed that Unix/RISC will not go down easy now that Windows Server 2003 Datacenter Edition and Itanium 2 processors can scale in SMP configurations. By responding quickly on Tuesday, HP, Microsoft and Intel are showing that the Wintel partners are willing to fight for TPC-C bragging rights.

Oracle and Sun have stayed out of this benchmark battle so far, choosing instead to criticize the current legitimacy of a benchmark that both vendors used extensively in the past.

When NEC posted its then-No. 1 result of 514,035 transactions per minute (tpmC) on April 23, it was the first new No. 1 posting on the non-clustered benchmark since Fujitsu claimed the top spot with a 128-processor behemoth in August 2001. The NEC system used 32 Itanium 2 processors, Windows Server 2003 Datacenter Edition and 64-bit SQL Server 2000 Enterprise Edition.

NEC's record stood for one day. On April 24, coinciding with the Windows Server 2003 launch, HP hit 658,277 tpmC with a 64-processor Superdome server running Intel's 64-bit "Madison" processors. "Madison" is the third-generation Itanium chip with 6 MB of cache and is expected to ship this summer. The 64-bit Microsoft software stack involved Windows Server 2003 Datacenter Edition and SQL Server Enterprise Edition.

IBM quickly upped the ante, improving on HP's result with 680,613 tpmC. The IBM Unix/RISC result came at a cost 14 percent higher than the HP result, at $11.13/tpmC compared with $9.80 for HP.

On Tuesday, HP, Microsoft and Intel came charging back with a raw performance result of 707,102 tpmC at a cost of $9.13 per tpmC. The results represent an improvement of 4 percent on the performance of the IBM system at a cost that is 18 percent lower per transaction.

HP and Microsoft used the exact same system in both runs. In addition to the massive database server, the $6.453 million configuration involves 32 dual-processor clients and a huge storage array.

"The April 24 benchmark that we had was very schedule driven," said SQL Server group product manager Tom Rizzo, referring to the goal of putting out a benchmark with the Windows Server 2003 launch. "We did no performance tuning on that benchmark. After we released that benchmark, we decided to go back and do some tuning. The number we have now is more the real number."

Meanwhile, Microsoft and its partners created a stir three years ago with the launch of Windows 2000 when they used clusters of Windows and SQL Server machines to anchor benchmark runs that set performance records far higher than the traditional big-iron, non-clustered Unix/RISC configurations that are more similar to what Microsoft and HP are running now.

The new result brings Microsoft's far less controversial non-clustered results on par with the best of its clustered results. The performance of the new HP Superdome benchmark trails the last of the clustered benchmark runs by less than 1 percent (707,102 tpmC to 709,220).