While I usually prefer to focus on technical solutions that can help readers solve specific business problems, after reading Microsoft’s latest press announcement about SQL Server 2008 #1 ranking in the TPC-E benchmark I felt compelled to discuss this somewhat misleading announcement. While SQL Server 2008 certainly did top the TPC-E benchmark, the fact is that no other vendors participate in the TPC-E benchmark. As one of my colleagues put it, they are number one in a field of one. Woo Hoo! Seriously, this is no accomplishment to be trumpeting to the world.
Don’t get me wrong. I’m one of SQL Server 2008’s best fans. And no one was more excited than me when SQL Server used to top the TPC-C benchmarks. However, that’s not what this is about. The TPC-C is where Microsoft and SQL Server used to compete. IBM and Oracle are still there. The TPC-E is a totally different benchmark. A couple of years back Microsoft quit issuing TPC-C benchmarks in favor of the new TPC-E benchmark. However, IBM and Oracle did not follow suit. Nor did anyone else.
Without multiple vendor participation TPC-E numbers are meaningless. They can’t be used to compare SQL Server 2008 against other database platforms (the true purpose behind the TPC in the first place) nor can they even be used to compare SQL Server 2008 against previous version of SQL Server. If no one besides Microsoft uses TPC-E then there can be no cross database vendor comparisons. It can’t even be used for comparing SQL Server 2008 and the earlier SQL Server 2005 release because they are run on different hardware platforms.
Industry standard benchmarks are about comparing the performance of multiple vendor solutions. At the best, announcing #1 TPC-E is silly and meaningless – unless Oracle and IBM begin producing TPC-E benchmarks as well. However, all of their latest OLTP tests have been in the TPC-C. At the worst, Microsoft TPC-E announcements are misleading as they imply that SQL Server is better than the other competing enterprise database platforms (isn’t that what it is to be #1?). But, in fact, that’s not demonstrated by the TPC-E. It makes you wonder if the problem is that SQL Server 2008 can’t post competitive scores in the TPC-C.
Order Your SQL Fundamentals CD Today! Learn how to use SQL Server, understand Office integration techniques and dive into the essentials of SQL Express and Visual Basic with this free SQL Fundamentals CD.
You've Deployed SharePoint...Now What? This one-day free online conference delivers the technical knowledge needed to kick MOSS up a notch. In one information-packed day, independent SharePoint experts will present practical, real-world information and provide take-away, ready-to-use solutions
What Would You Do If You Ran Microsoft? ITTV's 2008 inaugural video contest, "If I Ran Microsoft..." is your chance to tell it like it is. Be goofy or be serious, but don"t miss this chance to have fun, win prizes, and go viral in a major way.
Maximize Your SharePoint Investment This web seminar discusses how true bi-directional replication of SharePoint content from one server to another enables branch offices to maintain access to current SharePoint content.
Register
