WebMink
Simon Phipps's personal commentary

War of the Pets [WebTech]

If you're at all connected with the Java environment, you'll remember the furore over the Pet Store example. The synopsis: a verbose piece of sample code was created ages ago for J2EE that showed how to use pretty much all its features in a three-tier application intended for educational use - not pretty, but sample code for everyone. Someone at Microsoft built a two-tier application that looked similar but seemed to run faster and use less code, and their marketing department claimed that it proved .NET was better despite the fact it was a comparison of pumpkins with kumquats. Most thinking people spotted the huge holes in the argument and said that, if a benchmark was what was wanted, really it would be better it implement the same thing in both places - when people did, they unsurprisingly got neck-and-neck with the .NET version. Despite the demonstrated inaccuracy, Microsoft continued to make marketing claims that .NET was way better than J2EE on the basis of this comparison. End of potted history.

This week a consulting company has published a revised J2EE pet store implementation that they claim is optimised, and no doubt the Microsoft hype machine is spinning up to make all sorts of claims about .NET and J2EE as allegedly they were behind the work anyway - they also put two in-house engineers on tuning the .NET version. But already, analysis and comment show that the new J2EE application is (a) not designed the same way as the .NET version, (b) not optimised to use modern J2EE and (c) makes unneccessary calls. Seems it wasn't the consulting company's A-team doing the work.

Anyone who has ever been involved in benchmarking will know that it's a bad game to get into. Even when benchmarks are rigorously designed in a commodity marketplace, they are rarely if ever an accurate reflection of real-world deployment. But when the word 'benchmark' is used to describe code written for the marketing benefit of just one vendor in a marketplace where there is no philosophical unity let alone commodity implementation, the only conclusions one can come to are about integrity and intent of the participants rather than about products and technologies.

posted at 9:22 AM (UK) | |