Home / Opinions / Benchmarking Blues / Benchmarking Blues

Benchmarking Blues

Next up are notebooks. These are just the sort of prodcuts that manufacturers ought to be able to certify properly but all too often they are unreliable pre-production samples that won’t complete one test or another. It’s particularly galling when a benchmark fails to complete just towards the end of many hours of testing.

Perhaps the most ridiculous time wasting issues we encounter are those caused by administrative errors from a company or its PR agency. Both Riyad and I have wasted time benchmarking notebooks that despite being sent to us to review turn out to not actually be available for sale in this country, either in the particular hardware configuration supplied, or sometimes in any form at all.

Now we’re just saying all this for your sympathy. Testing hardware is what we do, and if it’s difficult it’s our problem, not yours. Right?

Well, the thing is that this clearly affects end users too. All this is an indicator that too many companies are ready to push technology out of the door that’s simply not ready for prime time. I'm not talking about making performance improvements over time available through driver or product revisions, or features being added later. I'm simply talking about products delivering to an acceptable level at launch - essentially doing what they say on the tin.

And who pays the price when they don't? Well we certainly do in terms of making testing harder. With the limited resources at our disposal we’re left with less time to produce our reviews, which affects how complete and informative they can be. But it also directly affects the consumer, especially those who like to play at the bleeding edge of technology, who will all too often find themselves getting stung by products that just don't work properly.

So what I’m saying is that while we’re as obsessed as anybody by the next big thing, it will be a lot more productive in the long run if products are more thoroughly tested before being rushed out to market. For example ATI seems to be taking it’s time getting its CrossFire technology out of the door. But if it means that it will work properly when it is released – and not five driver and BIOS revisions down the line, then it will be doing the right thing.

But this needs to be an industry wide initiative. If I knew that a product would work first time round for the sake of a couple of week's testing I'd be more than willing to wait that bit longer for the latest tech toys to appear. In the long run, we'd all benefit.

comments powered by Disqus