Brian Goetz on micro-benchmarks

Brian Goetz writes on micro-benchmarks and discourages people from writing any. The first part of the article details why some particular banchmark is flawed, which is only mildly interesting unless you're interested in lock performance in Java (and why wouldn't you be?), but the second part gives excellent advice on performance testing in general.

I thought I would give a pointer to his article as I've been guilty of micro-benchmarking myself on this blog more than once. To my defense, I've always said that these gave only a rough idea of performance in a real-life scenario, and that any performance testing should be made in the context of the real application.

The problem for us API developers is that our users ask for guidance on when to use this or that particular technique. We can still give some general answers based on reasonable micro-benchmarking and analysis of the IL code in some very simple cases, and that's enough for most users. But nothing will ever replace a good profiler and a lot of experimentation on the real application when it's being used in real-life conditions.

 

Read the article here: