Hi
I have to disagree with this. That is, I don't object to Don's explanation of why the shootout entries degraded in this particular case, but I do think that Andrzej was right to point this out: "Perhaps making a collective effort towards benchmarking Haskell programs and analyzing the results in some methodic way could prove helpful?" and I think that he *is* pointing out a fundamental issue here.
We have the nofib suite, it could do with some attention, but it is (or was) a pretty good performance tester.
Maybe it's just me, but as someone who has some amount of experience with implementing optimizations for Haskell, I find it nearly impossible to precisely understand and measure exactly how those optimizations improve (or degrade) program performance.
The problem is that something like GHC is very complex, with lots of transformations. When transformations are firing other transformations, which in turn fire other transformations, it doesn't take a great deal to disrupt this flow of optimisation and end up with a result which doesn't accurately reflect the particular change you made. Better knowledge of the end effects on a program isn't necessarily going to translate to better knowledge of the optimisations effect. Maybe if we had a greater number and variety of optimising compilers we'd be able to more freely experiment with optimisation techniques in different settings. At the moment GHC is all there is (with Jhc not ready for mainstream use yet) Thanks Neil