-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

I recently stumbled across this thread, related to the Shootout (which I run) and wanted to add a few comments.

First, I agree that I am a Haskell novice; however, I rely on people like yourselves to show me where I am doing things wrong. If our solutions are horribly inefficient, please tell us what we can do to resolve the problem. But note that in some cases, we are purposely being inefficient so that we can measure the cost of certain basic operations.

Doug Bagely started the Shootout as a comparison of scripting languages. After I revived it, I started focussing more on functional and logic languages, since I was curious how these languages would fare in comparison with the more typically-used languages like C/C++. Consequently, many of the tests show this heritage and are highly imperative. We have been working to relax some of these rules, and have added many new tests that try to be more amenable to comparison of lazy and 'strict' languages.

In our defense I would like to plead that it is not easy to devise short problems (e.g., that can be expressed in a page or so of code - -- we try to limit it to 100 lines of C++ code as a rough benchmark), and that also take enough run time for a meaningful measurement. So, many of the tests initially suffered from foolish looping and repetition as a means of extending the run times sufficiently to allow measurement. We have been changing tests so that fewer of them rely on these kinds of activities to generate results. We are always interested in new ideas, and would consider any benchmark ideas you have.

Finally, I don't agree at all that Haskell's scores show poorly. The GHC compiler is one of the better performers, and regularly trounces other languages when we let it do its job. For example, we had to jump through many hoops to convince the compiler to actually invoke all the loops required to do the "same way" tests on methodcall and similar benchmarks, because the compiler was so sharp at identifying wasted effort and optimizing it away. Perhaps we don't say this strongly enough on the site, but I was truly impressed at its ability to toss aside wasted calculations and focus on the core problem. I'll admit it was frustrating because we wanted to see how long it would take for Haskell to make N recursive calls, or what have you, but the fact that the compiler saw that there was no need to do the recursion to get the result is quite amazing! I'm sure this would be extremely useful when compiling real-world programs.

There have been many cases where the "obvious" solution in Haskell is a great performer, but we discover that it's so fast because it has tossed aside wasted cycles and just done the work necessary to arrive at a solution. It's unfortunate, since we then have to dirty the program with wasted steps to force it to exercise the tasks required. But in a "real world" test I think Haskell would show quite well.

As we evolve into the newer tests, I think you will see Haskell make a better showing. Simon Marlow (and others) have been a huge help in optimizing the existing tests in a "fair" fashion, and as we provide more "same thing" type tests, I think Haskell will pull further ahead.

Thanks,

- -Brent

P.S. I must say that I find Haskell to be the most esthetically pleasing language -- it's quite beautiful when printed!

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (Darwin) iD8DBQFDuWIlzGDdrzfvUpURAnJuAJ90G9ltXH5m+jsz6D0QoucoY6anNACfcfkQ Mg2K0NO0kbBYLy4ZX7LCoeM= =uGnw -----END PGP SIGNATURE-----