David Roundy wrote:
On Fri, Jan 26, 2007 at 10:17:28AM -0500, Al Falloon wrote:
The idea is to gather a bunch of programs written in Haskell, and which are representative for the Haskell community (i.e. apps, libraries, ...). A While ago I tried to write a Haskell version of John Harrops ray-tracer benchmark (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the
Kenneth Hoste wrote: performance was not very good (the OCaml version I based it on was at least 10x faster).
I would be happy to contribute my code to the benchmark suite if you are interested. Perhaps someone can point out obvious speed-ups that I missed while trying to improve the performance.
I would think that what we'd want to benchmark would be clean, optimized actually-used code. I.e. things like Data.Bytestring, so that we could see how compilers differed on important code, or how the code generated on different architectures differed. e.g. if jhc beats ghc on amd64, the ghc developers would probably be very curious as to why, and how to fix it.
Code that's not been properly optimized with respect to strictness, etc, would fail to focus the tests on important optimizations of the compiler. But of course, the benchmark code should also be clean, since we want to ensure that our compilers are good enough that we can write useful beautiful code that is also fast.
I tried to optimize it, but I couldn't approach the speed of the OCaml version. I followed the performance tuning advice from the Wiki, and had even resorted to writing the inner loop calculations using all unboxed doubles, but without significant improvements. This is exactly the kind of code that I write most often, and I would love to see improvements in the optimizations for this kind of numerically intensive code (especially without having to resort to compiler-specific unboxed representations). I agree that common libraries like ByteString need to be well represented, but the original request additionally included programs that are representative of applications. A ray-tracer (even with a fixed scene and only one type of scene primitive) is a fairly nice approximation of a real numerical batch-oriented application while still being small enough to understand and modify. I expect thats why John chose it as his benchmark application in the first place. I think it would also be a good idea to include SPJ's web server (I think its from an STM paper). A lot of the people outside the Haskell community will be able to relate better to metrics like pages/second.