Hi

I have to disagree with this. That is, I don't object to Don's explanation of why the shootout entries degraded in this particular case, but I do think that Andrzej was right to point this out: "Perhaps making a collective effort towards benchmarking Haskell programs and analyzing the results in some methodic way could prove helpful?" and I think that he *is* pointing out a fundamental issue here.

We have the nofib suite, it could do with some attention, but it is (or was) a pretty good performance tester.

Maybe it's just me, but as someone who has some amount of experience with implementing optimizations for Haskell, I find it nearly impossible to precisely understand and measure exactly how those optimizations improve (or degrade) program performance.

The problem is that something like GHC is very complex, with lots of transformations. When transformations are firing other transformations, which in turn fire other transformations, it doesn't take a great deal to disrupt this flow of optimisation and end up with a result which doesn't accurately reflect the particular change you made. Better knowledge of the end effects on a program isn't necessarily going to translate to better knowledge of the optimisations effect. Maybe if we had a greater number and variety of optimising compilers we'd be able to more freely experiment with optimisation techniques in different settings. At the moment GHC is all there is (with Jhc not ready for mainstream use yet) Thanks Neil

On Thu, 25 Jan 2007, Kirsten Chevalier wrote:

Anything better than staring at intermediate code would be an improvement, since time spent staring at intermediate code usually is time spent narrowing down the 2 lines out of 1000 that are relevant. Maybe it's possible to design tools that could help with that narrowing-down process.

I suspect a structure editor would be a good start - from there it's easier to start looking at visualisations both of the code itself and the result of queries on it, and if you bolt on hs-plugins to the side I imagine a library of commonly-used queries would arise over time. -- flippa@flippac.org A problem that's all in your head is still a problem. Brain damage is but one form of mind damage.

Neil Mitchell wrote:

That would be very neat. Another neat trick would be generalising optimisations so that there are fewer and more independant passes, this would make it easier to understand (and is what I was working on for Yhc).

Well, it's the nature of repeatedly applying local reductions that you will neither really know what's going nor truly understand how to improve it. One particular example is the GHC rules mechanism. It's much better than doing nothing, but it often fails to fuse yet another list. I think that one can only achieve deterministic optimization by carefully choosing and exploiting the type system of the intermediate language. For instance, short cut fusion and strictness analysis can be expressed as type inference. If you have an intermediate language where things like boxing and forcing of thunks is explicit and typed, you have a chance to eliminate such expensive constructs by type inference and conventional inlining magic. One point is that local optimization is hard to extend across the boundaries imposed by recursion and the fixed point combinator. But I can imagine that switching to a core language with non-totality (lifted types) manifest in the types, like System F with a fixed point combinator fix :: Pointed a => (a -> a) -> a is key to crossing this boundary.

Yhc has intermediate code that is substantially more Haskell like, and with the command:

loadCore "file.ycr" >>= writeFile "file.html" . coreHtml

You can generate an active Html document that lets you explore the Core in a more interactive way - including hyperlinks for function names, syntax hilighting etc.

i.e: http://www.cs.york.ac.uk/fp/yhc/Roman.html

All of these things make playing with Yhc Core much more fun than playing with GHC Core. There is absolutely no reason you couldn't add all these things to GHC Core, then perhaps you'd save some time when it does come to the "examine core" level.

Wow, the core looks really cool! One look and you see it all. I would even rename the local variables to single letters like a,b,c because the cryptic numbers are quite hard to track. This is something coreHtml can do. Also, I'd add more color, like making punctuation marks grey, but that's a very personal taste. Concerning the shootout, it deserves much laud for it is certainly not an easy task to set it up for so many language and it keep running. But I'm not very fond of the benchmarks themselves. The goal seems to be to measure how fast different languages can execute a hard wired algorithm which specifies memory management and data layout. While this is a valid goal, I don't think it is a worthy one. It simply does not get to the interesting facts, namely how fast the natural algorithms for each language are. It just compares highly artificial algorithm implementations. Random examples are the nsieves ("count the prime numbers from 2 to M [...] create a sequence of M boolean flags") and the k-nucleotide ("define a procedure/function to update a hashtable of k-nucleotide keys") benchmarks. Both boolean flags and hash tables are completely alien to Haskell. The former would be naturally implemented by a list of primes, the latter naturally with a generalized trie. Of course, disallowing unboxed arrays will certainly move Haskell down the ranking. But what do we gain from unlimited array necromancy? Do we get a fair account on how fast and short day to day Haskell really is? And how to tweak the compilers to make day to day Haskell an even more pleasant experience? I think not. (Do not misunderstand me, ByteStrings are fine for it is the purely functional style that counts). And sorry, but using the number of gzipped bytes for comparing the code length is just ridiculous. Regards, apfelmus

Neil Mitchell wrote:

The problem is that something like GHC is very complex, with lots of transformations. When transformations are firing other transformations, which in turn fire other transformations, it doesn't take a great deal to disrupt this flow of optimisation and end up with a result which doesn't accurately reflect the particular change you made. Better knowledge of the end effects on a program isn't necessarily going to translate to better knowledge of the optimisations effect.

Maybe if we had a greater number and variety of optimising compilers we'd be able to more freely experiment with optimisation techniques in different settings. At the moment GHC is all there is (with Jhc not ready for mainstream use yet)

Thanks

Neil _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Although there may not be a lot of optimizing Haskell compilers, there are compilers for languages similar to Haskell that consistently perform well. One could point to O'caml or others in the ML family, or even more interesting is the case of Clean, whose syntax heavily borrows from Haskell. What do the Clean folks do that has made their compiler so consistently competitive? Is it the abc machine? Frankly I'm amazed that a three stack based virtual machine can be translated into such efficient machine code in register centric CPU architecture. Can Haskell compiler writers learn something from this? Supposedly someone is working on a Haskell compiler that will use the clean compiler back end. I can't believe that Clean is so fundamentally different, even with uniqueness types, that it has an edge in compiler optimization. -- View this message in context: http://www.nabble.com/Re%3A--Haskell---Fwd%3A-Re%3A-Computer-Language-Shooto... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.

Uniqueness types does give some extra optimisation potential, such as destructive updates if you can guarantee a variable is only referred to once. But even with that, the language that has impressed me most on the shootout is Clean. Where the Haskell community spends significant time they can beat Clean, but generally the Clean does very well and looks much more idomatic FP than the Haskell.

Clean has strict, unboxed strings by default. We only got these in Haskell in the last few months. -- Don