Can Haskell outperform C++?

Wed May 16 16:40:26 CEST 2012, Gregg Lebovitz wrote: 2) ... I think the problem with current comparisons, is that they are designed to favor imperative languages. Please be specific: - Which current comparisons? - How do you know what they are designed to favor?

Isaac, I was looking at the debian coding contest benchmarks referenced by others in this discussion. All of the benchmarks algorithms, appear to be short computationally intensive programs with a fairly low level of abstraction. In almost all examples, the requirement says: you must implement the X functions as implemented in Java, or C#, or C++. The k-nucleotide even specifies a requirement to use an update a hash-table. I wonder if someone here could come up with a set of benchmarks that would out perform C++. Interesting that you would focus on this one comment in my post and not comment on one on countering the benchmarks with a new criteria for comparing languages. Gregg On 5/16/2012 12:59 PM, Isaac Gouy wrote:
Wed May 16 16:40:26 CEST 2012, Gregg Lebovitz wrote:
2) ... I think the problem with current comparisons, is that they are designed to favor imperative languages.
Please be specific:
- Which current comparisons?
- How do you know what they are designed to favor?
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 05/16/2012 09:02 PM, Gregg Lebovitz wrote:
Isaac,
I was looking at the debian coding contest benchmarks referenced by others in this discussion. All of the benchmarks algorithms, appear to be short computationally intensive programs with a fairly low level of abstraction.
In almost all examples, the requirement says: you must implement the X functions as implemented in Java, or C#, or C++. The k-nucleotide even specifies a requirement to use an update a hash-table.
I wonder if someone here could come up with a set of benchmarks that would out perform C++.
That's easy:
let ones = 1 : ones take 5 ones [1,1,1,1,1]
I'm not sure how much C++ code you'd have to write to produce the correct answer without butchering the intent too much, but the naïve translation to C++ loops infinitely. Obviously Haskell is infintely better than C++!11111oneone!
Interesting that you would focus on this one comment in my post and not comment on one on countering the benchmarks with a new criteria for comparing languages.
Comparing languages is a highly non-trivial matter involving various disciplines (including various squidgy ones) and rarely makes sense without a very specific context for comparison. So the short answer is: mu. Discovering the long answer requires a lifetime or more of research and may not actually result in an answer. Regards,

On 5/16/2012 3:57 PM, Bardur Arantsson wrote:
Comparing languages is a highly non-trivial matter involving various disciplines (including various squidgy ones) and rarely makes sense without a very specific context for comparison. So the short answer is: mu. Discovering the long answer requires a lifetime or more of research and may not actually result in an answer. Depends on your goal.
From the discussion on the cafe, it appears that many believe performance is not the only criteria for evaluating languages. I agree with the comment made by Ertugrul Söylemez. Some people focus on getting the best performance because they don't know any better. If given better evaluation criteria they might think differently. Asserting the other criteria publicly helps others understand the benefits of Haskell, otherwise they just see the lower performance numbers from places like the debian contest.

In a lecture today I presented (for quite other reasons) a simple combinatorial enumeration problem where the difference between two algorithms was far larger than any plausible difference between programming languages. If a programming language makes it easier to explore high level alternative algorithms, you may very easily end up with faster programs in a "slower" language. (Sorry about the very long line: broken return key.)

Richard, Thank you. This is an example of what I had in mind when I talked about changing the playing field. Do you have a slide deck for this lecture that you would be willing to share with me? I am very interested in learning more. Gregg On 5/16/2012 9:13 PM, Richard O'Keefe wrote:
In a lecture today I presented (for quite other reasons) a simple combinatorial enumeration problem where the difference between two algorithms was far larger than any plausible difference between programming languages. If a programming language makes it easier to explore high level alternative algorithms, you may very easily end up with faster programs in a "slower" language. (Sorry about the very long line: broken return key.)

On 5/16/12 3:57 PM, Bardur Arantsson wrote:
Comparing languages is a highly non-trivial matter involving various disciplines (including various squidgy ones) and rarely makes sense without a very specific context for comparison.
Exactly. That's what I was trying to get at re the problems of comparing Haskell to C++ (or indeed any pair of dissimilar languages). A legitimate comparison will involve far more than microbenchmarks, but then a legitimate comparison must always have a specific focus and context in order to be able to say anything interesting. The problems I see in language comparisons is that by and large they tend not to have any such focus[1], and consequently they shed little light on how the languages compare and shed a bit of darkness by serving only to confirm initial biases. [1] A nice counter-example to this trend are papers like: http://www.cse.unsw.edu.au/~chak/papers/modules-classes.pdf There was another one comparing the expressivity of Java-style polymorphism vs Haskell-style polymorphism, based on an analysis of in-the-wild code; but I can't seem to pull it up at the moment. -- Live well, ~wren

From: Gregg Lebovitz
Sent: Wednesday, May 16, 2012 12:02 PM
I was looking at the debian coding contest benchmarks referenced by others in this discussion.
"debian coding contest" ? It's been called many things but, until now, not that.
All of the benchmarks algorithms, appear to be short computationally intensive programs with a fairly low level of abstraction.
Tiny tiny toy programs - more or less 100 lines - not quite small enough to be microbenchmarks. Why might that be? Well, not IO bound. Do people usually mean IO performance when they compare programming languages? (I guess you must have excluded meteor-contest from your consideration. It's a coding contest and so doesn't specify an algorithm.)
In almost all examples, the requirement says: you must implement the X functions as implemented in Java, or C#, or C++.
binary-trees - No, it doesn't say that. thread-ring - No. chameneos-redux - No. pidigits - No. regex-dna - No. k-nucleotide - No. mandelbrot - No. reverse-complement - No. spectral-norm - "Each program must implement 4 separate functions / procedures / methods like the C# program." (The point being cross function calling so don't amalgamate 4 functions into a single function.) fasta-redux - No. fasta - No. meteor-contest - No. fannkuch-redux - "defined by programs in Performing Lisp Analysis of the FANNKUCH Benchmark" n-body - "Each program should model the orbits of Jovian planets, using the same simple symplectic-integrator - see the Java program."
The k-nucleotide even specifies a requirement to use an update a hash-table.
Probably not too onerous a requirement for a test of hash table lookup :-) Some people like hash tables, go figure. http://gregorycollins.net/posts/2011/06/11/announcing-hashtables -snip-
Interesting that you would focus on this one comment in my post and not comment on one on countering the benchmarks with a new criteria for comparing languages.
Too obvious to be interesting. Interesting that you haven't said how you know they are "designed to favor imperative languages" ;-)
On 5/16/2012 12:59 PM, Isaac Gouy wrote:
Wed May 16 16:40:26 CEST 2012, Gregg Lebovitz wrote:
2) ... I think the problem with current comparisons, is that they are designed to favor imperative languages.
Please be specific:
- Which current comparisons?
- How do you know what they are designed to favor?

Isaac, I see your point. Probably I shouldn't have made that assertion given my limited understanding of the benchmarks. I want to thank you for your kind and gentle way of pointing this out to me. I feel very welcomed and encourage. I still plan to work on the performance paper with the help of others on this list. I look forward to your support and constructive feedback. Gregg On 5/16/2012 6:59 PM, Isaac Gouy wrote:
-snip-
Interesting that you would focus on this one comment in my post and not comment on one on countering the benchmarks with a new criteria for comparing languages. Too obvious to be interesting.
Interesting that you haven't said how you know they are "designed to favor imperative languages" ;-)
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

From: Gregg Lebovitz
Sent: Thursday, May 17, 2012 5:50 AMI look forward to Subject: Re: [Haskell-cafe] Can Haskell outperform C++?
Isaac,
I see your point. Probably I shouldn't have made that assertion given my limited understanding of the benchmarks. I want to thank you for your kind and gentle way of pointing this out to me. I feel very welcomed and encourage.
I still plan to work on the performance paper with the help of others on this list. I look forward to your support and constructive feedback.
Gregg
Gregg, You wrote that you were looking at the benchmarks and then made a definite assertion about what was shown on the website. Unsuspecting readers would assume that you were truthfully reporting what you saw on the website. I cannot explain to them why you made an assertion which could be seen not to be true, simply by looking at the benchmarks game website. best wishes, Isaac

Isaac, I understand. Thank you. I will be more careful about my wording in the future. I really do appreciate your taking the time to point this out to me. I am here to learn and help where I can. Gregg On 5/17/2012 11:25 AM, Isaac Gouy wrote:
From: Gregg Lebovitz
Sent: Thursday, May 17, 2012 5:50 AMI look forward to Subject: Re: [Haskell-cafe] Can Haskell outperform C++? Isaac,
I see your point. Probably I shouldn't have made that assertion given my limited understanding of the benchmarks. I want to thank you for your kind and gentle way of pointing this out to me. I feel very welcomed and encourage.
I still plan to work on the performance paper with the help of others on this list. I look forward to your support and constructive feedback.
Gregg
Gregg,
You wrote that you were looking at the benchmarks and then made a definite assertion about what was shown on the website. Unsuspecting readers would assume that you were truthfully reporting what you saw on the website.
I cannot explain to them why you made an assertion which could be seen not to be true, simply by looking at the benchmarks game website.
best wishes, Isaac
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (5)
-
Bardur Arantsson
-
Gregg Lebovitz
-
Isaac Gouy
-
Richard O'Keefe
-
wren ng thornton