Hi, I think the mail server may have been acting up earlier. I sent this to Haskell-beginners, but it more properly belongs here. I found something interesting. "General wisdom" is that Clean (or OCaml) is faster than Haskell. The claim is often followed by a link to the Debian shootout. But on closer inspection, I question this conclusion. The Debian shoot out actually has four sets of benchmarks: 1) Intel 32-bit one core. 2) Intel 32-bit quad-core. 3) Intel 64-bit one core. 4) Intel 64-bit quad-core. It turns out that Clean is only faster for (1). For the others, Haskell is faster. Here I compare Haskell, Clean, OCaml, Lisp SBCL, C# Mono and Fortran because they are all in the same ball mark: 32-bit sing core [1]: Lisp, Fortran, Clean, Haskell, C# Mono. 32-bit quad-core [2]: Haskell, C# Mono, Lisp, Clean, Fortran. 64-bit sing core [3]: Fortran, OCaml, Haskell, Clean, C# Mono, Lisp. 64-bit quad-core [4]: Haskell, OCaml, Lisp, C# Mono, Fortran, Clean. Notes: * The order is "fast language first". * There are no results for OCaml for 32-bit. * "Lisp" is "List SCBL" whatever that is. Tentative conclusions: 1) Haskell makes very good use of multiple cores. It smokes Clean. 2) For single core, they are neck and neck. Whether Clean is faster depends non the architecture. What do you think? Daniel. P.S. [1]http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1 [2]http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1 [3]http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1 [4]http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1
On Sun, May 3, 2009 at 2:24 PM, Daniel Carrera
Hi,
I think the mail server may have been acting up earlier. I sent this to Haskell-beginners, but it more properly belongs here.
I found something interesting. "General wisdom" is that Clean (or OCaml) is faster than Haskell. The claim is often followed by a link to the Debian shootout. But on closer inspection, I question this conclusion. The Debian shoot out actually has four sets of benchmarks:
1) Intel 32-bit one core. 2) Intel 32-bit quad-core. 3) Intel 64-bit one core. 4) Intel 64-bit quad-core.
It turns out that Clean is only faster for (1). For the others, Haskell is faster. Here I compare Haskell, Clean, OCaml, Lisp SBCL, C# Mono and Fortran because they are all in the same ball mark:
32-bit sing core [1]: Lisp, Fortran, Clean, Haskell, C# Mono. 32-bit quad-core [2]: Haskell, C# Mono, Lisp, Clean, Fortran. 64-bit sing core [3]: Fortran, OCaml, Haskell, Clean, C# Mono, Lisp. 64-bit quad-core [4]: Haskell, OCaml, Lisp, C# Mono, Fortran, Clean.
Notes:
* The order is "fast language first". * There are no results for OCaml for 32-bit. * "Lisp" is "List SCBL" whatever that is.
Tentative conclusions:
1) Haskell makes very good use of multiple cores. It smokes Clean. 2) For single core, they are neck and neck. Whether Clean is faster depends non the architecture.
What do you think?
Daniel.
P.S. [1]http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1 [2]http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1 [3]http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1 [4]http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=all&d=data&ghc=on&ocaml=on&sbcl=on&csharp=on&ifc=on&clean=on&calc=calculate&box=1
Perhaps it's just that no one has parallelized the Clean programs? Haskellers seem to care about the shootout programs much more than Cleaners do. eg. I randomly looked at Mandelbrot on [2]. clean: http://shootout.alioth.debian.org/u32q/benchmark.php?test=mandelbrot&lang=clean&id=2 haskell: http://shootout.alioth.debian.org/u32q/benchmark.php?test=mandelbrot&lang=ghc&id=2 I can't really read Clean, but it certainly looks as if it's making no use of concurrency at all, while the Haskell one most certainly is. -- gwern
Gwern Branwen wrote:
Perhaps it's just that no one has parallelized the Clean programs? Haskellers seem to care about the shootout programs much more than Cleaners do.
I'm not sure about the second comment. I haven't seen the Haskell site mention the shootout, whereas web pages about Clean often do. Perhaps what happens is that Haskell has a larger community, so there's more chance that someone will submit an optimized program. Daniel.
Hello Daniel, Sunday, May 3, 2009, 10:42:06 PM, you wrote:
I'm not sure about the second comment. I haven't seen the Haskell site mention the shootout
just search cafe archives ;) -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
I haven't seen the Haskell site mention the shootout, whereas web pages about Clean often do.
Well, there certainly has been significant efforts on the shootout in the Haskell community. There's wiki pages about it [1] and it comes up on the Haskell reddit and proggit frequently. With regard to the original discussion on comparing Haskell shootout results to Clean I suggest people think more about making Haskell use as little memory as Clean does and not worrying so much about the Haskell vs. Clean CPU time. Thomas [1] http://haskell.org/haskellwiki/Shootout [2] http://www.reddit.com/r/haskell/search?q=shootout - and that doesn't show comments, just some submitted links.
Hello Gwern, Sunday, May 3, 2009, 10:29:37 PM, you wrote:
32-bit quad-core [2]: Haskell, C# Mono, Lisp, Clean, Fortran.
I can't really read Clean, but it certainly looks as if it's making no use of concurrency at all, while the Haskell one most certainly is.
probably other languages goes w/o built-in concurrency support and this test doesn't allow to use external libs :) -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Hello Daniel, Sunday, May 3, 2009, 10:24:52 PM, you wrote:
32-bit sing core [1]: Lisp, Fortran
:) this test measures speed of some programs, not "languages". results are depends mainly on bundled libraries and RTS. by no means it demonstrates speed of compiler-generated code of carefully-written program what is typically considered as "language speed". the reasons are: 1) C++ people (and probably Fortran too) aren't so interested in making fastest possible programs as Haskell community. it becomes popular a few years ago, you can find that Haskell becomes several faster at average since then, which doesn't reflect actual improvements in GHC code generation (10-20%) 2) Most programs there depend on speed of libraries. Moreover, there is limitation that it should be *bundled* libraries, so results greatly depends on what currently included in one compiler or another 3) it's prohibited to write your own fast code if bundled library is too slow (for example, because it's too general) -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Bulat Ziganshin wrote:
32-bit sing core [1]: Lisp, Fortran
:) this test measures speed of some programs, not "languages".
I know. But since I know that you know that too, I opted for brevity. "How can we benchmark a programming language? We can't - we benchmark programming language implementations. How can we benchmark language implementations? We can't - we measure particular programs." Daniel.
participants (4)
-
Bulat Ziganshin -
Daniel Carrera -
Gwern Branwen -
Thomas DuBuisson