On 1/16/06, Daniel Fischer
Hi,
Is it only my machime, or can you confirm that for the Ackermann benchmark, it's very good for C that they chose 9 and not a larger value? For 10, we are significantly faster and for 11,12,13, we can run rings around the C-programme: dafis@linux:~/Documents/haskell/shootout> time ./cacker3 10; time ./acker 10; time ./cacker3 11; time ./acker 11; time ./cacker3 12; time ./acker 12 Ack(3,10): 8189
real 0m0.664s user 0m0.660s sys 0m0.010s Ack(3,10): 8189
real 0m0.405s user 0m0.400s sys 0m0.000s Ack(3,11): 16381
real 0m6.255s user 0m6.220s sys 0m0.000s Ack(3,11): 16381
real 0m1.731s user 0m1.710s sys 0m0.000s Ack(3,12): 32765
real 0m35.270s user 0m35.050s sys 0m0.000s Ack(3,12): 32765
real 0m10.673s user 0m10.590s sys 0m0.000s dafis@linux:~/Documents/haskell/shootout> time ./acker 13; time ./cacker3 13 Ack(3,13): 65533
real 1m4.476s user 1m4.050s sys 0m0.010s Ack(3,13): 65533
real 2m50.645s user 2m47.020s sys 0m0.020s
This is interesting. Hopefully it's not intentional, but it's quite obvious that for benchmarks where the fastest time is only a few fractions of a second, languages with more complex runtime systems will be unfairly slow due to the startup cost. I suspect that e.g. Java suffers quite a bit from this in the shootout. There is already a startup benchmark in there (although for some lightweight langauges I suspect that the cost of writing "hello world" is larger than the startup, so it's perhaps not optimal) so it would be good to remove the startup factor in benchmarks which are not meant to test that. In other words I'd prefer if all benchmarks are reconfigured to target an execution time of at least a few seconds for the fastest benchmarks. /S -- Sebastian Sylvan +46(0)736-818655 UIN: 44640862