Re: [Haskell-cafe] Speed of Error handling with Continuations vs. Eithers

On Thu, May 13, 2010 at 8:13 PM, wren ng thornton wrote:

Andrea Vezzosi wrote:

...

On Thu, May 13, 2010 at 10:51 AM, wren ng thornton wrote:

...

Andrea Vezzosi wrote:

...

wren ng thornton  wrote:

...

With this change [1] I can't notice any difference for your benchmark[2]. Then again, all the runTest calls take 0 msec and I've had no luck making the computation take much time; perhaps your computer can detect a difference.

On my machine, with ghc-6.12.1, yours and the original ErrCPS give quite similar results, both ~2x slower than Either. However it's important to note that these results are highly dependent on the monadic expressions being evaluated, with a different benchmark you can get an huge speedup with the CPS versions.

That's very curious. After installing Criterion, my machine (OSX 10.5.8 2.8GHz Intel Core2Duo, GHC 6.12.1 with -O2) shows only 1% difference between my ErrCPS and Either on this benchmark. Alas, I can't print kernel density graphs since Crieterion charts are broken on 6.12. It seems buggy that your platform would behave so much differently...

I got the measurements from the original code, could you share the code that uses criterion instead?

The 1% number was buggy because I hadn't factored the generation of random lists out of the benchmark. But, having fixed that, I still can't replicate your numbers: I get 12us for Either, vs 17us for EitherCPS.

http://community.haskell.org/~wren/wren-extras/test/Control/Monad/ErrCPS/Cri...

Yet another version of the same benchmark, this time using Microbench:

http://community.haskell.org/~wren/wren-extras/test/Control/Monad/ErrCPS/Mic...

Microbench seems to replicate your numbers better: 2551.930ns vs 4466.832ns (or 391.86 vs 223.87 calls per second)--- though this is getting into the range where there might be Int overflow issues corrupting the results (a similar problem showed up when benchmarking Data.Trie vs Data.Map), so it may warrant further investigation.

That might be the case, i'm on 64bit: saizan@astarte:~$ uname -a Linux astarte 2.6.32-ARCH #1 SMP PREEMPT Tue Feb 23 19:43:46 CET 2010 x86_64 Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz GenuineIntel GNU/Linux saizan@astarte:~$ ./CriterionBenchmark warming up estimating clock resolution... mean is 6.834442 us (80001 iterations) found 1240 outliers among 79998 samples (1.6%) 1131 (1.4%) high severe estimating cost of a clock call... mean is 107.2316 ns (41 iterations) benchmarking Either collecting 100 samples, 1039 iterations each, in estimated 683.8220 ms bootstrapping with 100000 resamples mean: 6.563462 us, lb 6.553649 us, ub 6.570454 us, ci 0.950 std dev: 41.74602 ns, lb 23.76971 ns, ub 67.67842 ns, ci 0.950 found 8 outliers among 100 samples (8.0%) 2 (2.0%) low severe 4 (4.0%) high mild 2 (2.0%) high severe variance introduced by outliers: 0.990% variance is unaffected by outliers benchmarking ErrCPS collecting 100 samples, 1 iterations each, in estimated 1.334000 s bootstrapping with 100000 resamples mean: 13.14468 ms, lb 13.10442 ms, ub 13.18208 ms, ci 0.950 std dev: 198.3150 us, lb 182.0600 us, ub 220.7957 us, ci 0.950 variance introduced by outliers: 0.993% variance is unaffected by outliers If i'm reading it correctly this gives even worse results: 6us vs. 13ms

-- Live well, ~wren _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe