- Haskell-Cafe - Haskell.org

[14/16] SBM: Behind the measurements (rationale)
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

// I am getting sick and tired of working on this project and it's probably // better to get it fired off than polishing it any further. // // This email could benefit from being rewritten from a rough draft into a // well-crafted letter but that would take a couple of hours. // // So here it is, a lot rougher than I'd like -- but it *IS* :) why so big input files? easiest way to spot non-linearity and bad memory behaviour. Anyway, files should be big enough to overflow caches and kick the gc in. (short files interesting, too, but the big ones cause more complex behaviour of run-time system and CPU. If complex behaviour is behaved then simple behaviour probably is too -- but can still get its constant factors improved. If complex behaviour bad, then shouldn't that be fixed in any case?) waitpid4() has a struct w/ info about the child program's resource usage. unfortunately, the peakrss field is not filled in. Seems to be a general Unix problem. I've seen complaints on the net that Solaris doesn't fill it in, either. Other solution needed. pause-at-end, /proc/self/maps + /proc/self/status. VmmHWM = peak of VmmRSS, which is Resident (working) Set Size. It doesn't say what is shared with other processes or the operating system, though. In our case, we don't expect to share anything but some libraries -- which nobody else wants to share with us anyway (except for the C library). We are the only user of them. Discovered about a week ago that I could probably have used waitid() w/ WNOWAIT flag but didn't know. Was quick to write pause-at-end, anyway. It took about 15 minutes from the desire to know the peak memory use to having written and tested the first cut of it. Pause-at-end not completely bullet-proof in case of dyn libraries that get unloaded before the end of the program has been reached. On the other hand, plenty good enough for these tests + can conceivably allow more intricate poking around than waitid() solution. getting good measurements - eatmem, dd, probably should also dd library. good to have a "sacrificial run". Good to measure how good the measurements are (rel. std.dev + user/sys/real check). why average -- disturbances are mostly interrupts, daemons that everybody have anyway, slightly luckier/unluckier physical pages. These are real effects that nobody can control anyway. I'm not interested in the best possible times on an ideal, undisturbed machine with a helpful kernel. I'm interested in clean times under realistic circumstances. Therefore average instead of minimum. why I use real and not user/sys -- handling of blocking reads vs. mmap vs. madvise/fadvise vs. reading in separate thread in the future. User+sys would probably give me better numbers at the moment and I could change to real later. Still, I choose to stay with real (and the difference is marginal, anyway). Funny that the exact distribution of time between sys and user fluctuates a lot. In space-bslc8-lenfil-2 sys varies between 0.160s and 0.244s. Real is completely stable with 5x 1.396s and 1x 1.397s. look at /proc/interrupts, perhaps copy before/after to .intr? Warn if more than 100 (or 1000) Hz + 10%? write date/time + runlevel to platforminfo and/or sysinfo. barcharts why barcharts. should the time/mem barcharts be equal length? don't think so (hard to colour them in a text file. Would work with less -r and the console but not in an email or a text editor. Visual difference is good). But should perhaps not be /that/ different. visible markers if measurements bad. (5% real/user/sys check, typically within 0.1% on old laptop when doing a quick or thorough benchmark. Occasionally up to 1% - and 3% on c/byte-4k because it only takes 56ms in total.) prints out how tight the user/sys/real thing is. microarchitecture -- performance counters. Would be interesting to look at once the obvious performance problems have been handled. Let's fix the memory usage of bytestrings, the performance of lazy bytestrings, and start using registers in the machine code first. regularity of input file probably means that branch predictor on all three CPUs can remember pattern of spaces vs. non-spaces (or at least part of the pattern). Branch predictors not only use two-bit saturating counter for strongly non-taken/weakly non-taken/weakly taken/strongly taken. They also try to remember the pattern of jumps/non-jumps. A more realistic test would have less regular input file. This effect is very small given the current performance limiters, though. cache -- turned out to be pretty regular (by eyeballing cachegrind reports). Go up a factor of 10 in filesize and the number of access also went up a factor of 10. The miss ratios stayed the same. The miss ratios differed a bit between the benchmarks but I don't think it's time to look into that yet. The data are available, though, for those who can't wait to look into that. minor page faults we gather that through /usr/bin/time -- and could also get the same info from dumping the right file inside /proc/self/. Probably not important yet. Probably will be once all the low-lying fruit has been gathered up from the ground. More of a factor on slower OS'es than Linux. C files. Buffer size. reading it all in one go is slower than (re)using a small buffer. Cache effects, both in the operating system when copying (because the destination will be cached with a small buffer but non-cached with a big buffer) and in the application (everything will be cached with the small buffer, nothing with the big buffer). Note that at least the Core and the Athlon64 have automatic prefetchers that tries to fill the cache in advance so we don't have to wait for the cache misses. Doesn't quite seem to work. Older caches had a different write behaviour, they were write-through instead of the modern (lazy) write-back. For those caches the writing to the user-space buffer should be slow even when a small buffer is reused all the time (because we would have to wait for all the writes to be flushed out to main memory). C files. getchar/getchar_unlocked. A comparison with getchar() is NOT what the simple haskell program does [grammar!]. Thread-safe by default (cause of libraries). getchar_unlocked() is what the haskell programs do. getchar() and getchar_unlocked() use a single buffer for stdin. getwchar() and getwchar_unlocked() included at the insistence of wli. Much slower, because it is run-time dependent on locale (to choose encoding). Therefore, can't be a macro like getchar_unlocked() is. With an indirect jump, should be same speed as getchar() on Core and Athlon64 -- but curiously isn't. C and Haskell integer sizes and other limitations. Haskell uses unboxed 32-bit signed integers, except in lazy lenfil tests. Most of the C programs are simple and just use an int for the space count. One of them (space-megabuf)is more complicated. off_t is 64-bit. ssize_t is 32-bit. Potential overflow in c/space-megabuf. Potential 32-bit wrap-around in all my C tests. Same problem with all Haskell tests, except for the two lenfil tests that use lazy bytestrings, because they use a 64-bit int for the length of the intermediate string of just the spaces filtered out from stdin. Also potentially out of memory. In practice, they have almost the same limit because they use about 107MB for the 143MB input file. In other words, it will run out of virtual address space or RAM or swap at about the same time that the others will run out of bits in a 32-bit signed integer. -Peter

1 0

[13/16] SBM: Graphs that show the infidelity of -sstderr
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

Everybody seems to use +RTS -sstderr to measure the memory performance of their Haskell (ghc) code. That's what I started out doing, too. However, top seemed to occasionally disagree with the run-time system. I decided to dig deeper and ask the operating system at process exit time. I use LD_PRELOAD to get my code injected into each benchmarked process where it gets activated when the process is about to exit and copies /proc/self/maps and /proc/self/status to stderr where the test harness captures them. Unfortunately, -sstderr really *IS* unreliable! tools/regensstderr.sh and tools/sstderr.pl are two scripts that together generate bar graphs for an existing measurement tarball. I used the following shell code to generate graphs for 7 different ghc/bytestring combinations: (for F in ghc-thorough-6.6.1.tgz \ ghc-thorough-6.8.2.tgz ghc-thorough-6.8.2-bs0902.tgz \ ghc-thorough-6.9.tgz ghc-thorough-6.9-bs0902.tgz \ ghc-thorough-head.tgz ghc-thorough-head-bs0902.tgz; \ do \ printf "%s\n-------\n" $F; \ tools/regensstderr.sh $F; \ printf "\n\n"; \ done \ ) > xx The graphs clearly show that -sstderr sometimes severely underestimates the memory use and sometimes somewhat overestimates it. Moreover, the errors are not always for the same programs. This holds for all ghc versions I've tested: 6.6.1, 6.8.2, 6.9.20071119, and head as of noon 2007-12-19. -Peter ghc-thorough-6.6.1.tgz ------- hs/byte-bs----acc: 1 147448 ▏ |██████████████████▎ | hs/byte-bs----foldlx: 145 147468 ██████████▎ |██████████████████▎ | hs/byte-bs----foldrx: 1 147452 ▏ |██████████████████▎ | hs/byte-bsl---acc: 2 2872 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1572 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1568 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1572 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147448 ▏ |██████████████████▎ | hs/space-bs-c8-count: 1 147448 ▏ |██████████████████▎ | hs/space-bs-c8-foldlx-1: 145 147468 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldlx-2: 145 147468 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldrx: 1 147448 ▏ |██████████████████▎ | hs/space-bs-c8-lenfil: 145 147464 ██████████▎ |██████████████████▎ | hs/space-bslc8-acc-1: 2 2872 ▏ |▍ | hs/space-bslc8-acc-2: 2 2872 ▏ |▍ | hs/space-bslc8-acc-3: 2 2868 ▏ |▍ | hs/space-bslc8-count: 2 1476 ▏ |▏ | hs/space-bslc8-foldl: 2 1552 ▏ |▎ | hs/space-bslc8-foldlx-1: 2 1556 ▏ |▎ | hs/space-bslc8-foldlx-2: 2 1552 ▏ |▎ | hs/space-bslc8-foldr-1: 297 169480 █████████████████████|█████████████████████| hs/space-bslc8-foldr-2: 297 169480 █████████████████████|█████████████████████| hs/space-bslc8-lenfil-1: 38 38632 ██▋ |████▊ | hs/space-bslc8-lenfil-2: 38 38628 ██▋ |████▊ | hs/space-bsl---foldlx: 2 1560 ▏ |▎ | hs/space-xxxxx-acc-1: 1 1568 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1572 ▏ |▎ | hs/space-xxxxx-foldl: 1 1572 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1556 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.8.2.tgz ------- hs/byte-bs----acc: 1 147500 ▏ |██████████████████▎ | hs/byte-bs----foldlx: 145 147528 ██████████▎ |██████████████████▎ | hs/byte-bs----foldrx: 1 147500 ▏ |██████████████████▎ | hs/byte-bsl---acc: 2 2920 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1632 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1632 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147492 ▏ |██████████████████▎ | hs/space-bs-c8-count: 1 147500 ▏ |██████████████████▎ | hs/space-bs-c8-foldlx-1: 145 147524 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldlx-2: 145 147520 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldrx: 1 147496 ▏ |██████████████████▎ | hs/space-bs-c8-lenfil: 145 147524 ██████████▎ |██████████████████▎ | hs/space-bslc8-acc-1: 2 2920 ▏ |▍ | hs/space-bslc8-acc-2: 2 2924 ▏ |▍ | hs/space-bslc8-acc-3: 2 2924 ▏ |▍ | hs/space-bslc8-chunk-1: 2 1512 ▏ |▏ | hs/space-bslc8-chunk-2: 2 1512 ▏ |▏ | hs/space-bslc8-chunk-3: 2 1548 ▏ |▎ | hs/space-bslc8-chunk-4: 2 1512 ▏ |▏ | hs/space-bslc8-count: 2 1520 ▏ |▎ | hs/space-bslc8-foldl: 2 1620 ▏ |▎ | hs/space-bslc8-foldlx-1: 2 1620 ▏ |▎ | hs/space-bslc8-foldlx-2: 2 1616 ▏ |▎ | hs/space-bslc8-foldr-1: 297 169544 █████████████████████|█████████████████████| hs/space-bslc8-foldr-2: 297 169544 █████████████████████|█████████████████████| hs/space-bslc8-lenfil-1: 38 38644 ██▋ |████▊ | hs/space-bslc8-lenfil-2: 38 38648 ██▋ |████▊ | hs/space-bsl---foldlx: 2 1616 ▏ |▎ | hs/space-xxxxx-acc-1: 1 1636 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/space-xxxxx-foldl: 1 1636 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1608 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.8.2-bs0902.tgz ------- hs/byte-bs----acc: 1 147616 ▏ |██████████████████▌ | hs/byte-bs----foldlx: 1 147624 ▏ |██████████████████▌ | hs/byte-bs----foldrx: 1 147624 ▏ |██████████████████▌ | hs/byte-bsl---acc: 2 3040 ▎ |▍ | hs/byte-xxxxx-acc-1: 1 1636 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1636 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-count: 1 147612 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-1: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-2: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-foldrx: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-lenfil: 21 167176 ███ |█████████████████████| hs/space-bslc8-acc-1: 2 3048 ▎ |▍ | hs/space-bslc8-acc-2: 2 3048 ▎ |▍ | hs/space-bslc8-acc-3: 2 3052 ▎ |▍ | hs/space-bslc8-chunk-1: 2 1636 ▎ |▎ | hs/space-bslc8-chunk-2: 2 1640 ▎ |▎ | hs/space-bslc8-chunk-3: 2 1644 ▎ |▎ | hs/space-bslc8-chunk-4: 2 1636 ▎ |▎ | hs/space-bslc8-count: 2 1656 ▎ |▎ | hs/space-bslc8-foldl: 2 1632 ▎ |▎ | hs/space-bslc8-foldlx-1: 2 1636 ▎ |▎ | hs/space-bslc8-foldlx-2: 2 1636 ▎ |▎ | hs/space-bslc8-foldr-1: 149 149076 █████████████████████|██████████████████▊ | hs/space-bslc8-foldr-2: 149 149076 █████████████████████|██████████████████▊ | hs/space-bslc8-lenfil-1: 2 1940 ▎ |▎ | hs/space-bslc8-lenfil-2: 2 1940 ▎ |▎ | hs/space-bsl---foldlx: 2 1620 ▎ |▎ | hs/space-xxxxx-acc-1: 1 1636 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/space-xxxxx-foldl: 1 1636 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1604 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.9.tgz ------- hs/byte-bs----acc: 1 147752 ▏ |█████████████████▍ | hs/byte-bs----foldlx: 145 147748 █████████▊ |█████████████████▍ | hs/byte-bs----foldrx: 1 147744 ▏ |█████████████████▍ | hs/byte-bsl---acc: 2 3172 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1904 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1900 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147736 ▏ |█████████████████▍ | hs/space-bs-c8-count: 1 147744 ▏ |█████████████████▍ | hs/space-bs-c8-foldlx-1: 145 147756 █████████▊ |█████████████████▍ | hs/space-bs-c8-foldlx-2: 145 147752 █████████▊ |█████████████████▍ | hs/space-bs-c8-foldrx: 1 147732 ▏ |█████████████████▍ | hs/space-bs-c8-lenfil: 145 147756 █████████▊ |█████████████████▍ | hs/space-bslc8-acc-1: 2 3192 ▏ |▍ | hs/space-bslc8-acc-2: 2 3192 ▏ |▍ | hs/space-bslc8-acc-3: 2 3196 ▏ |▍ | hs/space-bslc8-chunk-1: 64 65192 ████▎ |███████▋ | hs/space-bslc8-chunk-2: 64 65192 ████▎ |███████▋ | hs/space-bslc8-chunk-3: 74 75240 █████ |████████▉ | hs/space-bslc8-chunk-4: 75 76188 █████ |█████████ | hs/space-bslc8-count: 78 79204 █████▎ |█████████▍ | hs/space-bslc8-foldl: 85 85868 █████▊ |██████████▏ | hs/space-bslc8-foldlx-1: 85 85864 █████▊ |██████████▏ | hs/space-bslc8-foldlx-2: 85 85868 █████▊ |██████████▏ | hs/space-bslc8-foldr-1: 297 169700 ████████████████████|████████████████████| hs/space-bslc8-foldr-2: 297 169700 ████████████████████|████████████████████| hs/space-bslc8-lenfil-1: 115 109404 ███████▊ |████████████▉ | hs/space-bslc8-lenfil-2: 115 109404 ███████▊ |████████████▉ | hs/space-bsl---foldlx: 85 85836 █████▊ |██████████▏ | hs/space-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1904 ▏ |▎ | hs/space-xxxxx-foldl: 1 1900 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1892 ▏ |▎ | hand/byte-bs----acc-a: 1 147764 ▏ |█████████████████▍ | hand/byte-bs----acc-b: 1 147764 ▏ |█████████████████▍ | hand/byte-bs----acc-c: 1 147760 ▏ |█████████████████▍ | hand/byte-bs----acc-d: 1 147760 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-a: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-b: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-c: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-d: 1 147756 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-e: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-f: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-g: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-h: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-i: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-j: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-k: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-l: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-m: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-n: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-o: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-p: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-q: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-r: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-s: 1 147744 ▏ |█████████████████▍ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.9-bs0902.tgz ------- hs/byte-bs----acc: 1 147732 ▏ |██████████████████▌ | hs/byte-bs----foldlx: 1 147736 ▏ |██████████████████▌ | hs/byte-bs----foldrx: 1 147740 ▏ |██████████████████▌ | hs/byte-bsl---acc: 2 3184 ▎ |▍ | hs/byte-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147740 ▏ |██████████████████▌ | hs/space-bs-c8-count: 1 147756 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-1: 1 147744 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-2: 1 147748 ▏ |██████████████████▌ | hs/space-bs-c8-foldrx: 1 147748 ▏ |██████████████████▌ | hs/space-bs-c8-lenfil: 21 167300 ███ |█████████████████████| hs/space-bslc8-acc-1: 2 3228 ▎ |▍ | hs/space-bslc8-acc-2: 2 3228 ▎ |▍ | hs/space-bslc8-acc-3: 2 3232 ▎ |▍ | hs/space-bslc8-chunk-1: 65 66216 █████████▏ |████████▎ | hs/space-bslc8-chunk-2: 65 66216 █████████▏ |████████▎ | hs/space-bslc8-chunk-3: 65 66208 █████████▏ |████████▎ | hs/space-bslc8-chunk-4: 78 79232 ███████████ |██████████ | hs/space-bslc8-count: 81 82248 ███████████▍ |██████████▍ | hs/space-bslc8-foldl: 78 79224 ███████████ |██████████ | hs/space-bslc8-foldlx-1: 78 79224 ███████████ |██████████ | hs/space-bslc8-foldlx-2: 78 79224 ███████████ |██████████ | hs/space-bslc8-foldr-1: 149 149704 █████████████████████|██████████████████▊ | hs/space-bslc8-foldr-2: 149 149708 █████████████████████|██████████████████▊ | hs/space-bslc8-lenfil-1: 81 82324 ███████████▍ |██████████▍ | hs/space-bslc8-lenfil-2: 81 82328 ███████████▍ |██████████▍ | hs/space-bsl---foldlx: 78 79184 ███████████ |██████████ | hs/space-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/space-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1892 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-head.tgz ------- hs/byte-bs----acc: 1 147772 ▏ |██████████████████▎ | hs/byte-bs----foldlx: 145 147796 ██████████▎ |██████████████████▎ | hs/byte-bs----foldrx: 1 147772 ▏ |██████████████████▎ | hs/byte-bsl---acc: 2 3212 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1900 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1900 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147752 ▏ |██████████████████▎ | hs/space-bs-c8-count: 1 147764 ▏ |██████████████████▎ | hs/space-bs-c8-foldlx-1: 145 147796 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldlx-2: 145 147796 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldrx: 1 147760 ▏ |██████████████████▎ | hs/space-bs-c8-lenfil: 145 147788 ██████████▎ |██████████████████▎ | hs/space-bslc8-acc-1: 2 3180 ▏ |▍ | hs/space-bslc8-acc-2: 2 3180 ▏ |▍ | hs/space-bslc8-acc-3: 2 3172 ▏ |▍ | hs/space-bslc8-chunk-1: 64 65204 ████▌ |████████▏ | hs/space-bslc8-chunk-2: 64 65204 ████▌ |████████▏ | hs/space-bslc8-chunk-3: 74 75216 █████▎ |█████████▎ | hs/space-bslc8-chunk-4: 75 76176 █████▎ |█████████▍ | hs/space-bslc8-count: 78 79220 █████▌ |█████████▊ | hs/space-bslc8-foldl: 85 85852 ██████ |██████████▋ | hs/space-bslc8-foldlx-1: 85 85852 ██████ |██████████▋ | hs/space-bslc8-foldlx-2: 85 85852 ██████ |██████████▋ | hs/space-bslc8-foldr-1: 297 169680 █████████████████████|█████████████████████| hs/space-bslc8-foldr-2: 297 169676 █████████████████████|█████████████████████| hs/space-bslc8-lenfil-1: 115 109388 ████████▏ |█████████████▌ | hs/space-bslc8-lenfil-2: 115 109392 ████████▏ |█████████████▌ | hs/space-bsl---foldlx: 85 85864 ██████ |██████████▋ | hs/space-xxxxx-acc-1: 1 1900 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1904 ▏ |▎ | hs/space-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1888 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-head-bs0902.tgz ------- hs/byte-bs----acc: 1 147780 ▏ |██████████████████▌ | hs/byte-bs----foldlx: 1 147780 ▏ |██████████████████▌ | hs/byte-bs----foldrx: 1 147784 ▏ |██████████████████▌ | hs/byte-bsl---acc: 2 3216 ▎ |▍ | hs/byte-xxxxx-acc-1: 1 1900 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1896 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1896 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147752 ▏ |██████████████████▌ | hs/space-bs-c8-count: 1 147776 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-1: 1 147752 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-2: 1 147756 ▏ |██████████████████▌ | hs/space-bs-c8-foldrx: 1 147756 ▏ |██████████████████▌ | hs/space-bs-c8-lenfil: 21 167320 ███ |█████████████████████| hs/space-bslc8-acc-1: 2 3196 ▎ |▍ | hs/space-bslc8-acc-2: 2 3200 ▎ |▍ | hs/space-bslc8-acc-3: 2 3208 ▎ |▍ | hs/space-bslc8-chunk-1: 65 66224 █████████▏ |████████▎ | hs/space-bslc8-chunk-2: 65 66228 █████████▏ |████████▎ | hs/space-bslc8-chunk-3: 65 66224 █████████▏ |████████▎ | hs/space-bslc8-chunk-4: 78 79204 ███████████ |██████████ | hs/space-bslc8-count: 81 82224 ███████████▍ |██████████▍ | hs/space-bslc8-foldl: 78 79196 ███████████ |██████████ | hs/space-bslc8-foldlx-1: 78 79196 ███████████ |██████████ | hs/space-bslc8-foldlx-2: 78 79196 ███████████ |██████████ | hs/space-bslc8-foldr-1: 149 149680 █████████████████████|██████████████████▊ | hs/space-bslc8-foldr-2: 149 149676 █████████████████████|██████████████████▊ | hs/space-bslc8-lenfil-1: 81 82304 ███████████▍ |██████████▍ | hs/space-bslc8-lenfil-2: 81 82300 ███████████▍ |██████████▍ | hs/space-bsl---foldlx: 78 79208 ███████████ |██████████ | hs/space-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/space-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1888 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - -

1 0

[12/16] SBM: Graphs for 7 ghc/bytestring combinations on a 2GHz Athlon64 300+
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This report combines the measurements from 7 ghc/bytestring combinations on a single machine, namely: ghc 6.6.1 ghc 6.8.2 ghc 6.8.2 + bytestring 0.9.0.2 ghc 6.9.20071119 ghc 6.9.20071119 + bytestring 0.9.0.2 ghc head-as-of-noon-2007-12-19 ghc head-as-of-noon-2007-12-19 + bytestring 0.9.0.2 This is the command I ran to generate the report: RESCALE=1 EXCLUDE='(^hand/|^c/)' \ tools/merge.pl \ ghc-thorough-6.6.1.tgz \ ghc-thorough-6.8.2.tgz ghc-thorough-6.8.2-bs0902.tgz \ ghc-thorough-6.9.tgz ghc-thorough-6.9-bs0902.tgz \ ghc-thorough-head.tgz ghc-thorough-head-bs0902.tgz \ > mergerep-amdy.txt We can readily see that there are some memory leaks with ghc 6.9.x. We can also see that Don Stewart's fix in bytestring 0.9.0.2 actually fixed things for hs/space-bslc8-lenfil-1 on ghc 6.8.2 but only improved things slightly for 6.9.x. It also makes things worse for hs/space-bs-c8-lenfil on 6.8.2 and both the 6.9.x's. Lazy bytestrings are slow, when used "manually", but quite acceptable, memory- wise. On the other hand, they cost dearly in memory on 6.9.x unless one does it all "by hand". Good old native strings seem to be the most predictable of the bunch but they are a bit slow. Surprisingly, they are not always /the/ slowest option measured! hs/space-bslc8-acc-1 and hs/space-bslc8-acc-2 are actually slower on ghc 6.6.1 than the native strings are in all combinations except hs/space-xxxxx-lenfil. -Peter PS: Note that this is the same report that I sent yesterday, except that it had the order of some of the benchmarks slightly wrong. charybdis ghc 6.6.1 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.8.2 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.8.2 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX=-bs0.9.0.2 charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX=-bs0.9.0.2 charybdis ghc 6.9.20071217 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071217 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX=-bs0.9.0.2 Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 1.020 40‰ 0.0 █████▏ | -- 0.703 5‰ 0.4 ███▌ | -- 0.702 7‰ 0.3 ███▌ | -- 0.705 7‰ 0.1 ███▋ | -- 0.712 3‰ 0.1 ███▋ | -- 0.706 7‰ 0.1 ███▋ | -- 0.707 7‰ 0.9 ███▋ | hs/byte-bs----foldlx: 0.789 3‰ 0.3 ████ | -- 0.993 2‰ 0.2 █████ | -- 1.102 1‰ 0.2 █████▋ | -- 1.002 1‰ 0.5 █████▏ | -- 1.112 1‰ 0.3 █████▋ | -- 1.024 2‰ 0.2 █████▏ | -- 1.111 1‰ 0.1 █████▋ | hs/byte-bs----foldrx: 0.813 3‰ 0.1 ████▏ | -- 1.102 2‰ 0.3 █████▋ | -- 1.100 1‰ 0.1 █████▋ | -- 1.112 2‰ 0.1 █████▋ | -- 1.114 1‰ 0.4 █████▋ | -- 1.113 2‰ 0.5 █████▋ | -- 1.112 1‰ 0.2 █████▋ | hs/byte-bsl---acc: 3.599 13‰ 0.0 ██████████████████▎ | -- 2.609 17‰ 0.0 █████████████▎ | -- 2.560 15‰ 0.1 █████████████ | -- 2.595 14‰ 0.1 █████████████▏ | -- 2.574 16‰ 0.1 █████████████ | -- 2.613 12‰ 0.2 █████████████▎ | -- 2.656 44‰ 0.1 █████████████▌ | hs/byte-xxxxx-acc-1: 4.606 5‰ 0.0 ███████████████████████▍ | -- 4.730 3‰ 0.0 ████████████████████████ | -- 4.722 3‰ 0.1 ███████████████████████▉ | -- 4.436 5‰ 0.0 ██████████████████████▌ | -- 4.500 6‰ 0.1 ██████████████████████▊ | -- 4.462 4‰ 0.1 ██████████████████████▋ | -- 4.499 7‰ 0.1 ██████████████████████▊ | hs/byte-xxxxx-acc-2: 4.595 5‰ 0.0 ███████████████████████▎ | -- 4.704 1‰ 0.0 ███████████████████████▉ | -- 4.722 6‰ 0.0 ███████████████████████▉ | -- 4.473 8‰ 0.0 ██████████████████████▋ | -- 4.493 5‰ 0.0 ██████████████████████▊ | -- 4.477 3‰ 0.1 ██████████████████████▋ | -- 4.446 2‰ 0.0 ██████████████████████▌ | hs/byte-xxxxx-foldl: 4.566 9‰ 0.0 ███████████████████████▏ | -- 4.754 6‰ 0.1 ████████████████████████▏ | -- 4.735 7‰ 0.0 ████████████████████████ | -- 4.412 10‰ 0.0 ██████████████████████▍ | -- 4.486 3‰ 0.0 ██████████████████████▊ | -- 4.473 3‰ 0.0 ██████████████████████▋ | -- 4.511 10‰ 0.1 ██████████████████████▉ | Memory: Peak ------- KB hs/byte-bs----acc: 147448 ████████████████████████████████████████ | -- 147500 ████████████████████████████████████████ | -- 147616 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | -- 147732 ████████████████████████████████████████ | -- 147772 ████████████████████████████████████████ | -- 147780 ████████████████████████████████████████ | hs/byte-bs----foldlx: 147468 ████████████████████████████████████████ | -- 147528 ████████████████████████████████████████ | -- 147624 ████████████████████████████████████████ | -- 147748 ████████████████████████████████████████ | -- 147736 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147780 ████████████████████████████████████████ | hs/byte-bs----foldrx: 147452 ████████████████████████████████████████ | -- 147500 ████████████████████████████████████████ | -- 147624 ████████████████████████████████████████ | -- 147744 ████████████████████████████████████████ | -- 147740 ████████████████████████████████████████ | -- 147772 ████████████████████████████████████████ | -- 147784 ████████████████████████████████████████ | hs/byte-bsl---acc: 2872 ▊ | -- 2920 ▊ | -- 3040 ▉ | -- 3172 ▉ | -- 3184 ▉ | -- 3212 ▉ | -- 3216 ▉ | hs/byte-xxxxx-acc-1: 1572 ▍ | -- 1632 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1900 ▌ | hs/byte-xxxxx-acc-2: 1568 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1900 ▌ | -- 1896 ▌ | hs/byte-xxxxx-foldl: 1572 ▍ | -- 1632 ▌ | -- 1636 ▌ | -- 1900 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1896 ▌ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 1.287 20‰ 0.2 ██████▌ | -- 1.131 2‰ 0.1 █████▊ | -- 1.132 2‰ 0.3 █████▊ | -- 1.145 1‰ 0.2 █████▊ | -- 1.150 1‰ 0.2 █████▉ | -- 1.147 2‰ 0.1 █████▊ | -- 1.147 2‰ 0.2 █████▊ | hs/space-bs-c8-count: 0.770 2‰ 0.0 ███▉ | -- 0.711 1‰ 0.4 ███▋ | -- 0.513 2‰ 0.2 ██▋ | -- 0.521 1‰ 0.2 ██▋ | -- 0.523 3‰ 0.2 ██▋ | -- 0.522 5‰ 0.2 ██▋ | -- 0.523 3‰ 0.6 ██▋ | hs/space-bs-c8-foldlx-1: 1.272 1‰ 0.2 ██████▌ | -- 1.205 2‰ 0.2 ██████▏ | -- 1.164 3‰ 0.0 █████▉ | -- 1.221 1‰ 0.1 ██████▏ | -- 1.176 1‰ 0.3 ██████ | -- 1.259 1‰ 0.1 ██████▍ | -- 1.176 1‰ 0.2 ██████ | hs/space-bs-c8-foldlx-2: 1.270 1‰ 0.2 ██████▍ | -- 1.207 1‰ 0.2 ██████▏ | -- 1.165 2‰ 0.1 █████▉ | -- 1.219 2‰ 0.2 ██████▏ | -- 1.176 1‰ 0.3 ██████ | -- 1.258 1‰ 0.2 ██████▍ | -- 1.177 1‰ 0.2 ██████ | hs/space-bs-c8-foldrx: 1.064 2‰ 0.1 █████▍ | -- 1.168 2‰ 0.2 █████▉ | -- 1.164 1‰ 0.3 █████▉ | -- 1.172 2‰ 0.3 █████▉ | -- 1.177 1‰ 0.2 ██████ | -- 1.176 1‰ 0.2 ██████ | -- 1.176 1‰ 0.1 ██████ | hs/space-bs-c8-lenfil: 1.271 1‰ 0.1 ██████▍ | -- 1.218 1‰ 0.2 ██████▏ | -- 0.541 3‰ 0.2 ██▊ | -- 1.223 1‰ 0.2 ██████▎ | -- 0.549 1‰ 0.4 ██▊ | -- 1.225 1‰ 0.2 ██████▎ | -- 0.551 5‰ 0.4 ██▊ | hs/space-bslc8-acc-1: 6.367 17‰ 0.2 ████████████████████████████████▎ | -- 3.396 7‰ 0.1 █████████████████▎ | -- 3.508 4‰ 0.1 █████████████████▊ | -- 3.388 10‰ 0.1 █████████████████▏ | -- 3.441 10‰ 0.1 █████████████████▍ | -- 3.438 3‰ 0.0 █████████████████▍ | -- 3.434 13‰ 0.1 █████████████████▍ | hs/space-bslc8-acc-2: 6.150 21‰ 0.0 ███████████████████████████████▏ | -- 3.470 5‰ 0.0 █████████████████▋ | -- 3.439 10‰ 0.1 █████████████████▍ | -- 3.386 5‰ 0.1 █████████████████▏ | -- 3.479 11‰ 0.1 █████████████████▋ | -- 3.467 6‰ 0.1 █████████████████▌ | -- 3.470 10‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-3: 4.041 21‰ 0.1 ████████████████████▌ | -- 2.959 13‰ 0.0 ███████████████ | -- 2.920 16‰ 0.1 ██████████████▊ | -- 2.832 4‰ 0.1 ██████████████▍ | -- 2.925 14‰ 0.1 ██████████████▊ | -- 2.987 9‰ 0.0 ███████████████▏ | -- 2.996 20‰ 0.1 ███████████████▏ | hs/space-bslc8-chunk-1: n/a -- 1.054 0‰ 0.3 █████▍ | -- 1.053 1‰ 0.1 █████▍ | -- 1.094 0‰ 0.3 █████▌ | -- 1.088 0‰ 0.0 █████▌ | -- 1.096 1‰ 0.1 █████▌ | -- 1.089 0‰ 0.6 █████▌ | hs/space-bslc8-chunk-2: n/a -- 1.055 0‰ 0.3 █████▍ | -- 1.053 0‰ 0.1 █████▍ | -- 1.094 1‰ 0.3 █████▌ | -- 1.088 1‰ 0.2 █████▌ | -- 1.096 1‰ 0.2 █████▌ | -- 1.089 0‰ 0.1 █████▌ | hs/space-bslc8-chunk-3: n/a -- 1.190 4‰ 0.2 ██████ | -- 1.199 1‰ 0.3 ██████▏ | -- 1.087 1‰ 0.3 █████▌ | -- 1.051 1‰ 0.4 █████▍ | -- 1.077 1‰ 0.1 █████▌ | -- 1.053 1‰ 0.2 █████▍ | hs/space-bslc8-chunk-4: n/a -- 1.054 0‰ 0.2 █████▍ | -- 1.054 0‰ 0.2 █████▍ | -- 1.099 0‰ 0.3 █████▋ | -- 1.097 1‰ 0.1 █████▌ | -- 1.094 1‰ 0.2 █████▌ | -- 1.096 0‰ 0.0 █████▌ | hs/space-bslc8-count: 0.713 1‰ 0.3 ███▋ | -- 0.735 1‰ 0.4 ███▊ | -- 0.431 0‰ 0.7 ██▏ | -- 0.477 2‰ 0.4 ██▍ | -- 0.482 1‰ 0.4 ██▌ | -- 0.479 1‰ 0.6 ██▍ | -- 0.482 2‰ 0.4 ██▌ | hs/space-bslc8-foldl: 1.431 1‰ 0.3 ███████▎ | -- 1.398 1‰ 0.2 ███████▏ | -- 1.197 1‰ 0.2 ██████ | -- 1.458 1‰ 0.3 ███████▍ | -- 1.059 0‰ 0.0 █████▍ | -- 1.459 1‰ 0.1 ███████▍ | -- 1.061 0‰ 0.4 █████▍ | hs/space-bslc8-foldlx-1: 1.430 1‰ 0.1 ███████▎ | -- 1.400 1‰ 0.1 ███████▏ | -- 1.197 1‰ 0.0 ██████ | -- 1.458 0‰ 0.1 ███████▍ | -- 1.060 1‰ 0.0 █████▍ | -- 1.457 1‰ 0.1 ███████▍ | -- 1.060 1‰ 0.2 █████▍ | hs/space-bslc8-foldlx-2: 1.431 1‰ 0.1 ███████▎ | -- 1.399 1‰ 0.1 ███████▏ | -- 1.196 1‰ 0.1 ██████ | -- 1.457 1‰ 0.1 ███████▍ | -- 1.060 1‰ 0.3 █████▍ | -- 1.458 1‰ 0.1 ███████▍ | -- 1.060 1‰ 0.0 █████▍ | hs/space-bslc8-foldr-1: 1.748 1‰ 0.0 ████████▉ | -- 1.579 2‰ 0.1 ████████ | -- 1.220 2‰ 0.2 ██████▏ | -- 1.312 1‰ 0.2 ██████▋ | -- 1.213 1‰ 0.2 ██████▏ | -- 1.325 4‰ 0.2 ██████▊ | -- 1.213 2‰ 0.2 ██████▏ | hs/space-bslc8-foldr-2: 1.745 1‰ 0.1 ████████▉ | -- 1.579 2‰ 0.1 ████████ | -- 1.220 1‰ 0.2 ██████▏ | -- 1.315 1‰ 0.2 ██████▋ | -- 1.210 1‰ 0.2 ██████▏ | -- 1.317 4‰ 0.2 ██████▋ | -- 1.213 2‰ 0.2 ██████▏ | hs/space-bslc8-lenfil-1: 1.521 7‰ 0.2 ███████▊ | -- 1.295 1‰ 0.2 ██████▌ | -- 0.429 0‰ 0.2 ██▏ | -- 1.323 1‰ 0.1 ██████▊ | -- 0.483 2‰ 0.4 ██▌ | -- 1.327 1‰ 0.2 ██████▊ | -- 0.482 2‰ 0.4 ██▌ | hs/space-bslc8-lenfil-2: 1.532 6‰ 0.1 ███████▊ | -- 1.296 0‰ 0.0 ██████▌ | -- 0.429 0‰ 0.2 ██▏ | -- 1.325 1‰ 0.1 ██████▊ | -- 0.483 1‰ 0.6 ██▌ | -- 1.327 1‰ 0.2 ██████▊ | -- 0.482 2‰ 0.4 ██▌ | hs/space-bsl---foldlx: 1.288 1‰ 0.2 ██████▌ | -- 1.333 1‰ 0.0 ██████▊ | -- 1.168 1‰ 0.2 █████▉ | -- 1.391 1‰ 0.1 ███████ | -- 1.006 1‰ 0.3 █████▏ | -- 1.395 1‰ 0.1 ███████▏ | -- 1.008 1‰ 0.2 █████▏ | hs/space-xxxxx-acc-1: 5.036 7‰ 0.1 █████████████████████████▌ | -- 5.381 6‰ 0.0 ███████████████████████████▎ | -- 5.358 8‰ 0.1 ███████████████████████████▏ | -- 4.991 1‰ 0.1 █████████████████████████▎ | -- 5.047 7‰ 0.1 █████████████████████████▌ | -- 5.051 3‰ 0.1 █████████████████████████▋ | -- 5.049 4‰ 0.1 █████████████████████████▋ | hs/space-xxxxx-acc-2: 5.009 7‰ 0.1 █████████████████████████▍ | -- 5.437 4‰ 0.1 ███████████████████████████▌ | -- 5.406 2‰ 0.0 ███████████████████████████▍ | -- 4.985 1‰ 0.0 █████████████████████████▎ | -- 5.057 4‰ 0.1 █████████████████████████▋ | -- 5.048 6‰ 0.1 █████████████████████████▌ | -- 5.089 6‰ 0.0 █████████████████████████▊ | hs/space-xxxxx-foldl: 5.024 5‰ 0.0 █████████████████████████▌ | -- 5.371 9‰ 0.0 ███████████████████████████▎ | -- 5.301 9‰ 0.1 ██████████████████████████▉ | -- 5.035 2‰ 0.1 █████████████████████████▌ | -- 5.037 3‰ 0.0 █████████████████████████▌ | -- 5.046 4‰ 0.0 █████████████████████████▌ | -- 5.042 4‰ 0.0 █████████████████████████▌ | hs/space-xxxxx-lenfil: 7.042 4‰ 0.0 ███████████████████████████████████▋ | -- 7.505 3‰ 0.0 ██████████████████████████████████████| -- 7.498 3‰ 0.0 ██████████████████████████████████████| -- 7.106 8‰ 0.1 ████████████████████████████████████ | -- 7.092 3‰ 0.1 ███████████████████████████████████▉ | -- 7.083 3‰ 0.0 ███████████████████████████████████▉ | -- 7.121 5‰ 0.1 ████████████████████████████████████ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147448 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147736 ████████████████████████████████████████ | -- 147740 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | hs/space-bs-c8-count: 147448 ████████████████████████████████████████ | -- 147500 ████████████████████████████████████████ | -- 147612 ████████████████████████████████████████ | -- 147744 ████████████████████████████████████████ | -- 147756 ████████████████████████████████████████ | -- 147764 ████████████████████████████████████████ | -- 147776 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147468 ████████████████████████████████████████ | -- 147524 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147756 ████████████████████████████████████████ | -- 147744 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147752 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-2: 147468 ████████████████████████████████████████ | -- 147520 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | -- 147748 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147756 ████████████████████████████████████████ | hs/space-bs-c8-foldrx: 147448 ████████████████████████████████████████ | -- 147496 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147732 ████████████████████████████████████████ | -- 147748 ████████████████████████████████████████ | -- 147760 ████████████████████████████████████████ | -- 147756 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147464 ████████████████████████████████████████ | -- 147524 ████████████████████████████████████████ | -- 167176 █████████████████████████████████████████████▍| -- 147756 ████████████████████████████████████████ | -- 167300 █████████████████████████████████████████████▍| -- 147788 ████████████████████████████████████████ | -- 167320 █████████████████████████████████████████████▍| hs/space-bslc8-acc-1: 2872 ▊ | -- 2920 ▊ | -- 3048 ▉ | -- 3192 ▉ | -- 3228 ▉ | -- 3180 ▉ | -- 3196 ▉ | hs/space-bslc8-acc-2: 2872 ▊ | -- 2924 ▊ | -- 3048 ▉ | -- 3192 ▉ | -- 3228 ▉ | -- 3180 ▉ | -- 3200 ▉ | hs/space-bslc8-acc-3: 2868 ▊ | -- 2924 ▊ | -- 3052 ▉ | -- 3196 ▉ | -- 3232 ▉ | -- 3172 ▉ | -- 3208 ▉ | hs/space-bslc8-chunk-1: n/a -- 1512 ▍ | -- 1636 ▌ | -- 65192 █████████████████▋ | -- 66216 ██████████████████ | -- 65204 █████████████████▋ | -- 66224 ██████████████████ | hs/space-bslc8-chunk-2: n/a -- 1512 ▍ | -- 1640 ▌ | -- 65192 █████████████████▋ | -- 66216 ██████████████████ | -- 65204 █████████████████▋ | -- 66228 ██████████████████ | hs/space-bslc8-chunk-3: n/a -- 1548 ▍ | -- 1644 ▌ | -- 75240 ████████████████████▍ | -- 66208 ██████████████████ | -- 75216 ████████████████████▍ | -- 66224 ██████████████████ | hs/space-bslc8-chunk-4: n/a -- 1512 ▍ | -- 1636 ▌ | -- 76188 ████████████████████▋ | -- 79232 █████████████████████▌ | -- 76176 ████████████████████▋ | -- 79204 █████████████████████▌ | hs/space-bslc8-count: 1476 ▍ | -- 1520 ▍ | -- 1656 ▌ | -- 79204 █████████████████████▌ | -- 82248 ██████████████████████▎ | -- 79220 █████████████████████▌ | -- 82224 ██████████████████████▎ | hs/space-bslc8-foldl: 1552 ▍ | -- 1620 ▌ | -- 1632 ▌ | -- 85868 ███████████████████████▎ | -- 79224 █████████████████████▌ | -- 85852 ███████████████████████▎ | -- 79196 █████████████████████▌ | hs/space-bslc8-foldlx-1: 1556 ▍ | -- 1620 ▌ | -- 1636 ▌ | -- 85864 ███████████████████████▎ | -- 79224 █████████████████████▌ | -- 85852 ███████████████████████▎ | -- 79196 █████████████████████▌ | hs/space-bslc8-foldlx-2: 1552 ▍ | -- 1616 ▌ | -- 1636 ▌ | -- 85868 ███████████████████████▎ | -- 79224 █████████████████████▌ | -- 85852 ███████████████████████▎ | -- 79196 █████████████████████▌ | hs/space-bslc8-foldr-1: 169480 ██████████████████████████████████████████████| -- 169544 ██████████████████████████████████████████████| -- 149076 ████████████████████████████████████████▍ | -- 169700 ██████████████████████████████████████████████| -- 149704 ████████████████████████████████████████▋ | -- 169680 ██████████████████████████████████████████████| -- 149680 ████████████████████████████████████████▋ | hs/space-bslc8-foldr-2: 169480 ██████████████████████████████████████████████| -- 169544 ██████████████████████████████████████████████| -- 149076 ████████████████████████████████████████▍ | -- 169700 ██████████████████████████████████████████████| -- 149708 ████████████████████████████████████████▋ | -- 169676 ██████████████████████████████████████████████| -- 149676 ████████████████████████████████████████▋ | hs/space-bslc8-lenfil-1: 38632 ██████████▌ | -- 38644 ██████████▌ | -- 1940 ▌ | -- 109404 █████████████████████████████▋ | -- 82324 ██████████████████████▍ | -- 109388 █████████████████████████████▋ | -- 82304 ██████████████████████▎ | hs/space-bslc8-lenfil-2: 38628 ██████████▌ | -- 38648 ██████████▌ | -- 1940 ▌ | -- 109404 █████████████████████████████▋ | -- 82328 ██████████████████████▍ | -- 109392 █████████████████████████████▋ | -- 82300 ██████████████████████▎ | hs/space-bsl---foldlx: 1560 ▍ | -- 1616 ▌ | -- 1620 ▌ | -- 85836 ███████████████████████▎ | -- 79184 █████████████████████▌ | -- 85864 ███████████████████████▎ | -- 79208 █████████████████████▌ | hs/space-xxxxx-acc-1: 1568 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1904 ▌ | hs/space-xxxxx-acc-2: 1572 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1904 ▌ | -- 1900 ▌ | hs/space-xxxxx-foldl: 1572 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1900 ▌ | -- 1904 ▌ | -- 1904 ▌ | -- 1904 ▌ | hs/space-xxxxx-lenfil: 1556 ▍ | -- 1608 ▍ | -- 1604 ▍ | -- 1892 ▌ | -- 1892 ▌ | -- 1888 ▌ | -- 1888 ▌ |

1 0

[11/16] SBM: Graphs for hand-tweaked assembly benchmarks
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This report compares the hand-tweaked assembly programs with the original untweaked programs on two vastly different microarchitectures. This is the command I ran to generate the report: EXCLUDE='(xxxx|-bsl|chunk|count|acc-[23]|fold|lenfil|^c/)' \ tools/merge.pl \ ghc-armada-thorough-6.9.tgz \ ghc-thorough-6.9.tgz \ > xx I cut out the memory sections manually since we've already seen them and inserted a few newlines for grouping purposes. The first one should note is that not all tweaks are better than the originals! The second is that the sequence of tweaks is not quite monotonically decreasing in run-time. The improvements don't really start until -e on the Athlon64 and -f on both. Not until then have the load pressure been sufficiently relieved on the L1 cache that the code actually runs faster. Note also how the two microarchitectures seem to have plateaus in different places. The Athlon64 seems to have the number 3 built into its silicon (efg, jkl, mno) which fits very well with what we know about it from AMD's documentation (the front end splits the instructions up into smaller pieces which then get distributed to three different "pipelines", each with its own out-of-order execution engine). The Pentium III seems to have trouble with the simple MMX code but does very well with the more advanced MMX code that keeps 8 space counters in a single MMX register for many iterations. The code I used to add those counters horizontally is the same in both -q and -r. Perhaps operations on both MMX and normal registers are slow? Loop unrolling (-s) doesn't seem to matter, in this case. -Peter ls-search ghc 6.9.20071119 Pentium III (Coppermine) 596.932 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 3.274 1‰ 0.1 ███████████████████████████ | -- 0.705 7‰ 0.1 █████████████████████▋ | hand/byte-bs----acc-a: 3.511 1‰ 0.0 ████████████████████████████▉ | -- 0.639 2‰ 0.2 ███████████████████▋ | hand/byte-bs----acc-b: 1.998 2‰ 0.1 ████████████████▌ | -- 0.414 2‰ 0.5 ████████████▊ | hand/byte-bs----acc-c: 1.876 2‰ 0.1 ███████████████▌ | -- 0.414 3‰ 0.2 ████████████▊ | hand/byte-bs----acc-d: 1.876 1‰ 0.1 ███████████████▌ | -- 0.415 3‰ 0.2 ████████████▊ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 4.318 1‰ 0.0 ███████████████████████████████████▋ | -- 1.145 1‰ 0.2 ███████████████████████████████████▏ | hand/space-bs-c8-acc-1-a: 4.318 1‰ 0.0 ███████████████████████████████████▋ | -- 1.177 2‰ 0.3 ████████████████████████████████████▏| hand/space-bs-c8-acc-1-b: 4.331 1‰ 0.0 ███████████████████████████████████▋ | -- 1.104 1‰ 0.2 █████████████████████████████████▉ | hand/space-bs-c8-acc-1-c: 4.492 1‰ 0.1 █████████████████████████████████████| -- 1.207 1‰ 0.3 █████████████████████████████████████| hand/space-bs-c8-acc-1-d: 4.354 1‰ 0.0 ███████████████████████████████████▉ | -- 1.191 1‰ 0.2 ████████████████████████████████████▌| hand/space-bs-c8-acc-1-e: 4.424 0‰ 0.1 ████████████████████████████████████▌| -- 0.937 1‰ 0.2 ████████████████████████████▊ | hand/space-bs-c8-acc-1-f: 4.164 1‰ 0.0 ██████████████████████████████████▎ | -- 0.921 1‰ 0.2 ████████████████████████████▎ | hand/space-bs-c8-acc-1-g: 4.309 1‰ 0.1 ███████████████████████████████████▌ | -- 0.927 2‰ 0.4 ████████████████████████████▍ | hand/space-bs-c8-acc-1-h: 4.202 1‰ 0.1 ██████████████████████████████████▋ | -- 0.886 2‰ 0.2 ███████████████████████████▏ | hand/space-bs-c8-acc-1-i: 3.820 1‰ 0.1 ███████████████████████████████▌ | -- 0.803 3‰ 0.4 ████████████████████████▋ | hand/space-bs-c8-acc-1-j: 3.472 1‰ 0.0 ████████████████████████████▋ | -- 0.706 2‰ 0.1 █████████████████████▋ | hand/space-bs-c8-acc-1-k: 3.474 1‰ 0.0 ████████████████████████████▋ | -- 0.705 1‰ 0.0 █████████████████████▋ | hand/space-bs-c8-acc-1-l: 3.498 1‰ 0.1 ████████████████████████████▉ | -- 0.710 2‰ 0.1 █████████████████████▊ | hand/space-bs-c8-acc-1-m: 3.397 1‰ 0.1 ████████████████████████████ | -- 0.642 6‰ 0.3 ███████████████████▋ | hand/space-bs-c8-acc-1-n: 3.373 1‰ 0.0 ███████████████████████████▊ | -- 0.636 4‰ 0.5 ███████████████████▌ | hand/space-bs-c8-acc-1-o: 3.118 1‰ 0.1 █████████████████████████▋ | -- 0.626 2‰ 0.0 ███████████████████▎ | hand/space-bs-c8-acc-1-p: 2.935 2‰ 0.0 ████████████████████████▏ | -- 0.565 3‰ 0.4 █████████████████▍ | hand/space-bs-c8-acc-1-q: 3.477 1‰ 0.1 ████████████████████████████▋ | -- 0.418 6‰ 0.7 ████████████▉ | hand/space-bs-c8-acc-1-r: 1.674 1‰ 0.1 █████████████▊ | -- 0.334 5‰ 0.6 ██████████▎ | hand/space-bs-c8-acc-1-s: 1.627 1‰ 0.2 █████████████▍ | -- 0.335 4‰ 0.9 ██████████▎ |

1 0

[10/16] SBM: Graphs for 6.9.x across four cpus
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This is what you get if you merge the previous four reports (and filter out the hand-tweaked assembly benchmarks). I generated the report with the following command: EXCLUDE='^hand/' \ tools/merge.pl \ ghc-armada-thorough-6.9.tgz \ ghc-fischer-thorough-6.9.tgz \ ghc-albatross-thorough-6.9.tgz \ ghc-thorough-6.9.tgz \ > xx This graph shows the memory usage to be (almost) exactly the same. The difference can easily be explained with slightly different versions of the C library (there was also a security update or two from Ubuntu in the last few days - the kernel was definitely updated and I think the C library was too). We are also talking about two versions of Ubuntu and one (old) version of SuSE. And we are talking about three not quite identical versions of ghc. The fischer and albatross runs were made before the inclusion of the three benchmarks from Don Stewart (hs/space-bs-c8-count, hs/space-bslc8-count, and hs/space-bslc8-chunk-4) so they are a couple of holes in the graphs. The speed pattern is more fun. It makes no sense to compare absolute times here so the graphs were not rescaled. One would naïvely expect the bars to be about the same when taken in groups of four but that really turns out to be far from the case! The explanation is most likely that we are looking at four very different microarchitectures. ********** This should hammer home the point that benchmarking on any single machine isn't enough! ********** Two probably aren't enough either... -Peter ls-search ghc 6.9.20071119 Pentium III (Coppermine) 596.932 MHz TESTKIND=THOROUGH SUFFIX= Fischer's machine ghc 6.9.20071124 AMD Duron(tm) processor 1200.089 MHz TESTKIND=THOROUGH SUFFIX= albatross ghc 6.9.20071208 (or thereabouts) Genuine Intel(R) CPU T2300 @ 1.66GHz (Core Duo) 1667.000 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 3.274 1‰ 0.1 ███▏ | -- 1.892 21‰ 0.4 ██▊ | -- 0.918 4‰ 0.1 █████▍ | -- 0.705 7‰ 0.1 ███▋ | hs/byte-bs----foldlx: 4.027 0‰ 0.0 ███▊ | -- 2.258 3‰ 0.1 ███▎ | -- 1.014 2‰ 0.2 ██████ | -- 1.002 1‰ 0.5 █████▎ | hs/byte-bs----foldrx: 4.184 1‰ 0.0 ███▉ | -- 2.933 0‰ 0.1 ████▎ | -- 0.999 2‰ 0.1 █████▉ | -- 1.112 2‰ 0.1 █████▊ | hs/byte-bsl---acc: 28.005 10‰ 0.0 ██████████████████████████▍ | -- 14.319 45‰ 0.1 ████████████████████▌ | -- 1.957 1‰ 0.1 ███████████▌ | -- 2.595 14‰ 0.1 █████████████▌ | hs/byte-xxxxx-acc-1: 25.852 4‰ 0.0 ████████████████████████▎ | -- 20.915 17‰ 4.0 █████████████████████████████▉ | -- 3.591 2‰ 0.1 █████████████████████▏ | -- 4.436 5‰ 0.0 ███████████████████████▏ | hs/byte-xxxxx-acc-2: 25.622 5‰ 0.0 ████████████████████████▏ | -- 20.691 8‰ 0.1 █████████████████████████████▌ | -- 3.598 4‰ 0.1 █████████████████████▏ | -- 4.473 8‰ 0.0 ███████████████████████▎ | hs/byte-xxxxx-foldl: 25.803 5‰ 0.0 ████████████████████████▎ | -- 20.610 5‰ 1.4 █████████████████████████████▍ | -- 3.594 1‰ 0.1 █████████████████████▏ | -- 4.412 10‰ 0.0 ███████████████████████ | c/byte-getchar: 13.016 0‰ 0.0 ████████████▎ | -- 9.042 0‰ 0.1 ████████████▉ | -- 3.761 2‰ 0.1 ██████████████████████▏ | -- 2.422 27‰ 0.1 ████████████▋ | c/byte-getchar-u: 1.662 1‰ 0.1 █▋ | -- 1.314 3‰ 0.2 █▉ | -- 0.719 2‰ 0.3 ████▎ | -- 0.632 3‰ 0.3 ███▎ | c/byte-4k: 0.543 2‰ 0.2 ▌ | -- 0.419 5‰ 0.5 ▋ | -- 0.101 10‰ 3.1 ▋ | -- 0.094 26‰ 2.2 ▌ | Memory: Peak ------- KB hs/byte-bs----acc: 147752 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147784 ████████████████████████████████████████▏ | -- 147752 ████████████████████████████████████████ | hs/byte-bs----foldlx: 147756 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147812 ████████████████████████████████████████▏ | -- 147748 ████████████████████████████████████████ | hs/byte-bs----foldrx: 147760 ████████████████████████████████████████ | -- 147488 ████████████████████████████████████████ | -- 147788 ████████████████████████████████████████▏ | -- 147744 ████████████████████████████████████████ | hs/byte-bsl---acc: 3180 ▉ | -- 2896 ▊ | -- 3228 ▉ | -- 3172 ▉ | hs/byte-xxxxx-acc-1: 1916 ▌ | -- 1612 ▌ | -- 1904 ▌ | -- 1904 ▌ | hs/byte-xxxxx-acc-2: 1912 ▌ | -- 1612 ▌ | -- 1908 ▌ | -- 1904 ▌ | hs/byte-xxxxx-foldl: 1912 ▌ | -- 1612 ▌ | -- 1908 ▌ | -- 1900 ▌ | c/byte-getchar: 436 ▏ | -- 384 ▏ | -- 432 ▏ | -- 440 ▏ | c/byte-getchar-u: 432 ▏ | -- 384 ▏ | -- 440 ▏ | -- 440 ▏ | c/byte-4k: 436 ▏ | -- 380 ▏ | -- 436 ▏ | -- 440 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 4.318 1‰ 0.0 ████ | -- 2.467 1‰ 0.3 ███▌ | -- 1.313 0‰ 0.2 ███████▊ | -- 1.145 1‰ 0.2 ██████ | hs/space-bs-c8-count: 3.118 1‰ 0.1 ██▉ | -- n/a -- n/a -- 0.521 1‰ 0.2 ██▊ | hs/space-bs-c8-foldlx-1: 4.631 1‰ 0.0 ████▍ | -- 2.585 2‰ 0.1 ███▊ | -- 1.326 1‰ 0.1 ███████▊ | -- 1.221 1‰ 0.1 ██████▍ | hs/space-bs-c8-foldlx-2: 4.632 1‰ 0.0 ████▍ | -- 2.576 2‰ 0.3 ███▋ | -- 1.328 1‰ 0.1 ███████▉ | -- 1.219 2‰ 0.2 ██████▍ | hs/space-bs-c8-foldrx: 4.678 0‰ 0.0 ████▍ | -- 2.982 8‰ 2.3 ████▎ | -- 1.128 1‰ 0.2 ██████▋ | -- 1.172 2‰ 0.3 ██████▏ | hs/space-bs-c8-lenfil: 4.634 1‰ 0.1 ████▍ | -- 2.599 1‰ 0.2 ███▊ | -- 1.327 2‰ 0.2 ███████▊ | -- 1.223 1‰ 0.2 ██████▍ | hs/space-bslc8-acc-1: 32.733 7‰ 0.0 ██████████████████████████████▊ | -- 15.228 8‰ 0.1 █████████████████████▊ | -- 3.002 1‰ 0.1 █████████████████▋ | -- 3.388 10‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-2: 33.112 11‰ 0.0 ███████████████████████████████▏ | -- 15.855 38‰ 0.0 ██████████████████████▋ | -- 3.010 2‰ 0.0 █████████████████▊ | -- 3.386 5‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-3: 30.294 12‰ 0.0 ████████████████████████████▌ | -- 14.980 14‰ 0.0 █████████████████████▍ | -- 2.586 1‰ 0.1 ███████████████▎ | -- 2.832 4‰ 0.1 ██████████████▊ | hs/space-bslc8-chunk-1: 3.888 0‰ 0.1 ███▋ | -- 2.443 2‰ 0.2 ███▌ | -- 1.192 0‰ 0.0 ███████ | -- 1.094 0‰ 0.3 █████▊ | hs/space-bslc8-chunk-2: 3.887 1‰ 0.0 ███▋ | -- 2.449 1‰ 0.3 ███▌ | -- 1.189 0‰ 0.1 ███████ | -- 1.094 1‰ 0.3 █████▊ | hs/space-bslc8-chunk-3: 3.881 0‰ 0.1 ███▋ | -- 2.534 3‰ 0.3 ███▋ | -- 1.272 1‰ 0.2 ███████▌ | -- 1.087 1‰ 0.3 █████▋ | hs/space-bslc8-chunk-4: 3.910 1‰ 0.0 ███▋ | -- n/a -- n/a -- 1.099 0‰ 0.3 █████▊ | hs/space-bslc8-count: 2.355 1‰ 0.1 ██▎ | -- n/a -- n/a -- 0.477 2‰ 0.4 ██▌ | hs/space-bslc8-foldl: 4.674 0‰ 0.0 ████▍ | -- 2.938 1‰ 0.2 ████▎ | -- 1.517 0‰ 0.1 ████████▉ | -- 1.458 1‰ 0.3 ███████▋ | hs/space-bslc8-foldlx-1: 4.676 0‰ 0.0 ████▍ | -- 2.928 1‰ 0.0 ████▏ | -- 1.516 0‰ 0.0 ████████▉ | -- 1.458 0‰ 0.1 ███████▋ | hs/space-bslc8-foldlx-2: 4.676 0‰ 0.0 ████▍ | -- 2.937 2‰ 0.2 ████▎ | -- 1.511 0‰ 0.0 ████████▉ | -- 1.457 1‰ 0.1 ███████▋ | hs/space-bslc8-foldr-1: 5.222 0‰ 0.0 ████▉ | -- 4.043 6‰ 0.1 █████▊ | -- 1.205 1‰ 0.2 ███████▏ | -- 1.312 1‰ 0.2 ██████▉ | hs/space-bslc8-foldr-2: 5.224 1‰ 0.1 ████▉ | -- 4.007 4‰ 0.1 █████▊ | -- 1.202 3‰ 0.1 ███████▏ | -- 1.315 1‰ 0.2 ██████▉ | hs/space-bslc8-lenfil-1: 4.712 1‰ 0.0 ████▍ | -- 3.240 1‰ 0.2 ████▋ | -- 1.400 0‰ 0.0 ████████▎ | -- 1.323 1‰ 0.1 ██████▉ | hs/space-bslc8-lenfil-2: 4.714 1‰ 0.0 ████▍ | -- 3.236 1‰ 0.2 ████▋ | -- 1.396 0‰ 0.0 ████████▎ | -- 1.325 1‰ 0.1 ██████▉ | hs/space-bsl---foldlx: 4.522 0‰ 0.1 ████▎ | -- 2.821 1‰ 0.1 ████ | -- 1.476 1‰ 0.1 ████████▋ | -- 1.391 1‰ 0.1 ███████▎ | hs/space-xxxxx-acc-1: 27.529 6‰ 0.0 █████████████████████████▉ | -- 21.002 4‰ 0.1 ██████████████████████████████ | -- 4.245 1‰ 0.0 █████████████████████████ | -- 4.991 1‰ 0.1 ██████████████████████████ | hs/space-xxxxx-acc-2: 27.200 3‰ 0.0 █████████████████████████▋ | -- 21.270 22‰ 4.7 ██████████████████████████████▍ | -- 4.241 2‰ 0.1 █████████████████████████ | -- 4.985 1‰ 0.0 ██████████████████████████ | hs/space-xxxxx-foldl: 27.436 8‰ 0.0 █████████████████████████▊ | -- 20.934 1‰ 0.1 █████████████████████████████▉ | -- 4.239 2‰ 0.1 █████████████████████████ | -- 5.035 2‰ 0.1 ██████████████████████████▎ | hs/space-xxxxx-lenfil: 39.343 3‰ 0.0 █████████████████████████████████████| -- 25.915 3‰ 0.0 █████████████████████████████████████| -- 6.287 2‰ 0.0 █████████████████████████████████████| -- 7.106 8‰ 0.1 █████████████████████████████████████| c/space-getchar: 13.268 0‰ 0.0 ████████████▌ | -- 9.354 0‰ 0.0 █████████████▍ | -- 4.031 18‰ 0.0 ███████████████████████▊ | -- 2.465 15‰ 0.1 ████████████▉ | c/space-getchar-u: 2.170 0‰ 0.1 ██ | -- 1.676 2‰ 0.2 ██▍ | -- 0.609 2‰ 0.7 ███▋ | -- 0.631 1‰ 0.5 ███▎ | c/space-4k: 1.815 1‰ 0.1 █▊ | -- 1.293 2‰ 0.5 █▉ | -- 0.465 2‰ 0.6 ██▊ | -- 0.319 3‰ 0.6 █▋ | c/space-megabuf: 2.763 2‰ 0.1 ██▋ | -- 1.830 3‰ 0.5 ██▋ | -- 0.613 1‰ 0.3 ███▋ | -- 0.515 3‰ 0.4 ██▋ | c/space-getwchar: 15.105 29‰ 0.0 ██████████████▎ | -- 14.721 1‰ 0.1 █████████████████████ | -- 4.584 1‰ 0.1 ███████████████████████████ | -- 3.956 6‰ 0.1 ████████████████████▋ | c/space-getwchar-u: 8.979 1‰ 0.0 ████████▌ | -- 4.814 0‰ 0.0 ██████▉ | -- 2.437 0‰ 0.1 ██████████████▍ | -- 2.217 1‰ 0.1 ███████████▌ | c/space-32k: 1.786 9‰ 0.1 █▋ | -- 1.276 2‰ 0.6 █▉ | -- 0.445 1‰ 0.5 ██▋ | -- 0.303 3‰ 1.0 █▋ | c/space-32k-8: 1.777 1‰ 0.1 █▋ | -- 1.275 2‰ 0.2 █▉ | -- 0.448 0‰ 0.0 ██▋ | -- 0.304 2‰ 0.3 █▋ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147748 ████████████████████████████████████████ | -- 147488 ████████████████████████████████████████ | -- 147772 ████████████████████████████████████████ | -- 147736 ████████████████████████████████████████ | hs/space-bs-c8-count: 147760 ████████████████████████████████████████ | -- n/a -- n/a -- 147744 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147760 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147800 ████████████████████████████████████████▏ | -- 147756 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-2: 147764 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147804 ████████████████████████████████████████▏ | -- 147752 ████████████████████████████████████████ | hs/space-bs-c8-foldrx: 147748 ████████████████████████████████████████ | -- 147488 ████████████████████████████████████████ | -- 147776 ████████████████████████████████████████ | -- 147732 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147764 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147756 ████████████████████████████████████████ | hs/space-bslc8-acc-1: 3196 ▉ | -- 2896 ▊ | -- 3192 ▉ | -- 3192 ▉ | hs/space-bslc8-acc-2: 3200 ▉ | -- 2896 ▊ | -- 3192 ▉ | -- 3192 ▉ | hs/space-bslc8-acc-3: 3204 ▉ | -- 2896 ▊ | -- 3188 ▉ | -- 3196 ▉ | hs/space-bslc8-chunk-1: 65204 █████████████████▋ | -- 65892 █████████████████▉ | -- 65208 █████████████████▋ | -- 65192 █████████████████▋ | hs/space-bslc8-chunk-2: 65200 █████████████████▋ | -- 65892 █████████████████▉ | -- 65212 █████████████████▋ | -- 65192 █████████████████▋ | hs/space-bslc8-chunk-3: 75248 ████████████████████▍ | -- 76472 ████████████████████▊ | -- 75232 ████████████████████▍ | -- 75240 ████████████████████▍ | hs/space-bslc8-chunk-4: 76204 ████████████████████▋ | -- n/a -- n/a -- 76188 ████████████████████▋ | hs/space-bslc8-count: 79224 █████████████████████▌ | -- n/a -- n/a -- 79204 █████████████████████▌ | hs/space-bslc8-foldl: 85880 ███████████████████████▎ | -- 86772 ███████████████████████▋ | -- 85856 ███████████████████████▎ | -- 85868 ███████████████████████▎ | hs/space-bslc8-foldlx-1: 85880 ███████████████████████▎ | -- 86772 ███████████████████████▋ | -- 85852 ███████████████████████▎ | -- 85864 ███████████████████████▎ | hs/space-bslc8-foldlx-2: 85880 ███████████████████████▎ | -- 86772 ███████████████████████▋ | -- 85856 ███████████████████████▎ | -- 85868 ███████████████████████▎ | hs/space-bslc8-foldr-1: 169704 ██████████████████████████████████████████████| -- 169360 ██████████████████████████████████████████████| -- 169684 ██████████████████████████████████████████████| -- 169700 ██████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169700 ██████████████████████████████████████████████| -- 169360 ██████████████████████████████████████████████| -- 169684 ██████████████████████████████████████████████| -- 169700 ██████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109408 █████████████████████████████▋ | -- 110704 ██████████████████████████████▏ | -- 109404 █████████████████████████████▋ | -- 109404 █████████████████████████████▋ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████▋ | -- 110704 ██████████████████████████████▏ | -- 109404 █████████████████████████████▋ | -- 109404 █████████████████████████████▋ | hs/space-bsl---foldlx: 85844 ███████████████████████▎ | -- 86776 ███████████████████████▋ | -- 85880 ███████████████████████▎ | -- 85836 ███████████████████████▎ | hs/space-xxxxx-acc-1: 1912 ▌ | -- 1612 ▌ | -- 1916 ▌ | -- 1904 ▌ | hs/space-xxxxx-acc-2: 1912 ▌ | -- 1612 ▌ | -- 1916 ▌ | -- 1904 ▌ | hs/space-xxxxx-foldl: 1912 ▌ | -- 1612 ▌ | -- 1916 ▌ | -- 1900 ▌ | hs/space-xxxxx-lenfil: 1904 ▌ | -- 1588 ▍ | -- 1892 ▌ | -- 1892 ▌ | c/space-getchar: 436 ▏ | -- 384 ▏ | -- 436 ▏ | -- 444 ▏ | c/space-getchar-u: 436 ▏ | -- 384 ▏ | -- 436 ▏ | -- 444 ▏ | c/space-4k: 472 ▏ | -- 412 ▏ | -- 468 ▏ | -- 468 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████▉ | -- 146904 ███████████████████████████████████████▉ | -- 146968 ███████████████████████████████████████▉ | -- 146968 ███████████████████████████████████████▉ | c/space-getwchar: 484 ▏ | -- 440 ▏ | -- 484 ▏ | -- 496 ▏ | c/space-getwchar-u: 484 ▏ | -- 440 ▏ | -- 480 ▏ | -- 500 ▏ | c/space-32k: 500 ▏ | -- 436 ▏ | -- 496 ▏ | -- 496 ▏ | c/space-32k-8: 496 ▏ | -- 436 ▏ | -- 500 ▏ | -- 500 ▏ |

1 0

[9/16] SBM: 6.9.20071119 Athlon64
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by me on my slightly old noname 2GHz Athlon64 3000+ (in 32-bit mode on a 32-bit kernel). It has 1GB RAM and runs Ubuntu 7.04. The benchmarks were run two days ago (2007-12-19) using ghc 6.9.20071119 in runlevel 1 (single-user mode). I did leave the network cable in, which might have produced some interrupts (not very likely -- 1) why would it run in promiscuous mode and 2) who broadcasts apart from my wired/wireless access point which broadcasts a packet per second?). Still, the quality of the timings proved to be good, with a standard deviation of at most 2.7% of the average run-time. This run includes the hand-tweaked assembly benchmarks (as does the Pentium III run in the previous email). Note how much the assembler tweaks help! -Peter charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 0.705 7‰ 0.1 ███▋ | hs/byte-bs----foldlx: 1.002 1‰ 0.5 █████▎ | hs/byte-bs----foldrx: 1.112 2‰ 0.1 █████▊ | hs/byte-bsl---acc: 2.595 14‰ 0.1 █████████████▌ | hs/byte-xxxxx-acc-1: 4.436 5‰ 0.0 ███████████████████████▏ | hs/byte-xxxxx-acc-2: 4.473 8‰ 0.0 ███████████████████████▎ | hs/byte-xxxxx-foldl: 4.412 10‰ 0.0 ███████████████████████ | hand/byte-bs----acc-a: 0.639 2‰ 0.2 ███▍ | hand/byte-bs----acc-b: 0.414 2‰ 0.5 ██▏ | hand/byte-bs----acc-c: 0.414 3‰ 0.2 ██▏ | hand/byte-bs----acc-d: 0.415 3‰ 0.2 ██▏ | c/byte-getchar: 2.422 27‰ 0.1 ████████████▋ | c/byte-getchar-u: 0.632 3‰ 0.3 ███▎ | c/byte-4k: 0.094 26‰ 2.2 ▌ | Memory: Peak ------- KB hs/byte-bs----acc: 147752 ███████████████████████████████████████▏ | hs/byte-bs----foldlx: 147748 ███████████████████████████████████████▏ | hs/byte-bs----foldrx: 147744 ███████████████████████████████████████▏ | hs/byte-bsl---acc: 3172 ▉ | hs/byte-xxxxx-acc-1: 1904 ▌ | hs/byte-xxxxx-acc-2: 1904 ▌ | hs/byte-xxxxx-foldl: 1900 ▌ | hand/byte-bs----acc-a: 147764 ███████████████████████████████████████▏ | hand/byte-bs----acc-b: 147764 ███████████████████████████████████████▏ | hand/byte-bs----acc-c: 147760 ███████████████████████████████████████▏ | hand/byte-bs----acc-d: 147760 ███████████████████████████████████████▏ | c/byte-getchar: 440 ▏ | c/byte-getchar-u: 440 ▏ | c/byte-4k: 440 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 1.145 1‰ 0.2 ██████ | hs/space-bs-c8-count: 0.521 1‰ 0.2 ██▊ | hs/space-bs-c8-foldlx-1: 1.221 1‰ 0.1 ██████▍ | hs/space-bs-c8-foldlx-2: 1.219 2‰ 0.2 ██████▍ | hs/space-bs-c8-foldrx: 1.172 2‰ 0.3 ██████▏ | hs/space-bs-c8-lenfil: 1.223 1‰ 0.2 ██████▍ | hs/space-bslc8-acc-1: 3.388 10‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-2: 3.386 5‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-3: 2.832 4‰ 0.1 ██████████████▊ | hs/space-bslc8-chunk-1: 1.094 0‰ 0.3 █████▊ | hs/space-bslc8-chunk-2: 1.094 1‰ 0.3 █████▊ | hs/space-bslc8-chunk-3: 1.087 1‰ 0.3 █████▋ | hs/space-bslc8-chunk-4: 1.099 0‰ 0.3 █████▊ | hs/space-bslc8-count: 0.477 2‰ 0.4 ██▌ | hs/space-bslc8-foldl: 1.458 1‰ 0.3 ███████▋ | hs/space-bslc8-foldlx-1: 1.458 0‰ 0.1 ███████▋ | hs/space-bslc8-foldlx-2: 1.457 1‰ 0.1 ███████▋ | hs/space-bslc8-foldr-1: 1.312 1‰ 0.2 ██████▉ | hs/space-bslc8-foldr-2: 1.315 1‰ 0.2 ██████▉ | hs/space-bslc8-lenfil-1: 1.323 1‰ 0.1 ██████▉ | hs/space-bslc8-lenfil-2: 1.325 1‰ 0.1 ██████▉ | hs/space-bsl---foldlx: 1.391 1‰ 0.1 ███████▎ | hs/space-xxxxx-acc-1: 4.991 1‰ 0.1 ██████████████████████████ | hs/space-xxxxx-acc-2: 4.985 1‰ 0.0 ██████████████████████████ | hs/space-xxxxx-foldl: 5.035 2‰ 0.1 ██████████████████████████▎ | hs/space-xxxxx-lenfil: 7.106 8‰ 0.1 █████████████████████████████████████| hand/space-bs-c8-acc-1-a: 1.177 2‰ 0.3 ██████▏ | hand/space-bs-c8-acc-1-b: 1.104 1‰ 0.2 █████▊ | hand/space-bs-c8-acc-1-c: 1.207 1‰ 0.3 ██████▎ | hand/space-bs-c8-acc-1-d: 1.191 1‰ 0.2 ██████▎ | hand/space-bs-c8-acc-1-e: 0.937 1‰ 0.2 ████▉ | hand/space-bs-c8-acc-1-f: 0.921 1‰ 0.2 ████▊ | hand/space-bs-c8-acc-1-g: 0.927 2‰ 0.4 ████▉ | hand/space-bs-c8-acc-1-h: 0.886 2‰ 0.2 ████▋ | hand/space-bs-c8-acc-1-i: 0.803 3‰ 0.4 ████▏ | hand/space-bs-c8-acc-1-j: 0.706 2‰ 0.1 ███▋ | hand/space-bs-c8-acc-1-k: 0.705 1‰ 0.0 ███▋ | hand/space-bs-c8-acc-1-l: 0.710 2‰ 0.1 ███▊ | hand/space-bs-c8-acc-1-m: 0.642 6‰ 0.3 ███▍ | hand/space-bs-c8-acc-1-n: 0.636 4‰ 0.5 ███▎ | hand/space-bs-c8-acc-1-o: 0.626 2‰ 0.0 ███▎ | hand/space-bs-c8-acc-1-p: 0.565 3‰ 0.4 ███ | hand/space-bs-c8-acc-1-q: 0.418 6‰ 0.7 ██▏ | hand/space-bs-c8-acc-1-r: 0.334 5‰ 0.6 █▊ | hand/space-bs-c8-acc-1-s: 0.335 4‰ 0.9 █▊ | c/space-getchar: 2.465 15‰ 0.1 ████████████▉ | c/space-getchar-u: 0.631 1‰ 0.5 ███▎ | c/space-4k: 0.319 3‰ 0.6 █▋ | c/space-megabuf: 0.515 3‰ 0.4 ██▋ | c/space-getwchar: 3.956 6‰ 0.1 ████████████████████▋ | c/space-getwchar-u: 2.217 1‰ 0.1 ███████████▌ | c/space-32k: 0.303 3‰ 1.0 █▋ | c/space-32k-8: 0.304 2‰ 0.3 █▋ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147736 ███████████████████████████████████████▏ | hs/space-bs-c8-count: 147744 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-1: 147756 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-2: 147752 ███████████████████████████████████████▏ | hs/space-bs-c8-foldrx: 147732 ███████████████████████████████████████▏ | hs/space-bs-c8-lenfil: 147756 ███████████████████████████████████████▏ | hs/space-bslc8-acc-1: 3192 ▉ | hs/space-bslc8-acc-2: 3192 ▉ | hs/space-bslc8-acc-3: 3196 ▉ | hs/space-bslc8-chunk-1: 65192 █████████████████▎ | hs/space-bslc8-chunk-2: 65192 █████████████████▎ | hs/space-bslc8-chunk-3: 75240 ████████████████████ | hs/space-bslc8-chunk-4: 76188 ████████████████████▎ | hs/space-bslc8-count: 79204 █████████████████████ | hs/space-bslc8-foldl: 85868 ██████████████████████▊ | hs/space-bslc8-foldlx-1: 85864 ██████████████████████▊ | hs/space-bslc8-foldlx-2: 85868 ██████████████████████▊ | hs/space-bslc8-foldr-1: 169700 █████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169700 █████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109404 █████████████████████████████ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████ | hs/space-bsl---foldlx: 85836 ██████████████████████▊ | hs/space-xxxxx-acc-1: 1904 ▌ | hs/space-xxxxx-acc-2: 1904 ▌ | hs/space-xxxxx-foldl: 1900 ▌ | hs/space-xxxxx-lenfil: 1892 ▌ | hand/space-bs-c8-acc-1-a: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-b: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-c: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-d: 147756 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-e: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-f: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-g: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-h: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-i: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-j: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-k: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-l: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-m: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-n: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-o: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-p: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-q: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-r: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-s: 147744 ███████████████████████████████████████▏ | c/space-getchar: 444 ▏ | c/space-getchar-u: 444 ▏ | c/space-4k: 468 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████ | c/space-getwchar: 496 ▏ | c/space-getwchar-u: 500 ▏ | c/space-32k: 496 ▏ | c/space-32k-8: 500 ▏ |

1 0

[8/16] SBM: 6.9.20071119 Pentium III
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by me on my old Compaq Armada E500 from around the year 2000. It has an Intel Coppermine Pentium III running at 600 MHz with 384 MB RAM and running Ubuntu 7.10. The benchmarks were run today (2007-12-21) using ghc 6.9.20071119 (freshly downloaded and reinstalled) in runlevel 1 (single-user mode). I did leave the wireless card in, which might have produced some interrupts. Still, the quality of the timings proved to be good, with a standard deviation of at most 1.2% of the average run-time. This run includes the hand-tweaked assembly benchmarks (as does the Athlon64 run in the next email). Note how much less the assembler tweaks help here (until one gets down to the wicked MMX tweaks) compared to the situation on the Athlon64 in the next mail. They still help, though. -Peter ls-search ghc 6.9.20071119 Pentium III (Coppermine) 596.932 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 3.274 1‰ 0.1 ███ | hs/byte-bs----foldlx: 4.027 0‰ 0.0 ███▋ | hs/byte-bs----foldrx: 4.184 1‰ 0.0 ███▉ | hs/byte-bsl---acc: 28.005 10‰ 0.0 █████████████████████████▋ | hs/byte-xxxxx-acc-1: 25.852 4‰ 0.0 ███████████████████████▋ | hs/byte-xxxxx-acc-2: 25.622 5‰ 0.0 ███████████████████████▌ | hs/byte-xxxxx-foldl: 25.803 5‰ 0.0 ███████████████████████▋ | hand/byte-bs----acc-a: 3.511 1‰ 0.0 ███▎ | hand/byte-bs----acc-b: 1.998 2‰ 0.1 █▉ | hand/byte-bs----acc-c: 1.876 2‰ 0.1 █▊ | hand/byte-bs----acc-d: 1.876 1‰ 0.1 █▊ | c/byte-getchar: 13.016 0‰ 0.0 ███████████▉ | c/byte-getchar-u: 1.662 1‰ 0.1 █▌ | c/byte-4k: 0.543 2‰ 0.2 ▌ | Memory: Peak ------- KB hs/byte-bs----acc: 147752 ███████████████████████████████████████▏ | hs/byte-bs----foldlx: 147756 ███████████████████████████████████████▏ | hs/byte-bs----foldrx: 147760 ███████████████████████████████████████▏ | hs/byte-bsl---acc: 3180 ▉ | hs/byte-xxxxx-acc-1: 1916 ▌ | hs/byte-xxxxx-acc-2: 1912 ▌ | hs/byte-xxxxx-foldl: 1912 ▌ | hand/byte-bs----acc-a: 147772 ███████████████████████████████████████▏ | hand/byte-bs----acc-b: 147776 ███████████████████████████████████████▏ | hand/byte-bs----acc-c: 147772 ███████████████████████████████████████▏ | hand/byte-bs----acc-d: 147776 ███████████████████████████████████████▏ | c/byte-getchar: 436 ▏ | c/byte-getchar-u: 432 ▏ | c/byte-4k: 436 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 4.318 1‰ 0.0 ████ | hs/space-bs-c8-count: 3.118 1‰ 0.1 ██▉ | hs/space-bs-c8-foldlx-1: 4.631 1‰ 0.0 ████▎ | hs/space-bs-c8-foldlx-2: 4.632 1‰ 0.0 ████▎ | hs/space-bs-c8-foldrx: 4.678 0‰ 0.0 ████▎ | hs/space-bs-c8-lenfil: 4.634 1‰ 0.1 ████▎ | hs/space-bslc8-acc-1: 32.733 7‰ 0.0 ██████████████████████████████ | hs/space-bslc8-acc-2: 33.112 11‰ 0.0 ██████████████████████████████▎ | hs/space-bslc8-acc-3: 30.294 12‰ 0.0 ███████████████████████████▊ | hs/space-bslc8-chunk-1: 3.888 0‰ 0.1 ███▌ | hs/space-bslc8-chunk-2: 3.887 1‰ 0.0 ███▌ | hs/space-bslc8-chunk-3: 3.881 0‰ 0.1 ███▌ | hs/space-bslc8-chunk-4: 3.910 1‰ 0.0 ███▋ | hs/space-bslc8-count: 2.355 1‰ 0.1 ██▏ | hs/space-bslc8-foldl: 4.674 0‰ 0.0 ████▎ | hs/space-bslc8-foldlx-1: 4.676 0‰ 0.0 ████▎ | hs/space-bslc8-foldlx-2: 4.676 0‰ 0.0 ████▎ | hs/space-bslc8-foldr-1: 5.222 0‰ 0.0 ████▊ | hs/space-bslc8-foldr-2: 5.224 1‰ 0.1 ████▊ | hs/space-bslc8-lenfil-1: 4.712 1‰ 0.0 ████▎ | hs/space-bslc8-lenfil-2: 4.714 1‰ 0.0 ████▍ | hs/space-bsl---foldlx: 4.522 0‰ 0.1 ████▏ | hs/space-xxxxx-acc-1: 27.529 6‰ 0.0 █████████████████████████▎ | hs/space-xxxxx-acc-2: 27.200 3‰ 0.0 ████████████████████████▉ | hs/space-xxxxx-foldl: 27.436 8‰ 0.0 █████████████████████████▏ | hs/space-xxxxx-lenfil: 39.343 3‰ 0.0 ████████████████████████████████████| hand/space-bs-c8-acc-1-a: 4.318 1‰ 0.0 ████ | hand/space-bs-c8-acc-1-b: 4.331 1‰ 0.0 ████ | hand/space-bs-c8-acc-1-c: 4.492 1‰ 0.1 ████▏ | hand/space-bs-c8-acc-1-d: 4.354 1‰ 0.0 ████ | hand/space-bs-c8-acc-1-e: 4.424 0‰ 0.1 ████ | hand/space-bs-c8-acc-1-f: 4.164 1‰ 0.0 ███▊ | hand/space-bs-c8-acc-1-g: 4.309 1‰ 0.1 ████ | hand/space-bs-c8-acc-1-h: 4.202 1‰ 0.1 ███▉ | hand/space-bs-c8-acc-1-i: 3.820 1‰ 0.1 ███▌ | hand/space-bs-c8-acc-1-j: 3.472 1‰ 0.0 ███▏ | hand/space-bs-c8-acc-1-k: 3.474 1‰ 0.0 ███▏ | hand/space-bs-c8-acc-1-l: 3.498 1‰ 0.1 ███▎ | hand/space-bs-c8-acc-1-m: 3.397 1‰ 0.1 ███▏ | hand/space-bs-c8-acc-1-n: 3.373 1‰ 0.0 ███▏ | hand/space-bs-c8-acc-1-o: 3.118 1‰ 0.1 ██▉ | hand/space-bs-c8-acc-1-p: 2.935 2‰ 0.0 ██▋ | hand/space-bs-c8-acc-1-q: 3.477 1‰ 0.1 ███▏ | hand/space-bs-c8-acc-1-r: 1.674 1‰ 0.1 █▌ | hand/space-bs-c8-acc-1-s: 1.627 1‰ 0.2 █▌ | c/space-getchar: 13.268 0‰ 0.0 ████████████▏ | c/space-getchar-u: 2.170 0‰ 0.1 ██ | c/space-4k: 1.815 1‰ 0.1 █▋ | c/space-megabuf: 2.763 2‰ 0.1 ██▌ | c/space-getwchar: 15.105 29‰ 0.0 █████████████▉ | c/space-getwchar-u: 8.979 1‰ 0.0 ████████▎ | c/space-32k: 1.786 9‰ 0.1 █▋ | c/space-32k-8: 1.777 1‰ 0.1 █▋ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147748 ███████████████████████████████████████▏ | hs/space-bs-c8-count: 147760 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-1: 147760 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-2: 147764 ███████████████████████████████████████▏ | hs/space-bs-c8-foldrx: 147748 ███████████████████████████████████████▏ | hs/space-bs-c8-lenfil: 147764 ███████████████████████████████████████▏ | hs/space-bslc8-acc-1: 3196 ▉ | hs/space-bslc8-acc-2: 3200 ▉ | hs/space-bslc8-acc-3: 3204 ▉ | hs/space-bslc8-chunk-1: 65204 █████████████████▎ | hs/space-bslc8-chunk-2: 65200 █████████████████▎ | hs/space-bslc8-chunk-3: 75248 ████████████████████ | hs/space-bslc8-chunk-4: 76204 ████████████████████▎ | hs/space-bslc8-count: 79224 █████████████████████ | hs/space-bslc8-foldl: 85880 ██████████████████████▊ | hs/space-bslc8-foldlx-1: 85880 ██████████████████████▊ | hs/space-bslc8-foldlx-2: 85880 ██████████████████████▊ | hs/space-bslc8-foldr-1: 169704 █████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169700 █████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109408 █████████████████████████████ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████ | hs/space-bsl---foldlx: 85844 ██████████████████████▊ | hs/space-xxxxx-acc-1: 1912 ▌ | hs/space-xxxxx-acc-2: 1912 ▌ | hs/space-xxxxx-foldl: 1912 ▌ | hs/space-xxxxx-lenfil: 1904 ▌ | hand/space-bs-c8-acc-1-a: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-b: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-c: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-d: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-e: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-f: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-g: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-h: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-i: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-j: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-k: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-l: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-m: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-n: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-o: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-p: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-q: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-r: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-s: 147760 ███████████████████████████████████████▏ | c/space-getchar: 436 ▏ | c/space-getchar-u: 436 ▏ | c/space-4k: 472 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████ | c/space-getwchar: 484 ▏ | c/space-getwchar-u: 484 ▏ | c/space-32k: 500 ▏ | c/space-32k-8: 496 ▏ |

1 0

[7/16] SBM: 6.9.20071208 Core Duo
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by me on a friend's machine at his place, namely an Intel Core Duo laptop running Ubuntu 7.10. The benchmarks were run on 2007-12-09 using ghc 6.9.20071208 (approx) in runlevel 1 (single-user mode). I used an early version of the benchmark suite which didn't capture the exact ghc version used and an early version of my ghc install scripts that didn't try for a precise version. It ran on what was ghc head at the time. -Peter albatross ghc 6.9.20071208 (or thereabouts) Genuine Intel(R) CPU T2300 @ 1.66GHz (Core Duo) 1667.000 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 0.918 4‰ 0.1 █████▌ | hs/byte-bs----foldlx: 1.014 2‰ 0.2 ██████▏ | hs/byte-bs----foldrx: 0.999 2‰ 0.1 ██████ | hs/byte-bsl---acc: 1.957 1‰ 0.1 ███████████▉ | hs/byte-xxxxx-acc-1: 3.591 2‰ 0.1 █████████████████████▊ | hs/byte-xxxxx-acc-2: 3.598 4‰ 0.1 █████████████████████▊ | hs/byte-xxxxx-foldl: 3.594 1‰ 0.1 █████████████████████▊ | c/byte-getchar: 3.761 2‰ 0.1 ██████████████████████▊ | c/byte-getchar-u: 0.719 2‰ 0.3 ████▍ | c/byte-4k: 0.101 10‰ 3.1 ▋ | Memory: Peak ------- KB hs/byte-bs----acc: 147784 ████████████████████████████████████████▏ | hs/byte-bs----foldlx: 147812 ████████████████████████████████████████▏ | hs/byte-bs----foldrx: 147788 ████████████████████████████████████████▏ | hs/byte-bsl---acc: 3228 ▉ | hs/byte-xxxxx-acc-1: 1904 ▌ | hs/byte-xxxxx-acc-2: 1908 ▌ | hs/byte-xxxxx-foldl: 1908 ▌ | c/byte-getchar: 432 ▏ | c/byte-getchar-u: 440 ▏ | c/byte-4k: 436 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 1.313 0‰ 0.2 ███████▉ | hs/space-bs-c8-foldlx-1: 1.326 1‰ 0.1 ████████ | hs/space-bs-c8-foldlx-2: 1.328 1‰ 0.1 ████████ | hs/space-bs-c8-foldrx: 1.128 1‰ 0.2 ██████▉ | hs/space-bs-c8-lenfil: 1.327 2‰ 0.2 ████████ | hs/space-bslc8-acc-1: 3.002 1‰ 0.1 ██████████████████▏ | hs/space-bslc8-acc-2: 3.010 2‰ 0.0 ██████████████████▎ | hs/space-bslc8-acc-3: 2.586 1‰ 0.1 ███████████████▋ | hs/space-bslc8-chunk-1: 1.192 0‰ 0.0 ███████▎ | hs/space-bslc8-chunk-2: 1.189 0‰ 0.1 ███████▏ | hs/space-bslc8-chunk-3: 1.272 1‰ 0.2 ███████▊ | hs/space-bslc8-foldl: 1.517 0‰ 0.1 █████████▏ | hs/space-bslc8-foldlx-1: 1.516 0‰ 0.0 █████████▏ | hs/space-bslc8-foldlx-2: 1.511 0‰ 0.0 █████████▏ | hs/space-bslc8-foldr-1: 1.205 1‰ 0.2 ███████▎ | hs/space-bslc8-foldr-2: 1.202 3‰ 0.1 ███████▎ | hs/space-bslc8-lenfil-1: 1.400 0‰ 0.0 ████████▌ | hs/space-bslc8-lenfil-2: 1.396 0‰ 0.0 ████████▌ | hs/space-bsl---foldlx: 1.476 1‰ 0.1 ████████▉ | hs/space-xxxxx-acc-1: 4.245 1‰ 0.0 █████████████████████████▋ | hs/space-xxxxx-acc-2: 4.241 2‰ 0.1 █████████████████████████▋ | hs/space-xxxxx-foldl: 4.239 2‰ 0.1 █████████████████████████▋ | hs/space-xxxxx-lenfil: 6.287 2‰ 0.0 ██████████████████████████████████████| c/space-getchar: 4.031 18‰ 0.0 ████████████████████████▍ | c/space-getchar-u: 0.609 2‰ 0.7 ███▋ | c/space-4k: 0.465 2‰ 0.6 ██▊ | c/space-megabuf: 0.613 1‰ 0.3 ███▊ | c/space-getwchar: 4.584 1‰ 0.1 ███████████████████████████▊ | c/space-getwchar-u: 2.437 0‰ 0.1 ██████████████▊ | c/space-32k: 0.445 1‰ 0.5 ██▊ | c/space-32k-8: 0.448 0‰ 0.0 ██▊ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147772 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147800 ████████████████████████████████████████▏ | hs/space-bs-c8-foldlx-2: 147804 ████████████████████████████████████████▏ | hs/space-bs-c8-foldrx: 147776 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147796 ████████████████████████████████████████▏ | hs/space-bslc8-acc-1: 3192 ▉ | hs/space-bslc8-acc-2: 3192 ▉ | hs/space-bslc8-acc-3: 3188 ▉ | hs/space-bslc8-chunk-1: 65208 █████████████████▋ | hs/space-bslc8-chunk-2: 65212 █████████████████▋ | hs/space-bslc8-chunk-3: 75232 ████████████████████▍ | hs/space-bslc8-foldl: 85856 ███████████████████████▎ | hs/space-bslc8-foldlx-1: 85852 ███████████████████████▎ | hs/space-bslc8-foldlx-2: 85856 ███████████████████████▎ | hs/space-bslc8-foldr-1: 169684 ██████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169684 ██████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109404 █████████████████████████████▋ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████▋ | hs/space-bsl---foldlx: 85880 ███████████████████████▎ | hs/space-xxxxx-acc-1: 1916 ▌ | hs/space-xxxxx-acc-2: 1916 ▌ | hs/space-xxxxx-foldl: 1916 ▌ | hs/space-xxxxx-lenfil: 1892 ▌ | c/space-getchar: 436 ▏ | c/space-getchar-u: 436 ▏ | c/space-4k: 468 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████▉ | c/space-getwchar: 484 ▏ | c/space-getwchar-u: 480 ▏ | c/space-32k: 496 ▏ | c/space-32k-8: 500 ▏ |

1 0

[6/16] SBM: 6.9.20071124 Athlon Duron
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by Daniel Fischer on one of his older machines, running SuSE 8.2, which has a Linux 2.4 kernel! The benchmarks were run on 2007-12-16 using ghc 6.9.20071124. Unfortunately, the ghc version is not quite the same as the one I've used for most measurements (6.9.20071119) so things may be a little different just for that reason. The results seemed a bit off at first, but now that I have graphs of all the runs on all the machines they don't seem strange at all. First of all, the memory use is about the same as on the other machines. Secondly, the timing differences for the C getchar/getwchar might be partly due to different versions of the C library. The remaining differences (a "steeper" profile than on the core duo and the Athlon64) may be due to different microarchitectures. -Peter Fischer's machine ghc 6.9.20071124 AMD Duron(tm) processor 1200.089 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 1.892 21‰ 0.4 ██▊ | hs/byte-bs----foldlx: 2.258 3‰ 0.1 ███▎ | hs/byte-bs----foldrx: 2.933 0‰ 0.1 ████▎ | hs/byte-bsl---acc: 14.319 45‰ 0.1 ████████████████████▌ | hs/byte-xxxxx-acc-1: 20.915 17‰ 4.0 █████████████████████████████▉ | hs/byte-xxxxx-acc-2: 20.691 8‰ 0.1 █████████████████████████████▌ | hs/byte-xxxxx-foldl: 20.610 5‰ 1.4 █████████████████████████████▍ | c/byte-getchar: 9.042 0‰ 0.1 ████████████▉ | c/byte-getchar-u: 1.314 3‰ 0.2 █▉ | c/byte-4k: 0.419 5‰ 0.5 ▋ | Memory: Peak ------- KB hs/byte-bs----acc: 147492 ████████████████████████████████████████ | hs/byte-bs----foldlx: 147492 ████████████████████████████████████████ | hs/byte-bs----foldrx: 147488 ████████████████████████████████████████ | hs/byte-bsl---acc: 2896 ▊ | hs/byte-xxxxx-acc-1: 1612 ▌ | hs/byte-xxxxx-acc-2: 1612 ▌ | hs/byte-xxxxx-foldl: 1612 ▌ | c/byte-getchar: 384 ▏ | c/byte-getchar-u: 384 ▏ | c/byte-4k: 380 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 2.467 1‰ 0.3 ███▌ | hs/space-bs-c8-foldlx-1: 2.585 2‰ 0.1 ███▊ | hs/space-bs-c8-foldlx-2: 2.576 2‰ 0.3 ███▋ | hs/space-bs-c8-foldrx: 2.982 8‰ 2.3 ████▎ | hs/space-bs-c8-lenfil: 2.599 1‰ 0.2 ███▊ | hs/space-bslc8-acc-1: 15.228 8‰ 0.1 █████████████████████▊ | hs/space-bslc8-acc-2: 15.855 38‰ 0.0 ██████████████████████▋ | hs/space-bslc8-acc-3: 14.980 14‰ 0.0 █████████████████████▍ | hs/space-bslc8-chunk-1: 2.443 2‰ 0.2 ███▌ | hs/space-bslc8-chunk-2: 2.449 1‰ 0.3 ███▌ | hs/space-bslc8-chunk-3: 2.534 3‰ 0.3 ███▋ | hs/space-bslc8-foldl: 2.938 1‰ 0.2 ████▎ | hs/space-bslc8-foldlx-1: 2.928 1‰ 0.0 ████▏ | hs/space-bslc8-foldlx-2: 2.937 2‰ 0.2 ████▎ | hs/space-bslc8-foldr-1: 4.043 6‰ 0.1 █████▊ | hs/space-bslc8-foldr-2: 4.007 4‰ 0.1 █████▊ | hs/space-bslc8-lenfil-1: 3.240 1‰ 0.2 ████▋ | hs/space-bslc8-lenfil-2: 3.236 1‰ 0.2 ████▋ | hs/space-bsl---foldlx: 2.821 1‰ 0.1 ████ | hs/space-xxxxx-acc-1: 21.002 4‰ 0.1 ██████████████████████████████ | hs/space-xxxxx-acc-2: 21.270 22‰ 4.7 ██████████████████████████████▍ | hs/space-xxxxx-foldl: 20.934 1‰ 0.1 █████████████████████████████▉ | hs/space-xxxxx-lenfil: 25.915 3‰ 0.0 █████████████████████████████████████| c/space-getchar: 9.354 0‰ 0.0 █████████████▍ | c/space-getchar-u: 1.676 2‰ 0.2 ██▍ | c/space-4k: 1.293 2‰ 0.5 █▉ | c/space-megabuf: 1.830 3‰ 0.5 ██▋ | c/space-getwchar: 14.721 1‰ 0.1 █████████████████████ | c/space-getwchar-u: 4.814 0‰ 0.0 ██████▉ | c/space-32k: 1.276 2‰ 0.6 █▉ | c/space-32k-8: 1.275 2‰ 0.2 █▉ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147488 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147492 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-2: 147492 ████████████████████████████████████████ | hs/space-bs-c8-foldrx: 147488 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147492 ████████████████████████████████████████ | hs/space-bslc8-acc-1: 2896 ▊ | hs/space-bslc8-acc-2: 2896 ▊ | hs/space-bslc8-acc-3: 2896 ▊ | hs/space-bslc8-chunk-1: 65892 █████████████████▉ | hs/space-bslc8-chunk-2: 65892 █████████████████▉ | hs/space-bslc8-chunk-3: 76472 ████████████████████▊ | hs/space-bslc8-foldl: 86772 ███████████████████████▋ | hs/space-bslc8-foldlx-1: 86772 ███████████████████████▋ | hs/space-bslc8-foldlx-2: 86772 ███████████████████████▋ | hs/space-bslc8-foldr-1: 169360 ██████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169360 ██████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 110704 ██████████████████████████████▏ | hs/space-bslc8-lenfil-2: 110704 ██████████████████████████████▏ | hs/space-bsl---foldlx: 86776 ███████████████████████▋ | hs/space-xxxxx-acc-1: 1612 ▌ | hs/space-xxxxx-acc-2: 1612 ▌ | hs/space-xxxxx-foldl: 1612 ▌ | hs/space-xxxxx-lenfil: 1588 ▍ | c/space-getchar: 384 ▏ | c/space-getchar-u: 384 ▏ | c/space-4k: 412 ▏ | c/space-megabuf: 146904 ███████████████████████████████████████▉ | c/space-getwchar: 440 ▏ | c/space-getwchar-u: 440 ▏ | c/space-32k: 436 ▏ | c/space-32k-8: 436 ▏ |

1 0

[5/16] SBM: Support scripts and scriptlets
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

Some of the scripts warrant a closer look. 'make zipdata' creates a nice tarball with all the data necessary to recreate a report AND to merge that report together with other reports, possible with rescaled bar charts. Very handy. All the files in the tarball are inside the 'ghc-measurements/' directory so the risk of things going wrong when unpacking the tarball is less. The names of the benchmarks are put in ghc-measurements/progs, mainly to ensure they end up in the right order when regenerating and merging reports. tools/genreport.pl [list of benchmarks to put in the report] It doesn't parse the command-line in any way because life is too short for command-line parsing. Instead, it is controlled via (too many) environment variables. ASCII - set to avoid using UTF-8 for bar charts and "per mille" character. NOSRC - the tool normally creates */*.srctimemem files containing the source code for each benchmark with bar charts for time/mem appended to the end. Setting this variable switches that off (necessary when regenerating and merging reports). EXCLUDE - disregard some of the benchmarks on the command line. Why is this necessary? Because it makes regenerating and merging reports easier. And because I was too lazy to filter the command line in tools/regenreport.sh and tools/merge.pl. FINDMAX - used by tools/merge.pl when rescaling. Outputs max time and max mem to stdout instead of the normal report. MAX_FILEWIDTH - used by tools/merge.pl to make merged reports look nice MAX_TIME, MAX_PEAKMEM - used by tools/merge.pl when rescaling Note that strictly speaking, there is a bug in the script(s) because it conflates the width of time/mem measurement represented as numbers (which you always want to take into account when merging) and MAX_TIME/MAX_PEAKMEM (which you only care about when rescaling). [FIXED now - 2007-12-21] tools/regenreport.sh unpacks a measurement tarball into a tmp directory and runs tools/genreport.pl to generate the report. Takes care not to disturb the normal files. tools/merge.pl [tarballs] Uses tools/regenreport.sh on each tarball in turn to generate a report which it reads in and stores on a benchmark-by-benchmark basis. At the end, synthetically combine all the pieces it cut out of the original report(s) into a brand-spanking new, merged report. Even the headers and the platforminfo at the top of each report is cut out and stored in data structures until they get spit out again at the end. The reading magic is in the state machine in gather(). It is not as bad as it looks. Some of the complications arise from marking repeated benchmark names as ' -- ', which improves the readability of the merged reports immensely. Another part of the complications arise due to the fact that not all tarballs contain the exact same benchmarks! Those that don't get a nice 'n/a' instead of numbers and a bar. And finally, the benchmarks should be in the right order. That is trickier than it sounds... When rescaling, tools/regenreport.sh is first run once for each tarball with the FINDMAX environment variable set. This results in tools/regenreport.sh outputting the maximum filename width, time, and peakmem for each tarball. ASCII - use ASCII instead of UTF-8 RESCALE - sometimes you want to rescale and sometimes you don't MAX_FILEWIDTH - if you want to force a specific width MAX_TIME, MAX_PEAKMEM - if you want to force a specific max -Peter

1 0