- Haskell-Cafe - Haskell.org

[13/16] SBM: Graphs that show the infidelity of -sstderr
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

Everybody seems to use +RTS -sstderr to measure the memory performance of their Haskell (ghc) code. That's what I started out doing, too. However, top seemed to occasionally disagree with the run-time system. I decided to dig deeper and ask the operating system at process exit time. I use LD_PRELOAD to get my code injected into each benchmarked process where it gets activated when the process is about to exit and copies /proc/self/maps and /proc/self/status to stderr where the test harness captures them. Unfortunately, -sstderr really *IS* unreliable! tools/regensstderr.sh and tools/sstderr.pl are two scripts that together generate bar graphs for an existing measurement tarball. I used the following shell code to generate graphs for 7 different ghc/bytestring combinations: (for F in ghc-thorough-6.6.1.tgz \ ghc-thorough-6.8.2.tgz ghc-thorough-6.8.2-bs0902.tgz \ ghc-thorough-6.9.tgz ghc-thorough-6.9-bs0902.tgz \ ghc-thorough-head.tgz ghc-thorough-head-bs0902.tgz; \ do \ printf "%s\n-------\n" $F; \ tools/regensstderr.sh $F; \ printf "\n\n"; \ done \ ) > xx The graphs clearly show that -sstderr sometimes severely underestimates the memory use and sometimes somewhat overestimates it. Moreover, the errors are not always for the same programs. This holds for all ghc versions I've tested: 6.6.1, 6.8.2, 6.9.20071119, and head as of noon 2007-12-19. -Peter ghc-thorough-6.6.1.tgz ------- hs/byte-bs----acc: 1 147448 ▏ |██████████████████▎ | hs/byte-bs----foldlx: 145 147468 ██████████▎ |██████████████████▎ | hs/byte-bs----foldrx: 1 147452 ▏ |██████████████████▎ | hs/byte-bsl---acc: 2 2872 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1572 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1568 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1572 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147448 ▏ |██████████████████▎ | hs/space-bs-c8-count: 1 147448 ▏ |██████████████████▎ | hs/space-bs-c8-foldlx-1: 145 147468 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldlx-2: 145 147468 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldrx: 1 147448 ▏ |██████████████████▎ | hs/space-bs-c8-lenfil: 145 147464 ██████████▎ |██████████████████▎ | hs/space-bslc8-acc-1: 2 2872 ▏ |▍ | hs/space-bslc8-acc-2: 2 2872 ▏ |▍ | hs/space-bslc8-acc-3: 2 2868 ▏ |▍ | hs/space-bslc8-count: 2 1476 ▏ |▏ | hs/space-bslc8-foldl: 2 1552 ▏ |▎ | hs/space-bslc8-foldlx-1: 2 1556 ▏ |▎ | hs/space-bslc8-foldlx-2: 2 1552 ▏ |▎ | hs/space-bslc8-foldr-1: 297 169480 █████████████████████|█████████████████████| hs/space-bslc8-foldr-2: 297 169480 █████████████████████|█████████████████████| hs/space-bslc8-lenfil-1: 38 38632 ██▋ |████▊ | hs/space-bslc8-lenfil-2: 38 38628 ██▋ |████▊ | hs/space-bsl---foldlx: 2 1560 ▏ |▎ | hs/space-xxxxx-acc-1: 1 1568 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1572 ▏ |▎ | hs/space-xxxxx-foldl: 1 1572 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1556 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.8.2.tgz ------- hs/byte-bs----acc: 1 147500 ▏ |██████████████████▎ | hs/byte-bs----foldlx: 145 147528 ██████████▎ |██████████████████▎ | hs/byte-bs----foldrx: 1 147500 ▏ |██████████████████▎ | hs/byte-bsl---acc: 2 2920 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1632 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1632 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147492 ▏ |██████████████████▎ | hs/space-bs-c8-count: 1 147500 ▏ |██████████████████▎ | hs/space-bs-c8-foldlx-1: 145 147524 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldlx-2: 145 147520 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldrx: 1 147496 ▏ |██████████████████▎ | hs/space-bs-c8-lenfil: 145 147524 ██████████▎ |██████████████████▎ | hs/space-bslc8-acc-1: 2 2920 ▏ |▍ | hs/space-bslc8-acc-2: 2 2924 ▏ |▍ | hs/space-bslc8-acc-3: 2 2924 ▏ |▍ | hs/space-bslc8-chunk-1: 2 1512 ▏ |▏ | hs/space-bslc8-chunk-2: 2 1512 ▏ |▏ | hs/space-bslc8-chunk-3: 2 1548 ▏ |▎ | hs/space-bslc8-chunk-4: 2 1512 ▏ |▏ | hs/space-bslc8-count: 2 1520 ▏ |▎ | hs/space-bslc8-foldl: 2 1620 ▏ |▎ | hs/space-bslc8-foldlx-1: 2 1620 ▏ |▎ | hs/space-bslc8-foldlx-2: 2 1616 ▏ |▎ | hs/space-bslc8-foldr-1: 297 169544 █████████████████████|█████████████████████| hs/space-bslc8-foldr-2: 297 169544 █████████████████████|█████████████████████| hs/space-bslc8-lenfil-1: 38 38644 ██▋ |████▊ | hs/space-bslc8-lenfil-2: 38 38648 ██▋ |████▊ | hs/space-bsl---foldlx: 2 1616 ▏ |▎ | hs/space-xxxxx-acc-1: 1 1636 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/space-xxxxx-foldl: 1 1636 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1608 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.8.2-bs0902.tgz ------- hs/byte-bs----acc: 1 147616 ▏ |██████████████████▌ | hs/byte-bs----foldlx: 1 147624 ▏ |██████████████████▌ | hs/byte-bs----foldrx: 1 147624 ▏ |██████████████████▌ | hs/byte-bsl---acc: 2 3040 ▎ |▍ | hs/byte-xxxxx-acc-1: 1 1636 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1636 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-count: 1 147612 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-1: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-2: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-foldrx: 1 147608 ▏ |██████████████████▌ | hs/space-bs-c8-lenfil: 21 167176 ███ |█████████████████████| hs/space-bslc8-acc-1: 2 3048 ▎ |▍ | hs/space-bslc8-acc-2: 2 3048 ▎ |▍ | hs/space-bslc8-acc-3: 2 3052 ▎ |▍ | hs/space-bslc8-chunk-1: 2 1636 ▎ |▎ | hs/space-bslc8-chunk-2: 2 1640 ▎ |▎ | hs/space-bslc8-chunk-3: 2 1644 ▎ |▎ | hs/space-bslc8-chunk-4: 2 1636 ▎ |▎ | hs/space-bslc8-count: 2 1656 ▎ |▎ | hs/space-bslc8-foldl: 2 1632 ▎ |▎ | hs/space-bslc8-foldlx-1: 2 1636 ▎ |▎ | hs/space-bslc8-foldlx-2: 2 1636 ▎ |▎ | hs/space-bslc8-foldr-1: 149 149076 █████████████████████|██████████████████▊ | hs/space-bslc8-foldr-2: 149 149076 █████████████████████|██████████████████▊ | hs/space-bslc8-lenfil-1: 2 1940 ▎ |▎ | hs/space-bslc8-lenfil-2: 2 1940 ▎ |▎ | hs/space-bsl---foldlx: 2 1620 ▎ |▎ | hs/space-xxxxx-acc-1: 1 1636 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1636 ▏ |▎ | hs/space-xxxxx-foldl: 1 1636 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1604 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.9.tgz ------- hs/byte-bs----acc: 1 147752 ▏ |█████████████████▍ | hs/byte-bs----foldlx: 145 147748 █████████▊ |█████████████████▍ | hs/byte-bs----foldrx: 1 147744 ▏ |█████████████████▍ | hs/byte-bsl---acc: 2 3172 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1904 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1900 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147736 ▏ |█████████████████▍ | hs/space-bs-c8-count: 1 147744 ▏ |█████████████████▍ | hs/space-bs-c8-foldlx-1: 145 147756 █████████▊ |█████████████████▍ | hs/space-bs-c8-foldlx-2: 145 147752 █████████▊ |█████████████████▍ | hs/space-bs-c8-foldrx: 1 147732 ▏ |█████████████████▍ | hs/space-bs-c8-lenfil: 145 147756 █████████▊ |█████████████████▍ | hs/space-bslc8-acc-1: 2 3192 ▏ |▍ | hs/space-bslc8-acc-2: 2 3192 ▏ |▍ | hs/space-bslc8-acc-3: 2 3196 ▏ |▍ | hs/space-bslc8-chunk-1: 64 65192 ████▎ |███████▋ | hs/space-bslc8-chunk-2: 64 65192 ████▎ |███████▋ | hs/space-bslc8-chunk-3: 74 75240 █████ |████████▉ | hs/space-bslc8-chunk-4: 75 76188 █████ |█████████ | hs/space-bslc8-count: 78 79204 █████▎ |█████████▍ | hs/space-bslc8-foldl: 85 85868 █████▊ |██████████▏ | hs/space-bslc8-foldlx-1: 85 85864 █████▊ |██████████▏ | hs/space-bslc8-foldlx-2: 85 85868 █████▊ |██████████▏ | hs/space-bslc8-foldr-1: 297 169700 ████████████████████|████████████████████| hs/space-bslc8-foldr-2: 297 169700 ████████████████████|████████████████████| hs/space-bslc8-lenfil-1: 115 109404 ███████▊ |████████████▉ | hs/space-bslc8-lenfil-2: 115 109404 ███████▊ |████████████▉ | hs/space-bsl---foldlx: 85 85836 █████▊ |██████████▏ | hs/space-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1904 ▏ |▎ | hs/space-xxxxx-foldl: 1 1900 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1892 ▏ |▎ | hand/byte-bs----acc-a: 1 147764 ▏ |█████████████████▍ | hand/byte-bs----acc-b: 1 147764 ▏ |█████████████████▍ | hand/byte-bs----acc-c: 1 147760 ▏ |█████████████████▍ | hand/byte-bs----acc-d: 1 147760 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-a: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-b: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-c: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-d: 1 147756 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-e: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-f: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-g: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-h: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-i: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-j: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-k: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-l: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-m: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-n: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-o: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-p: 1 147752 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-q: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-r: 1 147748 ▏ |█████████████████▍ | hand/space-bs-c8-acc-1-s: 1 147744 ▏ |█████████████████▍ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-6.9-bs0902.tgz ------- hs/byte-bs----acc: 1 147732 ▏ |██████████████████▌ | hs/byte-bs----foldlx: 1 147736 ▏ |██████████████████▌ | hs/byte-bs----foldrx: 1 147740 ▏ |██████████████████▌ | hs/byte-bsl---acc: 2 3184 ▎ |▍ | hs/byte-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147740 ▏ |██████████████████▌ | hs/space-bs-c8-count: 1 147756 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-1: 1 147744 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-2: 1 147748 ▏ |██████████████████▌ | hs/space-bs-c8-foldrx: 1 147748 ▏ |██████████████████▌ | hs/space-bs-c8-lenfil: 21 167300 ███ |█████████████████████| hs/space-bslc8-acc-1: 2 3228 ▎ |▍ | hs/space-bslc8-acc-2: 2 3228 ▎ |▍ | hs/space-bslc8-acc-3: 2 3232 ▎ |▍ | hs/space-bslc8-chunk-1: 65 66216 █████████▏ |████████▎ | hs/space-bslc8-chunk-2: 65 66216 █████████▏ |████████▎ | hs/space-bslc8-chunk-3: 65 66208 █████████▏ |████████▎ | hs/space-bslc8-chunk-4: 78 79232 ███████████ |██████████ | hs/space-bslc8-count: 81 82248 ███████████▍ |██████████▍ | hs/space-bslc8-foldl: 78 79224 ███████████ |██████████ | hs/space-bslc8-foldlx-1: 78 79224 ███████████ |██████████ | hs/space-bslc8-foldlx-2: 78 79224 ███████████ |██████████ | hs/space-bslc8-foldr-1: 149 149704 █████████████████████|██████████████████▊ | hs/space-bslc8-foldr-2: 149 149708 █████████████████████|██████████████████▊ | hs/space-bslc8-lenfil-1: 81 82324 ███████████▍ |██████████▍ | hs/space-bslc8-lenfil-2: 81 82328 ███████████▍ |██████████▍ | hs/space-bsl---foldlx: 78 79184 ███████████ |██████████ | hs/space-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/space-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1892 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-head.tgz ------- hs/byte-bs----acc: 1 147772 ▏ |██████████████████▎ | hs/byte-bs----foldlx: 145 147796 ██████████▎ |██████████████████▎ | hs/byte-bs----foldrx: 1 147772 ▏ |██████████████████▎ | hs/byte-bsl---acc: 2 3212 ▏ |▍ | hs/byte-xxxxx-acc-1: 1 1900 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1900 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147752 ▏ |██████████████████▎ | hs/space-bs-c8-count: 1 147764 ▏ |██████████████████▎ | hs/space-bs-c8-foldlx-1: 145 147796 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldlx-2: 145 147796 ██████████▎ |██████████████████▎ | hs/space-bs-c8-foldrx: 1 147760 ▏ |██████████████████▎ | hs/space-bs-c8-lenfil: 145 147788 ██████████▎ |██████████████████▎ | hs/space-bslc8-acc-1: 2 3180 ▏ |▍ | hs/space-bslc8-acc-2: 2 3180 ▏ |▍ | hs/space-bslc8-acc-3: 2 3172 ▏ |▍ | hs/space-bslc8-chunk-1: 64 65204 ████▌ |████████▏ | hs/space-bslc8-chunk-2: 64 65204 ████▌ |████████▏ | hs/space-bslc8-chunk-3: 74 75216 █████▎ |█████████▎ | hs/space-bslc8-chunk-4: 75 76176 █████▎ |█████████▍ | hs/space-bslc8-count: 78 79220 █████▌ |█████████▊ | hs/space-bslc8-foldl: 85 85852 ██████ |██████████▋ | hs/space-bslc8-foldlx-1: 85 85852 ██████ |██████████▋ | hs/space-bslc8-foldlx-2: 85 85852 ██████ |██████████▋ | hs/space-bslc8-foldr-1: 297 169680 █████████████████████|█████████████████████| hs/space-bslc8-foldr-2: 297 169676 █████████████████████|█████████████████████| hs/space-bslc8-lenfil-1: 115 109388 ████████▏ |█████████████▌ | hs/space-bslc8-lenfil-2: 115 109392 ████████▏ |█████████████▌ | hs/space-bsl---foldlx: 85 85864 ██████ |██████████▋ | hs/space-xxxxx-acc-1: 1 1900 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1904 ▏ |▎ | hs/space-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1888 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - - ghc-thorough-head-bs0902.tgz ------- hs/byte-bs----acc: 1 147780 ▏ |██████████████████▌ | hs/byte-bs----foldlx: 1 147780 ▏ |██████████████████▌ | hs/byte-bs----foldrx: 1 147784 ▏ |██████████████████▌ | hs/byte-bsl---acc: 2 3216 ▎ |▍ | hs/byte-xxxxx-acc-1: 1 1900 ▏ |▎ | hs/byte-xxxxx-acc-2: 1 1896 ▏ |▎ | hs/byte-xxxxx-foldl: 1 1896 ▏ |▎ | hs/space-bs-c8-acc-1: 1 147752 ▏ |██████████████████▌ | hs/space-bs-c8-count: 1 147776 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-1: 1 147752 ▏ |██████████████████▌ | hs/space-bs-c8-foldlx-2: 1 147756 ▏ |██████████████████▌ | hs/space-bs-c8-foldrx: 1 147756 ▏ |██████████████████▌ | hs/space-bs-c8-lenfil: 21 167320 ███ |█████████████████████| hs/space-bslc8-acc-1: 2 3196 ▎ |▍ | hs/space-bslc8-acc-2: 2 3200 ▎ |▍ | hs/space-bslc8-acc-3: 2 3208 ▎ |▍ | hs/space-bslc8-chunk-1: 65 66224 █████████▏ |████████▎ | hs/space-bslc8-chunk-2: 65 66228 █████████▏ |████████▎ | hs/space-bslc8-chunk-3: 65 66224 █████████▏ |████████▎ | hs/space-bslc8-chunk-4: 78 79204 ███████████ |██████████ | hs/space-bslc8-count: 81 82224 ███████████▍ |██████████▍ | hs/space-bslc8-foldl: 78 79196 ███████████ |██████████ | hs/space-bslc8-foldlx-1: 78 79196 ███████████ |██████████ | hs/space-bslc8-foldlx-2: 78 79196 ███████████ |██████████ | hs/space-bslc8-foldr-1: 149 149680 █████████████████████|██████████████████▊ | hs/space-bslc8-foldr-2: 149 149676 █████████████████████|██████████████████▊ | hs/space-bslc8-lenfil-1: 81 82304 ███████████▍ |██████████▍ | hs/space-bslc8-lenfil-2: 81 82300 ███████████▍ |██████████▍ | hs/space-bsl---foldlx: 78 79208 ███████████ |██████████ | hs/space-xxxxx-acc-1: 1 1904 ▏ |▎ | hs/space-xxxxx-acc-2: 1 1900 ▏ |▎ | hs/space-xxxxx-foldl: 1 1904 ▏ |▎ | hs/space-xxxxx-lenfil: 1 1888 ▏ |▎ | c/byte-getchar: - - c/byte-getchar-u: - - c/byte-4k: - - c/space-getchar: - - c/space-getchar-u: - - c/space-4k: - - c/space-megabuf: - - c/space-getwchar: - - c/space-getwchar-u: - - c/space-32k: - - c/space-32k-8: - -

1 0

[12/16] SBM: Graphs for 7 ghc/bytestring combinations on a 2GHz Athlon64 300+
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This report combines the measurements from 7 ghc/bytestring combinations on a single machine, namely: ghc 6.6.1 ghc 6.8.2 ghc 6.8.2 + bytestring 0.9.0.2 ghc 6.9.20071119 ghc 6.9.20071119 + bytestring 0.9.0.2 ghc head-as-of-noon-2007-12-19 ghc head-as-of-noon-2007-12-19 + bytestring 0.9.0.2 This is the command I ran to generate the report: RESCALE=1 EXCLUDE='(^hand/|^c/)' \ tools/merge.pl \ ghc-thorough-6.6.1.tgz \ ghc-thorough-6.8.2.tgz ghc-thorough-6.8.2-bs0902.tgz \ ghc-thorough-6.9.tgz ghc-thorough-6.9-bs0902.tgz \ ghc-thorough-head.tgz ghc-thorough-head-bs0902.tgz \ > mergerep-amdy.txt We can readily see that there are some memory leaks with ghc 6.9.x. We can also see that Don Stewart's fix in bytestring 0.9.0.2 actually fixed things for hs/space-bslc8-lenfil-1 on ghc 6.8.2 but only improved things slightly for 6.9.x. It also makes things worse for hs/space-bs-c8-lenfil on 6.8.2 and both the 6.9.x's. Lazy bytestrings are slow, when used "manually", but quite acceptable, memory- wise. On the other hand, they cost dearly in memory on 6.9.x unless one does it all "by hand". Good old native strings seem to be the most predictable of the bunch but they are a bit slow. Surprisingly, they are not always /the/ slowest option measured! hs/space-bslc8-acc-1 and hs/space-bslc8-acc-2 are actually slower on ghc 6.6.1 than the native strings are in all combinations except hs/space-xxxxx-lenfil. -Peter PS: Note that this is the same report that I sent yesterday, except that it had the order of some of the benchmarks slightly wrong. charybdis ghc 6.6.1 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.8.2 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.8.2 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX=-bs0.9.0.2 charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX=-bs0.9.0.2 charybdis ghc 6.9.20071217 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071217 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX=-bs0.9.0.2 Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 1.020 40‰ 0.0 █████▏ | -- 0.703 5‰ 0.4 ███▌ | -- 0.702 7‰ 0.3 ███▌ | -- 0.705 7‰ 0.1 ███▋ | -- 0.712 3‰ 0.1 ███▋ | -- 0.706 7‰ 0.1 ███▋ | -- 0.707 7‰ 0.9 ███▋ | hs/byte-bs----foldlx: 0.789 3‰ 0.3 ████ | -- 0.993 2‰ 0.2 █████ | -- 1.102 1‰ 0.2 █████▋ | -- 1.002 1‰ 0.5 █████▏ | -- 1.112 1‰ 0.3 █████▋ | -- 1.024 2‰ 0.2 █████▏ | -- 1.111 1‰ 0.1 █████▋ | hs/byte-bs----foldrx: 0.813 3‰ 0.1 ████▏ | -- 1.102 2‰ 0.3 █████▋ | -- 1.100 1‰ 0.1 █████▋ | -- 1.112 2‰ 0.1 █████▋ | -- 1.114 1‰ 0.4 █████▋ | -- 1.113 2‰ 0.5 █████▋ | -- 1.112 1‰ 0.2 █████▋ | hs/byte-bsl---acc: 3.599 13‰ 0.0 ██████████████████▎ | -- 2.609 17‰ 0.0 █████████████▎ | -- 2.560 15‰ 0.1 █████████████ | -- 2.595 14‰ 0.1 █████████████▏ | -- 2.574 16‰ 0.1 █████████████ | -- 2.613 12‰ 0.2 █████████████▎ | -- 2.656 44‰ 0.1 █████████████▌ | hs/byte-xxxxx-acc-1: 4.606 5‰ 0.0 ███████████████████████▍ | -- 4.730 3‰ 0.0 ████████████████████████ | -- 4.722 3‰ 0.1 ███████████████████████▉ | -- 4.436 5‰ 0.0 ██████████████████████▌ | -- 4.500 6‰ 0.1 ██████████████████████▊ | -- 4.462 4‰ 0.1 ██████████████████████▋ | -- 4.499 7‰ 0.1 ██████████████████████▊ | hs/byte-xxxxx-acc-2: 4.595 5‰ 0.0 ███████████████████████▎ | -- 4.704 1‰ 0.0 ███████████████████████▉ | -- 4.722 6‰ 0.0 ███████████████████████▉ | -- 4.473 8‰ 0.0 ██████████████████████▋ | -- 4.493 5‰ 0.0 ██████████████████████▊ | -- 4.477 3‰ 0.1 ██████████████████████▋ | -- 4.446 2‰ 0.0 ██████████████████████▌ | hs/byte-xxxxx-foldl: 4.566 9‰ 0.0 ███████████████████████▏ | -- 4.754 6‰ 0.1 ████████████████████████▏ | -- 4.735 7‰ 0.0 ████████████████████████ | -- 4.412 10‰ 0.0 ██████████████████████▍ | -- 4.486 3‰ 0.0 ██████████████████████▊ | -- 4.473 3‰ 0.0 ██████████████████████▋ | -- 4.511 10‰ 0.1 ██████████████████████▉ | Memory: Peak ------- KB hs/byte-bs----acc: 147448 ████████████████████████████████████████ | -- 147500 ████████████████████████████████████████ | -- 147616 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | -- 147732 ████████████████████████████████████████ | -- 147772 ████████████████████████████████████████ | -- 147780 ████████████████████████████████████████ | hs/byte-bs----foldlx: 147468 ████████████████████████████████████████ | -- 147528 ████████████████████████████████████████ | -- 147624 ████████████████████████████████████████ | -- 147748 ████████████████████████████████████████ | -- 147736 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147780 ████████████████████████████████████████ | hs/byte-bs----foldrx: 147452 ████████████████████████████████████████ | -- 147500 ████████████████████████████████████████ | -- 147624 ████████████████████████████████████████ | -- 147744 ████████████████████████████████████████ | -- 147740 ████████████████████████████████████████ | -- 147772 ████████████████████████████████████████ | -- 147784 ████████████████████████████████████████ | hs/byte-bsl---acc: 2872 ▊ | -- 2920 ▊ | -- 3040 ▉ | -- 3172 ▉ | -- 3184 ▉ | -- 3212 ▉ | -- 3216 ▉ | hs/byte-xxxxx-acc-1: 1572 ▍ | -- 1632 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1900 ▌ | hs/byte-xxxxx-acc-2: 1568 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1900 ▌ | -- 1896 ▌ | hs/byte-xxxxx-foldl: 1572 ▍ | -- 1632 ▌ | -- 1636 ▌ | -- 1900 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1896 ▌ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 1.287 20‰ 0.2 ██████▌ | -- 1.131 2‰ 0.1 █████▊ | -- 1.132 2‰ 0.3 █████▊ | -- 1.145 1‰ 0.2 █████▊ | -- 1.150 1‰ 0.2 █████▉ | -- 1.147 2‰ 0.1 █████▊ | -- 1.147 2‰ 0.2 █████▊ | hs/space-bs-c8-count: 0.770 2‰ 0.0 ███▉ | -- 0.711 1‰ 0.4 ███▋ | -- 0.513 2‰ 0.2 ██▋ | -- 0.521 1‰ 0.2 ██▋ | -- 0.523 3‰ 0.2 ██▋ | -- 0.522 5‰ 0.2 ██▋ | -- 0.523 3‰ 0.6 ██▋ | hs/space-bs-c8-foldlx-1: 1.272 1‰ 0.2 ██████▌ | -- 1.205 2‰ 0.2 ██████▏ | -- 1.164 3‰ 0.0 █████▉ | -- 1.221 1‰ 0.1 ██████▏ | -- 1.176 1‰ 0.3 ██████ | -- 1.259 1‰ 0.1 ██████▍ | -- 1.176 1‰ 0.2 ██████ | hs/space-bs-c8-foldlx-2: 1.270 1‰ 0.2 ██████▍ | -- 1.207 1‰ 0.2 ██████▏ | -- 1.165 2‰ 0.1 █████▉ | -- 1.219 2‰ 0.2 ██████▏ | -- 1.176 1‰ 0.3 ██████ | -- 1.258 1‰ 0.2 ██████▍ | -- 1.177 1‰ 0.2 ██████ | hs/space-bs-c8-foldrx: 1.064 2‰ 0.1 █████▍ | -- 1.168 2‰ 0.2 █████▉ | -- 1.164 1‰ 0.3 █████▉ | -- 1.172 2‰ 0.3 █████▉ | -- 1.177 1‰ 0.2 ██████ | -- 1.176 1‰ 0.2 ██████ | -- 1.176 1‰ 0.1 ██████ | hs/space-bs-c8-lenfil: 1.271 1‰ 0.1 ██████▍ | -- 1.218 1‰ 0.2 ██████▏ | -- 0.541 3‰ 0.2 ██▊ | -- 1.223 1‰ 0.2 ██████▎ | -- 0.549 1‰ 0.4 ██▊ | -- 1.225 1‰ 0.2 ██████▎ | -- 0.551 5‰ 0.4 ██▊ | hs/space-bslc8-acc-1: 6.367 17‰ 0.2 ████████████████████████████████▎ | -- 3.396 7‰ 0.1 █████████████████▎ | -- 3.508 4‰ 0.1 █████████████████▊ | -- 3.388 10‰ 0.1 █████████████████▏ | -- 3.441 10‰ 0.1 █████████████████▍ | -- 3.438 3‰ 0.0 █████████████████▍ | -- 3.434 13‰ 0.1 █████████████████▍ | hs/space-bslc8-acc-2: 6.150 21‰ 0.0 ███████████████████████████████▏ | -- 3.470 5‰ 0.0 █████████████████▋ | -- 3.439 10‰ 0.1 █████████████████▍ | -- 3.386 5‰ 0.1 █████████████████▏ | -- 3.479 11‰ 0.1 █████████████████▋ | -- 3.467 6‰ 0.1 █████████████████▌ | -- 3.470 10‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-3: 4.041 21‰ 0.1 ████████████████████▌ | -- 2.959 13‰ 0.0 ███████████████ | -- 2.920 16‰ 0.1 ██████████████▊ | -- 2.832 4‰ 0.1 ██████████████▍ | -- 2.925 14‰ 0.1 ██████████████▊ | -- 2.987 9‰ 0.0 ███████████████▏ | -- 2.996 20‰ 0.1 ███████████████▏ | hs/space-bslc8-chunk-1: n/a -- 1.054 0‰ 0.3 █████▍ | -- 1.053 1‰ 0.1 █████▍ | -- 1.094 0‰ 0.3 █████▌ | -- 1.088 0‰ 0.0 █████▌ | -- 1.096 1‰ 0.1 █████▌ | -- 1.089 0‰ 0.6 █████▌ | hs/space-bslc8-chunk-2: n/a -- 1.055 0‰ 0.3 █████▍ | -- 1.053 0‰ 0.1 █████▍ | -- 1.094 1‰ 0.3 █████▌ | -- 1.088 1‰ 0.2 █████▌ | -- 1.096 1‰ 0.2 █████▌ | -- 1.089 0‰ 0.1 █████▌ | hs/space-bslc8-chunk-3: n/a -- 1.190 4‰ 0.2 ██████ | -- 1.199 1‰ 0.3 ██████▏ | -- 1.087 1‰ 0.3 █████▌ | -- 1.051 1‰ 0.4 █████▍ | -- 1.077 1‰ 0.1 █████▌ | -- 1.053 1‰ 0.2 █████▍ | hs/space-bslc8-chunk-4: n/a -- 1.054 0‰ 0.2 █████▍ | -- 1.054 0‰ 0.2 █████▍ | -- 1.099 0‰ 0.3 █████▋ | -- 1.097 1‰ 0.1 █████▌ | -- 1.094 1‰ 0.2 █████▌ | -- 1.096 0‰ 0.0 █████▌ | hs/space-bslc8-count: 0.713 1‰ 0.3 ███▋ | -- 0.735 1‰ 0.4 ███▊ | -- 0.431 0‰ 0.7 ██▏ | -- 0.477 2‰ 0.4 ██▍ | -- 0.482 1‰ 0.4 ██▌ | -- 0.479 1‰ 0.6 ██▍ | -- 0.482 2‰ 0.4 ██▌ | hs/space-bslc8-foldl: 1.431 1‰ 0.3 ███████▎ | -- 1.398 1‰ 0.2 ███████▏ | -- 1.197 1‰ 0.2 ██████ | -- 1.458 1‰ 0.3 ███████▍ | -- 1.059 0‰ 0.0 █████▍ | -- 1.459 1‰ 0.1 ███████▍ | -- 1.061 0‰ 0.4 █████▍ | hs/space-bslc8-foldlx-1: 1.430 1‰ 0.1 ███████▎ | -- 1.400 1‰ 0.1 ███████▏ | -- 1.197 1‰ 0.0 ██████ | -- 1.458 0‰ 0.1 ███████▍ | -- 1.060 1‰ 0.0 █████▍ | -- 1.457 1‰ 0.1 ███████▍ | -- 1.060 1‰ 0.2 █████▍ | hs/space-bslc8-foldlx-2: 1.431 1‰ 0.1 ███████▎ | -- 1.399 1‰ 0.1 ███████▏ | -- 1.196 1‰ 0.1 ██████ | -- 1.457 1‰ 0.1 ███████▍ | -- 1.060 1‰ 0.3 █████▍ | -- 1.458 1‰ 0.1 ███████▍ | -- 1.060 1‰ 0.0 █████▍ | hs/space-bslc8-foldr-1: 1.748 1‰ 0.0 ████████▉ | -- 1.579 2‰ 0.1 ████████ | -- 1.220 2‰ 0.2 ██████▏ | -- 1.312 1‰ 0.2 ██████▋ | -- 1.213 1‰ 0.2 ██████▏ | -- 1.325 4‰ 0.2 ██████▊ | -- 1.213 2‰ 0.2 ██████▏ | hs/space-bslc8-foldr-2: 1.745 1‰ 0.1 ████████▉ | -- 1.579 2‰ 0.1 ████████ | -- 1.220 1‰ 0.2 ██████▏ | -- 1.315 1‰ 0.2 ██████▋ | -- 1.210 1‰ 0.2 ██████▏ | -- 1.317 4‰ 0.2 ██████▋ | -- 1.213 2‰ 0.2 ██████▏ | hs/space-bslc8-lenfil-1: 1.521 7‰ 0.2 ███████▊ | -- 1.295 1‰ 0.2 ██████▌ | -- 0.429 0‰ 0.2 ██▏ | -- 1.323 1‰ 0.1 ██████▊ | -- 0.483 2‰ 0.4 ██▌ | -- 1.327 1‰ 0.2 ██████▊ | -- 0.482 2‰ 0.4 ██▌ | hs/space-bslc8-lenfil-2: 1.532 6‰ 0.1 ███████▊ | -- 1.296 0‰ 0.0 ██████▌ | -- 0.429 0‰ 0.2 ██▏ | -- 1.325 1‰ 0.1 ██████▊ | -- 0.483 1‰ 0.6 ██▌ | -- 1.327 1‰ 0.2 ██████▊ | -- 0.482 2‰ 0.4 ██▌ | hs/space-bsl---foldlx: 1.288 1‰ 0.2 ██████▌ | -- 1.333 1‰ 0.0 ██████▊ | -- 1.168 1‰ 0.2 █████▉ | -- 1.391 1‰ 0.1 ███████ | -- 1.006 1‰ 0.3 █████▏ | -- 1.395 1‰ 0.1 ███████▏ | -- 1.008 1‰ 0.2 █████▏ | hs/space-xxxxx-acc-1: 5.036 7‰ 0.1 █████████████████████████▌ | -- 5.381 6‰ 0.0 ███████████████████████████▎ | -- 5.358 8‰ 0.1 ███████████████████████████▏ | -- 4.991 1‰ 0.1 █████████████████████████▎ | -- 5.047 7‰ 0.1 █████████████████████████▌ | -- 5.051 3‰ 0.1 █████████████████████████▋ | -- 5.049 4‰ 0.1 █████████████████████████▋ | hs/space-xxxxx-acc-2: 5.009 7‰ 0.1 █████████████████████████▍ | -- 5.437 4‰ 0.1 ███████████████████████████▌ | -- 5.406 2‰ 0.0 ███████████████████████████▍ | -- 4.985 1‰ 0.0 █████████████████████████▎ | -- 5.057 4‰ 0.1 █████████████████████████▋ | -- 5.048 6‰ 0.1 █████████████████████████▌ | -- 5.089 6‰ 0.0 █████████████████████████▊ | hs/space-xxxxx-foldl: 5.024 5‰ 0.0 █████████████████████████▌ | -- 5.371 9‰ 0.0 ███████████████████████████▎ | -- 5.301 9‰ 0.1 ██████████████████████████▉ | -- 5.035 2‰ 0.1 █████████████████████████▌ | -- 5.037 3‰ 0.0 █████████████████████████▌ | -- 5.046 4‰ 0.0 █████████████████████████▌ | -- 5.042 4‰ 0.0 █████████████████████████▌ | hs/space-xxxxx-lenfil: 7.042 4‰ 0.0 ███████████████████████████████████▋ | -- 7.505 3‰ 0.0 ██████████████████████████████████████| -- 7.498 3‰ 0.0 ██████████████████████████████████████| -- 7.106 8‰ 0.1 ████████████████████████████████████ | -- 7.092 3‰ 0.1 ███████████████████████████████████▉ | -- 7.083 3‰ 0.0 ███████████████████████████████████▉ | -- 7.121 5‰ 0.1 ████████████████████████████████████ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147448 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147736 ████████████████████████████████████████ | -- 147740 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | hs/space-bs-c8-count: 147448 ████████████████████████████████████████ | -- 147500 ████████████████████████████████████████ | -- 147612 ████████████████████████████████████████ | -- 147744 ████████████████████████████████████████ | -- 147756 ████████████████████████████████████████ | -- 147764 ████████████████████████████████████████ | -- 147776 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147468 ████████████████████████████████████████ | -- 147524 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147756 ████████████████████████████████████████ | -- 147744 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147752 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-2: 147468 ████████████████████████████████████████ | -- 147520 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147752 ████████████████████████████████████████ | -- 147748 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147756 ████████████████████████████████████████ | hs/space-bs-c8-foldrx: 147448 ████████████████████████████████████████ | -- 147496 ████████████████████████████████████████ | -- 147608 ████████████████████████████████████████ | -- 147732 ████████████████████████████████████████ | -- 147748 ████████████████████████████████████████ | -- 147760 ████████████████████████████████████████ | -- 147756 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147464 ████████████████████████████████████████ | -- 147524 ████████████████████████████████████████ | -- 167176 █████████████████████████████████████████████▍| -- 147756 ████████████████████████████████████████ | -- 167300 █████████████████████████████████████████████▍| -- 147788 ████████████████████████████████████████ | -- 167320 █████████████████████████████████████████████▍| hs/space-bslc8-acc-1: 2872 ▊ | -- 2920 ▊ | -- 3048 ▉ | -- 3192 ▉ | -- 3228 ▉ | -- 3180 ▉ | -- 3196 ▉ | hs/space-bslc8-acc-2: 2872 ▊ | -- 2924 ▊ | -- 3048 ▉ | -- 3192 ▉ | -- 3228 ▉ | -- 3180 ▉ | -- 3200 ▉ | hs/space-bslc8-acc-3: 2868 ▊ | -- 2924 ▊ | -- 3052 ▉ | -- 3196 ▉ | -- 3232 ▉ | -- 3172 ▉ | -- 3208 ▉ | hs/space-bslc8-chunk-1: n/a -- 1512 ▍ | -- 1636 ▌ | -- 65192 █████████████████▋ | -- 66216 ██████████████████ | -- 65204 █████████████████▋ | -- 66224 ██████████████████ | hs/space-bslc8-chunk-2: n/a -- 1512 ▍ | -- 1640 ▌ | -- 65192 █████████████████▋ | -- 66216 ██████████████████ | -- 65204 █████████████████▋ | -- 66228 ██████████████████ | hs/space-bslc8-chunk-3: n/a -- 1548 ▍ | -- 1644 ▌ | -- 75240 ████████████████████▍ | -- 66208 ██████████████████ | -- 75216 ████████████████████▍ | -- 66224 ██████████████████ | hs/space-bslc8-chunk-4: n/a -- 1512 ▍ | -- 1636 ▌ | -- 76188 ████████████████████▋ | -- 79232 █████████████████████▌ | -- 76176 ████████████████████▋ | -- 79204 █████████████████████▌ | hs/space-bslc8-count: 1476 ▍ | -- 1520 ▍ | -- 1656 ▌ | -- 79204 █████████████████████▌ | -- 82248 ██████████████████████▎ | -- 79220 █████████████████████▌ | -- 82224 ██████████████████████▎ | hs/space-bslc8-foldl: 1552 ▍ | -- 1620 ▌ | -- 1632 ▌ | -- 85868 ███████████████████████▎ | -- 79224 █████████████████████▌ | -- 85852 ███████████████████████▎ | -- 79196 █████████████████████▌ | hs/space-bslc8-foldlx-1: 1556 ▍ | -- 1620 ▌ | -- 1636 ▌ | -- 85864 ███████████████████████▎ | -- 79224 █████████████████████▌ | -- 85852 ███████████████████████▎ | -- 79196 █████████████████████▌ | hs/space-bslc8-foldlx-2: 1552 ▍ | -- 1616 ▌ | -- 1636 ▌ | -- 85868 ███████████████████████▎ | -- 79224 █████████████████████▌ | -- 85852 ███████████████████████▎ | -- 79196 █████████████████████▌ | hs/space-bslc8-foldr-1: 169480 ██████████████████████████████████████████████| -- 169544 ██████████████████████████████████████████████| -- 149076 ████████████████████████████████████████▍ | -- 169700 ██████████████████████████████████████████████| -- 149704 ████████████████████████████████████████▋ | -- 169680 ██████████████████████████████████████████████| -- 149680 ████████████████████████████████████████▋ | hs/space-bslc8-foldr-2: 169480 ██████████████████████████████████████████████| -- 169544 ██████████████████████████████████████████████| -- 149076 ████████████████████████████████████████▍ | -- 169700 ██████████████████████████████████████████████| -- 149708 ████████████████████████████████████████▋ | -- 169676 ██████████████████████████████████████████████| -- 149676 ████████████████████████████████████████▋ | hs/space-bslc8-lenfil-1: 38632 ██████████▌ | -- 38644 ██████████▌ | -- 1940 ▌ | -- 109404 █████████████████████████████▋ | -- 82324 ██████████████████████▍ | -- 109388 █████████████████████████████▋ | -- 82304 ██████████████████████▎ | hs/space-bslc8-lenfil-2: 38628 ██████████▌ | -- 38648 ██████████▌ | -- 1940 ▌ | -- 109404 █████████████████████████████▋ | -- 82328 ██████████████████████▍ | -- 109392 █████████████████████████████▋ | -- 82300 ██████████████████████▎ | hs/space-bsl---foldlx: 1560 ▍ | -- 1616 ▌ | -- 1620 ▌ | -- 85836 ███████████████████████▎ | -- 79184 █████████████████████▌ | -- 85864 ███████████████████████▎ | -- 79208 █████████████████████▌ | hs/space-xxxxx-acc-1: 1568 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1904 ▌ | hs/space-xxxxx-acc-2: 1572 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1904 ▌ | -- 1900 ▌ | -- 1904 ▌ | -- 1900 ▌ | hs/space-xxxxx-foldl: 1572 ▍ | -- 1636 ▌ | -- 1636 ▌ | -- 1900 ▌ | -- 1904 ▌ | -- 1904 ▌ | -- 1904 ▌ | hs/space-xxxxx-lenfil: 1556 ▍ | -- 1608 ▍ | -- 1604 ▍ | -- 1892 ▌ | -- 1892 ▌ | -- 1888 ▌ | -- 1888 ▌ |

1 0

[11/16] SBM: Graphs for hand-tweaked assembly benchmarks
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This report compares the hand-tweaked assembly programs with the original untweaked programs on two vastly different microarchitectures. This is the command I ran to generate the report: EXCLUDE='(xxxx|-bsl|chunk|count|acc-[23]|fold|lenfil|^c/)' \ tools/merge.pl \ ghc-armada-thorough-6.9.tgz \ ghc-thorough-6.9.tgz \ > xx I cut out the memory sections manually since we've already seen them and inserted a few newlines for grouping purposes. The first one should note is that not all tweaks are better than the originals! The second is that the sequence of tweaks is not quite monotonically decreasing in run-time. The improvements don't really start until -e on the Athlon64 and -f on both. Not until then have the load pressure been sufficiently relieved on the L1 cache that the code actually runs faster. Note also how the two microarchitectures seem to have plateaus in different places. The Athlon64 seems to have the number 3 built into its silicon (efg, jkl, mno) which fits very well with what we know about it from AMD's documentation (the front end splits the instructions up into smaller pieces which then get distributed to three different "pipelines", each with its own out-of-order execution engine). The Pentium III seems to have trouble with the simple MMX code but does very well with the more advanced MMX code that keeps 8 space counters in a single MMX register for many iterations. The code I used to add those counters horizontally is the same in both -q and -r. Perhaps operations on both MMX and normal registers are slow? Loop unrolling (-s) doesn't seem to matter, in this case. -Peter ls-search ghc 6.9.20071119 Pentium III (Coppermine) 596.932 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 3.274 1‰ 0.1 ███████████████████████████ | -- 0.705 7‰ 0.1 █████████████████████▋ | hand/byte-bs----acc-a: 3.511 1‰ 0.0 ████████████████████████████▉ | -- 0.639 2‰ 0.2 ███████████████████▋ | hand/byte-bs----acc-b: 1.998 2‰ 0.1 ████████████████▌ | -- 0.414 2‰ 0.5 ████████████▊ | hand/byte-bs----acc-c: 1.876 2‰ 0.1 ███████████████▌ | -- 0.414 3‰ 0.2 ████████████▊ | hand/byte-bs----acc-d: 1.876 1‰ 0.1 ███████████████▌ | -- 0.415 3‰ 0.2 ████████████▊ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 4.318 1‰ 0.0 ███████████████████████████████████▋ | -- 1.145 1‰ 0.2 ███████████████████████████████████▏ | hand/space-bs-c8-acc-1-a: 4.318 1‰ 0.0 ███████████████████████████████████▋ | -- 1.177 2‰ 0.3 ████████████████████████████████████▏| hand/space-bs-c8-acc-1-b: 4.331 1‰ 0.0 ███████████████████████████████████▋ | -- 1.104 1‰ 0.2 █████████████████████████████████▉ | hand/space-bs-c8-acc-1-c: 4.492 1‰ 0.1 █████████████████████████████████████| -- 1.207 1‰ 0.3 █████████████████████████████████████| hand/space-bs-c8-acc-1-d: 4.354 1‰ 0.0 ███████████████████████████████████▉ | -- 1.191 1‰ 0.2 ████████████████████████████████████▌| hand/space-bs-c8-acc-1-e: 4.424 0‰ 0.1 ████████████████████████████████████▌| -- 0.937 1‰ 0.2 ████████████████████████████▊ | hand/space-bs-c8-acc-1-f: 4.164 1‰ 0.0 ██████████████████████████████████▎ | -- 0.921 1‰ 0.2 ████████████████████████████▎ | hand/space-bs-c8-acc-1-g: 4.309 1‰ 0.1 ███████████████████████████████████▌ | -- 0.927 2‰ 0.4 ████████████████████████████▍ | hand/space-bs-c8-acc-1-h: 4.202 1‰ 0.1 ██████████████████████████████████▋ | -- 0.886 2‰ 0.2 ███████████████████████████▏ | hand/space-bs-c8-acc-1-i: 3.820 1‰ 0.1 ███████████████████████████████▌ | -- 0.803 3‰ 0.4 ████████████████████████▋ | hand/space-bs-c8-acc-1-j: 3.472 1‰ 0.0 ████████████████████████████▋ | -- 0.706 2‰ 0.1 █████████████████████▋ | hand/space-bs-c8-acc-1-k: 3.474 1‰ 0.0 ████████████████████████████▋ | -- 0.705 1‰ 0.0 █████████████████████▋ | hand/space-bs-c8-acc-1-l: 3.498 1‰ 0.1 ████████████████████████████▉ | -- 0.710 2‰ 0.1 █████████████████████▊ | hand/space-bs-c8-acc-1-m: 3.397 1‰ 0.1 ████████████████████████████ | -- 0.642 6‰ 0.3 ███████████████████▋ | hand/space-bs-c8-acc-1-n: 3.373 1‰ 0.0 ███████████████████████████▊ | -- 0.636 4‰ 0.5 ███████████████████▌ | hand/space-bs-c8-acc-1-o: 3.118 1‰ 0.1 █████████████████████████▋ | -- 0.626 2‰ 0.0 ███████████████████▎ | hand/space-bs-c8-acc-1-p: 2.935 2‰ 0.0 ████████████████████████▏ | -- 0.565 3‰ 0.4 █████████████████▍ | hand/space-bs-c8-acc-1-q: 3.477 1‰ 0.1 ████████████████████████████▋ | -- 0.418 6‰ 0.7 ████████████▉ | hand/space-bs-c8-acc-1-r: 1.674 1‰ 0.1 █████████████▊ | -- 0.334 5‰ 0.6 ██████████▎ | hand/space-bs-c8-acc-1-s: 1.627 1‰ 0.2 █████████████▍ | -- 0.335 4‰ 0.9 ██████████▎ |

1 0

[10/16] SBM: Graphs for 6.9.x across four cpus
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This is what you get if you merge the previous four reports (and filter out the hand-tweaked assembly benchmarks). I generated the report with the following command: EXCLUDE='^hand/' \ tools/merge.pl \ ghc-armada-thorough-6.9.tgz \ ghc-fischer-thorough-6.9.tgz \ ghc-albatross-thorough-6.9.tgz \ ghc-thorough-6.9.tgz \ > xx This graph shows the memory usage to be (almost) exactly the same. The difference can easily be explained with slightly different versions of the C library (there was also a security update or two from Ubuntu in the last few days - the kernel was definitely updated and I think the C library was too). We are also talking about two versions of Ubuntu and one (old) version of SuSE. And we are talking about three not quite identical versions of ghc. The fischer and albatross runs were made before the inclusion of the three benchmarks from Don Stewart (hs/space-bs-c8-count, hs/space-bslc8-count, and hs/space-bslc8-chunk-4) so they are a couple of holes in the graphs. The speed pattern is more fun. It makes no sense to compare absolute times here so the graphs were not rescaled. One would naïvely expect the bars to be about the same when taken in groups of four but that really turns out to be far from the case! The explanation is most likely that we are looking at four very different microarchitectures. ********** This should hammer home the point that benchmarking on any single machine isn't enough! ********** Two probably aren't enough either... -Peter ls-search ghc 6.9.20071119 Pentium III (Coppermine) 596.932 MHz TESTKIND=THOROUGH SUFFIX= Fischer's machine ghc 6.9.20071124 AMD Duron(tm) processor 1200.089 MHz TESTKIND=THOROUGH SUFFIX= albatross ghc 6.9.20071208 (or thereabouts) Genuine Intel(R) CPU T2300 @ 1.66GHz (Core Duo) 1667.000 MHz TESTKIND=THOROUGH SUFFIX= charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 3.274 1‰ 0.1 ███▏ | -- 1.892 21‰ 0.4 ██▊ | -- 0.918 4‰ 0.1 █████▍ | -- 0.705 7‰ 0.1 ███▋ | hs/byte-bs----foldlx: 4.027 0‰ 0.0 ███▊ | -- 2.258 3‰ 0.1 ███▎ | -- 1.014 2‰ 0.2 ██████ | -- 1.002 1‰ 0.5 █████▎ | hs/byte-bs----foldrx: 4.184 1‰ 0.0 ███▉ | -- 2.933 0‰ 0.1 ████▎ | -- 0.999 2‰ 0.1 █████▉ | -- 1.112 2‰ 0.1 █████▊ | hs/byte-bsl---acc: 28.005 10‰ 0.0 ██████████████████████████▍ | -- 14.319 45‰ 0.1 ████████████████████▌ | -- 1.957 1‰ 0.1 ███████████▌ | -- 2.595 14‰ 0.1 █████████████▌ | hs/byte-xxxxx-acc-1: 25.852 4‰ 0.0 ████████████████████████▎ | -- 20.915 17‰ 4.0 █████████████████████████████▉ | -- 3.591 2‰ 0.1 █████████████████████▏ | -- 4.436 5‰ 0.0 ███████████████████████▏ | hs/byte-xxxxx-acc-2: 25.622 5‰ 0.0 ████████████████████████▏ | -- 20.691 8‰ 0.1 █████████████████████████████▌ | -- 3.598 4‰ 0.1 █████████████████████▏ | -- 4.473 8‰ 0.0 ███████████████████████▎ | hs/byte-xxxxx-foldl: 25.803 5‰ 0.0 ████████████████████████▎ | -- 20.610 5‰ 1.4 █████████████████████████████▍ | -- 3.594 1‰ 0.1 █████████████████████▏ | -- 4.412 10‰ 0.0 ███████████████████████ | c/byte-getchar: 13.016 0‰ 0.0 ████████████▎ | -- 9.042 0‰ 0.1 ████████████▉ | -- 3.761 2‰ 0.1 ██████████████████████▏ | -- 2.422 27‰ 0.1 ████████████▋ | c/byte-getchar-u: 1.662 1‰ 0.1 █▋ | -- 1.314 3‰ 0.2 █▉ | -- 0.719 2‰ 0.3 ████▎ | -- 0.632 3‰ 0.3 ███▎ | c/byte-4k: 0.543 2‰ 0.2 ▌ | -- 0.419 5‰ 0.5 ▋ | -- 0.101 10‰ 3.1 ▋ | -- 0.094 26‰ 2.2 ▌ | Memory: Peak ------- KB hs/byte-bs----acc: 147752 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147784 ████████████████████████████████████████▏ | -- 147752 ████████████████████████████████████████ | hs/byte-bs----foldlx: 147756 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147812 ████████████████████████████████████████▏ | -- 147748 ████████████████████████████████████████ | hs/byte-bs----foldrx: 147760 ████████████████████████████████████████ | -- 147488 ████████████████████████████████████████ | -- 147788 ████████████████████████████████████████▏ | -- 147744 ████████████████████████████████████████ | hs/byte-bsl---acc: 3180 ▉ | -- 2896 ▊ | -- 3228 ▉ | -- 3172 ▉ | hs/byte-xxxxx-acc-1: 1916 ▌ | -- 1612 ▌ | -- 1904 ▌ | -- 1904 ▌ | hs/byte-xxxxx-acc-2: 1912 ▌ | -- 1612 ▌ | -- 1908 ▌ | -- 1904 ▌ | hs/byte-xxxxx-foldl: 1912 ▌ | -- 1612 ▌ | -- 1908 ▌ | -- 1900 ▌ | c/byte-getchar: 436 ▏ | -- 384 ▏ | -- 432 ▏ | -- 440 ▏ | c/byte-getchar-u: 432 ▏ | -- 384 ▏ | -- 440 ▏ | -- 440 ▏ | c/byte-4k: 436 ▏ | -- 380 ▏ | -- 436 ▏ | -- 440 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 4.318 1‰ 0.0 ████ | -- 2.467 1‰ 0.3 ███▌ | -- 1.313 0‰ 0.2 ███████▊ | -- 1.145 1‰ 0.2 ██████ | hs/space-bs-c8-count: 3.118 1‰ 0.1 ██▉ | -- n/a -- n/a -- 0.521 1‰ 0.2 ██▊ | hs/space-bs-c8-foldlx-1: 4.631 1‰ 0.0 ████▍ | -- 2.585 2‰ 0.1 ███▊ | -- 1.326 1‰ 0.1 ███████▊ | -- 1.221 1‰ 0.1 ██████▍ | hs/space-bs-c8-foldlx-2: 4.632 1‰ 0.0 ████▍ | -- 2.576 2‰ 0.3 ███▋ | -- 1.328 1‰ 0.1 ███████▉ | -- 1.219 2‰ 0.2 ██████▍ | hs/space-bs-c8-foldrx: 4.678 0‰ 0.0 ████▍ | -- 2.982 8‰ 2.3 ████▎ | -- 1.128 1‰ 0.2 ██████▋ | -- 1.172 2‰ 0.3 ██████▏ | hs/space-bs-c8-lenfil: 4.634 1‰ 0.1 ████▍ | -- 2.599 1‰ 0.2 ███▊ | -- 1.327 2‰ 0.2 ███████▊ | -- 1.223 1‰ 0.2 ██████▍ | hs/space-bslc8-acc-1: 32.733 7‰ 0.0 ██████████████████████████████▊ | -- 15.228 8‰ 0.1 █████████████████████▊ | -- 3.002 1‰ 0.1 █████████████████▋ | -- 3.388 10‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-2: 33.112 11‰ 0.0 ███████████████████████████████▏ | -- 15.855 38‰ 0.0 ██████████████████████▋ | -- 3.010 2‰ 0.0 █████████████████▊ | -- 3.386 5‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-3: 30.294 12‰ 0.0 ████████████████████████████▌ | -- 14.980 14‰ 0.0 █████████████████████▍ | -- 2.586 1‰ 0.1 ███████████████▎ | -- 2.832 4‰ 0.1 ██████████████▊ | hs/space-bslc8-chunk-1: 3.888 0‰ 0.1 ███▋ | -- 2.443 2‰ 0.2 ███▌ | -- 1.192 0‰ 0.0 ███████ | -- 1.094 0‰ 0.3 █████▊ | hs/space-bslc8-chunk-2: 3.887 1‰ 0.0 ███▋ | -- 2.449 1‰ 0.3 ███▌ | -- 1.189 0‰ 0.1 ███████ | -- 1.094 1‰ 0.3 █████▊ | hs/space-bslc8-chunk-3: 3.881 0‰ 0.1 ███▋ | -- 2.534 3‰ 0.3 ███▋ | -- 1.272 1‰ 0.2 ███████▌ | -- 1.087 1‰ 0.3 █████▋ | hs/space-bslc8-chunk-4: 3.910 1‰ 0.0 ███▋ | -- n/a -- n/a -- 1.099 0‰ 0.3 █████▊ | hs/space-bslc8-count: 2.355 1‰ 0.1 ██▎ | -- n/a -- n/a -- 0.477 2‰ 0.4 ██▌ | hs/space-bslc8-foldl: 4.674 0‰ 0.0 ████▍ | -- 2.938 1‰ 0.2 ████▎ | -- 1.517 0‰ 0.1 ████████▉ | -- 1.458 1‰ 0.3 ███████▋ | hs/space-bslc8-foldlx-1: 4.676 0‰ 0.0 ████▍ | -- 2.928 1‰ 0.0 ████▏ | -- 1.516 0‰ 0.0 ████████▉ | -- 1.458 0‰ 0.1 ███████▋ | hs/space-bslc8-foldlx-2: 4.676 0‰ 0.0 ████▍ | -- 2.937 2‰ 0.2 ████▎ | -- 1.511 0‰ 0.0 ████████▉ | -- 1.457 1‰ 0.1 ███████▋ | hs/space-bslc8-foldr-1: 5.222 0‰ 0.0 ████▉ | -- 4.043 6‰ 0.1 █████▊ | -- 1.205 1‰ 0.2 ███████▏ | -- 1.312 1‰ 0.2 ██████▉ | hs/space-bslc8-foldr-2: 5.224 1‰ 0.1 ████▉ | -- 4.007 4‰ 0.1 █████▊ | -- 1.202 3‰ 0.1 ███████▏ | -- 1.315 1‰ 0.2 ██████▉ | hs/space-bslc8-lenfil-1: 4.712 1‰ 0.0 ████▍ | -- 3.240 1‰ 0.2 ████▋ | -- 1.400 0‰ 0.0 ████████▎ | -- 1.323 1‰ 0.1 ██████▉ | hs/space-bslc8-lenfil-2: 4.714 1‰ 0.0 ████▍ | -- 3.236 1‰ 0.2 ████▋ | -- 1.396 0‰ 0.0 ████████▎ | -- 1.325 1‰ 0.1 ██████▉ | hs/space-bsl---foldlx: 4.522 0‰ 0.1 ████▎ | -- 2.821 1‰ 0.1 ████ | -- 1.476 1‰ 0.1 ████████▋ | -- 1.391 1‰ 0.1 ███████▎ | hs/space-xxxxx-acc-1: 27.529 6‰ 0.0 █████████████████████████▉ | -- 21.002 4‰ 0.1 ██████████████████████████████ | -- 4.245 1‰ 0.0 █████████████████████████ | -- 4.991 1‰ 0.1 ██████████████████████████ | hs/space-xxxxx-acc-2: 27.200 3‰ 0.0 █████████████████████████▋ | -- 21.270 22‰ 4.7 ██████████████████████████████▍ | -- 4.241 2‰ 0.1 █████████████████████████ | -- 4.985 1‰ 0.0 ██████████████████████████ | hs/space-xxxxx-foldl: 27.436 8‰ 0.0 █████████████████████████▊ | -- 20.934 1‰ 0.1 █████████████████████████████▉ | -- 4.239 2‰ 0.1 █████████████████████████ | -- 5.035 2‰ 0.1 ██████████████████████████▎ | hs/space-xxxxx-lenfil: 39.343 3‰ 0.0 █████████████████████████████████████| -- 25.915 3‰ 0.0 █████████████████████████████████████| -- 6.287 2‰ 0.0 █████████████████████████████████████| -- 7.106 8‰ 0.1 █████████████████████████████████████| c/space-getchar: 13.268 0‰ 0.0 ████████████▌ | -- 9.354 0‰ 0.0 █████████████▍ | -- 4.031 18‰ 0.0 ███████████████████████▊ | -- 2.465 15‰ 0.1 ████████████▉ | c/space-getchar-u: 2.170 0‰ 0.1 ██ | -- 1.676 2‰ 0.2 ██▍ | -- 0.609 2‰ 0.7 ███▋ | -- 0.631 1‰ 0.5 ███▎ | c/space-4k: 1.815 1‰ 0.1 █▊ | -- 1.293 2‰ 0.5 █▉ | -- 0.465 2‰ 0.6 ██▊ | -- 0.319 3‰ 0.6 █▋ | c/space-megabuf: 2.763 2‰ 0.1 ██▋ | -- 1.830 3‰ 0.5 ██▋ | -- 0.613 1‰ 0.3 ███▋ | -- 0.515 3‰ 0.4 ██▋ | c/space-getwchar: 15.105 29‰ 0.0 ██████████████▎ | -- 14.721 1‰ 0.1 █████████████████████ | -- 4.584 1‰ 0.1 ███████████████████████████ | -- 3.956 6‰ 0.1 ████████████████████▋ | c/space-getwchar-u: 8.979 1‰ 0.0 ████████▌ | -- 4.814 0‰ 0.0 ██████▉ | -- 2.437 0‰ 0.1 ██████████████▍ | -- 2.217 1‰ 0.1 ███████████▌ | c/space-32k: 1.786 9‰ 0.1 █▋ | -- 1.276 2‰ 0.6 █▉ | -- 0.445 1‰ 0.5 ██▋ | -- 0.303 3‰ 1.0 █▋ | c/space-32k-8: 1.777 1‰ 0.1 █▋ | -- 1.275 2‰ 0.2 █▉ | -- 0.448 0‰ 0.0 ██▋ | -- 0.304 2‰ 0.3 █▋ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147748 ████████████████████████████████████████ | -- 147488 ████████████████████████████████████████ | -- 147772 ████████████████████████████████████████ | -- 147736 ████████████████████████████████████████ | hs/space-bs-c8-count: 147760 ████████████████████████████████████████ | -- n/a -- n/a -- 147744 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147760 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147800 ████████████████████████████████████████▏ | -- 147756 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-2: 147764 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147804 ████████████████████████████████████████▏ | -- 147752 ████████████████████████████████████████ | hs/space-bs-c8-foldrx: 147748 ████████████████████████████████████████ | -- 147488 ████████████████████████████████████████ | -- 147776 ████████████████████████████████████████ | -- 147732 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147764 ████████████████████████████████████████ | -- 147492 ████████████████████████████████████████ | -- 147796 ████████████████████████████████████████▏ | -- 147756 ████████████████████████████████████████ | hs/space-bslc8-acc-1: 3196 ▉ | -- 2896 ▊ | -- 3192 ▉ | -- 3192 ▉ | hs/space-bslc8-acc-2: 3200 ▉ | -- 2896 ▊ | -- 3192 ▉ | -- 3192 ▉ | hs/space-bslc8-acc-3: 3204 ▉ | -- 2896 ▊ | -- 3188 ▉ | -- 3196 ▉ | hs/space-bslc8-chunk-1: 65204 █████████████████▋ | -- 65892 █████████████████▉ | -- 65208 █████████████████▋ | -- 65192 █████████████████▋ | hs/space-bslc8-chunk-2: 65200 █████████████████▋ | -- 65892 █████████████████▉ | -- 65212 █████████████████▋ | -- 65192 █████████████████▋ | hs/space-bslc8-chunk-3: 75248 ████████████████████▍ | -- 76472 ████████████████████▊ | -- 75232 ████████████████████▍ | -- 75240 ████████████████████▍ | hs/space-bslc8-chunk-4: 76204 ████████████████████▋ | -- n/a -- n/a -- 76188 ████████████████████▋ | hs/space-bslc8-count: 79224 █████████████████████▌ | -- n/a -- n/a -- 79204 █████████████████████▌ | hs/space-bslc8-foldl: 85880 ███████████████████████▎ | -- 86772 ███████████████████████▋ | -- 85856 ███████████████████████▎ | -- 85868 ███████████████████████▎ | hs/space-bslc8-foldlx-1: 85880 ███████████████████████▎ | -- 86772 ███████████████████████▋ | -- 85852 ███████████████████████▎ | -- 85864 ███████████████████████▎ | hs/space-bslc8-foldlx-2: 85880 ███████████████████████▎ | -- 86772 ███████████████████████▋ | -- 85856 ███████████████████████▎ | -- 85868 ███████████████████████▎ | hs/space-bslc8-foldr-1: 169704 ██████████████████████████████████████████████| -- 169360 ██████████████████████████████████████████████| -- 169684 ██████████████████████████████████████████████| -- 169700 ██████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169700 ██████████████████████████████████████████████| -- 169360 ██████████████████████████████████████████████| -- 169684 ██████████████████████████████████████████████| -- 169700 ██████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109408 █████████████████████████████▋ | -- 110704 ██████████████████████████████▏ | -- 109404 █████████████████████████████▋ | -- 109404 █████████████████████████████▋ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████▋ | -- 110704 ██████████████████████████████▏ | -- 109404 █████████████████████████████▋ | -- 109404 █████████████████████████████▋ | hs/space-bsl---foldlx: 85844 ███████████████████████▎ | -- 86776 ███████████████████████▋ | -- 85880 ███████████████████████▎ | -- 85836 ███████████████████████▎ | hs/space-xxxxx-acc-1: 1912 ▌ | -- 1612 ▌ | -- 1916 ▌ | -- 1904 ▌ | hs/space-xxxxx-acc-2: 1912 ▌ | -- 1612 ▌ | -- 1916 ▌ | -- 1904 ▌ | hs/space-xxxxx-foldl: 1912 ▌ | -- 1612 ▌ | -- 1916 ▌ | -- 1900 ▌ | hs/space-xxxxx-lenfil: 1904 ▌ | -- 1588 ▍ | -- 1892 ▌ | -- 1892 ▌ | c/space-getchar: 436 ▏ | -- 384 ▏ | -- 436 ▏ | -- 444 ▏ | c/space-getchar-u: 436 ▏ | -- 384 ▏ | -- 436 ▏ | -- 444 ▏ | c/space-4k: 472 ▏ | -- 412 ▏ | -- 468 ▏ | -- 468 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████▉ | -- 146904 ███████████████████████████████████████▉ | -- 146968 ███████████████████████████████████████▉ | -- 146968 ███████████████████████████████████████▉ | c/space-getwchar: 484 ▏ | -- 440 ▏ | -- 484 ▏ | -- 496 ▏ | c/space-getwchar-u: 484 ▏ | -- 440 ▏ | -- 480 ▏ | -- 500 ▏ | c/space-32k: 500 ▏ | -- 436 ▏ | -- 496 ▏ | -- 496 ▏ | c/space-32k-8: 496 ▏ | -- 436 ▏ | -- 500 ▏ | -- 500 ▏ |

1 0

[9/16] SBM: 6.9.20071119 Athlon64
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by me on my slightly old noname 2GHz Athlon64 3000+ (in 32-bit mode on a 32-bit kernel). It has 1GB RAM and runs Ubuntu 7.04. The benchmarks were run two days ago (2007-12-19) using ghc 6.9.20071119 in runlevel 1 (single-user mode). I did leave the network cable in, which might have produced some interrupts (not very likely -- 1) why would it run in promiscuous mode and 2) who broadcasts apart from my wired/wireless access point which broadcasts a packet per second?). Still, the quality of the timings proved to be good, with a standard deviation of at most 2.7% of the average run-time. This run includes the hand-tweaked assembly benchmarks (as does the Pentium III run in the previous email). Note how much the assembler tweaks help! -Peter charybdis ghc 6.9.20071119 AMD Athlon(tm) 64 Processor 3000+ 2009.160 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 0.705 7‰ 0.1 ███▋ | hs/byte-bs----foldlx: 1.002 1‰ 0.5 █████▎ | hs/byte-bs----foldrx: 1.112 2‰ 0.1 █████▊ | hs/byte-bsl---acc: 2.595 14‰ 0.1 █████████████▌ | hs/byte-xxxxx-acc-1: 4.436 5‰ 0.0 ███████████████████████▏ | hs/byte-xxxxx-acc-2: 4.473 8‰ 0.0 ███████████████████████▎ | hs/byte-xxxxx-foldl: 4.412 10‰ 0.0 ███████████████████████ | hand/byte-bs----acc-a: 0.639 2‰ 0.2 ███▍ | hand/byte-bs----acc-b: 0.414 2‰ 0.5 ██▏ | hand/byte-bs----acc-c: 0.414 3‰ 0.2 ██▏ | hand/byte-bs----acc-d: 0.415 3‰ 0.2 ██▏ | c/byte-getchar: 2.422 27‰ 0.1 ████████████▋ | c/byte-getchar-u: 0.632 3‰ 0.3 ███▎ | c/byte-4k: 0.094 26‰ 2.2 ▌ | Memory: Peak ------- KB hs/byte-bs----acc: 147752 ███████████████████████████████████████▏ | hs/byte-bs----foldlx: 147748 ███████████████████████████████████████▏ | hs/byte-bs----foldrx: 147744 ███████████████████████████████████████▏ | hs/byte-bsl---acc: 3172 ▉ | hs/byte-xxxxx-acc-1: 1904 ▌ | hs/byte-xxxxx-acc-2: 1904 ▌ | hs/byte-xxxxx-foldl: 1900 ▌ | hand/byte-bs----acc-a: 147764 ███████████████████████████████████████▏ | hand/byte-bs----acc-b: 147764 ███████████████████████████████████████▏ | hand/byte-bs----acc-c: 147760 ███████████████████████████████████████▏ | hand/byte-bs----acc-d: 147760 ███████████████████████████████████████▏ | c/byte-getchar: 440 ▏ | c/byte-getchar-u: 440 ▏ | c/byte-4k: 440 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 1.145 1‰ 0.2 ██████ | hs/space-bs-c8-count: 0.521 1‰ 0.2 ██▊ | hs/space-bs-c8-foldlx-1: 1.221 1‰ 0.1 ██████▍ | hs/space-bs-c8-foldlx-2: 1.219 2‰ 0.2 ██████▍ | hs/space-bs-c8-foldrx: 1.172 2‰ 0.3 ██████▏ | hs/space-bs-c8-lenfil: 1.223 1‰ 0.2 ██████▍ | hs/space-bslc8-acc-1: 3.388 10‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-2: 3.386 5‰ 0.1 █████████████████▋ | hs/space-bslc8-acc-3: 2.832 4‰ 0.1 ██████████████▊ | hs/space-bslc8-chunk-1: 1.094 0‰ 0.3 █████▊ | hs/space-bslc8-chunk-2: 1.094 1‰ 0.3 █████▊ | hs/space-bslc8-chunk-3: 1.087 1‰ 0.3 █████▋ | hs/space-bslc8-chunk-4: 1.099 0‰ 0.3 █████▊ | hs/space-bslc8-count: 0.477 2‰ 0.4 ██▌ | hs/space-bslc8-foldl: 1.458 1‰ 0.3 ███████▋ | hs/space-bslc8-foldlx-1: 1.458 0‰ 0.1 ███████▋ | hs/space-bslc8-foldlx-2: 1.457 1‰ 0.1 ███████▋ | hs/space-bslc8-foldr-1: 1.312 1‰ 0.2 ██████▉ | hs/space-bslc8-foldr-2: 1.315 1‰ 0.2 ██████▉ | hs/space-bslc8-lenfil-1: 1.323 1‰ 0.1 ██████▉ | hs/space-bslc8-lenfil-2: 1.325 1‰ 0.1 ██████▉ | hs/space-bsl---foldlx: 1.391 1‰ 0.1 ███████▎ | hs/space-xxxxx-acc-1: 4.991 1‰ 0.1 ██████████████████████████ | hs/space-xxxxx-acc-2: 4.985 1‰ 0.0 ██████████████████████████ | hs/space-xxxxx-foldl: 5.035 2‰ 0.1 ██████████████████████████▎ | hs/space-xxxxx-lenfil: 7.106 8‰ 0.1 █████████████████████████████████████| hand/space-bs-c8-acc-1-a: 1.177 2‰ 0.3 ██████▏ | hand/space-bs-c8-acc-1-b: 1.104 1‰ 0.2 █████▊ | hand/space-bs-c8-acc-1-c: 1.207 1‰ 0.3 ██████▎ | hand/space-bs-c8-acc-1-d: 1.191 1‰ 0.2 ██████▎ | hand/space-bs-c8-acc-1-e: 0.937 1‰ 0.2 ████▉ | hand/space-bs-c8-acc-1-f: 0.921 1‰ 0.2 ████▊ | hand/space-bs-c8-acc-1-g: 0.927 2‰ 0.4 ████▉ | hand/space-bs-c8-acc-1-h: 0.886 2‰ 0.2 ████▋ | hand/space-bs-c8-acc-1-i: 0.803 3‰ 0.4 ████▏ | hand/space-bs-c8-acc-1-j: 0.706 2‰ 0.1 ███▋ | hand/space-bs-c8-acc-1-k: 0.705 1‰ 0.0 ███▋ | hand/space-bs-c8-acc-1-l: 0.710 2‰ 0.1 ███▊ | hand/space-bs-c8-acc-1-m: 0.642 6‰ 0.3 ███▍ | hand/space-bs-c8-acc-1-n: 0.636 4‰ 0.5 ███▎ | hand/space-bs-c8-acc-1-o: 0.626 2‰ 0.0 ███▎ | hand/space-bs-c8-acc-1-p: 0.565 3‰ 0.4 ███ | hand/space-bs-c8-acc-1-q: 0.418 6‰ 0.7 ██▏ | hand/space-bs-c8-acc-1-r: 0.334 5‰ 0.6 █▊ | hand/space-bs-c8-acc-1-s: 0.335 4‰ 0.9 █▊ | c/space-getchar: 2.465 15‰ 0.1 ████████████▉ | c/space-getchar-u: 0.631 1‰ 0.5 ███▎ | c/space-4k: 0.319 3‰ 0.6 █▋ | c/space-megabuf: 0.515 3‰ 0.4 ██▋ | c/space-getwchar: 3.956 6‰ 0.1 ████████████████████▋ | c/space-getwchar-u: 2.217 1‰ 0.1 ███████████▌ | c/space-32k: 0.303 3‰ 1.0 █▋ | c/space-32k-8: 0.304 2‰ 0.3 █▋ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147736 ███████████████████████████████████████▏ | hs/space-bs-c8-count: 147744 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-1: 147756 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-2: 147752 ███████████████████████████████████████▏ | hs/space-bs-c8-foldrx: 147732 ███████████████████████████████████████▏ | hs/space-bs-c8-lenfil: 147756 ███████████████████████████████████████▏ | hs/space-bslc8-acc-1: 3192 ▉ | hs/space-bslc8-acc-2: 3192 ▉ | hs/space-bslc8-acc-3: 3196 ▉ | hs/space-bslc8-chunk-1: 65192 █████████████████▎ | hs/space-bslc8-chunk-2: 65192 █████████████████▎ | hs/space-bslc8-chunk-3: 75240 ████████████████████ | hs/space-bslc8-chunk-4: 76188 ████████████████████▎ | hs/space-bslc8-count: 79204 █████████████████████ | hs/space-bslc8-foldl: 85868 ██████████████████████▊ | hs/space-bslc8-foldlx-1: 85864 ██████████████████████▊ | hs/space-bslc8-foldlx-2: 85868 ██████████████████████▊ | hs/space-bslc8-foldr-1: 169700 █████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169700 █████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109404 █████████████████████████████ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████ | hs/space-bsl---foldlx: 85836 ██████████████████████▊ | hs/space-xxxxx-acc-1: 1904 ▌ | hs/space-xxxxx-acc-2: 1904 ▌ | hs/space-xxxxx-foldl: 1900 ▌ | hs/space-xxxxx-lenfil: 1892 ▌ | hand/space-bs-c8-acc-1-a: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-b: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-c: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-d: 147756 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-e: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-f: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-g: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-h: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-i: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-j: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-k: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-l: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-m: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-n: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-o: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-p: 147752 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-q: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-r: 147748 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-s: 147744 ███████████████████████████████████████▏ | c/space-getchar: 444 ▏ | c/space-getchar-u: 444 ▏ | c/space-4k: 468 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████ | c/space-getwchar: 496 ▏ | c/space-getwchar-u: 500 ▏ | c/space-32k: 496 ▏ | c/space-32k-8: 500 ▏ |

1 0

[8/16] SBM: 6.9.20071119 Pentium III
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by me on my old Compaq Armada E500 from around the year 2000. It has an Intel Coppermine Pentium III running at 600 MHz with 384 MB RAM and running Ubuntu 7.10. The benchmarks were run today (2007-12-21) using ghc 6.9.20071119 (freshly downloaded and reinstalled) in runlevel 1 (single-user mode). I did leave the wireless card in, which might have produced some interrupts. Still, the quality of the timings proved to be good, with a standard deviation of at most 1.2% of the average run-time. This run includes the hand-tweaked assembly benchmarks (as does the Athlon64 run in the next email). Note how much less the assembler tweaks help here (until one gets down to the wicked MMX tweaks) compared to the situation on the Athlon64 in the next mail. They still help, though. -Peter ls-search ghc 6.9.20071119 Pentium III (Coppermine) 596.932 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 3.274 1‰ 0.1 ███ | hs/byte-bs----foldlx: 4.027 0‰ 0.0 ███▋ | hs/byte-bs----foldrx: 4.184 1‰ 0.0 ███▉ | hs/byte-bsl---acc: 28.005 10‰ 0.0 █████████████████████████▋ | hs/byte-xxxxx-acc-1: 25.852 4‰ 0.0 ███████████████████████▋ | hs/byte-xxxxx-acc-2: 25.622 5‰ 0.0 ███████████████████████▌ | hs/byte-xxxxx-foldl: 25.803 5‰ 0.0 ███████████████████████▋ | hand/byte-bs----acc-a: 3.511 1‰ 0.0 ███▎ | hand/byte-bs----acc-b: 1.998 2‰ 0.1 █▉ | hand/byte-bs----acc-c: 1.876 2‰ 0.1 █▊ | hand/byte-bs----acc-d: 1.876 1‰ 0.1 █▊ | c/byte-getchar: 13.016 0‰ 0.0 ███████████▉ | c/byte-getchar-u: 1.662 1‰ 0.1 █▌ | c/byte-4k: 0.543 2‰ 0.2 ▌ | Memory: Peak ------- KB hs/byte-bs----acc: 147752 ███████████████████████████████████████▏ | hs/byte-bs----foldlx: 147756 ███████████████████████████████████████▏ | hs/byte-bs----foldrx: 147760 ███████████████████████████████████████▏ | hs/byte-bsl---acc: 3180 ▉ | hs/byte-xxxxx-acc-1: 1916 ▌ | hs/byte-xxxxx-acc-2: 1912 ▌ | hs/byte-xxxxx-foldl: 1912 ▌ | hand/byte-bs----acc-a: 147772 ███████████████████████████████████████▏ | hand/byte-bs----acc-b: 147776 ███████████████████████████████████████▏ | hand/byte-bs----acc-c: 147772 ███████████████████████████████████████▏ | hand/byte-bs----acc-d: 147776 ███████████████████████████████████████▏ | c/byte-getchar: 436 ▏ | c/byte-getchar-u: 432 ▏ | c/byte-4k: 436 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 4.318 1‰ 0.0 ████ | hs/space-bs-c8-count: 3.118 1‰ 0.1 ██▉ | hs/space-bs-c8-foldlx-1: 4.631 1‰ 0.0 ████▎ | hs/space-bs-c8-foldlx-2: 4.632 1‰ 0.0 ████▎ | hs/space-bs-c8-foldrx: 4.678 0‰ 0.0 ████▎ | hs/space-bs-c8-lenfil: 4.634 1‰ 0.1 ████▎ | hs/space-bslc8-acc-1: 32.733 7‰ 0.0 ██████████████████████████████ | hs/space-bslc8-acc-2: 33.112 11‰ 0.0 ██████████████████████████████▎ | hs/space-bslc8-acc-3: 30.294 12‰ 0.0 ███████████████████████████▊ | hs/space-bslc8-chunk-1: 3.888 0‰ 0.1 ███▌ | hs/space-bslc8-chunk-2: 3.887 1‰ 0.0 ███▌ | hs/space-bslc8-chunk-3: 3.881 0‰ 0.1 ███▌ | hs/space-bslc8-chunk-4: 3.910 1‰ 0.0 ███▋ | hs/space-bslc8-count: 2.355 1‰ 0.1 ██▏ | hs/space-bslc8-foldl: 4.674 0‰ 0.0 ████▎ | hs/space-bslc8-foldlx-1: 4.676 0‰ 0.0 ████▎ | hs/space-bslc8-foldlx-2: 4.676 0‰ 0.0 ████▎ | hs/space-bslc8-foldr-1: 5.222 0‰ 0.0 ████▊ | hs/space-bslc8-foldr-2: 5.224 1‰ 0.1 ████▊ | hs/space-bslc8-lenfil-1: 4.712 1‰ 0.0 ████▎ | hs/space-bslc8-lenfil-2: 4.714 1‰ 0.0 ████▍ | hs/space-bsl---foldlx: 4.522 0‰ 0.1 ████▏ | hs/space-xxxxx-acc-1: 27.529 6‰ 0.0 █████████████████████████▎ | hs/space-xxxxx-acc-2: 27.200 3‰ 0.0 ████████████████████████▉ | hs/space-xxxxx-foldl: 27.436 8‰ 0.0 █████████████████████████▏ | hs/space-xxxxx-lenfil: 39.343 3‰ 0.0 ████████████████████████████████████| hand/space-bs-c8-acc-1-a: 4.318 1‰ 0.0 ████ | hand/space-bs-c8-acc-1-b: 4.331 1‰ 0.0 ████ | hand/space-bs-c8-acc-1-c: 4.492 1‰ 0.1 ████▏ | hand/space-bs-c8-acc-1-d: 4.354 1‰ 0.0 ████ | hand/space-bs-c8-acc-1-e: 4.424 0‰ 0.1 ████ | hand/space-bs-c8-acc-1-f: 4.164 1‰ 0.0 ███▊ | hand/space-bs-c8-acc-1-g: 4.309 1‰ 0.1 ████ | hand/space-bs-c8-acc-1-h: 4.202 1‰ 0.1 ███▉ | hand/space-bs-c8-acc-1-i: 3.820 1‰ 0.1 ███▌ | hand/space-bs-c8-acc-1-j: 3.472 1‰ 0.0 ███▏ | hand/space-bs-c8-acc-1-k: 3.474 1‰ 0.0 ███▏ | hand/space-bs-c8-acc-1-l: 3.498 1‰ 0.1 ███▎ | hand/space-bs-c8-acc-1-m: 3.397 1‰ 0.1 ███▏ | hand/space-bs-c8-acc-1-n: 3.373 1‰ 0.0 ███▏ | hand/space-bs-c8-acc-1-o: 3.118 1‰ 0.1 ██▉ | hand/space-bs-c8-acc-1-p: 2.935 2‰ 0.0 ██▋ | hand/space-bs-c8-acc-1-q: 3.477 1‰ 0.1 ███▏ | hand/space-bs-c8-acc-1-r: 1.674 1‰ 0.1 █▌ | hand/space-bs-c8-acc-1-s: 1.627 1‰ 0.2 █▌ | c/space-getchar: 13.268 0‰ 0.0 ████████████▏ | c/space-getchar-u: 2.170 0‰ 0.1 ██ | c/space-4k: 1.815 1‰ 0.1 █▋ | c/space-megabuf: 2.763 2‰ 0.1 ██▌ | c/space-getwchar: 15.105 29‰ 0.0 █████████████▉ | c/space-getwchar-u: 8.979 1‰ 0.0 ████████▎ | c/space-32k: 1.786 9‰ 0.1 █▋ | c/space-32k-8: 1.777 1‰ 0.1 █▋ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147748 ███████████████████████████████████████▏ | hs/space-bs-c8-count: 147760 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-1: 147760 ███████████████████████████████████████▏ | hs/space-bs-c8-foldlx-2: 147764 ███████████████████████████████████████▏ | hs/space-bs-c8-foldrx: 147748 ███████████████████████████████████████▏ | hs/space-bs-c8-lenfil: 147764 ███████████████████████████████████████▏ | hs/space-bslc8-acc-1: 3196 ▉ | hs/space-bslc8-acc-2: 3200 ▉ | hs/space-bslc8-acc-3: 3204 ▉ | hs/space-bslc8-chunk-1: 65204 █████████████████▎ | hs/space-bslc8-chunk-2: 65200 █████████████████▎ | hs/space-bslc8-chunk-3: 75248 ████████████████████ | hs/space-bslc8-chunk-4: 76204 ████████████████████▎ | hs/space-bslc8-count: 79224 █████████████████████ | hs/space-bslc8-foldl: 85880 ██████████████████████▊ | hs/space-bslc8-foldlx-1: 85880 ██████████████████████▊ | hs/space-bslc8-foldlx-2: 85880 ██████████████████████▊ | hs/space-bslc8-foldr-1: 169704 █████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169700 █████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109408 █████████████████████████████ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████ | hs/space-bsl---foldlx: 85844 ██████████████████████▊ | hs/space-xxxxx-acc-1: 1912 ▌ | hs/space-xxxxx-acc-2: 1912 ▌ | hs/space-xxxxx-foldl: 1912 ▌ | hs/space-xxxxx-lenfil: 1904 ▌ | hand/space-bs-c8-acc-1-a: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-b: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-c: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-d: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-e: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-f: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-g: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-h: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-i: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-j: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-k: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-l: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-m: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-n: 147764 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-o: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-p: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-q: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-r: 147760 ███████████████████████████████████████▏ | hand/space-bs-c8-acc-1-s: 147760 ███████████████████████████████████████▏ | c/space-getchar: 436 ▏ | c/space-getchar-u: 436 ▏ | c/space-4k: 472 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████ | c/space-getwchar: 484 ▏ | c/space-getwchar-u: 484 ▏ | c/space-32k: 500 ▏ | c/space-32k-8: 496 ▏ |

1 0

[7/16] SBM: 6.9.20071208 Core Duo
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by me on a friend's machine at his place, namely an Intel Core Duo laptop running Ubuntu 7.10. The benchmarks were run on 2007-12-09 using ghc 6.9.20071208 (approx) in runlevel 1 (single-user mode). I used an early version of the benchmark suite which didn't capture the exact ghc version used and an early version of my ghc install scripts that didn't try for a precise version. It ran on what was ghc head at the time. -Peter albatross ghc 6.9.20071208 (or thereabouts) Genuine Intel(R) CPU T2300 @ 1.66GHz (Core Duo) 1667.000 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 0.918 4‰ 0.1 █████▌ | hs/byte-bs----foldlx: 1.014 2‰ 0.2 ██████▏ | hs/byte-bs----foldrx: 0.999 2‰ 0.1 ██████ | hs/byte-bsl---acc: 1.957 1‰ 0.1 ███████████▉ | hs/byte-xxxxx-acc-1: 3.591 2‰ 0.1 █████████████████████▊ | hs/byte-xxxxx-acc-2: 3.598 4‰ 0.1 █████████████████████▊ | hs/byte-xxxxx-foldl: 3.594 1‰ 0.1 █████████████████████▊ | c/byte-getchar: 3.761 2‰ 0.1 ██████████████████████▊ | c/byte-getchar-u: 0.719 2‰ 0.3 ████▍ | c/byte-4k: 0.101 10‰ 3.1 ▋ | Memory: Peak ------- KB hs/byte-bs----acc: 147784 ████████████████████████████████████████▏ | hs/byte-bs----foldlx: 147812 ████████████████████████████████████████▏ | hs/byte-bs----foldrx: 147788 ████████████████████████████████████████▏ | hs/byte-bsl---acc: 3228 ▉ | hs/byte-xxxxx-acc-1: 1904 ▌ | hs/byte-xxxxx-acc-2: 1908 ▌ | hs/byte-xxxxx-foldl: 1908 ▌ | c/byte-getchar: 432 ▏ | c/byte-getchar-u: 440 ▏ | c/byte-4k: 436 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 1.313 0‰ 0.2 ███████▉ | hs/space-bs-c8-foldlx-1: 1.326 1‰ 0.1 ████████ | hs/space-bs-c8-foldlx-2: 1.328 1‰ 0.1 ████████ | hs/space-bs-c8-foldrx: 1.128 1‰ 0.2 ██████▉ | hs/space-bs-c8-lenfil: 1.327 2‰ 0.2 ████████ | hs/space-bslc8-acc-1: 3.002 1‰ 0.1 ██████████████████▏ | hs/space-bslc8-acc-2: 3.010 2‰ 0.0 ██████████████████▎ | hs/space-bslc8-acc-3: 2.586 1‰ 0.1 ███████████████▋ | hs/space-bslc8-chunk-1: 1.192 0‰ 0.0 ███████▎ | hs/space-bslc8-chunk-2: 1.189 0‰ 0.1 ███████▏ | hs/space-bslc8-chunk-3: 1.272 1‰ 0.2 ███████▊ | hs/space-bslc8-foldl: 1.517 0‰ 0.1 █████████▏ | hs/space-bslc8-foldlx-1: 1.516 0‰ 0.0 █████████▏ | hs/space-bslc8-foldlx-2: 1.511 0‰ 0.0 █████████▏ | hs/space-bslc8-foldr-1: 1.205 1‰ 0.2 ███████▎ | hs/space-bslc8-foldr-2: 1.202 3‰ 0.1 ███████▎ | hs/space-bslc8-lenfil-1: 1.400 0‰ 0.0 ████████▌ | hs/space-bslc8-lenfil-2: 1.396 0‰ 0.0 ████████▌ | hs/space-bsl---foldlx: 1.476 1‰ 0.1 ████████▉ | hs/space-xxxxx-acc-1: 4.245 1‰ 0.0 █████████████████████████▋ | hs/space-xxxxx-acc-2: 4.241 2‰ 0.1 █████████████████████████▋ | hs/space-xxxxx-foldl: 4.239 2‰ 0.1 █████████████████████████▋ | hs/space-xxxxx-lenfil: 6.287 2‰ 0.0 ██████████████████████████████████████| c/space-getchar: 4.031 18‰ 0.0 ████████████████████████▍ | c/space-getchar-u: 0.609 2‰ 0.7 ███▋ | c/space-4k: 0.465 2‰ 0.6 ██▊ | c/space-megabuf: 0.613 1‰ 0.3 ███▊ | c/space-getwchar: 4.584 1‰ 0.1 ███████████████████████████▊ | c/space-getwchar-u: 2.437 0‰ 0.1 ██████████████▊ | c/space-32k: 0.445 1‰ 0.5 ██▊ | c/space-32k-8: 0.448 0‰ 0.0 ██▊ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147772 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147800 ████████████████████████████████████████▏ | hs/space-bs-c8-foldlx-2: 147804 ████████████████████████████████████████▏ | hs/space-bs-c8-foldrx: 147776 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147796 ████████████████████████████████████████▏ | hs/space-bslc8-acc-1: 3192 ▉ | hs/space-bslc8-acc-2: 3192 ▉ | hs/space-bslc8-acc-3: 3188 ▉ | hs/space-bslc8-chunk-1: 65208 █████████████████▋ | hs/space-bslc8-chunk-2: 65212 █████████████████▋ | hs/space-bslc8-chunk-3: 75232 ████████████████████▍ | hs/space-bslc8-foldl: 85856 ███████████████████████▎ | hs/space-bslc8-foldlx-1: 85852 ███████████████████████▎ | hs/space-bslc8-foldlx-2: 85856 ███████████████████████▎ | hs/space-bslc8-foldr-1: 169684 ██████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169684 ██████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 109404 █████████████████████████████▋ | hs/space-bslc8-lenfil-2: 109404 █████████████████████████████▋ | hs/space-bsl---foldlx: 85880 ███████████████████████▎ | hs/space-xxxxx-acc-1: 1916 ▌ | hs/space-xxxxx-acc-2: 1916 ▌ | hs/space-xxxxx-foldl: 1916 ▌ | hs/space-xxxxx-lenfil: 1892 ▌ | c/space-getchar: 436 ▏ | c/space-getchar-u: 436 ▏ | c/space-4k: 468 ▏ | c/space-megabuf: 146968 ███████████████████████████████████████▉ | c/space-getwchar: 484 ▏ | c/space-getwchar-u: 480 ▏ | c/space-32k: 496 ▏ | c/space-32k-8: 500 ▏ |

1 0

[6/16] SBM: 6.9.20071124 Athlon Duron
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

This set of measurements was captured by Daniel Fischer on one of his older machines, running SuSE 8.2, which has a Linux 2.4 kernel! The benchmarks were run on 2007-12-16 using ghc 6.9.20071124. Unfortunately, the ghc version is not quite the same as the one I've used for most measurements (6.9.20071119) so things may be a little different just for that reason. The results seemed a bit off at first, but now that I have graphs of all the runs on all the machines they don't seem strange at all. First of all, the memory use is about the same as on the other machines. Secondly, the timing differences for the C getchar/getwchar might be partly due to different versions of the C library. The remaining differences (a "steeper" profile than on the core duo and the Athlon64) may be due to different microarchitectures. -Peter Fischer's machine ghc 6.9.20071124 AMD Duron(tm) processor 1200.089 MHz TESTKIND=THOROUGH SUFFIX= Time (byte counting) std -------------------- avg dev slack hs/byte-bs----acc: 1.892 21‰ 0.4 ██▊ | hs/byte-bs----foldlx: 2.258 3‰ 0.1 ███▎ | hs/byte-bs----foldrx: 2.933 0‰ 0.1 ████▎ | hs/byte-bsl---acc: 14.319 45‰ 0.1 ████████████████████▌ | hs/byte-xxxxx-acc-1: 20.915 17‰ 4.0 █████████████████████████████▉ | hs/byte-xxxxx-acc-2: 20.691 8‰ 0.1 █████████████████████████████▌ | hs/byte-xxxxx-foldl: 20.610 5‰ 1.4 █████████████████████████████▍ | c/byte-getchar: 9.042 0‰ 0.1 ████████████▉ | c/byte-getchar-u: 1.314 3‰ 0.2 █▉ | c/byte-4k: 0.419 5‰ 0.5 ▋ | Memory: Peak ------- KB hs/byte-bs----acc: 147492 ████████████████████████████████████████ | hs/byte-bs----foldlx: 147492 ████████████████████████████████████████ | hs/byte-bs----foldrx: 147488 ████████████████████████████████████████ | hs/byte-bsl---acc: 2896 ▊ | hs/byte-xxxxx-acc-1: 1612 ▌ | hs/byte-xxxxx-acc-2: 1612 ▌ | hs/byte-xxxxx-foldl: 1612 ▌ | c/byte-getchar: 384 ▏ | c/byte-getchar-u: 384 ▏ | c/byte-4k: 380 ▏ | Time (space counting) std --------------------- avg dev slack hs/space-bs-c8-acc-1: 2.467 1‰ 0.3 ███▌ | hs/space-bs-c8-foldlx-1: 2.585 2‰ 0.1 ███▊ | hs/space-bs-c8-foldlx-2: 2.576 2‰ 0.3 ███▋ | hs/space-bs-c8-foldrx: 2.982 8‰ 2.3 ████▎ | hs/space-bs-c8-lenfil: 2.599 1‰ 0.2 ███▊ | hs/space-bslc8-acc-1: 15.228 8‰ 0.1 █████████████████████▊ | hs/space-bslc8-acc-2: 15.855 38‰ 0.0 ██████████████████████▋ | hs/space-bslc8-acc-3: 14.980 14‰ 0.0 █████████████████████▍ | hs/space-bslc8-chunk-1: 2.443 2‰ 0.2 ███▌ | hs/space-bslc8-chunk-2: 2.449 1‰ 0.3 ███▌ | hs/space-bslc8-chunk-3: 2.534 3‰ 0.3 ███▋ | hs/space-bslc8-foldl: 2.938 1‰ 0.2 ████▎ | hs/space-bslc8-foldlx-1: 2.928 1‰ 0.0 ████▏ | hs/space-bslc8-foldlx-2: 2.937 2‰ 0.2 ████▎ | hs/space-bslc8-foldr-1: 4.043 6‰ 0.1 █████▊ | hs/space-bslc8-foldr-2: 4.007 4‰ 0.1 █████▊ | hs/space-bslc8-lenfil-1: 3.240 1‰ 0.2 ████▋ | hs/space-bslc8-lenfil-2: 3.236 1‰ 0.2 ████▋ | hs/space-bsl---foldlx: 2.821 1‰ 0.1 ████ | hs/space-xxxxx-acc-1: 21.002 4‰ 0.1 ██████████████████████████████ | hs/space-xxxxx-acc-2: 21.270 22‰ 4.7 ██████████████████████████████▍ | hs/space-xxxxx-foldl: 20.934 1‰ 0.1 █████████████████████████████▉ | hs/space-xxxxx-lenfil: 25.915 3‰ 0.0 █████████████████████████████████████| c/space-getchar: 9.354 0‰ 0.0 █████████████▍ | c/space-getchar-u: 1.676 2‰ 0.2 ██▍ | c/space-4k: 1.293 2‰ 0.5 █▉ | c/space-megabuf: 1.830 3‰ 0.5 ██▋ | c/space-getwchar: 14.721 1‰ 0.1 █████████████████████ | c/space-getwchar-u: 4.814 0‰ 0.0 ██████▉ | c/space-32k: 1.276 2‰ 0.6 █▉ | c/space-32k-8: 1.275 2‰ 0.2 █▉ | Memory: Peak ------- KB hs/space-bs-c8-acc-1: 147488 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-1: 147492 ████████████████████████████████████████ | hs/space-bs-c8-foldlx-2: 147492 ████████████████████████████████████████ | hs/space-bs-c8-foldrx: 147488 ████████████████████████████████████████ | hs/space-bs-c8-lenfil: 147492 ████████████████████████████████████████ | hs/space-bslc8-acc-1: 2896 ▊ | hs/space-bslc8-acc-2: 2896 ▊ | hs/space-bslc8-acc-3: 2896 ▊ | hs/space-bslc8-chunk-1: 65892 █████████████████▉ | hs/space-bslc8-chunk-2: 65892 █████████████████▉ | hs/space-bslc8-chunk-3: 76472 ████████████████████▊ | hs/space-bslc8-foldl: 86772 ███████████████████████▋ | hs/space-bslc8-foldlx-1: 86772 ███████████████████████▋ | hs/space-bslc8-foldlx-2: 86772 ███████████████████████▋ | hs/space-bslc8-foldr-1: 169360 ██████████████████████████████████████████████| hs/space-bslc8-foldr-2: 169360 ██████████████████████████████████████████████| hs/space-bslc8-lenfil-1: 110704 ██████████████████████████████▏ | hs/space-bslc8-lenfil-2: 110704 ██████████████████████████████▏ | hs/space-bsl---foldlx: 86776 ███████████████████████▋ | hs/space-xxxxx-acc-1: 1612 ▌ | hs/space-xxxxx-acc-2: 1612 ▌ | hs/space-xxxxx-foldl: 1612 ▌ | hs/space-xxxxx-lenfil: 1588 ▍ | c/space-getchar: 384 ▏ | c/space-getchar-u: 384 ▏ | c/space-4k: 412 ▏ | c/space-megabuf: 146904 ███████████████████████████████████████▉ | c/space-getwchar: 440 ▏ | c/space-getwchar-u: 440 ▏ | c/space-32k: 436 ▏ | c/space-32k-8: 436 ▏ |

1 0

[5/16] SBM: Support scripts and scriptlets
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

Some of the scripts warrant a closer look. 'make zipdata' creates a nice tarball with all the data necessary to recreate a report AND to merge that report together with other reports, possible with rescaled bar charts. Very handy. All the files in the tarball are inside the 'ghc-measurements/' directory so the risk of things going wrong when unpacking the tarball is less. The names of the benchmarks are put in ghc-measurements/progs, mainly to ensure they end up in the right order when regenerating and merging reports. tools/genreport.pl [list of benchmarks to put in the report] It doesn't parse the command-line in any way because life is too short for command-line parsing. Instead, it is controlled via (too many) environment variables. ASCII - set to avoid using UTF-8 for bar charts and "per mille" character. NOSRC - the tool normally creates */*.srctimemem files containing the source code for each benchmark with bar charts for time/mem appended to the end. Setting this variable switches that off (necessary when regenerating and merging reports). EXCLUDE - disregard some of the benchmarks on the command line. Why is this necessary? Because it makes regenerating and merging reports easier. And because I was too lazy to filter the command line in tools/regenreport.sh and tools/merge.pl. FINDMAX - used by tools/merge.pl when rescaling. Outputs max time and max mem to stdout instead of the normal report. MAX_FILEWIDTH - used by tools/merge.pl to make merged reports look nice MAX_TIME, MAX_PEAKMEM - used by tools/merge.pl when rescaling Note that strictly speaking, there is a bug in the script(s) because it conflates the width of time/mem measurement represented as numbers (which you always want to take into account when merging) and MAX_TIME/MAX_PEAKMEM (which you only care about when rescaling). [FIXED now - 2007-12-21] tools/regenreport.sh unpacks a measurement tarball into a tmp directory and runs tools/genreport.pl to generate the report. Takes care not to disturb the normal files. tools/merge.pl [tarballs] Uses tools/regenreport.sh on each tarball in turn to generate a report which it reads in and stores on a benchmark-by-benchmark basis. At the end, synthetically combine all the pieces it cut out of the original report(s) into a brand-spanking new, merged report. Even the headers and the platforminfo at the top of each report is cut out and stored in data structures until they get spit out again at the end. The reading magic is in the state machine in gather(). It is not as bad as it looks. Some of the complications arise from marking repeated benchmark names as ' -- ', which improves the readability of the merged reports immensely. Another part of the complications arise due to the fact that not all tarballs contain the exact same benchmarks! Those that don't get a nice 'n/a' instead of numbers and a bar. And finally, the benchmarks should be in the right order. That is trickier than it sounds... When rescaling, tools/regenreport.sh is first run once for each tarball with the FINDMAX environment variable set. This results in tools/regenreport.sh outputting the maximum filename width, time, and peakmem for each tarball. ASCII - use ASCII instead of UTF-8 RESCALE - sometimes you want to rescale and sometimes you don't MAX_FILEWIDTH - if you want to force a specific width MAX_TIME, MAX_PEAKMEM - if you want to force a specific max -Peter

1 0

[4/16] SBM: How to use the Makefile (how to run benchmarks etc.)
by Peter Firefly Brodersen Lund 22 Dec '07

22 Dec '07

Introduction ------------ Most of the smarts of the benchmark harness is in the Makefile. If you want to rerun the benchmarks (or a single benchmark) or look at the intermediate code for a benchmark or the I/O trace or the memory consumption or the time spent or ... then you use the makefile. There are some support scripts in shell and Perl (and two C programs) that the Makefile uses to do its job. And there are some that you, the user, will want to interact directly with. The benchmarks are only expected to work on Linux. They have been tested on SuSE 8.2 (from 2003, with a 2.4 kernel), Ubuntu 7.04, and Ubuntu 7.10. Quick howto ----------- make phase1 -- compiles, generates test files, measures memory use. Safe to run on a busy machine if there's no active memory pressure. make phase2 -- timing runs. NOT safe to run on a busy machine. Should be run in runlevel 1 (= no X, no daemons, single-user mode) for best measurements. Outputs report at end. This is where you check the quality of the measurements. If you don't like them, run 'make redophase2' (or delete the .time and .stat files with low quality and run 'make phase2' again.) make zipdata -- make a tarball with all the measurements, suitable for emailing or putting on a website. The Makefile will beep after phase 1 and 2. The above will run a "NORMAL" run, which is fine during development if you want to see if you nailed a performance bug. It runs reasonably fast (about 43 seconds on my Athlon64). If you want better measurements, you should use: make TESTKIND=THOROUGH phase1 phase2 zipdata This will use a 150MB data file instead of a 15MB one and it will run the timing measurements 6 times (before throwing the first away) instead of 4 times (before throwing the first away). If you don't want to use single-user mode, you can improve the measurements by piping the output to a file (or run the test from the console) instead of involving a terminal and an X server (the screen update may kick in in the middle of a timing run and disturb things if for no other reason than their polluting the CPU caches). Filesystem layout ----------------- The benchmarks are in: hs/*.hs c/*.c hand/*.s hand/*.hs and hand/*.c are not compiled. The two *.hs files are the originals from which the tweaked assembly code has been derived. The two *.c files are sketches of how the MMX tweaks work (because MMX code by itself can be a bit off-putting). These are the support scripts: tools/genfiles.pl -- generate the test input files. tools/cutmem.pl tools/cutpid.pl -- both are used to disentangle the outputs of strace and pause-at-end (see below). I combine strace, memory info, and +RTS -sstderr into a single run to save time. This means that things end up in fewer files than I'd like. tools/cut.pl -- cut out main loop from disassembly ('make discut') tools/stat.pl -- looks at all timings for a single benchmark and calculates average and standard deviation and "time slack", that is the discrepancy between user+sys and real. It optionally throws away the first run. tools/eatmem.c -- allocates a chunk of memory and makes damn sure it really is in RAM! tools/pause-at-end.c -- part of a hack that copies /proc/self/maps and /proc/self/status to stderr just before a benchmark exits. tools/iosummary.pl -- takes an strace and sums up the I/O tools/genreport.pl -- generate a nice report with bar charts. Takes way too many options in the form of environment variables. tools/regenreport.sh -- regenerates the report from ANY measurement tarball. tools/merge.pl -- merge data from many measurement tarballs, with or without rescaling. Generated files: hs/*.core hs/*.stg hs/*.cmm hs/*.s -- intermediate code hs/*.hi -- "Haskell Interface" */*.o -- object code */* (the files in $(HSPROGS) $(CPROGS) $(HANDPROGS)) -- programs */*.dis */*.discut -- disassembled programs (and inner loops) */*.doc -- source + intermediate code + inner loops + timings */*.mem -- output from '+RTS -sstderr' + /proc/self/status + /proc/self/maps + output from /usr/bin/time (where the number of minor page faults is most interesting datum) */*.strace -- complete strace, taken together with */*.mem */*.iotrace -- only I/O operations from the strace (read/write/ select) */*.iosum -- summary of I/O operations */*.time -- time measurements */*.stat -- average + std.dev. + "time slack" */*.srctimespace -- source code + time/mem barchart (in ASCII) sysinfo -- description of the platform (uname, ghc, gcc, etc) platforminfo -- short description of the platform report.txt [8-16K] docs [1MB] -- sysinfo + all */*.doc concatenated Makefile targets ---------------- This is taken from 'make help': phase1 -- preparation + measurements that can run in background phase2 -- measurements that should run on unloaded machine redophase2 -- rerun phase2 doc, [ASCII=1] report, lastreport - reports zipdata -- zip up measurements (to ghc-measurements.tar.gz) prog,core,stg,cmm,asm,dis,discut -- compile, compile to core/stg/cmm/asm, disassemble, cut out main loop time,stat,mem,strace,iotrace,iosum,cache -- measure run-time, GHC heap + OS mem, syscalls, I/O patterns, cache cleartime, clean, distclean -- delete measurements etc TESTKIND=(SMOKETEST,NORMAL,THOROUGH), defaults to NORMAL STRACE=OLD, defaults to NEW Necessary tools --------------- Perl, sed, /usr/bin/time, bash (doesn't have to be the default shell as long as it its in PATH), strace, GNU Make, objdump (from the binutils package), gcc. Other things that could come in handy: A console and/or terminal that understands UTF-8. A less that understands UTF-8. An editor that understands UTF-8. A %!&# printing program that understands both UTF-8 and fonts. A2ps doesn't do UTF-8. Uniprint used 1) a proportional font which 2) didn't even have all the fractional-width blocks. U2ps used a with none of the block characters. I ended up resorting to gedit's print function :( That's enough for this email. -Peter

1 0