Performance, Optimization and Code Generation
This starts out with my being interested in darcs <-> git related issues. Since git uses sha1 I wanted to have the ability to calculate sha1 in an application where I was intending to use darcs as a back-end. The performance gap is > * 30 between Haskell and sha1sum. That seemed rather steep and so I started looking... Using ghc 6.4.2 The following code is from SHA1: -- {-# INLINE step #-} step :: ABCDE -> BS.ByteString -> ABCDE step abcde0@(ABCDE a b c d e) words = abcde5 where s16 = get_word_32s words s80 = s16 ++ (zipWith4 f0) (drop 13 s80) (drop 8 s80) (drop 2 s80) s80 f0 a b c d = rotL (a `xor` b `xor` c `xor` d) 1 (s20_0, s60) = splitAt 20 s80 (s20_1, s40) = splitAt 20 s60 (s20_2, s20) = splitAt 20 s40 (s20_3, _) = splitAt 20 s20 abcde1 = foldl (doit f1 0x5a827999) abcde0 s20_0 abcde2 = foldl (doit f2 0x6ed9eba1) abcde1 s20_1 abcde3 = foldl (doit f3 0x8f1bbcdc) abcde2 s20_2 ABCDE a' b' c' d' e' = foldl (doit f2 0xca62c1d6) abcde3 s20_3 f1 (XYZ x y z) = (x .&. y) .|. ((complement x) .&. z) f2 (XYZ x y z) = x `xor` y `xor` z f3 (XYZ x y z) = (x .&. y) .|. (x .&. z) .|. (y .&. z) abcde5 = ABCDE (a + a') (b + b') (c + c') (d + d') (e + e') -- {-# INLINE get_word_32s #-} get_word_32s :: BS.ByteString -> [Word32] get_word_32s s = map f [0..15] where f i = foldl (+) 0 $ map (\n -> toEnum (fromEnum (BS.index s (i*4+n))) `shiftL` (8 * (3-n))) [0..3] -- {-# INLINE doit #-} doit :: (XYZ -> Word32) -> Word32 -> ABCDE -> Word32 -> ABCDE doit f k (ABCDE a b c d e) w = ABCDE a' a (rotL b 30) c d where a' = rotL a 5 + f (XYZ b c d) + e + w + k -- {-# INLINE rotL #-} rotL :: Word32 -> Rotation -> Word32 rotL a s = shiftL a s .|. shiftL a (s-32) -- rotL a s = a `seq` rotate a s I want to focus on the *rotL* function --- get_word_32s might be faster with a rewrite. Using -prof -auto-all and -P at runtime produced the following summary... COST CENTRE MODULE %time %alloc ticks bytes get_word_32s MySHA1 39.3 35.0 10597 6952614960 rotL MySHA1 22.6 18.5 6089 3682275072 step MySHA1 21.2 24.1 5701 4783669848 doit MySHA1 16.2 21.8 4362 4332088320 As part of a larger program, but the sha1 portion was where 98% of the time went. So * I tried using the built-in "rotate", * I tried inlining, * I tried using -fasm directly * I tried generating C with -O2, * I tried using 'seq' in a number of places. None of this made much difference. Comment, the C code for just rotL would be unsigned int rotate(unsigned int a, int b) { return (a << b) | (a >> (b - 32)); } and the assembler at gcc -O2 is concise and straightforward: .file "rotate.c" .text .p2align 4,,15 .globl rotate .type rotate, @function rotate: pushl %ebp movl %esp, %ebp movl 8(%ebp), %edx movl 12(%ebp), %ecx popl %ebp movl %edx, %eax sall %cl, %eax subl $32, %ecx shrl %cl, %edx orl %edx, %eax ret .size rotate, .-rotate .section .note.GNU-stack,"",@progbits .ident "GCC: (GNU) 3.3.5 (Debian 1:3.3.5-3)" GHC's code on this part is nothing like competitive because of the need for closures --- I think. 1) What approaches to getting the compiler to optimize have I overlooked? Or, how would you change the code to get the compiler to generate better code for rotL? 2) Am I missing an easy way to get strictness on the rotL function? 3) Is there something I am missing that keeps ghc's code from becoming about as concise as gcc -O2? 4) Is the code generator simply not able to handle this? Thanks in advance, George
according to <http://darcs.haskell.org/packages/base/GHC/Word.hs>, GHC's rotate is implemented as (W32# x#) `rotate` (I# i#) | i'# ==# 0# = W32# x# | otherwise = W32# ((x# `shiftL32#` i'#) `or32#` (x# `shiftRL32#` (32# -# i'#))) where i'# = word2Int# (int2Word# i# `and#` int2Word# 31#) So you can see that it takes i modulo 32 first (by anding it with 31). Perhaps one needs an uncheckedRotate32# function (if it doesn't exist). Also, shouldn't the calls to shiftL32# and shiftRL32# be calls to uncheckedShiftL32# and uncheckedShiftR32# since i'# and (32# -# i'#) are provably safe? -- Russell O'Connor <http://r6.ca/> ``All talk about `theft,''' the general counsel of the American Graphophone Company wrote, ``is the merest claptrap, for there exists no property in ideas musical, literary or artistic, except as defined by statute.''
Hello roconnor, Saturday, September 23, 2006, 4:13:39 PM, you wrote:
Also, shouldn't the calls to shiftL32# and shiftRL32# be calls to uncheckedShiftL32# and uncheckedShiftR32# since i'# and (32# -# i'#) are provably safe?
yes. below is the code used in my lib: #ifdef __GLASGOW_HASKELL__ (I# a) <<# (I# b) = (I# (a `iShiftL#` b)) (I# a) >># (I# b) = (I# (a `uncheckedIShiftRL#` b)) #else /* ! __GLASGOW_HASKELL__ */ a <<# b = a `shiftL` b a >># b = a `shiftR` b #endif /* ! __GLASGOW_HASKELL__ */ -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
On Fri, Sep 22, 2006 at 08:30:20PM -0400, George Beshers wrote:
This starts out with my being interested in darcs <-> git related issues. Since git uses sha1 I wanted to have the ability to calculate sha1 in an application where I was intending to use darcs as a back-end.
The performance gap is > * 30 between Haskell and sha1sum. That seemed rather steep and so I started looking...
I did some work on optimising the SHA1.lhs in the darcs source a while ago. My memory is that I got to about a factor of 2 of C, and that most of the rest was due to GHC generating code with things like heap size checks unnecessarily inside loops. It uses darcs' FPS rather than the ByteString fork, of course, but it should be easy enough to port. http://www.abridgegame.org/repos/darcs-unstable/SHA1.lhs Thanks Ian
participants (4)
-
Bulat Ziganshin -
George Beshers -
Ian Lynagh -
roconnor@theorem.ca