Can't prevent memoizing in simple code
The buffer http://hpaste.org/68595 presents a simple code I tried to profile. I spotted what I strongly think to be an abusive memoization. The problem is that I don't see how to (simply) get rid of it. Compiled with -O2, it consumes 130MB of memory, however lines A and B executed separately consume each only 1MB. The infinite list (l 1), whatever I do, keeps being shared between lines A and B. I tried to wrap it in a function, as you can see, I also tried to make it explicitely polymorphic (bypassing monomorphic restriction), nothing solves it, GHC is just to good at memoizing. NB: When compiled without optimisations, the sharing does not happen (side note: but then lack of strictness analysis -- which is what I was testing at the first place -- makes line A (call to suminit2) consume a lot of memory, but this is normal).
On May 16, 2012, at 12:08 PM, Yves Parès wrote:
The buffer http://hpaste.org/68595 presents a simple code I tried to profile. I spotted what I strongly think to be an abusive memoization. The problem is that I don't see how to (simply) get rid of it. Compiled with -O2, it consumes 130MB of memory, however lines A and B executed separately consume each only 1MB.
The infinite list (l 1), whatever I do, keeps being shared between lines A and B. I tried to wrap it in a function, as you can see, I also tried to make it explicitely polymorphic (bypassing monomorphic restriction), nothing solves it, GHC is just to good at memoizing.
Adding a {-# NOINLINE l #-} annotation helps here. Syntactically, it must be located somewhere a type signature for l would also be valid. Anthony
Thanks ^^
My other solution was a dirty trick:
Changing the second (l 1) by a (l (div 2 2)), which would only be good
until GHC knows how to statically analyse it (2-1 wasn't working for
instance).
I also noticed (while profiling to confirm that this was the source of the
memory leak) that adding manually cost centres forces the re-evaluation:
main = do
print $ suminit2 ({-# SCC "list1" #-} l 1) 1000000 0
print $ fst $ suminit ({-# SCC "list2" #-} l 1) 1000000 0
where
l n = enumFrom n
CAF:main5 Main 97 0 0.0 0.0 26.6
50.0
main Main 118 0 0.0 0.0 26.6
50.0
list2 Main 119 0 0.0 0.0 26.6 *50.0
*
main.l Main 120 1 26.6 50.0 26.6
50.0
[...]
CAF:main8 Main 95 0 0.0 0.0 30.4
50.0
main Main 110 0 0.0 0.0 30.4
50.0
list1 Main 111 0 0.0 0.0 30.4
*50.0*
main.l Main 112 1 30.4 50.0 30.4
50.0
We see here that allocations are shared between list1 and list2 (I expected
list1 to get 100% and list2 0%, due to sharing).
Strange...
2012/5/16 Anthony Cowley
On May 16, 2012, at 12:08 PM, Yves Parès wrote:
The buffer http://hpaste.org/68595 presents a simple code I tried to profile. I spotted what I strongly think to be an abusive memoization. The problem is that I don't see how to (simply) get rid of it. Compiled with -O2, it consumes 130MB of memory, however lines A and B executed separately consume each only 1MB.
The infinite list (l 1), whatever I do, keeps being shared between lines A and B. I tried to wrap it in a function, as you can see, I also tried to make it explicitely polymorphic (bypassing monomorphic restriction), nothing solves it, GHC is just to good at memoizing.
Adding a {-# NOINLINE l #-} annotation helps here. Syntactically, it must be located somewhere a type signature for l would also be valid.
Anthony
participants (2)
-
Anthony Cowley -
Yves Parès