We've been trying to track down a memory leak, and by adding SCC annotations, pinned it down to a very brief expression. Unfortunately, the expression is just a variable! We even looked at the generated core: (case ([Soutei.Term Logic.BodyVar] -> (...) -> Logic.Frame -> Logic.QState -> FBackTrackT.Stream m_aYsk Logic.QState) lookup_aYPK pred_aXVH ctxIdx_aYpI of wild_B1 { Data.Maybe.Nothing -> GHC.Err.patError ... Data.Maybe.Just x_aXVL -> __scc {pqzifoozq Assertions} x_aXVL }); Note the __scc annotation just before x_aXVL at the end of the expression. Yet in the time and allocation profile, this SCC is charged for a lot of time and allocations in the "individual" column; individual inherited COST CENTRE no. entries %time %alloc %time %alloc pq.foo' 466 50001 8.0 12.1 8.0 13.2 and in the heap profile, it is a major memory leak. The Haskell code looks like predQuery pred = case lookup pred ctxIdx of Just f -> {-# SCC "pq.foo'" #-} f Note that f is a function, which we expect to be expensive when it is called. However, that should be charged to the definition of the function, right? We tried pretty hard to reproduce this in a small test case, but never succeeded. Always, the cost was accounted to the definition of the function in the intuitive way. We're using ghc 6.4.1 and compiling without optimization. Can any ghc hacker suggest how this could happen, or give us any hints on what to try next? Andrew [1] http://haskell.org/ghc/docs/latest/html/users_guide/profiling.html
On Wed, Feb 01, 2006 at 12:28:59AM -0800, Andrew Pimlott wrote:
The Haskell code looks like
predQuery pred = case lookup pred ctxIdx of Just f -> {-# SCC "pq.foo'" #-} f
Note that f is a function, which we expect to be expensive when it is called. However, that should be charged to the definition of the function, right?
We found a work-around: We replaced f to the right of the annotation with \z z2 z3 -> f z z2 z3 and suddenly the right SCCs showed up in the profile (which enabled us to find the real leak, which was fixed by one strictness annotation). Actually, f takes 4 arguments, and forcing only 2 of them was not sufficient; in fact, forcing only 2 caused a different misleading SCC to dominate the profile! I don't claim to understand why. Andrew
Andrew Pimlott wrote:
On Wed, Feb 01, 2006 at 12:28:59AM -0800, Andrew Pimlott wrote:
The Haskell code looks like
predQuery pred = case lookup pred ctxIdx of Just f -> {-# SCC "pq.foo'" #-} f
Note that f is a function, which we expect to be expensive when it is called. However, that should be charged to the definition of the function, right?
We found a work-around: We replaced f to the right of the annotation with
\z z2 z3 -> f z z2 z3
and suddenly the right SCCs showed up in the profile (which enabled us to find the real leak, which was fixed by one strictness annotation). Actually, f takes 4 arguments, and forcing only 2 of them was not sufficient; in fact, forcing only 2 caused a different misleading SCC to dominate the profile! I don't claim to understand why.
I don't understand why either, but I do know that cost-centre stacks behave strangely in some corner cases. I have some examples stashed away somewhere, and it's been a long term ToDo item to really pin down the semantics of CCSs. If you could encapsulate your example as a bug report (preferably make it as small as possible), that would be great. Cheers, Simon
This is a small example of problem I described (don't ask me what it does): foo :: (() -> () -> [()] -> [()]) -> () -> [()] -> [()] foo k = \_ xs -> concatMap ($ [head xs]) [bar] where bar = let k' = k undefined undefined in \xs -> let k'' = [k' xs] in (() : (foldr1 (>>) k'')) k = foo (\_ -> k) --k a = foo (\_ -> k) a r = k undefined [] !! 4000 main = print r It is very sensitive to small changes, so be sure to use a high-quality cut-n-paste implementation. Basically, this code has a memory leak through xs; the xs passed to k' comes from the xs passed to foo, and is never forced, so a chain of thunks is built. If you run the code as is, either with time or heap profiling, you will see everything charged to k. If you instead use the commented version of k, you will see everything charged to foo, as expected. The command I run to test this is ghc --make -prof -auto-all Logic.hs && ./a.out +RTS -hc && hp2ps -c a.out.hp I'm using ghc 6.4.1 from Debian unstable. I'll copy this into a bug report too. I can try to explain what I did to arrive at this code if it wolud help, but it's a long story.... Andrew
participants (2)
-
Andrew Pimlott -
Simon Marlow