Re: Strange memory usage problem
From: Ketil Malde
Hi,
I'm observing something I find quite strange. I have a program that normally streams through a file, and it runs in the expected constant space, consuming about 5MB, according to 'top'. When I uncomment a certain block of code in the library it uses, space consumption appears to grow linearly with the input size.
This is a bit surprising to me, since the code in question isn't used by the program at all, and yesterday, Cale confirmed this behavior on a recent GHC (6.10.3, I think - mine is 6.8.2 as shipped with Ubuntu 9.04).
I can confirm this behavior on GHC 6.10.4 on OSX 10.5.7. Try adding {-# INLINE getRB #-} above the getRB definition. That fixes it for me. I think that when the rest of the code is commented out, getRB is only called in one location, so GHC inlines it automatically. Since getRB is also called in the commented-out code, it probably isn't automatically inlined when that code is available. I suspect that the inlining allows GHC to infer some strictness property it otherwise can't. Cheers, John Lato
John Lato
I can confirm this behavior on GHC 6.10.4 on OSX 10.5.7.
Great, thanks!
Try adding {-# INLINE getRB #-} above the getRB definition. That fixes it for me.
And I especially appreciate this, of course. It appears to do the trick here, too.
I think that when the rest of the code is commented out, getRB is only called in one location, so GHC inlines it automatically. Since getRB is also called in the commented-out code, it probably isn't automatically inlined when that code is available. I suspect that the inlining allows GHC to infer some strictness property it otherwise can't.
Sounds reasonable. The getRB function 'get's a couple of bytestrings and lazy bytestrings from the input - apparently these aren't fully evaluated by the Get monad? I thought it might be the lazy ones, but I replaced them with strict bytestrings with no noticable success. I'm not entirely happy with the inline pragma as a solution, since it indicates that I'm depending on the optimizing-fu of GHC to make my program work, and this seems fragile. So far, it appears to be the best solution. -k -- If I haven't seen further, it is by standing in the footprints of giants
participants (2)
-
John Lato -
Ketil Malde