Peter, from 500 feet, we can't see much, but your strictness might actually be your problem depending on what "largish" looks like and whether you're reading your data from disc. It's entirely possible that your data structure updates or disc reads are head-strict and you're evaluating or loading the entirety of data in memory at a single update. -- Jeff On Thu, Jul 17, 2008 at 2:58 PM, Luke Palmer wrote:

On Thu, Jul 17, 2008 at 12:14 PM, Peter Gavin wrote:

...

Hello everyone,

I have this piece of code I've been working on, and I've been stuck on tracking down a space leak in it for some time now. The code is essentially a tight loop that updates a rather largish data structure with embedded functions that are called by the driver loop. The code doesn't accumulate any data as the loop runs (at least deliberately), so I would expect the memory profile to be flat. Unfortunately, the profile is a wedge :) I've added bangs and `seq` literally everywhere, and it looks (to me at least) like there's nothing left to be lazily evaluated anywhere. I've used retainer profiling, and the functions that are leaking space according to the profiler output are strict throughout.

I don't know what I can suggest as for general tactics. Without seeing the code it's hard to say what could be happening. Just remember that strictness is not always the answer!

...

From the very limited amount of information I got from this description, my first guess would be the data structure itself, or the functions inside it. If it's lazily generated, then you might not be seeing the full amount of space it's taking up at once. But that's just a guess.

Luke _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- I try to take things like a crow; war and chaos don't always ruin a picnic, they just mean you have to be careful what you swallow. -- Jessica Edwards

Replying to myself... Interesting. I removed all the bangs other than the obvious loop variables, and all the uses of seq that I had inserted, and there's still no leak. Does anyone know why the leak would disappear when GHC is using IO other than a generic (unspecified) monad? Is there something special about the >>= and return operators for IO that aren't true for other monads? Thanks, Pete Peter Gavin wrote:

Thanks for the responses.

This is basically what I've got looks like (grossly simplified):

data Monad m => Foo m a b = Foo { action :: m (Foo m a b, b) , update :: a -> Foo m a b }

The driver loop injects new values with update, and executes action whenever it's ready to, replacing the old Foo with the newly returned Foo.

I finally fixed the space leak it by inserting SPECIALIZE pragmas for Foo IO a b on every function that creates a Foo. I'm not sure if I can remove all the strictness annotations I've accumulated yet, though. This is a bit disconcerting, though, because in the future I'd like to not use IO and use a strict State instead. I hope I won't have to specialize for every monad that ends up getting used.

Thanks again, Pete

Jefferson Heard wrote:

...

Peter, from 500 feet, we can't see much, but your strictness might actually be your problem depending on what "largish" looks like and whether you're reading your data from disc. It's entirely possible that your data structure updates or disc reads are head-strict and you're evaluating or loading the entirety of data in memory at a single update.

-- Jeff

On Thu, Jul 17, 2008 at 2:58 PM, Luke Palmer wrote:

...

On Thu, Jul 17, 2008 at 12:14 PM, Peter Gavin wrote:

...

Hello everyone,

I have this piece of code I've been working on, and I've been stuck on tracking down a space leak in it for some time now. The code is essentially a tight loop that updates a rather largish data structure with embedded functions that are called by the driver loop. The code doesn't accumulate any data as the loop runs (at least deliberately), so I would expect the memory profile to be flat. Unfortunately, the profile is a wedge :) I've added bangs and `seq` literally everywhere, and it looks (to me at least) like there's nothing left to be lazily evaluated anywhere. I've used retainer profiling, and the functions that are leaking space according to the profiler output are strict throughout. I don't know what I can suggest as for general tactics. Without seeing the code it's hard to say what could be happening. Just remember that strictness is not always the answer!

...

From the very limited amount of information I got from this description, my first guess would be the data structure itself, or the functions inside it. If it's lazily generated, then you might not be seeing the full amount of space it's taking up at once. But that's just a guess.

Luke _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe