Hello, I'm still chasing down a memory leak in my server application written in Haskell using GHC 6.4.x under MinGW/MSYS. In the scenario described below, I am repeating the same server request once per second continuously. After utilizing some memory monitoring tools I've discovered that memory usage fluctuates within in a range of approximately 850 KB (I assume as garbage is collected), but at regular intervals the range gets bumped up by ~ 1 MB. So in effect I get a stair-stepping of memory usage that keeps repeating until memory runs out. WinDbg has revealed that this stepping coincides with the GHC runtime function getMBlocks(). My question is: what conditions affect when the runtime determines that it needs more memory? Is it a pure "no more room" trigger or is there some sort of algorithm behind it? I assume that this behavior means I still have a space leak somewhere in my Haskell code, though none of leak check tools I utilize indicate such. Thanks, Rich
Rich Fought wrote:
I'm still chasing down a memory leak in my server application written in Haskell using GHC 6.4.x under MinGW/MSYS. In the scenario described below, I am repeating the same server request once per second continuously.
After utilizing some memory monitoring tools I've discovered that memory usage fluctuates within in a range of approximately 850 KB (I assume as garbage is collected),
What tool(s) did you use to obtain this figure?
but at regular intervals the range gets bumped up by ~ 1 MB. So in effect I get a stair-stepping of memory usage that keeps repeating until memory runs out. WinDbg has revealed that this stepping coincides with the GHC runtime function getMBlocks().
My question is: what conditions affect when the runtime determines that it needs more memory? Is it a pure "no more room" trigger or is there some sort of algorithm behind it?
The block allocation layer grabs another Mb whenever it runs out, that's all. The vast majority of the blocks in use by the storage manager will be either free or assigned to the heap, so the most likely possibility is that you do have a space leak (although that disagrees with the 850Kb figure you gave above).
I assume that this behavior means I still have a space leak somewhere in my Haskell code, though none of leak check tools I utilize indicate such.
Did you try GHC's heap profiler? Or simply running your program with +RTS -Sstderr will give you a clue about the shape of the heap usage - each line is a single GC, and it includes the amount of live data at that point. If your program has a flat heap profile and yet is still grabbing more memory, then something else is going on. Cheers, Simon
Simon Marlow wrote:
What tool(s) did you use to obtain this figure?
This particular figure was gathered using perfmon logs collecting once per second from my application, while running in WinDbg to break on getMBlocks(). The particular memory variables tracked are "Private Bytes" and "Working Set."
Did you try GHC's heap profiler? Or simply running your program with +RTS -Sstderr will give you a clue about the shape of the heap usage - each line is a single GC, and it includes the amount of live data at that point.
If your program has a flat heap profile and yet is still grabbing more memory, then something else is going on.
Yes I did try the profiler, but I couldn't really garner any useful information from it. Most likely because I don't understand how to use it correctly. :) I should revisit this however, as I have since discovered and fixed some leaks. It should at least look different. Thanks, Rich
Did you try GHC's heap profiler?
Or simply running your program with +RTS -Sstderr will give you a clue about the shape of the heap usage - each line is a single GC, and it includes the amount of live data at that point.
If your program has a flat heap profile and yet is still grabbing more memory, then something else is going on.
Cheers, Simon
Well the profile is definitely not flat. I'm using the profiling tools and Network.accept is apparently the major offender using both -hc and -hr profiles. I take it this implies that the products of this function are being retained elsewhere. I do pass the handle off to another thread via forking - this shouldn't be an issue should it? Another retainer involves SYSTEM and Network.accept - what exactly does SYSTEM mean? Regards, Rich
Rich Fought wrote:
Did you try GHC's heap profiler?
Or simply running your program with +RTS -Sstderr will give you a clue about the shape of the heap usage - each line is a single GC, and it includes the amount of live data at that point.
If your program has a flat heap profile and yet is still grabbing more memory, then something else is going on.
Cheers, Simon
Well the profile is definitely not flat.
I'm using the profiling tools and Network.accept is apparently the major offender using both -hc and -hr profiles. I take it this implies that the products of this function are being retained elsewhere. I do pass the handle off to another thread via forking - this shouldn't be an issue should it? Another retainer involves SYSTEM and Network.accept - what exactly does SYSTEM mean?
SYSTEM is usually the stack, or some other root maintatined by the system (eg. a StablePtr). Cheers, Simon
participants (2)
-
Rich Fought -
Simon Marlow