On 26 January 2005 14:29, S. Alexander Jacobson wrote:
Ah, ok. So I ran the code with 100000 items, 50000 items, and 25000 items and got total memory in use of 28Mb, 15Mb, and 8Mb respectively. That comes to 260-280 bytes per record which is still an order of magnitude higher than the 20-30 bytes per record we would expect.
When using the ordinary 2-generation collector, memory in use will tend to be 2-3 times larger than the actual residency, because this is a copying collector.
On the other hand, I found 10mb, 5mb, and 2.5mb maximum residency, but that is still ~100 bytes per record.
Right.
Lastly, I tried "example +RTS -p -K5M -hc" and then looked at the resulting graph (attachment #2) and it shows a total of 1.6Mb heap allocated and if that data is correct, it does correspond roughly to our 20-30 bytes per record estimate.
That profile doesn't include the stack, which sounds reasonable.
Regarding stack, I tried "example +RTS -p -K5M -xt -hc" and then ran hp2ps and looked at the resulting graph (attachment #1) SYSTEM appears to use 4mb of stack at the very beginning (presumably to create "zipped"), but I don't know why it would. Then this stack is consumed by the rest of the program.
Stacks never get smaller in GHC, only bigger. If you need 4Mb of stack at one point in your program, you will forever have a 4Mb stack after that (fixing this wouldn't really buy you much; the memory doesn't get traversed or copied by the GC - but one thing you could do is madvise() to tell the OS it doesn't have to page the memory, though). I haven't looked at your code in detail, hopefully someone else can shed some light on why you're building up such a large stack in the first place. Cheers, Simon