Re: memory slop

On Tue, March 22, 2011 21:00:29 Tim Docker wrote:

I'm a bit shocked at the amount of wasted memory here. The sample data file has ~61k key/value pair. Hence ~122k ByteStrings - as you point out many of these are very small (1500 of them are empty). Assuming it's the bytestring that are generating slop, I am seeing ~500 bytes on average per bytestring!

It sounds like the space is allocated but unused pages. Unless you have messed with some kernel memory manager settings, unused virtual pages consume no physical RAM. You could confirm this by using ps to check how much RSS is actually used, compared to VSZ allocated (VSZ - RSS shouldn't include any actual data unless your system is actively swapping stuff to disk). If it is just unsued pages it's not a problem. Brandon