Pinning the arrays gives the GC much less flexibility. Especially if your objects are small. It means that the GC can't move things to compact the heap and you'll end up with lots of holes in between heap objects of things that were collected but the space couldn't be re-used because other objects didn't happen to fit in it. You end up with a very fragmented heap. You're essentially completely preventing the compacting GC from doing any compacting. Why can't it compact? Because you've told it that it isn't allowed to move anything! Honestly, I don't see why you think pinning all these little arrays should make things any better. It just means they can't be moved around it doesn't mean that magically they never have to be considered or freed by the GC. You only want to pin if you must do for some external reason or if the object is very big and therefore takes a significant amount of time to move. Duncan On Wed, 2006-10-18 at 19:50 +0400, Bulat Ziganshin wrote:
Hello glasgow-haskell-users,
i have program that holds in memory a lot of strings (filenames) and use my own packed-string library to represent these string. this library uses newByteArray# to allocate strings. in my benchmark run (with 300.000 filenames in memory) program prints the following statistics:
37,502,956 bytes maximum residency (16 sample(s))
2928 collections in generation 0 ( 6.90s) 16 collections in generation 1 ( 12.20s)
65 Mb total memory in use
most of this memory occupied by filenames. note that 28 mb of memory is used just for GC procedure (i use compacting GC)
then, i thought that using pinned byte arrays should significantly improve memory usage because these arrays can't be moved around and therefore, i thought, these arrays will not be involved in GC. amount of data which should be compacted by GC will decrease and amount of memory wasted by GC should also decrease
so i replaced in my packed str lib newByteArray# with newPinnedByteArray# and seen the following:
53,629,344 bytes maximum residency (14 sample(s))
2904 collections in generation 0 ( 6.97s) 14 collections in generation 1 ( 13.16s)
100 Mb total memory in use
why? why memory usage was increased? why another 47 megs is just wasted out? how GC works with pinned byte arrays? and why GC times was almost the same - i thought that GC for pinned data should be very different than GC for unpinned byte arrays