Hello Jeremy, Saturday, August 1, 2009, 3:15:02 AM, you wrote:
So, the desired experience would be:
1. A program starts running and populates an IxSet. At this point in time n MB of RAM are being used.
2. We use Binary to snapshot the entire IxSet to disk. Since encode outputs an lazy ByteString, I would expect only a modest amount of additional memory to be required during this process.
what really happens here: while you expect that intermediate list produced lazily (it's haskell, after all!) and immediately stored to disk in chunks, "length l" in Binary instance forces entire list to be calculated before producing any output it should be easy to fix: if your IxSet have builtin elements counter or way to measure it without allocating tons of memory, you just need to make something like the following instance: instance Binary a => Binary (IxSet a) where put l = put (IxSet.length l) >> mapM_ put l get = fmap list_to_IxSet get -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com