Re: [Haskell-cafe] why does the binary library require so much memory?

31 Jul 2009

      bos:
...
On Fri, Jul 31, 2009 at 1:56 PM, Jeremy Shaw  wrote:
Using encode/decode from Binary seems to permamently increase my
    memory consumption by 60x fold. I am wonder if I am doing something
    wrong, or if this is an issue with Binary.
It's an issue with the Binary instance for lists, which forces the entire spine
of the list too early. This gives you a gigantic structure to hold onto.
This is the current instance

    instance Binary a => Binary [a] where
        put l  = put (length l) >> mapM_ put l
        get    = do n <- get :: Get Int
                    getMany n

    -- | 'getMany n' get 'n' elements in order, without blowing the stack.
    getMany :: Binary a => Int -> Get [a]
    getMany n = go [] n
     where
        go xs 0 = return $! reverse xs
        go xs i = do x <- get
                     -- we must seq x to avoid stack overflows due to laziness in
                     -- (>>=)
                     x `seq` go (x:xs) (i-1)

It used to be this, though,

        xs <- replicateM n get     -- now the elems.

-- Don

Re: [Haskell-cafe] why does the binary library require so much memory?

Don Stewart