Re: [Haskell-cafe] two problems with Data.Binary and Data.ByteString

bos:

On Tue, Aug 12, 2008 at 5:13 PM, Tim Newsham wrote:

...

The data type I'm storing is a Map (of maps):

type DailyDb = M.Map Date Daily type InstrsDb = M.Map String DailyDb

What's going on here?

The default marshalling scheme that Binary uses for lists and maps (which are flattened to lists before writing out) is not streamable. Instead of writing out data in chunks, it computes the length of the list and writes that, followed by the elements. Presumably on the read side, a huge thunk is being built up before any actual Map creation starts.

Maybe it makes sense to have the streamble list instance in Binary as well, with some examples? -- Don