Re: [Haskell-cafe] The state of binary (de)serialization

On Mon, 2013-02-25 at 11:59 -0800, Johan Tibell wrote:

On Mon, Feb 25, 2013 at 4:30 AM, Nicolas Trangez wrote:

...

- cereal supports chunk-based 'partial' parsing (runGetPartial). It looks like support for this is introduced in recent versions of 'binary' as well (runGetIncremental)

Yes. Binary now support an incremental interface. We intend to make sure binary has all the same functionality as cereal. We'd like to move away from having two packages if possible and since binary has the larger installed user base we're trying to make that the go-to package.

This will certainly make things more obvious (and maybe ready for HP inclusion?).

...

- cereal can output a strict bytestring (runPut) or a lazy one (runPutLazy), whilst binary only outputs lazy ones (runPut)

The lazy one is more general and you can use toStrict (from bytestring) to get a strict ByteString from a lazy one, without loss of performance.

Sure. Turned out I was using lazy bs' anyway so switched to 'binary' for deserialization.

...

- Next to binary and cereal, there's bytestring's Builder interface for serialization, and Simon Meier's "blaze-binary" prototype

Simon's builder (originally developed in blaze-binary) has been merged into the bytestring package. In the future binary will just re-export that builder.

I was referring to https://github.com/meiersi/blaze-binary

...

Overall: what's the advised future-proof strategy of handling binary (de)serialization?

Use binary or the builder from bytestring whenever you can. Since the builder in bytestring was recently added you might have to fall back to blaze-builder if you believe your users can't rely on the latest version of bytestring.

I switched to Builder for serialization. It seems to create 'more strict' lazy bytestrings than the cereal based code (as in: cereal seems to create a new Chunk whenever appending a lazy bytestring, whilst Builder concats them into a single chunk, at least for the short strings I've been using). The Monoidal interface feels very natural, maybe even more natural than the Monad interface of PutM in binary/cereal: instance Argument a => Argument [a] where put l = word32LE cnt <> s where (cnt, s) = foldr (\e (c, m) -> (c + 1, put e <> m)) (0, mempty) l Thanks, Nicolas