Re: [Haskell-cafe] The state of binary (de)serialization

On Mon, Feb 25, 2013 at 4:51 PM, Alexander Solla wrote:

On Mon, Feb 25, 2013 at 11:59 AM, Johan Tibell wrote:

...

There are some blog posts and comments out there about merging cereal

...

and binary, is this what's the goal/going on (cfr runGetIncremental)?

It's most definitely the goal and it's basically done. The only thing I don't think we'll adopt from cereal is the instances from container types.

Why not? Those instances are useful. Without instances defined in binary/cereal, pretty much every Happstack (or, better said, every ixset/acidstate/safecopy stack) user will have to have orphan instances.

I will have to give a bit more context to answer this one. After the binary package was created we've realized that it should really have been two packages: * One package for serialization and deserialization of basic types, that have a well-defined serialization format even outside the package e.g. little and big endian integers, IEEE floats, etc. This package would correspond to Data.Binary.Get, Data.Binary.Builder, and Data.Binary.Put. * One package that defines a particular binary format useful for serializing arbitrary Haskell values. This package would correspond to Data.Binary. For the latter we need to decide what guarantees we make. For example, is the format stable between releases? Is the format public (such that other libraries can parse the output of binary)? Right now these two questions are left unanswered in both binary and cereal, making those packages less useful. Before we answer those questions we don't want to 1) add more dependencies to binary and 2) define serialization formats that we might break in the next release. So perhaps once we've settled these issues we'll include instances for containers. Also, cereal has a generic instance. Will the new binary?

That sounds reasonable. If someone sends a pull request Lennart or I will review and merge it. -- Johan