ANNOUNCE: binary 0.4: high performance, pure binary parsing and serialisation

Binary: high performance, pure binary encoding, decoding and serialisation for Haskell ---------------------------------------------------------------------- The Binary Strike Team is pleased to announce release 0.4 of Data.Binary, the pure, efficient binary serialisation library for Haskell, now available from Hackage: tarball: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary/0.4 darcs: darcs get http://darcs.haskell.org/binary The 'binary' package provides efficient serialisation of Haskell values to and from lazy ByteStrings, and a low level layer for high performance decoding and decoding binary data. ByteStrings constructed this way may then be written to disk, written to the network, or further processed (e.g. stored in memory directly, or compressed in memory with zlib or bzlib). *Very* high performance can be expected, with throughput over 1G/sec observed in practice (good enough for most networking scenarios, we suspect). Encoding and decoding are achieved by the functions: encode :: Binary a => a -> ByteString decode :: Binary a => ByteString -> a which mirror the read/show functions. Convenience functions for serialising to disk are also provided: encodeFile :: Binary a => FilePath -> a -> IO () decodeFile :: Binary a => FilePath -> IO a To serialise your Haskell data, all you need do is write an instance of Binary for your type. For example, suppose in an interpreter we had the data type: import Data.Binary import Control.Monad data Exp = IntE Int | OpE String Exp Exp We can serialise this to bytestring form with the following instance: instance Binary Exp where put (IntE i) = putWord8 0 >> put i put (OpE s e1 e2) = putWord8 1 >> put s >> put e1 >> put e2 get = do tag <- getWord8 case tag of 0 -> liftM IntE get 1 -> liftM3 OpE get get get The binary library has been heavily tuned for performance, particularly for writing speed. On average, Data.Binary is 10x faster than NewBinary, and has the advantage of a pure interface, and bytestring return values. Binary was developed by a team of 8 during the Haskell Hackathon, Hac 07, in January 2007, and this maintainence release has taken place during the second hackathon. Binary is portable, using the foreign function interface and cpp, and has been tested with Hugs and GHC. Happy hacking! -- Don

On 10/6/07, Don Stewart
The Binary Strike Team is pleased to announce release 0.4 of Data.Binary, the pure, efficient binary serialisation library for Haskell, now available from Hackage:
May I ask what are the changes? I didn't find some sort of changelog anywhere.
tarball: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary/0.4
...and this links gives a four-oh-four, not found. Thanks for this fine library, -- Felipe.

On 10/6/07, Felipe Almeida Lessa
May I ask what are the changes? I didn't find some sort of changelog anywhere.
There's the darcs changes list. The descriptions there in are .. terse :) But here's a selection: * Add getLazyByteStringNul. -- | Get a lazy ByteString that is terminated with a NUL byte. Fails -- if it reaches the end of input without hitting a NUL. getLazyByteStringNul :: Get L.ByteString * Fix strictness bug in runGetState that led to runtime errors. * Port binary to ghc 6.8 * add parallel driver (appears to be a parallel quicktest - which is cool because the tests took quite a while previously). -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org 650-283-9641

agl:
On 10/6/07, Felipe Almeida Lessa
wrote: May I ask what are the changes? I didn't find some sort of changelog anywhere.
There's the darcs changes list. The descriptions there in are .. terse :)
But here's a selection: * Add getLazyByteStringNul.
-- | Get a lazy ByteString that is terminated with a NUL byte. Fails -- if it reaches the end of input without hitting a NUL. getLazyByteStringNul :: Get L.ByteString
* Fix strictness bug in runGetState that led to runtime errors.
* Port binary to ghc 6.8
* add parallel driver (appears to be a parallel quicktest - which is cool because the tests took quite a while previously).
The main thing is porting to ghc 6.8 -- which means the new (*faster*) lazy bytestring representation, and the smp parallel quickcheck driver for the testsuite (it'll use N cores, watch the jobs migrate around). -- Don

don:
The main thing is porting to ghc 6.8 -- which means the new (*faster*) lazy bytestring representation, and the smp parallel quickcheck driver for the testsuite (it'll use N cores, watch the jobs migrate around).
Great news... Thanks for the collective work on this. I'm looking forward to firing up hpc. Tim

felipe.lessa:
On 10/6/07, Don Stewart
wrote: The Binary Strike Team is pleased to announce release 0.4 of Data.Binary, the pure, efficient binary serialisation library for Haskell, now available from Hackage:
May I ask what are the changes? I didn't find some sort of changelog anywhere.
tarball: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary/0.4
...and this links gives a four-oh-four, not found.
Thanks for this fine library,
Ah, hackages format changed, but my email template didn't: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary

Don Stewart wrote:
*Very* high performance can be expected, with throughput over 1G/sec observed in practice (good enough for most networking scenarios, we suspect).
Um... I wasn't aware that there was any harddrive or networking technology that goes this fast? Anyway, I'll have to take a look at the package and see if I can figure out how to work it. It is likely to prove quite useful...

On Sat, Oct 06, 2007 at 10:16:37PM +0100, Andrew Coppin wrote:
Don Stewart wrote:
*Very* high performance can be expected, with throughput over 1G/sec observed in practice (good enough for most networking scenarios, we suspect).
Um... I wasn't aware that there was any harddrive or networking technology that goes this fast?
Anyway, I'll have to take a look at the package and see if I can figure out how to work it. It is likely to prove quite useful...
It's what we call "ironic understatement", I suspect. Anyways, a single-mode optical fiber (as thin as a human hair), can transmit data at around 1,000,000 Gbit/s. Good luck finding a sufficiently advanced line driver; the last time I ran the Linux configuation program, it prompted up to 10,000 Mbit/s, so we're probably at least that far... Stefan

andrewcoppin:
Don Stewart wrote:
*Very* high performance can be expected, with throughput over 1G/sec observed in practice (good enough for most networking scenarios, we suspect).
Um... I wasn't aware that there was any harddrive or networking technology that goes this fast?
My bus is that fast. The point to take home is that serialising values (and possibly parsing stuff off the network) won't be a bottleneck in your Haskell code, for a while to come. -- Don
participants (6)
-
Adam Langley
-
Andrew Coppin
-
Don Stewart
-
Felipe Almeida Lessa
-
Stefan O'Rear
-
Tim Docker