ANNOUNCE: binary: high performance, pure binary serialisation
Binary: high performance, pure binary serialisation for Haskell ---------------------------------------------------------------------- The Binary Strike Team is pleased to announce the release of a new, pure, efficient binary serialisation library for Haskell, now available from Hackage: tarball: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary/0.2 darcs: darcs get http://darcs.haskell.org/binary haddocks: http://www.cse.unsw.edu.au/~dons/binary/Data-Binary.html The 'binary' package provides efficient serialisation of Haskell values to and from lazy ByteStrings. ByteStrings constructed this way may then be written to disk, written to the network, or further processed (e.g. stored in memory directly, or compressed in memory with zlib or bzlib). Encoding and decoding are achieved by the functions: encode :: Binary a => a -> ByteString decode :: Binary a => ByteString -> a which mirror the read/show functions. Convenience functions for serialising to disk are also provided: encodeFile :: Binary a => FilePath -> a -> IO () decodeFile :: Binary a => FilePath -> IO a To serialise your Haskell data, all you need do is write an instance of Binary for your type. For example, suppose in an interpreter we had the data type: import Data.Binary import Control.Monad data Exp = IntE Int | OpE String Exp Exp We can serialise this to bytestring form with the following instance: instance Binary Exp where put (IntE i) = putWord8 0 >> put i put (OpE s e1 e2) = putWord8 1 >> put s >> put e1 >> put e2 get = do tag <- getWord8 case tag of 0 -> liftM IntE get 1 -> liftM3 OpE get get get The binary library has been heavily tuned for performance, particularly for writing speed. Throughput of up to 160M/s has been achieved in practice, and in general speed is on par or better than NewBinary, with the advantage of a pure interface. Efforts are underway to improve performance still further. Plans are also taking shape for a parser combinator library on top of binary, for bit parsing and foreign structure parsing (e.g. network protocols). Several projects are using binary already for serialisation: lambdabot : state file serialisation hmp3 : mp3 file database hpaste.org : pastes are stored in memory as compressed bytestrings, and serialised to disk on MACID checkpoints Binary was developed by a team of 8 during the Haskell Hackathon, Hac 07, and received 200+ commits over that period. You can see the commit graph here: http://www.cse.unsw.edu.au/~dons/images/commits/community/binary-commits.png The use of QuickCheck was critical to the rapid, safe development of the library. The API was developed in conjunction with the QuickCheck properties that checked the API for sanity. We were thus able to improve performance while maintaining stability. We feel that QuickCheck should be an integral part of the development strategy for all new Haskell libraries. Don't write code without it! Binary is portable, using the foreign function interface and cpp, and is tested with Hugs and GHC. Happy hacking! The Binary Strike Team, Lennart Kolmodin Duncan Coutts Don Stewart Spencer Janssen David Himmelstrup Bjorn Bringert Ross Paterson Einar Karttunen
dons:
Binary: high performance, pure binary serialisation for Haskell ----------------------------------------------------------------------
The Binary Strike Team is pleased to announce the release of a new, pure, efficient binary serialisation library for Haskell, now available from Hackage:
Ok, I forgot one point. It is possible to automatically derive instances of Binary for your custom types, if they inhabit Data and Typeable, using an SYB trick. Load tools/derive/BinaryDerive.hs into ghci, and bring your type into scope, then run: *Main> mapM_ putStrLn . lines $ derive (undefined :: Drinks) To have the source for the Binary instance for the type Drinks derivied for you: *Main> mapM_ putStrLn . lines $ derive (undefined :: Drinks) instance Binary Main.Drinks where put (Beer a) = putWord8 0 >> put a put Coffee = putWord8 1 put Tea = putWord8 2 put EnergyDrink = putWord8 3 put Water = putWord8 4 put Wine = putWord8 5 put Whisky = putWord8 6 get = do tag_ <- getWord8 case tag_ of 0 -> get >>= \a -> return (Beer a) 1 -> return Coffee 2 -> return Tea 3 -> return EnergyDrink 4 -> return Water 5 -> return Wine 6 -> return Whisky The use of SYB techniques to provide a 'deriving' script along with a new typeclass seems to be quite handy. -- Don
DrIFT 2.2.1 is out and now has support for the Data.Binary module. The old 'Binary' has been moved to 'BitsBinary' and 'Binary' now refers to the new 'Data.Binary' version of the library. the homepage is at: http://repetae.net/~john/computer/haskell/DrIFT/ the current list of deriving rules it knows about is: Binary: Binary Data.Binary binary encoding of terms BitsBinary efficient binary encoding of terms GhcBinary byte sized binary encoding of terms Debugging: Arbitrary Derive reasonable Arbitrary for QuickCheck Observable HOOD observable General: NFData provides 'rnf' to reduce to normal form (deepSeq) Typeable derive Typeable for Dynamic Generics: FunctorM derive reasonable fmapM implementation HFoldable Strafunski hfoldr Monoid derive reasonable Data.Monoid implementation RMapM derive reasonable rmapM implementation Term Strafunski representation via Dynamic Prelude: Bounded Enum Eq Ord Read Show Representation: ATermConvertible encode terms in the ATerm format Haskell2Xml encode terms as XML (HaXml<=1.13) XmlContent encode terms as XML (HaXml>=1.14) Utility: Parse parse values back from standard 'Show' Query provide a QueryFoo class with 'is', 'has', 'from', and 'get' routines from provides fromFoo for each constructor get for label 'foo' provide foo_g to get it has hasfoo for record types is provides isFoo for each constructor test output raw data for testing un provides unFoo for unary constructors update for label 'foo' provides 'foo_u' to update it and foo_s to set it John -- John Meacham - ⑆repetae.net⑆john⑈
Dear Donald! Donald Bruce Stewart (Fri, Jan 26, 2007 at 01:51:01PM +1100):
Binary: high performance, pure binary serialisation for Haskell ----------------------------------------------------------------------
[..]
Encoding and decoding are achieved by the functions:
encode :: Binary a => a -> ByteString decode :: Binary a => ByteString -> a
which mirror the read/show functions. Convenience functions for serialising to disk are also provided:
How can you detect that a decode fails? Do you need to catch an error in the IO monad? How can you know that there is remaining input in the ByteString? Regards, -- S.Karrmann The truth of a proposition has nothing to do with its credibility. And vice versa.
Hi
How can you know that there is remaining input in the ByteString?
http://www.cse.unsw.edu.au/~dons/binary/Data-Binary-Get.html The underlying Get monad provides that kind of control, if you need it. Thanks Neil
participants (4)
-
dons＠cse.unsw.edu.au -
John Meacham -
Neil Mitchell -
Stefan Karrmann