Hello Is there any plans to add error handling for binary? When it comes to binary parsing most awkward part is error handling I presume answer will be like "there are some vague plans but no one to implement them" so the second question how could it be done? Naive implementation of error handliing which converts Get monad into hybrid of state monad and error monad (Either like) gives about 5 times slowdown.
On Fri, Nov 27, 2009 at 12:00 PM, Alexey Khudyakov
Hello
Is there any plans to add error handling for binary? When it comes to binary parsing most awkward part is error handling
I presume answer will be like "there are some vague plans but no one to implement them" so the second question how could it be done?
Naive implementation of error handliing which converts Get monad into hybrid of state monad and error monad (Either like) gives about 5 times slowdown.
There is a Cereal package on Hackage that does exactly that, which may have better performance characteristics than what you've tried. I haven't personally benchmarked it. However, it insists on strict bytestrings everywhere, including as the return type of encode. This has made it mostly a no-go for me. -- Svein Ove Aas
I'm in the same quandary: Data.Binary from the binary package has no error handling Data.Serialize from the cereal package uses only strict ByteString I was going to add error handling to Binary as a weekend project (it isn't that hard), but when I contacted the developers of binary, I was pointed at cereal. But as my project can parse multi-megabyte objects off the wire, I really want lazy ByteString support. I understand from the cereal creators that lazy ByteStrings are not in the future of cereal, since they got a big speed boost by going to strict ByteStrings only. I understand that Bryan O'Sullivan might have done work on adding errors to Binary... Bryan? If that's available, can we get it? If not, shall I do the work to add error handling? It's a long weekend... I've got time! - Mark Mark Lentczner http://www.ozonehouse.com/mark/ mark@glyphic.com
On Fri, Nov 27, 2009 at 10:36 PM, Mark Lentczner
I'm in the same quandary:
Data.Binary from the binary package has no error handling Data.Serialize from the cereal package uses only strict ByteString
I was going to add error handling to Binary as a weekend project (it isn't that hard), but when I contacted the developers of binary, I was pointed at cereal. But as my project can parse multi-megabyte objects off the wire, I really want lazy ByteString support.
I understand from the cereal creators that lazy ByteStrings are not in the future of cereal, since they got a big speed boost by going to strict ByteStrings only.
I understand that Bryan O'Sullivan might have done work on adding errors to Binary... Bryan? If that's available, can we get it? If not, shall I do the work to add error handling? It's a long weekend... I've got time!
As an experiment I ported error handling from cereal to binary. It does work, passes all tests shipped with binary. Perfomance became worse. According to benchmarks shipped with binary I observed ~25% perfomance drop. However when I ported my program I have 40% speedup. I think it's because I removed code which work around lack error handling and replaced with error handling in Get monad As for strictess concerns. Patch doesn't seem to change strictess from current state. At least my program which uses very big inputs works without any changes. Comments and suggestions are welcome
В сообщении от 27 ноября 2009 23:09:53 Don Stewart написал:
alexey.skladnoy:
Hello
Is there any plans to add error handling for binary? When it comes to binary parsing most awkward part is error handling
It is now available in the 'cereal' package. A strict binary with explicit, checked error handling.
It does but it was already noted that cereal uses strict bytestrings which are not really convenient when dealing with huge inputs. One may end up using both binary and cereal which is not really satisfactory. Also binary's Get monad is strict since 0.5.0.2 so addition of error handling wont bring strictness problems.
alexey.skladnoy:
В сообщении от 27 ноября 2009 23:09:53 Don Stewart написал:
alexey.skladnoy:
Hello
Is there any plans to add error handling for binary? When it comes to binary parsing most awkward part is error handling
It is now available in the 'cereal' package. A strict binary with explicit, checked error handling.
It does but it was already noted that cereal uses strict bytestrings which are not really convenient when dealing with huge inputs. One may end up using both binary and cereal which is not really satisfactory.
It is quite hard to do a good job of lazy input, but explicit checked errors (turning it into an Either Error a makes the whole stream strict!). You might want to look at designs that interleave error tokens in the stream. -- Don
В сообщении от Пятница 27 ноября 2009 23:55:47 вы написали:
It is quite hard to do a good job of lazy input, but explicit checked errors (turning it into an Either Error a makes the whole stream strict!).
It does. To a degree. For example if one need to read 8 bytes to decide whether parsing fails or succeed parsing will force only these 8 bytes. With lookalike of runGetState laziness could be reintroduced manually. runGetStateErr :: Get a -> ByteString -> (Either Err a, Int64, ByteString) Recovery from errors is not trivial task anyway. And only programmer knows how it should be done.
You might want to look at designs that interleave error tokens in the stream.
Could you point out any examples?
On Fri, Nov 27, 2009 at 3:14 PM, Khudyakov Alexey
You might want to look at designs that interleave error tokens in the stream.
Could you point out any examples?
http://hackage.haskell.org/packages/archive/tar/0.3.1.0/doc/html/Codec-Archi... Check the type of the return value in Codec.Archive.Tar.read Failures are reported in-line, otherwise the entire result would be forced in order to determine that there was no error. Antoine
В сообщении от 27 ноября 2009 23:55:47 Don Stewart написал:
alexey.skladnoy:
It does but it was already noted that cereal uses strict bytestrings which are not really convenient when dealing with huge inputs. One may end up using both binary and cereal which is not really satisfactory.
It is quite hard to do a good job of lazy input, but explicit checked errors (turning it into an Either Error a makes the whole stream strict!).
Surprisingly it seems that it's not possible to avoid "unexpected end of input error" without forcing input. Primitive parsers fail in that case and only reasonably way to check against this is 'remaining' function which forces input. Moreover it's very inconvenient to check before significant fraction of calls that input has sufficient length. Of course it's possible to implement robust parsers with getWord8 & isEmpty duo but this is theoretical possibility.
participants (6)
-
Alexey Khudyakov -
Antoine Latter -
Don Stewart -
Khudyakov Alexey
-
Mark Lentczner -
Svein Ove Aas