Yes. We decided that having the Get monad as a lazy state monad just doesn't make sense. It makes this one use case work, but causes more problems generally. Certainly we need to be able to do lazy binary deserialisation too, but our feeling is that it should be more explicit and integrate better with error handling. Using a lazy state monad gives us neither.
This is valid point. I think I'll just use variant with explicit laziness.
Note also that if we changed runGetState to do any error handling that this would also break the lazy state monad properties that you were previously relying upon.
I'd appreciate addition of error handling very much. Currently I have
to use ErrorT and do checks by hands. I have very simple parsers so
it's possible. It's not possible to do such things in general.
Trading laziness for error handling is good thing I believe. At the
moment it's hard to decode possibly malformed data.
On 10/17/09, Duncan Coutts
On Sat, 2009-09-19 at 00:36 +0400, Khudyakov Alexey wrote:
Hello
I run into problems with new binary package. Following function reads a list of elements one by one until end of stream. List is very long (won't fit into memory).
Sorry for the late reply.
In binary-0.5.0.1 and earlier it read list lazily. Now it seems that it tries to read whole list to memory. Program does not produce any output and memory usage steadily grows.
Yes. We decided that having the Get monad as a lazy state monad just doesn't make sense. It makes this one use case work, but causes more problems generally. Certainly we need to be able to do lazy binary deserialisation too, but our feeling is that it should be more explicit and integrate better with error handling. Using a lazy state monad gives us neither.
getStream :: Get a -> Get [a] getStream getter = do empty <- isEmpty if empty then return [] else do x <- getter xs <- getStream getter return (x:xs)
How could I add laziness to this function to revert to old behavior.
You can make it explicitly lazy using something like:
getStream :: Get a -> BS.ByteString -> [a] getStream getter bs = unfoldr step (bs, 0) where step (bs, off) = case runGetState getOne bs off of (Nothing, _, _ ) -> Nothing (Just x, bs', off') -> Just (x, (bs', off'))
getOne = do empty <- isEmpty if empty then return Nothing else fmap Just getter
Note the distinct lack of error handling, but if we had runGetState return an Either then you could still make this code lazy, but you'd have to decide what to do when you got a decoding error. You could at this point translate it into a pure error, or enrich your output stream type to account for the possibility of errors.
Note also that if we changed runGetState to do any error handling that this would also break the lazy state monad properties that you were previously relying upon.
Duncan