On Thu, Jul 29, 2010 at 10:35 AM, Jason Dagit
On Thu, Jul 29, 2010 at 6:15 AM, Duncan Coutts < duncan.coutts@googlemail.com> wrote:
No idea what WrappedByteString is.
WrappedByteString is a newtype wrapper around ByteString that has a phantom type. This allows instances of to be written such that ByteString can be used with the iteratee library. You can see the source here if you're interested:
http://hackage.haskell.org/packages/archive/iteratee/0.3.5/doc/html/src/Data...
It would look like attoparsec's resumable parser:
data Result a = Fail !ByteString | Partial (ByteString -> Result a) | Done !ByteString a
runGet :: Get a -> ByteString -> Result a
Point is you feed it strict bytestring chunks. Then decoding a lazy bytestring can be implemented on top easily, as can decoding a sequence lazily.
Like attoparsec you'll probably want to write some other utility functions for working with Results. Attoparsec defines feed, parseWith, maybeResult, and eitherResult. I think you'll want something similar here.
And I forgot to mention.... Given those constructors for Result, how will you decode a sequence lazily? I agree you can consume the input lazily but, decodings won't produce any values until you hit Done. This is something I noticed with attoparsec as well. You can feed it incrementally, but it doesn't produce output until it reaches a Done state. In one project, were the input was line based records, I used ByteString's hGetLine to read individual lines and then parse them individually. That gave me a Result for each line and I could build incremental processing of records on top of that. Perhaps I was using attoparsec incorrectly, but as far as I can tell it doesn't have a way to incrementally produce results. I think what needs to be added is a way to 'yield' and 'resume'. What I can't figure out, is how to meaningfully return 'partial results' in general. In cases where the top level combinator is something like manyTill it makes sense, but in other cases it's unclear to me. And I think you'd need to add a constructor above that is some variation of "PartiallyDone a ByteString (ByteString -> Result a)". Jason