Re: [Haskell-cafe] ANNOUNCE: enumerator 0.4.8

I noticed this problem some time ago. Beyond just breaking monadic associativity, there are many other issues with standard definitions of iteratees: 1. It does not make sense in general to bind with an iteratee that has already consumed input, but there's no type-level difference between a "virgin" iteratee and one that has already consumed input; 2. Error recovery is ill-defined because errors do not describe what portion of the input they have already consumed; 3. Iteratees sometimes need to manage resources, but they're not designed to do so which leads to hideous workarounds; 4. Iteratees cannot incrementally produce output, it's all or nothing, which makes them terrible for many real world problems that require both incremental input and incremental output. Overall, I regard iteratees as only a partial success. They're leaky and somewhat unsafe abstractions. I'm experimenting with Mealy machines because I think they have more long-term promise to solve the problems of iteratees. Regards, John A. De Goes Twitter: @jdegoes LinkedIn: http://linkedin.com/in/jdegoes On Mar 26, 2011, at 1:03 PM, John Millikin wrote:

On Mar 26, 10:46 am, Michael Snoyman wrote:

...

As far as the left-over data in a yield issue: does that require a breaking API change, or a change to the definition of >>= which would change semantics??

It requires a pretty serious API change, as the definition of 'Iteratee' itself is at fault. Unfortunately, Oleg's new definitions also have problems (they can yield extra on a continue step), so I'm at a bit of a loss as to what to do. Either way, underlying primitives allow users to create iteratees with invalid/undefined behavior. Not very Haskell-y.

All of the new high-level functions added in recent versions are part of an attempted workaround. I'd like to move the Iteratee definitions themselves to a ``Data.Enumerator.Internal`` module, and add some words discouraging their direct use. There would still be some API breaks (the >>== , $$, and >==> operators would go away) but at least clients wouldn't be subjected to a complete rewrite.

Since the API is being broken anyway, I'm also going to take the opportunity to change the Stream type so it can represent "EOF + some data". That should allow lots of interesting behaviors, such as arbitrary lookahead.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe