Re: [Haskell-cafe] ANNOUNCE: enumerator, an alternative iteratee package

I'd mentioned in my previous email that I was concerned that "consume" would traverse the entire list for each new record pulled from the database. I'll try to start using the combinators instead; do get started with, it was easier to just use the raw datatypes to more easily see what was happening. Would this version of consume perhaps be better: consume :: Monad m => Iteratee e a m [a] consume = liftI $ step id where step acc chunk = case chunk of Chunks [] -> Continue $ returnI . step acc Chunks xs -> Continue $ returnI . (step $ acc . (xs ++)) EOF -> Yield (acc []) EOF Michael On Thu, Aug 19, 2010 at 8:40 AM, John Millikin wrote:

I'm glad to hear you found it useful!

For implementing selectList, have you considered using the "consume" iteratee? I suspect it would simplify your code to something like:

selectList a b c d = do res <- run $ select a b c d ==<< consume case res of Left e -> error e Right x -> return x

You might also want to look at the iteratee combinators (returnI, yield, continue), which would reduce some boilerplate, eg:

"Iteratee $ return $ Continue k" becomes "continue k" "Iteratee $ return $ Yield x EOF" becomes "yield x EOF"

On Wed, Aug 18, 2010 at 22:24, Michael Snoyman wrote:

...

John, This package looks very promising. I used iteratee for the yaml package, but I had many of the concerns that you have mentioned below. Version 0.2 of persistent is going to have some form of an enumerator interface for getting the results of a query, and I eventually decided that iteratee was introducing too much complexity to be a good candidate. However, I was able to port the package[1] over to enumerator in about half an hour; I especially benefited from your example applications.

The only concern that I had was the possible inefficiency of representing all chunks as a list. In the case of persistent, the enumerator will *always* generate a one-lengthed list, and the most common operation is selectList, which returns all results as a list. If I used your consume function, I believe there would be a *lot* of list traversals. Instead, selectList[2] uses ([a] -> [a]) for building up the result internally. I haven't really thought the issue through fully, so I can recommend anything better. Perhaps more importantly, the simplification introduced by just dealing with lists is well received. Keep up the good work, I look forward to seeing more about enumerator. Michael [1] http://github.com/snoyberg/persistent/tree/enumerator [2] http://github.com/snoyberg/persistent/blob/enumerator/Database/Persist/Base....