John,
This package looks very promising. I used iteratee for the yaml package, but I had many of the concerns that you have mentioned below. Version 0.2 of persistent is going to have some form of an enumerator interface for getting the results of a query, and I eventually decided that iteratee was introducing too much complexity to be a good candidate. However, I was able to port the package[1] over to enumerator in about half an hour; I especially benefited from your example applications.
The only concern that I had was the possible inefficiency of representing all chunks as a list. In the case of persistent, the enumerator will *always* generate a one-lengthed list, and the most common operation is selectList, which returns all results as a list. If I used your consume function, I believe there would be a *lot* of list traversals. Instead, selectList[2] uses ([a] -> [a]) for building up the result internally. I haven't really thought the issue through fully, so I can recommend anything better. Perhaps more importantly, the simplification introduced by just dealing with lists is well received.
Keep up the good work, I look forward to seeing more about enumerator.
Michael
On Thu, Aug 19, 2010 at 7:31 AM, John Millikin
<jmillikin@gmail.com> wrote:
Most of you have probably read Oleg's essays on using left-fold
enumerators for incremental IO. In short, by encapsulating monadic
left-folds in an "Iteratee" type, incremental pure processing is
possible without using lazy IO. Sources to read:
Oleg: Streams and Iteratees < http://okmij.org/ftp/Streams.html >
Magnus Therning: Trying to work out iteratees <
http://therning.org/magnus/archives/735 >
cdsmith: Iteratees Step By Step (Part 1) <
http://cdsmith.wordpress.com/2010/05/23/iteratees-step-by-step-part-1/
>
John Millikin (me): Understanding Iteratees <
http://ianen.org/articles/understanding-iteratees/ >
Currently, the primary package for left-fold enumerators is John
Lato's "iteratee". It is based on Oleg's original code, extended to
support various forms of containers, platform-specific IO, and codecs
for the WAV and TIFF formats.
While I appreciate Mr. Lato's development of the package, I find it
far too large, and its documentation too sparse, to effectively use.
To correct this, I've written the "enumerator" package. It is also
derived from Oleg's IterateeM.hs , but with a simplified API and
significantly reduced dependency list.
Hackage entry: http://hackage.haskell.org/package/enumerator
Haddock docs: http://ianen.org/haskell/enumerator/api-docs/
Source code (literate PDF): http://ianen.org/haskell/enumerator/enumerator.pdf
darcs get http://ianen.org/haskell/enumerator/
Additionally, I've included examples of using enumerators to implement
simplified versions of the "cat" and "wc" utilities. These should
serve as a useful starting point for anybody who wants to use
enumerators in their own code:
http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs
http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs
There are already a few libraries using the existing "iteratee"
package (snap, attoparsec-iteratee, hexpat-iteratee); I am very
interested in advice from the authors of these libraries. In
particular, are any of the removed features (ListLike,
WrappedByteString, seeking) something your libraries depend on? Are
there any useful combinators you'd like to see included?
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe