John,
This package looks very promising. I used iteratee for the yaml package, but
I had many of the concerns that you have mentioned below. Version 0.2 of
persistent is going to have some form of an enumerator interface for getting
the results of a query, and I eventually decided that iteratee was
introducing too much complexity to be a good candidate. However, I was able
to port the package[1] over to enumerator in about half an hour; I
especially benefited from your example applications.
The only concern that I had was the possible inefficiency of representing
all chunks as a list. In the case of persistent, the enumerator will
*always* generate a one-lengthed list, and the most common operation is
selectList, which returns all results as a list. If I used your consume
function, I believe there would be a *lot* of list traversals. Instead,
selectList[2] uses ([a] -> [a]) for building up the result internally. I
haven't really thought the issue through fully, so I can recommend anything
better. Perhaps more importantly, the simplification introduced by just
dealing with lists is well received.
Keep up the good work, I look forward to seeing more about enumerator.
Michael
[1] http://github.com/snoyberg/persistent/tree/enumerator
[2]
http://github.com/snoyberg/persistent/blob/enumerator/Database/Persist/Base....
http://github.com/snoyberg/persistent/tree/enumerator
On Thu, Aug 19, 2010 at 7:31 AM, John Millikin
Most of you have probably read Oleg's essays on using left-fold enumerators for incremental IO. In short, by encapsulating monadic left-folds in an "Iteratee" type, incremental pure processing is possible without using lazy IO. Sources to read:
Oleg: Streams and Iteratees < http://okmij.org/ftp/Streams.html > Magnus Therning: Trying to work out iteratees < http://therning.org/magnus/archives/735 > cdsmith: Iteratees Step By Step (Part 1) < http://cdsmith.wordpress.com/2010/05/23/iteratees-step-by-step-part-1/
John Millikin (me): Understanding Iteratees < http://ianen.org/articles/understanding-iteratees/ >
Currently, the primary package for left-fold enumerators is John Lato's "iteratee". It is based on Oleg's original code, extended to support various forms of containers, platform-specific IO, and codecs for the WAV and TIFF formats.
While I appreciate Mr. Lato's development of the package, I find it far too large, and its documentation too sparse, to effectively use. To correct this, I've written the "enumerator" package. It is also derived from Oleg's IterateeM.hs , but with a simplified API and significantly reduced dependency list.
Hackage entry: http://hackage.haskell.org/package/enumerator Haddock docs: http://ianen.org/haskell/enumerator/api-docs/ Source code (literate PDF): http://ianen.org/haskell/enumerator/enumerator.pdf
darcs get http://ianen.org/haskell/enumerator/
Additionally, I've included examples of using enumerators to implement simplified versions of the "cat" and "wc" utilities. These should serve as a useful starting point for anybody who wants to use enumerators in their own code:
http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/...
http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/...
There are already a few libraries using the existing "iteratee" package (snap, attoparsec-iteratee, hexpat-iteratee); I am very interested in advice from the authors of these libraries. In particular, are any of the removed features (ListLike, WrappedByteString, seeking) something your libraries depend on? Are there any useful combinators you'd like to see included? _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe