safe lazy IO or Iteratee?

Hi everyone, This is not an attempt to start a flame war. I'm just trying to get a good feel for the advantages and disadvantages of the newer safe lazy io lib available on Hackage vs using Iteratee. It does appear to me that using something like Itereatee gives a bit of room to really tweak the low level enumerator/iteratee relationship, and, if you asked my old boss, I'm just a big fan of folds too, I use them a lot in my Erlang :-), so Iteratee really does appeal to me on many levels. Yet at the same time, I'm quite enamored with the beauty of "interact" and functions of that sort. I realize mixing the effects of the lazy IO and pure code may not be the clearest way to write code for everyone, but there is something about being able to get linewise data as interact (unlines . fmap someLineWiseFunction . lines) that is just kind of cool. Dave

Both have advantages and disadvantages. The primary advantage of lazy
IO over iteratees is that it's much, *much* easier to understand --
existing experience with monads can be used immediately. The downsides
of lazy IO, of course, are well documented[1][2][3].
Some are fixed by the safe/strict IO packages. However, safe lazy IO
is still "unpredictable" in that it's difficult to know how many
resources will be used, the order of some operations (eg, releasing
handles), or whether some particular expression will throw an
exception.
Iteratees are useful because the author can make and verify guarantees
about performance (eg, "this code will never read more than 4 KiB at
once"). They also allow pure code to determine the behavior of IO,
safely, which is useful when writing libraries which must deal with
large amounts of data (eg, streaming a file over HTTP). The Hyena web
server is written using iteratees, and from what I've heard the
authors have been quite happy with their properties.
I've also found iteratees to perform somewhat better than lazy IO,
though I don't know why.
Downside: iteratees are very hard to understand. I wrote a
decently-sized article about them[4] trying to figure out how to make
them useful, and some comments in one of Oleg's implementations[5]
suggest that the "iteratee" package is subtly wrong. Oleg has written
at least three versions (non-monadic, monadic, monadic CPS) and I've
no idea why or whether their differences are important. Even dons says
he didn't understand them until after writing his own iteratee-based
IO layer[6].
tl;dr: Lots of smart people, with a history of being right about this
sort of thing, say iteratees are better. Evidence suggests
iteratee-based IO is faster and more predictable than lazy IO.
Iteratees are really hard to understand.
[1] http://okmij.org/ftp/Haskell/Iteratee/Lazy-vs-correct.txt
[2] http://www.kuliniewicz.org/blog/archives/2010/01/27/happstack-and-streaming-...
[3] http://forums.xkcd.com/viewtopic.php?f=11&t=46780
[4] http://ianen.org/articles/understanding-iteratees/
[5] http://okmij.org/ftp/Haskell/Iteratee/IterateeM.hs , search for
"Drawbacks of this encoding"
[6] http://stackoverflow.com/questions/1319705/introduction-or-simple-examples-f...
On Thu, Feb 4, 2010 at 08:29, David Leimbach
Hi everyone, This is not an attempt to start a flame war. I'm just trying to get a good feel for the advantages and disadvantages of the newer safe lazy io lib available on Hackage vs using Iteratee. It does appear to me that using something like Itereatee gives a bit of room to really tweak the low level enumerator/iteratee relationship, and, if you asked my old boss, I'm just a big fan of folds too, I use them a lot in my Erlang :-), so Iteratee really does appeal to me on many levels. Yet at the same time, I'm quite enamored with the beauty of "interact" and functions of that sort. I realize mixing the effects of the lazy IO and pure code may not be the clearest way to write code for everyone, but there is something about being able to get linewise data as interact (unlines . fmap someLineWiseFunction . lines) that is just kind of cool. Dave _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Thanks for the detailed response below...
I must be able to understand how the resources will be used in my system for
mission-critical, long-running applications.
Dave
On Thu, Feb 4, 2010 at 12:51 PM, John Millikin
Both have advantages and disadvantages. The primary advantage of lazy IO over iteratees is that it's much, *much* easier to understand -- existing experience with monads can be used immediately. The downsides of lazy IO, of course, are well documented[1][2][3].
Some are fixed by the safe/strict IO packages. However, safe lazy IO is still "unpredictable" in that it's difficult to know how many resources will be used, the order of some operations (eg, releasing handles), or whether some particular expression will throw an exception.
Iteratees are useful because the author can make and verify guarantees about performance (eg, "this code will never read more than 4 KiB at once"). They also allow pure code to determine the behavior of IO, safely, which is useful when writing libraries which must deal with large amounts of data (eg, streaming a file over HTTP). The Hyena web server is written using iteratees, and from what I've heard the authors have been quite happy with their properties.
I've also found iteratees to perform somewhat better than lazy IO, though I don't know why.
Downside: iteratees are very hard to understand. I wrote a decently-sized article about them[4] trying to figure out how to make them useful, and some comments in one of Oleg's implementations[5] suggest that the "iteratee" package is subtly wrong. Oleg has written at least three versions (non-monadic, monadic, monadic CPS) and I've no idea why or whether their differences are important. Even dons says he didn't understand them until after writing his own iteratee-based IO layer[6].
tl;dr: Lots of smart people, with a history of being right about this sort of thing, say iteratees are better. Evidence suggests iteratee-based IO is faster and more predictable than lazy IO. Iteratees are really hard to understand.
[1] http://okmij.org/ftp/Haskell/Iteratee/Lazy-vs-correct.txt [2] http://www.kuliniewicz.org/blog/archives/2010/01/27/happstack-and-streaming-... [3] http://forums.xkcd.com/viewtopic.php?f=11&t=46780 [4] http://ianen.org/articles/understanding-iteratees/ [5] http://okmij.org/ftp/Haskell/Iteratee/IterateeM.hs , search for "Drawbacks of this encoding" [6] http://stackoverflow.com/questions/1319705/introduction-or-simple-examples-f...
Hi everyone, This is not an attempt to start a flame war. I'm just trying to get a good feel for the advantages and disadvantages of the newer safe lazy io lib available on Hackage vs using Iteratee. It does appear to me that using something like Itereatee gives a bit of room to really tweak the low level enumerator/iteratee relationship, and, if you asked my old boss, I'm just a big fan of folds too, I use them a lot in my Erlang :-), so Iteratee really does appeal to me on many levels. Yet at the same time, I'm quite enamored with the beauty of "interact" and functions of that sort. I realize mixing the effects of the lazy IO and pure code may not be the clearest way to write code for everyone, but
On Thu, Feb 4, 2010 at 08:29, David Leimbach
wrote: there is something about being able to get linewise data as interact (unlines . fmap someLineWiseFunction . lines) that is just kind of cool. Dave _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Hello John, Thursday, February 4, 2010, 11:51:59 PM, you wrote:
tl;dr: Lots of smart people, with a history of being right about this sort of thing, say iteratees are better. Evidence suggests iteratee-based IO is faster and more predictable than lazy IO. Iteratees are really hard to understand.
thank you! good time for iteratee tutorials? ;) -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Thu, Feb 4, 2010 at 1:26 PM, Bulat Ziganshin
Hello John,
Thursday, February 4, 2010, 11:51:59 PM, you wrote:
tl;dr: Lots of smart people, with a history of being right about this sort of thing, say iteratees are better. Evidence suggests iteratee-based IO is faster and more predictable than lazy IO. Iteratees are really hard to understand.
thank you! good time for iteratee tutorials? ;)
I put up my version of "cat" on my blog... http://leimy9.blogspot.com I'm just using runInteractiveCommand, and then putting the Handle returned through an enumerator/iteratee processing fold to get the contents.
-- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Thu, 4 Feb 2010 12:51:59 -0800, John Millikin
Both have advantages and disadvantages. The primary advantage of lazy IO over iteratees is that it's much, *much* easier to understand -- existing experience with monads can be used immediately. The downsides of lazy IO, of course, are well documented[1][2][3].
Some are fixed by the safe/strict IO packages. However, safe lazy IO is still "unpredictable" in that it's difficult to know how many resources will be used, the order of some operations (eg, releasing handles), or whether some particular expression will throw an exception.
I don't know if you include safe-lazy-io [1] here but its purpose is to cure these downsides. In particular file handles are properly released on time even when asking for a stream of a list of files. Moreover exceptions are not hidden and can be caught properly in the wrapping IO layer. Some restrictions are there though. For instance one cannot arbitrarily interleave different streams, one have to use one of the predefined interleaving scheme. Another limitation is that the client cannot react to errors without getting back to the non-lazy IO world, however this is a big selling point as well of lazy-IO: "reuse pure code". I don't want to claim that (safe-)lazy-IO should be used in all situations, this would be terribly wrong. However I recommend it for situations were the interleaving of input is standard and errors have not to be caught in the processing code. Best regards, [1]: http://www.haskell.org/pipermail/haskell/2009-March/021133.html -- Nicolas Pouillard http://nicolaspouillard.fr
participants (4)
-
Bulat Ziganshin
-
David Leimbach
-
John Millikin
-
Nicolas Pouillard