On Fri, Aug 20, 2010 at 1:12 PM, John Millikin
This thought occurred to me, but really, how often are you going to have a 10 GiB **text** file with no newlines? Remember, this is for text (log files, INI-style configs, plain .txt), not binary (HTML, XML, JSON). Off the top of my head, I can't think of any case where you'd expect to see 10 GiB in a single line.
In the worst case, you can just use "decode" to process bytes coming from the ByteString-based enumHandle, which should give nicely chunked text.
I was thinking about an attacker, not a use case. Think of a web server accepting queries using iteratees internally. This may open door to at least DoS attacks. And then, we use iteratees because we don't like the unpredictability of lazy IO. Why should iteratees be unpredictable when dealing with Text? Besides the memory consumption problem, there may be performance problems if the lines are too short. Cheers! =) -- Felipe.