John Goerzen writes:
Read and process the file in blocks:
I don't think that would really save much memory [...]
Given that the block-oriented approach has constant space requirements, I am fairly confident it would save memory.
and in fact, would likely just make the code a lot more complex. It seems like a simple wrapper around hGetContents over a file that uses block buffering would suffice.
Either your algorithm can process the input in blocks or it cannot. If it can, it doesn't make one bit a difference if you do I/O in blocks, because your algorithm processes blocks anyway. If your algorithm is *not* capable of processing blocks, then don't bother with block-oriented I/O -- and don't process large files unless you have 'seq' in all the right places. I happen to find adding those 'seq' calls much more difficult to get right than writing a computation that can be restarted. But your mileage may vary. Peter