On Thu, Oct 2, 2008 at 4:28 PM, Johan Tibell wrote:

On Thu, Oct 2, 2008 at 3:50 PM, John Lato wrote:

...

Hello,

I've recently been looking at Oleg's Iteratee approach to I/O (http://okmij.org/ftp/Haskell/Iteratee/), and I was wondering about something. In general this seems to be a very good approach to handling sequential access, but I don't see a means of applying it where non-sequential (esp. random) access is desired. In fact, the only way I see of doing so is to either hold references to all read data in memory until the file has finished processing, or to iterate over the file multiple times, depending on how the data is to be processed. I can't see how either of these approaches would scale well. Is this an actual problem? Is there already a solution that I'm not aware of?

You could have the iteratee return the next position in the file it would like to operate on. The enumerator would use e.g. `pread` to perform random access reads in constant memory. I would have to know a little more about your use case before I can say if this would be a good idea or not.

Cheers,

Johan

Well, I feel a little stupid now, because that seems pretty obvious in retrospect. I don't have a particular use case in mind other than a generic "algorithm that requires random access to a file." In fact, for a lot of cases other than true random access, I can think of better approaches (often interleaving multiple enumerators). Thanks, John