donn:
When using readFile to process a large number of files, I am exceeding the resource limits for the maximum number of open file descriptors on my system. How can I enhance my program to deal with this situation without making significant changes?
I note that if you use mmap(2) to map a disk file into virtual memory, you may close the file descriptor afterwards and still access the data. That might also be relatively economical in other respects. Pardon me if this has already been suggested.
I should know how to make ByteStrings from offsets into a mapped region, but all I can say right now is, I'm pretty sure that would be no problem. Don't give them any finalizer.
In fact, the (commented out code) for mmapFile :: FilePath -> IO ByteString is in Data.ByteString now. You can also provide munmap as the finaliser, and let the GC take care of that. -- Don