When using readFile to process a large number of files, I am exceeding the resource limits for the maximum number of open file descriptors on my system. How can I enhance my program to deal with this situation without making significant changes?
I note that if you use mmap(2) to map a disk file into virtual memory, you may close the file descriptor afterwards and still access the data. That might also be relatively economical in other respects. Pardon me if this has already been suggested. I should know how to make ByteStrings from offsets into a mapped region, but all I can say right now is, I'm pretty sure that would be no problem. Don't give them any finalizer. If you need to eventually unmap the files, that would be a problem, but I think if they're mapped right, you won't need that. They obviously have their own backing store, and if you just pretend they're unmapped, the host virtual memory management should ideally let you get away with that. And file data will be contiguous and sequential in memory, which in principle ought to be optimal for memory resources. Donn Cave, donn@drizzle.com