On 15/07/2009 05:16, Brandon S. Allbery KF8NH wrote:
On Jul 14, 2009, at 21:48 , Kazu Yamamoto (山本和彦) wrote:
running well at the beginning. But after several hours, it receives an error, "buildFdSets: file descriptor out of range".
I believe the runtime uses select(), which has a hard limit (enforced by the kernel) that the maximum file descriptor id be 1023. (select() uses bitmasks and there is a limit on the size of a bitmask; see FD_SETSIZE.)
Strictly speaking it's the IO library, not the runtime, that calls select() when you're using -threaded. But otherwise that's all correct.
- pushes the limit of file descriptors to 65536 with setResourceLimit.
Reduce this to 1024, otherwise the runtime will eventually find itself dealing with file descriptors beyond the select() limit mentioned above. Someone with more knowledge of the Haskell runtime will have to advise as to possible ways around it if you really need more than 1024 file descriptors.
There's no easy workaround. We could have the IO library switch to using blocking read() calls for the out-of-range FDs to avoid the error from the IO manager, but that is likely to lead to a different problem: too many OS threads. The right fix is to move to using epoll() instead. I understand it is being worked on, but I don't know the current status (Johan?). Cheers, Simon