Hi,
I'm experiencing the "accept: resource exhausted (Too many open
files)" exception when trying to use sockets in my Haskell program.
The situation:
- Around a dozen Linux machines running my Haskell program,
transmitting thousands of messages to each other, sometimes within a
small period of time.
- I'm using the Network.Socket.ByteString.Lazy module to send and
receive lazy bytestrings
- The socket sent to the getMsg is bound and is listening, and is not
closed until the program exits. getMsg is called in a loop to receive
lazy bytestring from remote nodes. The socket is initialized with:
sock <- Network.Socket.socket (addrFamily myAddr) Stream defaultProtocol
bindSocket sock (addrAddress myAddr)
listen sock 10
Here's the code:
sendMsg :: Maybe HostName -> Int -> Lazy.ByteString -> IO ()
sendMsg dest sckt msg = do
result <- try $ withSocketsDo $ do
addrinfos <- getAddrInfo Nothing dest (Just (show sckt))
let serveraddr = head addrinfos
sock <- socket (addrFamily serveraddr) Stream defaultProtocol
connect sock (addrAddress serveraddr)
sendAll sock msg
sClose sock
case result of
Left (ex::IOException) -> return () -- permit send failure
Right _ -> return ()
getMsg :: Socket -> IO Lazy.ByteString
getMsg sock = do
result <- try $ withSocketsDo $ do
(conn, addr) <- accept sock
getContents conn
case result of
Left (ex::IOException) -> putStrLn (show ex) >> getMsg sock
Right msg -> return msg
The current topology is a master/slave setup. For some programs that
use these functions above, `sendMsg' is called thousands of times in
quick succession on the remote nodes, where the destination of the
`sendAll' function is the master node. Here's the maximum number of
simultaneous sockets I am permitted to have open on my Linux machines:
$ ulimit -n
1024
Indeed, when I experience the "accept: resource exhausted (Too many
open files)" exception, I check the number of open sockets, which
exceeds 1024, by looking at the contents of the directory:
ls -lah /proc/