I notice that select is a bit of a bottleneck in the non-threaded RTS
once lots of sockets are in play. Between kernel and userspace, many a
cycle are wasted once we go past a few hundred clients. On some
operating systems, the fixed nature of fd_set imposes a surprisingly low
ceiling on the maximum number of concurrently open file descriptors.
I gather there's some kind of longish-term plan to build a more
pluggable RTS, but it seems like a nearer-term solution could be
implemented relatively cheaply, and wouldn't conflict with the ultimate
goal.
Would the GHC team be interested in receiving a patch that replaced
select, where possible, with an operating system's native, and hopefully
faster, event wait mechanism?
On Wed, May 30, 2007 at 09:21:36PM -0700, Bryan O'Sullivan wrote:
I notice that select is a bit of a bottleneck in the non-threaded RTS once lots of sockets are in play. Between kernel and userspace, many a cycle are wasted once we go past a few hundred clients. On some operating systems, the fixed nature of fd_set imposes a surprisingly low ceiling on the maximum number of concurrently open file descriptors.
I gather there's some kind of longish-term plan to build a more pluggable RTS, but it seems like a nearer-term solution could be implemented relatively cheaply, and wouldn't conflict with the ultimate goal.
Would the GHC team be interested in receiving a patch that replaced select, where possible, with an operating system's native, and hopefully faster, event wait mechanism?
I don't beleive the GHC team would be interested in receiving any patches to the non-threaded RTS, since it is scheduled for removal in 6.8 IIRC (leaving only threaded). Stefan
Stefan O'Rear wrote:
I don't beleive the GHC team would be interested in receiving any patches to the non-threaded RTS, since it is scheduled for removal in 6.8 IIRC (leaving only threaded).
But the threaded RTS calls select, too, only from Haskell instead of C.
So the problem remains, unless the plan to switch to the pluggable
event model will mature earlier than seems to be expected.
Bryan O'Sullivan wrote:
Stefan O'Rear wrote:
I don't beleive the GHC team would be interested in receiving any patches to the non-threaded RTS, since it is scheduled for removal in 6.8 IIRC (leaving only threaded).
But the threaded RTS calls select, too, only from Haskell instead of C. So the problem remains, unless the plan to switch to the pluggable event model will mature earlier than seems to be expected.
I'd be much more interested in seeing this done for the threaded RTS. This is the ticket: http://hackage.haskell.org/trac/ghc/ticket/635 AFAIK it's not a trivial swap-out, because epoll()/kevent() work quite differently from select(). You might want to take a look at Einar Karttunen's network-alt package which has epoll() support, I think. I don't think we can entirely remove the non-threaded RTS since some people still rely on being able to run Haskell code in a single-threaded environment. But I consider it deprecated. Cheers, Simon
participants (3)
-
Bryan O'Sullivan -
Simon Marlow -
Stefan O'Rear