#8400: Migrate the RTS to use libuv (or libev, or libevent) ----------------------------+---------------------------------------------- Reporter: schyler | Owner: simonmar Type: feature | Status: new request | Milestone: Priority: normal | Version: Component: Runtime | Keywords: System | Architecture: Unknown/Multiple Resolution: | Difficulty: Project (more than a week) Operating System: | Blocked By: Unknown/Multiple | Related Tickets: 635, 7353 Type of failure: | None/Unknown | Test Case: | Blocking: | ----------------------------+---------------------------------------------- Comment (by thoughtpolice): I think there are separate discussions happening here. One is about moving to libuv because it provides some nice abstractions we don't have to roll ourselves, and it works on Windows - and the other is about moving the scheduler and related stuff around for better I/O performance. These are somewhat separate discussions, IMO. To move the I/O manager into the RTS (and port it from C to Haskell) would be a sizeable amount of complex work, everything else aside (it's much, much easier to get all the tricky multicore stuff right in Haskell, obviously!) Of course this shouldn't kill the suggestion, but it's worth bringing this up - it adds a sizeable amount of complexity to an already extremely complex thing (the RTS.) It also adds other tradeoffs. This isn't a thing we're new to - we often carry the complexity burden for users - but we also don't want to totally overblow our complexity budget. Frankly I think we should look into improvements beyond "totally rewrite it in C" - this should be a last resort only, unless some numbers would suggest huge order-of-magnitude improvements. For windows, #7353 discusses some of the issues with Joey's I/O manager, the main one being that it needs some sort of scheduler integration to help mitigate the performance loss from round-tripping through the kernel on every I/O request. I can definitely buy that scheduler integration may help latency numbers on all platforms. There's also the somewhat-related issue that `-threaded` always adds a bit of latency anyway, and the Mio paper touches on this. Perhaps we should do a more thorough investigation and experiments first. And also, Joey really just wanted interruptible socket timeouts. Perhaps as a first step, we should: A) investigate if Joey's patches in #7353 can be resurrected, perhaps as a library (his work happened before the introduction of Mio!) and B) see if there's performance on the table. There's probably some scope for this if we can find a talented Windows engineer to run some good numbers. If A) can happen at least, and the improvements are stable, I may not be opposed to integrating this I/O manager into `base`, provided Simon thinks it's OK. Although bad performance may be unfortunate, it also brings feature parity to windows in this area (which was woefully limited in other ways.) As for the other stuff libuv provides - we already tend to have very lightweight wrappers around most system concurrency primitives so I'm not sure how much we need it, but being maintained and less code for us is a win too. (But I don't find this to be too much of a selling point to us, personally. Just a unified interface to IOCP/epoll/kqueue is enough if we were to use it.) Not to deter anyone, I just find the discussion here touching on a few related things, and it's all quite a lot to consider and think about! I think there's a ton of scope for improvement here, but it seems quite open ended in terms of design and implementation. Performance numbers and patches welcome! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8400#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler