Hi, I confirmed that the throughput of acme-http is also improved: No yield: 60,772 req/s Yield: 133,478 req/s I will try the internal-state branch of Warp next. --Kazu

On Mon, Jul 22, 2013 at 11:28 AM, Kazu Yamamoto wrote:

...

Hi all, Cc: Simon Marlow

I think I found a way to make the throughput of web servers much better. In short, put "Control.Concurrent.yield" after sending something. This enables good schedule shaping.

Typical code for web servers is like this:

loop = do keepAlive <- receiveRequest sendResponse when keepAlive loop

With this code, a typical sequence of system calls is like this:

recvfrom(13, ) -- thread A sendto(13, ) -- thread A recvfrom(13, ) = -1 EAGAIN -- thread A epoll_ctl(3, ) -- IO manager recvfrom(14, ) -- thread B sendto(14, ) -- thread B recvfrom(14, ) = -1 EAGAIN -- thread B epoll_ctl(3, ) -- IO manager

Since each thread calls recvfrom() immediately after sendto(), the possibility that recvfrom() can receive a request is low. This involves the IO manager.

To make the possibility higher, I put "yield" after sendResponse:

loop = do keepAlive <- receiveRequest sendResponse yield when keepAlive loop

Yield pushes its Haskell thread onto the end of thread queue. So, another thread can work. During the work of other threads, a request message would arrive.

recvfrom(13, ) -- thread A sendto(13, ) -- thread A recvfrom(14, ) -- thread B sendto(14, ) -- thread B recvfrom(13, ) -- thread A sendto(13, ) -- thread A

I tested this idea on SimpleServer and confirmed that its throughput is doubled:

No yield: 58,152 req/s ( https://gist.github.com/AndreasVoellmy/4184520#file-simpleserver-hs) Yield: 113,048 req/s ( https://gist.github.com/kazu-yamamoto/6051118#file-simpleserver2-hs)

Unfortunately, yield does not work for Warp as expected. It does not pass control to another thread and behaves as used to be.

So, my question: why doesn't "yield" work in Warp? ResourceT is doing a special thing? My code change is here:

https://github.com/yesodweb/wai/commit/4e9c6316ad59cd87be13000d15df8e6fd7c31...

I'm using GHC head (with multicore IO manager). I'm sure that the following patch to fix the bug of yield is applied:

https://github.com/ghc/ghc/commit/66839957a376dbe85608822c1820eb6c99210883

P.S.

It would be appreciated if someone tries this idea on other web servers.

--Kazu

_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

Hey Kazu,

As usual, exciting work on the I/O manager.

I've had an experiment in the back of my mind for a while now, and after this email I decided to finally try it out. On the internal-state branch[1] of the WAI repo, I've implemented a relatively simple change:

* All operations that used to live in ResourceT IO now live in IO. * The Request value has a new field called resourceInternalState, which holds onto the internal state of the ResourceT transformer. Using the internal state API[2], it should be trivial to convert ResourceT code to live in the IO monad instead.

Would you be able to see if this change makes any meaningful impact on the benchmarks you're running? If so, we can discuss the proper way to make this transition. Note that, as I've implemented it, this is a breaking change in WAI, though we could take less extreme measures and get most of the performance benefit most likely.

Michael

[1] https://github.com/yesodweb/wai/tree/internal-state [2] http://hackage.haskell.org/packages/archive/resourcet/0.4.7.1/doc/html/Contr...