
Hello, I would like ask to change WAI API a bit. Please include the following data constructor for Response: ResponseIO Status ResponseHeaders (Socket -> IO Bool) # Please choose better name than ResponseIO The function whose type is Socket -> IO Bool is to send a HTTP body to the socket and returns keepalive-or-not. This opens possibilities to find a better sending scheme than that which Warp adopts by default. I'm implementing a classical web server library on WAI (wai-app-file-cgi). It sends files to clients with ResponseFile. Warp uses the sendfile library for ResponseFile. The sendfile library uses (unnecessary) hSeek and hFileSize which call system calls. System calls causes context switches. Since GHC7's IO manager uses a single kernel thread, all Haskell threads stops. Calling system calls gives significant performance impact. ResponseIO enables to choose a function which sends files with minimum system calls for WAI users. I have already implemented this both in wai and warp, and confirmed that this works well. --Kazu

On Fri, Jun 3, 2011 at 10:40 AM, Kazu Yamamoto
Hello,
I would like ask to change WAI API a bit. Please include the following data constructor for Response: ResponseIO Status ResponseHeaders (Socket -> IO Bool)
# Please choose better name than ResponseIO
ResponseSocket?
The function whose type is Socket -> IO Bool is to send a HTTP body to the socket and returns keepalive-or-not. This opens possibilities to find a better sending scheme than that which Warp adopts by default.
I'm implementing a classical web server library on WAI (wai-app-file-cgi). It sends files to clients with ResponseFile. Warp uses the sendfile library for ResponseFile.
The sendfile library uses (unnecessary) hSeek and hFileSize which call system calls. System calls causes context switches. Since GHC7's IO manager uses a single kernel thread, all Haskell threads stops. Calling system calls gives significant performance impact.
ResponseIO enables to choose a function which sends files with minimum system calls for WAI users.
I have already implemented this both in wai and warp, and confirmed that this works well.
Obviously getting sendfile and Warp to be more efficient should be a goal here (which I know you're already following up on). My question is, barring issues with the sendfile package, is there any other use case for this constructor? I'm hesitant to add it, because it does not have a natural implementation for anything but a standalone server (i.e., CGI, FastCGI, wai-test will always have trouble with this). I just want to clarify the need before we make any changes. Michael

Hello Michael, Thank you for your reply.
ResponseIO enables to choose a function which sends files with minimum system calls for WAI users.
I have already implemented this both in wai and warp, and confirmed that this works well.
Obviously getting sendfile and Warp to be more efficient should be a goal here (which I know you're already following up on). My question is, barring issues with the sendfile package, is there any other use case for this constructor? I'm hesitant to add it, because it does not have a natural implementation for anything but a standalone server (i.e., CGI, FastCGI, wai-test will always have trouble with this). I just want to clarify the need before we make any changes.
If the sendfile library will be faster, this constructor is not necessary. I also agree that making the sendfile library faster is a right way to go. But I cannot contact the authors of the sendfile library. Do you have any advices on what I should do? --Kazu

On Mon, Jun 6, 2011 at 9:05 AM, Kazu Yamamoto
Hello Michael,
Thank you for your reply.
ResponseIO enables to choose a function which sends files with minimum system calls for WAI users.
I have already implemented this both in wai and warp, and confirmed that this works well.
Obviously getting sendfile and Warp to be more efficient should be a goal here (which I know you're already following up on). My question is, barring issues with the sendfile package, is there any other use case for this constructor? I'm hesitant to add it, because it does not have a natural implementation for anything but a standalone server (i.e., CGI, FastCGI, wai-test will always have trouble with this). I just want to clarify the need before we make any changes.
If the sendfile library will be faster, this constructor is not necessary. I also agree that making the sendfile library faster is a right way to go.
But I cannot contact the authors of the sendfile library. Do you have any advices on what I should do?
--Kazu
Hi Kazu, Well, if the authors of sendfile aren't responsive, another possibility would be to have Warp use your more efficient sendfile code directly (either by releasing it as its own package, or by moving that code into the Warp codebase). I would prefer this approach to changing WAI. Thoughts? Michael

Hello,
Well, if the authors of sendfile aren't responsive, another possibility would be to have Warp use your more efficient sendfile code directly (either by releasing it as its own package, or by moving that code into the Warp codebase). I would prefer this approach to changing WAI. Thoughts?
Please tell me which OSes Warp should support? I can provide sendfile functions to Linux/FreeBSD/MacOS but I cannot do for Windows. If Unix-like OSes are enough, I will take this approach. --Kazu

On Tue, Jun 7, 2011 at 8:14 AM, Kazu Yamamoto
Hello,
Well, if the authors of sendfile aren't responsive, another possibility would be to have Warp use your more efficient sendfile code directly (either by releasing it as its own package, or by moving that code into the Warp codebase). I would prefer this approach to changing WAI. Thoughts?
Please tell me which OSes Warp should support?
I can provide sendfile functions to Linux/FreeBSD/MacOS but I cannot do for Windows. If Unix-like OSes are enough, I will take this approach.
I think it should be possible to use conditional compilation to use your code for relevant OSes, and otherwise fall back to the sendfile package. Michael
--Kazu
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel

Conditional compilation should work fine but IIRC conditional dependencies aren't supported. You'll probably need to retain the sendFile dependency even if you integrate 山本様's code. Then just use his via conditional compilation as an optimization.
Max
On Jun 7, 2011, at 13:23, Michael Snoyman
On Tue, Jun 7, 2011 at 8:14 AM, Kazu Yamamoto
wrote: Hello,
Well, if the authors of sendfile aren't responsive, another possibility would be to have Warp use your more efficient sendfile code directly (either by releasing it as its own package, or by moving that code into the Warp codebase). I would prefer this approach to changing WAI. Thoughts?
Please tell me which OSes Warp should support?
I can provide sendfile functions to Linux/FreeBSD/MacOS but I cannot do for Windows. If Unix-like OSes are enough, I will take this approach.
I think it should be possible to use conditional compilation to use your code for relevant OSes, and otherwise fall back to the sendfile package.
Michael
--Kazu
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel
participants (3)
-
Kazu Yamamoto
-
Max Cantor
-
Michael Snoyman