It seems like HTTP pipelining and constant space usage are at odds with each other. A quick review of HTTP pipelining, http://en.wikipedia.org/wiki/HTTP_pipelining Handling persistent HTTP connections is no big deal, but HTTP pipelining is troublesome (which is why Opera is the only browser that has it enabled by default). The problem is that the client is allowed to send an unlimited number of Requests before it bothers to read any Responses. If your server uses the model of read a single Request and send the Response entirely before moving onto the next Request, then you could potentially deadlock because the client is still in the sending phase and you are blocked on a write(). So, we are basically forced to read all the Requests before sending a Response? But, then the client could perform a DoS by sending millions of requests.. Any thoughts on how best to handle this? Perhaps the only sane solution is to assume that any client that is going to using pipelining is going to be nice enough to actually read Responses even if it is still sending additional Requests (i.e., it has a writer thread and a reader thread)? In which case we can continue to read and respond to one request at a time[1]? I'm pretty sure this is what all the Haskell HTTP servers do at the moment and I have not heard any complaints... - jeremy [1] In theory, the server could processes multiple Requests in parallel as long as it returns the Responses in the correct order. Though it's not clear if this would actually help performance in most cases. Given the lack of pipelining support in clients anyway, it's nothing something I am especially interested in optimizing. :)
On Wed, Sep 18, 2013 at 5:45 PM, Jeremy Shaw
It seems like HTTP pipelining and constant space usage are at odds with each other.
A quick review of HTTP pipelining,
http://en.wikipedia.org/wiki/HTTP_pipelining
Handling persistent HTTP connections is no big deal, but HTTP pipelining is troublesome (which is why Opera is the only browser that has it enabled by default).
The problem is that the client is allowed to send an unlimited number of Requests before it bothers to read any Responses.
If your server uses the model of read a single Request and send the Response entirely before moving onto the next Request, then you could potentially deadlock because the client is still in the sending phase and you are blocked on a write().
Clients that pipeline need to be aware of this and use IO-multiplexing or separate threads for the read and write sides of the socket.
So, we are basically forced to read all the Requests before sending a Response? But, then the client could perform a DoS by sending millions of requests..
No, you don't read all of the requests, you read them one at a time. Exactly for that reason. Any thoughts on how best to handle this?
Perhaps the only sane solution is to assume that any client that is going to using pipelining is going to be nice enough to actually read Responses even if it is still sending additional Requests (i.e., it has a writer thread and a reader thread)? In which case we can continue to read and respond to one request at a time[1]? I'm pretty sure this is what all the Haskell HTTP servers do at the moment and I have not heard any complaints...
In fact, given the way the HTTP spec works, you're right that this is the
only sane thing to do.
G
--
Gregory Collins
On 2013-09-18 17:45, Jeremy Shaw wrote:
The problem is that the client is allowed to send an unlimited number of Requests before it bothers to read any Responses.
If your server uses the model of read a single Request and send the Response entirely before moving onto the next Request, then you could potentially deadlock because the client is still in the sending phase and you are blocked on a write().
So, we are basically forced to read all the Requests before sending a Response? But, then the client could perform a DoS by sending millions of requests..
Any thoughts on how best to handle this?
Implement SPDY instead or wait for HTTP 2.0 :). This is of course somewhat tongue-in-cheek, but really HTTP pipelining is notoriously complicated to get right and not sufficient for actually achieving what one wants. Multiplexing as in SPDY is much saner and based on actual use cases, admittedly from the "browser" portion of the HTTP client ecosystem, but actual use cases nonetheless. (Not saying that SDPY is all roses, btw, but that's irrelevant to this discussion.) If it were me, I'd simply not support HTTP pipelining.
Don't worry, I'm planning for SPDY and WebSockets as well :)
On Wed, Sep 18, 2013 at 12:40 PM, Bardur Arantsson
On 2013-09-18 17:45, Jeremy Shaw wrote:
The problem is that the client is allowed to send an unlimited number of Requests before it bothers to read any Responses.
If your server uses the model of read a single Request and send the Response entirely before moving onto the next Request, then you could potentially deadlock because the client is still in the sending phase and you are blocked on a write().
So, we are basically forced to read all the Requests before sending a Response? But, then the client could perform a DoS by sending millions of requests..
Any thoughts on how best to handle this?
Implement SPDY instead or wait for HTTP 2.0 :).
This is of course somewhat tongue-in-cheek, but really HTTP pipelining is notoriously complicated to get right and not sufficient for actually achieving what one wants. Multiplexing as in SPDY is much saner and based on actual use cases, admittedly from the "browser" portion of the HTTP client ecosystem, but actual use cases nonetheless.
(Not saying that SDPY is all roses, btw, but that's irrelevant to this discussion.)
If it were me, I'd simply not support HTTP pipelining.
_______________________________________________ web-devel mailing list web-devel@haskell.org http://www.haskell.org/mailman/listinfo/web-devel
participants (3)
-
Bardur Arantsson -
Gregory Collins -
Jeremy Shaw