Curt Sampson
Let's say on my server I store a lot of static content in gzip'd form. In this case, I certainly want to be handling the transfer-encoding myself, because I don't want to uncompress my compressed copy to send it to the server, only so that the server can compress it again as it sends it.
I think you're confusing content-encoding and transfer-encoding here. Servers shouldn't be arbitrarily mucking around with your content-encoding.
Another example: we treat responses differently based on whether you explicitly set a content length or not. If the connection is HTTP/1.0, and you don't set a content length, we have to set "Connection: close" in the response headers and close the socket when we're finished sending the response....
Another approach would be to have interface library buffer the response as it's being generated and, when it's complete, calculate the content length and pass it on to the web server. In the case of a web server doing a lot of relatively small requests, this would probably be considerably more efficient. But of course you still need to the ability to send out the response as it's being generated for large responses that quickly produce some data, but take a relatively long time to produce all of the data.
This breaks "being able to stream in O(1) space", so it's out. Concrete examples: comet, internet radio, etc.
How much of that sort of logic goes into the interface layer?
I'm not sure what you mean by "interface layer," but I feel that, as much as possible, that sort of logic should be in standalone libraries that go between the "standard" web server interface and the user application.
By "interface layer" for the purposes of this conversation I mean
roughly the level of abstraction that Hack/WAI/WSGI/Rack/Java Servlets
are inhabiting.
G
--
Gregory Collins