Thanks for the input! The sClose on line 21 is actually never called: the "dispatch_on_request" loops forever. The sClose implied by Network.HTTP.close on line 34 is called, but (in theory) after the "ErrorClosed" error is produced. This error comes from the "receiveHTTP" on line 37. Which should occur before the Network.HTTP.close. The reasoning of "client side has already closed the connection" makes sense, but I would imagine the client (JMeter in this case) would wait for a response before closing the connection. JMeter does receive a reply in the failure cases. Which is the reply formed on line 23. Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time. Cheers, Corey -Corey O'Connor coreyoconnor@gmail.com http://corebotllc.com/ On Mon, Oct 21, 2013 at 1:57 PM, Jeremy Shaw wrote:

my hypothesis is that you are getting 'ErrorClosed' when calling 'sClose' because the client side has already beat you to the punch. But that is just a hypothesis.

- jeremy

...

I attempted to write a tiny HTTP server using only Haskell Platform packages:

* https://github.com/coreyoconnor/tiny-http-hp

Which works... Unless the threaded runtime is used.

When compiled using the threaded runtime and run with +RTS -N this server fails to reply correctly to ~3% of requests. The expectation is that the reply will be exactly the request body. However, 3% of the time the HTTP request fails to be parsed due to an "ErrorClosed"

The server then executes:

* https://github.com/coreyoconnor/tiny-http-hp/blob/master/TinyHttp.hs#L23

which responds to the client. The client receives the response correctly most of the time.

My current hypothesis is that some aspect of lazy IO is not playing nice. The connection is being closed before the request can be completely

On Mon, Oct 21, 2013 at 3:34 PM, Corey O'Connor wrote: parsed.

...

Is this correct? What am I missing?

I know there are other HTTP server packages that could be used. However,

the

...

exercise was to build a HTTP server using only Haskell Platform packages.

Below is the main body of the code:

main = withSocketsDo $ do

http_socket <- listenOn $ PortNumber 9090

dispatch_on_accept http_socket $ either handle_failed_request handle_valid_request

sClose http_socket

handle_failed_request failure = return $ Response (4,0,0) "Bad Request" [mkHeader HdrConnection "close"]

(encodeUtf8 $ pack $ show failure)

handle_valid_request request = do

let request_body = rqBody request

return $ Response (2,0,0) "OK" [mkHeader HdrConnection "close"]

(encodeUtf8 $ pack $ show request_body)

dispatch_on_accept http_socket handler = forever $ accept http_socket >>= forkIO . httpHandler . fst

where httpHandler client_socket = bracket (socketConnection "client" 0 client_socket)

Network.HTTP.close

client_interact client_interact :: HandleStream BS.ByteString -> IO ()

client_interact byte_stream = receiveHTTP byte_stream >>= handler

...

...

= respondHTTP byte_stream

-Corey O'Connor coreyoconnor@gmail.com http://corebotllc.com/

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Mon, Oct 21, 2013 at 7:28 PM, Joey Adams wrote:

On Mon, Oct 21, 2013 at 5:15 PM, Corey O'Connor wrote:

...

Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

The client may be trying to reuse the connection; see http://en.wikipedia.org/wiki/HTTP_persistent_connection. This seems consistent with your results:

* With the close call: connection is closed right after client sends a second request, so client whines. * Without the close call: client waits around for a response, then gives up and establishes another connection.

To confirm, add another receiveHTTP call client_interact and see if it returns another Request.

​Thanks!​ I'll try your test and see if anything changes. Though I thought the code accounted for this: A connection close header is added to each response. Which I thought would cause the client to close the connection.​​ I could be totally wrong tho. There could be some additional aspect of HTTP connections I do not understand. Properly supporting persistent connections should be pretty easy. There is another Network.HTTP server package that does this: http://hackage.haskell.org/package/http-server-1/docs/Network-HTTP-Server.ht... So I'll give that a shot and report back. Thanks! Corey

My current guess: This is a bug in how the socket is set up or one of the underlying libraries. Using the stream debug hooks indicates the failure is due to "readLine" returning an empty line when parsing the request header. I cannot tell if an empty line is returned due to an actual 0 byte return from read/recv or some processing of the read data. Still, this would indicate the HTTP library considers the request to be malformed. The question is then: Are the requests from JMeter malformed? A wireshark capture was performed of the JMeter test traffic. This wireshark data indicated 5000 HTTP requests were made. Which is what JMeter produced. All 5000 of these requests were byte identical. This capture did not affect the test results: A similar number of requests failed. I tried to instrument the read/recv calls using dtrace. This indicated read, and not recv, was used when reading from teh socket. I was unable to determine anything else tho. Probably a better idea to instrument HTTP library a bit more. Cheers, Corey -Corey O'Connor coreyoconnor@gmail.com http://corebotllc.com/ On Tue, Nov 5, 2013 at 6:10 PM, Corey O'Connor wrote:

I've updated the code to handle persistent connections. As best I can figure anyways.. Unfortunately this does not resolve the issue. The server still reports the connection being closed before the full response is read.

The test is done via JMeter. See the perf_test.jmx file. I've confirmed JMeter is include "connection: close" headers in all requests. Which means the persistent connection code is not even being exercised. From instrumenting the code I can confirm this is the case: All requests include "connection: close" and the client never performs a second request using the same stream.

An additional data point: The same test when run with 1 capability (EG: +RTS -N1) does *not* fail. There are no connection close errors. Only when running with multiple capabilities does the test fail.

Cheers, Corey

-Corey O'Connor coreyoconnor@gmail.com http://corebotllc.com/

On Tue, Oct 22, 2013 at 4:54 PM, Corey O'Connor wrote:

...

On Mon, Oct 21, 2013 at 7:28 PM, Joey Adams wrote:

...

On Mon, Oct 21, 2013 at 5:15 PM, Corey O'Connor

...

wrote:

...

Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

The client may be trying to reuse the connection; see http://en.wikipedia.org/wiki/HTTP_persistent_connection. This seems consistent with your results:

* With the close call: connection is closed right after client sends a second request, so client whines. * Without the close call: client waits around for a response, then gives up and establishes another connection.

To confirm, add another receiveHTTP call client_interact and see if it returns another Request.

​Thanks!​ I'll try your test and see if anything changes.

Though I thought the code accounted for this: A connection close header is added to each response. Which I thought would cause the client to close the connection.​​ I could be totally wrong tho. There could be some additional aspect of HTTP connections I do not understand.

Properly supporting persistent connections should be pretty easy. There is another Network.HTTP server package that does this: http://hackage.haskell.org/package/http-server-1/docs/Network-HTTP-Server.ht...

So I'll give that a shot and report back.

Thanks! Corey

This is on Mac OS 10.8 and 10.7. I'll be testing NixOS (Linux) this weekend. Was having troubles getting jmeter running... Cheers, Corey -Corey O'Connor coreyoconnor@gmail.com http://corebotllc.com/ On Wed, Nov 6, 2013 at 4:37 PM, Joey Adams wrote:

On Tue, Nov 5, 2013 at 9:10 PM, Corey O'Connor wrote:

...

I've updated the code to handle persistent connections. As best I can figure anyways.. Unfortunately this does not resolve the issue. The server still reports the connection being closed before the full response is read.

What OS are you testing on?