On Jun 14, 2008, at 6:45 PM, Felipe Lessa wrote:

(Sorry, Sebastiaan, I hit send accidentally)

On Sat, Jun 14, 2008 at 1:18 PM, Sebastiaan Visser wrote:

...

readHttpMessage :: IO (Headers, Data.ByteString.Lazy.ByteString) readHttpMessage = do myStream <- <accept http connection from client> request <- hGetContents myStream header <- parseHttpHeader request bs <- Data.ByteString.Lazy.hGetContents myStream return (header, body)

Why not

readHttpMessage = do myStream <- <accept http connection from client> data <- Data.ByteString.Lazy.hGetContents myStream (header, rest) <- parseHttpHeader data return (header, rest)

i.e. make parseHttpHeader return the rest of the string it didn't parse?

Doesn't this imply that the parseHttpHeader must work on ByteStrings instead of regular Strings? Maybe this works for HTTP headers, but sometimes ByteStrings are not appropriate. Especially when you are not using the regular `System.IO.hGetContents' but the `System.IO.UTF8.hGetContents'.

In fact, may I ask why parseHttpHeader is not a pure function?

This is not a real-life example. It might as well be a pure function.

HTH,

-- Felipe.

-- Sebastiaan

On Jun 14, 2008, at 7:16 PM, Felipe Lessa wrote:

On Sat, Jun 14, 2008 at 1:50 PM, Sebastiaan Visser wrote:

...

Doesn't this imply that the parseHttpHeader must work on ByteStrings instead of regular Strings?

I made the change because it's easier and faster to go from ByteString to String than the converse.

...

Maybe this works for HTTP headers, but sometimes ByteStrings are not appropriate. Especially when you are not using the regular `System.IO.hGetContents' but the `System.IO.UTF8.hGetContents'.

You may use the package encoding instead, take a look at [1]. It will decode a lazy ByteString into a String. In your HTTP parsing example, you could break the ByteString on "\r\n\r\n" and then decodeLazy only the first part, while returning the second without modifying it.

Sounds interesting. This could indeed solve (a part of) my problem. The thing is, when decoding the header - in my web server architecture - it is not yet clear what should be done be the body. Some `handlers' may want to have it as a String, UTF8 String, ByteString or may not even need it at all.

[1] http://hackage.haskell.org/packages/archive/encoding/0.4.1/doc/ html/Data-Encoding.html#v%3AdecodeLazy

Because HTTP headers are line based and the parser does not need any look ahead - the first "\r\n\r\n" is the header delimiter - it might be possible to use the hGetLine for the headers. After this I am still `free' to decide what to do for the body: System.IO.hGetContents, System.IO.UTF8.hGetContents, Data.ByteString.Lazy.hGetContents, hClose, etc...

-- Felipe.

Thanks, Sebastiaan.

On Jun 14, 2008, at 6:49 PM, Isaac Dupree wrote:

Sebastiaan Visser wrote:

...

Hi, I've got a question about lazy IO in Haskell. The most well known function to do lazy IO is the `hGetContents', which lazily reads all the contents from a handle and returns this as a regular [Char]. The thing with hGetContents is that is puts the Handle in a semi- closed state, no one can use the handle anymore. This behaviour is understandable from the point of safety; it is not yet determined when the result of hGetContents will actually be computed, using the handle in the meantime is undesirable. The point is, I think I really have a situation in which I want to use the handle again `after' a call to hGetContents. I think I can best explain this using a code example. readHttpMessage :: IO (Headers, Data.ByteString.Lazy.ByteString) readHttpMessage = do myStream <- <accept http connection from client> request <- hGetContents myStream header <- parseHttpHeader request bs <- Data.ByteString.Lazy.hGetContents myStream return (header, body)

that's impure because parseHttpHeader doesn't return anything telling you how much of the stream it's looked at. Maybe it looked ahead more than it needed to, thus deleting part of the body. I was going to suggest, if you can't change parseHttpHeader to use ByteStrings,

...

bs <- Data.ByteString.Lazy.hGetContents myStream header <- parseHttpHeader (Data.ByteString.Lazy.unpack bs)

but you still have to get parseHttpHeader (or perhaps if it has similar friends) to tell you how much of the string it consumed! I don't know what parsing functions you have available to work with, so I can't tell you whether it's possible.

It is a regular Parsec parser and I am pretty sure it does not consume anything other than the header itself. Maybe I could rewrite my parser to work on Word8's instead of Char's, I don't think HTTP even allows Unicode characters within HTTP headers. Thanks. I think I'll try this. But I'm still curious about how to lazily parse messages with arbitrary size Unicode headers and plain (possibly) binary bodies.

-Isaac

-- Sebastiaan.

Sebastiaan Visser wrote:

On Jun 16, 2008, at 2:58 PM, Jules Bean wrote:

...

Sebastiaan Visser wrote:

...

Does anyone know a pattern in which I can do this easily?

Don't use hGetContents on a socket. That's asking for trouble.

Can you please explain why?

Because it's a broken abstraction. It's only correct if all you will ever do is read all the data into one String and don't care about it after that. In my experience this is almost never true of sockets : there is always protocol overhead, handshaking, and the "next request". It might be fine for unusually simple socket setups.

What is a more easier method to spool your HTTP post data to a file than:

Bs.hGetContens sock >>= Bs.hPut fd

?

Yes, that's fine.