
Hi all, I tried to turn off buffering with the command hSetBuffering (from System.IO) but my app still blocks on hGetContents (from Data.ByteString). Does anyone know what's happening? E.

Eric writes:
I tried to turn off buffering with the command hSetBuffering (from System.IO) but my app still blocks on hGetContents (from Data.ByteString). Does anyone know what's happening?
I very much doubt anyone will be able to help you unless you include some code for us all to look at. -- -David House, dmhouse@gmail.com

David House wrote:
Eric writes:
I tried to turn off buffering with the command hSetBuffering (from System.IO) but my app still blocks on hGetContents (from Data.ByteString). Does anyone know what's happening?
I very much doubt anyone will be able to help you unless you include some code for us all to look at.
import Network import System.IO import Data.ByteString as Bits(ByteString, hGetContents) soc <- listenOn $ PortNumber 2007 (hdl, host, port) <- accept soc hSetBuffering hdl No Buffering; bs <- Bits.hGetContents hdl; -- blocks here despite previous line E.

On Sat, Jun 16, 2007 at 04:17:39PM +0100, Eric wrote:
import Network import System.IO import Data.ByteString as Bits(ByteString, hGetContents)
soc <- listenOn $ PortNumber 2007 (hdl, host, port) <- accept soc hSetBuffering hdl No Buffering; bs <- Bits.hGetContents hdl; -- blocks here despite previous line
I think it's because you are using the strict ByteString module version. Its hGetContents just has to the whole input. Try with Data.ByteString.Lazy. Best regards Tomek

Eric wrote:
I tried to turn off buffering with the command hSetBuffering (from System.IO) but my app still blocks on hGetContents (from Data.ByteString). Does anyone know what's happening?
The hGetContents function can't behave the way you want, because it's defined to return the entire rest of the input stream. If you want to stick with strict ByteStrings, use hGetNonBlocking instead, but you'll need to block between reads of the handle yourself, using System.IO.hWaitForInput. Otherwise, use lazy ByteStrings. That version of hGetContents will lazily yield chunks that are as big as can be read without blocking as they arrive (up to a limit of 64KB), and will hWaitForInput for you.

Bryan O'Sullivan wrote:
Eric wrote:
I tried to turn off buffering with the command hSetBuffering (from System.IO) but my app still blocks on hGetContents (from Data.ByteString). Does anyone know what's happening?
The hGetContents function can't behave the way you want, because it's defined to return the entire rest of the input stream.
If you want to stick with strict ByteStrings, use hGetNonBlocking instead, but you'll need to block between reads of the handle yourself, using System.IO.hWaitForInput.
Otherwise, use lazy ByteStrings. That version of hGetContents will lazily yield chunks that are as big as can be read without blocking as they arrive (up to a limit of 64KB), and will hWaitForInput for you.
I've converted to lazy bytestrings. After reading in the bytes from a network connection I want to save them to a file but now the appendFile function blocks: import Data.ByteString.Lazy as LazyBits(ByteString, empty, hGetContents, writeFile, appendFile) ... LazyBits.appendFile filepath bits -- this blocks now! How can I fix this? E.

Eric wrote:
I've converted to lazy bytestrings. After reading in the bytes from a network connection I want to save them to a file but now the appendFile function blocks:
Well, yes. It's presumably waiting for data from the network connection, because it wants to write out the entire ByteString, and whoever you're receiving data from hasn't closed the connection. If you stepped back and stated the more general problem you are trying to solve, we could help you more effectively.

Bryan O'Sullivan wrote:
Eric wrote:
I've converted to lazy bytestrings. After reading in the bytes from a network connection I want to save them to a file but now the appendFile function blocks:
Well, yes. It's presumably waiting for data from the network connection, because it wants to write out the entire ByteString, and whoever you're receiving data from hasn't closed the connection.
If you stepped back and stated the more general problem you are trying to solve, we could help you more effectively.
I'm writing a simple HTTP server and am trying to implement the POST method. E.

Eric wrote:
I'm writing a simple HTTP server and am trying to implement the POST method.
That's a rather general problem statement, indeed :-) For an application like this, I'd suggest that explicit resource management is the way to go, and that you should not be using hGetContents at all, under any guise. For example, any scheme involving reading an entire stream is going to do completely the wrong thing in the face of HTTP keepalive. Also, code that leaves open sockets piling up in drifts, to eventually be shoveled up by the RTS, is going to be trivially easy to DoS.

Bryan O'Sullivan wrote:
Eric wrote:
I'm writing a simple HTTP server and am trying to implement the POST method.
That's a rather general problem statement, indeed :-) For an application like this, I'd suggest that explicit resource management is the way to go, and that you should not be using hGetContents at all, under any guise. For example, any scheme involving reading an entire stream is going to do completely the wrong thing in the face of HTTP keepalive. Also, code that leaves open sockets piling up in drifts, to eventually be shoveled up by the RTS, is going to be trivially easy to DoS. Are there any good resources/tutorials on Haskell network programming?
E.

eeoam:
Bryan O'Sullivan wrote:
Eric wrote:
I'm writing a simple HTTP server and am trying to implement the POST method.
That's a rather general problem statement, indeed :-) For an application like this, I'd suggest that explicit resource management is the way to go, and that you should not be using hGetContents at all, under any guise. For example, any scheme involving reading an entire stream is going to do completely the wrong thing in the face of HTTP keepalive. Also, code that leaves open sockets piling up in drifts, to eventually be shoveled up by the RTS, is going to be trivially easy to DoS. Are there any good resources/tutorials on Haskell network programming?
E.
Some articles have been written here, http://haskell.org/haskellwiki/Blog_articles#Network Intro stuff mostly. See also HAppS, and Simon Marlow's concurrent webserver paper. -- Don
participants (5)
-
Bryan O'Sullivan
-
David House
-
dons@cse.unsw.edu.au
-
Eric
-
Tomasz Zielonka