* Evan Laforge [2013-04-22 12:41:33+0530]

On Sun, Apr 21, 2013 at 9:25 PM, Donn Cave wrote:

...

Quoth Evan Laforge ,

...

sleep = Process.createProcess (Process.proc "sleep" ["5"])

sleep = Process.createProcess ((Process.proc "sleep" ["5"]) {Process.close_fds = True})

- Because the client uses buffered I/O (hGetContents in this case, but hGet-anything would be the same), it doesn't "see" the server response until a) buffer full, or b) end of file (server closes connection.)

- The server does close the connection, but after the "sleep" process has forked off with a copy of the connection fd. If it doesn't close that fd explicitly, it holds it open until process exit (5 seconds.)

Oh I see, because the subprocess inherits the socket connection. That makes sense, though it's tricky. Tricky tricky unix. Why does fork() have to be so complicated?

You probably want to set FD_CLOEXEC flag on your sockets. This will cause execve to close them in the child process. Roman

On Mon, Apr 22, 2013 at 7:39 PM, Donn Cave wrote:

quoth Evan Laforge , ...

...

Oh I see, because the subprocess inherits the socket connection. That makes sense, though it's tricky. Tricky tricky unix. Why does fork() have to be so complicated?

Well, it's very elegant really. It's one of the tools UNIX gives you to decompose a task into discrete modules, programs running in separate processes - but inheriting file descriptors and other common environmental stuff. And in some respects should feel familiar to a Haskell programmer - can't modify parent's environment, etc.

But the way I see it, it's a bunch of implicit state, which is not nicely encapsulated in a single data structure but scattered around invisibly, which is then implicitly copied by a function that claims to take no arguments... doesn't seem very haskelly to me :)

For me, moral in the story should be that buffered I/O is not robust, for a socket device or anything like it (UNIX pipe, whatever.) That isn't about UNIX, it's just inevitable. Maybe your client is really going to need the timely EOF anyway, but the immediate problem was that the server wrote to the socket and the client couldn't see it. Because the client read is buffered. Not robust.

Actually, my original code turns off buffering on the socket. I didn't include it in the example because it didn't have an effect on the result... but try modifying the server to (hdl, _host, _port) <- Network.accept socket IO.hSetBuffering hdl IO.NoBuffering and the client to hdl <- Network.connectTo "localhost" port IO.hSetBuffering hdl IO.NoBuffering and it still waits for the subprocess to complete. Actually, it looks like it's the client's hGetContents, since hGetChar comes back immediately. I guess that's understandable, but even hGetLine blocks. System.IO doesn't seem to have a hRead :: Int -> IO String, I guess you have to go down to the POSIX fd level, or read char-by-char. Though it mysteriously has readIO which doesn't actually do any IO. And I can't turn on CloseOnExec without closing the handle, converting to fd, and creating a new handle... but anyway we were talking about unix, not System.IO infelicities :)

Some programmers are so wedded to the buffered language platform I/O functions that they'll find a way to defeat the buffering or work around it. That can be "nonperformant" where the semantics of the read depend on buffering, like getLine; don't know what would happen with hGetContents. The path that makes sense to me is low level I/O (read/recv) that simply returns what's there, up to the specified limit.

The plan9 approach was that you get buffering only if you explicitly create a buffer and wrap it around the fd. And there was less buffering in general, perhaps motivated by the lack of hacks like isatty(). But I digress...