...

simple exe bytes args = do (i, o, e, p) <- runInteractiveProcess exe args Nothing Nothing hPut i bytes s <- hGetContents o hClose i return s

Yep, that's your problem. forkIO the hPut.

Maybe I didn't do enough here -- just wrapping in `forkIO` does not seem to actually help. -- _jsn import Data.ByteString.Lazy import System.Process import System.Environment import System.IO (hClose) import Control.Concurrent import Prelude hiding (writeFile, readFile) main = do exe:file:_ <- getArgs bytes <- readFile file foo <- simple exe bytes [] writeFile (file ++ ".foo") foo -- Manufactures a simple stream handler from a command line utility. simple :: String -> ByteString -> [String] -> IO ByteString simple exe bytes args = do (i, o, e, p) <- runInteractiveProcess exe args Nothing Nothing forkIO $ hPut i bytes s <- hGetContents o return s

On 2008 Nov 10, at 19:04, Jason Dusek wrote:

...

...

simple exe bytes args = do (i, o, e, p) <- runInteractiveProcess exe args Nothing Nothing hPut i bytes s <- hGetContents o hClose i return s

Yep, that's your problem. forkIO the hPut.

Maybe I didn't do enough here -- just wrapping in `forkIO` does not seem to actually help.

*sigh* I hate the ghc runtime... it works in ghci, or compiled with - threaded. Otherwise you still get the deadlock because it only switches threads under limited circumstances (garbage collections?) which isn't nearly often enough. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

Brandon S. Allbery KF8NH wrote:

On 2008 Nov 10, at 19:04, Jason Dusek wrote:

...

...

...

simple exe bytes args = do (i, o, e, p) <- runInteractiveProcess exe args Nothing Nothing hPut i bytes s <- hGetContents o hClose i return s

Yep, that's your problem. forkIO the hPut.

Maybe I didn't do enough here -- just wrapping in `forkIO` does not seem to actually help.

*sigh* I hate the ghc runtime... it works in ghci, or compiled with -threaded.

Would you hate it less if -threaded were the default?

Otherwise you still get the deadlock because it only switches threads under limited circumstances (garbage collections?)

No, the issue is that without real OS threads, a foreign call can't be pre-empted (pretty obvious when you think about it). waitForProcess ends up making a blocking foreign call - non-obvious, but at least it's documented. Cheers, Simon

On Thu, 2008-11-27 at 11:38 -0500, Brandon S. Allbery KF8NH wrote:

On 2008 Nov 27, at 8:51, Simon Marlow wrote:

...

No, the issue is that without real OS threads, a foreign call can't be pre-empted (pretty obvious when you think about it). waitForProcess ends up making a blocking foreign call - non-obvious, but at least it's documented.

The way this is usually handled in the non-threaded case is to either use SIGCHLD or non-blocking waitpid() so that "green" threads can continue running. I'm a little surprised this wasn't done.

Yes, we've discussed this in detail a few months back. We even have a partial implementation. However it stalled on needing a better signals API which we have not managed to get through the standardisation process. Unfortunately there is no non-blocking (non-polling) waitpid() and the global (process-scope) nature of signals is a pain. Duncan

Donn Cave wrote:

Quoth Duncan Coutts : | On Thu, 2008-11-27 at 11:38 -0500, Brandon S. Allbery KF8NH wrote: | |> The way this is usually handled in the non-threaded case is to either |> use SIGCHLD or non-blocking waitpid() so that "green" threads can |> continue running. I'm a little surprised this wasn't done. | | Yes, we've discussed this in detail a few months back. We even have a | partial implementation. However it stalled on needing a better signals | API which we have not managed to get through the standardisation | process. | | Unfortunately there is no non-blocking (non-polling) waitpid() and the | global (process-scope) nature of signals is a pain.

SIGCHLD can be a pain in its own unusual way. Once you have a SIGCHLD handler, process exits will interrupt "long" I/O, so every such read(), recv() or whatever must now check for EINTR and restart. Even though the authors of GHC go to great lengths to convert all I/O to non-blocking anyway, this will still apply to external library functions that are beyond GHC's reach. So it's a strategy I would use only if I were kind of desperate.

We already have this issue since GHC's runtime uses a SIGVTALRM timer signal for context switching and profiling. Indeed, it did cause trouble with the editline library which doesn't test for EINTR in one or two places, and we had to work around it by temporarily disabling the timer. Still, it's standard practice to test for EINTR and all library code should do it. Cheers, Simon

I am using 6.8.3 -- it is almost exciting to realize how close I came to 'impossible' instead of 'annoying'. Living on the edge with UNIX IO! -- _jsn