When is waitForProcess not necessary?
If one is calling runInteractiveCommand for a "sure-thing" returning a small amount of output (say, "ls" for a modest directory"), is it necessary to call waitForProcess? My waitForProcess calls came under scrutiny when I tried to GHC profile a threaded process, which isn't possible. It turns out that my programs run faster if they don't call waitForProcess, and it sure seems empirically that they never fail to run correctly. It would seem to me that in a lazy language, later code wouldn't presume it was looking at an entire string until the actual EOF came through, completing the string. So at the end of a program, with nothing else to do but wait, why not wait "implicitly" rather than via an explicit call to waitForProcess? What am I missing? (I've searched this forum; an interesting example is runCommand in http://happs.org/HAppS/src/HAppS/Util/Common.hs but I'm asking why this isn't overkill in my scenario. I'm actually calling Markdown.pl on tiny files (source code of lengths a human would read), and it is certainly sluggish enough to be a fair test.)
On Aug 2, 2007, at 23:52 , Dave Bayer wrote:
If one is calling runInteractiveCommand for a "sure-thing" returning a small amount of output (say, "ls" for a modest directory"), is it necessary to call waitForProcess?
Most operating systems only let you have a small number of outstanding child processes; once you hit it, you'll need to wait() (aka waitForProcess) to reap the dead children before you can launch any more. If you're only doing it once, you can ignore waitForProcess and let the OS's process exit stuff reap the child for you --- but I suggest documenting that you've done this, in case you later change the program to run more children and they start failing with "Too many processes". -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH
bayer:
If one is calling runInteractiveCommand for a "sure-thing" returning a small amount of output (say, "ls" for a modest directory"), is it necessary to call waitForProcess?
My waitForProcess calls came under scrutiny when I tried to GHC profile a threaded process, which isn't possible. It turns out that my programs run faster if they don't call waitForProcess, and it sure seems empirically that they never fail to run correctly. It would seem to me that in a lazy language, later code wouldn't presume it was looking at an entire string until the actual EOF came through, completing the string.
So at the end of a program, with nothing else to do but wait, why not wait "implicitly" rather than via an explicit call to waitForProcess? What am I missing?
(I've searched this forum; an interesting example is runCommand in
http://happs.org/HAppS/src/HAppS/Util/Common.hs
but I'm asking why this isn't overkill in my scenario. I'm actually calling Markdown.pl on tiny files (source code of lengths a human would read), and it is certainly sluggish enough to be a fair test.)
_______________________________________________
You might be interested in : http://www.cse.unsw.edu.au/~dons/code/newpopen/ In particular, readProcess :: FilePath -- ^ command to run -> [String] -- ^ any arguments -> String -- ^ standard input -> IO (Either ExitCode String) -- ^ either the stdout, or an exitcode Strict String IO for processes, without the zombies. -- Don Stewart (Authorised by the "People for System.IO.Strict" party)
On 03/08/07, Dave Bayer
I'm actually calling Markdown.pl on tiny files (source code of lengths a human would read), and it is certainly sluggish enough to be a fair test.)
I had to do this recently, so you might be interested in my approach: http://193.219.108.225/code/blogpost/BlogPost.hs The idea here is to run arbitrary text (blog posts) through Markdown and Smartypants before sending them out to the wider world. The code should be pretty self-contained and easy to follow. It does use waitForProcess but I've done my best to keep it clean. (It could still do with being refactored though; there are still bits or repeated code.) The part you want is about 80% of the way down and looks like this: educate = \s -> markdown s >>= smartypants markdown = convert "Markdown" smartypants = convert "SmartyPants" convertWith conv str = do (hin, hout, herr, proc) <- runInteractiveProcess conv [] Nothing Nothing forkIO $ hPutStr hin str >> hClose hin str' <- hGetContents hout return (str', proc) convert conv input = do bracket (convertWith conv input) (\(_,pc) -> do ret <- waitForProcess pc return (ret==ExitSuccess)) (return . fst) Cheers, D.
Dougal Stanton wrote:
I had to do this recently, so you might be interested in my approach:
http://193.219.108.225/code/blogpost/BlogPost.hs
The idea here is to run arbitrary text (blog posts) through Markdown and Smartypants before sending them out to the wider world.
Pardon me while I veer off-topic, but you could also use Pandoc to do
this. No forking required.
http://sophos.berkeley.edu/macfarlane/pandoc/
On 03/08/07, Bryan O'Sullivan
Pardon me while I veer off-topic, but you could also use Pandoc to do this. No forking required. http://sophos.berkeley.edu/macfarlane/pandoc/
I'll add that to the list of "things that must be done". That list seems, necessarily, to be longer than any list "things I have done". Last time I looked at PanDoc the docs gave the impression it was not very complete. It looks a lot better now. The idea of LaTeX embedded in Markdown sounds awesome... D.
Bryan O'Sullivan
Pardon me while I veer off-topic, but you could also use Pandoc to do this. No forking required. http://sophos.berkeley.edu/macfarlane/pandoc/
What I'm doing is neither Haskell nor Markdown specific; I allow any HTML markup filter that plays nice with the direct HTML I also write (a restriction I could easily drop), and I cooperate with language-specific library doc generators such as Haddock. For all the fuss one reads about Haskell-not-as-fast-as-C, it's amusing how sluggish Markdown.pl is. Someone should write a small BSD'd Haskell version as example code for programming in Haskell. I may, although I can't see myself writing anything called "SmartyPants". I admire pandoc and I allow its use as an alternative to Markdown.pl, as an external command. I don't want to link it into my code because * It is GPL'd and I'm writing BSD'd code * It is a library that does not come with GHC. * It is twice the length of my code so far. The Hackage/Cabal universe takes the perspective that one is a committed Haskell user, and one wants the same diversity of tools enjoyed, say, in the Perl universe. When one uses Haskell to write a tool whose use is standalone and not Haskell-specific, there's a very good chance that someone will come along and try to build it for a new platform, installing and using GHC for the first time in order to do so. The barrier to entry is easily doubled if one has to also figure out how to obtain libraries that do not come automatically with GHC. Plenty of us have the moxie to install a package like GHC for a single use, because we've heard that "hackers" can do such things easily, but we don't really want to join each treehouse. I've installed versions of, say, Perl, Python, Ruby, even if there was a possibly lame installation already present. Still, their package systems generally left me fuming. I know my audience; we mathematicians can be smart and incredibly stupid at the same time.
On Fri, 2007-08-03 at 14:51 +0000, Dave Bayer wrote:
The Hackage/Cabal universe takes the perspective that one is a committed Haskell user, and one wants the same diversity of tools enjoyed, say, in the Perl universe. When one uses Haskell to write a tool whose use is standalone and not Haskell-specific, there's a very good chance that someone will come along and try to build it for a new platform, installing and using GHC for the first time in order to do so. The barrier to entry is easily doubled if one has to also figure out how to obtain libraries that do not come automatically with GHC. Plenty of us have the moxie to install a package like GHC for a single use, because we've heard that "hackers" can do such things easily, but we don't really want to join each treehouse.
The plan/goal is for ghc and hopefully other implementations to come with cabal-install so that when you try and build your random cabal package it'll be able to download, build and install all the deps (without necessarily requiring root). So in the end you should not have to figure out all the deps yourself, it's just that at the moment we're kind of in an intermediate phase. Duncan
So I stared at the documentation in Control-Concurrent, learned about finally and MVar variables, and crossed the genes from the suggestions here to come up with runCommand :: String -> String -> IO (String,Bool) runCommand cmd input = do (inp,out,err,pid) <- runInteractiveCommand cmd let get h = do mvar <- newEmptyMVar let put xs = seq (length xs) (putMVar mvar xs) forkIO $ finally (hGetContents h >>= put) (put []) takeMVar mvar if null input then return () else hPutStr inp input output <- get out errmsg <- get err exit <- waitForProcess pid case exit of ExitSuccess -> return (output,True) ExitFailure _ -> do hPutStrLn stderr errmsg return (errmsg,False) which seems to work well; I haven't beat on it. I like the return type for my needs, e.g. I can write (out,ok) <- runCommand mark doc if ok then write out src else hPutStr stderr out So why don't the MVar examples in this thread bracket somehow, e.g. with finally as Control-Concurrent suggests: Note that we use finally from the Control.Exception module to make sure that the MVar is written to even if the thread dies or is killed for some reason. It seems to me that this could happen, with waitForProcess doing fine, yet the MVar never getting written. (I haven't written a test example to exercise this.)
participants (6)
-
Brandon S. Allbery KF8NH -
Bryan O'Sullivan -
Dave Bayer -
dons＠cse.unsw.edu.au -
Dougal Stanton -
Duncan Coutts