RE: Win32 process spawning, POpen and Hugs, revisited
On a related note, I recently implemented a System.Process library on Unix. Source code attached. The Windows implementation should be relatively straightforward. Porting it to Hugs will require System.Posix.forkProcess, but I don't see any great difficulties there. One warning: if you want to use it in a concurrent environment with GHC, you need to use GHC's threaded RTS (i.e. use the -threaded option in the forthcoming GHC 6.2.1), otherwise waitForProcess will block all threads. POpen-type abstractions can easily be layered on top of the functionality here - indeed the library should probably contain some higher-level versions which correspond to common usage. I think this is a good platform-independent abstraction for process management. What do others think? Cheers, Simon
On Wed, Mar 17, 2004 at 12:01:11PM -0000, Simon Marlow wrote:
On a related note, I recently implemented a System.Process library on Unix. Source code attached. The Windows implementation should be relatively straightforward. Porting it to Hugs will require System.Posix.forkProcess, but I don't see any great difficulties there.
...
I think this is a good platform-independent abstraction for process management. What do others think?
Looks pretty good to me. I'm currently using doing all my forking in C, since that seems the "easiest" way to go about it, particularly since I need to do it on both windows and POSIX systems (which I accomplish via #ifdefs). But this is all very nasty code that I'd like to get rid of (or more likely, nest in another layer of #ifdefs to deal with old versions of ghc...). The only thing I'd like to check on that I can't tell about is whether passing stdin and stdout to runProcess will behave "correctly". Some programs (e.g. gpg) can figure out how to get control of the tty by themselves (to ask for passwords in the case of gpg), but others (some versions of vi) will fail if their stdin and stdout correspond to a tty. It would be nice to be able to run "user-interactive" commands such as text editors in a safe and consistent manner. I'm not clear from your code whether there will be a problem with passing stdin and stdout to runProcess. If the process closes stdin, will it be closed for me? If the process changes the buffering of stdin, will that change it for me? Enquiring minds want to know! :) -- David Roundy http://www.abridgegame.org
I didn't (yet) receive Simon's original message, but I did look in the archive [1], where the source code attachment shows up as a "non-text" .obj file. So my comment may be based on incomplete information. Is "forkProcess" in the sense of a Unix fork? If so, this is something that is difficult to do cleanly on Windows -- normally, it requires the full Cygwin emulating environment, which as far as I'm concerned is a killer for deploying applications. Theoretically, Windows NT kernel does have a facility to "clone" an entire process, complete with address space contents, but, as far as I'm aware this functionality is not exposed by the Win32 API. (This is a shame, because the Unix fork model is extraordinarily elegant, and simplifies a whole raft of process-creation issues, but Windows is what we have to live with.) #g -- [1] http://www.haskell.org//pipermail/libraries/2004-March/001832.html At 07:40 17/03/04 -0500, David Roundy wrote:
On Wed, Mar 17, 2004 at 12:01:11PM -0000, Simon Marlow wrote:
On a related note, I recently implemented a System.Process library on Unix. Source code attached. The Windows implementation should be relatively straightforward. Porting it to Hugs will require System.Posix.forkProcess, but I don't see any great difficulties there.
...
I think this is a good platform-independent abstraction for process management. What do others think?
Looks pretty good to me. I'm currently using doing all my forking in C, since that seems the "easiest" way to go about it, particularly since I need to do it on both windows and POSIX systems (which I accomplish via #ifdefs). But this is all very nasty code that I'd like to get rid of (or more likely, nest in another layer of #ifdefs to deal with old versions of ghc...).
The only thing I'd like to check on that I can't tell about is whether passing stdin and stdout to runProcess will behave "correctly". Some programs (e.g. gpg) can figure out how to get control of the tty by themselves (to ask for passwords in the case of gpg), but others (some versions of vi) will fail if their stdin and stdout correspond to a tty. It would be nice to be able to run "user-interactive" commands such as text editors in a safe and consistent manner. I'm not clear from your code whether there will be a problem with passing stdin and stdout to runProcess. If the process closes stdin, will it be closed for me? If the process changes the buffering of stdin, will that change it for me? Enquiring minds want to know! :) -- David Roundy http://www.abridgegame.org _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
------------ Graham Klyne For email: http://www.ninebynine.org/#Contact
On Wed, Mar 17, 2004 at 02:30:30PM +0000, Graham Klyne wrote:
Is "forkProcess" in the sense of a Unix fork? If so, this is something that is difficult to do cleanly on Windows -- normally, it requires the full Cygwin emulating environment, which as far as I'm concerned is a killer for deploying applications. Theoretically, Windows NT kernel does have a facility to "clone" an entire process, complete with address space contents, but, as far as I'm aware this functionality is not exposed by the Win32 API.
I was mixed up and thinking about threads, which we do deal with via the win32 api. On windows we use system to emulate "fork and exec", which we *do* to via C and the FFI on POSIX systems. I suppose that "fork and exec" is easier to do on windows than a forkProcess itself, since the child doesn't need access to the parent's memory (except enough to redirect the stdio appropriately). -- David Roundy http://civet.berkeley.edu/droundy/
Graham Klyne wrote:
Is "forkProcess" in the sense of a Unix fork?
Yes; but Simon's code only uses that to provide the Unix
implementation of runProcess and runInteractiveProcess. Those
functions are essentially combined fork+exec operations, so it should
be possible to implement them using Windows' CreateProcess().
For anyone who can't read the Process.hs attachment in Simon's message
(it is base64 encoded), the documentation and types of those functions
are:
-- runProcess
{- | Runs a raw command, optionally specifying 'Handle's from which to
take the @stdin@, @stdout@ and @stderr@ channels for the new
process.
These 'Handle's may be created by 'createPipe', in order to set
up communication channels with the remote process (but see also
'runInteractiveProcess' if you want to do this).
-}
runProcess
:: FilePath -- ^ Filename of the executable
-> [String] -- ^ Arguments to pass to the executable
-> Maybe [(String,String)] -- ^ Optional environment (otherwise inherit)
-> Maybe Handle -- ^ Handle to use for @stdin@
-> Maybe Handle -- ^ Handle to use for @stdout@
-> Maybe Handle -- ^ Handle to use for @stderr@
-> IO ProcessHandle
-- runInteractiveProcess
{- | Runs a raw command, and returns 'Handle's that may be used to communicate
with the process via its @stdin@, @stdout@ and @stderr@ respectively.
'runInteractiveProcess' may not be implementable in terms of
'runProcess' and 'createPipe', because of the need to ensure
that the ends of the pipes that the child process does not use
are closed correctly (otherwise the child process will not be
able to receive EOF on stdin when the parent process closes
the stdin 'Handle', for example).
-}
runInteractiveProcess
:: FilePath -- ^ Filename of the executable
-> [String] -- ^ Arguments to pass to the executable
-> Maybe [(String,String)] -- ^ Optional environment (otherwise inherit)
-> IO (Handle,Handle,Handle,ProcessHandle)
--
Glynn Clements
David Roundy wrote:
The only thing I'd like to check on that I can't tell about is whether passing stdin and stdout to runProcess will behave "correctly". Some programs (e.g. gpg) can figure out how to get control of the tty by themselves (to ask for passwords in the case of gpg),
Programs which read passwords generally open the controlling TTY, directly, i.e. open("/dev/tty", O_RDONLY); this is quite distinct from stdin/stdout.
but others (some versions of vi) will fail if their stdin and stdout correspond to a tty.
Correspond, or don't correspond?
It would be nice to be able to run "user-interactive" commands such as text editors in a safe and consistent manner.
Are you referring to the case where you let the child inherit the parent's stdin/stdout, and suspend the parent for the duration of the child (e.g. system("$EDITOR ..."))? Or running the child on a slave pseudo-tty?
I'm not clear from your code whether there will be a problem with passing stdin and stdout to runProcess. If the process closes stdin, will it be closed for me?
I would expect so; once a descriptor is close()d, you can't "unclose" it.
If the process changes the buffering of stdin, will that change it for me?
*Buffering* involves user-space memory, so it can't be preserved across an exec() call. However, GHC's hSetBuffering doesn't just set the buffering; it also messes with the TTY settings. As those are per-device rather than per-process, any changes would inevitably affect the child (unless runProcess explicitly undoes the changes when passed a Handle which corresponds to a TTY). Realistically, the hSetBuffering misfeature should be discarded if Haskell is meant to be useful for anything beyond "simple, stupid programs".
Enquiring minds want to know! :)
In view of the smiley: was this an actual request for information, or
a way of pointing out that you need to define the semantics in order
for the function to be of any real use?
--
Glynn Clements
On Thu, Mar 18, 2004 at 05:10:04PM +0000, Glynn Clements wrote:
David Roundy wrote:
but others (some versions of vi) will fail if their stdin and stdout correspond to a tty.
Correspond, or don't correspond?
Ineed, I meant to say "don't correspond".
It would be nice to be able to run "user-interactive" commands such as text editors in a safe and consistent manner.
Are you referring to the case where you let the child inherit the parent's stdin/stdout, and suspend the parent for the duration of the child (e.g. system("$EDITOR ..."))? Or running the child on a slave pseudo-tty?
I'm thinking of suspending the parent. Certainly having two programs try to read from stdin would be a nightmare.
I'm not clear from your code whether there will be a problem with passing stdin and stdout to runProcess. If the process closes stdin, will it be closed for me?
I would expect so; once a descriptor is close()d, you can't "unclose" it.
Hmmmm. So either we'd have to dup stdin and stdout before passing them, or hope that the program doesn't close them (and the duping *could* be done by runInteractiveProcess, if we wanted it to actively support this usage).
If the process changes the buffering of stdin, will that change it for me?
*Buffering* involves user-space memory, so it can't be preserved across an exec() call. However, GHC's hSetBuffering doesn't just set the buffering; it also messes with the TTY settings. As those are per-device rather than per-process, any changes would inevitably affect the child (unless runProcess explicitly undoes the changes when passed a Handle which corresponds to a TTY).
Fortunately, as long as the *called* program behaves nicely (i.e. doesn't leave the tty in a different state than it started in), this shouldn't be a problem.
Enquiring minds want to know! :)
In view of the smiley: was this an actual request for information, or a way of pointing out that you need to define the semantics in order for the function to be of any real use?
It was an actual request for information. I'd really love to be able to just open an editor to let the user edit a file and have it just work, and currently I have no way of doing that which works with vi. On the other hand, if the answer is "No, don't pass stdin and stdout to runInteractiveProcess", it would still be a very useful call, it just wouldn't solve my vi problem. -- David Roundy http://www.abridgegame.org
David, on a whim I took a look at your website, and came across darcs, which it is claimed is: (a) written in Haskell, and (b) uses libcurl Does this mean you have a Haskell binding for libcurl? If so, it it for just GHC, or is it a fully-generalized FFI with (say) Hugs support as well? Does it work for Windows as well as Unix/Linux? #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact
On Fri, Mar 19, 2004 at 11:36:15AM +0000, Graham Klyne wrote:
David,
on a whim I took a look at your website, and came across darcs, which it is claimed is: (a) written in Haskell, and (b) uses libcurl
Does this mean you have a Haskell binding for libcurl? If so, it it for just GHC, or is it a fully-generalized FFI with (say) Hugs support as well?
Well, I have a very limited binding to libcurl, that's actually a bit ugly. I use ffi, and wrote two functions in C. One initializes curl, and saves a curl handle in a static global variable (so that curl can use a single http session for several file downloads). Another downloads a URL and saves it as a file. This second function involves four separate libcurl calls. For me (and perhaps for most libcurl users) this is enough.
Does it work for Windows as well as Unix/Linux?
Yes. -- David Roundy http://www.abridgegame.org
David Roundy wrote:
It would be nice to be able to run "user-interactive" commands such as text editors in a safe and consistent manner.
Are you referring to the case where you let the child inherit the parent's stdin/stdout, and suspend the parent for the duration of the child (e.g. system("$EDITOR ..."))? Or running the child on a slave pseudo-tty?
I'm thinking of suspending the parent. Certainly having two programs try to read from stdin would be a nightmare.
Right; the way that's normally done in C is with system(), plus saving and restoring the tty state if you've been messing with it (e.g. using curses; although that won't actually matter for a child program which itself uses curses, as it will set up the tty itself). For your purposes, runProcess + waitForProcess is probably the way to go. Except that runProcess appears to be missing the usual signal handling. system() ignores SIGINT and SIGQUIT in the parent, and resets them to their defaults in the child; it also blocks SIGCHLD in the parent. runProcess may need to do something similar; it should probably at least reset SIGINT and SIGQUIT in the child, in case they are ignored in the parent. The parent can set up its own signal handling, but the only place it can control the child's signal handling is between the fork() and the exec(), so if runProcess doesn't do it, it won't get done. FWIW, runInteractiveProcess isn't what you're looking for. That explicitly sets up pipes for stdin/stdout/stderr; i.e. it's meant for running a "slave" process, with the caller providing its input and consuming its output (like popen(), only more so).
I'm not clear from your code whether there will be a problem with passing stdin and stdout to runProcess. If the process closes stdin, will it be closed for me?
I would expect so; once a descriptor is close()d, you can't "unclose" it.
Hmmmm. So either we'd have to dup stdin and stdout before passing them, or hope that the program doesn't close them (and the duping *could* be done by runInteractiveProcess, if we wanted it to actively support this usage).
I think that I misunderstood you here. If the child process closes the
descriptors, that won't affect the parent, as fork() automatically
duplicates all descriptors. I was referring to the case where the
parent had closed some of the descriptors prior to calling runProcess.
--
Glynn Clements
On 2004 March 17 Wednesday 07:01, Simon Marlow wrote:
On a related note, I recently implemented a System.Process library on Unix. Source code attached. The Windows implementation should be relatively straightforward.
I think this is a good platform-independent abstraction for process management. What do others think?
The abstraction could use an option to control whether exiting or killing the parent process also terminates the child. I'm not sure on what Windows foundation you are thinking of implementing this, but it's not easy to handle the command and arguments on the Win32 API. The arguments are handled by the API as a raw string, so the caller must quote the arguments according to awkard (possibly undocumented) rules. I've worked with this both in the Win32 API and using Perl. Perl didn't have it right, so in one case I was unable to invoke a certain command using the full pathname, because it contained spaces. Nevertheless, supporting argument lists and command path names is the way to go.
participants (5)
-
David Roundy -
Glynn Clements -
Graham Klyne -
Scott Turner -
Simon Marlow