Hello, I try to write a service which execute an IO using forkProcess This service contain a web server whcih avwait for job published by users. for each job, I create a forkProcess of this job. I need to fork the process in order to change the uid and gid of the process for each of the job dependencing on who request the job. I have a least two questions 1) is this forkProcess a fork of all the current process, or is it just an executin of the IO in another process id ? I ask this because sometime one of my job hang and a process keep running. Since the original process is binded to a port, it is not possible to restart the server, saying that the port is already in use. 2 ) How can I catch the exceptions thrown from the child process in order to process them in the parent process. thanks for your help answering these questions. Frederic
I'm not familiar with forkProcess. As for exception handling, with asynchronous exceptions, it's a pretty in depth topic and I can't say I have any mastery of it. fpcomplete did a talk recently that I suspect could answer most of your questions. https://www.youtube.com/watch?v=T5y8sFmCFnA On Tue, Jul 17, 2018 at 4:48 AM, PICCA Frederic-Emmanuel < frederic-emmanuel.picca@synchrotron-soleil.fr> wrote:
Hello,
I try to write a service which execute an IO using forkProcess This service contain a web server whcih avwait for job published by users. for each job, I create a forkProcess of this job. I need to fork the process in order to change the uid and gid of the process for each of the job dependencing on who request the job.
I have a least two questions
1) is this forkProcess a fork of all the current process, or is it just an executin of the IO in another process id ?
I ask this because sometime one of my job hang and a process keep running. Since the original process is binded to a port, it is not possible to restart the server, saying that the port is already in use.
2 ) How can I catch the exceptions thrown from the child process in order to process them in the parent process.
thanks for your help answering these questions.
Frederic _______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Hello thanks for the link, I already saw theses slides :) but it seems to me that forkProcess is a different beast :))
Hi Frederic, Let me give a high level recommendation: don't use forkProcess. It's a very dangerous function, which can result in confusing behavior and impossible-to-debug problems. There are cases where it can be made to work, but (1) it's complicated, (2) I'm not sure anyone has ever figured out all of those caveats, and (3) it's certainly not documented properly. forkProcess is little more than a call to the fork() system call, creating a brand new child process which will run the IO action provided. The runtimes of the two processes will not be connected to each other at all. It would be impossible to, say, throw an exception from a thread in the parent process to a thread in the child process. I could say a lot more about this, but I think I'll just reiterate my original recommendation: don't use forkProcess :) Instead, for this kind of use case of changing user/group IDs, I'd recommend using a normal external process call via the process package[1]. I'm not sure of your use case exactly, but I see three ways of making this work: * Generate two Haskell executables * Put the code for both the parent and child into a single executable, and use command line arguments or env vars to control which behavior is run * If you don't have any real logic in the Haskell code, and instead are just using some other program: you can call that directly HTH, Michael [1] Or the typed-process package instead, which I typically recommend these days. Note that I'm the maintainer of process, but typed-process has a brand new API I think is easier to use correctly. More info at: https://haskell-lang.org/library/typed-process On Tue, Jul 17, 2018 at 3:36 PM PICCA Frederic-Emmanuel < frederic-emmanuel.picca@synchrotron-soleil.fr> wrote:
Hello,
I try to write a service which execute an IO using forkProcess This service contain a web server whcih avwait for job published by users. for each job, I create a forkProcess of this job. I need to fork the process in order to change the uid and gid of the process for each of the job dependencing on who request the job.
I have a least two questions
1) is this forkProcess a fork of all the current process, or is it just an executin of the IO in another process id ?
I ask this because sometime one of my job hang and a process keep running. Since the original process is binded to a port, it is not possible to restart the server, saying that the port is already in use.
2 ) How can I catch the exceptions thrown from the child process in order to process them in the parent process.
thanks for your help answering these questions.
Frederic _______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Hi Frederic,
Hello Michael Thanks a lot for your informative answer.
Let me give a high level recommendation: don't use forkProcess. It's a very dangerous function, which can result in confusing behavior and impossible-to-debug problems. There are cases where it can be made to work, but (1) it's complicated, (2) I'm not sure anyone has ever figured out all of those caveats, and (3) it's certainly not documented properly.
forkProcess is little more than a call to the fork() system call, creating a brand new child process which will run the IO action provided. The runtimes of the two processes will not be > connected to each other at all. It would be impossible to, say, throw an exception from a thread in the parent process to a thread in the child process.
I could say a lot more about this, but I think I'll just reiterate my original recommendation: don't use forkProcess :)
Instead, for this kind of use case of changing user/group IDs, I'd recommend using a normal external process call via the process package[1]. I'm not sure of your use case exactly, > but I see three ways of making this work:
* Generate two Haskell executables * Put the code for both the parent and child into a single executable, and use command line arguments or env vars to control which behavior is run * If you don't have any real logic in the Haskell code, and instead are just using some other program: you can call that directly
I am quite close to this model, since I have only once executable and the command line parameters allows to execute the different job like this autoprocessing-exe [exec|submit|server] <jobname> parameters I start my service via the server command without parameters. Then from another computer, I can submit jobs to that server. And locally I can execute jobs with the exec command. What you proposed is just to execute the autoprocessing-command line with s/submit/exec in order to execute a child process. I think that I can do this. Now sometimes this process hang and I want to add a timeout, can I just use timeout for this purpose. 2) I do not want to have communication between my server process and the child one, so in that case is it worth changing the code in order to use process instead of forkPRocess. With process I will need to had lot's of code in oder to convert my job type into command line. I have for now the right Parser command line -> jobtype, but not the other way around. This is why I used forkProcess et first. Once again thanks for your valuable comments. Frédéric
On Fri, Jul 20, 2018 at 10:19 AM PICCA Frederic-Emmanuel < frederic-emmanuel.picca@synchrotron-soleil.fr> wrote:
Hi Frederic,
Hello Michael
Thanks a lot for your informative answer.
Let me give a high level recommendation: don't use forkProcess. It's a very dangerous function, which can result in confusing behavior and impossible-to-debug problems. There are cases where it can be made to work, but (1) it's complicated, (2) I'm not sure anyone has ever figured out all of those caveats, and (3) it's certainly not documented properly.
forkProcess is little more than a call to the fork() system call, creating a brand new child process which will run the IO action provided. The runtimes of the two processes will not be > connected to each other at all. It would be impossible to, say, throw an exception from a thread in the parent process to a thread in the child process.
I could say a lot more about this, but I think I'll just reiterate my original recommendation: don't use forkProcess :)
Instead, for this kind of use case of changing user/group IDs, I'd recommend using a normal external process call via the process package[1]. I'm not sure of your use case exactly, > but I see three ways of making this work:
* Generate two Haskell executables * Put the code for both the parent and child into a single executable, and use command line arguments or env vars to control which behavior is run * If you don't have any real logic in the Haskell code, and instead are just using some other program: you can call that directly
I am quite close to this model, since I have only once executable and the command line parameters allows to execute the different job like this
autoprocessing-exe [exec|submit|server] <jobname> parameters
I start my service via the server command without parameters. Then from another computer, I can submit jobs to that server. And locally I can execute jobs with the exec command.
What you proposed is just to execute the autoprocessing-command line with s/submit/exec in order to execute a child process.
I think that I can do this.
Now sometimes this process hang and I want to add a timeout, can I just use timeout for this purpose.
For the most part, yes. You may need to reach deeper and use SIGKILL occasionally, depending on how stubborn the child process is. The `timeout` will only kill off the parent thread's call to block on the child's exit code.
2) I do not want to have communication between my server process and the child one, so in that case is it worth changing the code in order to use process instead of forkPRocess.
With process I will need to had lot's of code in oder to convert my job type into command line.
I have for now the right Parser command line -> jobtype, but not the other way around. This is why I used forkProcess et first.
Once again thanks for your valuable comments.
Yes, you should avoid forkProcess in this case, it will have unpredictable and confusing results.
Frédéric _______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Hello I have done the first solution and call the program changing the command line in oder to execute each child task. It works well and I can change the uid gid of these process.
For the most part, yes. You may need to reach deeper and use SIGKILL occasionally, depending on how stubborn the child process is. The `timeout` will only kill off the parent thread's call to block on the child's exit code.
Since all these tasks are executing also child process via other system cals. So I have a hyerarchy of process. Do you know how to manage that sort of things if something goes wrong and I need to kill the first child and all its children's ? Is it possible for CreateProcess to track all these process and kill them all on request ?
Yes, you should avoid forkProcess in this case, it will have unpredictable and confusing results.
I take you advices seriously and now I start to build something (I hope) more robust. Thansk Frédéric
On Tue, Jul 24, 2018 at 10:05 AM PICCA Frederic-Emmanuel < frederic-emmanuel.picca@synchrotron-soleil.fr> wrote:
Hello
I have done the first solution and call the program changing the command line in oder to execute each child task. It works well and I can change the uid gid of these process.
For the most part, yes. You may need to reach deeper and use SIGKILL occasionally, depending on how stubborn the child process is. The `timeout` will only kill off the parent thread's call to block on the child's exit code.
Since all these tasks are executing also child process via other system cals. So I have a hyerarchy of process.
Do you know how to manage that sort of things if something goes wrong and I need to kill the first child and all its children's ? Is it possible for CreateProcess to track all these process and kill them all on request ?
Funnily enough, there's another discussion going on about this right now: https://mail.haskell.org/pipermail/haskell-cafe/2018-July/129646.html The basic answer is: * You can create a process group and then kill the whole process group * But a misbehaving program would still be able to escape it by creating its own process group, only something like cgroups can be fully reliable here
Yes, you should avoid forkProcess in this case, it will have unpredictable and confusing results.
I take you advices seriously and now I start to build something (I hope) more robust.
Thansk
Frédéric _______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Funnily enough, there's another discussion going on about this right now:
https://mail.haskell.org/pipermail/haskell-cafe/2018-July/129646.html
:))
The basic answer is:
* You can create a process group and then kill the whole process group * But a misbehaving program would still be able to escape it by creating its own process group, only something like cgroups can be fully reliable here
My server will be managed via systemd, so I hope that it will be able to managed this when restarting. Is there an haskell API in order to work with cgoups ? Cheers Fred
On Wed, Jul 25, 2018 at 11:27 AM PICCA Frederic-Emmanuel < frederic-emmanuel.picca@synchrotron-soleil.fr> wrote:
Funnily enough, there's another discussion going on about this right now:
https://mail.haskell.org/pipermail/haskell-cafe/2018-July/129646.html
:))
The basic answer is:
* You can create a process group and then kill the whole process group * But a misbehaving program would still be able to escape it by creating its own process group, only something like cgroups can be fully reliable here
My server will be managed via systemd, so I hope that it will be able to managed this when restarting. Is there an haskell API in order to work with cgoups ?
None that I'm aware of.
Cheers
Fred _______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
participants (3)
-
David McBride -
Michael Snoyman -
PICCA Frederic-Emmanuel