Proposal: new signal-handling API

I've just submitted a proposal for a new signal-handling API, the ticket is here: http://hackage.haskell.org/trac/ghc/ticket/2451 Deadline: 1 Aug (2 weeks). Cheers, Simon

On Fri, 2008-07-18 at 13:55 +0100, Simon Marlow wrote:
I've just submitted a proposal for a new signal-handling API, the ticket is here:
http://hackage.haskell.org/trac/ghc/ticket/2451
Deadline: 1 Aug (2 weeks).
I'm happy with it. It's relatively simple now (earlier iterations had more states). The main question is if it's too simple. Can people who know about and use signals take a look and see if it covers everything? I'm cc'ing a couple more people who know about and use signals. In particular, one simplification we made from an earlier design is that it's not possible to go back to the default OS signal action without explicitly removing all handlers. Previously there was a set of handlers and an independent signal state, ignore, default or handled. Now the state follows the set of handlers so when the set is empty then we get the default signal action. There is no ignore state as such but one can override the default handler by adding a handler that does nothing. So each handler is relatively independent. You can't interfere with another handler unless you have access to its HandlerId. That would seem to be a good thing but I might be missing some use case. Duncan

Duncan Coutts wrote:
On Fri, 2008-07-18 at 13:55 +0100, Simon Marlow wrote:
I've just submitted a proposal for a new signal-handling API, the ticket is here:
http://hackage.haskell.org/trac/ghc/ticket/2451
Deadline: 1 Aug (2 weeks).
I'm happy with it. It's relatively simple now (earlier iterations had more states).
The main question is if it's too simple. Can people who know about and use signals take a look and see if it covers everything? I'm cc'ing a couple more people who know about and use signals.
Thanks for that. I'm on a train to OSCon right now, so my apologies for typos or late replies (my net connection is not terribly stable so I can't see the whole thing.) Overall I think the goal of being able to add signal handlers without disturbing what's there is a good one, and being able to add multiple handlers like that is a good plan too. I do have a few things I'd like to make sure are addressed, though. Maybe they are (again, a bit hard to read from my train)... 1) Instead of deprecating the low-level API, I would prefer to have it still available -- ideally with the new API implemented in terms of it. I can see cases where I really want to make the handler be what I say, not anything else. For instance, say some library was catching SIGABRT for its own purposes, but in a certain section of code, I really want SIGABRT to be handled in a certain way. Using the old API, I could save off the old handler, install a new handler that does what I want, and restore the old one later (assuming I didn't dump core). This is important because, for many signals, the "normal" way to handle it might be to do something and then crash. For instance, if you get sigabrt, maybe you print out some state, then crash. So if something installed a signal handler that crashes, you may be stuck if you don't want to crash. It could also be important if you really don't want other signals handlers doing anything. Say you've got a library that installs a handler for SIGINT that flushes out some buffer and prepares things for the progam to terminate nicely. But in a certain part of your code, you don't want to handle SIGINT at all (or you absolutely want it to be handled only by saying putStrLn "Don't press Ctrl-C now, you dolt!"). Since adding a signal handler could be opaque, this could be quite difficult. With the proposed system, you never know if some module has installed a signal handler, which could be an issue in itself. 2) It would be nice to at least have the option of atomic handlers in Haskell. I realize this is not a regression over 6.8.2. Failing that, it would be nice to have it documented whether forkIO or forkOS is used. 3) I hope you will not be deprecating System.Posix.Signals.Exts 4) Some signals such as SIGPWR are missing in both 6.8.2 and the new system Thanks for your work on this. These are probably not popular cases. On the other hand, I have tended to run into limitations in System.Posix in the past in areas that weren't anticipated to be popular cases ;-) -- John
In particular, one simplification we made from an earlier design is that it's not possible to go back to the default OS signal action without explicitly removing all handlers. Previously there was a set of handlers and an independent signal state, ignore, default or handled. Now the state follows the set of handlers so when the set is empty then we get the default signal action. There is no ignore state as such but one can override the default handler by adding a handler that does nothing.
So each handler is relatively independent. You can't interfere with another handler unless you have access to its HandlerId. That would seem to be a good thing but I might be missing some use case.
Duncan

On Fri, Jul 18, 2008 at 5:45 PM, John Goerzen
Duncan Coutts wrote:
The main question is if it's too simple. Can people who know about and use signals take a look and see if it covers everything? I'm cc'ing a couple more people who know about and use signals.
1) Instead of deprecating the low-level API, I would prefer to have it still available -- ideally with the new API implemented in terms of it. I can see cases where I really want to make the handler be what I say, not anything else. For instance, say some library was catching SIGABRT for its own purposes, but in a certain section of code, I really want SIGABRT to be handled in a certain way. Using the old API, I could save off the old handler, install a new handler that does what I want, and restore the old one later (assuming I didn't dump core).
[snip]
Thanks for your work on this. These are probably not popular cases. On the other hand, I have tended to run into limitations in System.Posix in the past in areas that weren't anticipated to be popular cases ;-)
I think John raises a very important point. This is something I've been thinking about since the discussion on the System.Process overhaul. It's my firm belief that when wrapping a low-level OS library it's important to provide as much of its functionality as possible even if that means that the resulting API is quite low-level. You can of course add some convenience functions on top of the low-level interface for the more common cases. In general, it's also important to export (maybe in a .Internal module) the low-level data types (e.g. file descriptors in the case of a file API) so that someone else can write an API using the FFI that integrates with yours. If you provide e.g. a completely abstract Socket type that wraps the `int' the OS uses to represent the socket then it's not possible to extend the API from another package. If you only add the high level functionality that cover say 95% of the use cases then larger programs will inevitably run into cases when the current API simply don't support the functionality that you need and they will have to use resort to using the FFI. This is quite unsatisfactory as wrapping an OS function so it works on different platforms and with different Haskell compilers is quite tricky. To summarize: Please always provide a low-level wrapping that is as close to the API provided by the OS as possible and then build convenience functions on top of that. Cheers, Johan

On 2008 Jul 18, at 11:45, John Goerzen wrote:
It could also be important if you really don't want other signals handlers doing anything. Say you've got a library that installs a handler for SIGINT that flushes out some buffer and prepares things for the progam to terminate nicely. But in a certain part of your code, you don't want to handle SIGINT at all (or you absolutely want it to be handled only by saying putStrLn "Don't press Ctrl-C now, you dolt!"). Since adding a signal handler could be opaque, this could be quite difficult. With the proposed system, you never know if some module has installed a signal handler, which could be an issue in itself.
This is where you use sigprocmask() in POSIX to simply disallow the signal during your critical section; safer than trusting the user not to interrupt. Your earlier example could be handled by pushing/ popping handlers (and I imagine a typesafe way of doing both of these, e.g. withSignalHandlerDo signal handler block / withSignalMaskDo mask block). -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

John Goerzen wrote:
Thanks for that. I'm on a train to OSCon right now, so my apologies for typos or late replies (my net connection is not terribly stable so I can't see the whole thing.)
Overall I think the goal of being able to add signal handlers without disturbing what's there is a good one, and being able to add multiple handlers like that is a good plan too.
I do have a few things I'd like to make sure are addressed, though. Maybe they are (again, a bit hard to read from my train)...
1) Instead of deprecating the low-level API, I would prefer to have it still available -- ideally with the new API implemented in terms of it. I can see cases where I really want to make the handler be what I say, not anything else. For instance, say some library was catching SIGABRT for its own purposes, but in a certain section of code, I really want SIGABRT to be handled in a certain way. Using the old API, I could save off the old handler, install a new handler that does what I want, and restore the old one later (assuming I didn't dump core).
This is important because, for many signals, the "normal" way to handle it might be to do something and then crash. For instance, if you get sigabrt, maybe you print out some state, then crash. So if something installed a signal handler that crashes, you may be stuck if you don't want to crash.
It could also be important if you really don't want other signals handlers doing anything. Say you've got a library that installs a handler for SIGINT that flushes out some buffer and prepares things for the progam to terminate nicely. But in a certain part of your code, you don't want to handle SIGINT at all (or you absolutely want it to be handled only by saying putStrLn "Don't press Ctrl-C now, you dolt!"). Since adding a signal handler could be opaque, this could be quite difficult. With the proposed system, you never know if some module has installed a signal handler, which could be an issue in itself.
So one reason I wanted to deprecate the old API is that with signals run in separate threads it didn't really make sene. For example, asking for a signal to be ignored isn't useful on its own: there might be signal handlers for that signal already running, or just started and about to run. Blocking signals suffers from the same problem. (I know I left the blocking primitives around in the new API, I should really deprecate them though). Synchronising with signal handlers should be done using ordinary thread synchronisation. Have one big signal lock if you like, or one per signal, or one per handler - it's up to you. It's true that you can't synchronise with other people's signal handlers this way. But I think that's a feature, or at least not a bug. Even using SIG_IGN or sigprocmask() you couldn't stop a handler that had already been started anyway. To do what you describe with SIGINT, you would: - say to the default ^C handler "don't deliver any ^C exceptions until I tell you otherwise" (GHC 6.9+ has a default ^C handler, and there will be a way to communicate/synchronise with it or disable it). - install another handler temporarily to do putStrLn "don't press ^C...". Random libraries would not be adding ^C handlers to do cleanup. Libraries that need cleanup should be providing a function withMyLib :: IO a -> IO a that does a "finally"-style action to perform the cleanup.
2) It would be nice to at least have the option of atomic handlers in Haskell. I realize this is not a regression over 6.8.2. Failing that, it would be nice to have it documented whether forkIO or forkOS is used.
The idea of atomic signal handlers in the context of GHC scares me :-) Either the handler would have to avoid touching any kind of shared resource (e.g. a Handle) to avoid deadlock, or we'd have to block signals whenever another thread was holding the resource (not really practical). The list of things you could do in a signal handler would be pretty small - most I/O would be out. Also signal handlers are not atomic even in POSIX when there are multiple threads, right? They're only atomic with respect to the current OS thread, which is not a concept we have in GHC. Yes I'm talking about GHC specifics here, but we want to design an API that makes sense in multiple settings, and GHC is a setting in which atomic handlers wouldn't work, I think.
3) I hope you will not be deprecating System.Posix.Signals.Exts
4) Some signals such as SIGPWR are missing in both 6.8.2 and the new system
Someone should add those to System.Posix.Signals.Exts, yes.

I wrote:
Synchronising with signal handlers should be done using ordinary thread synchronisation. Have one big signal lock if you like, or one per signal, or one per handler - it's up to you.
It's true that you can't synchronise with other people's signal handlers this way. But I think that's a feature, or at least not a bug. Even using SIG_IGN or sigprocmask() you couldn't stop a handler that had already been started anyway.
It just occurred to me that it would be relatively easy to add back support for blocking signals in a way that makes sense. We could add one MVar per signal, and have every signal handler do something like withMVar sigINT_lock $ \_ -> do ... (implemented under the hood in the API). That would let us provide a way to temporarily disable handlers for particular signals. It would have the effect of serialising the handlers for each signal too, I'm not sure if that's a good thing. We could provide a back door too, perhaps. Cheers, Simon

On Fri, Jul 18, 2008 at 5:55 AM, Simon Marlow
I've just submitted a proposal for a new signal-handling API, the ticket is here:
Thanks, Simon. I generally like the proposal, particularly the modular aspect of signal handling. I think that if Johan's request for a lower-level API were followed, that would also satisfy John's desire for more explicit control. I notice that there is no binding for sigwaitinfo/sigtimedwait. Is this deliberate?

On Sat, 2008-07-19 at 08:29 -0700, Bryan O'Sullivan wrote:
On Fri, Jul 18, 2008 at 5:55 AM, Simon Marlow
wrote: I've just submitted a proposal for a new signal-handling API, the ticket is here:
Thanks, Simon. I generally like the proposal, particularly the modular aspect of signal handling.
I think that if Johan's request for a lower-level API were followed, that would also satisfy John's desire for more explicit control.
I do understand the desire for that. The downside of course is that a lower level api has to be used very carefully to avoid breaking the promises provided by the higher level api. So we should discuss what lower level api is required exactly and document how people can use it without breaking the higher level api. I'd also prefer to have some use cases for the lower level api (rather than just having it there for its own sake) since I suspect that most of them can be solved in better and more portable ways than using signals directly (eg using exceptions). In particular the case of "I need to do this cleanup before the program terminates" is much more general than Posix and signals. For example a library installing a handler for SIGINT is the wrong thing to do (and not just because it's not very modular) but because the very decision about whether SIGINT is going to kill the process or not is not for the library to decide. What it needs is to be thrown an exception when the process does decide that it's going to terminate.
I notice that there is no binding for sigwaitinfo/sigtimedwait. Is this deliberate?
There is and it's not changed by Simon's proposal: awaitSignal :: Maybe SignalSet -> IO () Source awaitSignal iset suspends execution until an interrupt is received. If iset is Just s, awaitSignal calls sigsuspend, installing s as the new signal mask before suspending execution; otherwise, it calls pause. awaitSignal returns on receipt of a signal. If you have installed any signal handlers with handleSignal, for example, it may be wise to call yield directly after awaitSignal to ensure that the signal handler runs as promptly as possible. Incidentally this can be re-implemented using the new api in such a way that it does not block the whole OS thread as sigwaitinfo/sigtimedwait does. Actually the way it works now doesn't interact very well at all because it swallows the signal so the other handlers do not run. Duncan

On 2008 Jul 19, at 13:07, Duncan Coutts wrote:
In particular the case of "I need to do this cleanup before the program terminates" is much more general than Posix and signals. For example a library installing a handler for SIGINT is the wrong thing to do (and not just because it's not very modular) but because the very decision about whether SIGINT is going to kill the process or not is not for the library to decide. What it needs is to be thrown an exception when the process does decide that it's going to terminate.
...and here you see why I made my proposal about mapping signals to exceptions. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

On Sat, 2008-07-19 at 15:18 -0400, Brandon S. Allbery KF8NH wrote:
On 2008 Jul 19, at 13:07, Duncan Coutts wrote:
In particular the case of "I need to do this cleanup before the program terminates" is much more general than Posix and signals. For example a library installing a handler for SIGINT is the wrong thing to do (and not just because it's not very modular) but because the very decision about whether SIGINT is going to kill the process or not is not for the library to decide. What it needs is to be thrown an exception when the process does decide that it's going to terminate.
...and here you see why I made my proposal about mapping signals to exceptions.
Perhaps I misunderstood. I assumed you mean just a way of letting programs map signals to exceptions, which is something they can do already. What we'd need instead is to have all fatal signals be mapped to exceptions and then let programs selectively block or remap them (eg if command line progs want to use ^C for some alternative purpose). That way library code can just use ordinary exception handlers to clean up and would not have to think about signals at all. Duncan

On 2008 Jul 19, at 21:30, Duncan Coutts wrote:
On Sat, 2008-07-19 at 15:18 -0400, Brandon S. Allbery KF8NH wrote:
On 2008 Jul 19, at 13:07, Duncan Coutts wrote:
In particular the case of "I need to do this cleanup before the program terminates" is much more general than Posix and signals. For example a library installing a handler for SIGINT is the wrong thing to do (and not just because it's not very modular) but because the very decision about whether SIGINT is going to kill the process or not is not for the library to decide. What it needs is to be thrown an exception when the process does decide that it's going to terminate.
...and here you see why I made my proposal about mapping signals to exceptions.
Perhaps I misunderstood. I assumed you mean just a way of letting programs map signals to exceptions, which is something they can do
I may have spoken a bit too generically, but this is the essence of it: sometimes when you receive a signal it wants to be handled as an exception, as with this. Sometimes you want to handle interrupt/quit the same way; whether that is a good programming style or not may depend on what school of expression you come from. Handling the common case of debug mechanisms hanging of of SIGUSR[12] is almost certainly a poor match with exceptions. An example of when you might want to treat SIGINT as an example is a character-mode program with a user interface where you want to interrupt individual operations and return to the idle loop. (Consider yi as an example.) If SIGINT is delivered to user code as an InterruptException then you simply wrap the command dispatcher in a catch. Admittedly this is not necessarily easy in either case, and the code has to be ready to use critical sections or its own catch/rethrow to clean up resources.
That way library code can just use ordinary exception handlers to clean up and would not have to think about signals at all.
That's the idea, yes. At this level a signal *is* a kind of exception, especially with respect to operations cleaning up after themselves. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

On 2008-07-19 18:07 +0100 (Sat), Duncan Coutts wrote:
[Stuff about doing signals as exceptions.]
I think that the issue with this is, to which threads do you deliver the
exceptions?
cjs
--
Curt Sampson

Hello Curt, Monday, July 21, 2008, 7:58:07 AM, you wrote:
[Stuff about doing signals as exceptions.]
I think that the issue with this is, to which threads do you deliver the exceptions?
main one. it should handle other threads as needed -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On 2008 Jul 21, at 2:20, Bulat Ziganshin wrote:
Monday, July 21, 2008, 7:58:07 AM, you wrote:
[Stuff about doing signals as exceptions.]
I think that the issue with this is, to which threads do you deliver the exceptions?
main one. it should handle other threads as needed
I'm not so sure: if the point is to release resources on signal, you want to deliver the exception to (potentially) all threads, and the main thread may not have any idea of the needs of the library being used by another thread. I think I would handle these exceptions by sending them only to threads which have registered an interest (withSignalException sigMask $ do ... :: Either SignalException a). If a SignalException somehow ends up being rethrown back to the runtime, this is likely to be an error which should cause program termination --- but I can also see it being quietly ignored, This might be a +RTS option. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

Hello Brandon, Monday, July 21, 2008, 10:37:20 AM, you wrote:
I think that the issue with this is, to which threads do you deliver the exceptions?
main one. it should handle other threads as needed
I'm not so sure: if the point is to release resources on signal, you want to deliver the exception to (potentially) all threads, and the main thread may not have any idea of the needs of the library being used by another thread.
I think I would handle these exceptions by sending them only to threads which have registered an interest (withSignalException sigMask $ do ... :: Either SignalException a). If a SignalException somehow ends up being rethrown back to the runtime, this is likely to be an error which should cause program termination --- but I can also see it being quietly ignored, This might be a +RTS option.
well, is it what i'd in mind - not sure which variant is better: when main thread creates new thread, it saves its handle. if this thread creates new one - it does the same again. so when we send exception to main thread, it can do the same to all its subthreads, and so that recursive i've tried to use this model in my program (but not used yet), but not sure that it will really work and will be helpful generally speaking, we have an infrastructure of exception handling. when things go wrong, exception handlers kill files, burn out documents, so on. when signal arrives that means "things goes wrong" it will be wise to have opportunity to do the same. MAY BE, we should have just ability to send some exception to ALL threads in the program? how about adding to GHC API ability to get full list of threads? btw, i have even worse situation - in my program i call Haskell->C->Haskell and when operation should be terminated it's not easy to clean up smoothly -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Mon, 2008-07-21 at 02:37 -0400, Brandon S. Allbery KF8NH wrote:
On 2008 Jul 21, at 2:20, Bulat Ziganshin wrote:
Monday, July 21, 2008, 7:58:07 AM, you wrote:
[Stuff about doing signals as exceptions.]
I think that the issue with this is, to which threads do you deliver the exceptions?
main one. it should handle other threads as needed
I'm not so sure: if the point is to release resources on signal, you want to deliver the exception to (potentially) all threads, and the main thread may not have any idea of the needs of the library being used by another thread.
I think I would handle these exceptions by sending them only to threads which have registered an interest (withSignalException sigMask $ do ... :: Either SignalException a). If a SignalException somehow ends up being rethrown back to the runtime, this is likely to be an error which should cause program termination --- but I can also see it being quietly ignored, This might be a +RTS option.
And that was my point. That this issue of cleaning up when the program is about to die is a good deal more general than signals or even posix. Currently libs that launch threads have no easy means of getting notified when the main thread is going to exit. This is just as true for exitWith as for posix signals. One can use a withLib function but even that's not easy if not all the libs threads are going to be launched then and there. withLib would have to save some MVar collection of threads and other threads the lib launched would have to be added/removed from the set. Then withLib would send exceptions to all the threads in it's collection. My intuition is that we should look at what erlang does. As I understand it they have something which allows them to tie threads such that when one dies the other dies. Duncan

Hello Duncan, Monday, July 21, 2008, 2:56:24 PM, you wrote:
Currently libs that launch threads have no easy means of getting notified when the main thread is going to exit.
i think that problem is not that they launch threads. they just not have any exception sent to it. exceptions are standard way of cleanup, so exit/signals should send exceptions. to all threads that wants to clean up that, for example, about automatic sending special exceptions EXIT and SIGNAL x, to every haskell thread? i know one problem of this design - sometimes threads should synchronize their cleanup, so subthread cleans first, and superthread next -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Sat, Jul 19, 2008 at 10:07 AM, Duncan Coutts
I do understand the desire for that. The downside of course is that a lower level api has to be used very carefully to avoid breaking the promises provided by the higher level api.
True.
There is and it's not changed by Simon's proposal:
awaitSignal :: Maybe SignalSet -> IO ()
Actually, awaitSignal isn't the same as sigwaitinfo; the latter returns the siginfo_t for the received signal.

On Mon, 2008-07-21 at 14:23 -0700, Bryan O'Sullivan wrote:
On Sat, Jul 19, 2008 at 10:07 AM, Duncan Coutts
wrote: I do understand the desire for that. The downside of course is that a lower level api has to be used very carefully to avoid breaking the promises provided by the higher level api.
True.
There is and it's not changed by Simon's proposal:
awaitSignal :: Maybe SignalSet -> IO ()
Actually, awaitSignal isn't the same as sigwaitinfo; the latter returns the siginfo_t for the received signal.
Ah, another good reason to rewrite awaitSignal in terms of addSignalHandler. That's one of the new things with the internal implementation in ghc; it now captures the whole siginfo_t structure (and pushes it down an internal pipe) rather than just the signal number. Duncan

It seems a bit too simple to me. Mainly because since it is in the 'System.Posix' namespace, I would expect it to mirror the POSIX specification for signals. It seems that higher level handling of signal-esqe IPC should be handled in its own module. John -- John Meacham - ⑆repetae.net⑆john⑈
participants (9)
-
Brandon S. Allbery KF8NH
-
Bryan O'Sullivan
-
Bulat Ziganshin
-
Curt Sampson
-
Duncan Coutts
-
Johan Tibell
-
John Goerzen
-
John Meacham
-
Simon Marlow