Safe forking question
In response to question by Cat Dancer <cat@catdancer.ws> I wrote a few tests of sending asynchronous signal to a thread using GHC 6.6 The goal was to run a child thread via forkIO and use handle or finally to respond to the thread's demise. Unfortunately, it seems that there is an irreducible window where this fails. The forkIO returns but any exception handlers such as block/handle/catch/finally are not in place yet. So this fails:
module Main where
import Control.Exception import Control.Concurrent import Control.Concurrent.STM
forever x = x >> forever x
count tv = do val <- takeTMVar tv putTMVar tv $! (succ val) return val
-- Capture idiom of notifying a new MVar when a thread is finished fork todo = do doneMVar <- newEmptyMVar tid <- block $ forkIO $ block $ handle (\e -> print ("Exception",e) >> throw e) (finally (unblock todo) (print "dying!" >> putMVar doneMVar ())) return (doneMVar,tid)
spawn = do tv <- atomically (newTMVar 0) fork . forever $ atomically (count tv) >>= print
kill (mv,tid) = do -- yield print "killing.." killThread tid print "..checking corpse.." readMVar mv print "..confirmed dead"
main = spawn >>= kill
On my system, the above sends the killThread and destroys the thread before the handle or finally are setup, so the child thread never prints anything and never runs the "putMVar doneMVar ()". If I uncomment the yield statement then the child thread does start executing and the handle and finally work as desired. This makes it impossible to reliably do anything with the child thread. I cannot discern between a living and dead child thread at all, as there is way to know if it is waiting to be scheduled or if it has been killed waiting to be scheduled. The get/setUncaughtExceptionHandler does not seem to be inherited by the child thread, so this was not a useful guard. The best thing I can come up with is the ugly code:
fork todo = block $ do doneVar <- atomically (newEmptyTMVar) let putStarted = atomically (putTMVar doneVar False) putStopped = atomically (tryTakeTMVar doneVar >> putTMVar doneVar True) tid <- forkIO $ block $ (finally (putStarted >> unblock todo) putStopped) yield atomically $ do value <- takeTMVar doneVar when value (putTMVar doneVar True) return (doneVar,tid)
This does not return the ThreadId until the finally clause has started running. But if the thread is killed by any external force before getting that far then the main thread will hang on the "takeTMVar doneVar". Is there any remotely better way of forking a child with an exception handler? -- Chris
Chris Kuklewicz wrote:
In response to question by Cat Dancer <cat@catdancer.ws> I wrote a few tests of sending asynchronous signal to a thread using GHC 6.6
The goal was to run a child thread via forkIO and use handle or finally to respond to the thread's demise.
Unfortunately, it seems that there is an irreducible window where this fails. The forkIO returns but any exception handlers such as block/handle/catch/finally are not in place yet.
We might consider it a bug that a new thread is started in "async exceptions unblocked" mode. It's true that this does mean there's no way to reliably start a thread and guarantee to know if it is killed. Starting threads in 'blocked' mode isn't the answer. That would force everyone to add an 'unblock' to their forkIO's after setting up an exception handler, and that's too easy to forget (and not backwards compatible). So instead we could introduce a new abstraction: forkCatchIO :: (Exception -> IO ()) -> IO () -> IO ThreadId which combines forkIO with catch in an atomic way, such that the handler is guaranteed to execute if the thread receives an exception. You can implement this using an MVar and not returning the ThreadId until the child thread is inside the exception handler (as in your example): this is ok because the only way a thread can receive an exception is by knowing its ThreadId. However it's less than perfect in performance terms; if we had a primitive fork that created a thread in blocked mode we could do much better. If there's some agreement that this is the way to go, I'll file a task for GHC and try to get to it before 6.8. Cheers, Simon
Hi, Thanks for the reply. I have my own proposal below: Simon Marlow wrote:
Chris Kuklewicz wrote:
In response to question by Cat Dancer <cat@catdancer.ws> I wrote a few tests of sending asynchronous signal to a thread using GHC 6.6
The goal was to run a child thread via forkIO and use handle or finally to respond to the thread's demise.
Unfortunately, it seems that there is an irreducible window where this fails. The forkIO returns but any exception handlers such as block/handle/catch/finally are not in place yet.
We might consider it a bug that a new thread is started in "async exceptions unblocked" mode. It's true that this does mean there's no way to reliably start a thread and guarantee to know if it is killed.
Starting threads in 'blocked' mode isn't the answer. That would force everyone to add an 'unblock' to their forkIO's after setting up an exception handler, and that's too easy to forget (and not backwards compatible).
Starting in 'blocked' mode is more primitive, since you can always safely define the unblocked version from it but never the other way around.
So instead we could introduce a new abstraction:
forkCatchIO :: (Exception -> IO ()) -> IO () -> IO ThreadId
which combines forkIO with catch in an atomic way, such that the handler is guaranteed to execute if the thread receives an exception.
If I run "forkCatchIO handler (block io)" then it looks like there is still a vulnerable gap for throwTo to trigger the handler before the "block" goes into effect. Is this correct? If so then I need an MVar again.
You can implement this using an MVar and not returning the ThreadId until the child thread is inside the exception handler (as in your example):
I take great consolation in having found the right workaround. Note that my solution has been changed to remove the "unblock" to create "forkBlocked":
forkBlocked todo = block $ do doneVar <- atomically (newEmptyTMVar) let putStarted = atomically (putTMVar doneVar False) putStopped = atomically (tryTakeTMVar doneVar >> putTMVar doneVar True) tid <- forkIO $ block $ (finally (putStarted >> todo) putStopped) yield atomically $ do value <- takeTMVar doneVar when value (putTMVar doneVar True) return (doneVar,tid)
Now I have both an exception hander and a guarantee that "todo" is running inside "block" and I could recover the previous function with "fork = forkBlocked . unblock".
this is ok because the only way a thread can receive an exception is by knowing its ThreadId. However it's less than perfect in performance terms; if we had a primitive fork that created a thread in blocked mode we could do much better.
If there's some agreement that this is the way to go, I'll file a task for GHC and try to get to it before 6.8.
Cheers, Simon
Being able to open a new thread in blocked mode would also be useful, as it gives control over what the "safepoints" for async interruption are. The forkCatchIO does not ensure I can entered "block" mode without the kind of MVar trick I had to use with forkIO. forkInheritIO :: IO () -> IO ThreadId -- inherits parent's block or unblock status forkBlockedIO :: IO () -> IO ThreadId -- starts the action in "block" mode. Must manually "unblock" where forkBlockedIO could be written as "block . forkInheritIO" and forkIO is "unblock . forkInheritIO" but there is no way to write forkInheritIO since I can't query the current block|unblock status dynamically. I have no use for forkInheritIO but perhaps some library code would want to play nice with the calling application. If either of those two functions existed then one could write "forkCatchIO" (which is unblocked) using it: forkCatchBlockedIO hander io = forkBlockedIO (handle handler io) forkCatchIO handler = (forkChatchBlockedIO handler) . unblock I propose adding at least "forkInheritIO" or "forkBlockedIO" as a primitive and perhaps "forkCatchIO" or "forkCatchBlockedIO" as a primitive if there is a performance gain. Cheers, Chris
Chris Kuklewicz wrote:
forkInheritIO :: IO () -> IO ThreadId -- inherits parent's block or unblock status
forkBlockedIO :: IO () -> IO ThreadId -- starts the action in "block" mode. Must manually "unblock"
Either of these is certainly possible, and I agree that forkBlockedIO is more general than my forkCatchIO. One tantalising possibility is that we could make forkIO have the forkInheritIO behaviourthe default without breaking much code
where forkBlockedIO could be written as "block . forkInheritIO" and forkIO is "unblock . forkInheritIO" but there is no way to write forkInheritIO since I can't query the current block|unblock status dynamically. I have no use for forkInheritIO but perhaps some library code would want to play nice with the calling application.
If either of those two functions existed then one could write "forkCatchIO" (which is unblocked) using it:
forkCatchBlockedIO hander io = forkBlockedIO (handle handler io) forkCatchIO handler = (forkChatchBlockedIO handler) . unblock
I propose adding at least "forkInheritIO" or "forkBlockedIO" as a primitive and perhaps "forkCatchIO" or "forkCatchBlockedIO" as a primitive if there is a performance gain.
Cheers, Chris
One odd problem: The paper on async exception defines: safePoint = unblock (return ()) but this simply does not work in my testing. Ever. Even using {-# NOINLINE safePoint #-} or "-Onot" By comparision, this does work: safepoint = unblock (print "safe") So how can such a safePoint be written? -- Chris
[ duh, hit send by mistake, message continues... ] Chris Kuklewicz wrote:
forkInheritIO :: IO () -> IO ThreadId -- inherits parent's block or unblock status
forkBlockedIO :: IO () -> IO ThreadId -- starts the action in "block" mode. Must manually "unblock"
Either of these is certainly possible, and I agree that forkBlockedIO is more generally useful than my forkCatchIO. One tantalising possibility is that we could make forkIO have the forkInheritIO behaviour without breaking much code, since I bet forkIO is almost never called inside block. Cheers, Simon
participants (3)
-
Chris Kuklewicz -
Simon Marlow -
Simon Marlow