[GHC] #7970: Thread GC frees roots before thread actually finishes
#7970: Thread GC frees roots before thread actually finishes -----------------------------+---------------------------------------------- Reporter: joeyadams | Owner: Type: bug | Status: new Priority: normal | Component: Runtime System Version: 7.6.3 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Runtime crash | Blockedby: Blocking: | Related: 7170 -----------------------------+---------------------------------------------- In the following program, an IORef is garbage collected after a `NonTermination` exception, but is subsequently accessed: {{{ import Control.Exception as E import Data.IORef import System.Mem.Weak main :: IO () main = do ref <- newIORef 'x' weak <- mkWeakIORef ref $ putStrLn "IORef finalized" let check = deRefWeak weak >>= \m -> case m of Nothing -> putStrLn "IORef was GCed" Just ref' -> do x <- readIORef ref' putStrLn $ "IORef still alive, and contains " ++ show x let loop = loop check loop `catch` \ex -> do putStrLn $ "caught exception: " ++ show (ex :: SomeException) check readIORef ref >>= print }}} Output: {{{ IORef still alive, and contains 'x' IORef finalized caught exception: <<loop>> IORef was GCed 'x' }}} The same happens with other thread deadlocks, such as: * newEmptyMVar >>= takeMVar * atomically retry It does not happen when a `StackOverflow` or `UserInterrupt` exception is caught. This also affects `ForeignPtr`; see the attached "database" example. This is what really triggered #7170. I marked this "Runtime crash" because it can lead to a `ForeignPtr` being accessed after the garbage collector finalized it. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes ---------------------------------+------------------------------------------ Reporter: joeyadams | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Runtime crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: 7170 | ---------------------------------+------------------------------------------ Changes (by simonmar): * owner: => simonmar * difficulty: => Unknown * priority: normal => high * milestone: => 7.8.1 Comment: This is technically not a bug - the thread is unreachable when it is deadlocked, and so is the weak pointer. However, we have had a feature request in the past to change this behaviour, and I can see that in this case it is less than useful. I'll look into reversing the order so that we establish the reachability of threads before weak pointers. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes ---------------------------------+------------------------------------------ Reporter: joeyadams | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Runtime crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: 7170 | ---------------------------------+------------------------------------------ Comment(by joeyadams): Replying to [comment:1 simonmar]:
This is technically not a bug - the thread is unreachable when it is deadlocked, and so is the weak pointer.
It only makes sense to call a thread "unreachable" if it will never run again. But when the RTS detects an unreachable thread and prods it with `NonTermination`, the thread continues running and accessing its local variables, meaning it should be considered reachable again. Here are some valid solutions, based on my understanding: 1. Treat a thread as reachable until it actually finishes or dies. Downside: deadlocked threads would no longer be GCed. 2. Don't throw `BlockedIndefinitelyOnMVar`, `NonTermination`, etc., just free the thread. This keeps the "unreachable" interpretation valid. Downside: manually-managed resources (including those managed with bracket) would be leaked. This is like (1), but at least allows Haskell objects to be GCed. 3. When the RTS detects a deadlocked thread, treat it as reachable again before making it throw an exception. This keeps the current behavior, but seems difficult to implement. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes ---------------------------------+------------------------------------------ Reporter: joeyadams | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Runtime crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: 7170 | ---------------------------------+------------------------------------------ Comment(by simonmar):
It only makes sense to call a thread "unreachable" if it will never run again.
Which is the case now - at the point where the thread is unreachable, it will never run again. So we send it an exception to tell it (at which point it runs again of course, but if it were not for the exception it would not run). The specific problem in this ticket is the interaction between threads and weak pointers. Right now, it is the case that * a reference from the finalizer of a weak pointer to a thread makes the thread reachable, even if the key of the weak pointer is not reachable. This means that you can have a thread blocked waiting for a finalizer to run. There's an alternative design, which would make the program in this ticket work: * a reference from a thread to a weak pointer makes the weak pointer reachable, even if the thread itself is not reachable. You can choose one of these, or neither, but not both. The reason is that the GC must do these steps in some order: * decide which weak pointers are unreachable, and make their finalizers alive * decide which threads are unreachable, and then make them alive if we did both at the same time, you would get *neither* of the behaviours above. So I think what the example in this ticket argues for is switching the order. I'm fine with that (I buy the argument) but it does potentially break the use case of having a thread blocked on a finalizer. I'm not sure whether anyone actually does that or not. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes ---------------------------------+------------------------------------------ Reporter: joeyadams | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Runtime crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: 7170 | ---------------------------------+------------------------------------------ Comment(by simonmar): So I implemented the fix, and in doing so discovered that it broke a test (`conc031`), which is a test for the old behaviour, see #551. Let me be clear about what's going on. These two GC features: 1. A thread can be blocked on a finalizer without being considered deadlocked 2. A weak pointer will not be finalized if a deadlocked thread references it are mutually exclusive. We currently have (1) (because the lack of it was reported as a bug in #551), and this ticket complains about the lack of (2). But implementing (2) reverses (1). Here's another complaint about the lack of (2): http://www.haskell.org/pipermail/haskell-cafe/2010-August/081953.html (the whole thread is interesting) The evidence so far is that the lack of (2) is more surprising than the lack of (1), so I'm slightly tempted to go with (2). There's also a workaround: `mkStablePtr` can be used to keep anything alive (and prevents threads from getting the deadlock exception), but you have to manually `freeStablePtr` later. Finalizers are a pain :) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes ---------------------------------+------------------------------------------ Reporter: joeyadams | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Runtime crash Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: 7170 | ---------------------------------+------------------------------------------ Comment(by simonmar): ... and we also have a 5-year-old ticket reporting the lack of (2): #2161. The evidence in favour of (2) is mounting. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes
---------------------------------+------------------------------------------
Reporter: joeyadams | Owner: simonmar
Type: bug | Status: new
Priority: high | Milestone: 7.8.1
Component: Runtime System | Version: 7.6.3
Keywords: | Os: Unknown/Multiple
Architecture: Unknown/Multiple | Failure: Runtime crash
Difficulty: Unknown | Testcase:
Blockedby: | Blocking:
Related: 7170 |
---------------------------------+------------------------------------------
Comment(by marlowsd@…):
commit 1ae72ac45e51e63cbf6f3d627d77acc6a36aa0f9
{{{
Author: Simon Marlow
#7970: Thread GC frees roots before thread actually finishes -------------------------------+-------------------------------------------- Reporter: joeyadams | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Runtime crash | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: 7170 -------------------------------+-------------------------------------------- Changes (by simonmar): * status: new => closed * resolution: => fixed -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes -------------------------------+-------------------------------------------- Reporter: joeyadams | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Runtime crash | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: 7170 -------------------------------+-------------------------------------------- Comment(by simonpj): This is a long thread describing subtle properties of finalisers. Would it be worth some documentation in the user manual to describe the properties that you can (and cannot) now rely on? Simon -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7970: Thread GC frees roots before thread actually finishes -------------------------------+-------------------------------------------- Reporter: joeyadams | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Runtime crash | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: 7170 -------------------------------+-------------------------------------------- Comment(by simonmar): Done - it was hard to know where to document this, but I ended up putting a section in `Control.Concurrent` which has other detailed information about the workings of threads. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7970#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC