[GHC] #8235: Race condition in IO manager
#8235: Race condition in IO manager ------------------------------------+------------------------------------- Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: Runtime crash Difficulty: Unknown | Test Case: Blocked By: | Blocking: 910 Related Tickets: | ------------------------------------+------------------------------------- === Race.hs {{{ #!haskell import GHC.Conc main :: IO () main = do setNumCapabilities 2 setNumCapabilities 1 setNumCapabilities 2 }}} === Command Line {{{ $ ghc-stage2 -threaded Race.hs [1 of 1] Compiling Main ( Race.hs, Race.o ) Linking Race ... $ while true; do ./Race; echo -n .; done ...............................Race: GHC.Event.Manager.loop: state is already Finished }}} It looks like there is a race condition in `GHC.Event.Thread.restartPollLoop` where a write to `emState` is snuck in between releasing the executing thread and -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by akio): It looks like the write to emState was triggered in {{{shutdownHaskellAndExit}}}, after the main thread (thread 4) died. So perhaps this is a normal condition, and the call to {{{error}}} should simply be removed? (I'm assuming the following thread id mapping) {{{ thread 1: helper thread in ioManagerStart() thread 2: IO manager for cap 0 thread 3: time manager thread 4: main thread thread 5: helper thread in setNumCapabilities() (1st call) thread 6: IO manager for cap 1 thread 7: helper thread in setNumCapabilities() (2nd call) thread 8: helper thread in setNumCapabilities() (3rd call) thread 9: IO manager for cap 1, which called Prelude.error }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Changes (by akio): * cc: tkn.akio@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Changes (by AndreasVoellmy): * cc: andreas.voellmy@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by AndreasVoellmy): I have not been able to reproduce this on my server. Can you give more detail on which version of GHC you are using? I.e. is it HEAD? Can you give the last commit? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by parcs): This is what I think is happening: 1. The third call to `setNumCapabilities` causes the event manager on cap 1 to be restarted via `restartPollLoop`. 2. `restartPollLoop` forks a new poll loop. 3. `main` finishes and the RTS begins shutting down. 4. The RTS sends a die message to the timer manager. 5. The RTS flushes its capabilities' run queues. The timer-manager thread is run first. The timer manager reads the die message and shuts down the event managers, setting each event manager's state to `Dying`. 6. The original poll loop on cap 1 is run next. Its state variable reads `Dying`, so it shuts itself down. Its state variable is now `Finished`. 7. The new poll loop on cap 1 is finally run for the first time. Its state variable initially reads `Finished`, so it throws an error. Does this make sense? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by parcs): Replying to [comment:4 AndreasVoellmy]:
I have not been able to reproduce this on my server. Can you give more detail on which version of GHC you are using? I.e. is it HEAD? Can you give the last commit?
It's HEAD. I'm on d02a50ee1437a24a0fb584286cf8f16cff59d217 but it shouldn't matter much I think. How long did you run {{{ while true; do ./Race; echo -n .; done }}} for? It can take a while (i.e. up to a minute) for the bug to manifest. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Changes (by parcs): * status: new => patch Comment: I attached a simple patch that fixes the issue. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by AndreasVoellmy): It took a while, but finally the bug occurred a few times on my server. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by AndreasVoellmy): Thanks! The patch looks good to me. I'll try it out and see if the bug still occurs. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Comment (by AndreasVoellmy): I don't see the bug occurring anymore (after about 10 minutes of running), so I'll merge this patch if everything validates cleanly. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager
-------------------------------------+------------------------------------
Reporter: parcs | Owner:
Type: bug | Status: patch
Priority: normal | Milestone:
Component: libraries/base | Version: 7.7
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture: Unknown/Multiple
Type of failure: Runtime crash | Difficulty: Unknown
Test Case: | Blocked By:
Blocking: 910 | Related Tickets:
-------------------------------------+------------------------------------
Comment (by Andreas Voellmy
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: merge Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Changes (by AndreasVoellmy): * status: patch => merge -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8235: Race condition in IO manager -------------------------------------+------------------------------------ Reporter: parcs | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: libraries/base | Version: 7.7 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: 910 | Related Tickets: -------------------------------------+------------------------------------ Changes (by AndreasVoellmy): * status: merge => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8235#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC