#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHCi | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Runtime | Blocked By: crash | Related Tickets: Test Case: ghcirun004 | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Does anyone have a clue what is going on here? I regularly get a batch of failures {{{ Unexpected failures: ffi/should_run ffi018_ghci [bad stdout or stderr] (ghci) ghci/should_run T2589 [bad stdout or stderr] (ghci) ghci/should_run ghcirun001 [bad stdout or stderr] (ghci) ghci/should_run ghcirun002 [bad exit code] (ghci) ghci/should_run ghcirun003 [bad stdout or stderr] (ghci) ghci/should_run ghcirun004 [bad stdout or stderr] (ghci) }}} all of which are this `ioManagerWakeup:write: Bad file descriptor` thing. What is `ioManagerWakeup`? Who calls it? Why? What does it mean to have a bad file descriptor? It's bad because I can't get a clean validate these days. Incidentally {{{ make TEST=ghcirun001 }}} works fine! Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: GHCi | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Runtime | Blocked By: crash | Related Tickets: Test Case: ghcirun004 | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by AndreasVoellmy): * cc: AndreasVoellmy (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: GHCi | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Runtime | Blocked By: crash | Related Tickets: Test Case: ghcirun004 | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): For me they never occur when running the test on its own. They only occur during `sh validate --fast`. The difference, perhaps, is that many tests are running at the same time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: GHCi | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Runtime | Blocked By: crash | Related Tickets: Test Case: ghcirun004 | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by ezyang): I haven't seen this bug recently, but I don't see any commits which might have solved it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.1 Component: GHCi | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thoughtpolice): * priority: highest => high -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.1 Component: GHCi | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by edsko): * cc: edsko (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thomie): * cc: simonmar (added) * owner: => simonmar * component: GHCi => Runtime System Comment: I can reproduce the notorious 'ioManagerWakeup' bug. Not for `ghci004` itself, but for some other ghci tests. On my puny laptop with only 2 cores, I do the following: 1. Make sure the cpu is plenty busy, by running `ghc -e [0..] > /dev/null` in 3 different shells. 2. Run `make test EXTRA_HC_OPTS='+RTS -I0.1 -RTS' TEST='print002 print003 print006 print007 print008 print010'` For me, all tests always fail with: "ioManagerWakup: write: Bad file descriptor" Step 2 sets the 'GC idle time' for ghci script tests to 0.1 seconds (from the default for GHC/GHCi of 5 seconds, see #3408); the same value that is used for other tests with WAY=ghci (among which `ghci004`). The necessity of step 1 might explain why we don't see the bug when running a single test in isolation, but only when running validate, which runs a lot of tests at the same time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by thomie): Smaller test case (without any cpu loading) for the real bug: {{{ $ cat T9722.hs main = return () $ ghc-7.10.0.20150307 -threaded -rtsopts T9722.hs $ ./T9722 +RTS -I0.000000001 T9722: ioManagerWakeup: write: Bad file descriptor }}} This only happens ~50% of the time, and it doesn't happen with 7.8.4. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by AndreasVoellmy): Replying to [comment:17 AndreasVoellmy]:

Where is T9722.hs? Is it in master branch somewhere?

Nevermind. I see the contents of T9722 from your message. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by thomie): Andreas: if your computer has more than 2 cores you might need to start more copies of `ghc -e [0..] > /dev/null`. Another good one to keep the cpu is busy is to compile `ghc` with `-j<ncores>` at the same time. Do all of that, and then run the `print002` test with `+RTS -I<some small number>`. It should work... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by thomie): I am on 64 bit Ubuntu 14.04. I can reproduce with `ghc-7.10.0.20150307` and `ghc-7.11.20150307` from https://launchpad.net/~hvr/+archive/ubuntu/ghc (ghc-head). I have attached the output from running `./T9722 +RTS -I0.0001 -Ds`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by thomie): I build a fresh ghc head (commit 8b7534b39052c9cb44411bea0ca311a751564d6c) with the change to `ioManagerWakeup` from comment:23. {{{ $ ~/ghc-master-build/inplace/bin/ghc-stage2 -threaded -rtsopts T9722.hs $ ./T9722 +RTS -I0.000000001 ioManagerWakeup: write(A): Bad file descriptor T9722: ioManagerWakeup: write; fd: 9: Bad file descriptor }}} I don't understand why the output is different when I redirect it to a file. But in case it gives you a clue, here it is: {{{ $ ./T9722 +RTS -I0.000000001 > T9722.stderr $ cat T9722.stderr ioManagerWakeup: write(A): Bad file descriptor T9722: ioManagerWakeup: write; fd: 9: Invalid argument }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by thomie): I ran all the tests I mentioned before multiple times, and they all succeeded. Hurray! Before shipping: I did get a build warning about unused `UNPACK` in `W`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: patch Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714 -------------------------------------+------------------------------------- Comment (by Andreas Voellmy ): In [changeset:"74625d6847e970e8bdc6991c327515b3e10b231b/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="74625d6847e970e8bdc6991c327515b3e10b231b" RTS/IOManager: fix trac issue #9722. Summary: Whenever the RTS has been inactive for idleGCDelayTime, the idle timer fires and calls wakeUpRts(), which in turn calls ioManagerWakeup(), which in turn writes a byte (or a few) to a file descriptor (stored in the io_manager_wakeup_fd variable) registered by the TimerManager and on which the TimerManager will wait. (Note that the write will only occur if the file descriptor is non-negative.) When the RTS shuts down, it shuts down the TimerManager, and in this process the file descriptor stored in io_manager_wakeup_fd is closed. In the error case, the idle timer fires after the close of the file occurs, and then the write() call in ioManagerWakeup() fails and the aforementioned error message gets printed. This patch solves the problem by (1) having the TimerManager (via Control) write -1 to io_manager_wakeup_fd just before closing the file descriptor written in io_manager_wakeup_fd, and (2) having ioManagerWakeup() ignore an error returned by write() in the case that the write returned -1 and the io_manager_wakeup_fd is -1. Reviewers: austin, simonmar, hvr, thomie Reviewed By: thomie Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D722 GHC Trac Issues: #9722 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9722: ghcirun004 intermittently fails with ghc: ioManagerWakeup: write: Bad file descriptor -------------------------------------+------------------------------------- Reporter: ezyang | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 7.10.1 Component: Runtime System | Version: 7.9 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: ghcirun004 Related Tickets: | Blocking: | Differential Revisions: Phab:D714, | Phab:D722 -------------------------------------+------------------------------------- Changes (by thoughtpolice): * status: merge => closed * resolution: => fixed Comment: Merged (via fb2ab1e3e04fde78d8970815f83b90d54359ae82), thanks! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9722#comment:30 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler