[GHC] #10860: The test `setnumcapabilities001` sometimes fails on Phabricator
#10860: The test `setnumcapabilities001` sometimes fails on Phabricator -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.12.1 Component: Runtime | Version: 7.10.2 System | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- {{{ setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 406 }}} https://phabricator.haskell.org/harbormaster/build/5890/ Here it just timed out: https://phabricator.haskell.org/harbormaster/build/5904/?l=0 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: The test `setnumcapabilities001` sometimes fails on Phabricator -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 7.12.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by Thomas Miedema <thomasmiedema@…>): In [changeset:"23a301ac36bb7e62bcf91bbcf0eb9e0e37498664/ghc" 23a301ac/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="23a301ac36bb7e62bcf91bbcf0eb9e0e37498664" Testsuite: comment out `setnumcapabilities001` (#10860) Don't mark it expect_broken, because it fails only sometimes. }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: The test `setnumcapabilities001` sometimes fails on Phabricator -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Changes (by thomie): * Attachment "setnumcapabilities-assert-failure-Scheduler.txt" added. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: The test `setnumcapabilities001` sometimes fails on Phabricator -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): -------------------------------------+------------------------------------- Changes (by thomie): * Attachment "setnumcapabilities-assert-failure-GC.txt" added. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * cc: simonmar (added) Comment: I can reproduce two different assert failures quite reliably. The output of `+RTS -Ds` is attached. Below are the stack traces from gdb. To reproduce: {{{ $ ../../../../inplace/bin/ghc-stage2 -threaded -debug setnumcapabilities001.hs -fforce-recomp $ gdb setnumcapabilities001 (gdb) run 4 8 2000 +RTS -Ds > stdout 2> stderr & }}} Retry the `run` command multiple times, and keep the cpu busy by running for example `ghc -e [0..] > /dev/null` in a separate terminal. Another way to reproduce the assert failures is to run: {{{ $ for i in `seq 1 100`; do echo ok && ./setnumcapabilities001 100 8 2000 +RTS -Ds 2> out || break ; done }}} == Assert failure in the Garbage Colletor == {{{ Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe6ffd700 (LWP 4419)] 0x00007ffff6ea7cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); bt #0 0x00007ffff6ea7cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ffff6eab0d8 in __GI_abort () at abort.c:89 #2 0x000000000078c912 in rtsFatalInternalErrorFn (s=0x80a160 "ASSERTION FAILED: file %s, line %u\n", ap=0x7fffe6ffcc68) at rts/RtsMessages.c:182 #3 0x000000000078c54a in barf (s=0x80a160 "ASSERTION FAILED: file %s, line %u\n") at rts/RtsMessages.c:46 #4 0x000000000078c5ad in _assertFail (filename=0x80e9d0 "rts/sm/GC.c", linenum=904) at rts/RtsMessages.c:61 #5 0x00000000007a314b in dec_running () at rts/sm/GC.c:904 #6 0x00000000007a32ec in scavenge_until_all_done () at rts/sm/GC.c:977 #7 0x00000000007a3431 in gcWorkerThread (cap=0x7fffec0214d0) at rts/sm/GC.c:1035 #8 0x0000000000787020 in yieldCapability (pCap=0x7fffe6ffce30, task=0x7fffec035680, gcAllowed=rtsTrue) at rts/Capability.c:708 #9 0x000000000078ee7a in scheduleYield (pcap=0x7fffe6ffce68, task=0x7fffec035680) at rts/Schedule.c:674 #10 0x000000000078e3dd in schedule (initialCapability=0x7fffec0214d0, task=0x7fffec035680) at rts/Schedule.c:298 #11 0x00000000007914a2 in scheduleWorker (cap=0x7fffec0214d0, task=0x7fffec035680) at rts/Schedule.c:2378 #12 0x000000000079a3f0 in workerStart (task=0x7fffec035680) at rts/Task.c:437 #13 0x00007ffff723e182 in start_thread (arg=0x7fffe6ffd700) at pthread_create.c:312 #14 0x00007ffff6f6b47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 }}} == Assert failure in the Scheduler == {{{ Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe77fe700 (LWP 4394)] 0x00007ffff6ea7cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); bt #0 0x00007ffff6ea7cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ffff6eab0d8 in __GI_abort () at abort.c:89 #2 0x000000000078c912 in rtsFatalInternalErrorFn (s=0x80a160 "ASSERTION FAILED: file %s, line %u\n", ap=0x7fffe77fdd58) at rts/RtsMessages.c:182 #3 0x000000000078c54a in barf (s=0x80a160 "ASSERTION FAILED: file %s, line %u\n") at rts/RtsMessages.c:46 #4 0x000000000078c5ad in _assertFail (filename=0x80a529 "rts/Schedule.c", linenum=400) at rts/RtsMessages.c:61 #5 0x000000000078e5ef in schedule (initialCapability=0x7fffec0214d0, task=0x7fffe0000910) at rts/Schedule.c:400 #6 0x00000000007914a2 in scheduleWorker (cap=0x7fffec0214d0, task=0x7fffe0000910) at rts/Schedule.c:2378 #7 0x000000000079a3f0 in workerStart (task=0x7fffe0000910) at rts/Task.c:437 #8 0x00007ffff723e182 in start_thread (arg=0x7fffe77fe700) at pthread_create.c:312 #9 0x00007ffff6f6b47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.0.1 => 8.2.1 Comment: Sadly won't be getting fixed for 8.0.1. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: 12038 | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonmar): * blockedby: => 12038 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:5> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.1 Component: Runtime System | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: 12038 | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar): I believe this is fixed, I've re-enabled the test and there's a residual issue in #12038. In [changeset:"cfc5df43a7789832a2789e517d8270650cc31b7f/ghc" cfc5df4/ghc]: {{{ Fix ASSERT failure and re-enable setnumcapabilities001 The assertion failure was fairly benign, I think, but this fixes it. I've been running the test repeatedly for the last 30 mins and it hasn't triggered. There are other problems exposed by this test (see #12038), but I've worked around those in the test itself for now. I also copied the relevant bits of the parallel library here so that we don't need parallel for the test to run. }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:6> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 8.2.1 Component: Runtime System | Version: 7.10.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonmar): * status: new => closed * resolution: => fixed * blockedby: 12038 => -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 8.2.1 Component: Runtime System | Version: 7.10.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Simon Marlow <marlowsd@…>): In [changeset:"ce13a9a9f57d61170837532948fed8bc1924a7ab/ghc" ce13a9a9/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="ce13a9a9f57d61170837532948fed8bc1924a7ab" Fix an assertion that could randomly fail Summary: ASSERT_THREADED_CAPABILITY_INVARIANTS was testing properties of the returning_tasks queue, but that requires cap->lock to access safely. This assertion would randomly fail if stressed enough. Instead I've removed it from the catch-all ASSERT_PARTIAL_CAPABILITIY_INVARIANTS and made it a separate assertion only called under cap->lock. Test Plan: ``` cd testsuite/tests/concurrent/should_run make TEST=setnumcapabilities001 WAY=threaded1 EXTRA_HC_OPTS=-with- rtsopts=-DS CLEANUP=0 while true; do ./setnumcapabilities001.run/setnumcapabilities001 4 9 2000 || break; done ``` Reviewers: niteria, bgamari, ezyang, austin, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2440 GHC Trac Issues: #10860 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:8> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10860: setnumcapabilities001: internal error: ASSERTION FAILED: file rts/Schedule.c, line 400 -------------------------------------+------------------------------------- Reporter: thomie | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 8.0.2 Component: Runtime System | Version: 7.10.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.2.1 => 8.0.2 Comment: Merged to `ghc-8.0` as b7d6e20ce342c4a2b46089dfe64498d28337afd9. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10860#comment:9> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC