[GHC] #10345: Testsuite timeout_multiplier setting does not work as expected
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Other Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- I have a test case that - if failed - sends GHC into an infinite loop. The test case is small so I don't want to wait for default timeout (currently that's 5 minutes). The testsuite provides `timeout_multiplier` function to modify the default timeout but that setting seems to be ignored and the test is killed only after the default timeout time. I added my test to `all.T` file like this: {{{ test('tc265', timeout_multiplier(0.01), compile, ['']) }}} I also tried putting timeout setting into an array (`[timeout_multiplier(0.01)]`) but with no result. Merijn reports having same problems and bypassing them by writing a test that forks itself and kills the child process after a given timeout - see [[GhcFile(libraries/base/tests/T8089.hs)]]. I am adding a note on [[wiki:Building/RunningTests/Adding#Thesetupfield]] that this field does not work currently. Once this bug is fixed we need to remove that note. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): I can not reproduce this issue yet. Can you try a few things. I am assuming you are on Linux. When you run the following, does it stop after 3 seconds? * `python testsuite/timeout/timeout.py 3 'sleep 100'` * In `testsuite/tests/ghc-api/annotations/all.T`, make the following change: {{{ -test('annotations', normal, run_command, ['$MAKE -s --no-print-directory annotations']) +test('annotations', timeout_multiplier(0.01), run_command, ['$MAKE -s --no-print-directory annotations']) }}} When running `make TEST=annotations`, does it stop after 3 seconds? I guess it doesn't, otherwise you wouldn't have opened this ticket, but I want to be sure it's not something with the test you were writing in particular. It must be something specific to your platform, as timeout works for me. The issue only I seem to be having is that running `python testsuite/timeout/timeout.py 10 'echo ok' ` doesn't return immediately, but only after 10 seconds. Can you try that one as well? For some reason producing output is not allowed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by jstolarek): Replying to [comment:1 thomie]:
I am assuming you are on Linux. Yes, that's correct
When you run the following, does it stop after 3 seconds? * `python testsuite/timeout/timeout.py 3 'sleep 100'` Yes.
* In `testsuite/tests/ghc-api/annotations/all.T`, make the following change: {{{ -test('annotations', normal, run_command, ['$MAKE -s --no-print- directory annotations']) +test('annotations', timeout_multiplier(0.01), run_command, ['$MAKE -s --no-print-directory annotations']) }}} When running `make TEST=annotations`, does it stop after 3 seconds? Yes.
I guess it doesn't, otherwise you wouldn't have opened this ticket So the above tests work, but my tests still doesn't time out. An obvious explanation is that I've configured the test incorrectly. Another is that the timeout does not work if GHC becomes unresponsive (ie. hits a black hole). (In the latter case: why would a 5 minute timeout work?)
The issue only I seem to be having is that running `python testsuite/timeout/timeout.py 10 'echo ok' ` doesn't return immediately, but only after 10 seconds. Can you try that one as well? For some reason producing output is not allowed.
That work for me as it should - finishes immediately. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): There are 2 timeout programs. On Linux we use the one written in Python by default, on Windows the one in written in Haskell. Can you try the following: {{{ $ cd testsuite/timeout $ rm -r install-inplace $ make WINDOWS=YES $ cd ../ $ make TEST='<your test>' }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected
-------------------------------------+-------------------------------------
Reporter: jstolarek | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Test Suite | Version: 7.11
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
Type of failure: Other | Unknown/Multiple
Blocked By: | Test Case:
Related Tickets: | Blocking:
| Differential Revisions:
-------------------------------------+-------------------------------------
Comment (by thomie):
Sorry, the one in Haskell doesn't compile at the moment, I forgot. You
need this patch:
{{{
commit c5759a04c73de36f94bb7239f9ead91f8741e6af
Author: Thomas Miedema
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by jstolarek): Still the same. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): Should I try building Phab:D202 or do you have a smaller testcase? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by jstolarek): I'm afraid I don't have a smaller one :-/ If you want to build D202 please note that it changes format of interface files. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): Oh, I see it now. Fix coming. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected
-------------------------------------+-------------------------------------
Reporter: jstolarek | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Test Suite | Version: 7.11
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
Type of failure: Other | Unknown/Multiple
Blocked By: | Test Case:
Related Tickets: | Blocking:
| Differential Revisions:
-------------------------------------+-------------------------------------
Comment (by Thomas Miedema
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: closed Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.11 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: Phab:D982 -------------------------------------+------------------------------------- Changes (by thomie): * status: new => closed * differential: => Phab:D982 * resolution: => fixed * milestone: => 7.12.1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10345: Testsuite timeout_multiplier setting does not work as expected -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: closed Priority: normal | Milestone: 8.0.1 Component: Test Suite | Version: 7.11 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D982 Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): Replying to [comment:1 thomie]:
The issue only I seem to be having is that running `python testsuite/timeout/timeout.py 10 'echo ok' ` doesn't return immediately, but only after 10 seconds.
I finally figured this one out. I had `stty tostop` set in `~/.bashrc`. When `stty tostop` is set, a background process that tries to write to the terminal receives a SIGTTOU signal, which suspends it. In timeout.py, the first thing the child process does is call `os.setpgrp()`. This creates a new process group, with the child process itself as process group leader. Because only one process group per session can be the foreground process group, it will be a background process group. I'm not sure yet //why// timeout.py turns the child process that it creates into a background process. One implication is that `CTRL-C` doesn't kill it. References: * https://www.win.tue.nl/~aeb/linux/lk/lk-10.html * http://www.linusakesson.net/programming/tty/index.php -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10345#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC