How to treat testsuite timeout as success?

Ola! I was trying to re-re-fix https://ghc.haskell.org/trac/ghc/ticket/8089 and add a test-case for this fix. However, I'm having trouble creating a test that can actually be successful. The simplest case exhibiting the issue (on OSX) is that "main = threadDelay maxBound" crashes, I need a way to treat a timeout of say 5-10s as success, since the testsuite won't let my test run for 25,000 days :D I can't add a timeout inside the haskell code (along the lines of "threadDelay 10000000 >> exitSuccess" since installing any other timers with a timeout shorter than the crashing one masks the crash. Marking the test as expected to fail doesn't help, as this treats both a crash due to the bug and a timeout as success. Clearly only timeout should be considered successful. Cheers, Merijn PS - For the curious willing to help verify that the third time's the charm: https://phabricator.haskell.org/D407

Probably the easiest thing to do is write a little stub program which calls the actual test with a timeout. Edward Excerpts from Merijn Verstraaten's message of 2014-11-01 16:01:35 -0700:
Ola!
I was trying to re-re-fix https://ghc.haskell.org/trac/ghc/ticket/8089 and add a test-case for this fix. However, I'm having trouble creating a test that can actually be successful. The simplest case exhibiting the issue (on OSX) is that "main = threadDelay maxBound" crashes, I need a way to treat a timeout of say 5-10s as success, since the testsuite won't let my test run for 25,000 days :D
I can't add a timeout inside the haskell code (along the lines of "threadDelay 10000000 >> exitSuccess" since installing any other timers with a timeout shorter than the crashing one masks the crash. Marking the test as expected to fail doesn't help, as this treats both a crash due to the bug and a timeout as success. Clearly only timeout should be considered successful.
Cheers, Merijn
PS - For the curious willing to help verify that the third time's the charm: https://phabricator.haskell.org/D407
participants (2)
-
Edward Z. Yang
-
Merijn Verstraaten