#15374: Testsuite: compile_timeout_multiplier is fragile -------------------------------------+------------------------------------- Reporter: goldfire | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: Compiler | Version: 8.4.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I'm taking a sweep through the testsuite, trying to eliminate failures in a `DEBUG` compiler. But two tests are stymieing me: `pmcheck/should_compile/T11{276,374}`. Both of these use the `compile_timout_multiplier` option in their `all.T` entries. This option means that, instead of the normal 300s allotted to the test, these tests get only 3s. In my setup, with a `DEBUG` (read: unoptimized) stage2 compiler, these tests fail when I'm running the testsuite in parallel. Then, when I run the tests individually, they succeed. This poses two problems: 1. I can only see the failure when my machine is stressed, and it's disconcerting to have a failure appear and disappear depending on how I look. 2. There is (sometimes) a failure in `DEBUG` mode. What is the goal of `compile_timeout_multiplier`? If we're really testing performance (which #11276 and #11374 indicate), then these tests should be in `perf/compiler` and be "stat" tests, where some wibbling in the numbers is expected (and where `DEBUG` compilers are exempted). I would imagine that `compile_timeout_multiplier` should only ever get values greater than 1, when a non-performance test is so expensive to run that it takes more than 300s. As a local solution here, I think these two tests should move to `perf/compiler`. Other pattern-match tests also have `compile_timeout_multiplier`, too, and I can move these as well. Any objections? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15374 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler