#9315: Weird change in allocation numbers of T9203 -------------------------------------+------------------------------------- Reporter: nomeata | Owner: simonmar Type: bug | Status: new Priority: normal | Milestone: Component: Runtime | Version: 7.9 System | Keywords: Resolution: | Operating System: Unknown/Multiple Differential Revisions: | Type of failure: None/Unknown Architecture: | Test Case: Unknown/Multiple | Blocking: Difficulty: Unknown | Blocked By: | Related Tickets: | -------------------------------------+------------------------------------- Comment (by nomeata): I tried to bring it up on the mailing list, but nobody cared...
What I'd like to see instead is one of the following:
* Skip the test unless the libraries were built with -O2 * Skip the test *if* the libraries were built with -O2 * Have different results according to the optimisation levels
All of which fix the perceived problem without introducing the regression in validate time.
While the third option is the most complete, I don’t think it is maintainable. Given the aim of validate is to prevent breakage, and performance regressions are not breakage, I think it is ok to skip the tests there (even makes it faster!), and run the tests only when we are doing a release build. So how about this: * The expected numbers are adjusted to reflect the release settings. * `validate` in modes fast and normal pass `SKIP_PERF_TESTS` to the test suite * `validate` in slow mode uses the release settings (`-O2` to libraries and stage2), and does not `SKIP_PERF_TESTS` * Optionally: The test suite, when invoked manually, ignores performance tests if the flags are not as expected¹ Note that the test suite already skips compiler performance tests if `-DDEBUG` is set, so this has precedent. ¹ Is that possible in a reliable and not-too-complicate way, and without duplicating the “expected flags”? Maybe a warning “You have a custom `mk/build.mk`; failures in performance tests might be irrelevant.” would be enough -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9315#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler