#12758: Bring sanity to our performance testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: new Priority: high | Milestone: 8.8.1 Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3758 Wiki Page: | -------------------------------------+------------------------------------- Comment (by davide): In the case of expected changes in performance metrics, we still need the tests to pass. This requires some **convenient** mechanism by which the author can explicitly inform the CI of that change. The current plan is for the author to indicate such changes in the commit message in the following form: {{{ Metric (Increase | Decrease) ['metric' | \['metrics',..\]] [\((test_env|way)='abc',...\)]: TestName01, TestName02, ... }}} e.g. {{{ # All metrics decreased for test T1234. Metric Decrease: T1234 # max_bytes_used decreased for test T1234. Metric Decrease 'max_bytes_used': T1234 # bytes allocated and peak_megabytes_allocated increased for tests T005 and T006. Metric Increase ['bytes allocated', 'peak_megabytes_allocated']: T005, T00 # All metrics decreased for test T200 in the x86_linux environment Metric Decrease (test_env = 'x86_linux'): T200 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12758#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler