#12758: Bring sanity to our performance testsuite -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: new Priority: high | Milestone: 8.8.1 Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3758 Wiki Page: | -------------------------------------+------------------------------------- Comment (by domenkozar): Some insight how I've implemented a similar setting for Snabb (open source high performance networking drivers). Using Nix, we build a matrix of Snabb (multiple git branches, different versions of software dependencies like Qemu, etc) and plot all of those graphs. There is no smart automation to abort of fail for contributions, but if you run these tests before releases - and even as a cronjob every X days, you can spot regressions easily and then bisect the commit. Nix tags what derivations are benchmarks and those run on specialized hardware 1 at the time. Example report: https://hydra.snabb.co/build/3641949/download/2/report.html -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12758#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler