RE: [commit: ghc] master: Update various performance benchmarks (194107e)
| I started monitoring perfomance on a per-commit base. These seem to | be | off for a while now. Adjusting them, and from now I hope I can keep | closer tabs on them. Thank you. That's most helpful. One of the reasons that perf tests get worse is that each time the limit is reached, the author thinks "oh my bit probably only made it a tiny bit worse", so we miss big jumps. It's the big jumps we thereby miss, and nailing them to a particular commit would be great. I'ts important to monitor the actual bump, rather than the first time the limit is broken. (They aren't the same, of course.) Simon
Hi, Am Montag, den 14.07.2014, 11:13 +0000 schrieb Simon Peyton Jones:
I'ts important to monitor the actual bump, rather than the first time the limit is broken. (They aren't the same, of course.)
Quite right. I am in the process of setting up per-commit monitoring of _nofib_ results, and I just added “number of failing tests” to the list of “benchmark numbers”. But that doesn’t help with performance testcases Simple way forward: Always print the results from performance benchmarks (not only when they are out of the limits). Then, when they break the limit, one can go and grep through the old logs for changes. Next step after that: Include the numbers from all performance benchmarks in the same monitoring as the nofib numbers. Technically easy, but I’m not sure how well it scales for the large(?) number of perf tests we have. But one step at a time. Expect an announcement of the performance dashboard thingy soon. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
participants (2)
-
Joachim Breitner -
Simon Peyton Jones