I'm particularly interested in parallel performance in the >8 core space.
(In fact, we saw some regressions from 7.2->7.4 that we never tracked down
properly, but maybe can now.)
If the buildbot can make it easy to add a new "slave" machine that runs and
uploads its result to a central location, then I would be happy to donate a
few hours of dedicated time (no other logins) on a 32 core westmere
machine, and hopefully other architectures soon.
Maybe, this use case is well-covered by creating a jenkins/travis slave and
letting it move the data around? (CodeSpeed looks pretty nice too.)
Cheers,
-Ryan
On Wed, Dec 5, 2012 at 12:40 AM, Ben Lippmeier
On 01/12/2012, at 1:42 AM, Simon Peyton-Jones wrote:
| > While writing a new nofib benchmark today I found myself wondering | > whether all the nofib benchmarks are run just before each release,
I think we could do with a GHC Performance Tsar. Especially now that Simon has changed jobs, we need to try even harder to broaden the base of people who help with GHC. It would be amazing to have someone who was willing to:
* Run nofib benchmarks regularly, and publish the results
* Keep baseline figures for GHC 7.6, 7.4, etc so we can keep track of regressions
* Investigate regressions to see where they come from; ideally propose fixes.
* Extend nofib to contain more representative programs (as Johan is currently doing).
That would help keep us on the straight and narrow.
I was running a performance regression buildbot for a while a year ago, but gave it up because I didn't have time to chase down the breakages. At the time we were primarily worried about the asymptotic performance of DPH, and fretting about a few percent absolute performance was too much of a distraction.
However: if someone wants to pick this up then they may get some use out of the code I wrote for it. The dph-buildbot package in the DPH repository should still compile. This package uses http://hackage.haskell.org/package/buildbox-1.5.3.1 which includes code for running tests, collecting the timings, comparing against a baseline, making pretty reports etc. There is then a second package buildbox-tools which has a command line tool for listing the benchmarks that have deviated from the baseline by a particular amount.
Here is an example of a report that dph-buildbot made:
http://log.ouroborus.net/limitingfactor/dph/nightly-20110809_000147.txt
Ben.
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users