Dear ghc developers, last July I set up a performance dashboard for GHC, i.e. a website where you can see our performance numbers (nofib results, performance unit tests, build time etc.) for each commit. I used the software codespeed¹ as the backend, but I was dissatisfied with it: I found it was a tad too slow, the URLs were hard to share, but – most importantly – it tries to be VCS-agnostic, which just doesn’t work well. So in January, I wrote a new tool, called gipeda (Git Performance Dashboard)². The Haskell part generates static files, which are then displayed using JavaScript in the client (using bootstrap and handlebars), so it is reasonably fast. And it knows about (some) git concepts. I was about to announce it in February, but it ran on the same host as deb.haskell.org, which was compromised a few days before the announcement. Recently, though, davean of the haskell.org Admin team provided me with the required set (thanks for that!), so here it is: http://perf.haskell.org/ghc It should be relatively self-explanatory. Just note that by default it hides boring stuff (commits and results with no significant change), you can select what to show in the top-right corner. It does roughly what our codespeed setup did before, but I have some ideas that go beyond that: Comparison of arbitrary commits, better understanding of git branches and tags, aggregation of performance results from different hosts, e-mail notification. I hope that I’m not the only one who will work on them, and will be happy to receive help. The upcoming ZuriHac is a great possibility to join me here. There are also lot of ways you can contribute without touching Haskell (not that there are many people on this list to whom that matters :-)). Gipeda itself is in no way specific to GHC and you can use it for your own projects. We can possibly even host the results on http://perf.haskell.org/ – just talk to me. Also, see my gipeda announcement on haskell-cafe (which I’ll write next). Enjoy! Joachim ¹ https://github.com/tobami/codespeed ² https://github.com/nomeata/gipeda -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
On 05/17/15 02:19 PM, Joachim Breitner wrote:
provided me with the required set (thanks for that!), so here it is:
It should be relatively self-explanatory.
Joachim, this is indeed great stuff. But although it's self-explanatory I'm not able to find out what I'm looking for, i.e. simple way to see graph of whole nofib performance results. i.e. not graphs per benchmark, but per whole nofib suite. Is that possible? Thanks! Karel
Hi, Am Sonntag, den 17.05.2015, 20:00 +0200 schrieb Karel Gardas:
this is indeed great stuff. But although it's self-explanatory I'm not able to find out what I'm looking for, i.e. simple way to see graph of whole nofib performance results. i.e. not graphs per benchmark, but per whole nofib suite. Is that possible?
indeed, not yet! I’m not quite sure if this should be handled in gipeda, or rather in the project-specific script. After all, such an average is quite domain specific (average? weighted average? median? what else). Maybe in the case of nofib, nofib-analize should calculate an "overall performance number" in a sensible way, and gipeda can treat it just like any other benchmark number. I filed https://github.com/nomeata/gipeda/issues/7 for this. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
Outstanding work! Cheers, Edward Excerpts from Joachim Breitner's message of 2015-05-17 05:19:55 -0700:
Dear ghc developers,
last July I set up a performance dashboard for GHC, i.e. a website where you can see our performance numbers (nofib results, performance unit tests, build time etc.) for each commit. I used the software codespeed¹ as the backend, but I was dissatisfied with it: I found it was a tad too slow, the URLs were hard to share, but – most importantly – it tries to be VCS-agnostic, which just doesn’t work well.
So in January, I wrote a new tool, called gipeda (Git Performance Dashboard)². The Haskell part generates static files, which are then displayed using JavaScript in the client (using bootstrap and handlebars), so it is reasonably fast. And it knows about (some) git concepts.
I was about to announce it in February, but it ran on the same host as deb.haskell.org, which was compromised a few days before the announcement. Recently, though, davean of the haskell.org Admin team provided me with the required set (thanks for that!), so here it is:
It should be relatively self-explanatory. Just note that by default it hides boring stuff (commits and results with no significant change), you can select what to show in the top-right corner.
It does roughly what our codespeed setup did before, but I have some ideas that go beyond that: Comparison of arbitrary commits, better understanding of git branches and tags, aggregation of performance results from different hosts, e-mail notification. I hope that I’m not the only one who will work on them, and will be happy to receive help. The upcoming ZuriHac is a great possibility to join me here.
There are also lot of ways you can contribute without touching Haskell (not that there are many people on this list to whom that matters :-)).
Gipeda itself is in no way specific to GHC and you can use it for your own projects. We can possibly even host the results on http://perf.haskell.org/ – just talk to me. Also, see my gipeda announcement on haskell-cafe (which I’ll write next).
Enjoy!
Joachim
¹ https://github.com/tobami/codespeed ² https://github.com/nomeata/gipeda
participants (3)
-
Edward Z. Yang -
Joachim Breitner -
Karel Gardas