On 30/11/12 03:54, Johan Tibell wrote:
While writing a new nofib benchmark today I found myself wondering whether all the nofib benchmarks are run just before each release, which the drove me to go look for a document describing the release process. A quick search didn't turn up anything, so I thought I'd ask instead. Is there a documented GHC release process? Does it include running nofib? If not, may I propose that we do so before each release and compare the result to the previous release*.
* This likely means that nofib has to be run for the upcoming release and the prior release each time a release is made, as numbers don't translate well between machines so storing the results somewhere is likely not that useful.
I used to do this on an ad-hoc basis: the nightly builds at MSR spit out nofib results that I compared against previous releases. In practice you want to do this much earlier than just before a release, because it can take time to investigate and squash any discrepancies. On the subject of the release process, I believe Ian has a checklist that he keeps promising to put on the wiki (nudge :)). Cheers, Simon