
Hi all, I'd just like to query the status of the nightly builds. Is anything happening in that area? [1] is right on the front page of the GHC Trac even though no builds were ran for ~5 months. Perhaps it should be moved out of the way if there's no plan to resume these in the near future. Does anything specific need doing to get these to run again? [1]: https://ghc.haskell.org/trac/ghc/wiki/Builder -- Mateusz K.

As of right now, Pali's FreeBSD builds seem to be the only nightly
that is still consistently running (and thanks to him for that!)
The build infrastructure in its current status is mainly just
'unmaintained'. Furthermore there's not really a good roster of
machines that were/were not part of the system AFAIK aside from the
old list, and it's unclear what the status of many of those machines
are (as you said, many haven't checked in in a while.)
There is much interest in a better nightly infrastructure and people
have asked me several times about setting one up on IRC. We have
historically had some problems with the nightly infrastructure, mainly
things like network disconnectivity or firewalling policies, since
most people aren't running dedicated internet facing machines (or even
a dedicated machine at all. Firewalls have been a problem for places
like MSR from what I understand.)
Several individual people run Jenkins individually, and I like it, but
I'm not sure how well it does when spread across the globe in terms of
networking (and realistically builders will look like that, as we
can't possibly have a dedicated farm somewhere.) I was also at one
point worried about the size of such a tool on systems like ARM
machines where resources are at a premium, but in hindsight this looks
OK. I'd like any opinions on this if people have deployed things in
these highly distributed scenarios.
I have had some ideas for an extremely-minimal nightly build
infrastructure that would ideally require minimal setup and let
clients have power over choosing how and when to build, but I have yet
to find the time to finish the basic implementation to try it.
On Sat, Jan 25, 2014 at 8:22 PM, Mateusz Kowalczyk
Hi all,
I'd just like to query the status of the nightly builds. Is anything happening in that area? [1] is right on the front page of the GHC Trac even though no builds were ran for ~5 months. Perhaps it should be moved out of the way if there's no plan to resume these in the near future.
Does anything specific need doing to get these to run again?
[1]: https://ghc.haskell.org/trac/ghc/wiki/Builder -- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On 26/01/14 03:29, Austin Seipp wrote:
As of right now, Pali's FreeBSD builds seem to be the only nightly that is still consistently running (and thanks to him for that!)
The build infrastructure in its current status is mainly just 'unmaintained'. Furthermore there's not really a good roster of machines that were/were not part of the system AFAIK aside from the old list, and it's unclear what the status of many of those machines are (as you said, many haven't checked in in a while.)
There is much interest in a better nightly infrastructure and people have asked me several times about setting one up on IRC. We have historically had some problems with the nightly infrastructure, mainly things like network disconnectivity or firewalling policies, since most people aren't running dedicated internet facing machines (or even a dedicated machine at all. Firewalls have been a problem for places like MSR from what I understand.)
Why not simply have the clients post the results once a night? If the builds are nightly, is there really any need to have an open daemon listening? From what I can tell from http://darcs.haskell.org/ghcBuilder/builders/ it is simply the matter of building once a day/night and then posting the results in an e-mail to the list and uploading the binaries and test results elsewhere. Could we not simply have a wrapper script around GHC build process that in the end posts all these results to relevant places? The clients could simply have a nightly cron job and it'd be up to the slave owner to keep these builds going as often or as rarely as they want. The only downside is that you guys can't tell the clients precisely when to run but looking at build times, it's only once a day anyway.
Several individual people run Jenkins individually, and I like it, but I'm not sure how well it does when spread across the globe in terms of networking (and realistically builders will look like that, as we can't possibly have a dedicated farm somewhere.) I was also at one point worried about the size of such a tool on systems like ARM machines where resources are at a premium, but in hindsight this looks OK. I'd like any opinions on this if people have deployed things in these highly distributed scenarios.
I have had some ideas for an extremely-minimal nightly build infrastructure that would ideally require minimal setup and let clients have power over choosing how and when to build, but I have yet to find the time to finish the basic implementation to try it.
On Sat, Jan 25, 2014 at 8:22 PM, Mateusz Kowalczyk
wrote: Hi all,
I'd just like to query the status of the nightly builds. Is anything happening in that area? [1] is right on the front page of the GHC Trac even though no builds were ran for ~5 months. Perhaps it should be moved out of the way if there's no plan to resume these in the near future.
Does anything specific need doing to get these to run again?
[1]: https://ghc.haskell.org/trac/ghc/wiki/Builder -- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Mateusz K.

FWIW, this is pretty much what it was going to do. Except it also
needs to host things like publicly accessible binary snapshots, as
people want to use them. And that's a no-go for firewalls or otherwise
non-controllable infrastructure that people may have bots on, so it
must have a central place for the results to be located on
haskell.org. It can then send summarized reports to the list based
e.g. on a cron job (asking bot runners to manage their own emails for
bots is annoying and painful, and doesn't scale nicely for them as we
add more.) And also, of course, there needs to be some aspect of
non-repudiation to the results, so that people know builds and emails
are legitimate (i.e. signed by an GPG pubkey and verified on the
server.)
On Sat, Jan 25, 2014 at 9:50 PM, Mateusz Kowalczyk
On 26/01/14 03:29, Austin Seipp wrote:
As of right now, Pali's FreeBSD builds seem to be the only nightly that is still consistently running (and thanks to him for that!)
The build infrastructure in its current status is mainly just 'unmaintained'. Furthermore there's not really a good roster of machines that were/were not part of the system AFAIK aside from the old list, and it's unclear what the status of many of those machines are (as you said, many haven't checked in in a while.)
There is much interest in a better nightly infrastructure and people have asked me several times about setting one up on IRC. We have historically had some problems with the nightly infrastructure, mainly things like network disconnectivity or firewalling policies, since most people aren't running dedicated internet facing machines (or even a dedicated machine at all. Firewalls have been a problem for places like MSR from what I understand.)
Why not simply have the clients post the results once a night? If the builds are nightly, is there really any need to have an open daemon listening? From what I can tell from http://darcs.haskell.org/ghcBuilder/builders/ it is simply the matter of building once a day/night and then posting the results in an e-mail to the list and uploading the binaries and test results elsewhere. Could we not simply have a wrapper script around GHC build process that in the end posts all these results to relevant places? The clients could simply have a nightly cron job and it'd be up to the slave owner to keep these builds going as often or as rarely as they want. The only downside is that you guys can't tell the clients precisely when to run but looking at build times, it's only once a day anyway.
Several individual people run Jenkins individually, and I like it, but I'm not sure how well it does when spread across the globe in terms of networking (and realistically builders will look like that, as we can't possibly have a dedicated farm somewhere.) I was also at one point worried about the size of such a tool on systems like ARM machines where resources are at a premium, but in hindsight this looks OK. I'd like any opinions on this if people have deployed things in these highly distributed scenarios.
I have had some ideas for an extremely-minimal nightly build infrastructure that would ideally require minimal setup and let clients have power over choosing how and when to build, but I have yet to find the time to finish the basic implementation to try it.
On Sat, Jan 25, 2014 at 8:22 PM, Mateusz Kowalczyk
wrote: Hi all,
I'd just like to query the status of the nightly builds. Is anything happening in that area? [1] is right on the front page of the GHC Trac even though no builds were ran for ~5 months. Perhaps it should be moved out of the way if there's no plan to resume these in the near future.
Does anything specific need doing to get these to run again?
[1]: https://ghc.haskell.org/trac/ghc/wiki/Builder -- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Mateusz K.
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Austin, On 01/26/14 04:29 AM, Austin Seipp wrote:
As of right now, Pali's FreeBSD builds seem to be the only nightly that is still consistently running (and thanks to him for that!)
The build infrastructure in its current status is mainly just 'unmaintained'. Furthermore there's not really a good roster of machines that were/were not part of the system AFAIK aside from the old list, and it's unclear what the status of many of those machines are (as you said, many haven't checked in in a while.)
honestly speaking, last message from Ian was that builder server waits for "abbot" update. That's IIRC. So my i.MX/ARM buildbot and solaris buildbot waits for abbot to be update to connect again.
There is much interest in a better nightly infrastructure and people have asked me several times about setting one up on IRC. We have historically had some problems with the nightly infrastructure, mainly things like network disconnectivity or firewalling policies, since
I got those disconnectivity issue on builder client v2, I've not seen them on v1, but this may be just a coincidence.
Several individual people run Jenkins individually, and I like it, but I'm not sure how well it does when spread across the globe in terms of networking (and realistically builders will look like that, as we can't possibly have a dedicated farm somewhere.) I was also at one point worried about the size of such a tool on systems like ARM machines where resources are at a premium, but in hindsight this looks OK. I'd like any opinions on this if people have deployed things in these highly distributed scenarios.
ARM is all right, at least cortex-Ax boards provides 1GB usually and sometimes even more. Using NFS or attached drive I've been able to perform ghc build as dictated by the builder server in several days (4-5 IIRC). Pandaboard would be a lot faster (2 days IIRC) but is not that stable and I don't have modern cortex-a15 boards here or quad A9, those would be even faster. Anyway, if you do not require build every night than this is doable on one board. If you require better coverage, then more than one board will be needed.
I have had some ideas for an extremely-minimal nightly build infrastructure that would ideally require minimal setup and let clients have power over choosing how and when to build, but I have yet to find the time to finish the basic implementation to try it.
Why to waste your precious time on something which was basically done already several times in the past and what in its last incarnation done by Ian worked quite well? Just please start the venerable builder server and let's see people connect again and buildbots running... Thanks! Karel

Hi, Am Sonntag, den 26.01.2014, 02:22 +0000 schrieb Mateusz Kowalczyk:
I'd just like to query the status of the nightly builds. Is anything happening in that area? [1] is right on the front page of the GHC Trac even though no builds were ran for ~5 months. Perhaps it should be moved out of the way if there's no plan to resume these in the near future.
just to clarify: For what purpose do you want the nightlies? To check whether GHC validates cleanly, to compare performance numbers, or to get hold of up-to-date binary distributions? For the first, I’d really really like to see something that runs before a change enters master, so that non-validating mistakes like http://git.haskell.org/ghc.git/commitdiff/b26e2f92c5c6f77fe361293a128da637e7... (without the corresponding change in http://git.haskell.org/ghc.git/commitdiff/59f491a933ec7380698b776e14c3753c2a...) do not reach master in the first place. I’m happy to help setting up such an infrastructure, including designing the precise workflow. For the second and third, a build farm like the builders would of course be great. I actually once got a Igloo snowboall from Linaro for that purpose, but never finished setting it up properly. So once the builders are going to be revived, I’d like to finally do that. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org

On Sun, Jan 26, 2014 at 12:16 PM, Joachim Breitner
just to clarify: For what purpose do you want the nightlies? To check whether GHC validates cleanly, to compare performance numbers, or to get hold of up-to-date binary distributions?
Well, I run those clients primarily because that is (was?) one of the primary requirements for Tier-1 platforms :-) And this indeed greatly helps me to see if something has gone wrong on FreeBSD -- so I can track down the problems and fix them gradually continuously, therefore birthing a new release becomes a bit easier. But yes, I also feel useful to offer daily snapshots for the interested parties as a side effect.
For the first, I’d really really like to see something that runs before a change enters master
I am afraid that you may not want to pass each change through all the supported platforms before moving it to master. Of course, that is the ideal case, but it adds some operational cost, and can easily frustrate developers who do not have access to the given platform where it fails.
For the second and third, a build farm like the builders would of course be great.
I believe Ian's original project (the builder-server I use) [1] was to have a distributed farm of builders where anybody is allowed to dedicate a machine. Therefore GHC may be built on various platforms while the cost maintenance is shared between the operators of the respective platforms. I think it worked pretty well until the disappearance of the coordinator machine. We also used the binary tarballs produced by the builders for the latest releases -- Ian just set the release flag, waited for the next day, picked the release tarballs and published them, without any further interaction. [1] https://ghc.haskell.org/trac/ghc/wiki/Builder

On 26/01/14 11:16, Joachim Breitner wrote:
Hi,
Am Sonntag, den 26.01.2014, 02:22 +0000 schrieb Mateusz Kowalczyk:
I'd just like to query the status of the nightly builds. Is anything happening in that area? [1] is right on the front page of the GHC Trac even though no builds were ran for ~5 months. Perhaps it should be moved out of the way if there's no plan to resume these in the near future.
just to clarify: For what purpose do you want the nightlies? To check whether GHC validates cleanly, to compare performance numbers, or to get hold of up-to-date binary distributions?
Personally it's to see what validates that day and while we're at it, I don't see the reason to not use this to get the nightly binaries as well. I agree with pretty much everything that Páli said in his reply. If we can get the validate results from other people's machines, at the very least we have a sanity check: does it only fail for me or for everyone else too? I think that if we have a list of platforms with angry red everywhere, accessible to everyone, people are more likely to react to build failures and we're less likely to have e-mails on ghc-devs from people going ‘is it just me or is it failing for everyone?’.
For the first, I’d really really like to see something that runs before a change enters master, so that non-validating mistakes like http://git.haskell.org/ghc.git/commitdiff/b26e2f92c5c6f77fe361293a128da637e7... (without the corresponding change in http://git.haskell.org/ghc.git/commitdiff/59f491a933ec7380698b776e14c3753c2a...) do not reach master in the first place.
I’m happy to help setting up such an infrastructure, including designing the precise workflow.
I think doing a per-commit validate before something enters master would be difficult simply because one would have to wait a long time before their commit is allowed in. Even on the fast boxes, the quick validate from clean checkout seems to take about an hour at best.
For the second and third, a build farm like the builders would of course be great. I actually once got a Igloo snowboall from Linaro for that purpose, but never finished setting it up properly. So once the builders are going to be revived, I’d like to finally do that.
Greetings, Joachim
-- Mateusz K.

On Sun, Jan 26, 2014 at 6:35 PM, Mateusz Kowalczyk
If we can get the validate results from other people's machines, at the very least we have a sanity check: does it only fail for me or for everyone else too? I think that if we have a list of platforms with angry red everywhere, accessible to everyone, people are more likely to react to build failures and we're less likely to have e-mails on ghc-devs from people going ‘is it just me or is it failing for everyone?’.
And for the record, I do agree with this. I think a historic problem is the results have never been public enough to most developers, and unfortunately not everyone is trained to respond to just the emails sent to ghc-builds@haskell.org to diagnose a problem. Seeing a gigantic angry red build failure that blames you directly is likely much easier to for most people as opposed to sorting through emails from bots every morning. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On Sun, Jan 26, 2014 at 5:16 AM, Joachim Breitner
just to clarify: For what purpose do you want the nightlies? To check whether GHC validates cleanly, to compare performance numbers, or to get hold of up-to-date binary distributions?
In practice: all three. Developers want logs to see what went wrong. Users want snapshot consistent distribution of snapshots to test against. Both are legitimate uses that are covered by such infrastructure.
For the first, I’d really really like to see something that runs before a change enters master, so that non-validating mistakes like http://git.haskell.org/ghc.git/commitdiff/b26e2f92c5c6f77fe361293a128da637e7... (without the corresponding change in http://git.haskell.org/ghc.git/commitdiff/59f491a933ec7380698b776e14c3753c2a...) do not reach master in the first place.
I’m happy to help setting up such an infrastructure, including designing the precise workflow.
This is doable, but the question is to what extent? There are literally dozens of build configurations that could break with any given patch, without others breaking: * Profiling could break. Or profiling GHC (but not other smaller things) could break. * Dynamic linking could break. * Rarer configurations could break but only for some cases, e.g. threaded + profiling. Or LLVM + Profiling, or LLVM + dynamic linking, etc etc. * Static linking for GHCi could break on platforms that now use dynamic linking by default (as we saw happen when I broke it.) * GHC may only expose certain faulty behavior at certain optimization levels (both in bootstrapping itself and in the tests - so maybe ./validate looks mostly OK, but -O2 is not.) * Bootstrapping the build with different compilers may break (i.e. an unintentional backwards incompatible change is introduced in the stage1 build) * Any of these could theoretically break depending on things like the host platform. * The testsuite runs 'fast' by default. It would need to run slowly to potentially uncover more problems, but this greatly increases the runtime. * Not all machines are equal, and some will take dramatically longer or shorter amounts of time to build (and subsequently) uncover these problems. In my experience, all of the above are absolutely possible scenarios for something wrong to happen. Also, in practice, a lot of these things either need an incredible amount of cross-communication to fix (between the bot runner and the developer,) or require direct access to the machine in order to debug. Not everyone has that hardware, and not everyone will even be willing to give access (for legitimate reasons - some people have offered to run build bots, but behind corporate infrastructure at places like IBM.) And with the amount of time that many configurations requires, the turnaround time for some things could become incredibly large and frustrating. I think if we were to introduce pre-push validation, the only thing it could reasonably test would be ./validate and nothing else. And even then, e.g. on high-powered ARM platforms, this will still seriously take *hours*, and that's a significantly longer time-to-wait than most people are used to.
For the second and third, a build farm like the builders would of course be great. I actually once got a Igloo snowboall from Linaro for that purpose, but never finished setting it up properly. So once the builders are going to be revived, I’d like to finally do that.
If you're willing to contribute ARM builders, both Ben Gamari and I would be very happy to have you do so (me and him are the only people actively doing lots of ARM work, and frankly, Ben is doing most of it.)
Greetings, Joachim
-- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On Sun, Jan 26, 2014 at 6:57 PM, Austin Seipp
* Profiling could break. Or profiling GHC (but not other smaller things) could break. * Dynamic linking could break. * Rarer configurations could break but only for some cases, e.g. threaded + profiling. Or LLVM + Profiling, or LLVM + dynamic linking, etc etc. * Static linking for GHCi could break on platforms that now use dynamic linking by default (as we saw happen when I broke it.) * GHC may only expose certain faulty behavior at certain optimization levels (both in bootstrapping itself and in the tests - so maybe ./validate looks mostly OK, but -O2 is not.) * Bootstrapping the build with different compilers may break (i.e. an unintentional backwards incompatible change is introduced in the stage1 build) * Any of these could theoretically break depending on things like the host platform. * The testsuite runs 'fast' by default. It would need to run slowly to potentially uncover more problems, but this greatly increases the runtime.
Disregard all this, upon closer inspection I see you only wanted ./validate anyway.* :) * But it still will hurt more when you add in low-powered builders. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Good morning, Am Sonntag, den 26.01.2014, 19:18 -0600 schrieb Austin Seipp:
Disregard all this, upon closer inspection I see you only wanted ./validate anyway.* :)
* But it still will hurt more when you add in low-powered builders.
right, I did not want to highjack the thread with my wishlist. And even then I would not want low-powered builders, but rather one strong, “most typical” setup. I think it would already by a big win if we ensured statically (heh) that every change to master has been validated completely once somewhere. And if for changes like the one I liked (removing dead code), as a developer I don’t have to laboriously enforce this invariant (which I obviously then didn’t do), but can rely on the safety nets of the infrastructure. Greetings, Joachim PS: I’m subscribed to the list, no need to mail me personally. -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org

Guys, I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person b) to create binary snapshots that people can use without having to build GHC from scratch Ian invested quite a bit of effort in a home-grown system to do just that, and I have no idea of how it stacks up against more widely-supported systems. These days GHC HQ is really just ghc-devs. So please go right ahead... if you can converge to a solution and implement it, that'd be great. Thanks Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Joachim Breitner | Sent: 27 January 2014 09:47 | To: ghc-devs@haskell.org | Subject: Re: Nightlies | | Good morning, | | Am Sonntag, den 26.01.2014, 19:18 -0600 schrieb Austin Seipp: | > Disregard all this, upon closer inspection I see you only wanted | > ./validate anyway.* :) | > | > * But it still will hurt more when you add in low-powered builders. | | right, I did not want to highjack the thread with my wishlist. | | And even then I would not want low-powered builders, but rather one | strong, “most typical” setup. I think it would already by a big win if | we ensured statically (heh) that every change to master has been | validated completely once somewhere. And if for changes like the one I | liked (removing dead code), as a developer I don’t have to laboriously | enforce this invariant (which I obviously then didn’t do), but can rely | on the safety nets of the infrastructure. | | Greetings, | Joachim | | PS: I’m subscribed to the list, no need to mail me personally. | | -- | Joachim “nomeata” Breitner | mail@joachim-breitner.de • http://www.joachim-breitner.de/ | Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C | Debian Developer: nomeata@debian.org

Hi, Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones:
I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person
I believe we can do better, so that Austin’s big red message does not have to be written in the first place. Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions: Proposal ~~~~~~~~ Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch "validating/<some id>". One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master. If it does not validate, or if master has changed in between, the branch will be moved to failed/<some id>, and a message is sent to the pushing developer², including a tail of the log and a link to the full log. Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch "master-unchecked" will be moved to master, well, unchecked. Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource). Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.) Is this something you would what, or could live with? If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards. Greetings, Joachim ¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch. PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ... Stronger proposal ~~~~~~~~~~~~~~~~~ Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x "./validate" is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use "git merge --squash" to produce a single commit from their branch. This would make the git history more useful for things like bitsecting. -- Joachim Breitner e-Mail: mail@joachim-breitner.de Homepage: http://www.joachim-breitner.de Jabber-ID: nomeata@joachim-breitner.de

Yay!
I, for one, would probably submit more bugfix patches with this structure in place. I take validating seriously, but my daily schedule often involves switching machines and tasks, and once I press "validate", it's often hours before I actually look at the results... by which time, some other commit may have come through. My need (real or perceived) for an uninterrupted chunk of time to make a patch stops me from doing them, sometimes.
With Joachim's structure, I could spend a half hour doing a quick patch (especially comments!) and push and move on.
Please do it! :)
Thanks!
Richard
PS: Transparency in the git redirects would be nice, of course, but is in no way necessary. We could I suppose add a "push" script to the repo that handles some of the overhead.
On Feb 4, 2014, at 4:41 AM, Joachim Breitner
Hi,
Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones:
I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person
I believe we can do better, so that Austin’s big red message does not have to be written in the first place.
Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions:
Proposal ~~~~~~~~
Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch "validating/<some id>". One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master.
If it does not validate, or if master has changed in between, the branch will be moved to failed/<some id>, and a message is sent to the pushing developer², including a tail of the log and a link to the full log.
Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch "master-unchecked" will be moved to master, well, unchecked.
Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource).
Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.)
Is this something you would what, or could live with?
If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards.
Greetings, Joachim
¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch.
PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ...
Stronger proposal ~~~~~~~~~~~~~~~~~
Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x "./validate" is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use "git merge --squash" to produce a single commit from their branch.
This would make the git history more useful for things like bitsecting.
-- Joachim Breitner e-Mail: mail@joachim-breitner.de Homepage: http://www.joachim-breitner.de Jabber-ID: nomeata@joachim-breitner.de
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 04/02/14 09:41, Joachim Breitner wrote:
Hi,
Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones:
I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person
I believe we can do better, so that Austin’s big red message does not have to be written in the first place.
Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions:
Proposal ~~~~~~~~
Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch "validating/<some id>". One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master.
If it does not validate, or if master has changed in between, the branch will be moved to failed/<some id>, and a message is sent to the pushing developer², including a tail of the log and a link to the full log.
Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch "master-unchecked" will be moved to master, well, unchecked.
Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource).
Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.)
Is this something you would what, or could live with?
If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards.
Greetings, Joachim
¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch.
PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ...
Stronger proposal ~~~~~~~~~~~~~~~~~
Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x "./validate" is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use "git merge --squash" to produce a single commit from their branch.
This would make the git history more useful for things like bitsecting.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
You mention that it's enough for one machine to validate the patch. What if the change was for example to fix something on ARM (where builds take a long time) and has no effect on x86? The x86 machine is likely to come through first and give you the OK, pushing to master something that's effectively not checked. In fact, the fastest slave is likely to come through first every time and if it validates there (for example, I hear that 64-bit Linux is the golden platform for validate), it will be pushed to master. I think the proposal needs an enhancement: ability to specify which platform the commit needs to validate on (i.e. even if i686 comes through first, we'll wait for ARM anyway) before being pushed in. -- Mateusz K.

This actually raises an important question: we don't have any good policy
for making sure validate / etc stays clean on all the platforms. There's
enough variations that it might take an implausible amount of resources to
test all the targets we want to support on every commit. (Eg given how long
builds apparently take on certain platforms like arm, we'd need quite a few
wee arm machines to test arm at every commit. Or maybe not?)
Point being: I agree that support tooling to make sure builds don't stay
broken and don't get broken aren't where it should be. And I agree that
something like you describe should be done. I'm just wondering if we have
the resources to do that correctly.
On Tuesday, February 4, 2014, Mateusz Kowalczyk
On 04/02/14 09:41, Joachim Breitner wrote:
Hi,
Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones:
I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person
I believe we can do better, so that Austin’s big red message does not have to be written in the first place.
Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions:
Proposal ~~~~~~~~
Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch "validating/<some id>". One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master.
If it does not validate, or if master has changed in between, the branch will be moved to failed/<some id>, and a message is sent to the pushing developer², including a tail of the log and a link to the full log.
Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch "master-unchecked" will be moved to master, well, unchecked.
Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource).
Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.)
Is this something you would what, or could live with?
If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards.
Greetings, Joachim
¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch.
PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ...
Stronger proposal ~~~~~~~~~~~~~~~~~
Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x "./validate" is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use "git merge --squash" to produce a single commit from their branch.
This would make the git history more useful for things like bitsecting.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/ghc-devs
You mention that it's enough for one machine to validate the patch. What if the change was for example to fix something on ARM (where builds take a long time) and has no effect on x86? The x86 machine is likely to come through first and give you the OK, pushing to master something that's effectively not checked. In fact, the fastest slave is likely to come through first every time and if it validates there (for example, I hear that 64-bit Linux is the golden platform for validate), it will be pushed to master.
I think the proposal needs an enhancement: ability to specify which platform the commit needs to validate on (i.e. even if i686 comes through first, we'll wait for ARM anyway) before being pushed in.
-- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/ghc-devs

Hi, Am Dienstag, den 04.02.2014, 15:26 +0000 schrieb Mateusz Kowalczyk:
You mention that it's enough for one machine to validate the patch. What if the change was for example to fix something on ARM (where builds take a long time) and has no effect on x86? The x86 machine is likely to come through first and give you the OK, pushing to master something that's effectively not checked. In fact, the fastest slave is likely to come through first every time and if it validates there (for example, I hear that 64-bit Linux is the golden platform for validate), it will be pushed to master.
sorry, I was not clear. There would be _one_ dedicated machine for doing pre-master checks, and its main purpose is to detect obvious mistakes, i.e. broken code, forgotten test output updates etc. It will not replace the build farm that does daily checks on other systems. If someone fixes a problem on an different architecture, I have no doubts that we will have tested this fix. It is the forgotten update of the import list (which will make validate fail for others, independent o of the architecture) that I worry about. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org

I can see advantages here, similar to Richard. I'm just a bit worried about keeping all those branches straight in my head. Who names all these temporary branches? Does someone delete them so they don't accumulate and dominate the branch list? Would need careful documenting, so that new people know exactly what to do. Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Joachim Breitner | Sent: 04 February 2014 09:42 | To: ghc-devs@haskell.org | Subject: Re: Pre-Master checks (Was: Nightlies) | | Hi, | | Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones: | > I want just to say thank you for thinking about this. We badly need | better nightly-builds for GHC, on a variety of platforms | > a) to identify regressions, preferably to the actually commit that | > caused it, so Austin's big red message can go to the right person | | I believe we can do better, so that Austin’s big red message does not | have to be written in the first place. | | Here is what I have in mind, and I’ll volunteer to implement it if | people think it is a good idea and I get the resources/permissions: | | Proposal | ~~~~~~~~ | | Nobody gets to push to master directly. Instead, every push to master | is | diverted¹ to a temporary branch "validating/<some id>". One of our | servers detects the appearance of such a branch and will | * check it out, | * validate it, | * if ok: check if master can still be fast-forward’ed to it, | * if yes: push to master. | | If it does not validate, or if master has changed in between, the | branch | will be moved to failed/<some id>, and a message is sent to the pushing | developer², including a tail of the log and a link to the full log. | | Systems can fail, and maybe nothing validates anymore for reasons not | easily fixable. For that case, a backdoor is available: Pushes to the | branch "master-unchecked" will be moved to master, well, unchecked. | | Benefits: | * It is guaranteed that master has validated at least once somewhere. | I.e. no need to ask on the mailing list “does master validate for | you | right now?” | * It is now ok to do changes that are “obviously correct” (comment | changes, removing dead code, code reformatting) without having | to validate manually, which _saves developer time_ (our most | precious | resource). | | Downsides: | * When two commits are racing for master, one will be rejected for | being a non-fast-forward commit. The user will then have to merge | or rebase and try again. | But: The same would be true if he was validating locally (unless he | does not validate the merge/rebase, in which case we are again where | we don’t want to be: Unvalidated versions in master.) | | | Is this something you would what, or could live with? | | If it is technically feasible (given our hardware constraints, | repository structure and git’s feature) is a different question, which | needs to be discussed afterwards. | | Greetings, | Joachim | | | | ¹ Might not be possible transparently | (http://stackoverflow.com/questions/21362833), but for the sake of | argument and workflow design, assume it was. | ² As an approximation: The committer of the latest patch. | | PS: I’m also considering (but not pushing hard) for a stronger variant | as follows. We do not need to discuss that now and should, if at all, | start the with the proposal above. I’m just adding it to show where | this | is going ... | | Stronger proposal | ~~~~~~~~~~~~~~~~~ | | Every commit in master needs to be validated! | I tend to make sure that all patches on my branch validate individually | (git rebase -i -x "./validate" is a great tool here, you should use it! | ). Contributors who do not want to go through that trouble should then | use "git merge --squash" to produce a single commit from their branch. | | This would make the git history more useful for things like bitsecting. | | | | -- | Joachim Breitner | e-Mail: mail@joachim-breitner.de | Homepage: http://www.joachim-breitner.de | Jabber-ID: nomeata@joachim-breitner.de

one point in the design space (mind you one i'm not going to advocate) is
the testing / merging workflow that the Mozilla folks are doing for rust.
Every patch / pull request gets tested by their build bot before merging.
Does make the commits on master look pretty mess / mergy though.
mind you its a bit tightly coupled to github, but some of the ideas *might*
be a good strawman. Likewise, how does eg LLVM/Clang manage its
testing infrastructure? How does GCC manage it?
On Tue, Feb 4, 2014 at 12:56 PM, Simon Peyton Jones
I can see advantages here, similar to Richard. I'm just a bit worried about keeping all those branches straight in my head.
Who names all these temporary branches? Does someone delete them so they don't accumulate and dominate the branch list?
Would need careful documenting, so that new people know exactly what to do.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Joachim Breitner | Sent: 04 February 2014 09:42 | To: ghc-devs@haskell.org | Subject: Re: Pre-Master checks (Was: Nightlies) | | Hi, | | Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones: | > I want just to say thank you for thinking about this. We badly need | better nightly-builds for GHC, on a variety of platforms | > a) to identify regressions, preferably to the actually commit that | > caused it, so Austin's big red message can go to the right person | | I believe we can do better, so that Austin’s big red message does not | have to be written in the first place. | | Here is what I have in mind, and I’ll volunteer to implement it if | people think it is a good idea and I get the resources/permissions: | | Proposal | ~~~~~~~~ | | Nobody gets to push to master directly. Instead, every push to master | is | diverted¹ to a temporary branch "validating/<some id>". One of our | servers detects the appearance of such a branch and will | * check it out, | * validate it, | * if ok: check if master can still be fast-forward’ed to it, | * if yes: push to master. | | If it does not validate, or if master has changed in between, the | branch | will be moved to failed/<some id>, and a message is sent to the pushing | developer², including a tail of the log and a link to the full log. | | Systems can fail, and maybe nothing validates anymore for reasons not | easily fixable. For that case, a backdoor is available: Pushes to the | branch "master-unchecked" will be moved to master, well, unchecked. | | Benefits: | * It is guaranteed that master has validated at least once somewhere. | I.e. no need to ask on the mailing list “does master validate for | you | right now?” | * It is now ok to do changes that are “obviously correct” (comment | changes, removing dead code, code reformatting) without having | to validate manually, which _saves developer time_ (our most | precious | resource). | | Downsides: | * When two commits are racing for master, one will be rejected for | being a non-fast-forward commit. The user will then have to merge | or rebase and try again. | But: The same would be true if he was validating locally (unless he | does not validate the merge/rebase, in which case we are again where | we don’t want to be: Unvalidated versions in master.) | | | Is this something you would what, or could live with? | | If it is technically feasible (given our hardware constraints, | repository structure and git’s feature) is a different question, which | needs to be discussed afterwards. | | Greetings, | Joachim | | | | ¹ Might not be possible transparently | (http://stackoverflow.com/questions/21362833), but for the sake of | argument and workflow design, assume it was. | ² As an approximation: The committer of the latest patch. | | PS: I’m also considering (but not pushing hard) for a stronger variant | as follows. We do not need to discuss that now and should, if at all, | start the with the proposal above. I’m just adding it to show where | this | is going ... | | Stronger proposal | ~~~~~~~~~~~~~~~~~ | | Every commit in master needs to be validated! | I tend to make sure that all patches on my branch validate individually | (git rebase -i -x "./validate" is a great tool here, you should use it! | ). Contributors who do not want to go through that trouble should then | use "git merge --squash" to produce a single commit from their branch. | | This would make the git history more useful for things like bitsecting. | | | | -- | Joachim Breitner | e-Mail: mail@joachim-breitner.de | Homepage: http://www.joachim-breitner.de | Jabber-ID: nomeata@joachim-breitner.de
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 2/4/14, Carter Schonwald
one point in the design space (mind you one i'm not going to advocate) is the testing / merging workflow that the Mozilla folks are doing for rust. Every patch / pull request gets tested by their build bot before merging. Does make the commits on master look pretty mess / mergy though.
mind you its a bit tightly coupled to github, but some of the ideas *might* be a good strawman. Likewise, how does eg LLVM/Clang manage its testing infrastructure? How does GCC manage it?
LLVM devs need to run 'make check' before committing to the repo. This runs 8000+ tests in about 30 seconds. Then there is the bot infrastructure at http://lab.llvm.org:8011/console , and who breaks a target is alerted by mail. But this applies to all committers whose changes entered into (slow) builds. It is not commit-exact, though better on the fast bots. Cheers, Gabor
On Tue, Feb 4, 2014 at 12:56 PM, Simon Peyton Jones
wrote: I can see advantages here, similar to Richard. I'm just a bit worried about keeping all those branches straight in my head.
Who names all these temporary branches? Does someone delete them so they don't accumulate and dominate the branch list?
Would need careful documenting, so that new people know exactly what to do.
Simon
| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Joachim Breitner | Sent: 04 February 2014 09:42 | To: ghc-devs@haskell.org | Subject: Re: Pre-Master checks (Was: Nightlies) | | Hi, | | Am Dienstag, den 04.02.2014, 08:04 +0000 schrieb Simon Peyton Jones: | > I want just to say thank you for thinking about this. We badly need | better nightly-builds for GHC, on a variety of platforms | > a) to identify regressions, preferably to the actually commit that | > caused it, so Austin's big red message can go to the right person | | I believe we can do better, so that Austin's big red message does not | have to be written in the first place. | | Here is what I have in mind, and I'll volunteer to implement it if | people think it is a good idea and I get the resources/permissions: | | Proposal | ~~~~~~~~ | | Nobody gets to push to master directly. Instead, every push to master | is | diverted¹ to a temporary branch "validating/<some id>". One of our | servers detects the appearance of such a branch and will | * check it out, | * validate it, | * if ok: check if master can still be fast-forward'ed to it, | * if yes: push to master. | | If it does not validate, or if master has changed in between, the | branch | will be moved to failed/<some id>, and a message is sent to the pushing | developer², including a tail of the log and a link to the full log. | | Systems can fail, and maybe nothing validates anymore for reasons not | easily fixable. For that case, a backdoor is available: Pushes to the | branch "master-unchecked" will be moved to master, well, unchecked. | | Benefits: | * It is guaranteed that master has validated at least once somewhere. | I.e. no need to ask on the mailing list "does master validate for | you | right now?" | * It is now ok to do changes that are "obviously correct" (comment | changes, removing dead code, code reformatting) without having | to validate manually, which _saves developer time_ (our most | precious | resource). | | Downsides: | * When two commits are racing for master, one will be rejected for | being a non-fast-forward commit. The user will then have to merge | or rebase and try again. | But: The same would be true if he was validating locally (unless he | does not validate the merge/rebase, in which case we are again where | we don't want to be: Unvalidated versions in master.) | | | Is this something you would what, or could live with? | | If it is technically feasible (given our hardware constraints, | repository structure and git's feature) is a different question, which | needs to be discussed afterwards. | | Greetings, | Joachim | | | | ¹ Might not be possible transparently | (http://stackoverflow.com/questions/21362833), but for the sake of | argument and workflow design, assume it was. | ² As an approximation: The committer of the latest patch. | | PS: I'm also considering (but not pushing hard) for a stronger variant | as follows. We do not need to discuss that now and should, if at all, | start the with the proposal above. I'm just adding it to show where | this | is going ... | | Stronger proposal | ~~~~~~~~~~~~~~~~~ | | Every commit in master needs to be validated! | I tend to make sure that all patches on my branch validate individually | (git rebase -i -x "./validate" is a great tool here, you should use it! | ). Contributors who do not want to go through that trouble should then | use "git merge --squash" to produce a single commit from their branch. | | This would make the git history more useful for things like bitsecting. | | | | -- | Joachim Breitner | e-Mail: mail@joachim-breitner.de | Homepage: http://www.joachim-breitner.de | Jabber-ID: nomeata@joachim-breitner.de
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 04/02/2014 01:41, Joachim Breitner wrote:
Proposal ~~~~~~~~
Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch "validating/<some id>". One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master.
If it does not validate, or if master has changed in between, the branch will be moved to failed/<some id>, and a message is sent to the pushing developer², including a tail of the log and a link to the full log.
The system should try to rebase and re-validate automatically in this case. I think this is pretty important - otherwise pushes can fail for random reasons that are no fault of the developer. Actually I'd even go so far as to say that if the rebase succeeds without conflicts then the push should go ahead. Yes this can introduce bugs, but since a validate takes ~30 minutes at a certain rate of commits it becomes impossible to push anything if the tree must be validated and fast-forwarded to push. Also, bear in mind that since this stands between a developer and getting their changes in, it really must work, all the time. Be careful what you're signing up for :-) My feeling is that as a first step this system should be treated as a validation service: we don't force people to use it if they don't want to, so that we can see how well it works over time and developers can gradually migrate to using it. On that basis I'd be really happy to see it, and possibly to rely on it more over time. Cheers, Simon
Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch "master-unchecked" will be moved to master, well, unchecked.
Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource).
Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.)
Is this something you would what, or could live with?
If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards.
Greetings, Joachim
¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch.
PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ...
Stronger proposal ~~~~~~~~~~~~~~~~~
Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x "./validate" is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use "git merge --squash" to produce a single commit from their branch.
This would make the git history more useful for things like bitsecting.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Hi, Am Samstag, den 08.02.2014, 08:51 -0800 schrieb Simon Marlow:
The system should try to rebase and re-validate automatically in this case. I think this is pretty important - otherwise pushes can fail for random reasons that are no fault of the developer.
Actually I'd even go so far as to say that if the rebase succeeds without conflicts then the push should go ahead. Yes this can introduce bugs, but since a validate takes ~30 minutes at a certain rate of commits it becomes impossible to push anything if the tree must be validated and fast-forwarded to push.
I’d actually prefer that behavior; I just didn’t want to put it in the proposal because people might be much too scared by the prospect of patches entering master that no human eye has seen so far – but after someone suggested the same to me in private, and you now on the list, I guess it would be a viable.
Also, bear in mind that since this stands between a developer and getting their changes in, it really must work, all the time. Be careful what you're signing up for :-) My feeling is that as a first step this system should be treated as a validation service: we don't force people to use it if they don't want to, so that we can see how well it works over time and developers can gradually migrate to using it. On that basis I'd be really happy to see it, and possibly to rely on it more over time.
Correct: If I were to implement this (which has to wait for http://hackage.haskell.org/trac/ghc/ticket/8545 to be reasonably reliable) I’d set it up first as _alternative_ way to push, inviting validate-lazy developers to use it instead of master. And if it works great, we can consider making it mandatory (or at least the default). So there seems to be a consensus here that this is a worthwhile feature. Great! Is there a roadmap for #8545? Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
participants (10)
-
Austin Seipp
-
Carter Schonwald
-
Gabor Greif
-
Joachim Breitner
-
Karel Gardas
-
Mateusz Kowalczyk
-
Páli Gábor János
-
Richard Eisenberg
-
Simon Marlow
-
Simon Peyton Jones