Re: [GHC DevOps Group] The future of Phabricator

"Boespflug, Mathieu" writes:

Hi Ben,

On Tue, 30 Oct 2018 at 15:34, Ben Gamari wrote:

...

...

Some of the issues I list with GitHub are entirely orthogonal to GitHub's code review tool.

While Rust has shown that large open-source projects can use GitHub, they have also demonstrated that it requires a remarkable amount of automation.

Could you say more about how this would affect GHC? The issues with GitHub that were listed in your original email are all to do with reviews (and to my knowledge addressed by layering reviewable.io on top of GitHub as Manuel says), except a couple. Cribbing from followup emails as well, I end up with the following list:

It occurs to me that I never did sit down to write up my thoughts on reviewable. I tried doing a few reviews with it [1] and indeed it is quite good; in many ways it is comparable to Differential. It's a bit sluggish to load (loading a moderate-sized patch took over 15 seconds in Firefox) but after that it seems quite usable. The comment functionality is great; the ability to leave comments even on lines that were untouched by the patch is noted. However, it really feels like a band-aid, introducing another layer of indirection and a distinct conversation venue all to make up for what are plain deficiencies in GitHub's core product. Moreover, given that using it implies that we also need to buy in to the other deficiencies in GitHub's core product, it's not clear to me why we would go this direction when there are open-source, more featureful alternatives that also have a history of being adoption by large open-source projects. I suspect it will make little difference to contributors; one can authenticate to both with GitHub credentials and the UX is fairly similar. To me, the choice seems fairly clear-cut. [1] Admittedly my these were single-shot reviews and lacked the usual back-and-forth that one typically has during review, but on moderate-size patches, so I think they are fairly representative.

* Poor integration with external issue trackers (or at any rate with Trac), I assume meaning, hard to transactionally close issues upon PR merge and other ticket status updates. * No merge-on-green-CI button.

So keeping the review UX issues aside for a moment, are there other GitHub limitations that you anticipate would warrant automation bots à la Rust-lang?

Ultimately Rust's tools all exist for a reason. Bors works around GitHub's lacking ability to merge-on-CI-pass, Highfive addresses the lack of a flexible code owner notification system, among other things. Both of these are features that we either have already or would like to have. Furthermore, we already recognize that there are holes in our current CI story: relying on cross-compilation to validate non-Linux/amd64 architectures both complicates troubleshooting and requires that we fix issues in GHC's build system that I would rather not tie CI to. GitLab would allow us to potentially continue using CircleCI for "normal" platforms while giving us the ability to easily fill this holes with GitLab's native CI support (which I have used in a few projects; it is both easy to configure and flexible). On the whole, I simply see very few advantages to using GitHub over GitLab; the latter simply seems to me to be a generally superior product. Furthermore, we should remember that no product will be perfect; in light of this it is important to consider a) the openness of the implementation, and b) the responsiveness of the upstream developer. GitLab has the advantage of being open-source with an extremely responsive upstream; this stands in contrast to GitHub, where I have had straightforward pull requests [2] languish for the better part of a year before being merged and deployed. [2] https://github.com/github/markup/pull/925

I'm not too worried about the CI story. The hard part with CircleCI isn't CircleCI, it's getting to a green CircleCI. But once we're there, moving to a green OtherCI shouldn't be much work.

Right, and we are largely already there! Hadrian, Darwin, Fedora, and Debian/amd64 builds are all currently green; i386 is hours from passing (a regression recently snuck in which I pinned down this morning), the LLVM build is pending a fix for a long-standing critical bug (#14251), and we are now two tests away from slow validation being green. The remaining piece is moving differential validation to CircleCI. This was one of the motivations for starting this discussion. Cheers, - Ben