Congrats to Ben and everybody involved! This has been a long time coming and I’m super excited to see what it means for GHC in the future! _ara On 27 Dec 2018, at 11:56, Matthew Pickering wrote:

...

To ensure that GHC's git history remains linear ghc/ghc will use GitLab's "fast-forward without a merge commit" merge strategy.

Are merge requests squashed before they are merged?

It seems that the answer by default is no.. https://gitlab.com/gitlab-org/gitlab-ce/issues/27956

and the reason being that upsteam prefers "Convention over Configuration".. https://about.gitlab.com/handbook/product/#convention-over-configuration

However it seems that there is a per-mr option which can be checked if you are diligent to do it for each MR. Some comments indicate that it's possible to implement a webhook to change this behaviour.

Matt

...

On Thu, Dec 27, 2018 at 6:28 AM Ben Gamari wrote:

TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official upstream GHC repository. Various introductory notes are discussed. Let me know if you have any trouble.

Also, please do verify the correctness of the email address associated with your Trac account in the next few weeks. It will be used to map users when we transition Trac tickets to GitLab.

Hello everyone,

I am happy to announce that CI on GHC's GitLab instance [1] is now stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be considered the official upstream repository of GHC.

The rest of this email is meant to serve as a brief introduction and status update. It can also be viewed on the GitLab Wiki [2].

[1] https://gitlab.haskell.org/ [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome

# Getting started

To get started on GitLab you will first want to either create a new account [1] or login with your GitHub credentials [2].

Once you have an account you should add an SSH key [3] so that you can push to your repositories. If you currently have commit rights to GHC notify me (Ben Gamari) of your user name so I can grant you similar rights in GitLab.

[1] https://gitlab.haskell.org/users/sign_in [2] https://gitlab.haskell.org/users/auth/github [3] https://gitlab.haskell.org/profile/keys

# Updating your development environment

You can updated existing working directory (assuming the usual upstream remote name of `origin`) for the new upstream repository location by running the following:

git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git git remote set-url --push origin git@gitlab.haskell.org:ghc/ghc

This is all that should be necessary; a quick `git pull origin master` should verify that everything is working as expected.

# Continuous integration

Continuous integration is now provided by GitLab's native continuous integration infrastructure. We currently test a variety of configurations, including many that neither Phabricator nor CircleCI/Appveyor previously tested (see [1] for an example run):

* With the make build system: * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 * i386/Linux on Debian 9 * aarch64/Linux on Debian 9 (currently broken due to a variety of issues) * x86_64/Windows * x86_64/Darwin * x86_64/Linux on Debian 9 in a few special configurations: * unregisterised (still a bit fragile due to #16085) * integer-simple * building GHC with -fllvm * With Hadrian: * x86_64/Linux on Debian 9 * x86_64/Windows (currently broken due to #15950)

We also run a slightly larger set of jobs on a nightly basis. Note that binary distributions are saved from most builds and are available for download for a few weeks (we may put in place a longer retention policy for some builds in the future).

There are admittedly a few kinks that we are still working out, particularly in the case of Windows (specifically the long build times seen on Windows). If you suspect you are seeing spurious build failures do let us know.

To make the best use of our limited computational resources our CI builds occur in three stages:

* lint: the style and correctness checkers which would previously be run by `arc lint` and `git push`

* build: Debian 9 Linux x86_64 built with make and Hadrian

* full-build: the remaining configurations

If a build fails at an earlier phase no further phases will be run.

[1] https://gitlab.haskell.org/ghc/ghc/pipelines/568

# Structuring your merge request

With the transition to GitLab GHC is moving to a model similar to that used by GitHub. If you have a Differential on Phabricator we will finish review there. However, please post new patches as merge requests on GitLab.

Note that Phabricator and GitLab have quite different models for handling patches. Under Phabricator a Differential is a single patch with no further structure; larger changes can be composed of multiple dependent Differentials.

Under GitLab's model a merge request is a git branch consisting of one or more patches. Larger changes can be handled in one of two ways:

a. a set of dependent merge requests, each of which to be squashed when merged.

b. a single branch with each atomic change made in a single, buildable commit

Due to the difficulty of maintaining dependent merge requests, I would recommend that contributors making larger changes use method (b).

# Submitting your merge request for review

Depending upon whether you have push rights to the GHC repository there are two ways to submit a merge request:

* if you have push access you can push a branch directly to git@gitlab.haskell.org:ghc/ghc.git and open merge request.

In this case please do follow the usual branch naming conventions:

* prefix all branch names with `wip/`

* if you are fixing a particular ticket consider using the name `wip/TNNNN`

* if not you can create a fork using the "Fork" button on the project page [1] and push your branch there

In either case after you have pushed your branch open a merge request against ghc/ghc [2].

[1] https://gitlab.haskell.org/ghc/ghc/forks/new [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new

# Reviewing and merging merge requests

As always, all contributors are encouraged to help review proposed changes. If you are unfamiliar with GitLab's review interface please see GitLab's user documentation [1]. Here are a few quick highlights for those who are familiar with GitHub but haven't yet used GitLab:

* As with GitHub, GitLab supports both inline and out-of-line comments.

* Comments that are actionable (known as "discussions") can be marked as resolved and collapsed.

* Comments can be left on both changed and unchanged lines

* Revisions of a merge request can be viewed and compared using the two drop-down menus at the top of the Changes tab

* Merge requests can require approvals from particular users before considered as mergable

* Merge requests can be placed in "merge when CI passes" state, which will cause merge requests to be merged as soon as they are green

From this point onward all changes to GHC will be merged via GitLab's merge requests facility and must pass CI before being merged. To ensure that GHC's git history remains linear ghc/ghc will use GitLab's "fast-forward without a merge commit" merge strategy. Consequently you will be asked to rebase merge requests which are not fast-forward merges before merging (a convenient "Rebase" button will appear if the rebase can be carried out without conflicts.

[1] https://gitlab.com/help/user/discussions/index.md#discussions

# Status of the Trac migration

Tobias will be continuing work on the Trac ticket migration after a bit of a holiday break. Hopefully by mid-January we will be able to move forward on this part of the migration; I will share more details about this as they develop.

In the meantime, users of Trac should check and possibly update the email address associated with their account [1]. This address will be used to correlate Trac users with their GitLab equivalents so the correctness of this address will be important in preserving attribution information during the Trac import.

[1] https://ghc.haskell.org/trac/ghc/prefs

# Next steps

We are actively working on cleaning up a few remaining issues with CI:

* build times are still very long on Windows, despite the fact that we are only building the `quick` build flavour on that platform; consequently GitLab CI Windows builds do sometimes timeout when we are faced with long build queues.

* we at times run low on disk space on our Windows builder runners, resulting in occasional spurious build failures

* Appveyor builds (which are supposed to supplement the native GitLab builds) rarely seem to finish

GitLab upstream has been incredibly supportive of our transition effort and has expressed interest in assisting us with issues that we encounter. Our current requests can be found on our migration effort's tracking ticket [1]. If you find any additional bugs or workflows that could be improved please do let me know and I can raise the matter with GitLab.

[1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039

# Acknowledgments

I would like to acknowledge several parties for their contributions to this effort:

* Packet.net and Google X for their generous donation of hosting for continuous integration and web hosting

* GitLab and their Open Source program for many productive discussions, their generous support, and the GitLab Ultimate license used by gitlab.haskell.org.

* Davean Scies for his help procuring the hosting services that power our continuous integration.

* Tweag.io for their offer of help and advice

* Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work in setting up the new instance, sorting out the details of the migration, and debugging problems when they arose

Finally, thanks to GHC's contributors for their patience during this transition; it has been a long process which has stolen a significant amount of attention from other matters. My apologies we have been a bit less responsive than usual in code review and ticket triage over the past month or two. Regardless, I am hopeful that this wait will be worthwhile.

# Final thoughts

This is not only a milestone for the GitLab migration but also for GHC itself. For the first time GHC has fully-automated testing, proposed patch CI, and release generation across the full range of Tier 1 configurations it supports, with passing builds in all cases.

We are very excited to begin this next chapter of GHC's development and are looking forward to your feedback on how we can further improve our new infrastructure. Onward and upwards!

Cheers,

- Ben _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

---

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Along the lines of things needing to be adapted, are we moving the ghc-commits mailing list over to GL? _ara

On 27 Dec 2018, at 14:05, Gabor Greif wrote:

Great work, Ben!

Thanks for all the hard work setting up this CI. My hopes are high that our contributions' quality and ease will go up.

There seem to be a few loose ends that need to be wrapped up, like redirect the mirroring of https://github.com/ghc/ghc/ towards GitLab and possibly switch on mirroring for http://git.haskell.org/ghc.git . Or should we just add a redirect? Should pull requests be blocked on the former?

Anyway some readmes need to be adapted for the new world order.

Cheers,

Gabor

...

On 12/27/18, Ara Adkins wrote: Congrats to Ben and everybody involved! This has been a long time coming and I’m super excited to see what it means for GHC in the future!

_ara

On 27 Dec 2018, at 11:56, Matthew Pickering wrote:

...

...

To ensure that GHC's git history remains linear ghc/ghc will use GitLab's "fast-forward without a merge commit" merge strategy.

Are merge requests squashed before they are merged?

It seems that the answer by default is no.. https://gitlab.com/gitlab-org/gitlab-ce/issues/27956

and the reason being that upsteam prefers "Convention over Configuration".. https://about.gitlab.com/handbook/product/#convention-over-configuration

However it seems that there is a per-mr option which can be checked if you are diligent to do it for each MR. Some comments indicate that it's possible to implement a webhook to change this behaviour.

Matt

...

On Thu, Dec 27, 2018 at 6:28 AM Ben Gamari wrote:

TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official upstream GHC repository. Various introductory notes are discussed. Let me know if you have any trouble.

Also, please do verify the correctness of the email address associated with your Trac account in the next few weeks. It will be used to map users when we transition Trac tickets to GitLab.

Hello everyone,

I am happy to announce that CI on GHC's GitLab instance [1] is now stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be considered the official upstream repository of GHC.

The rest of this email is meant to serve as a brief introduction and status update. It can also be viewed on the GitLab Wiki [2].

[1] https://gitlab.haskell.org/ [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome

# Getting started

To get started on GitLab you will first want to either create a new account [1] or login with your GitHub credentials [2].

Once you have an account you should add an SSH key [3] so that you can push to your repositories. If you currently have commit rights to GHC notify me (Ben Gamari) of your user name so I can grant you similar rights in GitLab.

[1] https://gitlab.haskell.org/users/sign_in [2] https://gitlab.haskell.org/users/auth/github [3] https://gitlab.haskell.org/profile/keys

# Updating your development environment

You can updated existing working directory (assuming the usual upstream remote name of `origin`) for the new upstream repository location by running the following:

git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git git remote set-url --push origin git@gitlab.haskell.org:ghc/ghc

This is all that should be necessary; a quick `git pull origin master` should verify that everything is working as expected.

# Continuous integration

Continuous integration is now provided by GitLab's native continuous integration infrastructure. We currently test a variety of configurations, including many that neither Phabricator nor CircleCI/Appveyor previously tested (see [1] for an example run):

* With the make build system: * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 * i386/Linux on Debian 9 * aarch64/Linux on Debian 9 (currently broken due to a variety of issues) * x86_64/Windows * x86_64/Darwin * x86_64/Linux on Debian 9 in a few special configurations: * unregisterised (still a bit fragile due to #16085) * integer-simple * building GHC with -fllvm * With Hadrian: * x86_64/Linux on Debian 9 * x86_64/Windows (currently broken due to #15950)

We also run a slightly larger set of jobs on a nightly basis. Note that binary distributions are saved from most builds and are available for download for a few weeks (we may put in place a longer retention policy for some builds in the future).

There are admittedly a few kinks that we are still working out, particularly in the case of Windows (specifically the long build times seen on Windows). If you suspect you are seeing spurious build failures do let us know.

To make the best use of our limited computational resources our CI builds occur in three stages:

* lint: the style and correctness checkers which would previously be run by `arc lint` and `git push`

* build: Debian 9 Linux x86_64 built with make and Hadrian

* full-build: the remaining configurations

If a build fails at an earlier phase no further phases will be run.

[1] https://gitlab.haskell.org/ghc/ghc/pipelines/568

# Structuring your merge request

With the transition to GitLab GHC is moving to a model similar to that used by GitHub. If you have a Differential on Phabricator we will finish review there. However, please post new patches as merge requests on GitLab.

Note that Phabricator and GitLab have quite different models for handling patches. Under Phabricator a Differential is a single patch with no further structure; larger changes can be composed of multiple dependent Differentials.

Under GitLab's model a merge request is a git branch consisting of one or more patches. Larger changes can be handled in one of two ways:

a. a set of dependent merge requests, each of which to be squashed when merged.

b. a single branch with each atomic change made in a single, buildable commit

Due to the difficulty of maintaining dependent merge requests, I would recommend that contributors making larger changes use method (b).

# Submitting your merge request for review

Depending upon whether you have push rights to the GHC repository there are two ways to submit a merge request:

* if you have push access you can push a branch directly to git@gitlab.haskell.org:ghc/ghc.git and open merge request.

In this case please do follow the usual branch naming conventions:

* prefix all branch names with `wip/`

* if you are fixing a particular ticket consider using the name `wip/TNNNN`

* if not you can create a fork using the "Fork" button on the project page [1] and push your branch there

In either case after you have pushed your branch open a merge request against ghc/ghc [2].

[1] https://gitlab.haskell.org/ghc/ghc/forks/new [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new

# Reviewing and merging merge requests

As always, all contributors are encouraged to help review proposed changes. If you are unfamiliar with GitLab's review interface please see GitLab's user documentation [1]. Here are a few quick highlights for those who are familiar with GitHub but haven't yet used GitLab:

* As with GitHub, GitLab supports both inline and out-of-line comments.

* Comments that are actionable (known as "discussions") can be marked as resolved and collapsed.

* Comments can be left on both changed and unchanged lines

* Revisions of a merge request can be viewed and compared using the two drop-down menus at the top of the Changes tab

* Merge requests can require approvals from particular users before considered as mergable

* Merge requests can be placed in "merge when CI passes" state, which will cause merge requests to be merged as soon as they are green

From this point onward all changes to GHC will be merged via GitLab's merge requests facility and must pass CI before being merged. To ensure that GHC's git history remains linear ghc/ghc will use GitLab's "fast-forward without a merge commit" merge strategy. Consequently you will be asked to rebase merge requests which are not fast-forward merges before merging (a convenient "Rebase" button will appear if the rebase can be carried out without conflicts.

[1] https://gitlab.com/help/user/discussions/index.md#discussions

# Status of the Trac migration

Tobias will be continuing work on the Trac ticket migration after a bit of a holiday break. Hopefully by mid-January we will be able to move forward on this part of the migration; I will share more details about this as they develop.

In the meantime, users of Trac should check and possibly update the email address associated with their account [1]. This address will be used to correlate Trac users with their GitLab equivalents so the correctness of this address will be important in preserving attribution information during the Trac import.

[1] https://ghc.haskell.org/trac/ghc/prefs

# Next steps

We are actively working on cleaning up a few remaining issues with CI:

* build times are still very long on Windows, despite the fact that we are only building the `quick` build flavour on that platform; consequently GitLab CI Windows builds do sometimes timeout when we are faced with long build queues.

* we at times run low on disk space on our Windows builder runners, resulting in occasional spurious build failures

* Appveyor builds (which are supposed to supplement the native GitLab builds) rarely seem to finish

GitLab upstream has been incredibly supportive of our transition effort and has expressed interest in assisting us with issues that we encounter. Our current requests can be found on our migration effort's tracking ticket [1]. If you find any additional bugs or workflows that could be improved please do let me know and I can raise the matter with GitLab.

[1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039

# Acknowledgments

I would like to acknowledge several parties for their contributions to this effort:

* Packet.net and Google X for their generous donation of hosting for continuous integration and web hosting

* GitLab and their Open Source program for many productive discussions, their generous support, and the GitLab Ultimate license used by gitlab.haskell.org.

* Davean Scies for his help procuring the hosting services that power our continuous integration.

* Tweag.io for their offer of help and advice

* Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work in setting up the new instance, sorting out the details of the migration, and debugging problems when they arose

Finally, thanks to GHC's contributors for their patience during this transition; it has been a long process which has stolen a significant amount of attention from other matters. My apologies we have been a bit less responsive than usual in code review and ticket triage over the past month or two. Regardless, I am hopeful that this wait will be worthwhile.

# Final thoughts

This is not only a milestone for the GitLab migration but also for GHC itself. For the first time GHC has fully-automated testing, proposed patch CI, and release generation across the full range of Tier 1 configurations it supports, with passing builds in all cases.

We are very excited to begin this next chapter of GHC's development and are looking forward to your feedback on how we can further improve our new infrastructure. Onward and upwards!

Cheers,

- Ben _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

---

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

---

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Does this mean that the commit message doesn't come from the MR description? If I want to edit the commit message for a description I need to 1. squash the changes locally 2. force push the branch 3. Wait for CI to finish building (even though I just changed the commit message). Matt On Thu, Dec 27, 2018 at 3:11 PM Ben Gamari wrote:

Matthew Pickering writes:

...

...

To ensure that GHC's git history remains linear ghc/ghc will use GitLab's "fast-forward without a merge commit" merge strategy.

Are merge requests squashed before they are merged?

It seems that the answer by default is no.. https://gitlab.com/gitlab-org/gitlab-ce/issues/27956

Indeed there are not. Moreover, in the workflow that I suggest in the email not squashing is the desired behavior since each commit is atomic.

...

and the reason being that upsteam prefers "Convention over Configuration".. https://about.gitlab.com/handbook/product/#convention-over-configuration

However it seems that there is a per-mr option which can be checked if you are diligent to do it for each MR. Some comments indicate that it's possible to implement a webhook to change this behaviour.

I'm not sure there is a reasonable default here; not matter what you choose it will be wrong a good fraction of the time. The current plan is to just ensure that the person who merges an MR considers whether the history it introduces is useful and choose the correct option. I believe this should work fine although I'm happy to reconsider if not.

Cheers,

- Ben

Ben Gamari writes:

Matthew Pickering writes:

...

Does this mean that the commit message doesn't come from the MR description?

Correct. Like GitHub, the commit messages that make it into the version control history are precisely the commit messages of the commits in your branch.

I should note that the ability to specify the commit message when squashing is an open feature request [1]. It's not currently prioritized but I will bring it up with upstream. Cheers, - Ben [1] https://gitlab.com/gitlab-org/gitlab-ee/issues/1510

On December 31, 2018 6:27:33 PM EST, Wolfgang Jeltsch wrote:

Am Donnerstag, den 27.12.2018, 01:27 -0500 schrieb Ben Gamari:

...

In the meantime, users of Trac should check and possibly update the email address associated with their account [1].  This address will be used to correlate Trac users with their GitLab equivalents so the correctness of this address will be important in preserving attribution information during the Trac import.

[1] https://ghc.haskell.org/trac/ghc/prefs

Will this correlation also work if I don’t have a GitLab account at the time of migration but later will get a GitLab account that uses the e- mail address I have on Trac?

When the Trac migration is carried out new GitLab accounts will be created for Trac users who have not yet created an account. You will be able to request a password reset to gain access to this account. Cheers, - Ben

Wolfgang Jeltsch writes:

Am Dienstag, den 01.01.2019, 06:32 -0500 schrieb Ben Gamari:

...

When the Trac migration is carried out new GitLab accounts will be created for Trac users who have not yet created an account. You will be able to request a password reset to gain access to this account.

Hmm. I’d like to have a GitLab account linked to my GitHub account (login via GitHub credentials) rather than an independent GitLab account. What if I create a GitHub-linked GitLab account before the migration? Will this account be used for my Trac stuff in case my GitHub(!) e-mail address is the same as my Trac e-mail address?

I'm pretty certain it will although I have not verified this yet. I'll make a note to check. Cheers, - Ben

Simon Marlow writes:

Congrats Ben and co! This is a huge step forwards.

Thanks!

submodules are still pulling from git.haskell.org, is there an easy way to fix that?

Indeed, Omer's advice is spot on. Cheers, - Ben