February 2019 - ghc-devs

How to merge your patch
by Ben Gamari 07 Feb '19

07 Feb '19

tl;dr. Our beloved @marge-bot is behaving much more reliably now thanks to improvements in CI reliability and build batching. To merge an (accepted) merge request simply designate @marge-bot as its assignee. Marge will handle the rest. If things go awry let me know. Hi everyone, As you may have noticed, over the last weeks we have been feeling out how best to leverage our new CI infrastructure, particularly when it comes to merging patches. As mentioned a few weeks ago, we have introduced a bot, Marge, to help us work around some temporary limitations of GitLab's merge workflow. Unfortunately, Marge had her own set of quirks which have taken a while to sort out. In particular, fragile tests tended to result in repeated merge attempts which tended to clog up CI, leading to an avalanche of waiting, build failures, and general despair. However, we have been working to improve this situation in three ways: * provisioning more builder capacity to reduce wait times * fixing or disabling fragile tests to reduce the need for retries * enable Marge's batched merge functionality, reducing the number of builds necessary per merged patch [1] In light of this I just wanted to reiterate the previous guidance on merging patches. If you have a merge request you would like to merge simply do the following: 1. make sure that it has at least one approval. This should happen in the course of code review but do ping if this was forgotten. 2. assign the merge request to @marge-bot using the assignee field in the right-hand sidebar of the merge request page. 3. next time Marge does a batch of merges she will fold in your MR (leaving a helpful comment to let you know) and, if the batch passes CI, merge it. If not she will leave a comment letting you know there was an issue. I will try to step in to sort out the mess when this happens. Do let me know if you have any questions. Cheers, - Ben [1] thanks to Matthew Pickering for picking this up

3 6

Scaling back CI (for now)?
by Matthew Pickering 06 Feb '19

06 Feb '19

Hi all, Everyone has probably noticed that getting anything merged is a real effort at the moment. The main problem is that CI takes in the region of 5-7 hours and then spuriously fails at the end. After 5-7 hours you have to rebase and run CI again and so on. Therefore I propose to run just these four jobs on every MR: validate-x86_64-linux-deb9 validate-x86_64-linux-deb8-hadrian validate-x86_64-windows validate-x86_64-darwin The reasoning is as follows: validate-x86_64-linux-deb9 validate-x86_64-linux-deb8-hadrian These run first and are reliable and finish within an hour. Then we have lots of less reliable, lower priority jobs. Two windows jobs which take forever to run. validate-x86_64-windows validate-x86_64-windows-hadrian One darwin job validate-x86_64-darwin Many more linux jobs validate-x86_64-linux-deb9-unreg validate-x86_64-linux-deb9-integer-simple validate-x86_64-linux-fedora27 validate-x86_64-linux-deb9-llvm validate-x86_64-linux-deb8 validate-i386-linux-deb9 validate-aarch64-linux-deb9 So I don't argue that these are important to test but at the moment they produce too much friction on every commit through a combination of lack of resources and taking too long. Further to this, we really don't need to test fedora27, deb9 and deb8 for every build. When was the last time we broke one of these platforms but not the other, it's rare! So the concrete proposal is to slim back the per commit validation to four jobs. validate-x86_64-linux-deb9 validate-x86_64-linux-deb8-hadrian validate-x86_64-windows validate-x86_64-darwin which will test on the three major platforms. All the other flavours should be run once the commit reaches master. Thoughts? Cheers, Matt

5 7

WIP branches
by Sylvain Henry 06 Feb '19

06 Feb '19

Hi, Every time we fetch the main GHC repository, we get *a lot* of "wip/*" branches. That's a lot of noise, making the bash completion of "git checkout" pretty useless for instance: > git checkout <TAB> zsh: do you wish to see all 945 possibilities (329 lines)? Unless I'm missing something, they seem to be used to: 1) get the CI run on personal branches (e.g. wip/USER/whatever) 2) share code between different people (SVN like) 3) archival of not worth merging but still worth keeping code (cf https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches) Now that we have switched to Gitlab, can we keep the main repository clean of those branches? 1) The CI is run on user forks and on merge requests in Gitlab so we don't need this anymore 2 and 3) Can we have a Gitlab project ("ghc-wip" or something) that isn't protected and dedicated to this? The main project could be protected globally instead of per-branch so that only Ben and Marge could create release branches, merge, etc. Devs using wip branches would only have to add "ghc-wip" as an additional remote repo. Any opinion on this? Thanks, Sylvain

4 3

Cmm code of `id` function referring to `breakpoint`?
by Shao, Cheng 06 Feb '19

06 Feb '19

Hi devs, I just found that the Cmm code of `GHC.Base.id` refers to `breakpoint` in the same module, however, in the Haskell source of `GHC.Base`, the definition of `id` and `breakpoint` are totally unrelated: ``` id :: a -> a id x = x breakpoint :: a -> a breakpoint r = r ``` And here's the pretty-printed Cmm code: ``` base_GHCziBase_id_entry() // [R2] { [] } {offset chwa: // global R2 = R2; call base_GHCziBase_breakpoint_entry(R2) args: 8, res: 0, upd: 8; } base_GHCziBase_breakpoint_entry() // [R2] { [] } {offset chvW: // global R1 = R2; call stg_ap_0_fast(R1) args: 8, res: 0, upd: 8; } ``` This looks suspicious. I'm curious if this is intended behavior of ghc. Regards, Shao Cheng

2 1

Re: WIP branches
by Sylvain Henry 05 Feb '19

05 Feb '19

> What is the advantage of having ghc-wip instead of having all devs just have their own forks? I am all for each dev having its own fork. The ghc-wip repo would be just for devs having an SVN workflow (i.e. several people working with commit rights on the same branch/fork). If no-one uses this workflow or if Gitlab allows fine tuning of permissions on user forks, we may omit the ghc-wip repo altogether. Regards, Sylvain PS: you didn't send your answer to the list, only to me On 05/02/2019 19:44, Richard Eisenberg wrote: > I agree that movement in this direction would be good (though I don't feel the pain from the current mode -- it just seems suboptimal). What is the advantage of having ghc-wip instead of having all devs just have their own forks? > > Thanks, > Richard > >> On Feb 5, 2019, at 11:36 AM, Sylvain Henry <sylvain(a)haskus.fr> wrote: >> >> Hi, >> >> Every time we fetch the main GHC repository, we get *a lot* of "wip/*" branches. That's a lot of noise, making the bash completion of "git checkout" pretty useless for instance: >> >>> git checkout <TAB> >> zsh: do you wish to see all 945 possibilities (329 lines)? >> >> Unless I'm missing something, they seem to be used to: >> 1) get the CI run on personal branches (e.g. wip/USER/whatever) >> 2) share code between different people (SVN like) >> 3) archival of not worth merging but still worth keeping code (cf https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches) >> >> Now that we have switched to Gitlab, can we keep the main repository clean of those branches? >> 1) The CI is run on user forks and on merge requests in Gitlab so we don't need this anymore >> 2 and 3) Can we have a Gitlab project ("ghc-wip" or something) that isn't protected and dedicated to this? The main project could be protected globally instead of per-branch so that only Ben and Marge could create release branches, merge, etc. Devs using wip branches would only have to add "ghc-wip" as an additional remote repo. >> >> Any opinion on this? >> >> Thanks, >> Sylvain >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

2 1

Re: Request for comments on dry-run Trac -> GitLab migration
by Ryan Scott 05 Feb '19

05 Feb '19

There appears to be some impedance mismatches between GitLab's formatting and Trac's formatting in certain places. For example, see the bottom of this issue [1], which has a long, hyperlinked line with the phrase: Icanproducethe`missinginstance`issuewithouthavingtorecompileGHC,whichiswhyIthinkitmightbeindependentofthisbug. Ryan S. ----- [1] https://gitlab.staging.haskell.org/ghc/ghc/issues/16211

2 1

Request for comments on dry-run Trac -> GitLab migration
by Ben Gamari 05 Feb '19

05 Feb '19

TL;DR. Have a look at this [2] test import of GHC's Trac tickets. Tell us what issues you find. Hello everyone, As you likely know, we are currently in the process of consolidating GHC's infrastructure on GitLab [1]. The last step of this process is to migrate our tickets and wiki from Trac. Towards this end I am happy to announce the availability of a test GitLab instance [2] for the community's review. This is a clone of gitlab.haskell.org but with the addition of tickets [3] and GHC wiki [4] content imported from a Trac dump from earlier this month. There are a few known issues: * There are currently around 50 tickets missing from the import; we are working on identifying where they escaped to * The revision history of the Wiki is currently squashed due to performance issues [6] with GitLab's wiki implementation We intend to resolve both of these problems by the time of the final migration. If you find a ticket on the staging instance with problems that are not in the above list please do add the `import problems` label to it so we can have a look. There are a few aesthetic questions that could use community input: * Should the "Trac Metadata" boxes be always visible? * How do we make the Wiki index [7] more usable? Imposing more hierarchy to the page names would likely help but I'm still a bit worried that it will be hard to browse. Please do take a few minutes to peruse the test import and check for mistakes or potential usability issues. GHC's Trac tickets are one of the crown-jewels of the project; we want to make sure we get this migration right as will be living with the result for a long time to come. Cheers, - Ben [1] https://gitlab.haskell.org/ [2] https://gitlab.staging.haskell.org/ghc/ghc/ [3] https://gitlab.staging.haskell.org/ghc/ghc/issues [4] https://gitlab.staging.haskell.org/ghc/ghc/wiki [6] https://gitlab.com/gitlab-org/gitlab-ce/issues/57179 [7] https://ghc.haskell.org/trac/ghc/ticket/16212

5 8

How to teach Hadrian not to build ghctags and haddock
by Roland Senn 04 Feb '19

04 Feb '19

I switched to Hadrian. Normally I hack somewhere in the compiler. So I use "hadrian/build.sh --flavour=devel2 --freeze1" to quickly rebuild GHC after some little changes to the code. It normally builds - library 'ghc' (Stage1, way v) - program 'ghctags' (Stage1) - program 'ghc-bin' (Stage1) - program 'haddock' (Stage1) When hacking GHC, I never use ghctags or haddock. Building ghctags is very fast, 1-2 seconds. However building haddock takes 15 to 20 seconds. Is there a way to teach hadrian not to build haddock and ghctags? Many thanks! Roland

4 4

No email from gitlab
by Matthew Pickering 01 Feb '19

01 Feb '19

Hi all, I stopped getting emails from gitlab about 12 hours ago. Anyone know what is going on? Cheers, Matt

2 1