- ghc-devs - Haskell.org

Scaling back CI (for now)?
by Matthew Pickering 07 Feb '19

07 Feb '19

Hi all, Everyone has probably noticed that getting anything merged is a real effort at the moment. The main problem is that CI takes in the region of 5-7 hours and then spuriously fails at the end. After 5-7 hours you have to rebase and run CI again and so on. Therefore I propose to run just these four jobs on every MR: validate-x86_64-linux-deb9 validate-x86_64-linux-deb8-hadrian validate-x86_64-windows validate-x86_64-darwin The reasoning is as follows: validate-x86_64-linux-deb9 validate-x86_64-linux-deb8-hadrian These run first and are reliable and finish within an hour. Then we have lots of less reliable, lower priority jobs. Two windows jobs which take forever to run. validate-x86_64-windows validate-x86_64-windows-hadrian One darwin job validate-x86_64-darwin Many more linux jobs validate-x86_64-linux-deb9-unreg validate-x86_64-linux-deb9-integer-simple validate-x86_64-linux-fedora27 validate-x86_64-linux-deb9-llvm validate-x86_64-linux-deb8 validate-i386-linux-deb9 validate-aarch64-linux-deb9 So I don't argue that these are important to test but at the moment they produce too much friction on every commit through a combination of lack of resources and taking too long. Further to this, we really don't need to test fedora27, deb9 and deb8 for every build. When was the last time we broke one of these platforms but not the other, it's rare! So the concrete proposal is to slim back the per commit validation to four jobs. validate-x86_64-linux-deb9 validate-x86_64-linux-deb8-hadrian validate-x86_64-windows validate-x86_64-darwin which will test on the three major platforms. All the other flavours should be run once the commit reaches master. Thoughts? Cheers, Matt

5 7

WIP branches
by Sylvain Henry 06 Feb '19

06 Feb '19

Hi, Every time we fetch the main GHC repository, we get *a lot* of "wip/*" branches. That's a lot of noise, making the bash completion of "git checkout" pretty useless for instance: > git checkout <TAB> zsh: do you wish to see all 945 possibilities (329 lines)? Unless I'm missing something, they seem to be used to: 1) get the CI run on personal branches (e.g. wip/USER/whatever) 2) share code between different people (SVN like) 3) archival of not worth merging but still worth keeping code (cf https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches) Now that we have switched to Gitlab, can we keep the main repository clean of those branches? 1) The CI is run on user forks and on merge requests in Gitlab so we don't need this anymore 2 and 3) Can we have a Gitlab project ("ghc-wip" or something) that isn't protected and dedicated to this? The main project could be protected globally instead of per-branch so that only Ben and Marge could create release branches, merge, etc. Devs using wip branches would only have to add "ghc-wip" as an additional remote repo. Any opinion on this? Thanks, Sylvain

4 3

Cmm code of `id` function referring to `breakpoint`?
by Shao, Cheng 06 Feb '19

06 Feb '19

Hi devs, I just found that the Cmm code of `GHC.Base.id` refers to `breakpoint` in the same module, however, in the Haskell source of `GHC.Base`, the definition of `id` and `breakpoint` are totally unrelated: ``` id :: a -> a id x = x breakpoint :: a -> a breakpoint r = r ``` And here's the pretty-printed Cmm code: ``` base_GHCziBase_id_entry() // [R2] { [] } {offset chwa: // global R2 = R2; call base_GHCziBase_breakpoint_entry(R2) args: 8, res: 0, upd: 8; } base_GHCziBase_breakpoint_entry() // [R2] { [] } {offset chvW: // global R1 = R2; call stg_ap_0_fast(R1) args: 8, res: 0, upd: 8; } ``` This looks suspicious. I'm curious if this is intended behavior of ghc. Regards, Shao Cheng

2 1

Re: WIP branches
by Sylvain Henry 05 Feb '19

05 Feb '19

> What is the advantage of having ghc-wip instead of having all devs just have their own forks? I am all for each dev having its own fork. The ghc-wip repo would be just for devs having an SVN workflow (i.e. several people working with commit rights on the same branch/fork). If no-one uses this workflow or if Gitlab allows fine tuning of permissions on user forks, we may omit the ghc-wip repo altogether. Regards, Sylvain PS: you didn't send your answer to the list, only to me On 05/02/2019 19:44, Richard Eisenberg wrote: > I agree that movement in this direction would be good (though I don't feel the pain from the current mode -- it just seems suboptimal). What is the advantage of having ghc-wip instead of having all devs just have their own forks? > > Thanks, > Richard > >> On Feb 5, 2019, at 11:36 AM, Sylvain Henry <sylvain(a)haskus.fr> wrote: >> >> Hi, >> >> Every time we fetch the main GHC repository, we get *a lot* of "wip/*" branches. That's a lot of noise, making the bash completion of "git checkout" pretty useless for instance: >> >>> git checkout <TAB> >> zsh: do you wish to see all 945 possibilities (329 lines)? >> >> Unless I'm missing something, they seem to be used to: >> 1) get the CI run on personal branches (e.g. wip/USER/whatever) >> 2) share code between different people (SVN like) >> 3) archival of not worth merging but still worth keeping code (cf https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches) >> >> Now that we have switched to Gitlab, can we keep the main repository clean of those branches? >> 1) The CI is run on user forks and on merge requests in Gitlab so we don't need this anymore >> 2 and 3) Can we have a Gitlab project ("ghc-wip" or something) that isn't protected and dedicated to this? The main project could be protected globally instead of per-branch so that only Ben and Marge could create release branches, merge, etc. Devs using wip branches would only have to add "ghc-wip" as an additional remote repo. >> >> Any opinion on this? >> >> Thanks, >> Sylvain >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

2 1

Re: Request for comments on dry-run Trac -> GitLab migration
by Ryan Scott 05 Feb '19

05 Feb '19

There appears to be some impedance mismatches between GitLab's formatting and Trac's formatting in certain places. For example, see the bottom of this issue [1], which has a long, hyperlinked line with the phrase: Icanproducethe`missinginstance`issuewithouthavingtorecompileGHC,whichiswhyIthinkitmightbeindependentofthisbug. Ryan S. ----- [1] https://gitlab.staging.haskell.org/ghc/ghc/issues/16211

2 1

Request for comments on dry-run Trac -> GitLab migration
by Ben Gamari 05 Feb '19

05 Feb '19

TL;DR. Have a look at this [2] test import of GHC's Trac tickets. Tell us what issues you find. Hello everyone, As you likely know, we are currently in the process of consolidating GHC's infrastructure on GitLab [1]. The last step of this process is to migrate our tickets and wiki from Trac. Towards this end I am happy to announce the availability of a test GitLab instance [2] for the community's review. This is a clone of gitlab.haskell.org but with the addition of tickets [3] and GHC wiki [4] content imported from a Trac dump from earlier this month. There are a few known issues: * There are currently around 50 tickets missing from the import; we are working on identifying where they escaped to * The revision history of the Wiki is currently squashed due to performance issues [6] with GitLab's wiki implementation We intend to resolve both of these problems by the time of the final migration. If you find a ticket on the staging instance with problems that are not in the above list please do add the `import problems` label to it so we can have a look. There are a few aesthetic questions that could use community input: * Should the "Trac Metadata" boxes be always visible? * How do we make the Wiki index [7] more usable? Imposing more hierarchy to the page names would likely help but I'm still a bit worried that it will be hard to browse. Please do take a few minutes to peruse the test import and check for mistakes or potential usability issues. GHC's Trac tickets are one of the crown-jewels of the project; we want to make sure we get this migration right as will be living with the result for a long time to come. Cheers, - Ben [1] https://gitlab.haskell.org/ [2] https://gitlab.staging.haskell.org/ghc/ghc/ [3] https://gitlab.staging.haskell.org/ghc/ghc/issues [4] https://gitlab.staging.haskell.org/ghc/ghc/wiki [6] https://gitlab.com/gitlab-org/gitlab-ce/issues/57179 [7] https://ghc.haskell.org/trac/ghc/ticket/16212

5 8

How to teach Hadrian not to build ghctags and haddock
by Roland Senn 04 Feb '19

04 Feb '19

I switched to Hadrian. Normally I hack somewhere in the compiler. So I use "hadrian/build.sh --flavour=devel2 --freeze1" to quickly rebuild GHC after some little changes to the code. It normally builds - library 'ghc' (Stage1, way v) - program 'ghctags' (Stage1) - program 'ghc-bin' (Stage1) - program 'haddock' (Stage1) When hacking GHC, I never use ghctags or haddock. Building ghctags is very fast, 1-2 seconds. However building haddock takes 15 to 20 seconds. Is there a way to teach hadrian not to build haddock and ghctags? Many thanks! Roland

4 4

No email from gitlab
by Matthew Pickering 01 Feb '19

01 Feb '19

Hi all, I stopped getting emails from gitlab about 12 hours ago. Anyone know what is going on? Cheers, Matt

2 1

Thoughts on the Contributing page
by Ben Gamari 31 Jan '19

31 Jan '19

For those following along at home: David has been looking at revising our contributor documentation. He has started consolidating a variety of relevant content on the Contributing page of the Wiki [1]. Below are my thoughts; feel free to chime in with your own. David, The contributing page is looking quite good. However, I do wonder whether we could reduce the link "fan-out" a bit more: it still rather feels like a collection of links with no clear "beginning". The "Newcomers to GHC" section is a great start but I see two potential issues: * we don't clearly articulate the precise steps that a newcomer will need to take * the first thing we mention are four links which, while useful, constitute a significant volume of reading for a newcomer. I fear we may lose people at this point. Regarding the second point, I think that WorkingConventions/FixingBugs is a good model in that it clearly lists a series of concrete steps that the contributor should take. Admittedly, I think some of the detail should be dropped or moved (e.g. the mention of setting git's user.name variable is likely unnecessary in 2019). I think that something similar to this list should be the first thing one sees when they reach the contributing page. Ideally the "typical" case of a new contributor wanting to submit their first patch would be able to gather everything they need to get started in one or two screenfuls of text. After that prose can come additional links to more in-depth documentation. Other various issues I noticed: * We link to wiki:WorkingConventions from a variety of places (including wiki:Contributing) but it is now just a link to wiki:Contributing. * wiki:WorkingConventions/FixingBugs still has references to Phabricator. In general we should start culling such references. * wiki:WorkingConventions/FixingBugs also has references to Trac documentation. We should try to replace these with the relevant GitLab documentation when we migrate. * wiki:WorkingConventions/FixingBugs should be updated to reflect that GitLab CI is now the source of truth for validation. * I don't know exactly how this should look but I think we need to do a better job of concisely stating what we expect of contributions. That is: * commit messages should be readable and discuss what the patch does. "Fixes #NNNN" is not an adequate commit message. * changes should be commented where necessary (my usual rule of thumb is "write the comment you would have liked to read when you started writing your patch"). We should mention the Note convention. * commit messages and comments should refer to ticket numbers where appropriate (e.g. using usual #NNNN syntax; people often get this wrong). * commits should be either squashed or logically distinct, individually buildable changes. * changes to most of `base` need to go through the CLC (this may be optional as defining what set of `base` this applies to is a bit tricky; in the interest of keeping things concise we may be better off handling this personally on a case-by-case basis) * code changes should conform to GHC's (somewhat fluid, for better or worse) code style (https://ghc.haskell.org/trac/ghc/wiki/Commentary/CodingStyle) * thought should be given to testing Having a document which succinctly summarised these expectations as we could easily refer to it during code review. Even better, we could excerpt it as a checklist in our merge request description template (I have put up an initial attempt at this as !149). Anyways, there is more to be said here but this email is getting a bit long so let's leave it for future discussions. Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/wiki/Contributing

5 13

RE: Thoughts on the Contributing page
by Andrey Mokhov 30 Jan '19

30 Jan '19

Andreas, Alp: > > - On windows build.bat defaults to stack which I think has never > > worked on my box. > Andrey? (cc'd him) Actually `build.bat` calls the Cabal-based build script. This was a relatively recent change, and we plan to stick to it in the long term, while still providing an alternative `build.stack.bat`. > There are also a few quality of life issues like ctrl+c not canceling > the build on windows. > Which I hope will be resolved at some point but not sure if these > should be showstoppers. Indeed. I hope it's not difficult to fix this, but I'm not sure where to start. Any suggestions are very welcome. Cheers, Andrey

2 1