- ghc-devs - Haskell.org

Re: Welcome to GitLab!
by Ara Adkins 27 Dec '18

27 Dec '18

Thanks ben! Great to know. CC the list in case anyone else was interested. _ara > On 27 Dec 2018, at 15:06, Ben Gamari <ben(a)smart-cactus.org> wrote: > > Ara Adkins <me(a)ara.io> writes: > >> Along the lines of things needing to be adapted, are we moving the >> ghc-commits mailing list over to GL? >> > Yes, although I haven't quite worked out how best to do that yet. In the > meantime I have inverted the mirroring relationship between > git.haskell.org and gitlab.haskell.org such that the old commit > notification continue to fire. > > Cheers, > > - Ben >

1 0

Re: GitLab Migration (CI heads-up)
by Gabor Greif 23 Dec '18

23 Dec '18

Hi Ben, I was wondering why my pull request (merely to trigger a bit more of CI than what I have at my local disposal) was suddenly failing (1), when it worked in a previous incarnation (2). It turns out that either CI or the entire tree is broken since (3) being the last sound one. Looks like x86-64 is affected on the linux platform only. Darwin seems to work, i386 too, and my PR (2) has been validated for PPC64le recently. So this is just a heads up that there is something fishy with github->CircleCI (at least). Cheers, Gabor (1) https://circleci.com/gh/ghc/ghc/tree/pull/242 (2) https://circleci.com/gh/ghc/ghc/tree/pull/224 (3) https://circleci.com/gh/ghc/ghc/tree/pull/239 On 12/21/18, Ben Gamari <ben(a)smart-cactus.org> wrote: > Ara Adkins <me(a)ara.io> writes: > >> Hey All, >> >> Sorry for my confusion, but I'm a bit unclear as to when we're meant to >> start working against the GHC repo on the gitlab.haskell.org instance. I >> had in mind that the cutover was intended to be the 18th, but going on >> there it still appears as if it's mirrored from git.haskell.org. Can >> somebody clarify this for me? >> > Sorry for the confusion. I have been reluctant to formally announce the > cut-over until CI is green but this has taken a fair bit longer than > anticipated due to the long CI cycle time. Nevertheless people have > started submitting MRs against the GitLab instance and you are more than > welcome to do so. > > As far as the official upstream repository, indeed > gitlab.haskell.org/ghc/ghc is still mirroring git.haskell.org/ghc. This > will change when I formally announce the switchover. > > Cheers, > > - Ben > >

3 7

Linker speed on GHC
by Ben Gamari 22 Dec '18

22 Dec '18

Hi Tamar, I have recently been looking quite a bit at our continuous integration due to the transition to GitLab and have found that the Windows builder is almost always the rate-limiting step. Builds routinely take three hours or more, with much of the time being spent in the bfd linker. I am working to add more builders but ultimately there is only so much that we can do with the resources we have, especially given that Windows VM instances are essentially twice the cost of their Linux counterparts due to Windows licensing fees. I know in the past you have thought about ld performance on Windows. Has there been any development on this front? Is there anything obvious we can do to improve the situation? Cheers, - Ben

1 1

GitLab transition status
by Ben Gamari 22 Dec '18

22 Dec '18

Hello everyone, Over the past week we have been busily pushing ahead with the GitLab transition. While the original plan was to switch the upstream repository to the GitLab on last Tuesday, I have been too preoccupied pushing through the last mile of continuous integration stabilization to write the documentation that would make such a switch possible. Consequently, GitLab's ghc/ghc repository is still mirroring git.haskell.org. Continuous integration via GitLab pipelines is now very nearly green [1]. Once this happens I will send an email with final migration instructions and disable mirroring, signalling that the GitLab repository is the official upstream. After we migrate the upstream all code entering `master` will be merged via merge request, ensuring that no further breakage in CI. It would be greatly appreciated if those with commit rights could refrain from pushing unnecessary changes until the migration is finalized. I have had to fix a few breakages this week, each of which costs a few hours in effort and build time. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/pipelines/444

1 0

Notifications from GitLab
by Matthew Pickering 22 Dec '18

22 Dec '18

I am not receiving any emails from gitlab despite modifying the notification settings to indicate I want to be notified of all new diffs posted. Is anyone else getting notification emails? How can I debug this? Cheers, Matt

3 2

GitLab Migration
by Ara Adkins 21 Dec '18

21 Dec '18

Hey All, Sorry for my confusion, but I'm a bit unclear as to when we're meant to start working against the GHC repo on the gitlab.haskell.org instance. I had in mind that the cutover was intended to be the 18th, but going on there it still appears as if it's mirrored from git.haskell.org. Can somebody clarify this for me? Best, _ara

3 4

References for GHC usage of multiple capabilities
by Artem Pelenitsyn 20 Dec '18

20 Dec '18

Hello devs, I've been working on a short survey devoted to a topic of multithreading inside the GHC compiler and runtime. So far I was mostly looking at the following three papers [1] P. W. Trinder, K. Hammond, J. S. Mattson, Jr., A. S. Partridge, and S. L. Peyton Jones. Gum: A portable parallel implementation of Haskell. PLDI ’96 [2] Tim Harris, Simon Marlow, and Simon Peyton Jones. Haskell on a shared-memory multiprocessor. Haskell ’05 [3] Simon Marlow, Simon Peyton Jones, and Satnam Singh. Runtime support for multicore Haskell. ICFP ’09 Can you suggest any other papers adding insights on how GHC uses multiple capabilities for anything from GC to the implementation of Parallel/Concurrent Haskell? Perhaps, something more recent than the above, but preferably published in academic venues. The survey is meant to be of interest for systems folks. Therefore, I'm not paying so much attention to the programming model and how it is used in programs. -- Best wishes, Artem

4 5

GHC (API?) question: GHC Core for Base libraries
by Bill Hallahan 19 Dec '18

19 Dec '18

Hi, I'm writing a program analyzer that operates on GHC Core. Currently, I'm using the GHC API to get Core from .hs files. I'd like to be able to run this analysis on the standard libraries that come with GHC, which requires getting those as Core. Unfortunately, the build process for these libraries is not entirely straightforward, and relies on a make script. I eventually came up with the plan of writing a GHC plugin, which, rather than performing any optimizations, would simply run the analysis, and then print the results out to a file. I was able to write the plugin successfully, and test it on several files that were not from the base library. Then, i turned to modify the GhcLibHcOpts flag in mk/build.mk, so that the make script would call the plugin. I ended up with the following: GhcLibHcOpts = -package-db /usr/local/lib/ghc-8.0.2/package.conf.d -package-db /Users/BillHallahan/.ghc/x86_64-darwin-8.0.2/package.conf.d -package hplugin -fplugin HPlugin.Plugin -v The two package databases are to get to (1) the GHC API and (2) the plugin ("hplugin") itself. With this I get an error message, which I have not been able to find a way to resolve: "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -H32m -O -Wall -this-unit-id ghc-prim-0.5.0.0 -hide-all-packages -i -ilibraries/ghc-prim/. -ilibraries/ghc-prim/dist-install/build -ilibraries/ghc-prim/dist-install/build/autogen -Ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist-install/build/autogen -Ilibraries/ghc-prim/. -optP-include -optPlibraries/ghc-prim/dist-install/build/autogen/cabal_macros.h -package-id rts -this-unit-id ghc-prim -XHaskell2010 -package-db /usr/local/lib/ghc-8.0.2/package.conf.d -package-db /Users/BillHallahan/.ghc/x86_64-darwin-8.0.2/package.conf.d -package hplugin -fplugin HPlugin.Plugin -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist-install/build -hidir libraries/ghc-prim/dist-install/build -stubdir libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c libraries/ghc-prim/./GHC/Types.hs -o libraries/ghc-prim/dist-install/build/GHC/Types.o -dyno libraries/ghc-prim/dist-install/build/GHC/Types.dyn_o <command line>: not built for interactive use - can't load plugins (HPlugin.Plugin) make[1]: *** [libraries/ghc-prim/dist-install/build/GHC/Types.o] Error 1 make: *** [all] Error 2 So I'm now wondering (an answer to either of these two questions would be helpful): (1) Is this a viable path? That is, is it possible to use a plugin when building Base? If so, does anyone know what I might be doing wrong/what could be causing this error message? (2) Is there some other better/easier way I could get Core representations of the standard libraries? I guess, in theory, it must be possible to compile the standard libraries with the GHC API, but I have no idea how/where to look to figure out how? Thanks, Bill

4 8

GitLab migration status
by Ben Gamari 18 Dec '18

18 Dec '18

TL;DR. Given somewhat slower-than-expected progress on the Trac import I suggest that we implement a pared-down migration on Tuesday. See "The Plan" below. Hello everyone, Over the last few weeks we have been hard at work preparing the migration to GitLab. Currently the following things are ready: * Hosting of GHC's repositories and those of its mirrors have been prepared. * Continuous integration has been configured for GHC. All-in-all the GitLab migration has been quite timely since we were recently notified by CircleCI of billing changes which will soon make it quite difficult for us to continue using their services (see the thread on ghc-devops for details). Thankfully, moving CI to GitLab has been mostly painless and has even enabled us to introduce testing of platforms which were previously inaccessible to us under CircleCI. * The various linters which previously ran via `arc lint` and gitolite post-receive hooks have been ported to CI jobs. * The Trac ticket migration is looking good although there are still a significant number of details which need to be sorted out. * The Wiki migration is in a similar state. Over the past weeks we have been in constant contact with GitLab's FOSS outreach group, who have been quite helpful in getting the eyes of GitLab employees on the issues affecting our transition. Thanks to especially to David Planella for his help so far. Unfortunately, there is one issue in particular [1] which is currently blocking the Trac migration. From my discussions with GitLab's upstream it sounds like it may be possible for them to prioritize a fix in the short-term. However, our aggressive migration timeline is a fair bit faster than GitLab's development cycle and consequently this certainly won't happen before our planned migration on Tuesday. # The Plan Given what remains to be done in the Trac migration I believe it would be a mistake to move ahead with the full migration as planned. However, in the interest of re-gaining functional continuous integration of patches as soon as possible I propose that we move ahead with moving code review on Tuesday. The plan would be as follows: 1. We setup the final gitlab.haskell.org instance tomorrow; since the Trac migration will not be run will need to create new accounts on instance. 2. We begin officially accepting merge requests on this fresh GitLab instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will become GHC's official upstream repository. 3. We allow a week of transition time where new Differentials will continue to be accepted via Phabricator. 4. After this transition period we place Phabricator in read-only mode. 5. When we are confident in the Trac migration (likely after the new year) we move ahead with importing tickets and the wiki Previously I was skeptical of any plan that involved running the Trac migration against a live GitLab instance. However, further reflection I believe such a migration is safe and feasible. Moreover, given the constraints set upon us by the impending CircleCI changes, I think this is our best option to ensure continuity of CI. Thoughts? Cheers, - Ben [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980

6 7

Windows failures
by Simon Peyton Jones 16 Dec '18

16 Dec '18

Tamar Things are getting better. Here's my list of testsuite failure on Windows, based on last night's HEAD. Still 16 failures. Simon Unexpected failures: plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) rts/T10672/T10672_x64.run T10672_x64 [bad exit code] (normal) simplCore/should_compile/T7702.run T7702 [exit code non-0] (normal) th/T10596.run T10596 [exit code non-0] (normal) th/TH_spliceE1.run TH_spliceE1 [exit code non-0] (ext-interp) th/TH_PromotedList.run TH_PromotedList [exit code non-0] (ext-interp) libraries/Win32/tests/T4452.run T4452 [bad exit code] (normal) plugins/plugins07.run plugins07 [bad exit code] (normal) plugins/plugins09.run plugins09 [bad exit code] (normal) plugins/plugins10.run plugins10 [bad stdout] (normal) plugins/plugins11.run plugins11 [bad stdout] (normal) plugins/T10420.run T10420 [bad exit code] (normal) plugins/T10294a.run T10294a [bad exit code] (normal) plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal)

2 1