December 2019 - ghc-devs

The future of Phabricator
by Ben Gamari 18 Sep '21

18 Sep '21

TL;DR. For several reasons I think we should consider alternatives to Phabricator. My view is that GitLab seems like the best option. Hello everyone, Over the past year I have been growing increasingly weary of our continued dependence on Phabricator. Without a doubt, its code review interface is the best I have used. However, for a myriad of reasons I am recently questioning whether it is still the best tool for GHC's needs. # The problem There are a number of reasons why I am currently uncertain about Phabricator. For one, at this point we have no options for support in the event that something goes wrong as the company responsible for Phabricator, Phacility, has closed their support channels to non-paying customers. Furthermore, in the past year or two Phacility has been placing their development resources in the parts their customers pay them for, which appear to be much different that the parts that we actively use. For this reason, some parts that we rely on seem oddly half-finished. This concern was recently underscored by some rather unfortunate misfeatures in Harbormaster which resulted in broken CI after the Hadrian merge and now apparent bugs which have made it difficult to migrate to the CircleCI integration we previously prepared. Perhaps most importantly, in our recent development priorities survey our use of Phabricator was the most common complaint by a fair margin, both in case of respondents who have contributed patches and those who have not. On the whole, past contributors and potential future contributors alike have strongly indicated that they want a more Git-like experience. Of course, this wasn't terribly surprising; this is just the most recent case where contributors have made this preference known. Frankly, in a sense it is hard to blame them. The fact that users need to install a PHP tool, Arcanist, to contribute anything but documentation patches has always seemed like unnecessary friction to me and I would be quite happy to be rid of it. Indeed we have had a quite healthy number of GitHub documentation patches since we started accepting them. This makes me thing that there may indeed be potential contributoes that we are leaving on the table. # What to do With Rackspace support ending at the end of year, now may be a good time to consider whether we really want to continue on this road. Phabricator is great at code review but I am less and less certain that it is worth the maintenance uncertainty and potential lost contributors that it costs. Moreover, good alternatives seem closer at-hand than they were when we deployed Phabricator. ## Move to GitHub When people complain about our infrastructure, they often use GitHub as the example of what they would like to see. However, realistically I have a hard time seeing GitHub as a viable option. Its feature set is simply insufficient enough to handle the needs of a larger project like GHC without significant external tooling (as seen in the case of Rust-lang). The concrete reasons have been well-documented in previous discussions but, to summarize, * its review functionality is extremely painful to use with larger patches * rebased patches lead to extreme confusion and lost review comments * it lacks support for pre-receive hooks, which serve as a last line of defense to prevent unintended submodule changes * its inability to deal with external tickets is problematic * there is essentially no possibility that we could eventually migrate GHC's tickets to GitHub's ticket tracker without considerable data loss (much less manage the ticket load that would result), meaning that we would forever be stuck with maintaining Trac. * on a personal note, its search functionality has often left me empty-handed On the whole, these issues seem pretty hard to surmount. ## Move to GitLab In using GitLab for another project over the past months I have been positively surprised by its quality. It handles rebased merge requests far better than GitHub, has functional search, and quite a usable review interface. Furthermore, upstream has been extremely responsive to suggestions for improvement [1]. Even out-of-the-box it seems to be flexible enough to accommodate our needs, supporting integration with external issue trackers, having reasonable release management features, and support for code owners to automate review triaging (replacing much of the functionality of Phabricator's Herald). Finally, other FOSS projects' [3] recent migrations from Phabrictor to GitLab have shown that GitLab-the-company is quite willing to offer help when needed. I took some time this weekend to try setting up a quick GHC instance [2] to play around with. Even after just a few hours of playing around I think the result is surprisingly usable. Out of curiosity I also played around with importing some tickets from Trac (building on Matt Pickering's Trac-to-Maniphest migration tool). With relatively little effort I was even able to get nearly all of our tickets (as of 9 months ago) imported while preserving ticket numbers (although there are naturally a few wrinkles that would need to be worked out). Naturally, I think we should treat the question of ticket tracker migration as an orthogonal one to code review, but it is good to know that this is possible. ## Continue with Phabricator Continuing with Phabricator is of course an option. Its review functionality is great and it has served us reasonably well. However, compared to GitLab and even GitHub of today its features seem less distinguished than they once did. Moreover, the prospect of having to maintain a largely stagnant product with no support strikes me as a slightly dangerous game to play. Working around the issues we have recently encountered has already cost a non-negligible amount of time. # The bottom line If it wasn't clear already, I think that we should strongly consider a move to GitLab. At this point it seems clear that it isn't going to vanish, has a steady pace of development, is featureful, and available. However, these are just my thoughts. What do you think? Cheers, - Ben [1] 11.4 will ship with a file tree view in the code review interface, which I reported (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is one of the Phabricator features I missed the most during review [2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641 [3] The GNOME and freedesktop.org projects have recently migrated, the former from a hodge-podge of self-hosted services and the latter from Phabricator

17 40

New implementation for `ImpredicativeTypes`
by Alejandro Serrano Mena 08 Sep '21

08 Sep '21

Hi all, As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a… you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards, Alejandro

8 23

Residency profiles
by Simon Peyton Jones 23 Jan '20

23 Jan '20

Simon, Ben, Omer As you'll see in comments 55-72 of https://ghc.haskell.org/trac/ghc/ticket/9476, Sebastian has been a bit flummoxed by the task of measure residency profiles; that is, how much data is truly live during execution. A major GC measures that, but we are vulnerable to exactly when it happens (even with -G1) and that can lead to irreproducible results. I think what we want is a way to trigger GC at very regular intervals, after (say) each 10kbytes or 100kbytes or 1Mbyte of allocation. That might be expensive, but we'd get reproducible results. I don't think that is possible right now - see the ticket - but it would be easy enough to do wouldn't it? Just give only 10k or 100k or 1M to the allocator when setting it running again. Would you consider this? Or are we just missing something obvious? Needless to say, we want to do all this with full optimisation on, no cost-centre profiling. Thanks Simon

4 8

Superclasses of type families returning constraints?
by Alexis King 12 Jan '20

12 Jan '20

Hello all, I recently noticed that GHC rejects the following program: type family F a b :: Constraint where F a a = () eq :: F a b => a :~: b eq = Refl This is certainly not shocking, but it is a little unsatisfying: as far as I can tell, accepting this program would be entirely sound. That is, `a ~ b` is morally a “superclass” of `F a b`. In this example the type family is admittedly rather pointless, as `a ~ b` could be used instead, but it is possible to construct more sophisticated examples that cannot be so straightforwardly expressed in other ways. I am therefore curious: has this kind of scenario ever been discussed before? If yes, is there a paper/GitLab issue/email thread somewhere that discusses it? And if no, is there any fundamental reason that GHC does not propagate such information (i.e. it’s incompatible with some aspect of the type system or constraint solver), or is it simply something that has not been explored? (Maybe you think the above program is horrible and *shouldn’t* be accepted even if it were possible, but that is a different question entirely. :)) Thanks, Alexis

3 12

.hie files for pre-installed GHC libraries
by Csaba Hruska 05 Jan '20

05 Jan '20

Hi, Is it planned to include the .hie files of the base and other libraries in the GHC binary download package? Regards, Csaba Hruska

2 1

Fix for ticket 8095 in ghc-8.10.1
by Rene 05 Jan '20

05 Jan '20

Hello, I wondering if the fix for https://gitlab.haskell.org/ghc/ghc/issues/8095 is going to make it into ghc-8.10.1 ? Rene.

2 1

[ANNOUNCE] GHC 8.8.2-rc1 is now available
by Ben Gamari 02 Jan '20

02 Jan '20

Hello everyone, The GHC team is proud to announce the first release candidate of GHC 8.8.2. The source distribution, binary distributions, and documentation are available at https://downloads.haskell.org/~ghc/8.8.2-rc1 Release notes are also available [1]. This release fixes a handful of issues affecting 8.8.1: - A bug (#17088) in the compacting garbage collector resulting in segmentations faults under specific circumstances. Note that this may affect user programs even if they did not explicitly request the compacting GC (using the -c RTS flag) since GHC may fallback to compacting collection during times of high memory pressure. - A code generation bug (#17334) resulting in GHC panics has been fixed. - A bug in the `process` library causing builds using `hsc2hs` to fail non-deterministically on Windows has been fixed (Trac #17480) - A typechecker bug (#12088) resulting in programs being unexpectedly rejected has been fixed. - A bug in the implementation of compact normal forms resulting in segmentation faults in some uses (#17044) has been fixed. - A bug causing GHC to incorrectly complain about incompatible LLVM versions when using LLVM 7.0.1 has been fixed (#16990). As always, if anything looks amiss do let us know. Happy compiling! Cheers, - Ben [1] https://downloads.haskell.org/ghc/8.8.2-rc1/docs/html/users_guide/8.8.2-not…

4 3

[ANNOUNCE] Glasgow Haskell Compiler 8.10.1-alpha2 released
by Ben Gamari 02 Jan '20

02 Jan '20

Hello all, The GHC team is happy to announce the availability of the first alpha release in the GHC 8.10 series. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.1-alpha2/ GHC 8.10.1 will bring a number of new features including: * The new UnliftedNewtypes extension allowing newtypes around unlifted types. * The new StandaloneKindSignatures extension allows users to give top-level kind signatures to type, type family, and class declarations. * A new warning, -Wderiving-defaults, to draw attention to ambiguous deriving clauses * A number of improvements in code generation, including changes * A new GHCi command, :instances, for listing the class instances available for a type. * An upgraded Windows toolchain lifting the MAX_PATH limitation * A new, low-latency garbage collector. * Improved support profiling, including support for sending profiler samples to the eventlog, allowing correlation between the profile and other program events This release is the second alpha of the 8.10 pre-release cycle. The release will be a release candidate in roughly three weeks. If all goes well the final release will be cut roughly two weeks after the candidate, in mid-January. This being an alpha release, there are a few issues that are still outstanding: * The new Alpine Linux binary distribution is not present due to an apparent correctness issue [1]; any help Alpine users can offer here would be greatly appreciated. * We have yet to sort out compliance with Apple's notarization requirements [2] which will be likely be necessary for users of macOS Catalina. However, until this has been sorted out Catalina users can exempt the current macOS binary distribution from the notarization requirement themselves by running `xattr -cr .` on the unpacked tree before running `make install`. * There is one remaining non-regression correctness issue which we plan to fix for the final 8.10.1 but that is not fixed in this release. Please do test this release and let us know if you encounter any other issues. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17508 [2] https://gitlab.haskell.org/ghc/ghc/issues/17418

3 2

Implementation of Type level warnings
by Rowan Goemans 30 Dec '19

30 Dec '19

Hello everyone, A couple of days ago I tried to implement type level warnings [1]. I have got something now that sort of works but issues the same warning multiple times and not on the correct location. It's currently a WIP merge request [2]. I would like some help/pointers to continue the implementation The approach I have taken is to issue the warning once TypeWarning is rewritten. However,this has severe problems since now a warning is issued multiple times and at different locations due to therewritinghappening multiple times. I don't know what the best location is to do this check and issue the warning. Prior to constraint solving isn't a good place since the TypeWarning could be nested deeply in some constraint. Afterwards also isn't doable since then the TypeWarning constraint is gone. An alternative could be to not implement any instance in for TypeWarning similar to TypeError and special case it somehow. Any help is appreciated! [1] https://gitlab.haskell.org/ghc/ghc/issues/17027 [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/2361 Best regards, RowanGoemans

2 1

HEAD bootstrap error macOS
by Gabor Greif 25 Dec '19

25 Dec '19

Recently I started getting following error: ``` [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS -O0 -H64m -Wall -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts -this-unit-id rts -dcmm-lint -DDTRACE -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen -O2 -Wcpp-undef -Wnoncanonical-monad-instances -c rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by GHC version 8.6.5 *** initializing package database: Using binary package database: /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache package flags [] loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d wired-in package ghc-prim mapped to ghc-prim-0.6.1 wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0 wired-in package base mapped to base-4.14.0.0 wired-in package rts mapped to rts wired-in package template-haskell mapped to template-haskell-2.16.0.0 wired-in package ghc mapped to ghc-8.11.0.20191222 !!! initializing package database: finished in 8.51 milliseconds, allocated 8.770 megabytes Created temporary directory: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0 *** systool:cc: *** C Compiler: /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc -x c rts/sm/Scav_thr.c -o /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen -I/Users/ggreif/ghc/libraries/base/include -I/Users/ggreif/ghc/libraries/base/dist-install/build/include -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include -I/Users/ggreif/ghc/libraries/integer-gmp/include -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes -I/Users/ggreif/ghc/includes/dist-derivedconstants/header -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts' -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"' -DTHREADED_RTS In file included from rts/sm/Scav_thr.c:3:0: error: In file included from rts/sm/Scav.c:51:0: error: In file included from rts/sm/GCUtils.h:18:0: error: rts/sm/GCTDecl.h:113:1: error: error: register '%r13' unsuitable for global register variables on this target | 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base); | ^ GCT_REG_DECL(gc_thread*, gct, REG_Base); ^ rts/sm/GCTDecl.h:37:56: error: note: expanded from macro 'GCT_REG_DECL' | 37 | #define GCT_REG_DECL(type,name,reg) register type name REG(reg); | ^ #define GCT_REG_DECL(type,name,reg) register type name REG(reg); ^ includes/stg/MachRegs.h:160:24: error: note: expanded from macro 'REG' | 160 | #define REG(x) __asm__("%" #x) | ^ #define REG(x) __asm__("%" #x) ^ 1 error generated. *** Deleting temp files: Deleting: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_2.rsp Warning: deleting non-existent /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s *** Deleting temp dirs: Deleting: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0 `cc' failed in phase `C Compiler'. (Exit code: 1) ``` My $CC is clang-9 (but I also tried others). Has anybody seen something similar? Cheers nd thanks, Gabor

2 5