September 2020 - ghc-devs

The future of Phabricator
by Ben Gamari 18 Sep '21

18 Sep '21

TL;DR. For several reasons I think we should consider alternatives to Phabricator. My view is that GitLab seems like the best option. Hello everyone, Over the past year I have been growing increasingly weary of our continued dependence on Phabricator. Without a doubt, its code review interface is the best I have used. However, for a myriad of reasons I am recently questioning whether it is still the best tool for GHC's needs. # The problem There are a number of reasons why I am currently uncertain about Phabricator. For one, at this point we have no options for support in the event that something goes wrong as the company responsible for Phabricator, Phacility, has closed their support channels to non-paying customers. Furthermore, in the past year or two Phacility has been placing their development resources in the parts their customers pay them for, which appear to be much different that the parts that we actively use. For this reason, some parts that we rely on seem oddly half-finished. This concern was recently underscored by some rather unfortunate misfeatures in Harbormaster which resulted in broken CI after the Hadrian merge and now apparent bugs which have made it difficult to migrate to the CircleCI integration we previously prepared. Perhaps most importantly, in our recent development priorities survey our use of Phabricator was the most common complaint by a fair margin, both in case of respondents who have contributed patches and those who have not. On the whole, past contributors and potential future contributors alike have strongly indicated that they want a more Git-like experience. Of course, this wasn't terribly surprising; this is just the most recent case where contributors have made this preference known. Frankly, in a sense it is hard to blame them. The fact that users need to install a PHP tool, Arcanist, to contribute anything but documentation patches has always seemed like unnecessary friction to me and I would be quite happy to be rid of it. Indeed we have had a quite healthy number of GitHub documentation patches since we started accepting them. This makes me thing that there may indeed be potential contributoes that we are leaving on the table. # What to do With Rackspace support ending at the end of year, now may be a good time to consider whether we really want to continue on this road. Phabricator is great at code review but I am less and less certain that it is worth the maintenance uncertainty and potential lost contributors that it costs. Moreover, good alternatives seem closer at-hand than they were when we deployed Phabricator. ## Move to GitHub When people complain about our infrastructure, they often use GitHub as the example of what they would like to see. However, realistically I have a hard time seeing GitHub as a viable option. Its feature set is simply insufficient enough to handle the needs of a larger project like GHC without significant external tooling (as seen in the case of Rust-lang). The concrete reasons have been well-documented in previous discussions but, to summarize, * its review functionality is extremely painful to use with larger patches * rebased patches lead to extreme confusion and lost review comments * it lacks support for pre-receive hooks, which serve as a last line of defense to prevent unintended submodule changes * its inability to deal with external tickets is problematic * there is essentially no possibility that we could eventually migrate GHC's tickets to GitHub's ticket tracker without considerable data loss (much less manage the ticket load that would result), meaning that we would forever be stuck with maintaining Trac. * on a personal note, its search functionality has often left me empty-handed On the whole, these issues seem pretty hard to surmount. ## Move to GitLab In using GitLab for another project over the past months I have been positively surprised by its quality. It handles rebased merge requests far better than GitHub, has functional search, and quite a usable review interface. Furthermore, upstream has been extremely responsive to suggestions for improvement [1]. Even out-of-the-box it seems to be flexible enough to accommodate our needs, supporting integration with external issue trackers, having reasonable release management features, and support for code owners to automate review triaging (replacing much of the functionality of Phabricator's Herald). Finally, other FOSS projects' [3] recent migrations from Phabrictor to GitLab have shown that GitLab-the-company is quite willing to offer help when needed. I took some time this weekend to try setting up a quick GHC instance [2] to play around with. Even after just a few hours of playing around I think the result is surprisingly usable. Out of curiosity I also played around with importing some tickets from Trac (building on Matt Pickering's Trac-to-Maniphest migration tool). With relatively little effort I was even able to get nearly all of our tickets (as of 9 months ago) imported while preserving ticket numbers (although there are naturally a few wrinkles that would need to be worked out). Naturally, I think we should treat the question of ticket tracker migration as an orthogonal one to code review, but it is good to know that this is possible. ## Continue with Phabricator Continuing with Phabricator is of course an option. Its review functionality is great and it has served us reasonably well. However, compared to GitLab and even GitHub of today its features seem less distinguished than they once did. Moreover, the prospect of having to maintain a largely stagnant product with no support strikes me as a slightly dangerous game to play. Working around the issues we have recently encountered has already cost a non-negligible amount of time. # The bottom line If it wasn't clear already, I think that we should strongly consider a move to GitLab. At this point it seems clear that it isn't going to vanish, has a steady pace of development, is featureful, and available. However, these are just my thoughts. What do you think? Cheers, - Ben [1] 11.4 will ship with a file tree view in the code review interface, which I reported (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is one of the Phabricator features I missed the most during review [2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641 [3] The GNOME and freedesktop.org projects have recently migrated, the former from a hodge-podge of self-hosted services and the latter from Phabricator

17 40

New implementation for `ImpredicativeTypes`
by Alejandro Serrano Mena 08 Sep '21

08 Sep '21

Hi all, As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a… you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards, Alejandro

8 23

[ANNOUNCE] Glasgow Haskell Compiler 9.0.1-alpha1 released
by Ben Gamari 06 Dec '20

06 Dec '20

Hello all, The GHC team is very pleased to announce the availability of the first alpha release in the GHC 9.0 series. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/9.0.1-alpha1/ This first alpha comes quite a bit later than expected. However, we have done a significant amount of testing on this pre-release and therefore hope to be able to move forward quickly with a release candidate next week and with a final release in mid-October. GHC 9.0.1 will bring a number of new features: * A first cut of the new LinearTypes language extension [1], allowing use of linear function syntax and linear record fields. * A new bignum library (ghc-bignum), allowing GHC to be more easily used with integer libraries other than GMP. * Improvements in code generation, resulting in considerable performance improvements in some programs. * Improvements in pattern-match checking, allowing more precise detection of redundant cases and reduced compilation time. * Implementation of the "simplified subsumption" proposal [2] simplifying the type system and paving the way for QuickLook impredicativity in GHC 9.2. * Implementation of the QualifiedDo extension [3], allowing more convenient overloading of `do` syntax. * Improvements in compilation time. And many more. See the release notes [4] for a full accounting of the changes in this release. Do note that there are a few things that we expect will change before the final release: * We expect to sort out a notarization workflow for Apple Darwin, allowing our binary distributions to be used on macOS Catalina without hassle. Until this has been sorted out Catalina users can exempt the current macOS binary distribution from the notarization requirement themselves by running `xattr -cr .` on the unpacked tree before running `make install`. * We will likely transition the Alpine binary distribution to be fully statically-linked, providing a convenient, distribution-independent packaging option for Linux users. * We will be merging a robust solution for #17760 which will introduce a new primitive, `keepAlive#`, to the `base` library, subsuming most uses of `touch#`. As always, do test this release and open tickets for whatever issues you encounter. To help with this, we will be publishing a blog post describing use of our new `head.hackage` infrastructure to ease testing of larger projects with Hackage dependencies later this week. Cheers, - Ben [1] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0111-l… [2] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-s… [3] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0216-q… [4] https://downloads.haskell.org/ghc/9.0.1-alpha1/docs/html/users_guide/9.0.1-…

6 10

Using a development snapshot of happy
by Vladislav Zavialov 04 Dec '20

04 Dec '20

Hi ghc-devs, I’m working on the unification of parsers for terms and types, and one of the things I’d really like to make use of is a feature I implemented in ‘happy’ in October 2019 (9 months ago): https://github.com/simonmar/happy/pull/153 It’s been merged upstream, but there has been no release of ‘happy’, despite repeated requests: 1. I asked for a release in December: https://github.com/simonmar/happy/issues/164 2. Ben asked for a release a month ago: https://github.com/simonmar/happy/issues/168 I see two solutions here: a) Find a co-maintainer for ‘happy’ who could make releases more frequently (I understand the current maintainers probably don’t have the time to do it). b) Use a development snapshot of ‘happy’ in GHC Maybe we need to do both, but one reason I’d like to see (b) in particular happen is that I can imagine introducing more features to ‘happy’ for use in GHC, and it’d be nice not to wait for a release every time. For instance, there are some changes I’d like to make to happy/alex in order to implement #17750 So here are two questions I have: 1. Are there any objections to this idea? 2. If not, could someone more familiar with the build process guide me as to how this should be implemented? Do I add ‘happy’ as a submodule and change something in the ./configure script, or is there more to it? Do I need to modify make/hadrian, and if so, then how? Thanks, - Vlad

8 9

How is GHC.Prim.unpackInt8X64# meant to be used?
by Ryan Scott 28 Sep '20

28 Sep '20

GHC imposes an upper bound of 62 on the size of tuples one can use [1]. Currently, this upper bound applies to both boxed and unboxed tuples alike. For example, if you try to create an unboxed tuple of arity 64, then GHC will throw an error: error: A 64-tuple is too large for GHC (max size is 62) Workaround: use nested tuples or define a data type | | f = (#,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,#) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ However, I discovered recently that there are places where GHC *does* use unboxed tuples with arity greater than 62. For example, the GHC.Prim.unpackInt8X64# [2] function returns an unboxed tuple of size 64. I was confused for a while about how this was even possible, but I realized later than GHC only enforces the tuple size limit in expressions and patterns [3]. Simply having a type signature with a large unboxed tuple is fine in and of itself, and since unpackInt8X64# is implemented as a primop, no large unboxed tuples are ever used in the "body" of the function. (Indeed, primops don't have function bodies in the conventional sense.) Other functions in GHC.Prim that use unboxed tuples of arity 64 include unpackWord8X64# [4], packInt8X64# [5], and packWord8X64# [6]. But this makes me wonder: how on earth is it even possible to *use* unpackInt8X64#? The only thing that comes to mind is to pattern-match on the unboxed tuple that it returns, but if you try to do that, you'll get a "64-tuple is too large for GHC" error like shown above. What's more, I can't find any test cases in the GHC test suite that show how unpackInt8X64# or its cousins are supposed to be used. What am I missing? The comments for these functions mention "Warning: this is only available on LLVM", so perhaps someone familiar with the LLVM backend knows what the answer is? For some context, this came up in !4097, where I am attempting to extend the "too large for GHC" error message to include types, in addition to expressions and patterns [7]. However, doing so causes Haddock to error when processing unpackInt8X64# et al. Ryan S. ----- [1] https://gitlab.haskell.org/ghc/ghc/-/blob/a1f34d37b47826e86343e368a5c00f1a4… [2] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:unp… [3] https://gitlab.haskell.org/ghc/ghc/-/issues/18723 [4] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:unp… [5] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:pac… [6] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:pac… [7] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4097#note_301191

7 9

Native Code Generator for AArch64
by Moritz Angermann 27 Sep '20

27 Sep '20

Hi there! As some may know I've been working on a native code generation backend for aarch64[1]. When Ben initially wrote about The state of GHC on ARM[2], I was quite skeptical if a native code generator would really be what we should be doing. And the claim it would take a week or two, might have been underestimating the complexity a bit; but also the time needed to debug crashing programs. The idea of a NCG however intrigued me. I did work on an alternative llvm backend once, so I did know a bit about the code gen backend. I also knew a bit about aarch64 assembly from working on the rts linker for aarch64. So here we are today, with an aarch64 ncg for ghc[3], that has some basic optimizations included, but does not beat the llvm codegen yet in runtime performance. It is however substantially faster than the llvm codegen for compile time performance. I have performed nofib benchmarks for: - full llvm build vs full native build[4] - llvm nofib, with native libraries, vs full native build[5] to discriminate effects of compiling just the nofib programs vs. the impact the libraries have. I've only had time to take a cursory look over the generated assembly for the CSD test, and the llvm codegen seems to be able to produce quite different assembly, thus there seem to be some good optimizations llvm manages to exploit. I'll have to investigate this closer and probably look at the llvm IR we generate and the intermediate optimization steps llvm manages to apply to it, as the llvm assembly doesn't ressemble the ncg assembly much. I plan to look at aarch64/mach-o and performance over the coming weeks. I hope we can get this in for 9.2. Cheers, Moritz -- [1]: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3641 [2]: https://www.haskell.org/ghc/blog/20200515-ghc-on-arm.html [3]: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3641 [4]: https://gist.github.com/9d93454b832b769b5bdb4e731a10c068 [5]: https://gist.github.com/acc4dab7836f1f509716ac398a94d949

3 2

Re: Quick Look has landed
by Alejandro Serrano Mena 25 Sep '20

25 Sep '20

Awesome news! Thanks so much, Simon, for all the programming effort you've put into this :) Alejandro El jue., 24 sept. 2020 a las 22:04, Dimitrios Vytiniotis (< dimitriv(a)gmail.com>) escribió: > Wow this is great news!!! We finally landed something acceptable after 15 > years.... Thanks so much for all the hard work on the implementation side > Alejandro and Simon. > > On Thu, 24 Sep 2020, 21:21 Simon Peyton Jones, <simonpj(a)microsoft.com> > wrote: > >> Dimitrios, Jurriaan, Alejandro >> >> I’m happy to tell you that Quick Look has landed on GHC’s master branch. >> >> It’ll be in GHC 9.2, but is usable already by building HEAD. Good work! >> >> The implementation is very faithful to the paper, which was an invaluable >> guide. >> >> Simon >> >

1 0

Quick Look has landed
by Simon Peyton Jones 24 Sep '20

24 Sep '20

Dimitrios, Jurriaan, Alejandro I'm happy to tell you that Quick Look has landed on GHC's master branch. It'll be in GHC 9.2, but is usable already by building HEAD. Good work! The implementation is very faithful to the paper, which was an invaluable guide. Simon

1 0

GHC 8.10.2 build error: missing .cfi_startproc
by Björn Gohla 22 Sep '20

22 Sep '20

Hello, I'm trying to build ghc-8.10.2 on OpenBSD 6.7, the build finally fails with an error message from the assembler: ===--- building phase 0 gmake --no-print-directory -f ghc.mk phase=0 phase_0_builds gmake[1]: Nothing to be done for 'phase_0_builds'. ===--- building phase 1 gmake --no-print-directory -f ghc.mk phase=1 phase_1_builds gmake[1]: Nothing to be done for 'phase_1_builds'. ===--- building final phase gmake --no-print-directory -f ghc.mk phase=final all "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing -optc-fno-common -optc-Irts/dist/build/./autogen -optc-Werror=unused-but-set-variable -optc-Wno-error=inline -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-DRtsWay=\"rts_v\" -optc-w -static -O0 -H64m -Wall -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts -this-unit-id rts -dcmm-lint -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen -O2 -Wcpp-undef -Wnoncanonical-monad-instances -c rts/StgCRun.c -o rts/dist/build/StgCRun.o rts/StgCRun.c: Assembler messages: rts/StgCRun.c:382:0: error: Error: CFI instruction used without previous .cfi_startproc | 382 | "movq %%rbx,0(%%rax)\n\t" | ^ rts/StgCRun.c:383:0: error: Error: CFI instruction used without previous .cfi_startproc | 383 | "movq %%rbp,8(%%rax)\n\t" | ^ rts/StgCRun.c:384:0: error: Error: CFI instruction used without previous .cfi_startproc | 384 | "movq %%r12,16(%%rax)\n\t" | ^ rts/StgCRun.c:385:0: error: Error: CFI instruction used without previous .cfi_startproc | 385 | "movq %%r13,24(%%rax)\n\t" | ^ rts/StgCRun.c:386:0: error: Error: CFI instruction used without previous .cfi_startproc | 386 | "movq %%r14,32(%%rax)\n\t" | ^ rts/StgCRun.c:387:0: error: Error: CFI instruction used without previous .cfi_startproc | 387 | "movq %%r15,40(%%rax)\n\t" | ^ rts/StgCRun.c:388:0: error: Error: CFI instruction used without previous .cfi_startproc | 388 | #if defined(mingw32_HOST_OS) | ^ rts/StgCRun.c:389:0: error: Error: CFI instruction used without previous .cfi_startproc | 389 | /* | ^ rts/StgCRun.c:390:0: error: Error: CFI instruction used without previous .cfi_startproc | 390 | * Additional callee saved registers on Win64. This must match | ^ `egcc' failed in phase `Assembler'. (Exit code: 1) gmake[1]: *** [rts/ghc.mk:322: rts/dist/build/StgCRun.o] Error 1 gmake: *** [Makefile:128: all] Error 2 Looking in the source file rts/StgCRun.c and reading the GNU assembler manual concerning CFI directives, it seems to me that in the function StgRunIsImplementedInAssembler starting on line 368 the assembler block should perhaps have started with .cfi_startproc and ended with .cfi_endproc . Although I suspect it's perhaps a question of telling GCC not to expect those directives. I'm using gcc-8.3.0 and ghc-8.6.4 to compile. I would be grateful for any insight anyone might have to offer. -- Kindly, Björn Gohla

1 0

Parser depends on DynFlags, depends on Hooks, depends on TcM, DsM, ...
by Sebastian Graf 18 Sep '20

18 Sep '20

Hey Sylvain, In https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3971 I had to fight once more with the transitive dependency set of the parser, the minimality of which is crucial for ghc-lib-parser <https://hackage.haskell.org/package/ghc-lib-parser> and tested by the CountParserDeps test. I discovered that I need to make (parts of) `DsM` abstract, because it is transitively imported from the Parser for example through Parser.y -> Lexer.x -> DynFlags -> Hooks -> {DsM,TcM}. Since you are our mastermind behind the "Tame DynFlags" initiative, I'd like to hear your opinion on where progress can be/is made on that front. I see there is https://gitlab.haskell.org/ghc/ghc/-/issues/10961 and https://gitlab.haskell.org/ghc/ghc/-/issues/11301 which ask a related, but different question: They want a DynFlags-free interface, but I even want a DynFlags-free *module*. Would you say it's reasonable to abstract the definition of `PState` over the `DynFlags` type? I think it's only used for pretty-printing messages, which is one of your specialties (the treatment of DynFlags in there, at least). Anyway, can you think of or perhaps point me to an existing road map on that issue? Thank you! Sebastian

8 16