- ghc-devs - Haskell.org

can't get raw logs -- server down?
by Richard Eisenberg 24 Dec '20

24 Dec '20

Hi devs, I have a failing CI run at https://gitlab.haskell.org/ghc/ghc/-/jobs/533674 <https://gitlab.haskell.org/ghc/ghc/-/jobs/533674>. I can't repro the problem locally, so I wanted to see the raw log. But clicking the raw log button (leading to https://gitlab.haskell.org/ghc/ghc/-/jobs/533674/raw <https://gitlab.haskell.org/ghc/ghc/-/jobs/533674/raw>) gets nothing. It looks like a server is down. Help? Thanks! Richard

2 2

Troubles when building ghc
by Neophytos Michael 24 Dec '20

24 Dec '20

I am having some trouble building ghc on ubuntu. I have ghc 8.8.4 installed on the machine. I pulled from the main repository and followed the instructions here: https://gitlab.haskell.org/ghc/ghc/-/wikis/building/hadrian Building with: ./hadrian/build -j it builds for a long time until it finally fails with the message below. I would appreciate some advice on how to fix this. Thank you, Neo =========================== Exit code: 1 Stderr and Stdout: _build/stage1/compiler/build/GHC/Parser/Lexer.hs:3495:3: error: Variable not in scope: int16ToInt# :: Int# -> Int# | 3495 | int16ToInt# | ^^^^^^^^^^^ _build/stage1/compiler/build/GHC/Parser/Lexer.hs:3520:3: error: Variable not in scope: int32ToInt# :: Int# -> Int# | 3520 | int32ToInt# | ^^^^^^^^^^^ =========================== On that line (in Lexer.hs) we see: #if __GLASGOW_HASKELL__ >= 901 int16ToInt# #endif (indexInt16OffAddr# arr off) #endif so we are in the "then" clause and somehow "int16ToInt#" is nowhere to be found. -- Neo

3 3

Fat interface files?
by Ben Gamari 21 Dec '20

21 Dec '20

Hi Edward, While chatting with the ghc-ide folks recently I realized that it would be useful to be able to preserve Core such that compilation can be restarted (e.g. to be pushed down the bytecode pipeline to evaluate TH splices). As I recall this is precisely what you implemented in your "fat interface file" work. Do you recall what the state of this work was? Do you have a branch with last-known-good work? Do you recall any tricky questions that remained outstanding? Cheers, - Ben

5 6

[ANNOUNCE] Glasgow Haskell Compiler 8.10.3 released
by Ben Gamari 20 Dec '20

20 Dec '20

Hello all, The GHC team is happy to announce the release of GHC 8.10.3. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.3/ GHC 8.10.3 fixes a number of issues in present in GHC 8.10.2 including: * Numerous stability improves on Windows * More robust support for architectures with weak memory ordering guarantees (e.g. modern ARM hardware). * GHC can now split dynamic objects to accomodate macOS' RPATH size limitation when building large projects (#14444) * Several correctness bugs in the new low-latency garbage collector * Many, many other bug-fixes Note that at the moment we still require that macOS Catalina users exempt the binary distribution from the notarization requirement by running `xattr -cr .` on the unpacked tree before running `make install`. This situation will hopefully be improved for GHC 9.0.1 with the resolution of #17418 [1]. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17418

3 3

Status and GHC's treatment of Haddock
by Ben Gamari 16 Dec '20

16 Dec '20

Hi all, Sadly `master` has been broken for much of the day. The causal-chain is roughly the following: 1. Last week a haddock merge-request was prematurely merged by a haddock developer, despite not being compilable, to the `ghc-head` branch. 2. marge-bot failed during a merge on Sunday 3. On Monday I finished merging the half-merged batch manually, not aware of the inconsistent submodule state. However, while doing so some unintended changes crept in, relying on an unmerged haddock commit. 4. in an attempt to fix the inconsistent submodule state another developer merged the unmerged commit 5. I notice the unintentional commits on `master` and start to back them out 6. I note that even after backing out the unintentional commits the tree *still* doesn't build due to haddock 7. I perform a fair amount of head-scratching trying to figure out what is going on; eventually I find the non-building commit. 8. I eventually sort out what needs to be backed-out and in what order I take a few lessons from this: * manual merges, no matter how "straightforward", are always a vector for mistakes. I should have triple-checked my work before pushing manually pushing the half-merged branch. * haddock submodule changes going in prematurely come with a very high cost, especially if they don't build. It turns out I had run into the non-compilable haddock commit a few times last week but had chalked it up to in-flight changes. On the whole, I am really starting to wonder whether we can't improve our model for dealing with haddock; it is currently quite easy for haddock and GHC to enter an inconsistent state. Moreover, the fact that `haddock`'s `ghc-head` branch runs ahead of `master` means that we won't realize the problem until it's too late. One small improvement would be to move Haddock to GitLab. This would at least ensure that there is a clear connection between haddock MRs and their corresponding GHC changes. I have suggested this to Alex. Beyond this, it would be great if someone could step up to finish the hi-haddock change [1]. I suspect this refactoring has the potential to greatly improve this situation: the interface file AST tends to be more stable that the Core AST and has significantly less surface area. After this is in place Haddock could then use either its own AST or a frozen HsSyn AST from GHC (faciliated by TTG) internally, greatly reducing the coupling between Haddock and GHC. Any takers? Cheers, - Ben

1 0

Nested constructed product results?
by Alexis King 15 Dec '20

15 Dec '20

Hi all, I spent some time today looking into the performance of a program involving a parser type that looks something like this: data AnnotatedParser a = AnnotatedParser { annotation :: Annotation , parser :: String -> (Maybe a, String) } The `Annotation` records metadata about the structure of an `AnnotatedParser` that can be accessed statically (that is, without having to run the parser on some input). `AnnotatedParser`s are built from various primitive constructors and composed using various combinators. These combinators end up looking something like this: (<+>) :: AnnotatedParser a -> AnnotatedParser b -> AnnotatedParser (a, b) AnnotatedParser ann1 f <+> AnnotatedParser ann2 g = AnnotatedParser { annotation = Seq ann1 ann2 , parser = \s1 -> let !(a, s2) = f s1 !(b, s3) = g s2 in ((,) <$> a <*> b, s3) } Use of these combinators leads to the construction and subsequent case analysis of numerous `AnnotatedParser` closures. Happily, constructed product result[1] analysis kicks in and rewrites such combinators to cut down on the needless boxing, leading to worker/wrapper splits like this: $w<+> :: Annotation -> (String -> (Maybe a, String)) -> Annotation -> (String -> (Maybe b, String)) -> (# Annotation, String -> (Maybe (a, b), String) #) $w<+> ann1 f ann2 g = (# Seq ann1 ann2 , \s1 -> let !(a, s2) = f s1 !(b, s3) = g s2 in ((,) <$> a <*> b, s3) #) <+> :: AnnotatedParser a -> AnnotatedParser b -> AnnotatedParser (a, b) <+> (AnnotatedParser ann1 f) (AnnotatedParser ann2 g) = case $w<+> ann1 f ann2 g of (# a, b #) -> AnnotatedParser a b {-# INLINE <+> #-} This is great, and it cuts down on allocation significantly, but there is still something unsatisfying about it: the `parser` function inside the record is not affected by CPR! This is a shame, because essentially all use sites immediately deconstruct the pair, making it a prime candidate for unboxing. Ideally, we’d like to get this, instead: $w<+> :: Annotation -> (String -> (Maybe a, String)) -> Annotation -> (String -> (Maybe b, String)) -> (# Annotation, String -> (# Maybe (a, b), String #) #) $w<+> ann1 f ann2 g = (# Seq ann1 ann2 , \s1 -> let !(a, s2) = f s1 !(b, s3) = g s2 in (# (,) <$> a <*> b, s3 #) #) In practice, little combinators like `$w<+>` are marked INLINE, so `f` and `g` are usually known rather than unknown calls. This nested CPR transformation would allow the tuple construction to fuse with the tuple deconstruction, eliminating quite a lot of unnecessary boxing/unboxing. Unfortunately, it seems as though GHC’s implementation of CPR is entirely first-order: although function arguments are given rich demand signatures, results are only described one level deep. But as the above example hopefully illustrates, that’s leaving significant optimization opportunities on the table! Hence, my questions: 1. Has this notion of “nested CPR” been explored at all before? 2. Does such an extension to CPR sound worth its weight? I peeked a little at GHC.Types.Cpr and GHC.Core.Opt.CprAnal, and it seems quite manageable to me, but I haven’t actually looked into an implementation attempt just yet. I’m mostly interested in whether others have thought about something like this and/or run into similar issues in the past, or if this is really an unusual construction. Thanks, Alexis [1]: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/cpr.pdf

2 1

[ANNOUNCEMENT] Change in MR landing procedure
by Ben Gamari 10 Dec '20

10 Dec '20

tl;dr. On Thursday 10 Dec, Marge-bot will be relocated to a retirement facility. Her duties will henceforth be carried out by GitLab's Merge Train functionality. Hi all, When GHC moved to GitLab two years ago the platform lacked a critical feature: the ability to manage a queue of merge requests slated to be merged. For the past two years we have been using @marge-bot for this functionality. While Marge has served us well, she also introduces her fair share of failure modes. In the meantime GitLab has grown proper support for our use-case in the form of the "merge train" mechanism [1]. In light of Marge's growing intransigence [2], I will grant her a well-earned retirement and begin using merge trains for merging MRs starting on Thursday. As a result, the workflow for adding a merge request to the merge queue will become a bit more straightforward: Instead of setting the MR's "assignee" to @marge-bot rather push the blue "Start/add to merge train" button. This adds the merge request to the merge queue. This is documented in the Wiki [3]. Apart from freeing us from the need to maintain Marge, this change also enables us to use GitLab's branch-squashing functionality, which Marge did not support. Do let me know if you have any questions or concerns. So long, Marge, and thanks for all of the commits. Cheers, - Ben [1] https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_… [2] https://gitlab.haskell.org/ghc/ghc/-/issues/19034 [3] https://gitlab.haskell.org/ghc/ghc/-/wikis/Contributing-a-Patch

2 2

Help lead Haskell: Executive Director sought for Haskell Foundation
by Richard Eisenberg 08 Dec '20

08 Dec '20

Posting on behalf of the Haskell Foundation Working Group. Please forward widely! -------- The Haskell Foundation is seeking an Executive Director. Please find the job description here: https://haskell.foundation/ <https://haskell.foundation/executive-director>ed-job-description <https://haskell.foundation/ed-job-description/> The Haskell Foundation (HF) is an independent, non-profit organization dedicated to driving Haskell adoption and open-source development. HF seeks a full-time Executive Director (ED) to lead and develop the organization going forward. The ED is responsible for furthering the HF’s mission and vision, ensuring that resources are in place to accomplish its goals. The HF is supported by an active group of volunteers, and the ED will be a key focal point for that community, in addition to hiring support staff. This is a salaried position. Salary will be commensurate with the experience, qualities, and location of the individual, and will also reflect the Foundation’s status as a non-profit organisation funded by donations.

1 0

[ANNOUNCE] Glasgow Haskell Compiler 9.0.1-alpha1 released
by Ben Gamari 07 Dec '20

07 Dec '20

Hello all, The GHC team is very pleased to announce the availability of the first alpha release in the GHC 9.0 series. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/9.0.1-alpha1/ This first alpha comes quite a bit later than expected. However, we have done a significant amount of testing on this pre-release and therefore hope to be able to move forward quickly with a release candidate next week and with a final release in mid-October. GHC 9.0.1 will bring a number of new features: * A first cut of the new LinearTypes language extension [1], allowing use of linear function syntax and linear record fields. * A new bignum library (ghc-bignum), allowing GHC to be more easily used with integer libraries other than GMP. * Improvements in code generation, resulting in considerable performance improvements in some programs. * Improvements in pattern-match checking, allowing more precise detection of redundant cases and reduced compilation time. * Implementation of the "simplified subsumption" proposal [2] simplifying the type system and paving the way for QuickLook impredicativity in GHC 9.2. * Implementation of the QualifiedDo extension [3], allowing more convenient overloading of `do` syntax. * Improvements in compilation time. And many more. See the release notes [4] for a full accounting of the changes in this release. Do note that there are a few things that we expect will change before the final release: * We expect to sort out a notarization workflow for Apple Darwin, allowing our binary distributions to be used on macOS Catalina without hassle. Until this has been sorted out Catalina users can exempt the current macOS binary distribution from the notarization requirement themselves by running `xattr -cr .` on the unpacked tree before running `make install`. * We will likely transition the Alpine binary distribution to be fully statically-linked, providing a convenient, distribution-independent packaging option for Linux users. * We will be merging a robust solution for #17760 which will introduce a new primitive, `keepAlive#`, to the `base` library, subsuming most uses of `touch#`. As always, do test this release and open tickets for whatever issues you encounter. To help with this, we will be publishing a blog post describing use of our new `head.hackage` infrastructure to ease testing of larger projects with Hackage dependencies later this week. Cheers, - Ben [1] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0111-l… [2] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-s… [3] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0216-q… [4] https://downloads.haskell.org/ghc/9.0.1-alpha1/docs/html/users_guide/9.0.1-…

6 10

Using a development snapshot of happy
by Vladislav Zavialov 04 Dec '20

04 Dec '20

Hi ghc-devs, I’m working on the unification of parsers for terms and types, and one of the things I’d really like to make use of is a feature I implemented in ‘happy’ in October 2019 (9 months ago): https://github.com/simonmar/happy/pull/153 It’s been merged upstream, but there has been no release of ‘happy’, despite repeated requests: 1. I asked for a release in December: https://github.com/simonmar/happy/issues/164 2. Ben asked for a release a month ago: https://github.com/simonmar/happy/issues/168 I see two solutions here: a) Find a co-maintainer for ‘happy’ who could make releases more frequently (I understand the current maintainers probably don’t have the time to do it). b) Use a development snapshot of ‘happy’ in GHC Maybe we need to do both, but one reason I’d like to see (b) in particular happen is that I can imagine introducing more features to ‘happy’ for use in GHC, and it’d be nice not to wait for a release every time. For instance, there are some changes I’d like to make to happy/alex in order to implement #17750 So here are two questions I have: 1. Are there any objections to this idea? 2. If not, could someone more familiar with the build process guide me as to how this should be implemented? Do I add ‘happy’ as a submodule and change something in the ./configure script, or is there more to it? Do I need to modify make/hadrian, and if so, then how? Thanks, - Vlad

8 9