- ghc-devs - Haskell.org

Status and GHC's treatment of Haddock
by Ben Gamari 16 Dec '20

16 Dec '20

Hi all, Sadly `master` has been broken for much of the day. The causal-chain is roughly the following: 1. Last week a haddock merge-request was prematurely merged by a haddock developer, despite not being compilable, to the `ghc-head` branch. 2. marge-bot failed during a merge on Sunday 3. On Monday I finished merging the half-merged batch manually, not aware of the inconsistent submodule state. However, while doing so some unintended changes crept in, relying on an unmerged haddock commit. 4. in an attempt to fix the inconsistent submodule state another developer merged the unmerged commit 5. I notice the unintentional commits on `master` and start to back them out 6. I note that even after backing out the unintentional commits the tree *still* doesn't build due to haddock 7. I perform a fair amount of head-scratching trying to figure out what is going on; eventually I find the non-building commit. 8. I eventually sort out what needs to be backed-out and in what order I take a few lessons from this: * manual merges, no matter how "straightforward", are always a vector for mistakes. I should have triple-checked my work before pushing manually pushing the half-merged branch. * haddock submodule changes going in prematurely come with a very high cost, especially if they don't build. It turns out I had run into the non-compilable haddock commit a few times last week but had chalked it up to in-flight changes. On the whole, I am really starting to wonder whether we can't improve our model for dealing with haddock; it is currently quite easy for haddock and GHC to enter an inconsistent state. Moreover, the fact that `haddock`'s `ghc-head` branch runs ahead of `master` means that we won't realize the problem until it's too late. One small improvement would be to move Haddock to GitLab. This would at least ensure that there is a clear connection between haddock MRs and their corresponding GHC changes. I have suggested this to Alex. Beyond this, it would be great if someone could step up to finish the hi-haddock change [1]. I suspect this refactoring has the potential to greatly improve this situation: the interface file AST tends to be more stable that the Core AST and has significantly less surface area. After this is in place Haddock could then use either its own AST or a frozen HsSyn AST from GHC (faciliated by TTG) internally, greatly reducing the coupling between Haddock and GHC. Any takers? Cheers, - Ben

1 0

Nested constructed product results?
by Alexis King 15 Dec '20

15 Dec '20

Hi all, I spent some time today looking into the performance of a program involving a parser type that looks something like this: data AnnotatedParser a = AnnotatedParser { annotation :: Annotation , parser :: String -> (Maybe a, String) } The `Annotation` records metadata about the structure of an `AnnotatedParser` that can be accessed statically (that is, without having to run the parser on some input). `AnnotatedParser`s are built from various primitive constructors and composed using various combinators. These combinators end up looking something like this: (<+>) :: AnnotatedParser a -> AnnotatedParser b -> AnnotatedParser (a, b) AnnotatedParser ann1 f <+> AnnotatedParser ann2 g = AnnotatedParser { annotation = Seq ann1 ann2 , parser = \s1 -> let !(a, s2) = f s1 !(b, s3) = g s2 in ((,) <$> a <*> b, s3) } Use of these combinators leads to the construction and subsequent case analysis of numerous `AnnotatedParser` closures. Happily, constructed product result[1] analysis kicks in and rewrites such combinators to cut down on the needless boxing, leading to worker/wrapper splits like this: $w<+> :: Annotation -> (String -> (Maybe a, String)) -> Annotation -> (String -> (Maybe b, String)) -> (# Annotation, String -> (Maybe (a, b), String) #) $w<+> ann1 f ann2 g = (# Seq ann1 ann2 , \s1 -> let !(a, s2) = f s1 !(b, s3) = g s2 in ((,) <$> a <*> b, s3) #) <+> :: AnnotatedParser a -> AnnotatedParser b -> AnnotatedParser (a, b) <+> (AnnotatedParser ann1 f) (AnnotatedParser ann2 g) = case $w<+> ann1 f ann2 g of (# a, b #) -> AnnotatedParser a b {-# INLINE <+> #-} This is great, and it cuts down on allocation significantly, but there is still something unsatisfying about it: the `parser` function inside the record is not affected by CPR! This is a shame, because essentially all use sites immediately deconstruct the pair, making it a prime candidate for unboxing. Ideally, we’d like to get this, instead: $w<+> :: Annotation -> (String -> (Maybe a, String)) -> Annotation -> (String -> (Maybe b, String)) -> (# Annotation, String -> (# Maybe (a, b), String #) #) $w<+> ann1 f ann2 g = (# Seq ann1 ann2 , \s1 -> let !(a, s2) = f s1 !(b, s3) = g s2 in (# (,) <$> a <*> b, s3 #) #) In practice, little combinators like `$w<+>` are marked INLINE, so `f` and `g` are usually known rather than unknown calls. This nested CPR transformation would allow the tuple construction to fuse with the tuple deconstruction, eliminating quite a lot of unnecessary boxing/unboxing. Unfortunately, it seems as though GHC’s implementation of CPR is entirely first-order: although function arguments are given rich demand signatures, results are only described one level deep. But as the above example hopefully illustrates, that’s leaving significant optimization opportunities on the table! Hence, my questions: 1. Has this notion of “nested CPR” been explored at all before? 2. Does such an extension to CPR sound worth its weight? I peeked a little at GHC.Types.Cpr and GHC.Core.Opt.CprAnal, and it seems quite manageable to me, but I haven’t actually looked into an implementation attempt just yet. I’m mostly interested in whether others have thought about something like this and/or run into similar issues in the past, or if this is really an unusual construction. Thanks, Alexis [1]: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/cpr.pdf

2 1

[ANNOUNCEMENT] Change in MR landing procedure
by Ben Gamari 10 Dec '20

10 Dec '20

tl;dr. On Thursday 10 Dec, Marge-bot will be relocated to a retirement facility. Her duties will henceforth be carried out by GitLab's Merge Train functionality. Hi all, When GHC moved to GitLab two years ago the platform lacked a critical feature: the ability to manage a queue of merge requests slated to be merged. For the past two years we have been using @marge-bot for this functionality. While Marge has served us well, she also introduces her fair share of failure modes. In the meantime GitLab has grown proper support for our use-case in the form of the "merge train" mechanism [1]. In light of Marge's growing intransigence [2], I will grant her a well-earned retirement and begin using merge trains for merging MRs starting on Thursday. As a result, the workflow for adding a merge request to the merge queue will become a bit more straightforward: Instead of setting the MR's "assignee" to @marge-bot rather push the blue "Start/add to merge train" button. This adds the merge request to the merge queue. This is documented in the Wiki [3]. Apart from freeing us from the need to maintain Marge, this change also enables us to use GitLab's branch-squashing functionality, which Marge did not support. Do let me know if you have any questions or concerns. So long, Marge, and thanks for all of the commits. Cheers, - Ben [1] https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_… [2] https://gitlab.haskell.org/ghc/ghc/-/issues/19034 [3] https://gitlab.haskell.org/ghc/ghc/-/wikis/Contributing-a-Patch

2 2

Help lead Haskell: Executive Director sought for Haskell Foundation
by Richard Eisenberg 08 Dec '20

08 Dec '20

Posting on behalf of the Haskell Foundation Working Group. Please forward widely! -------- The Haskell Foundation is seeking an Executive Director. Please find the job description here: https://haskell.foundation/ <https://haskell.foundation/executive-director>ed-job-description <https://haskell.foundation/ed-job-description/> The Haskell Foundation (HF) is an independent, non-profit organization dedicated to driving Haskell adoption and open-source development. HF seeks a full-time Executive Director (ED) to lead and develop the organization going forward. The ED is responsible for furthering the HF’s mission and vision, ensuring that resources are in place to accomplish its goals. The HF is supported by an active group of volunteers, and the ED will be a key focal point for that community, in addition to hiring support staff. This is a salaried position. Salary will be commensurate with the experience, qualities, and location of the individual, and will also reflect the Foundation’s status as a non-profit organisation funded by donations.

1 0

[ANNOUNCE] Glasgow Haskell Compiler 9.0.1-alpha1 released
by Ben Gamari 07 Dec '20

07 Dec '20

Hello all, The GHC team is very pleased to announce the availability of the first alpha release in the GHC 9.0 series. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/9.0.1-alpha1/ This first alpha comes quite a bit later than expected. However, we have done a significant amount of testing on this pre-release and therefore hope to be able to move forward quickly with a release candidate next week and with a final release in mid-October. GHC 9.0.1 will bring a number of new features: * A first cut of the new LinearTypes language extension [1], allowing use of linear function syntax and linear record fields. * A new bignum library (ghc-bignum), allowing GHC to be more easily used with integer libraries other than GMP. * Improvements in code generation, resulting in considerable performance improvements in some programs. * Improvements in pattern-match checking, allowing more precise detection of redundant cases and reduced compilation time. * Implementation of the "simplified subsumption" proposal [2] simplifying the type system and paving the way for QuickLook impredicativity in GHC 9.2. * Implementation of the QualifiedDo extension [3], allowing more convenient overloading of `do` syntax. * Improvements in compilation time. And many more. See the release notes [4] for a full accounting of the changes in this release. Do note that there are a few things that we expect will change before the final release: * We expect to sort out a notarization workflow for Apple Darwin, allowing our binary distributions to be used on macOS Catalina without hassle. Until this has been sorted out Catalina users can exempt the current macOS binary distribution from the notarization requirement themselves by running `xattr -cr .` on the unpacked tree before running `make install`. * We will likely transition the Alpine binary distribution to be fully statically-linked, providing a convenient, distribution-independent packaging option for Linux users. * We will be merging a robust solution for #17760 which will introduce a new primitive, `keepAlive#`, to the `base` library, subsuming most uses of `touch#`. As always, do test this release and open tickets for whatever issues you encounter. To help with this, we will be publishing a blog post describing use of our new `head.hackage` infrastructure to ease testing of larger projects with Hackage dependencies later this week. Cheers, - Ben [1] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0111-l… [2] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-s… [3] https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0216-q… [4] https://downloads.haskell.org/ghc/9.0.1-alpha1/docs/html/users_guide/9.0.1-…

6 10

Using a development snapshot of happy
by Vladislav Zavialov 04 Dec '20

04 Dec '20

Hi ghc-devs, I’m working on the unification of parsers for terms and types, and one of the things I’d really like to make use of is a feature I implemented in ‘happy’ in October 2019 (9 months ago): https://github.com/simonmar/happy/pull/153 It’s been merged upstream, but there has been no release of ‘happy’, despite repeated requests: 1. I asked for a release in December: https://github.com/simonmar/happy/issues/164 2. Ben asked for a release a month ago: https://github.com/simonmar/happy/issues/168 I see two solutions here: a) Find a co-maintainer for ‘happy’ who could make releases more frequently (I understand the current maintainers probably don’t have the time to do it). b) Use a development snapshot of ‘happy’ in GHC Maybe we need to do both, but one reason I’d like to see (b) in particular happen is that I can imagine introducing more features to ‘happy’ for use in GHC, and it’d be nice not to wait for a release every time. For instance, there are some changes I’d like to make to happy/alex in order to implement #17750 So here are two questions I have: 1. Are there any objections to this idea? 2. If not, could someone more familiar with the build process guide me as to how this should be implemented? Do I add ‘happy’ as a submodule and change something in the ./configure script, or is there more to it? Do I need to modify make/hadrian, and if so, then how? Thanks, - Vlad

8 9

Use of forall as a sigil
by Bryan Richter 03 Dec '20

03 Dec '20

Dear forall ghc-devs. ghc-devs, As I read through the "Visible 'forall' in types of terms" proposal[1], I stumbled over something that isn't relevant to the proposal itself, so I thought I would bring it here. Given f :: forall a. a -> a (1) I intuitively understand the 'forall' in (1) to represent the phrase "for all". I would read (1) as "For all objects a in Hask, f is some relation from a to a." After reading the proposal, I think my intuition may be wrong, since I discovered `forall a ->`. This means something similar to, but practically different from, `forall a.`. Thus it seems like 'forall' is now simply used as a sigil to represent "here is where some special parameter goes". It could as well be an emoji. What's more, the practical difference between the two forms is *only* distinguished by ` ->` versus `.`. That's putting quite a lot of meaning into a rather small number of pixels, and further reduces any original connection to meaning that 'forall' might have had. I won't object to #281 based only on existing syntax, but I *do* object to the existing syntax. :) Is this a hopeless situation, or is there any possibility of bringing back meaning to the syntax of "dependent quantifiers"? -Bryan [1]: https://github.com/ghc-proposals/ghc-proposals/pull/281

9 17

PSA: type checker plugins to be affected by upcoming changes to GHC
by Richard Eisenberg 30 Nov '20

30 Nov '20

Hi all, I'm hard at work on two significant refactorings within GHC's constraint solver. The first is at https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4149. It removes flattening meta-variables and flattening skolems. This is a very nice simplification. Instead, it just reduces type families directly. My other patch (held up by the first) is at https://gitlab.haskell.org/ghc/ghc/-/tree/wip/derived-refactor and will remove Derived constraints, to be replaced by a little bit of cleverness in suppressing certain confusing error messages. My guess is that either or both of these will invalidate the current behavior of type-checker plugins. Sadly, this is not just a case of finding a new function that replicates the old behavior -- enough is changing under the hood that you might actually have to rewrite chunks of your code. I have never written a type-checker plugin, and so I don't currently have advice for you. But if you are a plugin author affected by this change and want help, please reach out -- I would be happy to walk you through the changes, and then hopefully make a little video explaining the process to other plugin authors. Neither patch will make it for 9.0, but I expect both to be in 9.2. There may be more where this came from (https://gitlab.haskell.org/ghc/ghc/-/issues/18965) in the future, but it's all for a good cause. (I have bcc'd plugin authors that I'm aware of. Just adding this in case you're surprised at receiving this email.) Thanks, Richard

5 11

Hadrian: Error "cannot find -lHSrts-1.0_thr_l" when linking ghc executable.
by Roland Senn 25 Nov '20

25 Nov '20

Hi there! I have two GHC trees. One is called ghcdebug and is a little bit outdated (16. September 2020). In this tree, I can work normally: Change code, compile and run the resulting GHC compiler. The second tree is up-to-date! Compiling works fine, however the linker steps for the ghc executable fails. When I restart the failing link step, I get the following error: $ hadrian/build --flavour=devel2 --freeze1 -j2 stage2:exe:ghc-bin Up to date | Run Ghc LinkHs Stage1: _build/stage1/ghc/build/c/hschooks.o (and 7 more) => _build/stage1/bin/ghc /usr/bin/ld.gold: error: cannot find -lHSrts-1.0_thr_l _build/stage1/ghc/build/GHCi/Leak.o(.text+0xd5): error: undefined reference to 'stg_upd_frame_info' _build/stage1/ghc/build/GHCi/Leak.o(.text+0x1c2): error: undefined reference to 'stg_upd_frame_info' _build/stage1/ghc/build/GHCi/Leak.o(.text+0x1e6): error: undefined reference to 'stg_ap_ppp_info' ... and many many more lines with "error: undefined reference to ..." I get the same error when I use a freshly cloned ghc tree. I also get the error independently of the GHC version used to compile (8.8.3 or 8.10.2). I didn't modify any settings files. I do: hadrian/build clean hadrian/build --flavour=devel2 -j2 I'm using a normal x68_64 box with plain vanilla Linux Debian 10. No Docker and no Nix. The directory _build/stage/rts/build contains: roland@goms:~/Projekte/ghc$ ls -all _build/stage1/rts/build/ insgesamt 19396 drwxr-xr-x 5 roland roland 4096 Nov 24 17:58 . drwxr-xr-x 3 roland roland 4096 Nov 24 17:56 .. drwxr-xr-x 2 roland roland 4096 Nov 24 17:56 autogen drwxr-xr-x 7 roland roland 12288 Nov 24 17:58 c drwxr-xr-x 2 roland roland 4096 Nov 24 17:58 cmm -rw-r--r-- 1 roland roland 31349 Nov 24 17:58 DerivedConstants.h -rw-r--r-- 1 roland roland 13764 Nov 24 17:57 ffi.h -rw-r--r-- 1 roland roland 4343 Nov 24 17:57 ffitarget.h -rw-r--r-- 1 roland roland 15128 Nov 24 17:58 ghcautoconf.h -rw-r--r-- 1 roland roland 618 Nov 24 17:58 ghcplatform.h -rw-r--r-- 1 roland roland 732 Nov 24 17:58 ghcversion.h -rw-r--r-- 1 roland roland 82486 Nov 24 17:57 libCffi.a lrwxrwxrwx 1 roland roland 9 Nov 24 17:57 libCffi_thr.a -> libCffi.a -rw-r--r-- 1 roland roland 9166966 Nov 24 17:58 libHSrts-1.0.a -rw-r--r-- 1 roland roland 10490730 Nov 24 17:58 libHSrts-1.0_thr.a Any ideas what's wrong or how to fix? Many thanks and Cheers, Roland

2 3

Questions about System F and data types
by Jan van Brügge 24 Nov '20

24 Nov '20

Hi, I know that this is probably not the correct place to ask as my question is not directly related to GHC itself, but I thought here are the people that can most likely answer this. To better understand PL papers, especially those involving System Fc and its extensions, I started to write a formal proof of type safety including alpha-equivalence. Currently I have a complete proof for System F <https://github.com/jvanbruegge/isabelle-lambda-calculus> (without coercions and data types yet). I mainly used the System Fc paper <https://www.microsoft.com/en-us/research/wp-content/uploads/2007/01/tldi22-…> as reference, ignoring all the parts about coercions. 1. In the Fc paper, the rule `AppT` has a judgement delta as assumption, which does not exist. I assumed the `ty` judgement was meant there by looking at the arguments. Is this correct? 2. In the `Abs` rule, the type of the variable is required to be valid and of kind star by the judgement `ty`. In the `Let` rule, this assumption is missing. I tried it like that, but was not able to complete the proofs. Is such an assumption missing there or should I be able to proof it without? 3. In the Fc paper, the types and kinds of datatype declarations are added to the context with a separate judgement that interprets the datatype declarations. In the System Fc pro paper <https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/p53-yor…> (from what I can tell) those types and kinds are assumed to be already part of the initial context. At the moment I prove progress against the empty context, so I guess I would have to relax that to an initial context that only contains types and kinds of type constants and nothing else. Is that correct? What is here the "best practice" in terms of PL research? Thank you all Jan

2 2