- ghc-devs - Haskell.org

Why TcLclEnv and DsGblEnv need to store the same IORef for errors?
by Alfredo Di Napoli 01 Apr '21

01 Apr '21

Hello folks, as some of you might know me and Richard are reworking how GHC constructs, emits and deals with errors and warnings (See https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values and #18516). To summarise very briefly the spirit, we will have (eventually) proper domain-specific types instead of SDocs. The idea is to have very precise and "focused" types for the different phases of the compilation pipeline, and a "catch-all" monomorphic `GhcMessage` type used for the final pretty-printing and exception-throwing: data GhcMessage where GhcPsMessage :: PsMessage -> GhcMessage GhcTcRnMessage :: TcRnMessage -> GhcMessage GhcDsMessage :: DsMessage -> GhcMessage GhcDriverMessage :: DriverMessage -> GhcMessage GhcUnknownMessage :: forall a. (Diagnostic a, Typeable a) => a -> GhcMessage While starting to refactor GHC to use these types, I have stepped into something bizarre: the `DsGblEnv` and `TcLclEnv` envs both share the same `IORef` to store the diagnostics (i.e. warnings and errors) accumulated during compilation. More specifically, a function like `GHC.HsToCore.Monad.mkDsEnvsFromTcGbl` simply receives as input the `IORef` coming straight from the `TcLclEnv`, and stores it into the `DsGblEnv`. This is unfortunate, because it would force me to change the type of this `IORef` to be `IORef (Messages GhcMessage)` to accommodate both diagnostic types, but this would bubble up into top-level functions like `initTc`, which would now return a `Messages GhcMessage`. This is once again unfortunate, because is "premature": ideally it might still be nice to return `Messages TcRnMessage`, so that GHC API users could get a very precise diagnostic type rather than the bag `GhcMessage` is. It also violates an implicit contract: we are saying that `initTc` might return (potentially) *any* GHC diagnostic message (including, for example, driver errors/warnings), which I think is misleading. Having said all of that, it's also possible that returning `Messages GhcMessage` is totally fine here and we don't need to be able to do this fine-grained distinction for the GHC API functions. Regardless, I would like to ask the audience: * Why `TcLclEnv` and `DsGblEnv` need to share the same IORef? * Is this for efficiency reasons? * Is this because we need the two monads to independently accumulate errors into the same IORef? Thanks! Alfredo

4 10

Type inference of singular matches on GADTs
by Alexis King 30 Mar '21

30 Mar '21

Hi all, Today I was writing some code that uses a GADT to represent heterogeneous lists: data HList as where HNil :: HList '[] HCons :: a -> HList as -> HList (a ': as) This type is used to provide a generic way to manipulate n-ary functions. Naturally, I have some functions that accept these n-ary functions as arguments, which have types like this: foo :: Blah as => (HList as -> Widget) -> Whatsit The idea is that Blah does some type-level induction on as and supplies the function with some appropriate values. Correspondingly, my use sites look something like this: bar = foo (\HNil -> ...) Much to my dismay, I quickly discovered that GHC finds these expressions quite unfashionable, and it invariably insults them: • Ambiguous type variable ‘as0’ arising from a use of ‘foo’ prevents the constraint ‘(Blah as0)’ from being solved. The miscommunication is simple enough. I expected that when given an expression like \HNil -> ... GHC would see a single pattern of type HList '[] and consequently infer a type like HList '[] -> ... Alas, it was not to be. It seems GHC is reluctant to commit to the choice of '[] for as, lest perhaps I add another case to my function in the future. Indeed, if I were to do that, the choice of '[] would be premature, as as ~ '[] would only be available within one branch. However, I do not in fact have any such intention, which makes me quietly wish GHC would get over its anxiety and learn to be a bit more of a risk-taker. I ended up taking a look at the OutsideIn(X) paper, hoping to find some commentary on this situation, but in spite of the nice examples toward the start about the trickiness of GADTs, I found no discussion of this specific scenario: a function with exactly one branch and an utterly unambiguous pattern. Most examples come at the problem from precisely the opposite direction, trying to tease out a principle type from a collection of branches. The case of a function (or perhaps more accurately, a case expression) with only a single branch does not seem to be given any special attention. Of course, fewer special cases is always nice. I have great sympathy for generality. Still, I can’t help but feel a little unsatisfied here. Theoretically, there is no reason GHC cannot treat \(a `HCons` b `HCons` c `HCons` HNil) -> ... and \a b c -> ... almost identically, with a well-defined principle type and pleasant type inference properties, but there is no way for me to communicate this to the typechecker! So, my questions: 1. Have people considered this problem before? Is it discussed anywhere already? 2. Is my desire here reasonable, or is there some deep philosophical argument for why my program should be rejected? 3. If it /is/ reasonable, are there any obvious situations where a change targeted at what I’m describing (vague as that is) would affect programs negatively, not positively? I realize this gets rather at the heart of the typechecker, so I don’t intend to imply a change of this sort should be made frivolously. Indeed, I’m not even particularly attached to the idea that a change must be made! But I do want to understand the tradeoffs better, so any insight would be much appreciated. Thanks, Alexis

6 15

Multiple versions of happy
by Simon Peyton Jones 30 Mar '21

30 Mar '21

What's the approved mechanism to install multiple versions of happy/alex etc? Eg I tried to build ghc-9.0 and got this: checking for makeinfo... no checking for python3... /usr/bin/python3 checking for ghc-pkg matching /opt/ghc/bin/ghc... /opt/ghc/bin/ghc-pkg checking for happy... /home/simonpj/.cabal/bin/happy checking for version of happy... 1.20.0 configure: error: Happy version 1.19 is required to compile GHC. I so I have to 1. Install happy-1.19 without overwriting the installed happy-1.20 2. Tell configure to use happy-1.19 What's the best way to do those two things? Thanks Simon

3 3

Options for targeting Windows XP?
by Clinton Mead 29 Mar '21

29 Mar '21

I'm currently trying to bring my company around to using a bit of Haskell. One issue is that a number of our clients are based in South East Asia and need software that runs on Windows XP. Unfortunately it seems the last version of GHC that produces executables that run on Windows XP is GHC 7.10. Whilst this table <https://gitlab.haskell.org/ghc/ghc/-/wikis/platforms/windows> suggests the issue may only running GHC 8.0+ on Windows XP, I've confirmed that GHC 8.0 executables (even "Hello World") will not run on Windows XP, presumably because a non-XP WinAPI call in the runtime. My first thought would be to restrict myself to GHC 7.10 features (i.e. 2015). This would be a slight annoyance but GHC 7.10 still presents a reasonable language. But my concern would be that increasingly I'll run into issues with libraries that use extensions post GHC 7.10, particularly libraries with large dependency lists. So there's a few options I've considered at this point: 1. Use GHCJS to compile to Javascript, and then dig out a version of NodeJS that runs on Windows XP. GHCJS seems to at least have a compiler based on GHC 8.6. 2. Patch GHC with an additional command line argument to produce XP/Vista compatible executables, perhaps by looking at the changes between 7.10 -> 8.0, and re-introducing the XP approach as an option. The issue with 1 is that is that as well as being limited by how up to date GHCJS is, this will increase install size, memory usage and decrease performance on Windows XP machines, which are often in our environments quite old and resource and memory constrained. Approach 2 is something I'd be willing to put some work into if it was practical, but my thought is that XP support was removed for a reason, presumably by using newer WinAPI functions simplified things significantly. By re-adding in XP support I'd be complicating GHC once again, and GHC will effectively have to maintain two approaches. In addition, in the long term, whenever a new WinAPI call is added one would now have to check whether it's available in Windows XP, and if it's not produce a Windows XP equivalent. That might seem like just an extra burden of support for already busy GHC developers. But on the other hand, if the GHC devs would be happy to merge a patch and keep up XP support this would be the cleanest option. But then I had a thought. If GHC Core isn't supposed to change much between versions is it? Which made me come up with these approaches: 3. Hack up a script to compile programs using GHC 9 to Core, then feed that Core output into GHC 7.10. OR 4. Produce a chimera style GHC by importing the GHC 9.0 API and the GHC 7.10 API, and making a version of GHC that does Haskell -> Core in GHC 9.0 and the rest of the code generation in GHC 7.10. One issue with 4 will be that presumably that because I'm importing GHC 9.0 API and the 7.10 API separately, all their data types will technically be separate, so I'll need to basically deep copy the GHC 9.0 core datatype (and perhaps others) to GHC 7.10 datatypes. But presuming their largely similar this should be fairly mechanical. So are any of these approaches (well, particularly 2 and 4) reasonable? Or am I going to run into big problems with either of them? Is there another approach I haven't thought of? Thanks, Clinton

8 14

GHC 9.2 has branched
by Ben Gamari 29 Mar '21

29 Mar '21

Hello everyone, At this point the ghc-9.2 branch has officially branched off from master. If there was anything you were holding back from `master`, feel free to now send it off to Marge. I'll be working on doing some release prep and pushing out an alpha in the next few days. This alpha will very likely lack the new NCG (which is still being worked on by Moritz) but otherwise should be functionally complete. Cheers, - Ben

1 0

Pattern matching desugaring regression? Re: [Haskell-cafe] Why does my module take so long to compile?
by Carter Schonwald 28 Mar '21

28 Mar '21

Ccing ghc devs since that’s a better forum perhaps Crazy theory: this is a regression due the the partial changes to pattern matching coverage checking in 8.10 that finished / landed in ghc 9 Why: Desugaring is when pattern/ case statement translation happens I think? And the only obvious “big thing” is that you have some huge , albeit sane for a compiler, pattern matching I’d first check if the new ghc 9 release doesn’t have that regression in build time that you experienced. And if it does file a ticket. I may be totally wrong, but that seems like a decent likelihood ! On Mon, Feb 15, 2021 at 12:20 PM Troels Henriksen <athas(a)sigkill.dk> wrote: > Carter Schonwald <carter.schonwald(a)gmail.com> writes: > > > How big are your data types in the file In question? Do you ghc generics > or > > deriving or template Haskell? Could you share a link to the file in > > question ? > > The file does not define any large data types itself, but it operates on > some fairly large data types (an AST, it's a compiler). But so do many > other modules that work just fine. It uses no generics, nontrivial > deriving, or Template Haskell. It's this file: > > https://github.com/diku-dk/futhark/blob/master/src/Futhark/Pass/ExtractKern… > > I also found a handful of other modules in my project that are > significantly slower to compile in GHC 8.10, and seemingly also because > of the desugarer, but none of them have any obvious smoking guns like > generics or TH. > > The only commonality I can find is that the affected modules contain > functions with a relatively large typeclass context. I use > ConstraintKinds to make them more concise, and I guess unfolded there > may be 30-40 equality/class constraints in total. Like this: > > type DistLore lore = > ( Bindable lore, > HasSegOp lore, > BinderOps lore, > LetDec lore ~ Type, > ExpDec lore ~ (), > BodyDec lore ~ () > ) > > where the 'Bindable' constraint in particular then implies a further > handful of "~" constraints: > > class > ( ASTLore lore, > FParamInfo lore ~ DeclType, > LParamInfo lore ~ Type, > RetType lore ~ DeclExtType, > BranchType lore ~ ExtType, > SetType (LetDec lore) > ) => > Bindable lore > where > ... > > FParamInfo/LParamInfo/etc are type families. Are such constraints > particularly slow? > > -- > \ Troels > /\ Henriksen >

4 7

config.sub
by Simon Peyton Jones 26 Mar '21

26 Mar '21

Folks I'm getting a lot of this simonpj@MSRC-3645512:~/code/HEAD-3$ git status On branch wip/T19569 Your branch is up to date with 'origin/wip/T19569'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) (commit or discard the untracked or modified content in submodules) modified: libraries/unix (modified content) Untracked files: (use "git add <file>..." to include in what will be committed) libraries/base/config.sub libraries/ghc-bignum/config.sub What has changed in unix? Answer: simonpj@MSRC-3645512:~/code/HEAD-3$ cd libraries/unix simonpj@MSRC-3645512:~/code/HEAD-3/libraries/unix$ git status HEAD detached at 21437f2 Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: config.sub Ugh. Why is config.sub modified if it's a repo file? And should I ignore the untracked base/config.sub and ghc-bignum? What am I doing wrong? Thanks Simon

2 1

HLint in the GHC CI, an eight-months retrospective
by Hécate 26 Mar '21

26 Mar '21

Hello fellow devs, this email is an activity report on the integration of the HLint[0] tool in the Continuous Integration (CI) pipelines. On Jul. 5, 2020 I opened a discussion ticket[1] on the topic of code linting in the several components of the GHC code-base. It has served as a reference anchor for the Merge Requests (MR) that stemmed from it, and allowed us to refine our expectations and processes. If you are not acquainted with its content, I invite you to read the whole conversation. Subsequently, several Hadrian lint rules have been integrated in the following months, in order to run HLint on targeted components of the GHC repository (the base library, the compiler code-base, etc). Being satisfied with the state of the rules we applied to the code-base, such as removing extraneous pragmata and keywords, it was decided to integrate the base library linting rule in the CI. This was five months ago, in September[2], and I am happy to report that developer friction has been so far minimal. In parallel to this work on the base library, I took care of cleaning-up the compiler, and harmonised the various micro coding styles that have emerged quite organically during the decades of development that are behind us (I never realised how many variations of the same ten lines of pragmata could coexist in the same folders). Upon feedback from stakeholders of this sub-code base, the rules file was altered to better suit their development needs, such as not removing extraneous `do` keywords, as they are useful to introduce a block in which debug statements can be easily inserted. Since today, the linting of the compiler code-base has been integrated in our CI pipelines, without further burdening our CI times. Things seem to run smoothly, and I welcome comments and requests of any kind related to this area of our code quality process. Regarding our future plans, there has been a discussion about integrating such a linting mechanism for our C code-base, in the RTS. Nothing is formally established yet, so I would be grateful if people who have experience and wisdom about it can chime in to contribute to the discussion: https://gitlab.haskell.org/ghc/ghc/-/issues/19437. And I would like to say that I am overall very thankful for the involvement of the people who have been giving us feedback and have been reviewing the resulting MRs. Have a very nice day, Hécate --- [0]: https://github.com/ndmitchell/hlint [1]: https://gitlab.haskell.org/ghc/ghc/-/issues/18424 [2]: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4147 -- Hécate ✨ 🐦: @TechnoEmpress IRC: Uniaika WWW: https://glitchbra.in RUN: BSD

2 2

On CI
by Moritz Angermann 24 Mar '21

24 Mar '21

Hi there! Just a quick update on our CI situation. Ben, John, Davean and I have been discussion on CI yesterday, and what we can do about it, as well as some minor notes on why we are frustrated with it. This is an open invitation to anyone who in earnest wants to work on CI. Please come forward and help! We'd be glad to have more people involved! First the good news, over the last few weeks we've seen we *can* improve CI performance quite substantially. And the goal is now to have MR go through CI within at most 3hs. There are some ideas on how to make this even faster, especially on wide (high core count) machines; however that will take a bit more time. Now to the more thorny issue: Stat failures. We do not want GHC to regress, and I believe everyone is on board with that mission. Yet we have just witnessed a train of marge trials all fail due to a -2% regression in a few tests. Thus we've been blocking getting stuff into master for at least another day. This is (in my opinion) not acceptable! We just had five days of nothing working because master was broken and subsequently all CI pipelines kept failing. We have thus effectively wasted a week. While we can mitigate the latter part by enforcing marge for all merges to master (and with faster pipeline turnaround times this might be more palatable than with 9-12h turnaround times -- when you need to get something done! ha!), but that won't help us with issues where marge can't find a set of buildable MRs, because she just keeps hitting a combination of MRs that somehow together increase or decrease metrics. We have three knobs to adjust: - Make GHC build faster / make the testsuite run faster. There is some rather interesting work going on about parallelizing (earlier) during builds. We've also seen that we've wasted enormous amounts of time during darwin builds in the kernel, because of a bug in the testdriver. - Use faster hardware. We've seen that just this can cut windows build times from 220min to 80min. - Reduce the amount of builds. We used to build two pipelines for each marge merge, and if either of both (see below) failed, marge's merge would fail as well. So not only did we build twice as much as we needed, we also increased our chances to hit bogous build failures by 2. We need to do something about this, and I'd advocate for just not making stats fail with marge. Build errors of course, but stat failures, no. And then have a separate dashboard (and Ben has some old code lying around for this, which someone would need to pick up and polish, ...), that tracks GHC's Performance for each commit to master, with easy access from the dashboard to the offending commit. We will also need to consider the implications of synthetic micro benchmarks, as opposed to say building Cabal or other packages, that reflect more real-world experience of users using GHC. I will try to provide a data driven report on GHC's CI on a bi-weekly or month (we will have to see what the costs for writing it up, and the usefulness is) going forward. And my sincere hope is that it will help us better understand our CI situation; instead of just having some vague complaints about it. Cheers, Moritz

11 23

GHC 8.10 backports?
by Moritz Angermann 24 Mar '21

24 Mar '21

Hi there! Does anyone have any backports they'd like to see for consideration for 8.10.5? Cheers, Moritz

5 8