- ghc-devs - Haskell.org

How does GHC implement layout?
by Alexis King 05 Apr '21

05 Apr '21

Hi all, I’m wondering if there are any resources that discuss the design of GHC’s implementation of layout. (I haven’t been able to find any.) From looking at the code, here’s what I’ve gathered so far: * Layout is implemented in the lexer (compiler/GHC/Parser/Lexer.x). * The implementation is similar in some respects to the approach described in the Haskell Report, but still fairly different. Virtual braces and semicolons are inserted during the lexing process itself with the assistance of Alex lexer states (aka “start codes”). * In order to handle particularly tricky cases like if e then do x; y else z where the virtual close brace must be inserted in the middle of a line, tokens such as in and else are given special context-sensitive treatment. This appears to be quite subtle. Overall, I can mostly follow the code, but I still have a few unanswered questions: * The layout-related code consistently uses the phrase “alternative layout rule”—what does “alternative” mean here? Does it refer to GHC’s implementation of layout? Or maybe it refers to NondecreasingIndentation? It isn’t clear. * The implementation of layout seems quite complex, in large part because it has to worry about parsing concerns in the lexer in order to handle tricky cases like the one I provided above. Is there are reason all this is done in the lexer, rather than deferring some more of the work to the parser? I’ve found remarkably little information about implementing layout in general, so perhaps I’m missing some resources or keywords to search for, but any information or perspectives would be appreciated! Thanks, Alexis

5 5

GHC Proposal: GHC Maintainer Preview
by Hécate 03 Apr '21

03 Apr '21

Hi everyone! I would like to share with you all the proposal I opened, called "GHC Maintainer Preview", to shape the release schedule of our beloved compiler. The concept is that we let a certain period of time pass between the final "release" of a major (X.Y) GHC version, and its official, public, user-targeting release, so that we can focus our efforts in helping library maintainers to migrate and change their code to accommodate the whatever breaking change the new major version introduces. The proposal is located here: https://github.com/ghc-proposals/ghc-proposals/pull/417 Have a nice week-end! -- Hécate ✨ 🐦: @TechnoEmpress IRC: Uniaika WWW: https://glitchbra.in RUN: BSD

1 0

Error and Warning Messages practical migration guidelines
by Alfredo Di Napoli 03 Apr '21

03 Apr '21

*TL;DR If you recently had merge conflicts related to error and messages in GHC, here I give migration guidelines. Otherwise, feel free to ignore this email.* Good morning all, If you had some work-in-progress branch that was emitting errors or warnings (either adding new ones or modifying existing ones), chances are you noticed that now the API and types changed slightly and you had to deal with the conflicts. What you noticed was the initial chunk of work for [#18516]. See the [Wiki] for more info. Our work is not done yet, but the API changed enough to warrant me writing this little email to help folks out with their rebasing (because we are *all* rebasing at least daily, aren't we? :) ). First of all, sorry for all the churning! I am trying to stabilise the external interfaces (i.e the type signatures of the GHC API functions) so things will become a bit more stable in the future. In the meantime, I wanted to give you some practical guidelines on how to migrate your code: * Rather than "errors" or "warnings", we are trying to shift our vocabulary to talk about "diagnostics". In GHC this distinction is not so "strict", in the sense warnings can become errors and errors can become warnings if deferred. Rather we talk about diagnostics, i.e. facts about the compiled program, that can be, at the very end, classified as errors or warnings based on their `Severity`; * *If you skip this whole email, read just this:* the key insight is that there is a clear separation between the *reason* this diagnostic was created and the final `Severity`! Example: we emit a diagnostic because `Opt_WarnMonomorphism` is on. That's the raison d'être of this diagnostic. Therefore, we would create a new message (more on this later) with reason `WarningWithFlag Opt_WarnMonomorphism`; * A `Severity` of a diagnostic is computed using the `DynFlags`, because that's the only way to correctly classify a diagnostic. Back to our little example above, if `Opt_WarnMonomorphism` was set as fatal (or -Werror enabled), now the final `Severity` of the diagnostic above would be `SevError`. That's why it's important to keep them separate; * We separated out the logging messages with the diagnostics proper, this is why we now have a new [MessageClass] type which can be used to log all sort of messages (debug ones, informational etc) *and* diagnostics, which has a dedicated type constructor; * If you want to emit something which is not an error or a warning, use [putLogMsg]. Here is an example: https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Core/Lint.hs#… (we could argue if the example shouldn't be a diagnostic, but we didn't go that far in the refactoring yet, this is just to give you an idea of the usage); * The most general function which can be used to produce a diagnostic is [mkMsgEnvelope], which takes as input the `DynFlags` to compute the `Severity` under the hood. * You can't just pass a random `DynFlags` to `mkMsgEnvelope`, but the former should be acquired as close as possible to the construction/emission site of the diagnostics. This is because `DynFlags` evolves and are mangled all the time inside GHC, and we want to produce the `Severity` using the most faithful "snapshot" of the state of the world at the time the diagnostic was produced. * Most of the times you don't have to worry about all of this; if you know for sure a diagnostic is an error and is not recoverable, then you can use `mkMsgErrorEnvelope` that doesn't need the `DynFlags` and always produces a diagnostic with `SevError` and `ErrorWithoutFlags`. * Once #18516 will be completed, this will require a bit less cognitive overhead. We will have proper types (not SDocs or generic `DecoratedMessage`s), and all the working GHC hacker will have to worry about would be to create a brand new Haskell type for the diagnostic, give it a `Diagnostic` instance (or extend the instance declaration if that's an existing type) and then just call whichever high-level API we will have available. Hope this helps! Alfredo [#18516]: https://gitlab.haskell.org/ghc/ghc/-/issues/18516 [Wiki]: https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values [MessageClass]: https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.h… [putLogMsg]: https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Utils/Logger.… [mkMsgEnvelope]: https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Utils/Error.h…

1 0

[ANNOUNCE] GHC 9.2.1-alpha1 now available
by Ben Gamari 02 Apr '21

02 Apr '21

Hi all, The GHC developers are very happy to announce the availability of the first alpha release in the 9.2.1 series. Binary distributions, source distributions, and documentation are available at https://downloads.haskell.org/ghc/9.2.1-alpha1 GHC 9.2 will bring a number of exciting features including: * Many changes in the area of records, including the new `RecordDotSyntax` and `NoFieldSelectors` language extensions, as well as Support for `DuplicateRecordFields` with `PatternSynonyms`. * Introduction of the new `GHC2021` language extension set, giving users convenient access to a larger set of language extensions which have been long considered stable. * Merge of `ghc-exactprint` into the GHC tree, providing infrastructure for source-to-source program rewriting out-of-the-box. * Introduction of a `BoxedRep` `RuntimeRep`, allowing for polymorphism over levity of boxed objects (#17526) * Implementation of the `UnliftedDataTypes` extension, allowing users to define types which do not admit lazy evaluation ([proposal]) * The new [-hi profiling] mechanism which provides significantly improved insight into thunk leaks. * Support for the `ghc-debug` out-of-process heap inspection library [ghc-debug] * Support for profiling of pinned objects with the cost-centre profiler (#7275) * Introduction of Haddock documentation support in TemplateHaskell (#5467) In addition, the final 9.2.1 release will bring a new native code generator for ARM, providing fast, first-class for Haskell on Apple ARM hardware [apple-m1], although this functionality is not yet present in this alpha. As always, do give this a try and open a [ticket] if you see anything amiss. Happy testing, - Ben [apple-m1]: https://www.haskell.org/ghc/blog/20210309-apple-m1-story.html [proposal]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0265-u… [-hi profiling]: https://well-typed.com/blog/2021/01/first-look-at-hi-profiling-mode/ [ghc-debug]: http://ghc.gitlab.haskell.org/ghc-debug/ [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new

3 4

Why TcLclEnv and DsGblEnv need to store the same IORef for errors?
by Alfredo Di Napoli 01 Apr '21

01 Apr '21

Hello folks, as some of you might know me and Richard are reworking how GHC constructs, emits and deals with errors and warnings (See https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values and #18516). To summarise very briefly the spirit, we will have (eventually) proper domain-specific types instead of SDocs. The idea is to have very precise and "focused" types for the different phases of the compilation pipeline, and a "catch-all" monomorphic `GhcMessage` type used for the final pretty-printing and exception-throwing: data GhcMessage where GhcPsMessage :: PsMessage -> GhcMessage GhcTcRnMessage :: TcRnMessage -> GhcMessage GhcDsMessage :: DsMessage -> GhcMessage GhcDriverMessage :: DriverMessage -> GhcMessage GhcUnknownMessage :: forall a. (Diagnostic a, Typeable a) => a -> GhcMessage While starting to refactor GHC to use these types, I have stepped into something bizarre: the `DsGblEnv` and `TcLclEnv` envs both share the same `IORef` to store the diagnostics (i.e. warnings and errors) accumulated during compilation. More specifically, a function like `GHC.HsToCore.Monad.mkDsEnvsFromTcGbl` simply receives as input the `IORef` coming straight from the `TcLclEnv`, and stores it into the `DsGblEnv`. This is unfortunate, because it would force me to change the type of this `IORef` to be `IORef (Messages GhcMessage)` to accommodate both diagnostic types, but this would bubble up into top-level functions like `initTc`, which would now return a `Messages GhcMessage`. This is once again unfortunate, because is "premature": ideally it might still be nice to return `Messages TcRnMessage`, so that GHC API users could get a very precise diagnostic type rather than the bag `GhcMessage` is. It also violates an implicit contract: we are saying that `initTc` might return (potentially) *any* GHC diagnostic message (including, for example, driver errors/warnings), which I think is misleading. Having said all of that, it's also possible that returning `Messages GhcMessage` is totally fine here and we don't need to be able to do this fine-grained distinction for the GHC API functions. Regardless, I would like to ask the audience: * Why `TcLclEnv` and `DsGblEnv` need to share the same IORef? * Is this for efficiency reasons? * Is this because we need the two monads to independently accumulate errors into the same IORef? Thanks! Alfredo

4 10

Type inference of singular matches on GADTs
by Alexis King 31 Mar '21

31 Mar '21

Hi all, Today I was writing some code that uses a GADT to represent heterogeneous lists: data HList as where HNil :: HList '[] HCons :: a -> HList as -> HList (a ': as) This type is used to provide a generic way to manipulate n-ary functions. Naturally, I have some functions that accept these n-ary functions as arguments, which have types like this: foo :: Blah as => (HList as -> Widget) -> Whatsit The idea is that Blah does some type-level induction on as and supplies the function with some appropriate values. Correspondingly, my use sites look something like this: bar = foo (\HNil -> ...) Much to my dismay, I quickly discovered that GHC finds these expressions quite unfashionable, and it invariably insults them: • Ambiguous type variable ‘as0’ arising from a use of ‘foo’ prevents the constraint ‘(Blah as0)’ from being solved. The miscommunication is simple enough. I expected that when given an expression like \HNil -> ... GHC would see a single pattern of type HList '[] and consequently infer a type like HList '[] -> ... Alas, it was not to be. It seems GHC is reluctant to commit to the choice of '[] for as, lest perhaps I add another case to my function in the future. Indeed, if I were to do that, the choice of '[] would be premature, as as ~ '[] would only be available within one branch. However, I do not in fact have any such intention, which makes me quietly wish GHC would get over its anxiety and learn to be a bit more of a risk-taker. I ended up taking a look at the OutsideIn(X) paper, hoping to find some commentary on this situation, but in spite of the nice examples toward the start about the trickiness of GADTs, I found no discussion of this specific scenario: a function with exactly one branch and an utterly unambiguous pattern. Most examples come at the problem from precisely the opposite direction, trying to tease out a principle type from a collection of branches. The case of a function (or perhaps more accurately, a case expression) with only a single branch does not seem to be given any special attention. Of course, fewer special cases is always nice. I have great sympathy for generality. Still, I can’t help but feel a little unsatisfied here. Theoretically, there is no reason GHC cannot treat \(a `HCons` b `HCons` c `HCons` HNil) -> ... and \a b c -> ... almost identically, with a well-defined principle type and pleasant type inference properties, but there is no way for me to communicate this to the typechecker! So, my questions: 1. Have people considered this problem before? Is it discussed anywhere already? 2. Is my desire here reasonable, or is there some deep philosophical argument for why my program should be rejected? 3. If it /is/ reasonable, are there any obvious situations where a change targeted at what I’m describing (vague as that is) would affect programs negatively, not positively? I realize this gets rather at the heart of the typechecker, so I don’t intend to imply a change of this sort should be made frivolously. Indeed, I’m not even particularly attached to the idea that a change must be made! But I do want to understand the tradeoffs better, so any insight would be much appreciated. Thanks, Alexis

6 15

Multiple versions of happy
by Simon Peyton Jones 30 Mar '21

30 Mar '21

What's the approved mechanism to install multiple versions of happy/alex etc? Eg I tried to build ghc-9.0 and got this: checking for makeinfo... no checking for python3... /usr/bin/python3 checking for ghc-pkg matching /opt/ghc/bin/ghc... /opt/ghc/bin/ghc-pkg checking for happy... /home/simonpj/.cabal/bin/happy checking for version of happy... 1.20.0 configure: error: Happy version 1.19 is required to compile GHC. I so I have to 1. Install happy-1.19 without overwriting the installed happy-1.20 2. Tell configure to use happy-1.19 What's the best way to do those two things? Thanks Simon

3 3

Options for targeting Windows XP?
by Clinton Mead 30 Mar '21

30 Mar '21

I'm currently trying to bring my company around to using a bit of Haskell. One issue is that a number of our clients are based in South East Asia and need software that runs on Windows XP. Unfortunately it seems the last version of GHC that produces executables that run on Windows XP is GHC 7.10. Whilst this table <https://gitlab.haskell.org/ghc/ghc/-/wikis/platforms/windows> suggests the issue may only running GHC 8.0+ on Windows XP, I've confirmed that GHC 8.0 executables (even "Hello World") will not run on Windows XP, presumably because a non-XP WinAPI call in the runtime. My first thought would be to restrict myself to GHC 7.10 features (i.e. 2015). This would be a slight annoyance but GHC 7.10 still presents a reasonable language. But my concern would be that increasingly I'll run into issues with libraries that use extensions post GHC 7.10, particularly libraries with large dependency lists. So there's a few options I've considered at this point: 1. Use GHCJS to compile to Javascript, and then dig out a version of NodeJS that runs on Windows XP. GHCJS seems to at least have a compiler based on GHC 8.6. 2. Patch GHC with an additional command line argument to produce XP/Vista compatible executables, perhaps by looking at the changes between 7.10 -> 8.0, and re-introducing the XP approach as an option. The issue with 1 is that is that as well as being limited by how up to date GHCJS is, this will increase install size, memory usage and decrease performance on Windows XP machines, which are often in our environments quite old and resource and memory constrained. Approach 2 is something I'd be willing to put some work into if it was practical, but my thought is that XP support was removed for a reason, presumably by using newer WinAPI functions simplified things significantly. By re-adding in XP support I'd be complicating GHC once again, and GHC will effectively have to maintain two approaches. In addition, in the long term, whenever a new WinAPI call is added one would now have to check whether it's available in Windows XP, and if it's not produce a Windows XP equivalent. That might seem like just an extra burden of support for already busy GHC developers. But on the other hand, if the GHC devs would be happy to merge a patch and keep up XP support this would be the cleanest option. But then I had a thought. If GHC Core isn't supposed to change much between versions is it? Which made me come up with these approaches: 3. Hack up a script to compile programs using GHC 9 to Core, then feed that Core output into GHC 7.10. OR 4. Produce a chimera style GHC by importing the GHC 9.0 API and the GHC 7.10 API, and making a version of GHC that does Haskell -> Core in GHC 9.0 and the rest of the code generation in GHC 7.10. One issue with 4 will be that presumably that because I'm importing GHC 9.0 API and the 7.10 API separately, all their data types will technically be separate, so I'll need to basically deep copy the GHC 9.0 core datatype (and perhaps others) to GHC 7.10 datatypes. But presuming their largely similar this should be fairly mechanical. So are any of these approaches (well, particularly 2 and 4) reasonable? Or am I going to run into big problems with either of them? Is there another approach I haven't thought of? Thanks, Clinton

8 14

GHC 9.2 has branched
by Ben Gamari 29 Mar '21

29 Mar '21

Hello everyone, At this point the ghc-9.2 branch has officially branched off from master. If there was anything you were holding back from `master`, feel free to now send it off to Marge. I'll be working on doing some release prep and pushing out an alpha in the next few days. This alpha will very likely lack the new NCG (which is still being worked on by Moritz) but otherwise should be functionally complete. Cheers, - Ben

1 0

Pattern matching desugaring regression? Re: [Haskell-cafe] Why does my module take so long to compile?
by Carter Schonwald 28 Mar '21

28 Mar '21

Ccing ghc devs since that’s a better forum perhaps Crazy theory: this is a regression due the the partial changes to pattern matching coverage checking in 8.10 that finished / landed in ghc 9 Why: Desugaring is when pattern/ case statement translation happens I think? And the only obvious “big thing” is that you have some huge , albeit sane for a compiler, pattern matching I’d first check if the new ghc 9 release doesn’t have that regression in build time that you experienced. And if it does file a ticket. I may be totally wrong, but that seems like a decent likelihood ! On Mon, Feb 15, 2021 at 12:20 PM Troels Henriksen <athas(a)sigkill.dk> wrote: > Carter Schonwald <carter.schonwald(a)gmail.com> writes: > > > How big are your data types in the file In question? Do you ghc generics > or > > deriving or template Haskell? Could you share a link to the file in > > question ? > > The file does not define any large data types itself, but it operates on > some fairly large data types (an AST, it's a compiler). But so do many > other modules that work just fine. It uses no generics, nontrivial > deriving, or Template Haskell. It's this file: > > https://github.com/diku-dk/futhark/blob/master/src/Futhark/Pass/ExtractKern… > > I also found a handful of other modules in my project that are > significantly slower to compile in GHC 8.10, and seemingly also because > of the desugarer, but none of them have any obvious smoking guns like > generics or TH. > > The only commonality I can find is that the affected modules contain > functions with a relatively large typeclass context. I use > ConstraintKinds to make them more concise, and I guess unfolded there > may be 30-40 equality/class constraints in total. Like this: > > type DistLore lore = > ( Bindable lore, > HasSegOp lore, > BinderOps lore, > LetDec lore ~ Type, > ExpDec lore ~ (), > BodyDec lore ~ () > ) > > where the 'Bindable' constraint in particular then implies a further > handful of "~" constraints: > > class > ( ASTLore lore, > FParamInfo lore ~ DeclType, > LParamInfo lore ~ Type, > RetType lore ~ DeclExtType, > BranchType lore ~ ExtType, > SetType (LetDec lore) > ) => > Bindable lore > where > ... > > FParamInfo/LParamInfo/etc are type families. Are such constraints > particularly slow? > > -- > \ Troels > /\ Henriksen >

4 7