September 2021 - ghc-devs

Re: [Haskell-cafe] Bundle patterns with type aliases
by Carter Schonwald 17 Sep '21

17 Sep '21

These are great ideas! Could you please create a ghc tracker ticket with a tiny examples or two? There may be specific technical reasons we might not be able to do so for type synonyms in ghc, but I don’t see any obvious barriers in the case of David’s excellent idea, I’ve def seen lots of great code out there where you’d really want either associated pattern synonyms or to bundle pattern synonyms with the exported public interface for a type class. I’m sure there’s some devil in the details but these sound lovely. Step -1 is making up 1-2 toy examples and explaining what and why you want it on a ghc ticket! On Wed, Sep 8, 2021 at 1:25 PM David Feuer <david.feuer(a)gmail.com> wrote: > I would like that, along with the ability to bundle patterns with classes. > > On Wed, Sep 8, 2021, 1:13 PM Keith <keith.wygant(a)gmail.com> wrote: > >> Is there currently a way to 'bundle' a pattern with a type alias? And if >> not, could that capability be added to the PatternSynonyms GHC extension? >> (Is this the right place to ask, or should I be asking a GHC list?) >> >> --Keith >> Sent from my phone with K-9 Mail. >> _______________________________________________ >> Haskell-Cafe mailing list >> To (un)subscribe, modify options or view archives go to: >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >> Only members subscribed via the mailman list are allowed to post. > > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post.

2 2

Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
by Simon Peyton Jones 15 Sep '21

15 Sep '21

Alfredo writes (below for full thread) That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer): * Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way); * The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep; * GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.h…<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…> . Yes Haddock doesn't understand Notes but that's a deficiency in Haddock! There so much in GHC that simply does not fit well with the Haddocks attached to a particular data decl or function. We need Notes to explain how all the moving parts fit together, and to point to them. Even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" But I don't think this is better - I think it is significantly worse! In the case you cite, the Haddock is about DiagnosticReason, and mentions Severity only incidentally. I bet that the Haddock for Severity doesn't refer to this. Nor is there a clear "Note [Severity vs DiagnosticReason]" title that bits of code across GHC can refer to by saying "See Note [Severity vs DiagnosticReason]". It's far less satisfactory (to me) than a single Note that * covers just one topic (the difference between Severity and DiagnosticReason, rather than fully describing either * can be pointed to symmetrically from both Severity and DiagnosticReason * can be pointed to by many other bits of code The way it is better is that today's Haddock doesn't understand Notes. But we could fix that if we were minded to. Returning to how to document the error-message architecture, if you'd prefer to use a Note than a wiki page, that's fine. But please write that Overview Note that explains all the pieces, points to them one by one. And then copiously refer to that Note from all those places, so people will update it. Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature". I really don't think that will happen unless there is a Note that explains what the new design is! Lacking this explicit design, everyone will infer their own mental model of how it all works from sundry scattered clues - and those mental models will differ. So instead of one thing "spreading" a dozen subtly different things will spread. And then the next one, confused by these slightly different clues, will be even less coherent. Let's have one, fully-explicit version of The Plan that we constantly refer to. cc'ing ghc-devs because we must constantly question and refine how we describe and document GHC. Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj(a)microsoft.com<mailto:simonpj@microsoft.com> will cease to work. Use simon.peytonjones(a)gmail.com<mailto:simon.peytonjones@gmail.com> instead. (For now, it just forwards to simonpj(a)microsoft.com.) From: Alfredo Di Napoli <alfredo.dinapoli(a)gmail.com> Sent: 26 August 2021 07:25 To: Simon Peyton Jones <simonpj(a)microsoft.com> Cc: rae(a)richarde.dev Subject: Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure Hello Simon! On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> wrote: Alfredo Thanks for all the work you are doing on GHC's error message infrastructure. Your blog post gives a great overview. Thanks, and I am glad you enjoyed it :) As you know I'm very keen for GHC to have a Note or wiki page that gives a solid, up-to-date overview of all the moving parts. (NOT the design alternatives, nor the time sequence; just the outcome.) This is incredibly useful for our future selves; and it helps ensure that people understand (say) the difference between Severity and DiagnosticReason, and use them correctly. So the question is: where is the canonical overview? It could be * Your blog post below. But that is a snapshot... you aren't going to go back to edit it as the design evolves. And it's not in the repo. * The wiki page: https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…>. But it's hard to keep up to date (it was last edited 3 months ago). * Note(s) in the code. We seem to use this increasingly, and it has the great merit of being part of the source code itself. But then we need clear pointer to the canonical overview Notes, and need to make sure they are up to date. I'm not advocating any particular path here... just wanting to be sure that we end up with a good overview somewhere! What is your view? TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some carefully added Notes (and Haddock comments) in GHC might do the trick. That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer): * Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way); * The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep; * GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.h…<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…> . So, in practical terms, I suggest we (I) give the Wiki a little overhaul to add at the top the current design (or anything not captured directly in GHC's source code) and I will keep an eye on the GHC notes and Haddock comments to see if there is anything worth adding. Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature". Hope it helps, and sorry for the long ramble! Alfredo Thanks Simon From: Alfredo Di Napoli via Haskell Community <discourse(a)haskell.org<mailto:discourse@haskell.org>> Sent: 23 August 2021 11:26 To: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Subject: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure [Image removed by sender.] adinapoli<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse…> August 23 [Image removed by sender.]well-typed.com<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-type…> Error! Filename not specified. The new GHC diagnostic infrastructure - Well-Typed: The Haskell Consultants<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-type…> ________________________________ Visit Topic<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse…> or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, click here<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse…>.

6 9

More type safety in Core?
by Ari Fordsham 15 Sep '21

15 Sep '21

I don't know if this is the right forum for this, I apologise if I'm intruding... Are there any plans to use the type system to enforce safety in Core, via e.g. GADTs? This would replace much of core-lint with static checking. Conal Eliottt has done something similar in a blog post ( http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20l…) and it seems relatively straightforward. This would be especially beneficial to those working at the cutting edge of GHC features, statically ensuring their Core manipulations are correct. I would be surprised if existing compiler bugs wouldn't be found while implementing this. What would the performance impact be? would using GADTs incur extra overhead? I'd assume you'd save something by lugging around less type information in Core. Ari Fordsham

5 6

Resuming normal service
by Simon Peyton Jones 13 Sep '21

13 Sep '21

Friends What with the summer holidays and leaving MSR, I've been rather preoccupied and have done almost nothing on GHC for the last six weeks. I'm getting back in action now - I hope. But my GHC backlog is huge. If you are waiting on some response from me, please feel free to ping me. Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj(a)microsoft.com<mailto:simonpj@microsoft.com> will cease to work. Use simon.peytonjones(a)gmail.com<mailto:simon.peytonjones@gmail.com> instead. (For now, it just forwards to simonpj(a)microsoft.com.)

1 0

How to set idScope properly when converting LhsExpr GhcPs to CoreExpr?
by Yiyun Liu 11 Sep '21

11 Sep '21

Hi all, We have a function in Liquidhaskell's code base for converting refinement expressions into GHC's core expressions using the GHC API. Its implementation is based on tcRnExpr <https://gitlab.haskell.org/ghc/ghc/-/blob/ghc-8.10.6-release/compiler/typec…>except we keep the typechecked expression and desugar it before returning. However, our function fails to set the idScope information properly when LiquidHaskell is invoked as a plugin (installed as a typeCheckResultAction). This repo <https://github.com/yiyunliu/ghc-elaboration-test>contains a minimal example that demonstrates the issue. In the CoreExpr returned by elabRnExpr <https://github.com/yiyunliu/ghc-elaboration-test/blob/cb7397884b7949e6b7fb6…>, every symbol from the current module that's being compiled is labeled as local. The README.md file shows how the exported symbol testPlus is marked as local in QQQ.hs, the module where it's defined. Is there a function we should call to set the idScope information properly? Thanks, -Yiyun

2 2

new label: diagnostic quality
by Richard Eisenberg 10 Sep '21

10 Sep '21

Hi devs, I've just created a new label in GitLab, called "diagnostic quality". (Diagnostics = errors ∪ warnings.) I intend for this label to be used when the report is about the quality of an error message or warning. That is, the existing message is not wrong, per se, but it's somehow not as helpful as it can be to users. I think it's helpful to separate out such issues from those describing error messages that are just wrong. In particular, if you spot a ticket arising from https://github.com/haskell/error-messages, you may want to give it the "diagnostic quality" label. If you disagree with the choice to make this label, please push back! Thanks, Richard

2 3

Re: Registering orphan instances and filling the `deps` field of ModIface [RE: Where (else) do I need to register instances from loaded modules?]
by Erdi, Gergo 08 Sep '21

08 Sep '21

PUBLIC Hi, Any other ideas I could try? Gergo -----Original Message----- From: Erdi, Gergo Sent: Friday, August 27, 2021 2:42 PM To: 'Matthew Pickering' <matthewtpickering(a)gmail.com> Cc: 'GHC' <ghc-devs(a)haskell.org>; 'gergo(a)erdi.hu' <gergo(a)erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com> Subject: Registering orphan instances and filling the `deps` field of ModIface [RE: Where (else) do I need to register instances from loaded modules?] PUBLIC Re: #1., I've simplified the code by removing all the Unit shuffling: everything is now in mainUnit. Unfortunately, this doesn't change anything other than cutting a cool 100 lines from the MWE code; I'm attaching the new smaller & simpler version. It is based on GHC 9.0.1 with an extra patch to export the first half of mkIfaceTc, just before calling mkFullIface. With this change, it really should be filling everything in ModIface the same way as the real GHC (so e.g. `deps` should be properly filled), modulo the fingerprints. Unfortunately, the `deps` field of the `ModIface`s created this way still doesn't contain any orphan instance modules: `Class.src`'s deps are empty (which is what I'd expect), `Instance.src` has a dependency in dep_mods (but not in dep_orphs) on `Class {-# SORUCE #-}` (I'm not sure why it's marked as a source import when `Class` module was already loaded into the module providers map before typechecking `Instance`...), and of course `Use.src` still fails so I can't get to its dependencies. It is confusing to me that the problem should be because of the dep_orphs, because `Instance.src` does NOT use any orphan instances from `Class.src; merely, it provides a new orphan instance for other modules to depend on. Can someone explain this part to me? Thanks, Gergo -----Original Message----- From: Erdi, Gergo Sent: Wednesday, August 25, 2021 10:37 AM To: Matthew Pickering <matthewtpickering(a)gmail.com> Cc: GHC <ghc-devs(a)haskell.org>; gergo(a)erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com> Subject: Re: Where (else) do I need to register instances from loaded modules? Hi, Thanks Matt! Unfortunately, I couldn't get it working even with these suggestions. 1. I changed registerModule so that it only changes the HPT when toUnitId (moduleUnit mod) == homeUnitId (hsc_dflags env). But I'm pretty sure this will be vacuously true, since the unit of the module comes from prepareModule setting the home unit before the call to `downsweep`. TBH I don't understand the whole idea behind the home package, and maybe all my pain comes from misusing packages in the first place. As you can see in the original code I've sent, I'm loading modules from different units; in my real use case, I'd even like to do things like load module1 from unit1, then module2 from unit2, then module3 from unit1 again -- basically just using the units as a cheap way to classify modules so that later downstream, I can make decisions based on the unit of the module of the source of a given definition. Should I give up on this idea, is this going to lead to problems later on? 2. I also changed mkModIface to fill in the `deps` and while I'm at it, the `usages` field, by copy-pasting more code from mkIfaceTc. I'm really starting to think that I'd be better off just changing mkIfaceTc to return a PartialModIface... Anyway, the new definitions of `deps` and `usages` are: let pluginModules = map lpModule (cachedPlugins (hsc_dflags hsc_env)) unit_id = homeUnitId (hsc_dflags hsc_env) deps <- mkDependencies unit_id (map mi_module pluginModules) tcg dep_files <- readIORef dependent_files let used_names = mkUsedNames tcg usages <- mkUsageInfo hsc_env this_mod (imp_mods imports) used_names dep_files merged pluginModules With these changes applied, I am still seeing the same error. Printing shows that `dep_orphs deps` is empty for Instance.src; further tracing, in turn, shows that `imp_orphs . tcg_imports $ tcg` is also empty! I have also found the wording in the documentation regarding dep_orphs confusing: it sounds like, in my case, this would mean that Use has a dependency on Instance, because it is using an orphan instance from there. But my problems are way before I am making a ModIface for Use, since it fails already during typechecking. But you saying the problem is the incomplete deps in ModIface must mean that dep_orphs needs to be filled in the ModIface for Instance (or, oh god I hope not, Class), right? If I sound somewhat confused, rest assured that in reality I am even MORE confused! Any further advice welcome, Gergo -----Original Message----- From: Matthew Pickering <matthewtpickering(a)gmail.com> Sent: Tuesday, August 24, 2021 5:16 PM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Cc: GHC <ghc-devs(a)haskell.org>; gergo(a)erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com> Subject: [External] Re: Where (else) do I need to register instances from loaded modules? Hi Gergo, Some things I noticed: * The HPT only contains modules from the current home unit. It seems that registerModule ignores this so you break this invariant. * It seems that `Instance` contains an orphan, but you don't fill in the `deps` field when making a `ModIface`, which should contain all the orphan modules below the module you are compiling. I think that if you fill in `deps` correctly then things may work out. `Dependencies` eventually becomes `ImportAvail`, which gets consulted by `tcVisibleOrphanMods` which is called in `tcGetInstEnvs` which is called by `matchInstEnv`. Matt This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

1 0

New implementation for `ImpredicativeTypes`
by Alejandro Serrano Mena 08 Sep '21

08 Sep '21

Hi all, As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a… you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards, Alejandro

8 23

Question about specialization
by Harendra Kumar 07 Sep '21

07 Sep '21

Hi GHC devs, I have a simple program using the streamly library, as follows, the whole code is in the same module: {-# INLINE iterateState #-} {-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-} iterateState :: MonadState Int m => Int -> SerialT m Int iterateState n = do x <- get if x > n then do put (x - 1) iterateState n else return x main :: IO () main = do State.evalStateT (S.drain (iterateState 0)) 100000 Earlier the SPECIALIZE pragma was not required on iterateState, but after some refactoring in the library (the monad bind of SerialT changed a bit), this program now requires a SPECIALIZE on iterateState to trigger specialization, just INLINE also does not help. My question is whether this may be expected in some conditions or is this something which can be considered a bug in the compiler? I am also curious what specifically could have made the compiler not specialize this anymore, is it the size of the function or some other threshold? Thanks, Harendra

2 2

Potential improvement in compacting GC
by Ömer Sinan Ağacan 03 Sep '21

03 Sep '21

Hello, I recently implemented the algorithm used by GHC's compacting GC in another project. The algorithm (after marking) makes two passes over the heap /generation. In GHC, these passes are implemented in [1] and in the next function. In my implementation I tried 3 ways of implementing these passes, one of which is the same as GHC's code, and benchmarked each version. I found that the fastest implementation is not what's used in GHC, but it could easily be used. I should mention that my code targets Wasm, and I benchmarked Wasm instructions executed. Not CPU cycles, CPU instructions, or anything like that. It's very likely that the results will be different when benchmarking code running on actual hardware. Secondly, in my case the heap is mostly dead (residency is low). In GHC, compaction for the oldest generation is enabled when residency passes a threshold, so the assumption is the heap is mostly live. I'm guessing this should also make some difference. Anyway, the first implementation I tried was similar to GHC's scan, but I did scan += object_size(scan); instead of bumping scan by one, as GHC does in [2]. This was the slowest version. Second implementation did the same as GHC (bumped scan by one). This was faster, but still slower than the next version. What I found to be the best is scanning the bitmap, not the heap. The bitmap iterator reads one word at a time. In each iteration it checks if the bitmap word is 0. In GHC, in the best case this can skip 16 words on heap on 32-bit systems, and 32 words on 64-bit. Reminder: we need two bits per object in the bitmap, see [3]. (this is not the case in my implementation so the payoff is better) When the bitmap word is not 0 I use "count trailing zeros" (or "count leading zeros" depending on the bitmap implementation) to get the number of words to skip. This is a single instruction on Wasm and x86 (TZCNT or LZCNT, available via __builtin_ctzl and __builtin_clzl in gcc). So instead of skipping one word at a time, this can potentially skip 16 words (or 32 on 64-bit architectures). When that's not possible, it can still skip multiple words by using ctz/clz. Ömer [1]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt… [2]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt… [3]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt…

5 6