July 2022 - ghc-devs - Haskell.org

Implementing a compilation server
by Domínguez, Facundo 11 Oct '22

11 Oct '22

Dear ghc devs, I'm using the ghc API to write a compilation server (a.k.a. persistent worker). The idea is to serve requests to compile individual modules. In this fashion, we can compile modules with different compilation flags and yet pay only once for the startup overheads of the compiler. One challenge of this approach is to reuse as much as possible from the ghc API session/environment from one compilation request to the next, so we save the trouble of reconstructing it each time. This message is to ask for advise on how to better accomplish this reuse. I tried reusing the whole environment for multiple requests, but I'm conjecturing that this might cause troubles when the requests require building modules with different values of -this-unit-id. Another problem that streams from this is that recompiling a module which defines a type class instance fails because it encounters in the environment the type class instance from the previous compilation. My work-in-progress implementation is here [1]. There appears to be multiple ways to compile a module in the API, so far I have been trying DriverPipeline.compileFile. My best lead right now is to look for inspiration in how GHCi implements the load command, but this does a sort of --make compilation while I'm going here for the one-shot style. Thanks in advance, Facundo [1] https://github.com/tweag/rules_haskell/blob/16ba422457ea4daa5dbf40d46327ebc…

2 3

Question about ambiguous predicates in pattern bindings
by Benjamin Redelings 16 Aug '22

16 Aug '22

Hi, 1. I'm reading "A Static semantics for Haskell" and trying to code it up. I came across some odd behavior with pattern bindings, and I was wondering if someone could explain or point me in the right direction. Suppose you have the declaration (x,y) = ('a',2) My current code is yielding: x :: Num a => Char y :: Num a => a However, I notice that ghci gives x the type Char, with no constraint, which is what I would expect. It also gives y the type 'Num b => b', so I don't think it is defaulting a to Int here. The weird results from my code stem from rule BIND-PRED in Figure 15 of https://homepages.inf.ed.ac.uk/wadler/papers/staticsemantics/static-semanti… E |- bind ~~> \dicts : theta -> monobinds in bind : (LIE_{encl}, theta => LVE) Here theta = ( Num a ) and LVE = { x :: Char, y :: a }. So, theta => LVE is { x :: Num a => Char, y :: Num a => a } The obvious thing to do is avoid changing a type T to Num a => T if T does not contain a. Also I'm not totally sure if that trick gets to the bottom of the issue. However, the paper doesn't mention define theta => LVE that way. Is there something else I should read on this? 2. If we just chop out predicates which mention variables not in the type ( == ambiguous predicates?) then I'm not totally sure how to create code for this. I would imagine that we have something like tup dn = ('a', fromInteger dn 2) x = case (tup dn) of (x,y) -> x y dn case (tup dn) of (x,y) -> y In this case its not clear where to get the `dn` argument of `tup` from, in the definition of `x`. Can we pass in `undefined`? Should we do something else? If anyone can shed light on this, I would be grateful :-) -BenRI

4 8

Gitlab labels
by Simon Peyton Jones 05 Aug '22

05 Aug '22

Whenever I try to add a label to a ticket I get "an error occurred while updating labels". See below. When I repeat the exact same sequence, it works. If I add another label, again the error; repeating again works. This is tiresome and peculiar. Does anyone have any ideas? Simon [image: image.png]

4 3

Re: Wildcards in type synonyms
by ÉRDI Gergő 03 Aug '22

03 Aug '22

(TL;DR: `newMetaTyVarX` gives me type metavars that behave weirdly and I don't understand why. What shoudl I use instead?) OK so I have two half-done implementations now: * Doing `HsType`-level substitution in the middle of `tc_infer_hs_type` (see my exchange with Richard on why this needs to happen in `tc_infer_hs_type` instead of `rnHsTyKi`) * Doing Core `Type`-level substitution in the middle of `tc_infer_hs_type` The advantage of the first one is that it works :) The disadvantage is that it involves storing a `HsType` in a `TyCon`, which in turn means making it work inter-module will require an `Iface` representation for `HsType`s. Hence the second attempt. I think that would be a more principled solution anyway. This approach is based on typechecking the macro's right-hand side into a core `Type`, and storing that, and the list of wildcard-originating `TyVar`s, in the `TyCon`. At every occurrence site, I take this core `Type` and apply a substitution on it that is the composition of the following two: * A substitution from macro type synonym type parameters to the type arguments * An instantiation of each wildcard variable into a fresh metavariable Unfortunately, it is this second step that is tripping me up. If I use `newMetaTyVarX` to make these "refreshing" metavars, then while the substitution looks OK when eyeballing it, the resulting *type* metavariables seem to be handled by GHC as if they were *kind* metavariables?! Here's an example. The source input is: ``` {-# LANGUAGE NoPolyKinds, NoStarIsType #-} -- Makes it easier to see how it goes wrong data MyData a b c = MkMyData a b c type MySyn a = MyData a _ Int f1 :: MyData a b c -> b f1 (MkMyData _ x _) = x f2 :: MySyn a -> Double f2 = f1 ``` I start with the following "macro type template" (using `-dppr-debug` format): ``` TySynWildcard.MyData{tc r3} (a{tv auq} Nothing [sk:1] :: GHC.Types.Type{(w) tc 32Q}) ((w_awX{tv} Nothing [tau:0] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})) |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I} GHC.Types.Type{(w) tc 32Q} GHC.Types.Type{(w) tc 32Q} (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}) GHC.Types.Type{(w) tc 32Q})}) GHC.Types.Int{(w) tc 3u} ``` The substitution applied: ``` [TCvSubst In scope: InScope {a{tv auu} k_awW{tv} w_axc{tv}} Type env: [auq :-> (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q})), awX :-> (w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))] Co env: []] ``` Note that the second type substitution, (w_awX :: k_awW) :-> (w_axc :: k_awW) is the one that should take care of instantiating the wildcard metavariable. And the result of applying this substitution still looks OK: ``` TySynWildcard.MyData{tc r3} (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q})) ((w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})) |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I} GHC.Types.Type{(w) tc 32Q} GHC.Types.Type{(w) tc 32Q} (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}) GHC.Types.Type{(w) tc 32Q})}) GHC.Types.Int{(w) tc 3u} ``` But soon after, typechecking fails: ``` • Couldn't match type ‘Type’ with ‘Double’ Expected: MyData a Type Int -> Double Actual: MyData a Type Int -> Type • In the expression: f1 In an equation for ‘f2’: f2 = f1 ``` So this is weird. Instead of unification solving `w_axc ~ Double`, it seems `w_axc` is left unrestricted, and then `NoPolyKinds` picks it up as a kind variable (why?) and defaults it to `Type`. As an experiment, I have also tried *not* refreshing `w_awX`, only substituting in the type arguments. Now, of course, this can't possibly work as soon as I have more than one occurrence of `MySyn` due to the interference between the wildcard metavars, but if I only have one, then the program typechecks. So to me this suggests I'm doing things mostly right, except that the metavar returned by `newMetaTyVarX` is not fit for my use case. What should I use instead of `newMetaTyVarX` to instantiate / "refresh" the (wildcard-originating) type metavariables in my "macro type template"? Thanks, Gergo On Mon, 25 Jul 2022, Simon Peyton Jones wrote: > I'm afraid I don't understand, but it sounds delicate. By all means try! > > Simon > > On Mon, 25 Jul 2022 at 11:04, ÉRDI Gergő <gergo(a)erdi.hu> wrote: > On Mon, 25 Jul 2022, Simon Peyton Jones wrote: > > > Do we have an existing way of substituting types over type > variables, *in > > HsType instead of Core Type*? > > > > > > I'm afraid not. Currently HsType is not processed much -- just renamed > and typechecked > > into a Type. > > I wonder if, instead, I could expand the rhs, typecheck it "abstractly" > (i.e. in the context of the synonym's binders), and THEN do the > substitution. If I typecheck the rhs for every occurrence, I should get > fresh metavars for each wildcard, which is pretty much what I want. I just > have to make sure I don't zonk before the substitution. > > Does this make sense? > > >

2 4

[ANNOUNCE] GHC 9.4.1-rc1 is now available
by Ben Gamari 02 Aug '22

02 Aug '22

The GHC developers are happy to announce the availability of the first (and likely last) release candidate of GHC 9.4.1. Binary distributions, source distributions, and documentation are available at [downloads.haskell.org] This major release will include: - A new profiling mode, `-fprof-late`, which adds automatic cost-center annotations to all top-level functions *after* Core optimisation has run. This provides informative profiles while interfering significantly less with GHC's aggressive optimisations, making it easier to understand the performance of programs which depend upon simplification.. - A variety of plugin improvements including the introduction of a new plugin type, *defaulting plugins*, and the ability for typechecking plugins to rewrite type-families. - An improved constructed product result analysis, allowing unboxing of nested structures, and a new boxity analysis, leading to less reboxing. - Introduction of a tag-check elision optimisation, bringing significant performance improvements in strict programs. - Generalisation of a variety of primitive types to be levity polymorphic. Consequently, the `ArrayArray#` type can at long last be retired, replaced by standard `Array#`. - Introduction of the `\cases` syntax from [GHC proposal 0302] - A complete overhaul of GHC's Windows support. This includes a migration to a fully Clang-based C toolchain, a deep refactoring of the linker, and many fixes in WinIO. - Support for multiple home packages, significantly improving support in IDEs and other tools for multi-package projects. - A refactoring of GHC's error message infrastructure, allowing GHC to provide diagnostic information to downstream consumers as structured data, greatly easing IDE support. - Significant compile-time improvements to runtime and memory consumption. - On overhaul of our packaging infrastructure, allowing full traceability of release artifacts and more reliable binary distributions. - Reintroduction of deep subsumption (which was previously dropped with the *simplified subsumption* change) as a language extension. - ... and much more. See the [release notes] for a full accounting. Note that, as 9.4.1 is the first release for which the released artifacts will all be generated by our Hadrian build system, it is possible that there will be packaging issues. If you enounter trouble while using a binary distribution, please open a [ticket]. Likewise, if you are a downstream packager, do consider migrating to [Hadrian] to run your build; the Hadrian build system can be built using `cabal-install`, `stack`, or the in-tree [bootstrap script]. We will be publishing a blog post describing the migration process to Hadrian in the coming weeks. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy testing, - Ben [downloads.haskell.org] https://downloads.haskell.org/ghc/9.4.1-rc1/ [GHC proposal 0302]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-c… [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… [Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… [release notes]: https://downloads.haskell.org/~ghc/9.4.1-rc1/docs/users_guide/9.4.1-notes.h…

5 11

[Haskell] [ANNOUNCE] GHC 9.2.4 released
by Zubin Duggal 28 Jul '22

28 Jul '22

The GHC developers are very happy to at announce the availability of GHC 9.2.4. Binary distributions, source distributions, and documentation are available at [`downloads.haskell.org`](https://downloads.haskell.org/ghc/9.2.4). Download Page: https://www.haskell.org/ghc/download_ghc_9_2_4.html Blog Post: https://www.haskell.org/ghc/blog/20220728-ghc-9.2.4-released.html This release will include: - The new `DeepSubsumption` language extension which reverses the effects of the [Simplified Subsumption Proposal] introduced in GHC 9.0. This is an attempt to make GHC 9.2.4 more backwards compatible with GHC 8.10 and eases migration for users who depended on this feature. This extension is enabled by default with the `Haskell2010` and `Haskell98` languages but disabled with the `GHC2021` language originally introduced in GHC 9.2.1. See the [Deep Subsumption Proposal] for more details. - Fixes for segfaults that may arise due to a bug in the implementation of the `keepAlive#` primop. This may regress performance for certain programs which use this primop or functions which use the primop, such as `withForeignPtr`. These regressions are mostly small, but can be larger in certain edge cases. Judicious use of `unsafeWithForeignPtr` when its argument is known not to statically diverge can mitigate these in many cases. It is our judgment that the critical correctness issues justify the regression in performance and that it is important to get a release out with the fix while we work on a better approach which will improve performance for future releases (#21708). We have a [wiki page](https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator) that tracks possible solutions to this problem, and Ben wrote a [blog post](https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html) detailing the introduction of the `keepAlive#` primop and its history. - Fixes for a number of miscompilations on AArch64 and other platforms (#21624, #21773, #20735, #21685). - Fixes for segfaults due to bugs in the RTS and GC (#21708, #21880, #21885). - Fixing the behaviour of Ctrl-C with GHCi on Windows (#21889). - ... and much more. See the [release notes] for a full accounting. As some of the fixed issues do affect correctness users are encouraged to upgrade promptly. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy compiling, - Zubin [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [release notes]: https://downloads.haskell.org/~ghc/9.2.4/docs/users_guide/9.2.4-notes.html [Simplified Subsumption Proposal]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-s… [Deep Subsumption Proposal]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0511-d…

3 2

Test suite
by Simon Peyton Jones 28 Jul '22

28 Jul '22

Doing *hadrian/build test* takes absolutely ages. It seems that it is building haddock, running haddock, building check_exact, and other things. Eg right now it is doing Run Haddock BuildPackage: libraries/parsec/src/Text/Parsec.hs (and 24 more) => _build/doc/html/libraries/parsec-3.1.15.0/parsec.haddock But I didn't ask to do Haddock. I just wanted to run the testsuite. How can I do that? I would prefer never to build haddock. Thanks Simon

3 3

Wildcards in type synonyms
by ÉRDI Gergő 26 Jul '22

26 Jul '22

Hi, I'd like to implement type synonyms containing wildcards. The idea is that if you have `type MySyn a = MyType a _ Int`, then during typechecking, every occurrence of `MySyn T` would be expanded into `MyType T w123 Int`, with a fresh type (meta)variable `w123`. One worrying thing I noticed in my initial exploration of the GHC codebase is that the Core representation of `Type`s can still contain type synonym occurrences. And this doesn't seem like just an artefact of sharing the `Type` representation with `TcType`, since the `coreView` function also has code to look through type synonyms. What is the reason for this? I would have expected type synonyms to be only relevant during typechecking, and then fully resolved in the elaborated Core output. If that were the case, then a new version of `expand_syn` could live in `TcM` and take care of making these fresh metavariables. Beside this concrete question, taking a step back, I would also like to hear from people who know their way around this part of GHC, what they think about this and how they'd approach implementing it. Thanks, Gergo

5 14

GHC development asks too much of the host system
by Hécate 20 Jul '22

20 Jul '22

Hello ghc-devs, I hadn't made significant contributions to the GHC code base in a while, until a few days ago, where I discovered that my computer wasn't able to sustain running the test suite, nor handle HLS well. Whether it is my OS automatically killing the process due to oom-killer or just the fact that I don't have a war machine, I find it too bad and I'm frankly discouraged. This is not the first time such feedback emerges, as the documentation task force for the base library was unable to properly onboard some people from third-world countries who do not have access to hardware we'd consider "standard" in western Europe or some parts of North America. Or at least "standard" until even my standard stuff didn't cut it anymore. So yeah, I'll stay around but I'm afraid I'm going to have to focus on projects for which the feedback loop is not on the scale of hours , as this is a hobby project. Hope this will open some eyes. Cheers, Hécate -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD

10 19

Hadrian problem
by Simon Peyton Jones 12 Jul '22

12 Jul '22

I'm in a GHC tree, built with Hadrian, I'm getting this red problem. But compilation has got way past compiling base. why is it looking in my .ghc/... directory? It should be looking in my build tree. Simon bash$ ~/code/HEAD-1/_build/ghc-stage1 -c Foo.hs Loaded package environment from /home/simonpj/.ghc/x86_64-linux-9.5.20220628/environments/default <command line>: cannot satisfy -package-id base-4.17.0.0 (use -v for more information) bash$ cat ~/code/HEAD-1/_build/ghc-stage1 "/home/simonpj/code/HEAD-1/_build/stage0/bin/ghc" "-no-global-package-db" "-package-db /home/simonpj/code/HEAD-1/_build/stage1/lib/package.conf.d" "$@"

3 2