August 2022 - ghc-devs

Recompilation avoidance on ABI hash changes
by Domínguez, Facundo 11 Aug '22

11 Aug '22

Dear devs, rules_haskell [1] recently earned the ability to skip recompiling modules when the ABI hashes of dependencies don't change in interface files. We have noticed since then that sometimes ghc --make does avoid rebuilding even when ABI hashes change. However, so far I've been unable to reproduce this in the small. The wiki page on recompilation avoidance does remark that a changing ABI hash is a necessary but not sufficient condition for recompilation [2]. Does anyone have a small example showing how ghc avoids rebuilding even when the ABI hash of a dependency changes? Thanks in advance! Facundo [1] https://github.com/tweag/rules_haskell/issues/1760#issuecomment-1179102232 [2] https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/recompilatio…

2 2

GHC 9.4 symlink wrappers installed incorrectly on FreeBSD 12
by Viktor Dukhovni 10 Aug '22

10 Aug '22

I just built a GHC 9.4 bindist on FreeBSD 12, and found that the bindist Makefile installed unusable wrappers when the source was a symlink. It seems plausible that the issue may not be FreeBSD-specific, but that's all I've tried so far. The problem wrappers do not have execute permissions, and lack the variable settings at the top of the scripts needed to make them work: $ ls -l ~/.local/ghc-9.4/bin total 98 -rw-r--r-- 1 viktor viktor 45 Aug 10 21:46 ghc -rwxr-xr-x 1 viktor viktor 406 Aug 10 21:46 ghc-9.4.1 -rw-r--r-- 1 viktor viktor 97 Aug 10 21:46 ghc-pkg -rwxr-xr-x 1 viktor viktor 466 Aug 10 21:46 ghc-pkg-9.4.1 -rw-r--r-- 1 viktor viktor 67 Aug 10 21:46 ghci -rwxr-xr-x 1 viktor viktor 430 Aug 10 21:46 ghci-9.4.1 -rw-r--r-- 1 viktor viktor 57 Aug 10 21:46 haddock -rwxr-xr-x 1 viktor viktor 434 Aug 10 21:46 haddock-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 33 Aug 10 21:46 hp2ps -rwxr-xr-x 1 viktor viktor 406 Aug 10 21:46 hp2ps-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 33 Aug 10 21:46 hpc -rwxr-xr-x 1 viktor viktor 402 Aug 10 21:46 hpc-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 734 Aug 10 21:46 hsc2hs -rwxr-xr-x 1 viktor viktor 1109 Aug 10 21:46 hsc2hs-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 50 Aug 10 21:46 runghc -rwxr-xr-x 1 viktor viktor 417 Aug 10 21:46 runghc-9.4.1 -rw-r--r-- 1 viktor viktor 50 Aug 10 21:46 runhaskell -rwxr-xr-x 1 viktor viktor 425 Aug 10 21:46 runhaskell-9.4.1 $ (cd ~/.local/ghc-9.4; grep . $(find bin ! -name hsc2hs ! -perm -001)) bin/ghci:executable="$bindir/ghc-9.4.1" bin/ghci:exec $executable --interactive "$@" bin/ghc:exec "$executablename" -B"$libdir" ${1+"$@"} bin/haddock:exec "$executablename" -B"$libdir" -l"$libdir" ${1+"$@"} bin/hp2ps:exec "$executablename" ${1+"$@"} bin/ghc-pkg:PKGCONF="$libdir/package.conf.d" bin/ghc-pkg:exec "$executablename" --global-package-db "$PKGCONF" ${1+"$@"} bin/runghc:exec "$executablename" -f "$exedir/ghc" ${1+"$@"} bin/hpc:exec "$executablename" ${1+"$@"} bin/runhaskell:exec "$executablename" -f "$exedir/ghc" ${1+"$@"} The same holds for "hsc2hs", but the "meat" of the script is longer so I chose to skip it. -- Viktor.

1 1

Re: A macOS static linking mystery
by Ryan Scott 10 Aug '22

10 Aug '22

I should clarify that I'm using a borrowed macOS on which I don't have admin privileges, so I'm unable to install pkg-config. As a result, I'm commenting out the pkgconfig-depends: line in libffi.cabal and manually specifying the extra-lib-dirs and include-dirs via a cabal.project.local file. I've configured the project to use both the macOS system libffi (located at /usr/lib/libffi.dylib) as well as the version that Homebrew provides. Both configurations exhibit the strange, apparently-statically-linked behavior. Even stranger, the macOS system libffi _only_ provides a .dylib file, not an .a file. If I examine the x86 code of the compiled executable using `objdump -d`, I can see various libffi definitions (e.g., _ffi_call), so perhaps these definitions are being inlined through some kind of link-time optimization? I'm really not sure how else to explain what's happening. Ryan

3 8

[ANNOUNCE] GHC 9.4.1 is now available
by Ben Gamari 09 Aug '22

09 Aug '22

The GHC developers are very pleased to announce the availability of GHC 9.4.1. Binary distributions, source distributions, and documentation are available at downloads.haskell.org: https://downloads.haskell.org/ghc/9.4.1 This release includes: - A new profiling mode, `-fprof-late`, which adds automatic cost-center annotations to all top-level functions *after* Core optimisation has run. This provides informative profiles while interfering significantly less with GHC's aggressive optimisations, making it easier to understand the performance of programs which depend upon simplification.. - A variety of plugin improvements including the introduction of a new plugin type, *defaulting plugins*, and the ability for typechecking plugins to rewrite type-families. - An improved constructed product result analysis, allowing unboxing of nested structures, and a new boxity analysis, leading to less reboxing. - Introduction of a tag-check elision optimisation, bringing significant performance improvements in strict programs. - Generalisation of a variety of primitive types to be levity polymorphic. Consequently, the `ArrayArray#` type can at long last be retired, replaced by standard `Array#`. - Introduction of the `\cases` syntax from [GHC proposal 0302]. - A complete overhaul of GHC's Windows support. This includes a migration to a fully Clang-based C toolchain, a deep refactoring of the linker, and many fixes in WinIO. - Support for multiple home packages, significantly improving support in IDEs and other tools for multi-package projects. - A refactoring of GHC's error message infrastructure, allowing GHC to provide diagnostic information to downstream consumers as structured data, greatly easing IDE support. - Significant compile-time improvements to runtime and memory consumption. - On overhaul of our packaging infrastructure, allowing full traceability of release artifacts and more reliable binary distributions. - Reintroduction of deep subsumption (which was previously dropped with the *simplified subsumption* change) as a language extension. - ... and much more. See the [release notes] for a full accounting. Note that, as 9.4.1 is the first release for which the released artifacts will all be generated by our Hadrian build system, it is possible that there will be packaging issues. If you enounter trouble while using a binary distribution, please open a [ticket]. Likewise, if you are a downstream packager, do consider migrating to [Hadrian] to run your build; the Hadrian build system can be built using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See the accompanying [blog post] for details on migrating packaging to Hadrian. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy testing, - Ben [GHC proposal 0302]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-c… [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… [Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… [release notes]: https://downloads.haskell.org/~ghc/9.4.1/docs/users_guide/9.4.1-notes.html [blog post]: https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html

1 1

ghc doesn't work after installing 9.4.1 on my Mac
by George Colpitts 09 Aug '22

09 Aug '22

Unfortunately ghc doesn't work after installing 9.4.1 on my Mac. Does it work for others? After the install finishes when I do the following: $ ghc --version bash: /usr/local/bin/ghc: Permission denied $ sudo chmod +x /usr/local/bin/ghc $ghc --version /usr/local/bin/ghc: line 1: exec: : not found $ cat /usr/local/bin/ghc exec "$executablename" -B"$libdir" ${1+"$@"} $ cat /usr/local/bin/ghc-9.4.1 #!/bin/sh exedir="/usr/local/lib/ghc-9.4.1/bin" exeprog="ghc-9.4.1" executablename="/usr/local/lib/ghc-9.4.1/bin/ghc-9.4.1" bindir="/usr/local/bin" libdir="/usr/local/lib/ghc-9.4.1/lib" docdir="/usr/local/share/doc/ghc-9.4.1" includedir="/usr/local/include" exec "$executablename" -B"$libdir" ${1+"$@"} Thanks George On Sun, Aug 7, 2022 at 6:30 PM Ben Gamari <ben(a)well-typed.com> wrote: > The GHC developers are very pleased to announce the availability of GHC > 9.4.1. Binary distributions, source distributions, and documentation are > available at downloads.haskell.org: > > https://downloads.haskell.org/ghc/9.4.1 > > This release includes: > > - A new profiling mode, `-fprof-late`, which adds automatic cost-center > annotations to all top-level functions *after* Core optimisation has > run. This provides informative profiles while interfering > significantly less with GHC's aggressive optimisations, making it > easier to understand the performance of programs which depend upon > simplification.. > > - A variety of plugin improvements including the introduction of a new > plugin type, *defaulting plugins*, and the ability for typechecking > plugins to rewrite type-families. > > - An improved constructed product result analysis, allowing unboxing of > nested structures, and a new boxity analysis, leading to less reboxing. > > - Introduction of a tag-check elision optimisation, bringing > significant performance improvements in strict programs. > > - Generalisation of a variety of primitive types to be levity > polymorphic. Consequently, the `ArrayArray#` type can at long last be > retired, replaced by standard `Array#`. > > - Introduction of the `\cases` syntax from [GHC proposal 0302]. > > - A complete overhaul of GHC's Windows support. This includes a > migration to a fully Clang-based C toolchain, a deep refactoring of > the linker, and many fixes in WinIO. > > - Support for multiple home packages, significantly improving support > in IDEs and other tools for multi-package projects. > > - A refactoring of GHC's error message infrastructure, allowing GHC to > provide diagnostic information to downstream consumers as structured > data, greatly easing IDE support. > > - Significant compile-time improvements to runtime and memory consumption. > > - On overhaul of our packaging infrastructure, allowing full > traceability of release artifacts and more reliable binary > distributions. > > - Reintroduction of deep subsumption (which was previously dropped with > the > *simplified subsumption* change) as a language extension. > > - ... and much more. See the [release notes] for a full accounting. > > Note that, as 9.4.1 is the first release for which the released artifacts > will > all be generated by our Hadrian build system, it is possible that there > will be > packaging issues. If you enounter trouble while using a binary > distribution, > please open a [ticket]. Likewise, if you are a downstream packager, do > consider > migrating to [Hadrian] to run your build; the Hadrian build system can be > built > using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See the > accompanying > [blog post] for details on migrating packaging to Hadrian. > > We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, > Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and > other anonymous contributors whose on-going financial and in-kind support > has > facilitated GHC maintenance and release management over the years. Finally, > this release would not have been possible without the hundreds of > open-source > contributors whose work comprise this release. > > As always, do give this release a try and open a [ticket] if you see > anything amiss. > > Happy testing, > > - Ben > > > [GHC proposal 0302]: > https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-c… > [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new > [bootstrap script]: > https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… > [Hadrian]: > https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… > [release notes]: > https://downloads.haskell.org/~ghc/9.4.1/docs/users_guide/9.4.1-notes.html > [blog post]: > https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html > _______________________________________________ > ghc-devs mailing list > ghc-devs(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >

2 1

A macOS static linking mystery
by Ryan Scott 08 Aug '22

08 Aug '22

I'm trying to diagnose strange GHC linking behavior which, as far as I can tell, only occurs on macOS. I feel a bit out of my league debugging this, so I'm hoping someone knows what is causing this. When building the Haskell libffi library [1], it will link pass -lffi to GHC. Typically, most installations of libffi will include both a static library (libffi.a) and a shared library (libffi.{so,dylib,dll}). My impression is that when the linker finds both, it will default to the shared library. This is definitely the case on Linux and Windows, at least. An exception to this rule is macOS, however. On macOS, building libffi always appears to default to linking against the static version of libffi, even when a dynamic version is also available. To reproduce this phenomenon, check out libffi [1] and run the following commands: $ brew install libffi # If it is not already installed $ cabal build ctime $ otool -L $(cabal list-bin ctime) The output reveals that the compiled executable does not dynamically link against libffi.dylib: $ otool -L $(cabal list-bin ctime) /Users/rscott/Documents/Hacking/Haskell/libffi/dist-newstyle/build/x86_64-osx/ghc-8.10.7/libffi-examples-0.1/x/ctime/build/ctime/ctime: /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) /usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0) This is exceedingly strange, since my Hombrew installation does in fact provide libffi.dylib: $ ls -alh ~/Software/homebrew/Cellar/libffi/3.4.2/lib/ total 232 drwxr-xr-x 6 rscott 1340850540 192B Aug 7 09:38 . drwxr-xr-x 11 rscott 1340850540 352B Aug 7 08:51 .. -rw-r--r-- 1 rscott 1340850540 70K Aug 7 08:51 libffi.8.dylib -r--r--r-- 1 rscott 1340850540 41K Jun 28 2021 libffi.a lrwxr-xr-x 1 rscott 1340850540 14B Jun 28 2021 libffi.dylib -> libffi.8.dylib drwxr-xr-x 3 rscott 1340850540 96B Aug 7 08:51 pkgconfig What's more, this only seems to happen when building the libffi library in particular. If I try building another library that has a C extra-libraries dependency, such as HsOpenSSL, then I can find its dynamic dependencies (libssl.dylib and libcrypto.dylib) using otool: $ otool -L dist-newstyle/build/x86_64-osx/ghc-8.10.7/HsOpenSSL-0.11.7.2/build/test-dsa/test-dsa dist-newstyle/build/x86_64-osx/ghc-8.10.7/HsOpenSSL-0.11.7.2/build/test-dsa/test-dsa: /usr/local/opt/openssl(a)1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0) /usr/local/opt/openssl(a)1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) /usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0) I'm at a bit of a loss trying to figure out why this only happens for libffi, and only on macOS. To make things even stranger, this only seems to happen when using GHC. If I try to compile a simple libffi C program using -lffi, for instance, then otool finds libffi.dylib. Is there something about GHC's linking behavior that would cause this? Ryan S. ----- [1] https://github.com/remiturk/libffi

3 3

Re: [Haskell-cafe] [ANNOUNCE] GHC 9.4.1 is now available
by Ben Gamari 08 Aug '22

08 Aug '22

Bruno Damour <bruno(a)ruomad.net> writes: > Hello, > Thanks for this new release ! > Do you plan to add FreeBSD binaries ? Yes, I have recently been working on the FreeBSD CI infrastructure [1] and hope to have this finished in time for 9.4.2. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/6318

1 0

Gitlab labels
by Simon Peyton Jones 05 Aug '22

05 Aug '22

Whenever I try to add a label to a ticket I get "an error occurred while updating labels". See below. When I repeat the exact same sequence, it works. If I add another label, again the error; repeating again works. This is tiresome and peculiar. Does anyone have any ideas? Simon [image: image.png]

4 3

Mixed boxed/unboxed arrays?
by J. Reinders 03 Aug '22

03 Aug '22

Hi GHC devs, I’ve been investigating fast hash table implementations. In particular hash tables used for counting unique items. For this use case, I believe the most performant hash tables are, in C terms, arrays of structures with a (boxed) pointer to the key, which is the item that we are counting, and an (unboxed) integer which holds the actual count. I already know of the ‘vector-hashtables’ package which uses two separate arrays, for example one boxed to hold the keys and one unboxed to hold the counts. However, I believe it can be quite important to store all the elements in the same array as that can reduce the number of cache misses. Because with random access to two arrays there is a higher chance that there will be two cache misses even if it immediately finds the right key in the hash table. So, I have also been looking at the low level arrays from the ‘primitive’ package and even in GHC.Exts, but I don’t believe it is currently possible to create a single array that contains both boxed and unboxed elements. Have I overlooked something? Or else, would it be possible to support this use case in a future version of GHC? Cheers, Jaro

4 10

Re: Wildcards in type synonyms
by ÉRDI Gergő 03 Aug '22

03 Aug '22

(TL;DR: `newMetaTyVarX` gives me type metavars that behave weirdly and I don't understand why. What shoudl I use instead?) OK so I have two half-done implementations now: * Doing `HsType`-level substitution in the middle of `tc_infer_hs_type` (see my exchange with Richard on why this needs to happen in `tc_infer_hs_type` instead of `rnHsTyKi`) * Doing Core `Type`-level substitution in the middle of `tc_infer_hs_type` The advantage of the first one is that it works :) The disadvantage is that it involves storing a `HsType` in a `TyCon`, which in turn means making it work inter-module will require an `Iface` representation for `HsType`s. Hence the second attempt. I think that would be a more principled solution anyway. This approach is based on typechecking the macro's right-hand side into a core `Type`, and storing that, and the list of wildcard-originating `TyVar`s, in the `TyCon`. At every occurrence site, I take this core `Type` and apply a substitution on it that is the composition of the following two: * A substitution from macro type synonym type parameters to the type arguments * An instantiation of each wildcard variable into a fresh metavariable Unfortunately, it is this second step that is tripping me up. If I use `newMetaTyVarX` to make these "refreshing" metavars, then while the substitution looks OK when eyeballing it, the resulting *type* metavariables seem to be handled by GHC as if they were *kind* metavariables?! Here's an example. The source input is: ``` {-# LANGUAGE NoPolyKinds, NoStarIsType #-} -- Makes it easier to see how it goes wrong data MyData a b c = MkMyData a b c type MySyn a = MyData a _ Int f1 :: MyData a b c -> b f1 (MkMyData _ x _) = x f2 :: MySyn a -> Double f2 = f1 ``` I start with the following "macro type template" (using `-dppr-debug` format): ``` TySynWildcard.MyData{tc r3} (a{tv auq} Nothing [sk:1] :: GHC.Types.Type{(w) tc 32Q}) ((w_awX{tv} Nothing [tau:0] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})) |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I} GHC.Types.Type{(w) tc 32Q} GHC.Types.Type{(w) tc 32Q} (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}) GHC.Types.Type{(w) tc 32Q})}) GHC.Types.Int{(w) tc 3u} ``` The substitution applied: ``` [TCvSubst In scope: InScope {a{tv auu} k_awW{tv} w_axc{tv}} Type env: [auq :-> (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q})), awX :-> (w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))] Co env: []] ``` Note that the second type substitution, (w_awX :: k_awW) :-> (w_axc :: k_awW) is the one that should take care of instantiating the wildcard metavariable. And the result of applying this substitution still looks OK: ``` TySynWildcard.MyData{tc r3} (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q})) ((w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})) |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I} GHC.Types.Type{(w) tc 32Q} GHC.Types.Type{(w) tc 32Q} (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}) GHC.Types.Type{(w) tc 32Q})}) GHC.Types.Int{(w) tc 3u} ``` But soon after, typechecking fails: ``` • Couldn't match type ‘Type’ with ‘Double’ Expected: MyData a Type Int -> Double Actual: MyData a Type Int -> Type • In the expression: f1 In an equation for ‘f2’: f2 = f1 ``` So this is weird. Instead of unification solving `w_axc ~ Double`, it seems `w_axc` is left unrestricted, and then `NoPolyKinds` picks it up as a kind variable (why?) and defaults it to `Type`. As an experiment, I have also tried *not* refreshing `w_awX`, only substituting in the type arguments. Now, of course, this can't possibly work as soon as I have more than one occurrence of `MySyn` due to the interference between the wildcard metavars, but if I only have one, then the program typechecks. So to me this suggests I'm doing things mostly right, except that the metavar returned by `newMetaTyVarX` is not fit for my use case. What should I use instead of `newMetaTyVarX` to instantiate / "refresh" the (wildcard-originating) type metavariables in my "macro type template"? Thanks, Gergo On Mon, 25 Jul 2022, Simon Peyton Jones wrote: > I'm afraid I don't understand, but it sounds delicate. By all means try! > > Simon > > On Mon, 25 Jul 2022 at 11:04, ÉRDI Gergő <gergo(a)erdi.hu> wrote: > On Mon, 25 Jul 2022, Simon Peyton Jones wrote: > > > Do we have an existing way of substituting types over type > variables, *in > > HsType instead of Core Type*? > > > > > > I'm afraid not. Currently HsType is not processed much -- just renamed > and typechecked > > into a Type. > > I wonder if, instead, I could expand the rhs, typecheck it "abstractly" > (i.e. in the context of the synonym's binders), and THEN do the > substitution. If I typecheck the rhs for every occurrence, I should get > fresh metavars for each wildcard, which is pretty much what I want. I just > have to make sure I don't zonk before the substitution. > > Does this make sense? > > >

2 4