- ghc-devs - Haskell.org

hadrian-util: An experiment in a more usable hadrian UX
by Ben Gamari 06 Jan '20

06 Jan '20

Hi everyone, For the past few months I have been using Hadrian for the majority of my GHC builds. In due course I have encountered a few papercuts: * hadrian/cabal.build.sh is quite wordy (#16250); moreover, you need to be in the source root to invoke it (#16667) * editing hadrian.settings is quite difficult due to the lack of availability of tab-completion in vim * maintaining multiple build roots is quite error-prone since you must remember which build flavour you used for each (#16481, #16638) * there is no equivalent to setting `stage=2` in `mk/build.mk` to make the stage-1-freeze persistent To address these I cobbled together a small wrapper, hadrian-util. I have this installed in my home-manager environment with a shell alias, `hu`, meaning that building GHC is as easy as typing `hu run` anywhere in the tree. As discussed in the README, `hadrian-util` supports multiple build roots, has a moderately convenient interface for manipulating hadrian.settings (with completion!) and has enough persistent state to eliminate most of the error-prone boilerplate from Hadrian invocations without being confusing. There is the question of what the long-term future of hadrian-util should be. Arguably it is merely a hack papering over some of the shortcomings of Hadrian's current UX; perhaps eventually these will be fixed. However, in the meantime, I've found that hadrian-util makes hadrian quite pleasant to use. I hope others also find this useful. Cheers, - Ben

3 2

.hie files for pre-installed GHC libraries
by Csaba Hruska 05 Jan '20

05 Jan '20

Hi, Is it planned to include the .hie files of the base and other libraries in the GHC binary download package? Regards, Csaba Hruska

2 1

Fix for ticket 8095 in ghc-8.10.1
by Rene 05 Jan '20

05 Jan '20

Hello, I wondering if the fix for https://gitlab.haskell.org/ghc/ghc/issues/8095 is going to make it into ghc-8.10.1 ? Rene.

2 1

A small but useful tool for performance characterisation
by Ben Gamari 05 Jan '20

05 Jan '20

Hi everyone, I have recently been doing a fair amount of performance characterisation and have long wanted a convenient means of collecting GHC runtime statistics for later analysis. For this I quickly developed a small wrapper utility [1]. To see what it does, let's consider an example. Say we made a change to GHC which we believe might affect the runtime performance of Program.hs. We could quickly check this by running, $ ghc-before/_build/stage1/bin/ghc -O Program.hs $ ghc_perf.py -o before.json ./Program $ ghc-before/_build/stage1/bin/ghc -O Program.hs $ ghc_perf.py -o after.json ./Program This will produce two files, before.json and after.json, which contain the various runtime statistics emitted by +RTS -s --machine-readable. These files are in the same format as is used by my nofib branch [2] and therefore can be compared using `nofib-compare` from that branch. In addition to being able to collect runtime metrics, ghc_perf is also able to collect performance counters (on Linux only) using perf. For instance, $ ghc_perf.py -o program.json \ -e instructions,cycles,cache-misses ./Program will produce program.json containing not only RTS statistics but also event counts from the perf instructions, cycles, and cache-misses events. Alternatively, passing simply `ghc_perf.py --perf` enables a reasonable default set of events (namely instructions, cycles, cache-misses, branches, and branch-misses). Finally, ghc_perf can also handle repeated runs. For instance, $ ghc_perf.py -o program.json -r 5 --summarize \ -e instructions,cycles,cache-misses ./Program will run Program 5 times, emit all of the collected samples to program.json, and produce a (very basic) statistical summary of what it collected on stdout. Note that there are a few possible TODOs that I've been considering: * I chose JSON as the output format to accomodate structured data (e.g. capture experimental parameters in a structured way). However, in practice this choice has lead to significantly more inconvenience than I would like, especially given that so far I've only used the format to capture basic key/value pairs. Perhaps reverting to CSV would be preferable. * It might be nice to also add support for cachegrind. Anyways, I hope that others find this as useful as I have. Cheers, - Ben [1] https://gitlab.haskell.org/bgamari/ghc-utils/blob/master/ghc_perf.py [2] https://gitlab.haskell.org/ghc/nofib/merge_requests/24

2 2

[ANNOUNCE] GHC 8.8.2-rc1 is now available
by Ben Gamari 02 Jan '20

02 Jan '20

Hello everyone, The GHC team is proud to announce the first release candidate of GHC 8.8.2. The source distribution, binary distributions, and documentation are available at https://downloads.haskell.org/~ghc/8.8.2-rc1 Release notes are also available [1]. This release fixes a handful of issues affecting 8.8.1: - A bug (#17088) in the compacting garbage collector resulting in segmentations faults under specific circumstances. Note that this may affect user programs even if they did not explicitly request the compacting GC (using the -c RTS flag) since GHC may fallback to compacting collection during times of high memory pressure. - A code generation bug (#17334) resulting in GHC panics has been fixed. - A bug in the `process` library causing builds using `hsc2hs` to fail non-deterministically on Windows has been fixed (Trac #17480) - A typechecker bug (#12088) resulting in programs being unexpectedly rejected has been fixed. - A bug in the implementation of compact normal forms resulting in segmentation faults in some uses (#17044) has been fixed. - A bug causing GHC to incorrectly complain about incompatible LLVM versions when using LLVM 7.0.1 has been fixed (#16990). As always, if anything looks amiss do let us know. Happy compiling! Cheers, - Ben [1] https://downloads.haskell.org/ghc/8.8.2-rc1/docs/html/users_guide/8.8.2-not…

4 3

[ANNOUNCE] Glasgow Haskell Compiler 8.10.1-alpha2 released
by Ben Gamari 02 Jan '20

02 Jan '20

Hello all, The GHC team is happy to announce the availability of the first alpha release in the GHC 8.10 series. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.1-alpha2/ GHC 8.10.1 will bring a number of new features including: * The new UnliftedNewtypes extension allowing newtypes around unlifted types. * The new StandaloneKindSignatures extension allows users to give top-level kind signatures to type, type family, and class declarations. * A new warning, -Wderiving-defaults, to draw attention to ambiguous deriving clauses * A number of improvements in code generation, including changes * A new GHCi command, :instances, for listing the class instances available for a type. * An upgraded Windows toolchain lifting the MAX_PATH limitation * A new, low-latency garbage collector. * Improved support profiling, including support for sending profiler samples to the eventlog, allowing correlation between the profile and other program events This release is the second alpha of the 8.10 pre-release cycle. The release will be a release candidate in roughly three weeks. If all goes well the final release will be cut roughly two weeks after the candidate, in mid-January. This being an alpha release, there are a few issues that are still outstanding: * The new Alpine Linux binary distribution is not present due to an apparent correctness issue [1]; any help Alpine users can offer here would be greatly appreciated. * We have yet to sort out compliance with Apple's notarization requirements [2] which will be likely be necessary for users of macOS Catalina. However, until this has been sorted out Catalina users can exempt the current macOS binary distribution from the notarization requirement themselves by running `xattr -cr .` on the unpacked tree before running `make install`. * There is one remaining non-regression correctness issue which we plan to fix for the final 8.10.1 but that is not fixed in this release. Please do test this release and let us know if you encounter any other issues. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17508 [2] https://gitlab.haskell.org/ghc/ghc/issues/17418

3 2

Implementation of Type level warnings
by Rowan Goemans 30 Dec '19

30 Dec '19

Hello everyone, A couple of days ago I tried to implement type level warnings [1]. I have got something now that sort of works but issues the same warning multiple times and not on the correct location. It's currently a WIP merge request [2]. I would like some help/pointers to continue the implementation The approach I have taken is to issue the warning once TypeWarning is rewritten. However,this has severe problems since now a warning is issued multiple times and at different locations due to therewritinghappening multiple times. I don't know what the best location is to do this check and issue the warning. Prior to constraint solving isn't a good place since the TypeWarning could be nested deeply in some constraint. Afterwards also isn't doable since then the TypeWarning constraint is gone. An alternative could be to not implement any instance in for TypeWarning similar to TypeError and special case it somehow. Any help is appreciated! [1] https://gitlab.haskell.org/ghc/ghc/issues/17027 [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/2361 Best regards, RowanGoemans

2 1

HEAD bootstrap error macOS
by Gabor Greif 25 Dec '19

25 Dec '19

Recently I started getting following error: ``` [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS -O0 -H64m -Wall -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts -this-unit-id rts -dcmm-lint -DDTRACE -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen -O2 -Wcpp-undef -Wnoncanonical-monad-instances -c rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by GHC version 8.6.5 *** initializing package database: Using binary package database: /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache package flags [] loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d wired-in package ghc-prim mapped to ghc-prim-0.6.1 wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0 wired-in package base mapped to base-4.14.0.0 wired-in package rts mapped to rts wired-in package template-haskell mapped to template-haskell-2.16.0.0 wired-in package ghc mapped to ghc-8.11.0.20191222 !!! initializing package database: finished in 8.51 milliseconds, allocated 8.770 megabytes Created temporary directory: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0 *** systool:cc: *** C Compiler: /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc -x c rts/sm/Scav_thr.c -o /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen -I/Users/ggreif/ghc/libraries/base/include -I/Users/ggreif/ghc/libraries/base/dist-install/build/include -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include -I/Users/ggreif/ghc/libraries/integer-gmp/include -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes -I/Users/ggreif/ghc/includes/dist-derivedconstants/header -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts' -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"' -DTHREADED_RTS In file included from rts/sm/Scav_thr.c:3:0: error: In file included from rts/sm/Scav.c:51:0: error: In file included from rts/sm/GCUtils.h:18:0: error: rts/sm/GCTDecl.h:113:1: error: error: register '%r13' unsuitable for global register variables on this target | 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base); | ^ GCT_REG_DECL(gc_thread*, gct, REG_Base); ^ rts/sm/GCTDecl.h:37:56: error: note: expanded from macro 'GCT_REG_DECL' | 37 | #define GCT_REG_DECL(type,name,reg) register type name REG(reg); | ^ #define GCT_REG_DECL(type,name,reg) register type name REG(reg); ^ includes/stg/MachRegs.h:160:24: error: note: expanded from macro 'REG' | 160 | #define REG(x) __asm__("%" #x) | ^ #define REG(x) __asm__("%" #x) ^ 1 error generated. *** Deleting temp files: Deleting: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_2.rsp Warning: deleting non-existent /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s *** Deleting temp dirs: Deleting: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0 `cc' failed in phase `C Compiler'. (Exit code: 1) ``` My $CC is clang-9 (but I also tried others). Has anybody seen something similar? Cheers nd thanks, Gabor

2 5

Perf metrics
by Simon Peyton Jones 20 Dec '19

20 Dec '19

When I run `sh validate -legacy`, towards the end I see stuff like this Performance Metrics (test environment: local): Conversions(normal) runtime/bytes allocated 107696.000 DeriveNull(normal) runtime/bytes allocated 112050960.000 InlineArrayAlloc(normal) runtime/bytes allocated 1600041088.000 InlineByteArrayAlloc(normal) runtime/bytes allocated 1440041088.000 InlineCloneArrayAlloc(normal) runtime/bytes allocated 1600041248.000 ManyAlternatives(normal) compile_time/bytes allocated 840985728.000 ManyConstructors(normal) compile_time/bytes allocated 4540766560.000 MethSharing(normal) runtime/peak_megabytes_allocated 2.000 MethSharing(normal) runtime/bytes allocated 480098136.000 MultiLayerModules(normal) compile_time/bytes allocated 5856970504.000 ... It is intermingled with other apparently unrelated output. What should I conclude form this? Is it good or bad? By what amount have these figures changed, and relative to what? How can I run a single perf test? Thanks Simon

2 3

Re: haskell/bytestring stalled?
by Simon Jakobi 19 Dec '19

19 Dec '19

Hi Herbert, I apologize for the overly harsh and accusatory tone in my last email. After looking through the existing PRs and issues, I was admittedly somewhat disappointed over the current situation with bytestring issues and PRs, but I shouldn't have expressed my feelings in this way. Also, thanks a lot for responding so quickly today to my many comments on issues and PRs. To be clear, I'm not interested in putting blame on you or Duncan or anyone else. I do however strongly believe that bytestring needs a more responsive maintainer: 1. Contributors IMO deserve to get a response within weeks or ideally days. 2. bytestring users (and that transitively includes all GHC users) could benefit massively if some of the current PRs were finally merged. For example in [#175], Sylvain Henry reports improvements of **11%** for some compilation tasks when he used the PR to build GHC. And I admittedly don't want to ask you, Herbert, to lead the effort of clearing up the maintenance backlog in bytestring. Of course, _any_ Haskell project you're involved in benefits from your massive experience! However, I feel that since your involvement is so crucial in so many other essential projects in the Haskell ecosystem, if you would shift your focus towards bytestring, these other projects would suffer! I also believe that projects like cabal or Hackage need your domain knowledge more dearly than bytestring does. So to ask you to allocate more time to bytestring, would feel a bit like a zero-sum game, or possibly a negative-sum game. A second, more personal reason, why I would prefer to see a different maintainer for bytestring, is your preference for coordinating things in private communication. The many PRs and issues in the bytestring project, and this email thread demonstrate that there are a lot of people who would like to be involved in improving bytestring. Private coordination would exclude most of them, and make it more difficult to make use of all the help that this project currently needs. I understand that your preference for private communication is related to your experiences with previous discussions over Haskell projects, and I am sorry that it has come so far, but I believe that a more open and transparent style of communication is more healthy. So, while I'm very grateful for your work on bytestring, and while I hope that you stay involved in the project, I hope that that someone else can take over official maintainer duties. Now I am not sure who could take on that job. Clearly we need someone whom Duncan trusts. That's why I wonder whether someone involved in the performance work on GHC could help. Ultimately I don't think that their time involvement would need to be very high – good judgment and responsiveness seem more important to me. I'm CC'ing the ghc-devs list, hoping that someone with the right experience and availability would come forward. Thanks, Simon [#175] https://github.com/haskell/bytestring/pull/175#issuecomment-567233984

2 2