- ghc-devs - Haskell.org

Gitlab corrupt
by Simon Peyton Jones 19 Jun '20

19 Jun '20

Ben Something is amiss. "corrupt" sounds bad! Simon bash$ git fetch error: garbage at end of loose object '62be92e5be07b34fc25c77e36223268c4a6eadec' fatal: loose object 62be92e5be07b34fc25c77e36223268c4a6eadec (stored in ./objects/62/be92e5be07b34fc25c77e36223268c4a6eadec) is corrupt fatal: The remote end hung up unexpectedly

2 1

Receiving type information from environment instead of hardcoding.
by Rinat Stryungis 19 Jun '20

19 Jun '20

Hi. I have a question about a possible way of unification of Nat and Natural. I've almost done that, but only in case of using integer-gmp. If I use integer-simple there is a completely different definition of Natural. How I construct now naturalTyCon (to make `naturalTy` to use it instead of `typeNatKind`) : ```naturalTyCon :: TyCon naturalTyCon = pcTyCon naturalTyConName Nothing [] [natSDataCon,natJDataCon] natSDataCon :: DataCon natSDataCon = pcDataCon natSDataConName [] [wordPrimTy] naturalTyCon etc... ``` Now I have to check`DynFlags` in a few places to reimplement `naturalTyCon` in case of using `integer-simple`. Is there a way to avoid hardcoding of `naturalTy`? My colleague said that it would be nice to get `naturalTy` from an environment by something like `lookupTyCon`, but there are many functions whose don't use any environment like functions from `typeNatTyCons` list in `GHC.Builtin.Types.Literals`. Now I am going to use `DynFlags` checking, but it looks like an ugly way... -- Best regards. Rinat Striungis

3 3

Gitlab 503
by Simon Peyton Jones 19 Jun '20

19 Jun '20

I'm getting a lot of "Gitlab is unavailable" 503 messages, after editing a wiki page. Does anyone know what is happening? Simon

1 0

Linear types performance characterisation
by Ben Gamari 17 Jun '20

17 Jun '20

Hi everyone, Last week I discussed the plan for merging the LinearTypes branch into GHC 8.12 with Arnaud, Richard, Andreas, and Simon. Many thanks to all of them for their respective roles in pushing this patch over the finish line. One thing that we wanted to examine prior to merge is compiler performance across a larger collection of packages. For this I used the head.hackage patch-set, comparing the Linear Types branch with its corresponding base commit in `master`. Here I will describe the methodology used for this comparison and briefly summarize the (happily, quite positive) results. # Methodology I collected total bytes allocated (as reported by the runtime system), elapsed runtime (as reported by the runtime system), and instructions (as reported by `perf stat`) of head.hackage builds in two configurations: * the `opt` configuration * the `noopt` configuration, which passed `--disable-optimisation` to cabal-install These configurations were evaluated on two commits: * `master`: 2b792facab46f7cdd09d12e79499f4e0dcd4293f * `linear-bang`: 481cf412d6e619c0e47960f4c70fb21f19d6996d Unfortunately, the `noopt` configuration appears to be be affected by a few cabal-install bugs [1,2] and consequently some packages may *still* be compiled with optimisation, so take these numbers with a grain of salt. The test environment was a reasonably quiet Ryzen 7 1800X with 32 GBytes of RAM. The test was run by first building the two tested commits in Hadrian's default build flavour. The head.hackage CI driver was then invoked as follows: # Don't parallelize for stable performance measurements export CPUS=1 export USE_NIX=1 export EXTRA_HC_OPTS=-ddump-timings export COLLECT_PERF_STATS=1 mkdir -p runs # master export GHC=/home/ben/ghc/ghc-compare-2/_build/stage1/bin/ghc ./run-ci --cabal-option=--disable-optimisation mv ci/run runs/master-noopt ./run-ci mv ci/run runs/master-opt # linear-bang export GHC=/home/ben/ghc/ghc-compare-1/_build/stage1/bin/ghc ./run-ci --cabal-option=--disable-optimisation mv ci/run runs/linear-noopt ./run-ci mv ci/run runs/linear-opt As we are building all packages (nearly 300 in total) serially, the full run takes quite a while (around 8 hours IIRC). The final run of this test used head.hackage commit e7e5c5cfbfd42c41b1e62d42bb18483a83b78701 (on the `rts-stats` branch). # Results I examined several different metrics of compiler performance * the total_wall_seconds RTS metric gives an picture of overall compilation effort * time reported by -ddump-timings, summed by module, gives a slightly finer-grained measurement of per-module compilation time * the RTS's bytes_allocated metric gives overall compiler allocations * the RTS"s max_bytes_used metric gives a sense of AST size (and potentially the existence of leaks) To cut straight to the chase, the measurements show the following: metric -O0 -O1 ------------------- --------- ---------- total_wall_seconds +0.3% +0.6% total_cpu_seconds +0.3% +0.7% max_bytes_used +4.2% +4.8% GC_cpu_seconds +1.5% +2.1% mut_cpu_seconds no change no change sum(per-module-time) +4.2% +4.2% sum(per-module-alloc) +0.8% +0.8% There are a few things to point out here: the overall change in compiler runtime is thankfully quite reasonable. However, max_bytes_used increases rather considerably. This seems to give rise to an appreciable regression in GC time. It would be interesting to know whether this can be improved with optimisation to data representation. The fact that the cumulative per-module metrics didn't change between -O0 and -O1 indicate to me that there is a methodological problem which needs to be addressed in the test infrastructure. I investigated this a bit and have a hypothesis for what might be going on here; nevertheless, in the interest of publishing these measurements I'm ignoring these measurements for the time being. I have attached the Jupyter notebook that gave rise to these numbers. This gives a finer-grained breakdown of the data including histograms showing the variance of each metric. Perhaps this will be helpful in better understanding the effects. I would be happy to share my run data as well although it is a bit large. All-in-all, the Tweag folks have done a great job in squashing the performance numbers noticed a few weeks ago. The current numbers look quite acceptable for GHC 8.12. Congratulations to Arnaud, Krzysztof, and Richard on landing this feature! I'm very much looking forward to see what the community does with it in the coming years. Cheers, - Ben [1] https://github.com/haskell/cabal/issues/5353 [2] https://github.com/haskell/cabal/issues/3883

2 1

Search in GitLab
by Simon Peyton Jones 15 Jun '20

15 Jun '20

Does anyone know how to search better in GitLab. Currently I'm using the standard GitLab search. I'm searching for "<<loop>>" where I intend the quotes meaning exactly that string as usual in a search term. But I get lots of results mentioning loop, without the angle brackets. Moreover I want to sort the results by date or ticket number, and I can't see how to do that. Does Google index our repo? Can I use Google to search it somehow? Thanks Simon

6 8

Looking for a recent error reporting change
by Chris Smith 15 Jun '20

15 Jun '20

Hi. In GHC 8.6.5, if I compile a main module that both fails to define `main` and also contains other errors, the other errors are not reported because GHC aborts after realizing that there is no `main`. In GHC 8.10.1, all errors are reported. I'm looking for where this change was made, in the hopes that it's easy to backport to a local branch of GHC 8.6.5. Does anyone know off-hand?

4 4

libgmp dependency on alpine linux binary release
by depsterr 12 Jun '20

12 Jun '20

I apologize if this is the wrong mailing list, however I couldn't find another one that seemed fit. On the official binary release page (https://www.haskell.org/ghc/download_ghc_8_10_1.html) under the x86_64 alpine release it states the following: > Unlike our other binary distributions, this links against the`integer-simple`big-integer backend and therefore does not require`libgmp`. However, ldd shows that it DOES link to gmp: libgmp.so.10 => /lib/libgmp.so.10 (0x7f47f4c94000) Additionally, if gmp is not installed then a number of errors appear: Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_tdiv_qr: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_gcd_1: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_add_1: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_andn_n: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpz_gcdext: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_lshift: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_rshift: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_and_n: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpz_powm: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_mod_1: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_cmp: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_add: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_sub_1: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpn_xor_n: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpz_clear: symbol not found Error relocating /usr/bin/../lib/x86_64-linux-ghc-8.10.1/libHSinteger-gmp-1.0.3.0-ghc8.10.1.so: __gmpz_invert: symbol not found Either this build isn't configured properly or the text on the release page needs to be edited to reflect this.

1 0

Evac.c
by Simon Peyton Jones 11 Jun '20

11 Jun '20

I'm getting this in HEAD. Should I worry? It says "error"! Simon rts/sm/Evac.c:463:5: error: note: called from here ACQUIRE_SPIN_LOCK(&gen->sync); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | 463 | ACQUIRE_SPIN_LOCK(&gen->sync); | ^ In file included from includes/Rts.h:196:0: error: 0, from rts/sm/Evac.c:15: includes/rts/SpinLock.h:43:20: error: warning: inlining failed in call to 'ACQUIRE_SPIN_LOCK': call is unlikely and code size would grow [-Winline] INLINE_HEADER void ACQUIRE_SPIN_LOCK(SpinLock * p) ^~~~~~~~~~~~~~~~~ | 43 | INLINE_HEADER void ACQUIRE_SPIN_LOCK(SpinLock * p) | ^ rts/sm/Evac.c:515:31: error: note: called from here if (new_gen != gen) { ACQUIRE_SPIN_LOCK(&new_gen->sync); } ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | 515 | if (new_gen != gen) { ACQUIRE_SPIN_LOCK(&new_gen->sync); } | ^

3 2

Marge Bot repeatedly failing due to stat decreases
by Ryan Scott 08 Jun '20

08 Jun '20

Hello, I've been receiving a flurry of e-mails all weekend about Marge Bot failing to validate a batch on CI. Moreover, the failures appear to be consistent each time. From the most recent one [1]: Unexpected stat failures: /builds/ghc/ghc/tmp/ghctest-mel1y5lo/test spaces/testsuite/tests/perf/compiler/T12150.run T12150 [stat decreased from "x86_64-linux-deb9-hadrian" baseline @ HEAD~6] (optasm) /builds/ghc/ghc/tmp/ghctest-mel1y5lo/test spaces/testsuite/tests/perf/compiler/T12234.run T12234 [stat decreased from "x86_64-linux-deb9-hadrian" baseline @ HEAD~6] (optasm) Where T12150(optasm) compile_time/bytes allocated 75982592.000 (baseline @ HEAD~6) 76945688.000 [decreased, -1.3%] T12234(optasm) compile_time/bytes allocated 82149240.000 (baseline @ HEAD~6) 83109896.000 [decreased, -1.2%] Can something be done about this? It's holding one one of my patches. Ryan S. ----- [1] https://gitlab.haskell.org/ghc/ghc/-/jobs/359994

2 4

Change in ~"backport needed" label convention
by Ben Gamari 04 Jun '20

04 Jun '20

Hi everyone, Recently with the increased rate of GHC releases we have had to handle more concurrent stable branches. In particular, currently we have GHC-8.8.4 and GHC-8.10.2 releases both in flight. It has become clear that our current scheme for tracking backports, the ~"baackport needed" GitLab label, doesn't accomodate multiple concurrent stable branches particularly well. Consequently, I have created a new set of labels which are specific to each release series. For instance, ~"backport needed:8.10" means that the MR needs to be backported to the ghc-8.10 branch. The old ~"backport needed" label has been removed. Please do use these new labels on merge requests to indicate that backporting is needed. Cheers, - Ben

1 0