March 2021 - ghc-devs

Build failure -- missing dependency? Help!
by Simon Peyton Jones 16 Mar '21

16 Mar '21

I'm getting this (with 'sh validate -legacy'). Oddly * It does not happen on HEAD * It does happen on wip/T19495, a tiny patch with one innocuous change to GHC.Tc.Gen.HsType I can't see how my patch could possible cause "missing files" in ghc-bignum! I'm guessing that there is a missing dependency that someone doesn't show up in master, but does in my branch, randomly. There's something funny about ghc-bignum; it doesn't seem to be a regular library Can anyone help? Thanks Simon "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id base-4.16.0.0 -hide-all-packages -package-env - -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -Ilibraries/base/dist-install/build/include -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id ghc-bignum-1.0 -package-id ghc-prim-0.8.0 -package-id rts -this-unit-id base -Wcompat -Wnoncanonical-monad-instances -XHaskell2010 -O -dcore-lint -ticky -Wwarn -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -outputdir libraries/base/dist-install/build -dynamic-too -c libraries/base/./GHC/Exception/Type.hs-boot -o libraries/base/dist-install/build/GHC/Exception/Type.o-boot -dyno libraries/base/dist-install/build/GHC/Exception/Type.dyn_o-boot Failed to load interface for 'GHC.Num.Integer' There are files missing in the 'ghc-bignum' package, try running 'ghc-pkg check'. Use -v (or `:set -v` in ghci) to see a list of the files searched for. make[1]: *** [libraries/base/ghc.mk:4: libraries/base/dist-install/build/GHC/Exception/Type.o-boot] Error 1 make[1]: *** Waiting for unfinished jobs....

4 7

GSOC Idea: Bytecode serialization and/or Fat Interface files
by Zubin Duggal 14 Mar '21

14 Mar '21

Hi all, This is following up on this recent discussion on the list concerning fat interface files: https://mail.haskell.org/pipermail/ghc-devs/2020-October/019324.html Now that we have been accepted as a GSOC organisation, I think it would be a good project idea for a sufficiently motivated and advanced student. This is a call for mentors (and students as well!) who would be interested in this project The problem is the following: Haskell Language Server (and ghci with `-fno-code`) have very fast startup times for codebases which don't make use of Template Haskell, and thus don't require any code-gen to typecheck. This is because they can simply read the cached iface files generated by a previous compile and don't need to re-invoke the typechecker. However, as soon as TH is involved, we are forced to retypecheck and compile files, since it is not possible to restart the code-gen process starting with only a iface file. I can think of two ways to address this problem: 1. Allow bytecode to be serialized 2. Serialize desugared Core into iface files (fat interfaces), so that (byte)code-gen can be restarted from this point and doesn't need (1) might be challenging, but offers a few more advantages over (2), in that we can reduce the work done to load TH-heavy codebases to just a load of the cached bytecode objects from disk, and could make the load process (and times) for these codebases directly comparable to their TH-free cousins. It would also make ghci startup a lot faster with a warm cache of bytecode objects, bringing ghci startup times in line with those of -fno-code However (2) might be much easier to achieve and offers many of the same advantages, in that we would not need to re-run the compiler frontend or core-to-core optimisation phases. There is also already a (slightly bitrotted) implementation of (2) thanks to the work of Edward Yang. If any of this sounds exciting to you as a student or a mentor, please get in touch. In particular, I think (2) is a feasible project that can be completed with minimal mentoring effort. However, I'm only vaguely familiar with the details of the byte code generator, so if (1) is a direction we want to pursue, we would need a mentor familiar with the details of this part of GHC. Cheers, Zubin

5 5

GHC Exactprint merge process
by Alan & Kim Zimmerman 12 Mar '21

12 Mar '21

I have been running a branch in !2418[1] for just over a year to migrate the ghc-exactprint functionality directly into the GHC AST[2], and I am now satisfied that it is able to provide all the same functionality as the original. This is one of the features intended for the impending 9.2.1 release, and it needs to be reviewed to be able to land. But the change is huge, as it mechanically affects most files that interact with the GHC AST. So I have split out a precursor !5158 [3] with just the new types that are used to represent the annotations, so it can be a focal point for discussion. It is ready for review, please comment if you have time and interest. Regards Alan [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/2418 [2] https://gitlab.haskell.org/ghc/ghc/-/issues/17638 [3] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5158

3 4

Inlining of `any @[]` vs `elem @[]`
by ÉRDI Gergő 12 Mar '21

12 Mar '21

Hi, The inlining behaviour of `any @[]` and `elem @[]` differs in a way that I am not sure is intentional, and it is affecting Clash (see https://github.com/clash-lang/clash-compiler/issues/1691) I would think that if it is a good idea to inline `any` then inlining `elem` would be just as good an idea, or vice versa. However, `any` is defined polymorphically over `Foldable`, via `foldMap` using `foldr`, with all steps between (and `foldr @[]`!) marked as `INLINE`. The result is that if you use `any (x ==) [1, 5, 7]` you get the following beautiful Core: ``` topEntity = \ (x_agAF :: Int) -> case x_agAF of { GHC.Types.I# y_ahao -> case y_ahao of { __DEFAULT -> GHC.Types.False; 1# -> GHC.Types.True; 5# -> GHC.Types.True; 7# -> GHC.Types.True } } ``` As the kids these days would say: *chef's kiss*. `elem`, on the other hand, is a typeclass method of `Foldable`, with a default implementation in terms of `any`, but overridden for lists with the following implementation: ``` GHC.List.elem :: (Eq a) => a -> [a] -> Bool GHC.List.elem _ [] = False GHC.List.elem x (y:ys) = x==y || GHC.List.elem x ys {-# NOINLINE [1] elem #-} {-# RULES "elem/build" forall x (g :: forall b . Eq a => (a -> b -> b) -> b -> b) . elem x (build g) = g (\ y r -> (x == y) || r) False #-} ``` This is marked as non-inlineable until phase 1 (so that `elem/build` has a chance of firing), but it seems that when build fusion doesn't apply (since `[1, 5, 7]` is, of course, not built via `build`), no inlining happens AT ALL, even in later phases, so we end up with this: ``` topEntity = \ (x_agAF :: Int) -> GHC.List.elem @ Int GHC.Classes.$fEqInt x_agAF (GHC.Types.: @ Int (GHC.Types.I# 1#) (GHC.Types.: @ Int (GHC.Types.I# 5#) (GHC.Types.: @ Int (GHC.Types.I# 7#) (GHC.Types.[] @ Int)))) ``` So not only does it trip up Clash, it would also result in less efficient code in software when using "normal" GHC. Is this all intentional? Wouldn't it make more sense to mark `GHC.List.elem` as `INLINE [1]` instead of `NOINLINE [1]`, so that any calls remaining after build fusion would be inlined? Thanks, Gergo

2 6

Re: WSL2
by Tom Ellis 12 Mar '21

12 Mar '21

SPJ Wrote: > I've just installed WSL2 and built GHC. I get this (single) > validation failure in libraries/unix/tests/getGroupEntryForName. It > seems to be just an error message wibble, but I can't push a change > to master because that'll affect everyone else. Interesting, I've only ever built GHC on WSL and WSL2. I've seen this error message on WSL2 during every test run, I think. I didn't realise that it never occurred on other platforms, let alone that it was WSL2 specific! Tom

4 11

What type of performance regression testing does GHC go through?
by Tom Ellis 12 Mar '21

12 Mar '21

A user posted the following to the ghc-proposals repository. Both JB and RAE suggested ghc-devs as a more appropriate forum. Since I have no idea whether the user has even ever used a mailing list before I thought I would lower the activation energy by posting their message for them. https://github.com/ghc-proposals/ghc-proposals/issues/410 > Hi, > > Does the GHC release or development process include regression > testing for performance? > > Is this the place to discuss ideas for implementing such a thing and > to eventually craft a proposal? > > I believe the performance impact of changes to GHC needs to be > verified/validated before release. I also believe this would be > feasible if we tracked metrics on building a wide variety of > real-world packages. Using real-world packages is one of the best > ways to see the actual impact users will experience. It's also a > great way to broaden the scope of tests, particularly with the > combination of language pragmas and enabled features within the > compiler.

2 1

WSL2
by Simon Peyton Jones 10 Mar '21

10 Mar '21

Friends I've just installed WSL2 and built GHC. I get this (single) validation failure in libraries/unix/tests/getGroupEntryForName. It seems to be just an error message wibble, but I can't push a change to master because that'll affect everyone else. Any ideas? Simon =====> 1 of 1 [0, 0, 0] ]0;getGroupEntryForName(normal) 1 of 1 [0, 0, 0]Actual stderr output differs from expected: --- getGroupEntryForName.run/getGroupEntryForName.stderr.normalised 2021-03-09 22:36:01.300421100 +0000 +++ getGroupEntryForName.run/getGroupEntryForName.run.stderr.normalised 2021-03-09 22:36:01.300421100 +0000 @@ -1 +1 @@ -getGroupEntryForName: getGroupEntryForName: does not exist (no such group) +getGroupEntryForName: getGroupEntryForName: does not exist (No such process) *** unexpected failure for getGroupEntryForName(normal)

2 2

Running ghc-debug on ghc
by Matthew Pickering 09 Mar '21

09 Mar '21

Hi, I now have some simple instructions for running ghc-debug on GHC. 1. Cherry-pick 4be70967f1f1ab70cbe31aad8ae69aea87c6f4c4 commit 4be70967f1f1ab70cbe31aad8ae69aea87c6f4c4 (HEAD -> wip/ghc-with-debug) Author: Matthew Pickering <matthewtpickering(a)gmail.com> Date: Fri Jan 8 11:26:17 2021 +0000 Add support for ghc-debug to ghc executable 2. Build GHC * Add the following to _build/hadrian.settings ``` stage1.*.ghc.hs.opts += -finfo-table-map -fdistinct-constructor-tables ``` * Build GHC as normal ``` ./hadrian/build -j8 ``` * The result is a ghc-debug enabled compiler # Building a debugger * Use the compiler you just built to build ghc-debug ``` cd ghc-debug cabal update cabal new-build debugger -w ../_build/stage1/bin/ghc ``` # Running the debugger Modify `test/Test.hs` to implement the debugging thing you want to do. Perhaps start with `p30`, which is a program to generate a profile. * Start the process you want to debug ``` GHC_DEBUG_SOCKET=/tmp/ghc-debug build-cabal ``` * Start the debugger ``` cabal new-run debugger -w ... ``` * Open a ticket about the memory issue you find. There is the start of some more documentation here - http://ghc.gitlab.haskell.org/ghc-debug/docs.html These instructions are also in the instructions.md file in the commit. Cheers, Matt

2 1

What changed between GHC 8.8 and 8.10 that could cause this?
by ÉRDI Gergő 09 Mar '21

09 Mar '21

Hi, I'm trying to figure out a Clash problem and managed to track it down to a GHC upgrade; specifically, a given Clash version, when based on GHC 8.8, has no problem synthesizing one module after another from one process; but the same Clash version with GHC 8.10 fails with link-time errors on the second compilation. The details are at https://github.com/clash-lang/clash-compiler/issues/1686 but for now I'm just hoping that some lightbulb will go off for someone if some handling of internal state has changed in GHC that could mean that the symbol tables of loaded modules could persist between GHC invocations from the same process. So, does this ring a bell for anyone? Thanks, Gergo

5 8

Requesting help with unregisterized backend failure of !4717
by John Ericson 09 Mar '21

09 Mar '21

Hi everyone, https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4717 fails some numerical boundary checking tests with the unregisterized backend. In particular, `minBound / (-1)` and `pred minBound` are *not* diverging as expected. This stumped me a few days ago, and no new ideas have struct me since. I would very much appreciate some help. This does seem like something that is very likely to be backend-dependent, as different instructions/arches handle such edge cases in different ways. What makes this so peculiar, however, is that the boundary condition checking/branching is done *in regular Haskell*. I'm thus quite baffled as to what could be going wrong. If anyone wants to dive in, see my last comment https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4717#note_329840, which has a minimization to declutter the code without doing enough constant folding that the problem is avoided entirely. (I went with a compilation unit trick, but in hindsight NOINLINE should also work and be simpler.) Finally, let my provide some context for why I am hoping to get this merged soon. To be clear, !4492 was the main MR relating to the numerics primops critical to get in for 9.2 and it thankfully already landed. But landing this MR too would be nice: It shrinks the intermediate representations of numerical code probably smaller than it was originally, whereas now it is perhaps larger than it was before due to more size<->native conversions. Also, I think this MR will help get !3658 over the finish line, which, while again not as critical for 9.2 as !4492 was, would be awfully nice to do to fully realize the new simplicity of the plan set forth in https://gitlab.haskell.org/ghc/ghc/-/wikis/Unboxed-Numerics. Thanks, John N.B. I just rebased and repushed the MR, so CI might be still running or failing due to something else, but based on local testing this is still the an issue. https://gitlab.haskell.org/ghc/ghc/-/pipelines/31002 is an earlier pipeline run that one can look at until CI finishes again.

2 2