- ghc-devs - Haskell.org

slow load and typecheck
by Sam Halliday 09 Oct '19

09 Oct '19

Hello all, I am writing an interactive tool using the ghc api. It is able to load and typecheck a source file in a user's package. I obtain the flags that cabal uses to compile the user's package via the hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool, then I `setTargets' all the home modules (with targetAllowObjCode=True). I use HscNothing and NoLink because I only want access to the trees, I don't want to produce any output files. For the file that I wish to inspect, I `removeTarget' the module and `addTarget` it again but this time providing the full path to the file and don't allow object code. Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume that the file under inspection only has a module definiton and no imports or top levels. Functionally, my code is working great and I am able to do what I want with the typechecked tree. However, load is very slow (~10 seconds user time) on large projects. Here is a cpu time trace of my program (milliseconds): main 1 parse flags 93 load 20436 typecheck 20437 I can enable a bit more ghc timing info via -Rghc-timings and I see !!! Chasing dependencies: finished in 157.20 milliseconds, allocated 528.112 megabytes This seems fine, anything sub-second is ok. But then I see a bunch of home modules in CodeGen that I was not expecting: !!! CodeGen [My.Module.Dependency]: finished in 3335.62 milliseconds, allocated 270.615 megabytes So it looks like the targetAllowObjCode is being ignored... is there any way to force it? Actually I'd prefer to fail fast than to ever compile or codegen a dependency module. I know that it should be possible to load the module a lot faster because if I make a small change in the file under inspection and ask cabal to recompile the module it is super fast (less than a second). Could somebody who understands how incremental/partial compiles work please help me out? PS: If this textual description is confusing, I could put together a minimal reproduction and example project but it will take me some time to do that. -- Best regards, Sam

4 14

[ANNOUNCE] GHC 8.8.1 is now available
by Ben Gamari 08 Oct '19

08 Oct '19

Hello everyone, The GHC team is pleased to announce the release candidate for GHC 8.8.1. The source distribution, binary distributions, and documentation are available at https://downloads.haskell.org/ghc/8.8.1 This release is the culmination of over 3000 commits by over one hundred contributors and has several new features and numerous bug fixes relative to GHC 8.6: * Visible kind applications are now supported (GHC Proposal #15) * Profiling now works correctly on 64-bit Windows (although still may be problematic on 32-bit Windows due to platform limitations; see #15934) * A new code layout algorithm for amd64's native code generator significantly improving the runtime performance of some kernels * The introduction of a late lambda-lifting pass which may reduce allocations significantly for some programs. * Further work on Trees That Grow, enabling improved code re-use of the Haskell AST in tooling * Users can write `forall` in more contexts (GHC Proposal #7) * The pattern-match checker is now more precise in the presence of strict fields with uninhabited types. * A comprehensive audit of GHC's memory ordering barriers has been performed, resulting in a number of fixes that should significantly improve the reliability of programs on architectures with weakly-ordered memory models (e.g. PowerPC, many ARM and AArch64 implementations). * A long-standing linker limitation rendering GHCi unusable with projects with cyclic symbol dependencies has been fixed (#13786) * Further work on the Hadrian build system * Countless miscellaneous bug-fixes Unfortunately, due to a build issue (#17108) found late in the release process i386 Windows builds are currently unavailable. These will be provided in the coming weeks. As always, if anything looks amiss do let us know. Happy compiling! Cheers, - Ben [1] https://downloads.haskell.org/ghc/8.8.1/docs/html/users_guide/8.8.1-notes.h…

4 5

Re: Coercible questions
by Michal Terepeta 07 Oct '19

07 Oct '19

Adding +ghc-devs <ghc-devs(a)haskell.org> to continue the thread Hi Sandy, Thanks for the answer! Do you think there is some fundamental reason for this? Or just a matter of implementing this in GHC? It seems to me that this should work just fine as long as the runtime representation is the same. And a related question--is it safe to `unsafeCoerce` an `Int` to a `Word`? The only reason for why this could be problematic that comes to my mind is that there could be an assumption that different `data`s do not alias each other (although `newtype`s can due to `Coercible` functionality). But I'm not sure this is ever used by GHC? Are there any other reasons why this could be problematic? Thanks! - Michal On Sat, Oct 5, 2019 at 5:27 PM Sandy Maguire <sandy(a)sandymaguire.me> wrote: > Hi Michal, > > Datas aren't coercible, only newtypes. This is why you can't coerce Ints > and Words, and why Foo and Bar don't work. > > Sandy > > On Sat, Oct 5, 2019 at 4:17 PM Michal Terepeta <michal.terepeta(a)gmail.com> > wrote: > >> Hi, >> >> I've started looking into using `Data.Coerce` (and the `Coercible` >> type-class) for a personal project and was wondering why coercing between >> `Int` and `Word` is not allowed? I don't see any fundamental reason why >> this shouldn't work... >> >> Perhaps, it's just a matter of GHC's implementation details leaking out? >> IIRC internally GHC has separate `RuntimeRep`/`PrimRep` for a `Word#` and >> for an `Int#`. If that's the case, would it make sense to unify these? >> Their actual runtime representation should be the same and I'd expect most >> (all?) of their differences should be attached to `PrimOp`s. >> >> And that leads me to another question--what exactly goes wrong here: >> ``` >> data Foo = Foo Int# >> data Bar = Bar Int# >> >> test :: Bar >> test = coerce (Foo 42#) >> ``` >> Which fails with: "Couldn't match representation of type ‘Foo’ with that >> of ‘Bar’ arising from a use of ‘coerce’" >> >> Perhaps I'm just misunderstanding exactly how `Coercible` works? >> >> Thanks in advance! >> >> - Michal >> >> PS. The ability to coerce through things like lists is amazing :) >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > > -- > I'm currently travelling the world, sleeping on people's couches and doing > full-time collaboration on Haskell projects. If this seems interesting to > you, please consider signing up as a host! > https://isovector.github.io/erdos/ >

3 2

Should coercion binders (arguments or binders in patterns) be TyVars?
by Ömer Sinan Ağacan 07 Oct '19

07 Oct '19

Hi, I just realized that coercion binders are currently Ids and not TyVars (unlike other type arguments). This means that we don't drop coercion binders in CoreToStg. Example: {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs, TypeApplications, MagicHash #-} module UnsafeCoerce where import Data.Type.Equality ((:~:)(..)) import GHC.Prim import GHC.Types unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b unsafeEqualityProof = error "unsafeEqualityProof evaluated" unsafeCoerce :: forall a b . a -> b unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x If I build this with -ddump-stg this is what I get for `unsafeCoerce`: UnsafeCoerce.unsafeCoerce :: forall a b. a -> b [GblId, Arity=1, Unf=OtherCon []] = [] \r [x_s2jn] case UnsafeCoerce.unsafeEqualityProof of { Data.Type.Equality.Refl co_a2fd -> x_s2jn; }; See the binder in `Refl` pattern. Unarise drops this binder because it's a "void" argument (doesn't have a runtime representation), but still it's a bit weird that we drop types but not coercions in CoreToStg. Is this intentional? Ömer

4 5

Hadrian failure after worktree update
by Bryan Richter 05 Oct '19

05 Oct '19

Below is an error message I have seen a number of times after rebasing my git worktree, i.e. this is how to "reproduce": git fetch origin git rebase origin/master git submodule update ./hadrian/build.sh (Obviously some details are elided, and this is not a complete reproduction.) I don't have many copies of the error, but I do remember seeing the complaint about "No generator for .../PmOracle.hs" on other occasions. Any idea what this comes from? -Bryan --- 8< --- Error when running Shake build system: at action, called at src/Rules.hs:71:19 in main:Rules at need, called at src/Rules.hs:93:5 in main:Rules * Depends on: _build/stage0/lib/package.conf.d/ghc-8.9.0.20190929.conf at need, called at src/Rules/Register.hs:113:5 in main:Rules.Register * Depends on: _build/stage0/compiler/build/libHSghc-8.9.0.20190929.a * Depends on: _build/stage0/compiler/build/GHCi.o * Depends on: _build/stage0/compiler/build/GHCi.o _build/stage0/compiler/build/GHCi.hi * Depends on: OracleQ (KeyValues ("_build/stage0/compiler/.dependencies","_build/stage0/compiler/build/GHCi.o")) * Depends on: _build/stage0/compiler/.dependencies * Depends on: _build/stage0/compiler/.dependencies.mk at need, called at src/Rules/Dependencies.hs:26:9 in main:Rules.Dependencies * Depends on: _build/stage0/compiler/build/PmOracle.hs at error, called at src/Rules/Generate.hs:105:34 in main:Rules.Generate * Raised the exception: No generator for _build/stage0/compiler/build/PmOracle.hs.

2 1

Coercible questions
by Michal Terepeta 05 Oct '19

05 Oct '19

Hi, I've started looking into using `Data.Coerce` (and the `Coercible` type-class) for a personal project and was wondering why coercing between `Int` and `Word` is not allowed? I don't see any fundamental reason why this shouldn't work... Perhaps, it's just a matter of GHC's implementation details leaking out? IIRC internally GHC has separate `RuntimeRep`/`PrimRep` for a `Word#` and for an `Int#`. If that's the case, would it make sense to unify these? Their actual runtime representation should be the same and I'd expect most (all?) of their differences should be attached to `PrimOp`s. And that leads me to another question--what exactly goes wrong here: ``` data Foo = Foo Int# data Bar = Bar Int# test :: Bar test = coerce (Foo 42#) ``` Which fails with: "Couldn't match representation of type ‘Foo’ with that of ‘Bar’ arising from a use of ‘coerce’" Perhaps I'm just misunderstanding exactly how `Coercible` works? Thanks in advance! - Michal PS. The ability to coerce through things like lists is amazing :)

1 0

Compiling projects with dependencies using GHC HEAD
by Alexis King 05 Oct '19

05 Oct '19

Hi all, I have a project that I’d like to try to build with GHC HEAD, but it’s a large project with a lot of dependencies, and while there’s a lot of information on building GHC itself, I haven’t been able to find very many resources on what workflow people use to actually build projects in the wild using it. I found this wiki page, “Installing packages in your test compiler,” which has a little guidance, but the details are sparse: https://gitlab.haskell.org/ghc/ghc/wikis/debugging/installing-packages-inpl… However, I’ve been struggling to actually get my project to build using cabal-install paired with a stage 2 compiler. I realize some of this is just fundamental—various libraries need to be updated for new versions of GHC—but I’m willing to go to the effort to manually resolve those conflicts so long as I know the right way to go about doing it. One simple thing I did try (and failed at) is building an updated version of cabal-install itself using my updated GHC. From inside my checkout of the GHC repo, I tried the following: $ cd libraries/Cabal/cabal-install $ cabal --version cabal --version cabal-install version 2.4.1.0 compiled using version 2.4.1.0 of the Cabal library $ cabal --with-compiler=<path to ghc> --package-db=<path to package db> \ new-build cabal-install However, this failed to compile due to out-of-bounds versions of certain packages, so I tried again with --allow-newer: $ cabal --with-compiler=<path to ghc> --package-db=<path to package db> \ new-build cabal-install --allow-newer=base,ghc-prim,template-haskell This time, it managed to produce a build plan, but for reasons I don’t understand, both Cabal-3.1.0.0 and Cabal-2.4.1.0 are in the build plan, and the latter fails to compile against newer versions of base (due to the MonadFail changes). I tried to figure out what was causing Cabal-2.4.1.0 to end up in the build plan, but running new-build with the -v3 option didn’t seem to give me any more explanation. Apologies if this is really a request for support with cabal-install, and it doesn’t belong on this list, but I figured I’d start here, since I imagine the people on this list are best-equipped to answer questions involving using GHC HEAD. I’d appreciate any pointers or resources anyone has! Thanks, Alexis

2 4

How to work out why a data constructor is allocated using gdb?
by Matthew Pickering 04 Oct '19

04 Oct '19

Hi, I am trying to work out questions such as * Why are there thousands of Module data constructors allocated when building something with GHC * What is allocating all a lot of strings when building GHC? In order to do this I can use gdb in order to find some of the Module/String closures but then I'm a bit stuck about what to do. gdb displays a list of all the Module closures for example and then you can usually try to find the retainer for Module by using findPtr. If the retainer is a THUNK closure, it would be easy, as THUNK closures have DWARF information which maps straight to a particular line. However if the retainer is just some other data constructor, for example, the Module is stored in a Map, it's data constructors all the way up and none of them have DWARF info. I need to fall back to domain specific knowledge to work out where such a sequence of constructors might appear in my program. * Is there anything better I can do to map a constructor allocation to a more precise source location? The string closures were causing me some particular issues as `findPtr` was not showing any retainers so it's hard to work out why they are not GCd. * What situations can an object be retained but show no retainer when using findPtr? Cheers, Matt

2 3

SPAM snippets on GitLab
by Takenobu Tani 03 Oct '19

03 Oct '19

Hi Ben, devs, It looks like GHC gitlab snipeets [1] are exposed to spam. Could we control them? [1] https://gitlab.haskell.org/explore/snippets Regards, Takenobu

2 2

GitLab upgrade soon
by Ben Gamari 02 Oct '19

02 Oct '19

Hi everyone, In about 30 minutes I will start an upgrade of gitlab.haskell.org. I expect it shouldn't take long but there will be a few minutes of outage. Cheers, - Ben

1 2