- ghc-devs - Haskell.org

typePrimRep invariants
by Csaba Hruska 01 Dec '18

01 Dec '18

Hi, I'd like to export the PrimRep of every binder and data con from Haskell modules. I have a modified GHC 8.6 which serializes the PrimRep for every binder during the compilation. It uses the *typePrimRep* function. When my customised GHC compiles the base library it always thows the following error for the *libraries/base/GHC/Err.hs* <https://github.com/grin-tech/ghc-grin/blob/master/ghc-8.6.2/libraries/base/…> module. 1. Does that mean that there is hidden requirement for typePrimRep input? 2. If so what are the restrictions? 3. Can you see anything in the Err module source code that could not be a proper input for typePrimRep function? Thanks, Csaba "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -H32m -O -Wall -this-unit-id base-4.12.0.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -Ilibraries/base/dist-install/build/include -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id ghc-prim-0.5.3 -package-id integer-gmp-1.0.2.0 -package-id rts -this-unit-id base -Wcompat -Wnoncanonical-monad-instances -XHaskell2010 -O2 -haddock -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build -split-sections -dynamic-too -c libraries/base/./GHC/Err.hs -o libraries/base/dist-install/build/GHC/Err.o -dyno libraries/base/dist-install/build/GHC/Err.dyn_o ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.6.2 for x86_64-unknown-linux): runtimeRepPrimRep typePrimRep (a_a1ML :: TYPE r_a1MK) r_a1MK Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/utils/Outputable.hs:1160:37 in ghc:Outputable pprPanic, called at compiler/simplStg/RepType.hs:358:5 in ghc:RepType Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug libraries/base/ghc.mk:4: recipe for target 'libraries/base/dist-install/build/GHC/Err.o' failed make[1]: *** [libraries/base/dist-install/build/GHC/Err.o] Error 1 Makefile:122: recipe for target 'all' failed make: *** [all] Error 2

1 2

Moving to GitLab...
by Ben Gamari 01 Dec '18

01 Dec '18

tl;dr. Have a look at [1], feel free to submit merge requests, issues, comments, code review, etc. Some documentation describing a few common tasks can be found here[2]. However, be aware that we this is not the final instance and will be cleared before the final migration in around two weeks. Hello everyone, A few weeks ago I wrote to this list proposing that we consider moving GHC's development infrastructure to GitLab. While the original proposal provided a small test instance to play with, it wasn't complete enough to use in earnest. Today I would like to announce the availability of https://gitlab.staging.haskell.org for your perusal and usage. While this is not the final migrated instance, it does have all of the features that one can expect from the final migration. These include, * a full import of Trac tickets (as of last week), including attachments * continuous integration via CircleCI * mirrors of all boot libraries * the ability to login using GitHub credentials There are a few issues that we are still working on sorting out: * the timestamps associated with ticket open and close events aren't quite right * some milestone changes aren't properly imported * CircleCI currently fails on forks (this should be resolved shortly) * we currently don't import Trac Wiki pages All of these issues have either already been resolved in the import tool or are in-progress. # The plan moving forward The goal of this instance is to allow contributors to gain experience using GitLab and identify potential friction points. Towards that end, please do make good use of it. In particular we are interested in identifying: * workflows that will become harder under GitLab (and ways that we could improve these) * remaining issues in the Trac import * areas lacking in documentation Please do let us know if you encounter any of the above. Ultimately the goal remains to cut over to GitLab on December 18. This will require that we bring down this instance for roughly a day to seed it with a new import. Note that we will not make any attempt to preserve any comments, merge requests, or issues created on this instance. Cheers, - Ben [1] https://gitlab.staging.haskell.org/ghc/ghc [2] https://gitlab.staging.haskell.org/ghc/ghc/wikis/home

1 0

Considerations for Control flow hint implementation.
by Andreas Klebinger 01 Dec '18

01 Dec '18

Hello Devs, I've started thinking about the implementation of https://github.com/ghc-proposals/ghc-proposals/pull/182 recently. (Add control flow hint pragmas.) For this purpose I've rebased D4327 "WIP: Add likelyhood to alternatives from stg onwards" which already does a lot of the work at the Cmm/Stg level. The issue I ask you for feedback now is how to best attach branch weights to case alternatives in core. My prefered approach would be to expand core data types to include them unconditionally. While this is quite far reaching in the amount of code it touches it would be rather straight forward to implement: Alternative 1: Putting the weights directly into the case alternative tuple: + It it's trivial to check which places manipulate case alternatives as they will initially fail to compile. + It's very mechanical, almost all use sites won't actually change the weight. + It's easy to keep this working going forward as any new optimizations can't "forget" they have to consider them. - It will introduce a cost in compiler performance. - New optimization who don't have to care about branchweights still have to at least pipe them through. - While syntactically heavy in terms of real complexity it's a simply approach. Alternative 2: Putting the weights into the case constructor. + Might give better compiler performance as I expect us to rebuild cases less often than alternatives. - Seems kind of clunky. - Weaker coupling between case alternatives and their weights. Or we could use ticks: + There is some machinery already there + Can be turned off for -O0 + Can be ignored when convenient. - Can be ignored when convenient. - Very weak coupling between case alternatives and their weights. - The existing machinery doesn't exactly match the needs of this. - We would have to extend tick semantics to a degree where complexity might grow too large for me to successfully implement this. - If new optimizations end up just removing these ticks because they are allowed to then the whole exercise becomes rather pointless. - Makes it harder to ensure all relevant code paths in GHC are actually updated. In particular there is currently no tick category which can stick to case alternatives but just get's removed in case it get's in the way of optimizations. The closest match is SoftScope which allows ticks to be floated up, something that could impact performance quite badly in this case. As then we might float something intended to mark a branch as unlikely into another branch that is actually along the hot path. I think the core variant(s) mostly stand and fall with the actual compile time impact. For -O0 the impact would be negligible as the compile time is already dominated by codegen and typechecking. For the rest it's hard to say. So I'm looking for feedback on this. Maybe you have other suggestions I haven't considered? How much compile time cost increase would be acceptable for what kind of performance boost? Cheers, Andreas Klebinger

2 2

Writing a simple Core evaluator, having trouble with name lookups
by Christopher Done 01 Dec '18

01 Dec '18

Hi all, I'm attempting to make a simple evaluator for GHC core, but I'm not clear on how to reliably looking up names. I'm compiling each of ghc-prim, integer-simple and base with a patched version of GHC which performs an extra output step with the Core AST to a file for each module. Later, I load those files in. So for an input Haskell file like this: module Main (main,Foo(..)) where class Foo a where foo :: a -> Int instance Foo Int where foo x = x * x instance Foo Char where foo x = 99 main = print (foo (123 :: Int)) I have an output set of bindings like this: https://gist.github.com/chrisdone/cb05a77d3fcb081a4580b5f85289674a One thing that I immediately notice is that the names of things are completely non-unique, especially in generated names. So here are two implementations of the method "foo" for the class "Foo": ( Id {idStableName = "main:Main:$cfoo", idUnique = Unique 6989586621679010917}, ...) -- Int ( Id {idStableName = "main:Main:$cfoo", idUnique = Unique 6989586621679010923}, ...) -- Char So e.g. the instance for "Foo Int" refers to the above method implementation via its Unique (6989586621679010923): ( Id {idStableName = "main:Main:$fFooInt", idUnique = Unique 8214565720323784705} , CastE (VarE (Id { idStableName = "main:Main:$cfoo" , idUnique = Unique 6989586621679010923 <---- HERE }))) At first, I thought I would use the Unique associated with every Name to make a lookup. This is completely reliable within one GHC compilation, but I've read in the docs that it's not stable across multiple invocations? What does that mean for my case? Another thing I notice is that type-class methods are not generated at the core level. I have, for example, this method call that provides it the instance dictionary, (AppE (AppE (VarE (Id { idStableName = "main:Main:foo" , idUnique = Unique 8214565720323784707 <---- MISSING })) (TypE (Typ "Int"))) (VarE (Id { idStableName = "main:Main:$fFooInt" , idUnique = Unique 8214565720323784705 }))) But the "main:Main:foo" (8214565720323784707) is not produced in the CoreProgram, it seems. My compile step is very simple: compile :: GHC.GhcMonad m => GHC.ModSummary -> m [CoreSyn.Bind GHC.Var] compile modSummary = do parsedModule <- GHC.parseModule modSummary typecheckedModule <- GHC.typecheckModule parsedModule desugared <- GHC.desugarModule typecheckedModule let binds = GHC.mg_binds (GHC.dm_core_module desugared) pure binds It simply gets the bindings and that's all from the ModGuts. mg_binds :: !CoreProgram Two questions: 1) How do I recognize class methods when I see one, like the "main:Main:foo" above? Maybe this? isClassOpId_maybe :: Id -> Maybe Class Is an "op" what GHC calls type-class methods? 2) If I compile e.g. ghc-prim and that generates a binding Name with ID 123, and then I compile base -- will the ID 123 be re-used by base for something else, or will any reference to 123 in the compiled Names for base refer ONLY to that one in ghc-prim? In other words, when GHC loads the iface for ghc-prim, does it generate a fresh set of names for everything in ghc-prim, or does it load them from file? Cheers!

4 11

make slow test suite failures (at least on OSX ) for 8.6 branch
by Carter Schonwald 30 Nov '18

30 Nov '18

to reproduce cd testsuite make slow TEST="EtaExpandLevPoly ManyAlternatives ManyConstructors MultiLayerModules T10370 T11535 T12707 T13379 T13701 T13719 T14683 T14697 T14936 T15349 T4334 T6132 T7702 T9203 T9630 ghci063 haddock.Cabal haddock.base haddock.compiler hpc_fork signals004 space_leak_001" -------------------- Unexpected results from: TEST="EtaExpandLevPoly ManyAlternatives ManyConstructors MultiLayerModules T10370 T11535 T12707 T13379 T13701 T13719 T14683 T14697 T14936 T15349 T4334 T6132 T7702 T9203 T9630 ghci063 haddock.Cabal haddock.base haddock.compiler hpc_fork signals004 space_leak_001" SUMMARY for test run started at Thu Nov 29 21:03:34 2018 EST 0:55:09 spent to go through 26 total tests, which gave rise to 137 test cases, of which 20 were skipped 0 had missing libraries 63 expected passes 0 expected failures 4 caused framework failures 0 caused framework warnings 6 unexpected passes 6 unexpected failures 38 unexpected stat failures Unexpected passes: runghc/T6132.run T6132 [unexpected] (normal) runghc/T6132.run T6132 [unexpected] (hpc) runghc/T6132.run T6132 [unexpected] (optasm) runghc/T6132.run T6132 [unexpected] (profasm) typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [unexpected] (profasm) typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [unexpected] (profthreaded) Unexpected failures: deriving/should_run/T11535.run T11535 [bad exit code] (ghci) ghci/scripts/ghci063.run ghci063 [bad stderr] (ghci) simplCore/should_compile/T7702.run T7702 [exit code non-0] (profasm) ../../libraries/base/tests/T15349.run T15349 [bad exit code] (ghci) ../../libraries/hpc/tests/fork/hpc_fork.run hpc_fork [bad heap profile] (profasm) ../../libraries/unix/tests/signals004.run signals004 [bad exit code] (threaded2) Unexpected stat failures: perf/compiler/T10370.run T10370 [stat not good enough] (optasm) perf/compiler/T12707.run T12707 [stat not good enough] (hpc) perf/compiler/T12707.run T12707 [stat not good enough] (optasm) perf/compiler/T12707.run T12707 [stat not good enough] (profasm) perf/compiler/T13379.run T13379 [stat not good enough] (hpc) perf/compiler/MultiLayerModules.run MultiLayerModules [stat not good enough] (optasm) perf/compiler/ManyConstructors.run ManyConstructors [stat not good enough] (hpc) perf/compiler/ManyConstructors.run ManyConstructors [stat not good enough]

1 0

perf.haskell.org functional again
by Joachim Breitner 30 Nov '18

30 Nov '18

Hi, after a system upgrade to avoid weird linker errors, and after some fixes in the nofib submodule, http://perf.haskell.org/ghc is running again. I am collecting instruction counts instead of runtime, because the latter was just too often varying too wildly. I hope this will yield less false alarms. I am also running nofib with mode=fast. This way, building GHC, running the testsuite and nofib takes a bit over one hour. I hope this can keep up with y'all's commits (when it took 2h it couldn't). Now nothing of the setup requires a quiet dedicated machine, so if there is a need, we could move these builds into the cloud or into some CI infrastructure - but no changes are immediately planned. Enjoy, Joachim

5 11

Windows test failures
by Simon Peyton Jones 30 Nov '18

30 Nov '18

Ben, Phyx, and other CI folk 'sh validate -fast' still gives a lot of failures on Window. The output is below. I would be so good to get this to zero! (another minor thing is that it would be great to eliminate the long path at the beginning - this is platform independent - I have to delete manually many times each day. Thanks Simon Unexpected passes: /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/rts/T9405.run T9405 [unexpected] (normal) Unexpected failures: /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/driver/T14452.run T14452 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/linking/ghcilink001.run ghcilink001 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/prog010/ghci.prog010.run ghci.prog010 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/scripts/ghci050.run ghci050 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/should_run/T14963a.run T14963a [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/print012.run print012 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/print016.run print016 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/break019.run break019 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/break028.run break028 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/T2740.run T2740 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/indexed-types/should_fail/T7102a.run T7102a [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/rts/T10672/T10672_x64.run T10672_x64 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/TH_spliceGuard.run TH_spliceGuard [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/TH_overlaps.run TH_overlaps [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/T5886.run T5886 [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/T10697_decided_3.run T10697_decided_3 [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/T11629.run T11629 [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/libraries/base/tests/T4006.run T4006 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/libraries/base/tests/IO/environment001.run environment001 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/should_run/T15633b.run T15633b [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins07.run plugins07 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins09.run plugins09 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins10.run plugins10 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins11.run plugins11 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/T10420.run T10420 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) Framework failures: /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/perf/compiler/MultiLayerModules.run MultiLayerModules [runTest] (Unhandled exception during cleanup: Unable to remove folder '/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/perf/compiler/MultiLayerModules.run': [Errno 13] Permission denied: '/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/perf/compiler/MultiLayerModules.run' Unable to start current test.) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/libraries/base/tests/IO/T3307.run T3307 [normal] ('latin-1' codec can't encode characters in position 24-25: ordinal not in range(256))

3 2

Kind inference
by Simon Peyton Jones 29 Nov '18

29 Nov '18

I have been working for longer than I care to admit on a significant refactoring of kind inference. I've just pushed it. It's a pretty big patch, which Richard and I have been discussing for some weeks. It validates, and fixes at least nine troublesome open tickets. But of course I could have made a mistake. Be vigilant. My: there will almost certainly be conflicts with your visible kind application patch, especially in tcInferApps. But that function has gotten substantially simpler in my patch so I don't think it'll be hard to fix things up. Phew! Simon

1 0

llvm way only tests?
by Gabor Greif 29 Nov '18

29 Nov '18

Hi all, what is the magic incantation for running tests only when the llvm tools (e.g. `opt`) are around? I am currently declaring the test as test('T15155l', [ only_ways(llvm_ways), ], run_command, ['$MAKE -s --no-print-directory T15155l']) but it won't be included neither in the optasm builds (which is okay) nor in the llvm builds (gets skipped unexpetedly) I looked at the other test which are declared similarly, and they behave just like mine. What I am doing wrong? Thanks, Gabor

2 2

condensed summary of vanilla mojave ghc 8.6.2 errors with clang
by Carter Schonwald 27 Nov '18

27 Nov '18

the vast majority of the errors i previously shared seem to be "can't find atom for stabs BNSYM at 0000AF80 in /Users/carter/dev-checkouts/ghc-tree/ghc-8.6.X-checkout-clang-build/libraries/base/dist-install/build/libHSbase-4.12.0.0.a(Internals.o)" which corresponds with this code in the apple linker https://github.com/llvm-mirror/lld/blob/master/lib/ReaderWriter/MachO/MachO… i dont know to what to make of this

1 2