- ghc-devs - Haskell.org

Confused about PAP object layout
by Ömer Sinan Ağacan 27 Feb '20

27 Feb '20

Hi Simon, In this code: (slightly simplified) StgPtr scavenge_PAP (StgPAP *pap) { evacuate(&pap->fun); return scavenge_PAP_payload (pap->fun, pap->payload, pap->n_args); } StgPtr scavenge_PAP_payload (StgClosure *fun, StgClosure **payload, StgWord size) { const StgFunInfoTable *fun_info = get_fun_itbl(UNTAG_CONST_CLOSURE(fun)); StgPtr p = (StgPtr)payload; switch (fun_info->f.fun_type) { case ARG_GEN_BIG: scavenge_large_bitmap(p, GET_FUN_LARGE_BITMAP(fun_info), size); p += size; break; ... } return p; } Here the `size` argument in `scavenge_PAP_payload` is the number of arguments applied to the function in `pap->fun`. But when scavenging the function's bitmap we're using this number as the size of the bitmap which doesn't make sense to me, because I think size of the function's bitmap and size of the PAP's payload may be different. Or in other words I may have the same function used in many PAPs with different n_args, but that'd be buggy if this code is correct. I haven't checked every single place where we build a PAP but for example the `NEW_PAP` macro uses the argument's (another PAP) function directly, without making any bitmap-related changes, but bumps n_args by one. If the code above is right, then this new PAP will be scavenged incorrectly. Am I missing anything? Thanks, Ömer

5 13

ghc 8.8.x change in behaviour when loading in-memory modules
by Tseen She 26 Feb '20

26 Feb '20

Dear ghc devs, I am in the process of updating my dev tool, which uses the ghc api, to support ghc-8.8.x. However, I have noticed that one of my regression tests is failing, and I have isolated the problem to a change in behaviour in typecheckModule when using a Target with an in-memory buffer (i.e. targetContents is a Just). Using debug tracing, I can confirm that in the following code, the ms_hspp_buf of a ModSummary has my in-memory version: https://gitlab.com/tseenshe/hsinspect/-/blob/dfcb50a4e3437fbb4d0a3eb786b99c… minf_rdr_env' :: GHC.GhcMonad m => GHC.ModuleName -> m GlobalRdrEnv minf_rdr_env' m = do modSum <- GHC.getModSummary m pmod <- GHC.parseModule modSum tmod <- GHC.typecheckModule pmod let (tc_gbl_env, _) = GHC.tm_internals_ tmod pure $ tcg_rdr_env tc_gbl_env You can see how I setup the session in https://gitlab.com/tseenshe/hsinspect/-/blob/dfcb50a4e3437fbb4d0a3eb786b99c… And whereas in 8.4.4 and 8.6.5 the typecheckModule call is working on the in-memory representation, in 8.8.1 and 8.8.2 it is falling back to the version on disk and therefore failing, because I have made in-memory changes that are necessary. Aside: I am able to observe this because I have a function that trims everything except the module declaration and import section, and can therefore tolerate typos in the rest of the file. Now my code fails if the entire file does not typecheck, which is also incredibly slow, because I only want access to the import section. Best regards, Tseen She

1 0

Time-profiling the compiler pipeline
by Sergej Jaskiewicz 26 Feb '20

26 Feb '20

Hello everyone, Recently I've been exploring ways to improve the LLVM support in GHC. While doing that, I've encountered an issue that I think is worth looking into. Namely, the compilation times with LLVM are considerably longer than with the native code generator. To be more concrete, on my machine I've managed to build the whole toolchain in just half an hour with NCG, but the same build with LLVM-powered GHC took several hours. If we want to improve compilation times, we should be able to measure it first. As far as I understand, the only way for profiling the compiler pipeline today is building GHC with special flags: https://gitlab.haskell.org/ghc/ghc/wikis/debugging/profiling-ghc This has three important downsides: * We need build the whole compiler ourselves. If I'm just a user interested in why my program is so slow to compile and how I should fix it, this is too much for me to ask to do. * There's just too much irrelevant data in the output. For example, it doesn't tell me which part of my Haskell code is taking so long to compile. It would be nice if the report contained all the metadata that I need to go and fix my code. * If we're using LLVM, we're pretty much out of luck, because we're using an existing LLVM binary which is likely not built for profiling. There have been the same problem with the Clang compiler until recently. In Clang 9, which was released in the last year, there's a new flag '-ftime-trace' that produces a beautiful flame chart with all the metadata needed to identify bottlenecks. This flag tells Clang to produce a JSON file that can be visualized in Chrome's chrome://tracing page, or on https://www.speedscope.app. This is described in more details here: https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profile… I'm thinking about implementing similar functionality in GHC. One option would be to utilize the plugin infrastructure. Maybe I'm missing something and there are better options? — Sergej.

2 1

Windows build broken
by Andreas Klebinger 26 Feb '20

26 Feb '20

Hi devs, it seems the windows build is broken. (Can't build stage2 locally). Quickest way to reproduce is a validate of master. It also happens on CI: https://gitlab.haskell.org/ghc/ghc/-/jobs/270248 I remember ben mentioning something along the lines of builds with "old" (8.6) boot compilers being broken. Is that the culprint? How do I get back to a functioning environment? Cheers, Andreas

2 1

[ANNOUNCE] GHC 8.8.2 is now available
by Ben Gamari 25 Feb '20

25 Feb '20

Hello everyone, The GHC team is proud to announce the release of GHC 8.8.2. The source distribution, binary distributions, and documentation are available at https://downloads.haskell.org/~ghc/8.8.2 Release notes are also available [1]. This release fixes a handful of issues affecting 8.8.1: - A bug (#17088) in the compacting garbage collector resulting in segmentations faults under specific circumstances. Note that this may affect user programs even if they did not explicitly request the compacting GC (using the -c RTS flag) since GHC may fallback to compacting collection during times of high memory pressure. - A code generation bug (#17334) resulting in GHC panics has been fixed. - A bug in the `process` library causing builds using `hsc2hs` to fail non-deterministically on Windows has been fixed (Trac #17480) - A typechecker bug (#12088) resulting in programs being unexpectedly rejected has been fixed. - A bug in the implementation of compact normal forms resulting in segmentation faults in some uses (#17044) has been fixed. - A bug causing GHC to incorrectly complain about incompatible LLVM versions when using LLVM 7.0.1 has been fixed (#16990). As always, if anything looks amiss do let us know. Happy compiling! Cheers, - Ben [1] https://downloads.haskell.org/ghc/8.8.2/docs/html/users_guide/8.8.2-notes.h…

2 4

Calling an unknown function from low-level Cmm
by Alexis King 24 Feb '20

24 Feb '20

Hi all, I’m trying to understand how to properly call an unknown function from low-level Cmm code. If I’m just applying a function to a state token, it’s easy; I can just do R1 = io; jump stg_ap_v_fast [R1]; since the calling convention is consistent in that case. But what if my function takes actual arguments? I can’t do R1 = fun; R2 = arg; jump stg_ap_p_fast [R1, R2]; because if the calling convention doesn’t pass any arguments in registers, that would be wrong. I could check if NO_ARG_REGS is defined and generate different code in that situation, but that seems extreme. One option I think would work would be to do R1 = fun; Sp_adj(-2); Sp(1) = arg; jump RET_LBL(stg_ap_p) [R1]; but that seems wasteful if I have the argument in a register already anyway. Am I missing something? Thanks, Alexis

2 2

is backport planned for 8.8.X of MR 1304?
by Csaba Hruska 23 Feb '20

23 Feb '20

Hello, Will MR 1304 <https://gitlab.haskell.org/ghc/ghc/merge_requests/1304> be backported to GHC 8.8.X? Thanks, Csaba

2 1

I'm concerned about the affine-types extension and its impact on ghc and haskell at large
by Carter Schonwald 19 Feb '20

19 Feb '20

https://github.com/ghc-proposals/ghc-proposals/pull/111#issuecomment-583664… https://www.reddit.com/r/haskell/comments/f0jigv/im_concerned_about_the_lon… As current maintainer of vector, and a member of the CLC, both of which roles really should be AFFIRMATIONAL stakeholders in this, I only see costs and concerns. As someone who's spent the past few years doing a LOT of modelling and prototyping around marrying linear logic, formal methods, and functional programming in an applied industrial setting, i should be an Affirmational stakeholder. yet again I am not. theres very real costs in the complexity and scope of impact that impact EVERY single user and stakeholder of ghc and haskell. And I do not see any concrete population that benefits. cale even makes a very constructive and articulate point > I don't know how much my opinion matters at this point, but I'd really like to see the non-toy real-world use cases of this before I can even consider thinking that it would be a good idea to merge to the main compiler. It has such a huge potential impact on so many people in the Haskell community: > > * Library maintainers who start getting PRs that make "improvements" to linearity while making their libraries harder to maintain because their interface becomes more rigid, and harder to understand because the types are more complicated. > > * Beginners, or simply ordinary users of the language who have to pay the mental overhead of living with the linear types and polymorphism spreading everywhere as types are adjusted to make terms more usable in places where one is concerned with linearity. > > * Commercial users of the language who pay for the additional time taken by all their employees waiting for the compiler to run, regardless of whether or not they're using the extension. If Carter's 6-7% slowdown is real even in cases where one doesn't care about the extension, I can imagine wanting to make a fork without the extension. The compiler is already 2 orders of magnitude slower than I'd like. If that weren't the case, maybe 6-7% wouldn't be a huge deal. While ghci is often helpful at shortening the feedback loop, it's not always a viable solution. > > > But really, I just want to know how anyone would put this to practical use in a real setting -- it needs to be _really_ compelling to make up for the social cost. It can't be that hard for Tweag, or someone enthusiastic, to live on a fork until they have a decent case study to show the world, so we can say "okay, that's actually really cool, maybe we actually want that everywhere".

3 3

source-dist from GHC master branch
by Csaba Hruska 18 Feb '20

18 Feb '20

Hello, I'd like to create a source distribution (source-dist) from GHC git master branch. I tried hadrian (hadrian/build.sh source-dist) and make (make sdist-ghc) but both failed in different ways. Should this work (at all) for any git snapshot? Hadrian successfully built a source dist (ghc-8.11.0.20200215-src.tar.xz) but it did not compile. Regards, Csaba The error output (make sdist-ghc): > make sdist-ghc > make --no-print-directory -f ghc.mk sdist-ghc NO_INCLUDE_DEPS=YES > NO_INCLUDE_PKGDATA=YES > ghc.mk:1221: warning: overriding recipe for target > 'sdist_compiler_stage2_Cmm' > ghc.mk:1220: warning: ignoring old recipe for target > 'sdist_compiler_stage2_Cmm' > make[1]: *** No rule to make target 'compiler/stage2/build/Cmm.hs', needed > by 'sdist_compiler_stage2_Cmm'. Stop. > Makefile:162: recipe for target 'sdist-ghc' failed > make: *** [sdist-ghc] Error 2 > > csaba@x1:~/haskell/grin-compiler/ghc-playground/ghc$ make sdist-ghc > make --no-print-directory -f ghc.mk sdist-ghc NO_INCLUDE_DEPS=YES > NO_INCLUDE_PKGDATA=YES > ghc.mk:1221: warning: overriding recipe for target > 'sdist_compiler_stage2_Cmm' > ghc.mk:1220: warning: ignoring old recipe for target > 'sdist_compiler_stage2_Cmm' > make[1]: *** No rule to make target 'compiler/stage2/build/Cmm.hs', needed > by 'sdist_compiler_stage2_Cmm'. Stop. > Makefile:162: recipe for target 'sdist-ghc' failed > make: *** [sdist-ghc] Error 2 Error output hadrian: > | Run GenPrimopCode: _build/stage0/compiler/build/primops.txt => > _build/stage0/compiler/build/primop-vector-tys-exports.hs-incl > | Run Ghc FindHsDependencies Stage0: compiler/main/Annotations.hs (and 475 > more) => _build/stage0/compiler/.dependencies.mk > > <no location info>: error: > module ‘ghc-8.6.2:Lexer’ is defined in multiple files: > compiler/GHC/Cmm/Lexer.hs > > compiler/parser/Lexer.hs > Error when running Shake build system: > at action, called at src/Rules.hs:71:19 in main:Rules > at need, called at src/Rules.hs:93:5 in main:Rules > * Depends on: _build/stage0/lib/package.conf.d/ghc-8.11.0.20200215.conf > at need, called at src/Rules/Register.hs:116:5 in main:Rules.Register > * Depends on: _build/stage0/compiler/build/libHSghc-8.11.0.20200215.a > at need, called at src/Rules/Library.hs:146:5 in main:Rules.Library > * Depends on: _build/stage0/compiler/build/Cpr.o > at &%>, called at src/Rules/Compile.hs:62:9 in main:Rules.Compile > * Depends on: _build/stage0/compiler/build/Cpr.o > _build/stage0/compiler/build/Cpr.hi > at apply1, called at > src/Development/Shake/Internal/Rules/Oracle.hs:159:32 in > shake-0.18.3-a2e904f154f09728357733d7a3e8955a2a87773d2d7eb963a6f88d9d883fdde3:Development.Shake.Internal.Rules.Oracle > * Depends on: OracleQ (KeyValues > ("_build/stage0/compiler/.dependencies","_build/stage0/compiler/build/Cpr.o")) > at need, called at src/Hadrian/Oracles/TextFile.hs:96:9 in > main:Hadrian.Oracles.TextFile > * Depends on: _build/stage0/compiler/.dependencies > at readFile', called at src/Rules/Dependencies.hs:34:19 in > main:Rules.Dependencies > at need, called at src/Development/Shake/Internal/Derived.hs:118:15 in > shake-0.18.3-a2e904f154f09728357733d7a3e8955a2a87773d2d7eb963a6f88d9d883fdde3:Development.Shake.Internal.Derived > * Depends on: _build/stage0/compiler/.dependencies.mk > * Raised the exception: > user error (Development.Shake.cmd, system command failed >

2 1

core to core plugin before inline
by Florian Engel 18 Feb '20

18 Feb '20

Hello, I'm trying to fix an error in https://github.com/conal/concat. The problem is the core to core passes of the plugin are way to slow for simple functions. For slightly more complicated one but still simple functions, it fails after a long time of compiling (see the errGrad tests in examples/test/Examples.hs). We suppose that the problem is related to inline. Is there a way to run the core-to-core passes of the plugin before inline. Best regards Florian Engel

2 1