- ghc-devs - Haskell.org

Time-profiling the compiler pipeline
by Sergej Jaskiewicz 26 Feb '20

26 Feb '20

Hello everyone, Recently I've been exploring ways to improve the LLVM support in GHC. While doing that, I've encountered an issue that I think is worth looking into. Namely, the compilation times with LLVM are considerably longer than with the native code generator. To be more concrete, on my machine I've managed to build the whole toolchain in just half an hour with NCG, but the same build with LLVM-powered GHC took several hours. If we want to improve compilation times, we should be able to measure it first. As far as I understand, the only way for profiling the compiler pipeline today is building GHC with special flags: https://gitlab.haskell.org/ghc/ghc/wikis/debugging/profiling-ghc This has three important downsides: * We need build the whole compiler ourselves. If I'm just a user interested in why my program is so slow to compile and how I should fix it, this is too much for me to ask to do. * There's just too much irrelevant data in the output. For example, it doesn't tell me which part of my Haskell code is taking so long to compile. It would be nice if the report contained all the metadata that I need to go and fix my code. * If we're using LLVM, we're pretty much out of luck, because we're using an existing LLVM binary which is likely not built for profiling. There have been the same problem with the Clang compiler until recently. In Clang 9, which was released in the last year, there's a new flag '-ftime-trace' that produces a beautiful flame chart with all the metadata needed to identify bottlenecks. This flag tells Clang to produce a JSON file that can be visualized in Chrome's chrome://tracing page, or on https://www.speedscope.app. This is described in more details here: https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profile… I'm thinking about implementing similar functionality in GHC. One option would be to utilize the plugin infrastructure. Maybe I'm missing something and there are better options? — Sergej.

2 1

Windows build broken
by Andreas Klebinger 25 Feb '20

25 Feb '20

Hi devs, it seems the windows build is broken. (Can't build stage2 locally). Quickest way to reproduce is a validate of master. It also happens on CI: https://gitlab.haskell.org/ghc/ghc/-/jobs/270248 I remember ben mentioning something along the lines of builds with "old" (8.6) boot compilers being broken. Is that the culprint? How do I get back to a functioning environment? Cheers, Andreas

2 1

[ANNOUNCE] GHC 8.8.2 is now available
by Ben Gamari 25 Feb '20

25 Feb '20

Hello everyone, The GHC team is proud to announce the release of GHC 8.8.2. The source distribution, binary distributions, and documentation are available at https://downloads.haskell.org/~ghc/8.8.2 Release notes are also available [1]. This release fixes a handful of issues affecting 8.8.1: - A bug (#17088) in the compacting garbage collector resulting in segmentations faults under specific circumstances. Note that this may affect user programs even if they did not explicitly request the compacting GC (using the -c RTS flag) since GHC may fallback to compacting collection during times of high memory pressure. - A code generation bug (#17334) resulting in GHC panics has been fixed. - A bug in the `process` library causing builds using `hsc2hs` to fail non-deterministically on Windows has been fixed (Trac #17480) - A typechecker bug (#12088) resulting in programs being unexpectedly rejected has been fixed. - A bug in the implementation of compact normal forms resulting in segmentation faults in some uses (#17044) has been fixed. - A bug causing GHC to incorrectly complain about incompatible LLVM versions when using LLVM 7.0.1 has been fixed (#16990). As always, if anything looks amiss do let us know. Happy compiling! Cheers, - Ben [1] https://downloads.haskell.org/ghc/8.8.2/docs/html/users_guide/8.8.2-notes.h…

2 4

Calling an unknown function from low-level Cmm
by Alexis King 24 Feb '20

24 Feb '20

Hi all, I’m trying to understand how to properly call an unknown function from low-level Cmm code. If I’m just applying a function to a state token, it’s easy; I can just do R1 = io; jump stg_ap_v_fast [R1]; since the calling convention is consistent in that case. But what if my function takes actual arguments? I can’t do R1 = fun; R2 = arg; jump stg_ap_p_fast [R1, R2]; because if the calling convention doesn’t pass any arguments in registers, that would be wrong. I could check if NO_ARG_REGS is defined and generate different code in that situation, but that seems extreme. One option I think would work would be to do R1 = fun; Sp_adj(-2); Sp(1) = arg; jump RET_LBL(stg_ap_p) [R1]; but that seems wasteful if I have the argument in a register already anyway. Am I missing something? Thanks, Alexis

2 2

is backport planned for 8.8.X of MR 1304?
by Csaba Hruska 23 Feb '20

23 Feb '20

Hello, Will MR 1304 <https://gitlab.haskell.org/ghc/ghc/merge_requests/1304> be backported to GHC 8.8.X? Thanks, Csaba

2 1

I'm concerned about the affine-types extension and its impact on ghc and haskell at large
by Carter Schonwald 19 Feb '20

19 Feb '20

https://github.com/ghc-proposals/ghc-proposals/pull/111#issuecomment-583664… https://www.reddit.com/r/haskell/comments/f0jigv/im_concerned_about_the_lon… As current maintainer of vector, and a member of the CLC, both of which roles really should be AFFIRMATIONAL stakeholders in this, I only see costs and concerns. As someone who's spent the past few years doing a LOT of modelling and prototyping around marrying linear logic, formal methods, and functional programming in an applied industrial setting, i should be an Affirmational stakeholder. yet again I am not. theres very real costs in the complexity and scope of impact that impact EVERY single user and stakeholder of ghc and haskell. And I do not see any concrete population that benefits. cale even makes a very constructive and articulate point > I don't know how much my opinion matters at this point, but I'd really like to see the non-toy real-world use cases of this before I can even consider thinking that it would be a good idea to merge to the main compiler. It has such a huge potential impact on so many people in the Haskell community: > > * Library maintainers who start getting PRs that make "improvements" to linearity while making their libraries harder to maintain because their interface becomes more rigid, and harder to understand because the types are more complicated. > > * Beginners, or simply ordinary users of the language who have to pay the mental overhead of living with the linear types and polymorphism spreading everywhere as types are adjusted to make terms more usable in places where one is concerned with linearity. > > * Commercial users of the language who pay for the additional time taken by all their employees waiting for the compiler to run, regardless of whether or not they're using the extension. If Carter's 6-7% slowdown is real even in cases where one doesn't care about the extension, I can imagine wanting to make a fork without the extension. The compiler is already 2 orders of magnitude slower than I'd like. If that weren't the case, maybe 6-7% wouldn't be a huge deal. While ghci is often helpful at shortening the feedback loop, it's not always a viable solution. > > > But really, I just want to know how anyone would put this to practical use in a real setting -- it needs to be _really_ compelling to make up for the social cost. It can't be that hard for Tweag, or someone enthusiastic, to live on a fork until they have a decent case study to show the world, so we can say "okay, that's actually really cool, maybe we actually want that everywhere".

3 3

source-dist from GHC master branch
by Csaba Hruska 18 Feb '20

18 Feb '20

Hello, I'd like to create a source distribution (source-dist) from GHC git master branch. I tried hadrian (hadrian/build.sh source-dist) and make (make sdist-ghc) but both failed in different ways. Should this work (at all) for any git snapshot? Hadrian successfully built a source dist (ghc-8.11.0.20200215-src.tar.xz) but it did not compile. Regards, Csaba The error output (make sdist-ghc): > make sdist-ghc > make --no-print-directory -f ghc.mk sdist-ghc NO_INCLUDE_DEPS=YES > NO_INCLUDE_PKGDATA=YES > ghc.mk:1221: warning: overriding recipe for target > 'sdist_compiler_stage2_Cmm' > ghc.mk:1220: warning: ignoring old recipe for target > 'sdist_compiler_stage2_Cmm' > make[1]: *** No rule to make target 'compiler/stage2/build/Cmm.hs', needed > by 'sdist_compiler_stage2_Cmm'. Stop. > Makefile:162: recipe for target 'sdist-ghc' failed > make: *** [sdist-ghc] Error 2 > > csaba@x1:~/haskell/grin-compiler/ghc-playground/ghc$ make sdist-ghc > make --no-print-directory -f ghc.mk sdist-ghc NO_INCLUDE_DEPS=YES > NO_INCLUDE_PKGDATA=YES > ghc.mk:1221: warning: overriding recipe for target > 'sdist_compiler_stage2_Cmm' > ghc.mk:1220: warning: ignoring old recipe for target > 'sdist_compiler_stage2_Cmm' > make[1]: *** No rule to make target 'compiler/stage2/build/Cmm.hs', needed > by 'sdist_compiler_stage2_Cmm'. Stop. > Makefile:162: recipe for target 'sdist-ghc' failed > make: *** [sdist-ghc] Error 2 Error output hadrian: > | Run GenPrimopCode: _build/stage0/compiler/build/primops.txt => > _build/stage0/compiler/build/primop-vector-tys-exports.hs-incl > | Run Ghc FindHsDependencies Stage0: compiler/main/Annotations.hs (and 475 > more) => _build/stage0/compiler/.dependencies.mk > > <no location info>: error: > module ‘ghc-8.6.2:Lexer’ is defined in multiple files: > compiler/GHC/Cmm/Lexer.hs > > compiler/parser/Lexer.hs > Error when running Shake build system: > at action, called at src/Rules.hs:71:19 in main:Rules > at need, called at src/Rules.hs:93:5 in main:Rules > * Depends on: _build/stage0/lib/package.conf.d/ghc-8.11.0.20200215.conf > at need, called at src/Rules/Register.hs:116:5 in main:Rules.Register > * Depends on: _build/stage0/compiler/build/libHSghc-8.11.0.20200215.a > at need, called at src/Rules/Library.hs:146:5 in main:Rules.Library > * Depends on: _build/stage0/compiler/build/Cpr.o > at &%>, called at src/Rules/Compile.hs:62:9 in main:Rules.Compile > * Depends on: _build/stage0/compiler/build/Cpr.o > _build/stage0/compiler/build/Cpr.hi > at apply1, called at > src/Development/Shake/Internal/Rules/Oracle.hs:159:32 in > shake-0.18.3-a2e904f154f09728357733d7a3e8955a2a87773d2d7eb963a6f88d9d883fdde3:Development.Shake.Internal.Rules.Oracle > * Depends on: OracleQ (KeyValues > ("_build/stage0/compiler/.dependencies","_build/stage0/compiler/build/Cpr.o")) > at need, called at src/Hadrian/Oracles/TextFile.hs:96:9 in > main:Hadrian.Oracles.TextFile > * Depends on: _build/stage0/compiler/.dependencies > at readFile', called at src/Rules/Dependencies.hs:34:19 in > main:Rules.Dependencies > at need, called at src/Development/Shake/Internal/Derived.hs:118:15 in > shake-0.18.3-a2e904f154f09728357733d7a3e8955a2a87773d2d7eb963a6f88d9d883fdde3:Development.Shake.Internal.Derived > * Depends on: _build/stage0/compiler/.dependencies.mk > * Raised the exception: > user error (Development.Shake.cmd, system command failed >

2 1

core to core plugin before inline
by Florian Engel 18 Feb '20

18 Feb '20

Hello, I'm trying to fix an error in https://github.com/conal/concat. The problem is the core to core passes of the plugin are way to slow for simple functions. For slightly more complicated one but still simple functions, it fails after a long time of compiling (see the errGrad tests in examples/test/Examples.hs). We suppose that the problem is related to inline. Is there a way to run the core-to-core passes of the plugin before inline. Best regards Florian Engel

2 1

Re: [smartos-discuss] GHC 8.8.2 + cabal-install 3.0.0 finally working on SmartOS x64 20200117 (yet GHC 8.6.5 left less usable)
by Compl Yue 18 Feb '20

18 Feb '20

Great! And I suggest add 8.6.5 to the list, as I feel it may have wider adoption than other versions as time being, and fortunately the previous ghci problem with it has been fixed by backporting merge request 87, my repository has updated the build steps and the backporting patch. FYI. On 2020/2/18 下午7:41, Jonathan Perkin wrote: > * On 2020-02-10 at 08:26 GMT, Compl Yue via smartos-discuss wrote: > >> And regarding stack, as it's focused on managing binary distributions of GHC >> (build-from-src not working very well per my experience), it'll need a >> trusted binary repository for stack to work with on SmartOS. A private http >> server hosting the bindists should be straight forward, but I'm not >> interested to host a publicly available binary repository with those bindist >> tarballs digitally signed (or there're security concerns). If anyone is >> interested, I'd like to help as far as I can. > Thanks again for your work in paving the path forward with this. I've > used it to produce some official bootstraps for SmartOS here: > > https://us-east.manta.joyent.com/pkgsrc/public/pkg-bootstraps/ghc-7.6.3-boo… > https://us-east.manta.joyent.com/pkgsrc/public/pkg-bootstraps/ghc-7.10.3-bo… > https://us-east.manta.joyent.com/pkgsrc/public/pkg-bootstraps/ghc-8.0.2-boo… > https://us-east.manta.joyent.com/pkgsrc/public/pkg-bootstraps/ghc-8.4.4-boo… > > For some reason I'm seeing build errors with the 8.8.1 bootstrap > related to the rts/fs.h header but the normal release build works > fine, I should hopefully have that fixed soon. > > There are also release packages now available in the SmartOS trunk > package repository ready for users to install: > > $ pkgin se ^ghc- > ghc-8.8.1nb1 Compiler for the functional language Haskell - 8.8 Release Series > ghc-8.4.4 Compiler for the functional language Haskell - 8.4 Release Series > ghc-8.0.2 Compiler for the functional language Haskell - 8.0 Release Series > ghc-7.10.3nb2 Compiler for the functional language Haskell - 7.10 Release Series > > I'd be particularly interested in whether Haskell users can find any > issues with them. Each of the release packages were built using the > previous bootstrap kit listed above. > > Many thanks, >

1 0

GHC 8.8.2 miscompiles program with -fcatch-bottoms
by Csaba Hruska 16 Feb '20

16 Feb '20

Hello, I've tried to compile GHC from source using -fcatch-bottoms on the stage2. I compiled GHC with having *GhcStage2HcOpts += -fcatch-bottoms* in build.mk. Unfortunately it breaks programs and causes runtime error. I wanted to use this feature in an experiment. My motivation: *I'm working on a typed external STG IR were the type system is based according GHC's PrimRep ADT. In my Ext-STG IR each case alt must have the same PrimRep result type. The vanilla Core to Stg pass breaks this invariant but the catch-bottoms option preserves it. * Cheers, Csaba Error output: make ===--- building phase 0 make --no-print-directory -f ghc.mk phase=0 phase_0_builds make[1]: Nothing to be done for 'phase_0_builds'. ===--- building phase 1 make --no-print-directory -f ghc.mk phase=1 phase_1_builds make[1]: Nothing to be done for 'phase_1_builds'. ===--- building final phase make --no-print-directory -f ghc.mk phase=final all "inplace/bin/ghc-stage2" -hisuf dyn_hi -osuf dyn_o -hcsuf dyn_hc -fPIC -dynamic -H32m -O -Wall -hide-all-packages -i -iutils/haddock/driver -iutils/haddock/haddock-api/src -iutils/haddock/haddock-library/src -iutils/haddock/dist/build -Iutils/haddock/dist/build -iutils/haddock/dist/build/haddock/autogen -Iutils/haddock/dist/build/haddock/autogen -optP-DIN_GHC_TREE -optP-include -optPutils/haddock/dist/build/haddock/autogen/cabal_macros.h -package-id array-0.5.4.0 -package-id base-4.13.0.0 -package-id bytestring-0.10.10.0 -package-id containers-0.6.2.1 -package-id deepseq-1.4.4.0 -package-id directory-1.3.4.0 -package-id filepath-1.4.2.1 -package-id ghc-8.8.2 -package-id ghc-boot-8.8.2 -package-id parsec-3.1.14.0 -package-id text-1.2.4.0 -package-id transformers-0.5.6.2 -package-id xhtml-3000.2.2.1 -funbox-strict-fields -Wall -fwarn-tabs -O2 -threaded -XHaskell2010 -no-user-package-db -rtsopts -Wno-unused-imports -Wno-deprecations -Wnoncanonical-monad-instances -outputdir utils/haddock/dist/build -c utils/haddock/haddock-library/src/Documentation/Haddock/Types.hs -o utils/haddock/dist/build/Documentation/Haddock/Types.dyn_o *ghc-stage2: internal error: evacuate(static): strange closure type 0 (GHC version 8.8.2 for x86_64_unknown_linux) Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug <https://www.haskell.org/ghc/reportabug>* utils/haddock/ghc.mk:20: recipe for target 'utils/haddock/dist/build/Documentation/Haddock/Types.dyn_o' failed make[1]: *** [utils/haddock/dist/build/Documentation/Haddock/Types.dyn_o] Aborted (core dumped) Makefile:123: recipe for target 'all' failed make: *** [all] Error 2

2 1