- ghc-devs - Haskell.org

RE: ghc 8.0.2 vs 8.4.1 compilation time and performance
by Simon Peyton Jones 23 Mar '18

23 Mar '18

Evan That’s truly great information, thank you. Could you start a GHC ticket for it? Stuff gets lots in email, less so in tickets. The big take-away I got from your message is this: score max mb total mb prd derive lily perform ghc 6 72.26 3279.22 0.88 0.79~0.84 0.70~0.74 0.31~0.33 8.0.2 6 76.63 3419.20 0.58 1.45~1.59 1.05~1.07 0.33~0.36 8.4.1 bloom 70.69 2456.14 0.89 1.32~1.36 0.15~0.16 8.0.2 bloom 67.86 2589.97 0.62 1.94~1.99 0.20~0.22 8.4.1 The bytes-allocated number has gone up a bit. (Not too surprising… the compiler is doing more.) But the productivity number is down sharply, and consistently so, which translates directly into longer compile times. Somehow, although residency is not increasing, GC time is greatly increased. We have some compiler/perf regression tests that track bytes-allocated, but not that track productivity, which is why we have noticed. It’d be good to figure out what’s gone wrong here. Maybe a change in nursery size or something stupid like that? It’d be great if someone felt up to investigating a bit Thanks for gathering this data. Simon From: Glasgow-haskell-users <glasgow-haskell-users-bounces(a)haskell.org> On Behalf Of Evan Laforge Sent: 21 March 2018 22:05 To: GHC users <glasgow-haskell-users(a)haskell.org> Subject: ghc 8.0.2 vs 8.4.1 compilation time and performance I just upgraded from 8.0.2 to 8.4.1, and I took the opportunity to do a few informal compile time and run time tests. There's been a lot of talk about compile time regressions, so maybe these will be of interest, informal as they are. I wound up skipping 8.2.1 due to https://ghc.haskell.org/trac/ghc/ticket/13604, but I could still test with 8.2 perfectly well. Just haven't done it yet. In this context, RunTests is more code with no optimization (and -fhpc, if it matters). debug/seq and opt/seq are the same code but with no optimization and -O respectively. I found that -O2 hurt compile time but didn't improve run time, but it's been a long time so I should run that experiment again. compile times: OS X, macbook pro: RunTests 549.10s user 118.45s system 343% cpu 3:14.53 total 8.0.2 RunTests 548.71s user 117.10s system 347% cpu 3:11.78 total 8.0.2 RunTests 450.92s user 109.63s system 343% cpu 2:43.13 total 8.4.1 RunTests 445.48s user 107.99s system 341% cpu 2:42.19 total 8.4.1 debug/seq 284.47s user 55.95s system 345% cpu 1:38.58 total 8.0.2 debug/seq 283.33s user 55.27s system 343% cpu 1:38.53 total 8.0.2 debug/seq 220.92s user 50.21s system 337% cpu 1:20.32 total 8.4.1 debug/seq 218.39s user 49.20s system 345% cpu 1:17.47 total 8.4.1 opt/seq 732.63s user 70.86s system 338% cpu 3:57.30 total 8.0.2 opt/seq 735.21s user 71.48s system 327% cpu 4:06.31 total 8.0.2 opt/seq 785.12s user 65.42s system 327% cpu 4:19.84 total 8.4.1 opt/seq 765.52s user 64.01s system 321% cpu 4:18.29 total 8.4.1 Linux, PC: RunTests 781.31s user 58.21s system 363% cpu 3:50.70 total 8.0.2 RunTests 613.11s user 49.84s system 357% cpu 3:05.52 total 8.4.1 debug/seq 429.44s user 31.34s system 362% cpu 2:07.03 total 8.0.2 debug/seq 329.67s user 23.86s system 352% cpu 1:40.38 total 8.4.1 opt/seq 1277.20s user 45.85s system 358% cpu 6:08.68 total 8.0.2 opt/seq 1339.73s user 39.87s system 341% cpu 6:43.50 total 8.4.1 So it looks like non-optimized compile times have gotten significantly better since 8.0.2. However, optimized has gotten a little worse, but not much. The performance numbers are a bit more disappointing. At first it appeared that allocation went down in 8.4.1 while overall time is up significantly. However, the 8.4.1 used newer dependencies, so to try to control for those, I tested again after using a cabal freeze from the 8.4.1 test. Of course I had to remove the ghc distributed packages, like container and 'ghc' itself, but the rest of the deps should be the same. Those have the 'libs' suffix on Linux. From that, it looks like the improved memory in 8.4.1 was due to external dependencies, and in fact 8.4.1 bumped memory usage up again. Ow. In the graphs, 'score' is just the input file. 'max mb' and 'total mb' and 'prd' come from the post-run GC report, specifically '* bytes maximum residency', '* bytes allocated in the heap', and productivity fields. 'derive', 'lily' and 'perform' are just different kinds of processes. They are CPU time bracketing the specific action, after initialization, and the range is min and max over 6 runs, so no fancy criterion-like analysis. Each run is a separate process, so they should be independent. I was hoping for some gains due to the join points stuff, but it kind of looks like things get worse across the board. I don't know why productivity goes down so much, and I don't know why the effect seems so much worse on OS X. Of course the obvious next step is to see where 8.2.1 lies, but I thought I'd see if there's interest before going to the trouble. Of course, I should track down the regressions for my own purposes, but it's a bit of a daunting task. The step of reducing to a minimal example seems a lot harder for performance than for a bug! Probably some old fashioned SCC annotations await me, but that can be a long and confusing process. OS X, macbook pro: score max mb total mb prd derive lily perform ghc 6 72.26 3279.22 0.88 0.79~0.84 0.70~0.74 0.31~0.33 8.0.2 6 76.63 3419.20 0.58 1.45~1.59 1.05~1.07 0.33~0.36 8.4.1 bloom 70.69 2456.14 0.89 1.32~1.36 0.15~0.16 8.0.2 bloom 67.86 2589.97 0.62 1.94~1.99 0.20~0.22 8.4.1 cerucuk-punyah 138.01 10080.55 0.93 6.98~7.16 1.24~1.30 8.0.2 cerucuk-punyah 130.78 9617.35 0.68 8.91~9.22 1.57~1.68 8.4.1 hex 32.86 2120.95 0.91 0.76~0.88 0.16~0.19 8.0.2 hex 32.67 2194.82 0.66 1.09~1.16 0.28~0.30 8.4.1 p1 67.01 4039.82 0.92 2.63~2.73 0.47~0.50 8.0.2 p1 64.80 3899.85 0.68 3.35~3.43 0.58~0.59 8.4.1 viola-sonata 69.32 6083.65 0.92 2.48~2.56 2.07~2.13 0.25~0.26 8.0.2 viola-sonata 66.76 6120.26 0.68 3.32~3.43 2.90~2.93 0.32~0.34 8.4.1 Linux, PC: score max mb total mb prd derive lily perform ghc 6 79.76 3310.69 0.89 0.88~0.89 0.73~0.75 0.27~0.27 8.0.2 6 72.21 3421.45 0.90 0.87~0.87 0.72~0.79 0.28~0.28 8.0.2 libs 6 76.56 3419.05 0.77 1.16~1.17 0.87~0.93 0.33~0.33 8.4.1 bloom 69.82 2461.95 0.89 1.35~1.36 0.17~0.17 8.0.2 bloom 63.45 2554.89 0.90 1.33~1.35 0.18~0.18 8.0.2 libs bloom 67.79 2589.85 0.79 1.64~1.65 0.20~0.20 8.4.1 cerucuk-punyah 137.05 10113.41 0.94 7.44~7.50 1.31~1.33 8.0.2 cerucuk-punyah 128.09 10278.03 0.94 7.50~7.55 1.37~1.38 8.0.2 libs cerucuk-punyah 131.20 9617.22 0.84 7.35~7.40 1.49~1.50 8.4.1 hex 32.02 2096.87 0.92 0.73~0.74 0.18~0.18 8.0.2 hex 32.05 2200.30 0.91 0.73~0.80 0.19~0.19 8.0.2 libs hex 32.46 2144.22 0.83 0.89~0.90 0.20~0.20 8.4.1 p1 65.88 4054.66 0.93 2.84~2.87 0.49~0.50 8.0.2 p1 62.60 4127.68 0.94 2.83~2.92 0.51~0.51 8.0.2 libs p1 64.72 3899.72 0.81 2.80~2.81 0.54~0.55 8.4.1 viola-sonata 68.68 6086.49 0.93 2.55~2.56 2.10~2.12 0.27~0.27 8.0.2 viola-sonata 65.05 6212.57 0.93 2.52~2.55 2.07~2.16 0.28~0.28 8.0.2 libs viola-sonata 66.85 6120.15 0.83 2.91~2.92 2.48~2.51 0.30~0.31 8.4.1

2 1

Why does SpecConstr run far later than specialisation
by Matthew Pickering 20 Mar '18

20 Mar '18

As far as I understand it, the goals of SpecConstr and specialisation are the same. One works for normal value arguments and one works for special type class dictionary arguments. However, looking at the pass order in `SimplCore`, specialisation runs very early before any major simplification but SpecConstr runs very late right at the end of the pipeline. Does anyone know the justification for this? It seems intuitively that they should work in the same way. Cheers, Matt

3 2

What does "return" keyword mean in INFO_TABLE_RET declarations?
by Ömer Sinan Ağacan 20 Mar '18

20 Mar '18

Hi, I'm trying to understand what a "return" list in INFO_TABLE_RET declaration line specifies. As far as I understand a "return" in the declaration line is something different than a "return" in the body. For example, in this definition: (in HeapStackCheck.cmm) INFO_TABLE_RET ( stg_ret_p, RET_SMALL, W_ info_ptr, P_ ptr ) return (/* no return values */) { return (ptr); } The return list is empty and it even says "no return values" explicitly, yet it returns something. My guess is that the "return" list in the header is actually for arguments. I found this info table which has an argument: (in StgMiscClosures.cmm) INFO_TABLE_RET (stg_restore_cccs_eval, RET_SMALL, W_ info_ptr, W_ cccs) return (P_ ret) { unwind Sp = Sp + WDS(2); #if defined(PROFILING) CCCS = cccs; #endif jump stg_ap_0_fast(ret); } This is the use site: (in Interpreter.c) #if defined(PROFILING) // restore the CCCS after evaluating the closure Sp_subW(2); SpW(1) = (W_)cap->r.rCCCS; SpW(0) = (W_)&stg_restore_cccs_eval_info; #endif Sp_subW(2); SpW(1) = (W_)tagged_obj; SpW(0) = (W_)&stg_enter_info; RETURN_TO_SCHEDULER_NO_PAUSE(ThreadRunGHC, ThreadYielding); If I understand this correctly, the "tagged_obj" code will put the return value in R1, pop the stack (which will have stg_restore_ccs_eval_info at the bottom) and jump to this the info table code shown above. So `P_ ret` is the value of `tagged_obj`, and the "return" list is actually for parameters. Did I get this right? If I did, I'm curious why it's called "return" and not "args" or something like that. Thanks, Ömer

4 7

ghcpkg05
by Simon Peyton Jones 20 Mar '18

20 Mar '18

On my Linux platform, with an in-place build, I get a consistent failure of ghcpkg05. Any ideas how I can fix it? Thanks Simon =====> ghcpkg05(normal) 1 of 1 [0, 0, 0] cd "./ghcpkg05.run" && $MAKE -s --no-print-directory ghcpkg05 Actual stderr output differs from expected: diff -uw "./ghcpkg05.run/ghcpkg05.stderr.normalised" "./ghcpkg05.run/ghcpkg05.run.stderr.normalised" --- ./ghcpkg05.run/ghcpkg05.stderr.normalised 2018-03-19 11:31:12.533918366 +0000 +++ ./ghcpkg05.run/ghcpkg05.run.stderr.normalised 2018-03-19 11:31:12.533918366 +0000 @@ -10,6 +10,13 @@ cannot find any of ["C/D.hi","C/D.p_hi","C/D.dyn_hi"] cannot find any of ["C/E.hi","C/E.p_hi","C/E.dyn_hi"] cannot find any of ["libtestpkg-2.0-XXX.a","libtestpkg-2.0-XXX.p_a","libtestpkg-2.0-XXX-ghc<VERSION>.so","libtestpkg-2.0-XXX-ghc<VERSION>.dylib","testpkg-2.0-XXX-ghc<VERSION>.dll"] on library path +Warning: include-dirs: /5playpen/simonpj/HEAD-1/compiler/stage2/build/utils doesn't exist or isn't a directory +Warning: include-dirs: /5playpen/simonpj/HEAD-1/compiler/stage2/build/../rts/dist/build doesn't exist or isn't a directory +Warning: include-dirs: /5playpen/simonpj/HEAD-1/compiler/stage2/build/stage2 doesn't exist or isn't a directory +Warning: include-dirs: /5playpen/simonpj/HEAD-1/libraries/haskeline/dist-install/build/includes doesn't exist or isn't a directory +Warning: include-dirs: /5playpen/simonpj/HEAD-1/libraries/text/dist-install/build/include doesn't exist or isn't a directory +Warning: include-dirs: /5playpen/simonpj/HEAD-1/libraries/containers/dist-install/build/include doesn't exist or isn't a directory +Warning: include-dirs: /5playpen/simonpj/HEAD-1/libraries/bytestring/dist-install/build/include doesn't exist or isn't a directory The following packages are broken, either because they have a problem listed above, or because they depend on a broken package. *** unexpected failure for ghcpkg05(normal) Unexpected results from: TEST="ghcpkg05"

2 2

Viewing the types declared in GHC.Prim in local machine
by Abhiroop Sarkar 19 Mar '18

19 Mar '18

Is it possible to view the types generated in the GHC.Prim module on my local machine? The commentary[1] states that it is a virtual module and a source file including dummy declarations are available in this location: ` libraries/base/GHC/Prim.hs`, however, upon downloading and building the latest GHC codebase I could not find this dummy file on my machine. Do I have to modify my Makefile somehow to generate these dummy types? For building, I have followed the instructions mentioned here[2] and set `BuildFlavour = devel2`. Thanks, Abhiroop Sarkar [1]https://ghc.haskell.org/trac/ghc/wiki/Commentary/PrimOps [2]https://ghc.haskell.org/trac/ghc/wiki/Newcomers

3 2

Haskell Report 2010: pattern bindings
by Simon Peyton Jones 19 Mar '18

19 Mar '18

Thanks Jose. The other issue I have in mind is that of pattern bindings. Currently in the 2010 Haskell Report * The text on the monomorphism restriction<https://www.haskell.org/onlinereport/haskell2010/haskellch4.html#x10-880004…> (4.5.5) refers to “simple pattern bindings”. * It claims that a simple pattern binding “is a pattern binding in which the pattern consists of a single variable” * You’d expect “simple pattern binding” to be defined in 3.17.1 Patterns<https://www.haskell.org/onlinereport/haskell2010/haskellch3.html#x8-580003.…>. But it isn’t. * Instead, 4.4.3.2 Pattern bindings<https://www.haskell.org/onlinereport/haskell2010/haskellch4.html#x10-800004…> explicitly contradicts 4.5.5 by saying “a simple pattern binding has form p=e”. What a mess! The simplest way out is probably: * Define “simple pattern binding” to be of form x=e, in 3.17.1. * Fix the first para of 4.4.3.2. * Fix the cross-ref in 4.5.5 to refer to the definition of simple pattern. Sadly, 4.4.3.2 still is not very well defined. It’s supposed to say what pattern bindings mean, including top-level ones; but it does so by referring to 3.12 which does not handle top-level bindings. I’m not sure if this is worth fixing. Simon From: José Manuel Calderón Trilla <jmct(a)jmct.cc> Sent: 15 March 2018 23:17 To: Simon Peyton Jones <simonpj(a)microsoft.com>; haskell-prime(a)haskell.org; ghc-devs(a)haskell.org Subject: Re: [Haskell] The Haskell Report: who maintains it? All of us on the Haskell Language Committee have the ability to commit on that repo. I think typos are uncontroversial and I'll happily merge pull requests like that. I think the pull-request you point out suffered from the bystander effect, unfortunately. I'll review and merge it now. If you let me know the typo you'd like fixed I'll make sure that gets done as well. Cheers, José Manuel On Thu, Mar 15, 2018, at 6:52 PM, Simon Peyton Jones via Haskell wrote: Friends Does anyone know who, if anyone, feels responsible for committing updates to the Haskell 2010 Report? Who even has commit rights? There’s Frank’s pull request below, and I have another important typo to fix. Thanks Simon From: Frank Steffahn [mailto:notifications@github.com] Sent: 11 March 2018 17:03 To: haskell/haskell-report <haskell-report(a)noreply.github.com> Cc: Subscribed <subscribed(a)noreply.github.com> Subject: [haskell/haskell-report] Fix a typo in: Semantics of Case Expressions, Part 3 (s) (#4) Hi. I noticed this in the Haskell 2010 report, which is an obvious typo / mistake. I’m not 100% sure if this is the right branch (or even in general the right place) to note this, but I hope it will get fixed ;-) This seems like it is an artifact of copy-and-pasting from “Semantics of Case Expressions, Part 1 (c)” without properly adapting the thing, especially in commit bc94554. ________________________________ You can view, comment on, or merge this pull request online at: https://github.com/haskell/haskell-report/pull/4<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> Commit Summary * Fix a typo in: Semantics of Case Expressions, Part 3 (s) File Changes * M report/exps.verb<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> (1) Patch Links: * https://github.com/haskell/haskell-report/pull/4.patch<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> * https://github.com/haskell/haskell-report/pull/4.diff<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…>. _______________________________________________ Haskell mailing list Haskell(a)haskell.org<mailto:Haskell@haskell.org> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskel…>

1 0

Is "cml_cont" of CmmCall used in practice?
by Shao, Cheng 19 Mar '18

19 Mar '18

Hi all, Is the "cml_cont" field of the CmmCall variant is really used in practice? I traversed the output of raw Cmm produced by ghc compiling the whole base package, but the value of cml_cont is always Nothing. Regards, Shao Cheng

6 5

Re: [GHC] #13776: -ddump-splices produces unnecessarily qualified names for tuple and list types
by Soham Chowdhury 17 Mar '18

17 Mar '18

That sounds great! I never got around to working on this myself, feel free :) On Sun, Mar 18, 2018, 00:45 GHC <ghc-devs(a)haskell.org> wrote: > #13776: -ddump-splices produces unnecessarily qualified names for tuple > and list > types > -------------------------------------+------------------------------------- > Reporter: RyanGlScott | Owner: mrkgnao > Type: bug | Status: new > Priority: normal | Milestone: > Component: Template Haskell | Version: 8.0.1 > Resolution: | Keywords: newcomer > Operating System: Unknown/Multiple | Architecture: > | Unknown/Multiple > Type of failure: None/Unknown | Test Case: > Blocked By: | Blocking: > Related Tickets: | Differential Rev(s): > Wiki Page: | > -------------------------------------+------------------------------------- > > Comment (by ckoparkar): > > I'd like to work on this, if that's OK with @mrkgnao. > > I have a patch on my local machine > that does the right thing for the examples posted in the > description (and some tests: `th/T3319`, `th/T5700`, > `th/TH_foreignInterruptible`). > While I work on submitting that on Phabricator, I wanted to post a summary > here and get some early feedback. > > > (1) It seems that `showName` doesn't play a role in pretty-printing the > splices > with `-ddump-splices`. Instead, the `Outputable` instances in GHC do most > of the > work. Specifically, `Outputable RdrName` is responsible for printing out > the fully > qualified names in question. > > (2) When the Renamer typechecks & runs a splice (`RnSplice.runRnSplice`), > it > converts the splice to `HsSyn RdrName` (hence the `Outputable RdrName`). > `TcSplice.lookupThName` is involved in the process, which converts > a `TH.Name` to `Name` via `Convert.thRdrNameGuesses`. > > (3) For primitives like `[]`, `(:)` etc. `TH.dataToQa` generates a fully > qualified > global name, i.e `NameG NameSpace PkgName ModName`. > And the corresponding `RdrName` generated by `thRdrNameGuesses` is also > fully > qualified (`Orig Module OccName`). But this is not what we want for built- > in syntax. > > (4) So the "patch" is a simple change to modify this behavior. If > `thOrigRdrName` is > dealing with built-in syntax, it returns an `Exact Name` instead. > > {{{#!haskell > > thOrigRdrName :: String -> TH.NameSpace -> PkgName -> ModName -> RdrName > thOrigRdrName occ th_ns pkg mod = > let occ' = mk_occ (mk_ghc_ns th_ns) occ > in case isBuiltInOcc_maybe occ' of > Just name -> nameRdrName name > Nothing -> (mkOrig $! (mkModule (mk_pkg pkg) (mk_mod mod))) $! occ' > > }}} > > I ran the testsuite, and apart from some `perf` tests, almost everything > else worked. > > These tests fail: > > {{{ > ghci/linking/ghcilink003.run ghcilink003 [bad exit code] (normal) > > ghci/linking/ghcilink006.run ghcilink006 [bad exit code] (normal) > > th/T13366.run -- (gcc: error trying to exec 'cc1plus': > execvp: > -- No such file or directory) > }}} > > but there's a good chance that this is unrelated to the patch. > > > Does the overall approach seem reasonable ? I'll submit a patch soon. > > -- > Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13776#comment:5> > GHC <http://www.haskell.org/ghc/> > The Glasgow Haskell Compiler >

1 0

Custom ghcPrimIface for cross-compilation?
by Shao Cheng 16 Mar '18

16 Mar '18

Hi all, is it possible for a 64-bit ghc to emit 32-bit code, if I supply a custom ghcPrimIface via Hooks and also modify the platform flags in DynFlags? The module does not import Prelude and has no dependencies other than GHC.Prim.

3 2

The Haskell Report: who maintains it?
by Simon Peyton Jones 15 Mar '18

15 Mar '18

Friends Does anyone know who, if anyone, feels responsible for committing updates to the Haskell 2010 Report? Who even has commit rights? There’s Frank’s pull request below, and I have another important typo to fix. Thanks Simon From: Frank Steffahn [mailto:notifications@github.com] Sent: 11 March 2018 17:03 To: haskell/haskell-report <haskell-report(a)noreply.github.com> Cc: Subscribed <subscribed(a)noreply.github.com> Subject: [haskell/haskell-report] Fix a typo in: Semantics of Case Expressions, Part 3 (s) (#4) Hi. I noticed this in the Haskell 2010 report, which is an obvious typo / mistake. I’m not 100% sure if this is the right branch (or even in general the right place) to note this, but I hope it will get fixed ;-) This seems like it is an artifact of copy-and-pasting from “Semantics of Case Expressions, Part 1 (c)” without properly adapting the thing, especially in commit bc94554<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…>. ________________________________ You can view, comment on, or merge this pull request online at: https://github.com/haskell/haskell-report/pull/4<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> Commit Summary * Fix a typo in: Semantics of Case Expressions, Part 3 (s) File Changes * M report/exps.verb<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> (1) Patch Links: * https://github.com/haskell/haskell-report/pull/4.patch<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> * https://github.com/haskell/haskell-report/pull/4.diff<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com…>.

3 2