September 2018 - ghc-devs

GHC's module hierarchy
by Simon Peyton Jones 02 Oct '19

02 Oct '19

Dear ghc-devs hsyl20 proposes a radical overhaul of the module structure of GHC itself. He or she suggested it six months ago in https://ghc.haskell.org/trac/ghc/ticket/13009 and has now offered a monster patch https://phabricator.haskell.org/D3647 It's clearly the result of a lot of work, but I was the only one who responded on the original ticket, and it'll affect all of your lives in a very immediate way. So, would you like to * consider the idea * look at the actual re-mapping of modules hsyl20 proposals * express an opinion about whether to go ahead Probably the ticket, rather than Phab, is the best place to comment on the general idea. I'd like to thank hsyl20. GHC's rather flat module structure has grown incrementally over years. But still, there are pros and cons. Simon

2 1

Any ways to test a GHC build against large set of packages (including test suites)?
by Ömer Sinan Ağacan 25 Jul '19

25 Jul '19

Hi, I'd like to test some GHC builds + some compile and runtime flag combinations against a large set of packages by building them and running test suites. For this I need - A set of packages that are known to work with latest GHC - A way to build them and run their test suites (if I could specify compile and runtime flags that'd be even better) I think stackage can serve as (1) but I don't know how to do (2). Can anyone point me to the right direction? I vaguely remember some nix-based solution for this that was being discussed on the IRC channel, but can't recall any details. Thanks, Ömer

9 15

Default options for -threaded
by lonetiger＠gmail.com 04 Jan '19

04 Jan '19

Hi All, A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why -N -qa isn’t the default for -threaded. I’m afraid I don’t know a good reason why it’s not. Can anyone help shed some light on this? If there is no good reason, Would anyone object to making it the default? Thanks, Tamar

7 10

perf.haskell.org functional again
by Joachim Breitner 30 Nov '18

30 Nov '18

Hi, after a system upgrade to avoid weird linker errors, and after some fixes in the nofib submodule, http://perf.haskell.org/ghc is running again. I am collecting instruction counts instead of runtime, because the latter was just too often varying too wildly. I hope this will yield less false alarms. I am also running nofib with mode=fast. This way, building GHC, running the testsuite and nofib takes a bit over one hour. I hope this can keep up with y'all's commits (when it took 2h it couldn't). Now nothing of the setup requires a quiet dedicated machine, so if there is a need, we could move these builds into the cloud or into some CI infrastructure - but no changes are immediately planned. Enjoy, Joachim

5 11

[ANNOUNCE] GHC 8.6.1 released
by Ben Gamari 22 Oct '18

22 Oct '18

Hello everyone, The GHC team is pleased to announce the availability of GHC 8.6.1, the fourth major release in the GHC 8 series. The source distribution, binary distributions, and documentation for this release are available at https://downloads.haskell.org/~ghc/8.6.1 The 8.6 release fixes over 400 bugs from the 8.4 series and introduces a number of exciting features. These most notably include: * A new deriving mechanism, `deriving via`, providing a convenient way for users to extend Haskell's typeclass deriving mechanism * Quantified constraints, allowing forall quantification in constraint contexts * An early version of the GHCi `:doc` command * The `ghc-heap-view` package, allowing introspection into the structure of GHC's heap * Valid hole fit hints, helping the user to find terms to fill typed holes in their programs * The BlockArguments extension, allowing the `$` operator to be omitted in some unambiguous contexts * An exciting new plugin mechanism, source plugins, allowing plugins to inspect and modify a wide variety of compiler representations. * Improved recompilation checking when plugins are used * Significantly better handling of macOS linker command size limits, avoiding linker errors while linking large projects * The next phase of the MonadFail proposal, enabling -XMonadFailDesugaring by default A full list of the changes in this release can be found in the release notes: https://downloads.haskell.org/~ghc/8.6.1/docs/html/users_guide/8.6.1-notes.… Perhaps of equal importance, GHC 8.6 is the second major release made under GHC's accelerated six-month release schedule and the first set of binary distributions built primarily using our new continuous integration scheme. While the final 8.6 release is around three weeks later than initially scheduled due to late-breaking bug reports, we expect that the 8.8 release schedule shouldn't be affected. Thanks to everyone who has contributed to developing, documenting, and testing this release! As always, let us know if you encounter trouble. How to get it ~~~~~~~~~~~~~ The easy way is to go to the web page, which should be self-explanatory: https://www.haskell.org/ghc/ We supply binary builds in the native package format for many platforms, and the source distribution is available from the same place. Packages will appear as they are built - if the package for your system isn't available yet, please try again later. Background ~~~~~~~~~~ Haskell is a standard lazy functional programming language. GHC is a state-of-the-art programming suite for Haskell. Included is an optimising compiler generating efficient code for a variety of platforms, together with an interactive system for convenient, quick development. The distribution includes space and time profiling facilities, a large collection of libraries, and support for various language extensions, including concurrency, exceptions, and foreign language interfaces. GHC is distributed under a BSD-style open source license. A wide variety of Haskell related resources (tutorials, libraries, specifications, documentation, compilers, interpreters, references, contact information, links to research groups) are available from the Haskell home page (see below). On-line GHC-related resources ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Relevant URLs on the World-Wide Web: GHC home page https://www.haskell.org/ghc/ GHC developers' home page https://ghc.haskell.org/trac/ghc/ Haskell home page https://www.haskell.org/ Supported Platforms ~~~~~~~~~~~~~~~~~~~ The list of platforms we support, and the people responsible for them, is here: https://ghc.haskell.org/trac/ghc/wiki/Contributors Ports to other platforms are possible with varying degrees of difficulty. The Building Guide describes how to go about porting to a new platform: https://ghc.haskell.org/trac/ghc/wiki/Building Developers ~~~~~~~~~~ We welcome new contributors. Instructions on accessing our source code repository, and getting started with hacking on GHC, are available from the GHC's developer's site run by Trac: https://ghc.haskell.org/trac/ghc/ Mailing lists ~~~~~~~~~~~~~ We run mailing lists for GHC users and bug reports; to subscribe, use the web interfaces at https://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets There are several other haskell and ghc-related mailing lists on www.haskell.org; for the full list, see https://mail.haskell.org/cgi-bin/mailman/listinfo Some GHC developers hang out on #haskell on IRC, too: https://www.haskell.org/haskellwiki/IRC_channel Please report bugs using our bug tracking system. Instructions on reporting bugs can be found here: https://www.haskell.org/ghc/reportabug

9 12

Goals for GHC 8.8
by Ben Gamari 16 Oct '18

16 Oct '18

Hi everyone, Since GHC 8.6 is almost released it's time to start thinking about our goals for 8.8, which will branch in November. I have added the items that I know are in-flight to the 8.8 status page[1]. If you have a project that you would like to see present in 8.8 then please do add it as well. If you are uncertain of whether you can have a patch up for review by late October then please do speak to me and we can discuss options. Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-8.8.1

5 8

Feedback request: GHC performance test-suite
by David Eichmann 10 Oct '18

10 Oct '18

Hello all, I've recently resumed some work started by Jared Weakly on the GHC test suite. This specifically regards performance tests. The work aims to, among some other things, reduce manual work and to log performance test results from our CI server. The proposed change is described in more detail on this wiki page: https://ghc.haskell.org/trac/ghc/wiki/Performance/Tests. I'd appreciate any feedback or questions on this. Thank you and have a great day, David Eichmann -- David Eichmann, Haskell Consultant Well-Typed LLP, http://www.well-typed.com Registered in England & Wales, OC335890 118 Wymering Mansions, Wymering Road, London W9 2NF, England

3 4

nofib oldest GHC to support?
by Ömer Sinan Ağacan 04 Oct '18

04 Oct '18

Do we have a policy on the oldest GHC to support in nofib? I'm currently doing some hacking on nofib to parse some new info printed by a modified GHC, and I think we can do a lot of cleaning (at the very least remove some regexes and parsers) if we decide on which GHCs to support. I checked the README and RunningNoFib wiki page but couldn't see anything relevant. Thanks Ömer

3 3

Re: Stable name allocation
by David Feuer 27 Sep '18

27 Sep '18

Simon seems a bit busy right now. Can anyone else advise me on the basics of heap allocation in primops? On Tue, Sep 25, 2018 at 1:42 PM, David Feuer <david.feuer(a)gmail.com> wrote: > Let's forget about allocate(). I can definitely handle this part in > C--. But I'm still lost in the macros and such. For example, I'm very > unclear on the differences among the ALLOC, HP_CHK, and MAYBE_GC > classes of macro. I can't find anything in the commentary, and the > source code documentation is very sparse. I'm okay with either of the > following approaches, but either way I need a bit more info. > > 1. First see if we need to allocate a StableName#. If so, check > whether GC would be required to allocate the StableName# (how?). If > so, drop the lock, run GC (how?) and start over. This looks cleanest > to me if it can be done easily. > > 2. First run the GC if we're low on memory (how?). Then if we need to > allocate a StableName#, we'll be sure to have room. > > > On Tue, Sep 25, 2018 at 6:25 AM, Simon Marlow <marlowsd(a)gmail.com> wrote: >> You can do it unconditionally before taking the lock, or you can do it >> conditionally as long as you release the lock if the heap check fails. I >> think in the latter case there might not be a macro that allows this, but >> you could use the `allocate()` method for allocating memory (like >> newByteArray#) and then you could write a heap check like the MAYBE_GC() >> macro. Doing it unconditionally is easier and probably not a big performance >> hit, but note that you'll have to retreat Hp if you don't use the memory. >> >> Cheers >> Simon >> >> On Sat, 22 Sep 2018 at 13:08, David Feuer <david.feuer(a)gmail.com> wrote: >>> >>> How do I check if GC will be required, and how do I trigger it? Should I >>> perform the check unconditionally at the beginning of the operation so I >>> don't have to drop the lock, GC, then retake? I don't know the right ways to >>> deal with this stuff, and the macros are mostly undocumented. >>> >>> On Sep 22, 2018 3:53 AM, "Simon Marlow" <marlowsd(a)gmail.com> wrote: >>> >>> Yes, the current implementation looks like it creates the object after >>> adding the entry to the StableName table and releasing the lock, which is >>> unsafe because another thread could read that same entry before the object >>> has been created. The easiest solution to that is to take and release the >>> lock in C-- in the right places instead of in the C lookupStableName() >>> function (you might need to make a separate version of lookupStableName() >>> that doesn't take the lock). >>> >>> Cheers >>> Simon >>> >>> >>> On Fri, 21 Sep 2018 at 12:53, David Feuer <david.feuer(a)gmail.com> wrote: >>>> >>>> It seems awkward to do it in C--, but maybe you can help me work out how. >>>> The allocation facilities definitely seem much nicer there, and allocating a >>>> small heap object in C feels like an abuse of the facilities we have there. >>>> The essential challenge, as I see it, is that we need the key to point to a >>>> valid stable name object by the time we drop the hash table lock. The >>>> process, as I imagine it: >>>> >>>> 1. Follow indirections, untag, choose the right generation. (All this is >>>> in C) >>>> 2. Take the appropriate hash table lock. (C) >>>> 3. Look up the key in the hash table (C). >>>> >>>> Now there's a branch. If we found the key, then we don't need to allocate >>>> an SNO. We just drop the lock and return. Otherwise >>>> >>>> 4. Allocate an SNO and set its info pointer (most easily done in C--). If >>>> this necessitates GC, we need to drop the lock first and might as well just >>>> go back to the very beginning afterwards. >>>> 5. Populate the SNO with its "hash value" (we can do this anywhere we >>>> like, I imagine). >>>> 6. Insert the key and SNO into the hash table and drop the hash table >>>> lock (C) >>>> 7. If necessary, insert the SNO into the remembered set (C) >>>> >>>> How would you recommend structuring this back-and-forth? >>>> >>>> On Fri, Sep 21, 2018, 3:19 AM Simon Marlow <marlowsd(a)gmail.com> wrote: >>>>> >>>>> I'm a bit sceptical that you need to allocate a heap object in C instead >>>>> of C--, but still, here's an example: >>>>> https://phabricator.haskell.org/diffusion/GHC/browse/master/rts%2FThreads.c… >>>>> >>>>> It's slightly less efficient to do this in C than C--, because >>>>> `allocate()` is slower than allocating by bumping `Hp`. >>>>> >>>>> On Mon, 17 Sep 2018 at 21:25, David Feuer <david.feuer(a)gmail.com> wrote: >>>>>> >>>>>> How can I allocate a heap object in C code in the rts? I've only seen >>>>>> heap objects allocated in C--, and doing that here would be lousy for >>>>>> performance and worse for clarity. >>>>>> >>>>>> David >>> >>> >>

2 1

RE: Windows testsuite failures
by Simon Peyton Jones 25 Sep '18

25 Sep '18

Tamar Thank you, that’s great! For the ones that need more work, can we mark them as expect-broken, so that they don’t pollute the testsuite output? Simon From: lonetiger(a)gmail.com <lonetiger(a)gmail.com> Sent: 24 September 2018 07:13 To: Simon Peyton Jones <simonpj(a)microsoft.com> Subject: RE: Windows testsuite failures Hi Simon, I created some patches to fix the majority of these https://phabricator.haskell.org/D5174 https://phabricator.haskell.org/D5175 https://phabricator.haskell.org/D5176 The remaining ones I’ve either pinged the patches that caused the issues or created tickets For them because they’re actual bugs that require a bit more time to find the cause and fix. https://ghc.haskell.org/trac/ghc/ticket/15668 https://ghc.haskell.org/trac/ghc/ticket/15669 https://ghc.haskell.org/trac/ghc/ticket/15670 https://ghc.haskell.org/trac/ghc/ticket/15671 these should bring down the amount of failing tests to about 5. Kind Regards, Tamar From: Simon Peyton Jones<mailto:simonpj@microsoft.com> Sent: Thursday, September 20, 2018 12:05 To: Phyx<mailto:lonetiger@gmail.com> Subject: RE: Windows testsuite failures Thanks Tamar. I’ll look forward to hearing back S From: Phyx <lonetiger(a)gmail.com<mailto:lonetiger@gmail.com>> Sent: 20 September 2018 12:02 To: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Cc: ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> Subject: Re: Windows testsuite failures Hi Simon, Thanks for the email. I haven't been building head much as I'm working on top of some older commits. From a quick look it seems like the plugin ones are probably testisms, the plugins aren't found so likely a missing path entry somewhere. The linker ones are are weird, I'll need to take a closer look at those, likely culprit is my recent patch, I had been testing in the 32 bit build and didn't notice these. There are a few worrying segfault that shouldn't be there on some random tests so I'll take a closer look at those too. And the stat changes need to be updated. The framework failures I don't see on harbormaster, so think they are again a threading artifact. Need to figure out a more effective way to debug these to find a permanent fix. The ones that harbormaster does see are encoding related. touch is failing on non-ascii names. I will take a look this weekend. Kind regards, Tamar On Thu, Sep 20, 2018, 11:33 Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> wrote: Hi Tamar The list of testsuite failure on Windows has grown quite long – see below. Most seem to concern plugins or linking. Do you know what is going on here? If they can’t be fixed, can we mark them as expect_broken on Windows, so that it’s easier (when developing) to know when I’ve introduced a regression. Currently I have to do a manual diff against a rather long list. Thanks! Simon SUMMARY for test run started at Thu Sep 20 00:13:20 2018 GMTST 1:03:11 spent to go through 6530 total tests, which gave rise to 18728 test cases, of which 12206 were skipped 33 had missing libraries 6278 expected passes 173 expected failures 9 caused framework failures 1 caused framework warnings 0 unexpected passes 31 unexpected failures 7 unexpected stat failures Unexpected failures: ghci/linking/dyn/T10955dyn.run T10955dyn [bad exit code] (normal) ghci/linking/dyn/T10955.run T10955 [bad stderr] (ghci) ghci/linking/dyn/T11072gcc.run T11072gcc [bad exit code] (normal) numeric/should_run/FloatFnInverses.run FloatFnInverses [bad stdout] (normal) plugins/T11244.run T11244 [bad stderr] (normal) plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) rts/T7040_ghci.run T7040_ghci [bad stdout] (ghci) rts/linker_unload.run linker_unload [bad exit code] (normal) rts/linker_error1.run linker_error1 [bad exit code] (normal) rts/linker_error2.run linker_error2 [bad exit code] (normal) rts/T12771/T12771.run T12771 [bad exit code] (normal) rts/T13082/T13082_good.run T13082_good [bad exit code] (normal) rts/T14611/T14611.run T14611 [bad exit code] (normal) simplCore/should_compile/T7702.run T7702 [exit code non-0] (normal) rts/T10672/T10672_x64.run T10672_x64 [bad exit code] (normal) libraries/Win32/tests/T4452.run T4452 [bad exit code] (normal) plugins/plugins01.run plugins01 [bad exit code] (normal) plugins/plugins07.run plugins07 [bad exit code] (normal) plugins/plugins09.run plugins09 [bad exit code] (normal) plugins/plugins11.run plugins11 [bad exit code] (normal) plugins/plugins12.run plugins12 [bad exit code] (normal) plugins/plugins13.run plugins13 [bad exit code] (normal) plugins/plugins14.run plugins14 [bad exit code] (normal) plugins/plugins15.run plugins15 [bad exit code] (normal) plugins/T10420.run T10420 [bad exit code] (normal) plugins/T10294.run T10294 [bad exit code] (normal) plugins/T10294a.run T10294a [bad exit code] (normal) plugins/T12567a.run T12567a [bad exit code] (normal) plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) Unexpected stat failures: perf/compiler/T9872d.run T9872d [stat not good enough] (normal) perf/compiler/T12425.run T12425 [stat not good enough] (optasm) perf/compiler/T12234.run T12234 [stat not good enough] (optasm) perf/compiler/T12150.run T12150 [stat not good enough] (optasm) perf/should_run/T15226.run T15226 [stat too good] (normal) perf/should_run/T15226a.run T15226a [stat too good] (normal) perf/compiler/MultiLayerModules.run MultiLayerModules [stat not good enough] (normal) Framework failures: ghci/linking/dyn/T10955.run T10955 [ghci] (pre_cmd failed: 2) plugins/T11244.run T11244 [normal] (pre_cmd failed: 2) plugins/plugin-recomp-change.run plugin-recomp-change [normal] (pre_cmd failed: 2) plugins/plugins07.run plugins07 [normal] (pre_cmd failed: 2) plugins/T10420.run T10420 [normal] (pre_cmd failed: 2) plugins/T10294a.run T10294a [normal] (pre_cmd failed: 2) plugins/plugin-recomp-pure.run plugin-recomp-pure [normal] (pre_cmd failed: 2) plugins/plugin-recomp-impure.run plugin-recomp-impure [normal] (pre_cmd failed: 2) plugins/plugin-recomp-flags.run plugin-recomp-flags [normal] (pre_cmd failed: 2) Framework warnings: . T13701 [numfield-no-expected] (No expected value found for bytes allocated in num_field check)

2 1