- ghc-devs - Haskell.org

Lazy ST vs concurrency
by David Feuer 01 Feb '17

01 Feb '17

I forgot to CC ghc-devs the first time, so here's another copy. I was working on #11760 this weekend, which has to do with concurrency breaking lazy ST. I came up with what I thought was a pretty decent solution ( https://phabricator.haskell.org/D3038 ). Simon Peyton Jones, however, is quite unhappy about the idea of sticking this weird unsafePerformIO-like code (noDup, which I originally implemented as (unsafePerformIO . evaluate), but which he finds ugly regardless of the details) into fmap and (>>=). He's also concerned that the noDuplicate# applications will kill performance in the multi-threaded case, and suggests he would rather leave lazy ST broken, or even remove it altogether, than use a fix that will make it slow sometimes, particularly since there haven't been a lot of reports of problems in the wild. My view is that leaving it broken, even if it only causes trouble occasionally, is simply not an option. If users can't rely on it to always give correct answers, then it's effectively useless. And for the sake of backwards compatibility, I think it's a lot better to keep it around, even if it runs slowly multithreaded, than to remove it altogether. Note to Simon PJ: Yes, it's ugly to stick that noDup in there. But lazy ST has always been a bit of deep magic. You can't *really* carry a moment of time around in your pocket and make its history happen only if necessary. We can make it work in GHC because its execution model is entirely based around graph reduction, so evaluation is capable of driving execution. Whereas lazy IO is extremely tricky because it causes effects observable in the real world, lazy ST is only *moderately* tricky, causing effects that we have to make sure don't lead to weird interactions between threads. I don't think it's terribly surprising that it needs to do a few more weird things to work properly. David

5 12

Join points revised
by Luke Maurer 01 Feb '17

01 Feb '17

Revised version of the join points patch is up: https://phabricator.haskell.org/D2853 Hoping to commit ASAP. Thanks all! - Luke Maurer University of Oregon maurerl(a)cs.uoregon.edu

3 2

Freeze update
by Ben Gamari 31 Jan '17

31 Jan '17

Hello everyone, As you likely know, the feature freeze date for the 8.2.1 was yesterday. Unfortunately, it looks like that date may have been a bit optimistic as there are there are still several major patches outstanding that we would like to have for 8.2.1. These are namely, * Join points rework (D2853) * Overloaded Record Fields (D2708) * Separation of the Constraint and Type types within GHC (D3023) * Generalization of the kind of (->) (D2038) * Type-indexed Typeable (D2010) * Use top-level instance to solve superclasses where possible (D2714) * A variety of DWARF patches While many of these patches are very close to completion, in sum they represent a significant amount of work. While I would love to be able to offer a concrete timeline for how we will proceed, at the moment I can only say that things are still rather up in the air. In particular, there are some thorny, late-rising design questions in D2038 and D3023 which are currently under discussion. If there are no obviously easy paths forward on these questions then we may be forced to retarget one or more of these patches to 8.4. I believe within the next few days we'll have a better idea of where we stand on these. Finally, note that unless you have spoken to me previously about merging your work for 8.2 we will not be considering any additional features at this point. Cheers, - Ben

1 0

Re: Lazy ST vs concurrency
by David Feuer 31 Jan '17

31 Jan '17

I think Ben's eagerlyBlackhole is what I called noDup. And it is indeed a bit tricky to use correctly. It needs the same sort of care around inlining that unsafePerformIO does. A bit of summary: 1. It's okay to duplicate a runST thunk or to enter it twice, because each copy will have its own set of references and arrays. This is important, because we have absolutely no control over what the user will do with such a thunk. 2. With the exception of a runST thunk, we must never duplicate or double-enter a thunk if it performs or suspends ST work. Based on my tests and the intuition I've developed about what's going on here, (2) breaks down into two pieces: 2a. Any time we perform or suspend ST work, we must use NOINLINE to avoid duplication. 2b. Any time we suspend ST work, we must set up the thunk involved with noDuplicate# or similar. For example, the code I wrote yesterday for the Applicative instance looks like this: fm <*> xm = ST $ \ s -> let {-# NOINLINE res1 #-} !res1 = unST fm s !(f, s') = res1 {-# NOINLINE res2 #-} res2 = noDup (unST xm s') (x, s'') = res2 in (f x, s'') I NOINLINE res1. If it were to inline (could that happen?), we'd get let res2 = noDup (unST xm (snd (unST fm s))) (x, s'') = res2 in (fst (unST fm s) x, s') and that would run the fm computation twice. But I don't noDup res1, because we force it immediately on creation; no one else ever handles it. I NOINLINE res2 for a similar reason, but I also use noDup on it. The res2 thunk escapes into the wild via x and s'' in the result; we need to make sure that it is not entered twice. I believe can use a few rewrite rules to reduce costs substantially in some situations. I will add those to the differential. -------- Original message -------- From: Simon Marlow <marlowsd(a)gmail.com> Date: 1/31/17 3:59 AM (GMT-05:00) To: Simon Peyton Jones <simonpj(a)microsoft.com> Cc: David Feuer <david(a)well-typed.com>, ghc-devs(a)haskell.org Subject: Re: Lazy ST vs concurrency On 30 January 2017 at 22:56, Simon Peyton Jones <simonpj(a)microsoft.com[1]> wrote: We don’t want to do this on a per-module basis do we, as -fatomic-eager-blackholing would suggest. Rather, on per-thunk basis, no? Which thunks, precisely? I think perhaps *precisely thunks one of whose free variables has type (Sttate# s) for some s.* These are thunks that consume a state token, and must do so no more than once.

1 0

Constant functions and selectors make for interesting arguments
by David Feuer 31 Jan '17

31 Jan '17

GHC's inliner has a notion of "interesting argument" it uses to encourage inlining of functions called with (I think) dictionary arguments. I think another class of argument is very interesting, by being very boring. Any argument that looks like either \ _ ... (Con _ ... x ... _ ) ... _ -> coerce x or \ _ ... _ -> k Has a pretty good chance of doing a lot of good when inlined, perhaps plugging a space leak. Would it make sense to try to identify such functions and consider them interesting for inlining?

4 5

Overloaded record fields for 8.2?
by Adam Gundry 30 Jan '17

30 Jan '17

Hi Ben, devs, I'd like to propose that we merge my latest ORF patch (https://phabricator.haskell.org/D2708) for 8.2 (modulo any code review improvements, of course). The corresponding proposal discussion (https://github.com/ghc-proposals/ghc-proposals/pull/6) seems to have reached a broad consensus. But it's not clear to me if some kind of final approval from the GHC committee is needed, or how to obtain that. This patch makes breaking changes to the experimental OverloadedLabels extension, and I think it would be good to make those quickly rather than having another release with the "old" version. Moreover, the work has been in progress for a long time, so it would be great to see it finally make it into a GHC release. Any objections? Adam -- Adam Gundry, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

2 1

Next steps of the trac-to-maniphest migration?
by Matthew Pickering 29 Jan '17

29 Jan '17

Hello devs, Thanks to everyone so far who has looked at and commented on the prototype. It seems that the response is generally positive so I would like to drive the process forwards. In order for that to happen, someone needs to decide whether we as a community think it is a good idea. It seems to make sense if those who use the tracker most make this decision so I propose that Simon and Ben should ultimately be the ones to do this. Therefore, I propose this timeline 1. Before 11th Feb (3 weeks from today) we decide whether we want to migrate the issue tracker. 2. A working group is established who will work through the details of the migration with the minimum of a final prototype built from a clone of the actual installation. 3. Migration would happen before the end of March. I think Ben summarised the discussions quite well on the wiki page - https://ghc.haskell.org/trac/ghc/wiki/Phabricator/Maniphest And the prototype continues to exist here. http://ec2-52-214-147-146.eu-west-1.compute.amazonaws.com/ As always, any comments welcome. Matt

3 8

quick building the user's guide
by Takenobu Tani 29 Jan '17

29 Jan '17

Dear devs, Can We build the user's guide without building the ghc binary? If so, new contributors can easily check the generated html. When I execute the following command, a binary build is always executed. # vi mk/build.mk (BuildFlavour = quick; BUILD_SPHINX_HTML = YES) # ./boot # ./configure # cd docs/users_guide/ # make html Is there a way to skip binary builds? Regards, Takenobu

2 2

simple pictures about GHC development flow
by Takenobu Tani 28 Jan '17

28 Jan '17

Hi devs, For myself and new contributors, I drew overview pictures about GHC development flow. GHC development flow http://takenobu-hs.github.io/downloads/ghc_development_flow.pdf https://github.com/takenobu-hs/ghc-development-flow If I misunderstood something, please teach me. I'll correct or remove them. Regards, Takenobu

4 15

Cannot build GHC using the Newcomers guide
by Marius Ghita 27 Jan '17

27 Jan '17

Following is a list of steps that I ran and their output linked: - clone repo https://gist.github.com/mhitza/f5d4516b6c8386fe8e064f95b5ad62 0b - build.mk configuration https://gist.github.com/mhitza/ 2d979c64a646bdd3e097f65fd650c675 - boot https://gist.github.com/mhitza/e23df8b9ed2aac5b1b8881c70165bf3f - configure https://gist.github.com/mhitza/88c09179be3bb82024192bf6181aef13 - make FAILS https://gist.github.com/mhitza/95738bf49c8c87ce46c9319b4c266a 2c I'm using a 'stack-ghc' executable, that's only a shell wrapper to run ghc from stack (since I don't have a globally installed ghc) (source https://gist.github.com/mhitza/38fe96fb440daab28e57a50de47863d5 ), and I also have 'ghc-pkg' wrapped in the same way with a stack-ghc-pkg script (source https://gist.github.com/mhitza/ 6c2b1978ef802707161041abe1d2699e ) -- Google+: https://plus.google.com/111881868112036203454

5 7