- ghc-devs - Haskell.org

scopedSort and kind variable left-biasing
by Ryan Scott 14 Feb '19

14 Feb '19

Consider this function: f :: Proxy (a :: j) -> Proxy (b :: k) If you just collect the free type variables of `f`'s type in left-to-right order, you'd be left with [a,j,b,k]. But the type of `f` is not `forall (a :: j) j (b :: k) k. Proxy a -> Proxy b`, as that would be ill scoped. `j` must come before `a`, since `j` appears in `a`'s kind, and similarly, `k` must come before `b`. Fortunately, GHC is quite smart about sorting free variables such that they respect dependency order. If you ask GHCi what the type of `f` is (with -fprint-explicit-foralls enabled), it will tell you this: λ> :type +v f f :: forall j k (a :: j) (b :: k). Proxy a -> Proxy b As expected, `j` appears before `a`, and `k` appears before `b`. In a different context, I've been trying to implement a type variable sorting algorithm similar to the one that GHC is using. My previous understanding was that the entirely of this sorting algorithm was implemented in `Type.scopedSort`. To test my understanding, I decided to write a program using the GHC API which directly uses `scopedSort` on the example above: main :: IO () main = do let tv :: String -> Int -> Type -> TyVar tv n uniq ty = mkTyVar (mkSystemName (mkUniqueGrimily uniq) (mkTyVarOcc n)) ty j = tv "j" 0 liftedTypeKind a = tv "a" 1 (TyVarTy j) k = tv "k" 2 liftedTypeKind b = tv "b" 3 (TyVarTy k) sorted = scopedSort [a, j, b, k] putStrLn $ showSDocUnsafe $ ppr sorted To my surprise, however, running this program does /not/ give the answer [j,k,a,b], like what :type reported: λ> main [j_0, a_1, k_2, b_3] Instead, it gives the answer [j,a,k,b]! Strictly speaking, this answer meets the specification of ScopedSort, since it respects dependency order and preserves the left-to-right ordering of variables that don't depend on each other (i.e., `j` appears to the left of `k`, and `a` appears to the left of `b`). But it's noticeably different that what :type reports. The order that :type reports, [j,k,a,b], appears to bias kind variables to the left such that all kind variables (`j` and `k`) appear before any type variables (`a` and `b`). >From what I can tell, scopedSort isn't the full story here. That is, something else appears to be left-biasing the kind variables. My question is: which part of GHC is doing this left-biasing? Ryan S.

3 8

Re: Performance of pattern synonyms
by Carter Schonwald 14 Feb '19

14 Feb '19

Hey Matthew, One dimension of analsysis that would be instructive would be to characterize the differences in core / stg for these different versions Also : am I correct in believing that these all are the exact same algorithm in terms of representation or am I overlooking some differences between the 3 different codes ? On Mon, Feb 11, 2019 at 4:07 PM Matthew Roberts <matthew.roberts(a)mq.edu.au> wrote: > My apologies, > > The link to the source was broken by some repo work - I have fixed it and > it should be stable now. This page was intended just to be a way of > showing the results to my collaborators, not a full explanation that anyone > can follow, but I thought the graphs at least show off what I am seeing. > > Regardless, it is all there in the code and hopefully not too obtuse. I > can improve the discussion on the page if enough people are interested :) > > Matt > > On 12 Feb 2019, at 5:14 AM, Carter Schonwald <carter.schonwald(a)gmail.com> > wrote: > > I'm looking at these links, but i'm actually having a hard time finding > the actual different definitions of this microbenchmark... > > On Mon, Feb 11, 2019 at 9:22 AM Richard Eisenberg <rae(a)cs.brynmawr.edu> > wrote: > >> >> >> On Feb 11, 2019, at 8:55 AM, Simon Peyton Jones <simonpj(a)microsoft.com> >> wrote: >> >> What exactly is “the alternative compile-time implementation”? >> >> >> In my response, I interpreted this to be macro-expansion, the alternative >> we discuss in the paper. The paper includes a nice discussion of how the >> semantics differs between what we currently have and macro-expansion. >> >> Richard >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> <https://protect-au.mimecast.com/s/DjjUCK1DOrC8NwxohMgjBd?domain=mail.haskel…> >> > >

1 0

Marge has been stabilised
by Matthew Pickering 13 Feb '19

13 Feb '19

Hi all, I think I have finally managed to stablise the merge bot (Marge). If you have a patch ready to merge then 1. Make sure that CI shows as passing 2. Make sure it has been approved by at least one person 3. Make sure it is not marked as WIP Once these three conditions are met, assign the PR to "Marge Bot" and then she will pick it up and merge it, hopefully within 24 hours depending on her current state. Any more questions, feel free to ask. I included some more specific detail below. Cheers, Matt ------ Here is her current mode of operation. Every 30 minutes whilst idle she will try to find new MRs to batch together. If she finds at least two MRs to batch together then she creates a batch as a new merge request. A batch is the series of MRs rebased on top of each other from oldest to newest. She then waits for this merge request to pass CI, checking every 10 minutes to see if it has done so. Once the MR has passed CI, she rebases and merges each MR in the batch one by one to the target branch. In between each MR she merges she waits 5 minutes. Due to the fact each MR is merged one by one: 1. The original MR will be closed automatically. 2. A CI job will trigger for each MR as it lands on master.

4 10

Re: TTG: Handling Source Locations
by Ryan Scott 13 Feb '19

13 Feb '19

Yes, I agree. This will require sprinkling the codebase with EmptyCase due to [1], but that's still a sight better than calling `panic`. After GHC 8.10 is released (and the minimum version of GHC that HEAD supports is 8.8), we can even remove these empty cases by making the empty data type fields strict (see [2]). Ryan S. ----- [1] https://ghc.haskell.org/trac/ghc/ticket/15247#comment:4 [2] https://ghc.haskell.org/trac/ghc/ticket/15305

2 2

Re: TTG: Handling Source Locations
by Ryan Scott 13 Feb '19

13 Feb '19

> Yes, I have reported it while back. I don't know of the progress in fixing this. Reported what? #15884? [1] You do realize that there is a very simple workaround for that issue, right? Instead of writing this, which is subject to the pattern-guard completeness issues observed in #15753: f :: Maybe a -> Bool f (id->Nothing) = False f (id->(Just _)) = True You can instead write this: f :: Maybe a -> Bool f (id -> x) = case x of Nothing -> False Just _ -> True This will get proper coverage checking, which means that this technique could be used to remove all of the panicking catch-all cases brought about by dL view patterns. Ryan S. ----- [1] https://ghc.haskell.org/trac/ghc/ticket/15884 [2] https://ghc.haskell.org/trac/ghc/ticket/15753

1 0

TTG: Handling Source Locations
by Richard Eisenberg 12 Feb '19

12 Feb '19

Hi devs, I just came across [TTG: Handling Source Locations], as I was poking around in RdrHsSyn and found wondrous things like (dL->L wiz waz) all over the place. General outline: https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow/HandlingSou… <https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow/HandlingSou…> Phab diff: https://phabricator.haskell.org/D5036 <https://phabricator.haskell.org/D5036> Trac ticket: https://ghc.haskell.org/trac/ghc/ticket/15495 <https://ghc.haskell.org/trac/ghc/ticket/15495> Commit: https://gitlab.haskell.org/ghc/ghc/commit/509d5be69c7507ba5d0a5f39ffd1613a5… <https://gitlab.haskell.org/ghc/ghc/commit/509d5be69c7507ba5d0a5f39ffd1613a5…> I see why this change is wanted and how the new version works. It seems to me, though, that this move makes us *less typed*. That is, it would be very easy (and disastrous) to forget to match on a location node. For example, I can now do this: > foo :: LPat p -> ... > foo (VarPat ...) = ... Note that I have declared that foo takes a located pat, but then I forgot to extract the location with dL. This would type-check, but it would fail. Previously, the type checker would ensure that I didn't forget to match on the L constructor. This error would get caught after some poking about, because foo just wouldn't work. However, worse, we might forget to *add* a location when downstream functions expect one. This would be harder to detect, for two reasons: 1. The problem is caught at deconstruction, and figuring out where an object was constructed can be quite hard. 2. The problem might silently cause trouble, because dL won't actually fail on a node missing a location -- it just gives noSrcSpan. So the problem would manifest as a subtle degradation in the quality of an error message, perhaps not caught until several patches (or years!) later. So I'm uncomfortable with this direction of travel. Has this aspect of this design been brought up before? I have to say I don't have a great solution to suggest. Perhaps the best I can think of is to make Located a type family. It would branch on the type index to HsSyn types, introducing a Located node for GhcPass but not for other types. This Isn't really all that extensible (I think) and it gives special status to GHC's usage of the AST. But it seems to solve the immediate problems without the downside above. Sorry for reopening something that has already been debated, but (unless I'm missing something) the current state of affairs seems like a potential wellspring of subtle bugs. Thanks, Richard

6 11

RE: Performance of pattern synonyms
by Simon Peyton Jones 11 Feb '19

11 Feb '19

Are you as convinced as you were of the semantics of pattern synonyms? I can see they are the right semantics given a run-time implementation but the alternative compile-time implementation has a lot to recommend it. What exactly is "the alternative compile-time implementation"? Remember, although pattern synonyms are set up to call a matching function, that matching function is often inlined, which reduces the overhead to zero. I say "often" inlined. An INLINE pragma for pattern synonyms would be a good feature. Widening to ghc-devs. Simon From: Matthew Roberts <matthew.roberts(a)mq.edu.au> Sent: 11 February 2019 00:10 To: matthew.pickering(a)cs.ox.ac.uk; gergo(a)erdi.hu; Simon Peyton Jones <simonpj(a)microsoft.com>; rae(a)cs.brynmawr.edu Subject: Performance of pattern synonyms Hi all, I am working with someone on a compile-time pattern matching extension and the most important prior work is pattern synonyms in Haskell. I hope you might indulge a couple of questions I have not been able to answer myself from the literature: * [In my testing](http://pattern-benchmarks.herokuapp.com/posts/2019-02-09-peano.html<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpattern-be…>), pattern synonyms have remarkable performance, has anyone ever benchmarked them before? * Do you have - on hand - the hackage data you used in investigating uses of pattern synonyms? I am looking for real-world usage of pattern matching extensions. * Are you as convinced as you were of the semantics of pattern synonyms? I can see they are the right semantics given a run-time implementation but the alternative compile-time implementation has a lot to recommend it. Thanks for your time, Matt Roberts Department of Computing, Macquarie University

3 2

RE: [commit: ghc] wip/T16188: testsuite: Report stdout and stderr in JUnit output (224fec6)
by Simon Peyton Jones 11 Feb '19

11 Feb '19

Are these commits really for #16188? For which we have a MR in progress https://gitlab.haskell.org/ghc/ghc/merge_requests/207 I was just surprised to see wip/T16188 going by. Simon | -----Original Message----- | From: ghc-commits <ghc-commits-bounces(a)haskell.org> On Behalf Of | git(a)git.haskell.org | Sent: 10 February 2019 21:32 | To: ghc-commits(a)haskell.org | Subject: [commit: ghc] wip/T16188: testsuite: Report stdout and stderr in | JUnit output (224fec6) | | Repository : ssh://git@git.haskell.org/ghc | | On branch : wip/T16188 | Link : | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fghc.hask | ell.org%2Ftrac%2Fghc%2Fchangeset%2F224fec6983e16ecfc44a80d47e591a2425468e | af%2Fghc&data=02%7C01%7Csimonpj%40microsoft.com%7Cec8ee4f808cd416cf0d | d08d68f9f2e48%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C63685431128635 | 6725&sdata=ZS4Bp%2FMsebLALA27waYiWgtyx%2FyrKSkiCLcFXvSfXBE%3D&res | erved=0 | | >--------------------------------------------------------------- | | commit 224fec6983e16ecfc44a80d47e591a2425468eaf | Author: Ben Gamari <ben(a)smart-cactus.org> | Date: Thu Jan 24 14:20:11 2019 -0500 | | testsuite: Report stdout and stderr in JUnit output | | This patch makes the JUnit output more useful as now we also report | the | stdout/stderr in the message which can be used to quickly identify | why a | test is failing without downloading the log. | | This also introduces TestResult, | previously we were simply passing around tuples, making things the | implementation rather difficult to follow and harder to extend. | | | >--------------------------------------------------------------- | | 224fec6983e16ecfc44a80d47e591a2425468eaf | testsuite/driver/junit.py | 23 ++++++++-------- | testsuite/driver/testglobals.py | 15 +++++++++++ | testsuite/driver/testlib.py | 58 ++++++++++++++++++++++++++--------- | ------ | testsuite/driver/testutil.py | 7 +++-- | 4 files changed, 69 insertions(+), 34 deletions(-) | | Diff suppressed because of size. To see it, use: | | git diff-tree --root --patch-with-stat --no-color --find-copies- | harder --ignore-space-at-eol --cc | 224fec6983e16ecfc44a80d47e591a2425468eaf | _______________________________________________ | ghc-commits mailing list | ghc-commits(a)haskell.org | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | commits&data=02%7C01%7Csimonpj%40microsoft.com%7Cec8ee4f808cd416cf0dd | 08d68f9f2e48%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636854311286366 | 720&sdata=RH6Jnvs7cNsnGyLpwONSZ0Cldighb85HZtuWfn4Gc9Q%3D&reserved | =0

2 1

Commit comments - call for opinions
by Tobias Dammers 11 Feb '19

11 Feb '19

Dear all, So far, we had a feature in Trac where git commit messages mentioning a ticket would automatically be copied into a comment on that commit. See, for example, this comment: https://ghc.haskell.org/trac/ghc/ticket/13615#comment:47 GitLab does things slightly differently. It has "live" git repository information available, and rather than the relatively static comment list we see in Trac, it synthesizes a "timeline" style list of events relevant to an issue, which includes comments ("notes"), but also issue status updates, git commits, and other things. So when a commit mentions a GitLab issue, then GitLab will insert a link to that commit into the issue's event timeline. This means that in principle, copying commit messages into notes would be redundant, and we initially decided to ignore commit comments during the Trac/GitLab migration. However, it also means that the issue timeline will no longer display the actual commit message anymore, just a link to the commit ("@username mentioned this issue in commit 1234beef" or similar). So what used to be a narrative that you could read top-down to retrace the issue's history is now a bit more scattered. Now, the ideal solution would be for GitLab to instead display the full commit message, but I don't see this happening anytime soon, because it would require a patch to GitLab itself. So we're left with two options: a) Import commit comments as notes, duplicating the commit message into the note, and having both the full commit note and a hyperlinked commit reference in the issue timeline. b) Don't import commit comments as notes, just rely on GitLab to insert the hyperlinked commit reference. If any of you have any preference either way, please do tell. Thanks! -- Tobias Dammers - tdammers(a)gmail.com

5 5

Memory ordering and thunk update
by Ben Gamari 10 Feb '19

10 Feb '19

Hi Simon and Peter, I just wanted to draw your attention to !337 [1], where I try to fix #15994. While it's possible I have missed something, it looks to me like there has been a bug lurking here for quite some time. In particular on a weak memory model machine we do not place a write barrier between the construction of the result of a thunk evaluation and making the results visible to other cores via the indirection's indirectee field. This essentially means that a thread looking at the indirectee may essentially see uninitialized memory. Specifically, updateWithIndirection is currently: # Evaluation result p2 initialized by caller ... ((StgInd *)p1)->indirectee = p2; write_barrier(); SET_INFO(p1, &stg_BLACKHOLE_info); ... whereas I think this should rather be # Evaluation result p2 initialized by caller ... write_barrier(); ((StgInd *)p1)->indirectee = p2; SET_INFO(p1, &stg_BLACKHOLE_info); ... I describe the reasoning for this in more detail in the Note added in !337. All-in-all, I'm rather surprised to find this bug. While we won't see this issue manifest on x86_64 due to this architecture's memory model, we have long supported ARM, PowerPC, and SPARC, all of which have weakly-ordered execution modes. I can't find any relevant open tickets from these platforms. The fact that this wasn't noticed on these platforms makes me wonder whether I am just missing something. Let me know what you think. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/merge_requests/337

1 0