- ghc-devs - Haskell.org

RE: Performance of pattern checker on OptCoercions
by Simon Peyton Jones 11 Dec '15

11 Dec '15

`f` will generate an empty uncovered set while g will generate: uncovered = { x |> { x ~ [], x ~ (_:_) } , x |> { x ~ (_:_), x ~ [] } } which is also semantically empty but this cannot be detected until we call the term oracle on it to see the inconsistency. So, why not call the term oracle more aggressively (at every node of the tree), to prune empty sets? It should not be hard to spot common cases; in what you show above it’s very easy. Simon From: George Karachalias [mailto:george.karachalias@gmail.com] Sent: 11 December 2015 11:27 To: Ben Gamari <ben(a)well-typed.com> Cc: Simon Peyton Jones <simonpj(a)microsoft.com>; Richard Eisenberg <eir(a)cis.upenn.edu>; GHC developers <ghc-devs(a)haskell.org> Subject: Re: Performance of pattern checker on OptCoercions Hello Ben, On Fri, Dec 11, 2015 at 11:58 AM, Ben Gamari <ben(a)well-typed.com<mailto:ben@well-typed.com>> wrote: Hi George, Richard has encountered a bit of a performance cliff when merging his no-kinds work. In particular OptCoercions now results in multiple gigabytes of memory consumption during compilation due to the pattern checker. The problem seems to be the opt_trans_rule binding, which has numerous equations, each of which has patterns of various complexities and guards. Might this be another case where disabling the pattern checker is unavoidable? I am afraid so. I have just responded to the ticket about it. The essence is the difference between `f` and `g` below: f x = case x of [] -> ... (_:_) -> ... g y | [] <- y = ... | (_:_) <- y = ... `f` will generate an empty uncovered set while g will generate: uncovered = { x |> { x ~ [], x ~ (_:_) } , x |> { x ~ (_:_), x ~ [] } } which is also semantically empty but this cannot be detected until we call the term oracle on it to see the inconsistency. Since pattern guards can pattern match against any variable whilst case expressions match a single expression (`x` above) I can not make the check treat them the same. From what I see, until now the pattern guards in opt_trans_co involved mostly pattern matching with Maybe which has only two constructors. I can easily assume that this is the reason we did not have such a problem until now. So many guards are already a challenge for the pattern match checker but maybe changing this (by this I mean to not use pattern guards on types with many constructors because this is the most expensive thing for the whole check) is enough to make GHC bootstrap. I hope this helps, I am really confident that this is the cause of the problem. Nevertheless, I will look into it more to see if I can find another source. George -- things you own end up owning you

1 0

Re: Performance of pattern checker on OptCoercions
by Richard Eisenberg 11 Dec '15

11 Dec '15

If I understand correctly, you're saying that the problem is due to my use of guards, not anything with fancy types. If that's correct, then this sort of thing might be a trap anyone could stumble into. (The code in OptCoercion is fairly routine for Haskell.) It seems like disabling checking guards by default may be best, but I'd love a flag (not with -Wall) to enable it optionally. Richard On Dec 11, 2015, at 6:27 AM, George Karachalias <george.karachalias(a)gmail.com> wrote: > Hello Ben, > > On Fri, Dec 11, 2015 at 11:58 AM, Ben Gamari <ben(a)well-typed.com> wrote: > > Hi George, > > Richard has encountered a bit of a performance cliff when merging his > no-kinds work. In particular OptCoercions now results in multiple > gigabytes of memory consumption during compilation due to the pattern > checker. The problem seems to be the opt_trans_rule binding, which has > numerous equations, each of which has patterns of various complexities > and guards. Might this be another case where disabling the pattern > checker is unavoidable? > > I am afraid so. I have just responded to the ticket about it. The essence is the > difference between `f` and `g` below: > > f x = case x of > [] -> ... > (_:_) -> ... > > g y | [] <- y = ... > | (_:_) <- y = ... > > `f` will generate an empty uncovered set while g will generate: > > uncovered = > { x |> { x ~ [], x ~ (_:_) } > , x |> { x ~ (_:_), x ~ [] } } > > which is also semantically empty but this cannot be detected until we call the > term oracle on it to see the inconsistency. Since pattern guards can pattern match > against any variable whilst case expressions match a single expression (`x` above) > I can not make the check treat them the same. > > From what I see, until now the pattern guards in opt_trans_co involved mostly > pattern matching with Maybe which has only two constructors. I can easily assume > that this is the reason we did not have such a problem until now. So many guards > are already a challenge for the pattern match checker but maybe changing this (by > this I mean to not use pattern guards on types with many constructors because this > is the most expensive thing for the whole check) is enough to make GHC bootstrap. > > I hope this helps, I am really confident that this is the cause of the problem. > Nevertheless, I will look into it more to see if I can find another source. > > George > > -- > things you own end up owning you

2 1

type operators and colon in GHC
by John Leo 11 Dec '15

11 Dec '15

I sent this to haskell-cafe a little while ago but didn't get a response, so I thought I'd try here. I'd guess this is a case of the GHC user guide needing an update, but I'd like an expert opinion. --- According to sections 7.4.3 and 7.4.4 of the latest GHC documentation https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/data-type-e… you can define (7.4.3) an infix type constructor as long as it begins with a colon, for example data a :*: b = Foo a b and furthermore (7.4.4) you can define an infix operator without having to use a colon if you enable the TypeOperators extension: data a * b = Foo a b However if I try the former without using TypeOperators I get this compiler error in 7.10.2: Illegal declaration of a type or class operator ‘:*:’ Use TypeOperators to declare operators in type and declarations Using TypeOperators fixes this, but then * without colon also works so I don't see the point of using colon anymore. My guess is this was some some kind of historical distinction which is no longer valid and the documentation needs to be updated. Is this true, or am I missing something? John

3 3

[ANNOUNCE] Glasgow Haskell Compiler version 7.10.3
by Ben Gamari 11 Dec '15

11 Dec '15

Hello everyone, We are pleased to announce the release of GHC 7.10.3: https://downloads.haskell.org/~ghc/7.10.3/ There can be found source tarballs and binary distributions for 64-bit and 32-bit modern Linux (GMP 5.0 or later), CentOS (GMP 4.0), Windows, and 64-bit Mac OS X platforms. These binaries and tarballs have an accompanying SHA256SUMS file signed by my GPG key id (0x97DB64AD). Significant fixes in release include changes to the simplifier's treatment of rules, the handling of Mac OS X frameworks, and support for response files to work around the restrictive command line length limit on Windows. As always, a full accounting of the changes present in this release can be found in the release notes [1]. The previous release, 7.10.2, was well-behaved save a couple notable bugs; while we have merged a good number of bug fixes in 7.10.3 they were were largely low risk and so we expect that this release should be similiarly stable. A notable exception is the upgrade of the Windows compiler toolchain to GCC 5.2. Typically we would refrain from making such large changes in a point release but Windows users have been long suffering at the hand of the old toolchain (e.g. lack of response file support, #8596, and lack of SEH support). We expect that this change should fix far more than breaks. Unfortunately, due to some oversights in the release process there are two source tarballs for this release. The first, ghc-7.10.3-src.tar.{bz2,xz}, does not include the release notes in the users guide. This is fixed in the second patchlevel release, ghc-7.10.3a-src.tar.{bz2,xz}. It is recommended that users wanting a source release use the ghc-7.10.3a-src tarballs. GHC 7.10.3 will very likely be the last release in the GHC 7 series. In the coming weeks we will be beginning the release process for GHC 8.0, which will include a number of exciting features including the merger of kinds with types, injective type families, imporved DWARF debugging information, applicative-do syntax, a Strict language extension synonyms mechanism, and more. See the GHC 8.0.1 status page for details [2]. Happy compiling! Cheers, - Ben [1] http://downloads.haskell.org/~ghc/7.10.3/docs/html/users_guide//release-7-1… [2] https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-8.0.1

3 3

Performance of pattern checker on OptCoercions
by Ben Gamari 11 Dec '15

11 Dec '15

Hi George, Richard has encountered a bit of a performance cliff when merging his no-kinds work. In particular OptCoercions now results in multiple gigabytes of memory consumption during compilation due to the pattern checker. The problem seems to be the opt_trans_rule binding, which has numerous equations, each of which has patterns of various complexities and guards. Might this be another case where disabling the pattern checker is unavoidable? I've attached a testcase to the ticket (#11195) which depends only on the `ghc` library built from Richard's nokinds-dev branch. If it would help I could make it a fully free-standing testcase. It's not time critical as we have just disabled the pattern checker for the time being, but it would be nice to know whether you have any insight into what is happening here. Cheers, - Ben

1 0

arcanist-external-json-linter missing commits
by Edward Z. Yang 10 Dec '15

10 Dec '15

On a git submodule update on HEAD, I get: fatal: reference is not a tree: 85ece0f8653e7b1e7de6024d372cfeaf36ab5fa9 Unable to checkout '85ece0f8653e7b1e7de6024d372cfeaf36ab5fa9' in submodule path '.arc-linters/arcanist-external-json-linter' Edward

2 2

Does the Strict extension make monadic bindings strict?
by Rob Stewart 10 Dec '15

10 Dec '15

Are the following two programs equivalent with respect to the strictness of `readFile`? --8<---------------cut here---------------start------------->8--- {-# LANGUAGE BangPatterns #-} module Main where main = do !contents <- readFile "foo.txt" print contents --8<---------------cut here---------------end--------------->8--- And: --8<---------------cut here---------------start------------->8--- {-# LANGAUGE Strict #-} module Main where main = do contents <- readFile "foo.txt" print contents --8<---------------cut here---------------end--------------->8--- The documentation on "Strict-by-default pattern bindings" gives let/where binding as an example, but there is not a monadic bind example. http://downloads.haskell.org/~ghc/master/users-guide/glasgow_exts.html#stri… Inspecting GHC Core for these two programs suggests that !contents <- readFile "foo.txt" is not equivalent to (with Strict enabled): contents <- readFile "foo.txt" Here's core using BangPatterns: (readFile (unpackCString# "foo.txt"#)) (\ (contents_asg :: String) -> case contents_asg of contents1_Xsk { __DEFAULT -> print @ String $dShow_rYy contents1_Xsk }) Here's core using Strict: (readFile (unpackCString# "foo.txt"#)) (\ (contents_asg :: String) -> print @ String $dShow_rYv contents_asg) Does this core align with the design of the Strict extension? If it does, are users going to understand that using Strict is going to make let/where bindings strict, but is not going to make <- or >>= bindings strict? -- Rob Stewart

5 5

Re: [Haskell-cafe] [ANNOUNCE] Glasgow Haskell Compiler version 7.10.3
by Ben Gamari 09 Dec '15

09 Dec '15

Ivan Lazar Miljenovic <ivan.miljenovic(a)gmail.com> writes: > On 9 December 2015 at 23:17, Ben Gamari <ben(a)well-typed.com> wrote: >> >> >> [1] http://downloads.haskell.org/~ghc/7.10.3/docs/html/users_guide//release-7-1… > > The links to Trac issues in that page seem to redirect to the same page. > Indeed, I'm trying to work out what went wrong here. I am quite looking forward to being able to drop DocBook in 8.0. Cheers, - Ben

1 0

RE: D808 progress report
by Simon Peyton Jones 09 Dec '15

09 Dec '15

Kind equalities are the Big New Thing in 8.0. Let's just get it in and deal with the fallout. After all, there is no reason for performance to be worse. For programs that 7.10 accepts, 8.0 should yield essentially the same coercions. They might need a bit of optimisation to squeeze them down but the result should be essentially identical. If not, let's investigate. I could imagine the typechecker being a bit slower, but not a lot. For T3738, compile the compiler before and after with -ticky and compare. | In light of all this, I propose the following: | - Scramble to fix all non-perf failures. I expect I can finish this by | Wed evening. | - Hope that one of you (or another dev) can take a look at T3738 and | friends. That clearly needs to get fixed. | - Adjust perf targets to get validation to work, clearly labeling the | remaining problems as the fault of type=kind. | - Commit to fixing #8095 in the next two weeks. But probably not by | early next week, I'm afraid. | In short, yes. Simon | -----Original Message----- | From: Richard Eisenberg [mailto:eir@cis.upenn.edu] | Sent: 08 December 2015 03:35 | To: Simon Peyton Jones <simonpj(a)microsoft.com>; Ben Gamari <ben@well- | typed.com>; Austin Seipp <aseipp(a)pobox.com> | Subject: D808 progress report | | Hi Simon, Ben, Austin, | | First, the bad news: | I'm a bit stalled on performance issues. When I sent my earlier email, | I was celebrating having gotten one test case from 319M of allocation | down to 182M via several seemingly general-purpose optimizations. But | this was with -fno-opt-coercion. Once I re-enabled coercion | optimization, that particular test case still fails | (pert/compiler/T5030), along with 22 others. This is bad. But many ~4 | hours of effort this evening I've made no substantive progress at all, | shaving off maybe 1% of allocation via a few tiny tweaks. Even | characterizing what's going wrong is proving difficult. I've only | analyzed a few of the failing tests, but each one is stubbornly | refusing to break, so I'm losing hope about the others. | | Then, the good news: | I think the idea posited in #8095 (not to bother building coercions | unless -dcore-lint is on) will solve all of these problems and more, | as long as users don't use -dcore-lint. With one exception that I've | noticed (see below), my performance failures aren't catastrophic: on | the performance tests, which tend to be pathological, my branch is | running 10-20% worse than HEAD. Not good, but not so bad that -dcore- | lint users can't cope. So, with #8095 addressed, I think we'll be OK. | And #8095 should be very straightforward and done in a few hours' | work. | | Finally, the ugly: | The exception to the non-catastrophic nature of the failures is this: | perf/should_run/T3738 fails with 3479.1% overage. (Yes, the percentage | is in the thousands.) Worse, this is at runtime, not in the compiler. | Yet the Core produced in my branch (as observed by -ddump-simpl) and | in HEAD appears identical. There are a few other should_run failures | that have me nervous, but my guess is that they're all from one | source. I'd love an offer of help to debug this. | | | In light of all this, I propose the following: | - Scramble to fix all non-perf failures. I expect I can finish this by | Wed evening. | - Hope that one of you (or another dev) can take a look at T3738 and | friends. That clearly needs to get fixed. | - Adjust perf targets to get validation to work, clearly labeling the | remaining problems as the fault of type=kind. | - Commit to fixing #8095 in the next two weeks. But probably not by | early next week, I'm afraid. | | What do we think? | | Thanks, | Richard

3 7

Re: runghc fails on Windows with GHC HEAD
by Ryan Scott 08 Dec '15

08 Dec '15

Actually, this might not be Windows-related at all. It turns out that runghc was failing because there wasn't a ghc.exe located in inplace/bin (just ghc-stage1.exe and ghc-stage2.exe). Adding a ghc.exe symlink to ghc-stage2.exe made runghc.exe work correctly. Is this a bug, or the intended behavior? Ryan S.

2 1