September 2014 - ghc-devs

Feedback request for #9628 AST Annotations
by Alan & Kim Zimmerman 02 Oct '14

02 Oct '14

Hi All I am looking for some input on #9628 for source annotations. The intent behind this is to make it easier for tool developers to add extra information to the AST. This is to assist in structured editing or other source to source conversions, such as refactoring. One of the use cases would be to maintain the ParsedSource as tightly coupled to the editor, where changes would be made directly to the AST and only indirectly to the text source. This is the approach enabled by structured haskell mode, or demonstrated in lamdu. Ideally it should be possible to perform a change to the ParsedSource, and then submit it to the renamer and typechecker for error and warning feedback. As such the renamer / typechecker should be able to accept the annotated AST. Eventually this process should be incremental, since a given edit takes place in a structured way in a single position at a time, so it should be possible to identify which part of previously done work is invalidated by it and needs to be redone. This would happen in the context of a ghci session operating as a server. In order to lay the initial groundwork for this, I have updated the AST in hsSyn to replace all of the `Located` elements with `GenLocated l`, where `l` is constrained to be a member of class `ApiAnnotation`. class (Ord l, Outputable l, OutputableBndr l) => ApiAnnotation l where annGetSpan :: l -> SrcSpan annSetSpan :: l -> SrcSpan -> l annNoSpan :: l This guarantees that the annotation does in fact keep a `SrcSpan` in it. I have also worked this through the renamer, typechecker, and desugarer, and the monad for core. I have kept the `Located` / `GenLocated SrcSpan` boundary for the parser, and within the `RdrName`,`Name`,`Id` types. In this context the hooks become especially important, but they need to get the annotation type parameter. It seemed extreme to force the `DynamicFlags` to have to have the type parameter, so I am proposing to move the hooks to `HscEnv`, which needs a type parameter, even as a phantom type, to make sure that the right data structures are instantiated in the monads for the various passes. This is a big change, and I am very much working in the dark, so I would love some feedback or discussion. 1. Is this change too big, should I scale it back to just update the HsSyn structures and then lock it down to Located SrcSpan for all the rest? 2. Will the ApiAnnotation constraint and resolution cause performance degradation? 2. Is it ok to move the hooks into HscEnv? 3. In order to have a single session for a project, it needs a single annotation parameter. To support this the parser needs to produce annotated output. It should be a simple case of updating the L0/L1/LL macros and production types. Does it make sense to do this? This is intended to be an enabling change, without changing the structure of any code or the AST. A later change would introduce additional annotations in HsSyn to track the locations of the various keywords and other punctuation so that exactly regenerating source is much easier, without requiring extensive examiniation of the tokens. References 1. https://ghc.haskell.org/trac/ghc/ticket/9628 2. https://ghc.haskell.org/trac/ghc/wiki/GhcAstAnnotations 3. https://phabricator.haskell.org/D246 4. https://github.com/alanz/ghc/commits/wip/ast-ann-location Regards Alan

5 18

Re: Build time regressions
by Edward Z. Yang 01 Oct '14

01 Oct '14

Hello Joachim, This was halfway known, but it sounds like we haven't solved it completely. The beginning of the sordid tale was when Cabal HEAD switched to using derived binary instances: https://ghc.haskell.org/trac/ghc/ticket/9583 SPJ fixed the infinite loop bug in the simplifier, but apparently the deriving binary generates a lot of code, meaning a lot of memory. https://ghc.haskell.org/trac/ghc/ticket/9630 hvr's fix was specifically to solve this problem. But it sounds like it didn't eliminate the regression entirely? If there's an unrelated regression, we should suss it out. It would be helpful if someone could revert just the deriving changes, and see if this reverts the compilation time. Edward Excerpts from Joachim Breitner's message of 2014-09-30 13:36:27 -0700: > Hi, > > the attached graph shows a noticable increase in build time caused by > > Update Cabal submodule & ghc-pkg to use new module re-export types > author Edward Z. Yang <ezyang(a)cs.stanford.edu> > https://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176… > > and only halfway mitigated by > > Update `binary` submodule in an attempt to address #9630 > author Herbert Valerio Riedel <hvr(a)gnu.org> > https://git.haskell.org/ghc.git/commit/3ecca02516af5de803e4ff667c8c969c5bff… > > > I am not sure if the improvement is related to the regression, but in > any case: Edward, was such an increase expected by you? If not, can you > explain it? Can it be avoided? > > Or maybe Cabal just became much larger... +38% in allocations when > running haddock on it seems to confirm this. > > Greetings, > Joachim >

4 4

The future of the haskell2010/haskell98 packages - AKA Trac #9590
by Austin Seipp 01 Oct '14

01 Oct '14

Hello developers, users, friends, I'd like you all to weigh in on something - a GHC bug report, that has happened as a result of making Applicative a superclass of Monad: https://ghc.haskell.org/trac/ghc/ticket/9590 The very condensed version is this: because haskell2010/haskell98 packages try to be fairly strictly conforming, they do not have modules like Control.Applicative. Unfortunately, due to the way these packages are structured, many things are simply re-exported from base, like `Monad`. But `Applicative` is not, and cannot be imported if you use -XHaskell2010 and the haskell2010 package. The net result here is that haskell98/haskell2010 are hopelessly broken in the current state: it's impossible to define an instance of `Monad`, because you cannot define an instance of `Applicative`, because you can't import it in the first place! This leaves us in quite a pickle. So I ask: Friends, what do you think we should do? I am particularly interested in users/developers of current Haskell2010 packages - not just code that may *be* standard Haskell - code that implies a dependency on it. There was a short discussion between me and Simon Marlow about this in the morning, and again on IRC this morning between me, Duncan, Edward K, and Herbert. Basically, I only see one of two options: - We could make GHC support both: a version of `Monad` without `Applicative`, and one with it. This creates some complication in the desugarer, where GHC takes care of `do` syntax (and thus needs to be aware of `Monad`'s definition and location). But it is, perhaps, quite doable. - We change both packages to export `Applicative` and follow the API changes in `base` accordingly. Note that #1 above is contingent on three things: 1) There is interest in this actually happening, and these separate APIs being supported. If there is not significant interest in maintaining this, it's unclear if we should go for it. 2) It's not overly monstrously complex (I don't think it necessarily will be, but it might be.) 3) You can't like `haskell2010` packages and `base` packages together in the general case, but, AFAIK, this wasn't the case before either. I'd really appreciate your thoughts. This must be sorted out for 7.10 somehow; the current situation is hopelessly busted. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

3 3

GHC Weekly news
by Austin Seipp 01 Oct '14

01 Oct '14

Hi *, Here's the latest weekly news post: https://ghc.haskell.org/trac/ghc/blog/weekly20140930 Included verbatim for those too lazy to open new tabs: ---------------- Hi *, Here's some news for y'all! Apologizes about the unavailability last week; the internet wasn't exactly a very fun place for a system administrator... So without delay, here's the current recap of the past two weeks: - Lots of merged code and code reviews have gone in, and a lot of commits: in the past two weeks since the last update, `ghc.git` has seen just over 100 commits, from about a dozen different developers. - As part of those patches, a significant amount of them have gone towards implementing the "Burning Bridges Proposal" or BBP for the `base` library. This is a set of changes to `base` that have generalized many parts of the API, by putting `Traversable` and `Foldable` in the Prelude. This required a bit of shoveling by Herbert, but now this is all now in GHC HEAD, and will be part of 7.10: - `Prelude` combinators, like `mapM`, have been generalized to the `Traversable` and `Foldable` classes. - Several other modules, like `Control.Monad` and `Data.List`, have been generalized to `Traversable` and `Foldable` where applicable. - `Control.Monad` combinators generalized to `Applicative` where possible. - Similarly, `MonadPlus` combinators like `guard` are generalized to `Alternative`. - `Foldable` has been extended with new methods, like `length` and `null`. - But also, GHC's compiler is now tab-free! That's right, after what seemed like a million years, a very large portion of the code has been detabbed, and `-fwarn-tabs` is now on by default in the GHC build. - There are an assortment of other changes: GHC's linker is not as loud[1], and various documentation improvements. - The windows build is broken *again* unfortunately, this time due to what seems to be a Cabal update, I think. Austin is once again on the case. - The HCAR draft for October has seen some nice improvements. If you're a developer, please amend things. If you're a user, read with eager anticipation of all the new features![2] - It turns out the new Applicative/Monad changes have unfortunately broken the `haskell98` and `haskell2010` packages, with an unclear migration strategy for the future: see #9590. For GHC 7.10, it seems the `haskell2010` packages will need to change to accomodate these standard deviations. If any users of the `haskell98` or `haskell2010` packages would speak up to help, that would be fantastic. The discussion will surely continue for a little bit - 7.10 is still a ways off. In miscellaneous news: - ghc.haskell.org may have been temporarily unavailable during this weekend due to an emergency downtime with our provider for a security update, but the window was quite small. - Relatedly (but not the exact same scenario), the internet also caught fire in several other places, requiring quite a lot of careful watching and remediation after the Bash "ShellShock" bug hit last week. And I think that sums it up quite nicely, folks! Closed tickets for the past two weeks include (there are a lot of them!): #9650, #7068, #5190, #5880, #8374, #9603, #3303, #3470, #3509, #781, #8115, #9641, #9191, #9515, #9326, #7987, #9634, #9576, #9612, #9635, #8593, #7544, #8529, #9338, #5794, #9535, #3646, #617, #8026, #8480, #8881, #9366, #8251, #7673, #8983, #8369, #8897, #8070, #9550, #9057, #9629, #8836, #8960, #8941, #9565, #9589, #5248, #8918, #9568, #9620, #1042, #9557, #7863, #5647, #9610, #5395, #9580, #9529, #4426, #9607, #9592, #8097, #9559, #9560, #4218, #9602, #9528, #9530, #9423, #9400, #1407, #9598, #9597. I'd like to mention that for the above tickets, a *huge* amount of them were closed by one of our newest contributors, **Thomas Miedema**, who went through the bug tracker and confirmed or closed a large majority of them. I lost track of how many. Thanks Thomas! [1] https://github.com/ghc/ghc/commit/9f7e3633c692dce75c27607131bd386178fb0fcf [2] https://ghc.haskell.org/trac/ghc/wiki/Status/Oct14 -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

2 1

Build time regressions
by Joachim Breitner 30 Sep '14

30 Sep '14

Hi, the attached graph shows a noticable increase in build time caused by Update Cabal submodule & ghc-pkg to use new module re-export types author Edward Z. Yang <ezyang(a)cs.stanford.edu> https://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176… and only halfway mitigated by Update `binary` submodule in an attempt to address #9630 author Herbert Valerio Riedel <hvr(a)gnu.org> https://git.haskell.org/ghc.git/commit/3ecca02516af5de803e4ff667c8c969c5bff… I am not sure if the improvement is related to the regression, but in any case: Edward, was such an increase expected by you? If not, can you explain it? Can it be avoided? Or maybe Cabal just became much larger... +38% in allocations when running haddock on it seems to confirm this. Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata(a)debian.org

1 0

Re: Aliasing current module qualifier
by John Meacham 30 Sep '14

30 Sep '14

You don't need a new language construct, what i do is: module AnnoyinglyLongModuleName (M.length, M.null) where import AnnoyinglongLongModuleName as M I think ghc would need to be extended a little to make this convienient as it doesn't handle recursive module imports as transparently. John On Mon, Sep 29, 2014 at 8:47 AM, Brandon Allbery <allbery.b(a)gmail.com> wrote: > On Mon, Sep 29, 2014 at 4:19 AM, Herbert Valerio Riedel <hvr(a)gnu.org> > wrote: > >> Now it'd be great if I could do the following instead: >> >> module AnnoyinglyLongModuleName (M.length, M.null) where >> >> import AnnoyinglyLongModuleName as M -- <- does not work >> > > I think if I wanted this syntax, I'd go for: > > module AnnoyinglyLongModuleName as M where ... > > -- > brandon s allbery kf8nh sine nomine > associates > allbery.b(a)gmail.com > ballbery(a)sinenomine.net > unix, openafs, kerberos, infrastructure, xmonad > http://sinenomine.net > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users > > -- John Meacham - http://notanumber.net/

3 3

Aliasing current module qualifier
by Herbert Valerio Riedel 29 Sep '14

29 Sep '14

Hello *, Here's a situation I've encountered recently, which mades me wish to be able to define a local alias (in order to avoid CPP use). Consider the following stupid module: module AnnoyinglyLongModuleName ( AnnoyinglyLongModuleName.length , AnnoyinglyLongModuleName.null ) where length :: a -> Int length _ = 0 null :: a -> Bool null = (== 0) . AnnoyinglyLongModuleName.length Now it'd be great if I could do the following instead: module AnnoyinglyLongModuleName (M.length, M.null) where import AnnoyinglyLongModuleName as M -- <- does not work length :: a -> Int length _ = 0 null :: a -> Bool null = (== 0) . M.length However, if I try to compile this, GHC complains about AnnoyinglyLongModuleName.hs:4:1: Bad interface file: AnnoyinglyLongModuleName.hi AnnoyinglyLongModuleName.hi: openBinaryFile: does not exist (No such file or directory) while GHCi tells me: Module imports form a cycle: module ‘AnnoyinglyLongModuleName’ (AnnoyinglyLongModuleName.hs) imports itself Is there some other way (without CPP) to create a local alias for the current module-name? If not, is there a reason GHC couldn't support this special case of self-aliasing the current module name? PS: Alternatively, this could be done as a language extension but that'd require extending the Haskell grammar: module AnnoyinglyLongModuleName as M (M.length, M.null) where Cheers, hvr

5 5

Re: [commit: ghc] master: Set default-impl of `mapM`/`sequence` methods to `traverse`/`sequenceA` (f636faa)
by Joachim Breitner 29 Sep '14

29 Sep '14

Hi, Am Samstag, den 27.09.2014, 21:07 +0000 schrieb git(a)git.haskell.org: > commit f636faa7b2b7fc1d0663f994ad08f365d39a746d > Author: Herbert Valerio Riedel <hvr(a)gnu.org> > Date: Sat Sep 27 22:55:19 2014 +0200 > > Set default-impl of `mapM`/`sequence` methods to `traverse`/`sequenceA` > > This is made possible by the AMP, as we don't need the `WrappedMonad` > helper for that anymore. according to http://ghcspeed-nomeata.rhcloud.com/changes/?rev=f636faa7b2b7fc1d0663f994ad… this has caused a 11% regression in fibheaps allocation: http://ghcspeed-nomeata.rhcloud.com/timeline/?ben=nofib/allocs/fibheaps&env… Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata(a)debian.org

2 1

Re: [commit: ghc] master: Extend `Foldable` class with `length` and `null` methods (e5cca4a)
by Joachim Breitner 29 Sep '14

29 Sep '14

Hi, Am Sonntag, den 28.09.2014, 13:07 +0000 schrieb git(a)git.haskell.org: > commit e5cca4ab246ca2d1ecdd7c39eefd3157547cb6aa > Author: Herbert Valerio Riedel <hvr(a)gnu.org> > Date: Sun Sep 28 13:02:53 2014 +0200 > > Extend `Foldable` class with `length` and `null` methods > > This completes the `Foldable` class by two important operations which > this way can be optimised for the underlying structure more easily. > > A minor fix for the `containers` submodule was needed to due name clash > > Addresses #9621 > > Reviewed By: ekmett, dfeuer, austin > > Differential Revision: https://phabricator.haskell.org/D250 this broke nofib: ==nofib== para: time to compile Main follows... /data1/ghc-builder/logs/ghc-tmp-REV/inplace/bin/ghc-stage2 -O2 -Rghc-timing -H32m -hisuf hi -rtsopts -c Main.lhs -o Main.o Main.lhs:517:32: No instance for (Foldable t0) arising from a use of ‘step’ The type variable ‘t0’ is ambiguous Relevant bindings include par1' :: [t0 a] -> [[t0 a]] (bound at Main.lhs:516:2) Note: there are several potential instances: instance Foldable ((,) a) -- Defined in ‘Data.Foldable’ instance GHC.Arr.Ix i => Foldable (GHC.Arr.Array i) -- Defined in ‘Data.Foldable’ instance Foldable (Either a) -- Defined in ‘Data.Foldable’ ...plus three others In the first argument of ‘fold1’, namely ‘step’ In the second argument of ‘(.)’, namely ‘fold1 step start’ In the second argument of ‘(.)’, namely ‘minWith cost . fold1 step start’ <<ghc: 93451064 bytes, 44 GCs, 4435640/8217616 avg/max bytes residency (4 samples), 31M in use, 0.001 INIT (0.001 elapsed), 0.051 MUT (0.056 elapsed), 0.057 GC (0.057 elapsed) :ghc>> Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata(a)debian.org

2 1

tar --help is failing on Solaris.
by Karel Gardas 27 Sep '14

27 Sep '14

Hello Mikhail, you are the author of the patch eaf2aae18603965ff668384b391fcaa14de19823 in libraries/Cabal which you have merged into GHC HEAD on September 25 in the patch 4be50969491aa471d6c9ab3b9339036a355d45c6 The problem is that your patch breaks build on Solaris, here is the log from the Solaris/x86 builder: http://haskell.inf.elte.hu/builders/solaris-x86-head/178/10.html Could you be so kind and revert this patch or fix it to also work on Solaris? For Solaris the change should be easy, simply if you invoke Solaris' tar with --help, it'll fail with exit code 1 and help message printed into the stderr. If you are curious if it supports --format, then no. Please let me know if you are able to push fix or revert during next week or if I shall deal with it myself. I can't just fix that now myself since I'm overloaded with other tasks till Wednesday night... Thanks! Karel

3 3