January 2016 - ghc-devs

[ANNOUNCE] Glasgow Haskell Compiler 8.0.1, release candidate 1
by Ben Gamari 02 Feb '16

02 Feb '16

The GHC Team is very pleased to announce the first release candidate of the Glasgow Haskell Compiler 8.0.1 release. Source and binary distributions as well as the newly revised users guide can be found at http://downloads.haskell.org/~ghc/8.0.1-rc1/ This is the first in a series of release candidates which will allow us to get wider testing of the significant changes that have occurred since the 7.10 series. These include, * the TypeInType extension, which unifies types and kinds allowing for promotion of more Haskell constructs to the type-level * the introduction of type application in source programs * support for recursive superclass relationships * support for Applicative do notation * introduction of the DuplicateRecordFields language extension * a rewritten and substantially more thorough pattern match checker * the introduction of injective type classes * introduction of the Strict and StrictData language extensions, allowing modules to be compiled with strict-by-default evaluation of bindings * the ability to run the GHCi interpreter in a separate process, allowing a callstacks in GHCi, easier integration with tooling, and more and much more. Changes of this magnitude will invariably bring bugs. This release candidate in particular is known to suffer from a few significant issues which are being actively worked upon, * The new -XInjectiveTypeFamilies language extension will likely be renamed to -XTypeFamilyDependencies * #11120: Type representations are missing for some types and promoted constructors * #11334: Solving for Typeable (Proxy :: Proxy 'Compose) fails * #11276: Pattern checker performance can degrade significantly in presence of pattern matches with guards * #11405: Type-level skolem-escape check fails incorrectly * #11414: Use of -XStrict results in compiler abort * #11379: Instance solver fails to terminate * #11419: Haddock documentation is currently not included in the binary distributions (and hence is missing on downloads.haskell.org) * #11370: -Wredundant-constraints being included in -Wall breaks the three-release compatibility policy In the coming weeks we will continue to iterate on these issues. We will also look at Trac tickets marked with "highest" priority on the release status page [2]. If you have a ticket that you would like to see addressed that does not meet one of these criteria, please bring this to our attention. Likewise, if you encounter an issue please open a ticket if one does not already exist. Also note that we currently cannot offer 32-bit Windows builds due to breaking changing in a recent Windows 10 upgrade. We'll work to resolve this before the 8.0 release but please let us know if this poses a significant problem for you. Cheers, - Ben [1] https://cygwin.com/ml/cygwin/2015-12/msg00003.htm [2] https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-8.0.1

13 35

Dropping bzip2 release tarballs?
by Ben Gamari 02 Feb '16

02 Feb '16

tl;dr do you rely on the .bz2 release tarballs on downloads.haskell.org? If so, let us know! Hello everyone, As you may have noticed, a few releases ago we started producing xz-compressed binary distributions in addition to the usual bzip2 tarballs. While preparing 8.0.1-rc1 it was suggested that we move to distributing xz tarballs exclusively. Not only would this move reduce storage and bandwidth demands on our infrastructure but it would also simplify the job of producing the distributions. Indeed there is plenty of precendent for projects who have moved exclusively to xz (the Linux kernel and git being two examples). Of course, these reasons alone aren't sufficient to abandon those who might rely on our bzip2 tarballs. If you feel strongly that we should continue to distribute bzip2 tarballs, please let us know. Thanks! - Your friendly GHC packaging gnomes

4 4

Nailing down what we expect IO to do and not do - and why
by Christopher Allen 01 Feb '16

01 Feb '16

I'm writing a book, I'd like to get this nailed down and to get it right. If anyone on here that's familiar with the various ways in which IO/State#/realWorld# work in GHC and you have time to reply, anything at all would be welcome. Any pointers, links, references, details, anecdotes, or faint memories of GHC bugs will be greatly appreciated! Getting this written up (possibly for addition to Michael Snoyman's wiki article?) would make me, and I imagine others, a lot happier with trying to understand how the different bits and bobs fit together. I will be dumping my notes as I don't want to get linked to stuff that can be googled because I've already lost 10-15 hours to just that in the past 3-4 days. Digging it up in the compiler is hard because compiler behavior that influences how IO actions are treated don't necessarily have "IO" or "realWorld" mentioned in the relevant parts of the compiler, optimizations, etc. What I'm hoping for is answers on what specifically preserves the listed properties we want from IO in the compiler, prims, or structure of how we write IO actions. What we expect IO to do: - Disable sharing of results, even when it's not a lambda and is evaluated multiple times by the same name. ie, getCurrentTime :: IO UTCTime should get evaluated more than once. - Not reorder sequential IO actions, such as in a do-block. Called "linearity" below - Not duplicate the effects of IO actions. Effects shouldn't be spuriously duplicated during optimization passes. - Effects should not be discarded separately of the value returned by an IO action, merged, or elided. # Sharing A friend suggested that perhaps one-shot semantics via the state hack for State# in the IO type is responsible for disabling sharing, I don't believe so, but here are my notes. >-fno-state-hack >Turn off the "state hack" whereby any lambda with a State# token as argument is considered to be single-entry, hence it is considered OK to inline things inside it. This can improve performance of IO and ST monad code, but it runs the risk of reducing sharing. >A one shot lambda >State hack, makes the lambda over State# assume it's one-shot universally by default. >one-shot/state hack is an anti-inlining heuristic, suggesting that inlining is costly. Also I found this on Trac, does anyone know the answer to this? Is the summary above accurate? >Can the IO state hack be avoided if oneShot is used in the right places in library code, e.g. in IO’s definition of >>=? This seems related how the state token works, for differentiating which IO action is which and how many times an IO action should run, when it should run, etc. >From the prims: >data State# s >State# is the primitive, unlifted type of states. It has one type parameter, thus State# RealWorld, or State# s, where s is a type variable. The only purpose of the type parameter is to keep different state threads separate. It is represented by nothing at all. >data RealWorld >RealWorld is deeply magical. It is primitive, but it is not unlifted (hence ptrArg). We never manipulate values of type RealWorld; it's only used in the type system, to parameterise State#. # Linearity Is this from the nesting of lambdas? It doesn't seem like that's enough based on the various examples using State/State# in GHC Trac bug tickets. The RealWorld token seems to be what's driving this but precisely how that works hasn't been easy to find. # Discarding, not inlining effects I believe these are addressed by has_side_effects in the prim ops. I could very well be wrong. can_fail has_side_effects Discard NO NO Float in YES YES Float out NO NO Duplicate YES NO * Duplication. You cannot duplicate a has_side_effect primop. You might wonder how this can occur given the state token threading, but just look at Control.Monad.ST.Lazy.Imp.strictToLazy! We get something like this p = case readMutVar# s v of (# s', r #) -> (S# s', r) s' = case p of (s', r) -> s' r = case p of (s', r) -> r I believe duplication addresses inlining IO actions more generally but I could be wrong. Here's a note I found regarding elision/merging: * Use the compiler flag @-fno-cse@ to prevent common sub-expression elimination being performed on the module, which might combine two side effects that were meant to be separate. A good example is using multiple global variables (like @test@ in the example below). Any help or pointers for nailing down and documenting this would be greatly appreciated. Also if there's a more detailed explanation of what behavior is expected out of each unsafe function, that would help as well. There are bits and pieces I've been able to aggregate from the GHC trac tickets. References used (not exhaustive): - Referential Transparency; Haskell Wiki https://wiki.haskell.org/Referential_transparency - IO Inside; Haskell Wiki https://wiki.haskell.org/IO_inside - Unraveling the mystery of the IO Monad; Edward Z. Yang http://blog.ezyang.com/2011/05/unraveling-the-mystery-of-the-io-monad/ - Evaluation order and state tokens; Michael Snoyman https://wiki.haskell.org/Evaluation_order_and_state_tokens - Haskell GHC Illustrated; Takenobu Tani - Tackling the Awkward Squad; Simon Peyton Jones http://research.microsoft.com/en-us/um/people/simonpj/papers/marktoberdorf/… - Note [IO hack in the demand analyser]; GHC source code - Monadic I/O in Haskell 1.3; Andrew D. Gordon and Kevin Hammond - Haskell Report 1.2 http://haskell.cs.yale.edu/wp-content/uploads/2011/01/haskell-report-1.2.pdf Thank you for your time, Chris

2 2

[ANNOUNCE] Shaking up GHC
by Andrey Mokhov 31 Jan '16

31 Jan '16

Dear GHC developers, I am happy to announce that the Shaking up GHC project has finally reached the first milestone. The goal of the project is to design a new GHC build system based on Shake that will eventually replace the current make-based one. See the project page for more details: https://github.com/snowleopard/shaking-up-ghc. There is still a long way until we can match the capabilities of the current build system. At the moment we only build vanilla way. Validation, documentation, build flavours and cross-compilation are not yet implemented. Known limitations are listed here: https://github.com/snowleopard/shaking-up-ghc#current-limitations. The purpose of this announcement is to attract alpha testers and collect early feedback across multiple platforms and build configurations. We already have several success reports on Linux, OS X, Solaris and Windows. However, things will inevitably break and we hope to catch and fix as many of these cases as possible with your help. The instructions on how to try the new build system can be found here: https://github.com/snowleopard/shaking-up-ghc#your-first-build. We plan to be ready to become a part of the GHC tree around 1 March 2016, and catch up with the make build system around 1 June 2016. The dates are tentative and depend on how much time it takes us to resolve the remaining issues: https://github.com/snowleopard/shaking-up-ghc/milestones. I would like thank everyone who contributed to this project so far: Simon Peyton Jones, Neil Mitchell and Simon Marlow came up with the idea and guided me throughout the project; Moritz Angermann, Ben Gamari, Karel Gardas, David Luposchainsky, and Neil Mitchell contributed to the codebase. Thank you all! Kind regards, Andrey

4 8

Build broken
by Geoffrey Mainland 31 Jan '16

31 Jan '16

I attempted to rebase my fix for #11487, but I can no longer validate. The culprit is 6c7760b2. I am seeing the errors below under linux, building with 7.10.3. Herbert, I hope there is a quick fix? Thanks, Geoff libraries/unix/System/Posix/IO/Common.hsc:316:61: error: • Couldn't match type ‘()’ with ‘Base.CFLock’ Expected type: Ptr Base.CFLock Actual type: Ptr CFLock • In the third argument of ‘Base.c_fcntl_lock’, namely ‘p_flock’ In the second argument of ‘throwErrnoIfMinus1_’, namely ‘(Base.c_fcntl_lock fd (5) p_flock)’ In a stmt of a 'do' block: throwErrnoIfMinus1_ "getLock" (Base.c_fcntl_lock fd (5) p_flock) libraries/unix/System/Posix/IO/Common.hsc:364:61: error: • Couldn't match type ‘()’ with ‘Base.CFLock’ Expected type: Ptr Base.CFLock Actual type: Ptr CFLock • In the third argument of ‘Base.c_fcntl_lock’, namely ‘p_flock’ In the second argument of ‘throwErrnoIfMinus1_’, namely ‘(Base.c_fcntl_lock fd (6) p_flock)’ In the expression: throwErrnoIfMinus1_ "setLock" (Base.c_fcntl_lock fd (6) p_flock) libraries/unix/System/Posix/IO/Common.hsc:371:35: error: • Couldn't match type ‘()’ with ‘Base.CFLock’ Expected type: Ptr Base.CFLock Actual type: Ptr CFLock • In the third argument of ‘Base.c_fcntl_lock’, namely ‘p_flock’ In the second argument of ‘throwErrnoIfMinus1_’, namely ‘(Base.c_fcntl_lock fd (7) p_flock)’ In the expression: throwErrnoIfMinus1_ "waitToSetLock" (Base.c_fcntl_lock fd (7) p_flock)

1 0

More aggressive dictionary removal?
by Conal Elliott 30 Jan '16

30 Jan '16

I'm looking for pointers on getting GHC to eliminate more overloading & polymorphism. I think this sort of thing mainly happens in the Specialise module. The default GHC flag settings get me a couple levels of monomorphization and dictionary removal, but I want to go further. I've tried -fspecialise-aggressively, but it didn't seem to make a difference, and I haven't found this flag described in the GHC user's guide. Anyone have pointers to more information? Thanks, - Conal

4 7

a reliable way of dropping levity args?
by Ömer Sinan Ağacan 30 Jan '16

30 Jan '16

Hi all, I'm looking for a reliable way of dropping levity args from TyCon applications. When I know that a particular TyCon gets some number of levity args, I can just drop the args manually (for example, I can drop the first half of arguments of a tuple TyCon application) but the code looks fragile (what happens if I use a different TyCon in the future) and confusing to the reader because it looks like this: drop (length args `div` 2) args Ideally it'd look like this: dropWhile isLevityArg args Now, there's a function called isLevityTy, but I don't understand what it's supposed to do. This doesn't do anyting to 'Boxed and 'Unboxed arguments: dropWhile (isLevityArg . idType) args Any ideas on this? Thanks..

3 8

Adding a "release" setting in build.mk.sample (and some other build system questions)
by Ömer Sinan Ağacan 29 Jan '16

29 Jan '16

I'm trying to figure out how to generate a release build. I thought it should be "perf" setting, but then I realized ghc-stage1 is called with -O (instead of -O2) when building stage2 with perf setting. So either perf is not the release setting, or I need stage3 which is probably compiled with stage2 using -O2? In any case, it'd be great to document this process of generating a release build in build.mk.sample. If it's "perf" setting, then maybe we can rename it to "release" or at least add a comment saying that that's the release build and you should wait until stage3 is done. Another thing that always confuses me is these settings: SRC_HC_OPTS = -O -H64m GhcStage1HcOpts = -O GhcStage2HcOpts = -O2 GhcLibHcOpts = -O2 As far as I can see, there's no documentation about these in the source tree. I figured that `GhcStage1HcOpts` is passed to system GHC when compiling stage1, and `GhcStage2Hcopts` is passed to ghc-stage1 when compiling stage 2. The lib one is obvious, and it's passed to stage 1 as libs are built using stage 1. But I still don't understand how is `SRC_HC_OPTS` is used, and what is used when building stage 3.

2 1

Unexpected lack of change in ghcspeed results
by Reid Barton 29 Jan '16

29 Jan '16

Hi Joachim, I was interested to see what effect the recent commit "Restore original alignment for info tables" (0dc7b36c) would have on performance. However, when I look at http://ghcspeed-nomeata.rhcloud.com/changes/?rev=0dc7b36c3c261b3eccf8460581…, I don't see the expected binary size increase (about 1%) that I got in local testing. Instead, the size increase appears to be attached to commit 0d92d9cb6d65fd00f9910c3f6f85bc6c68f5543b. I notice that these two commits, along with three others, were committed at exactly the same time (Wed Jan 27 11:32:15 2016 +0100), presumably in a rebase. Could this be confusing ghcspeed? Regards, Reid Barton

2 4

GHC 8.0.1 plans
by Ben Gamari 27 Jan '16

27 Jan '16

tl;dr: The march towards 8.0.1 continues. Release candidate 2 will likely be cut late next week. If you have a ticket which you'd like to see addressed but suspect we aren't working on then please say so. Hello again GHC-ers, As you likely know, we are currently in the stabilization period of GHC's 8.0.1 release. Release candidate 1 has been out for nearly two weeks now and issues have been steadily trickling in. The problems we've heard about thusfar have largely fallen into a few broad categories, * OS X linker issues due to botched preparation of the first binary distribution (#11423) * pattern match checker performance. These we largely knew about prior to the release, although it has been surprising to see how easily one can wander into the new checker's performance abyss. (#11374, #11276, #11163, #11302, #11303, #11195) * Regressions due to visible type applications (#11329, #11458) * Issues uncovered by generalizations allowed under TypeInType (#11429, #11334, #11473, #11405, #11407, #11399) * Trouble caused by new warnings (#11370, #11316, #11474) Since the release, there has been significant progress on many of these fronts: * George Karachalias has been hard at work further improving the pattern checker. The result should have far more predictable performance characteristics at less cost in reasoning power. * Simon Peyton Jones has been squashing bugs at a furious rate, including a variety of long-standing issues (#11248, #11330 and many others) * Tamar Christina has added support for remote GHCi on Windows and resolved a long-standing linking issue (#11223, #11100) * A variety of limitations in our implementation of Typeable have been fixed (#11120) although there is still more to be done (#11334). * InjectiveTypeFamilies has been renamed to TypeFamiliesDependencies * Custom type errors are now more robust (#11391) * We now have a more conservative default warning set, as well as better mechanisms for managing warning changes in the future. (#11429, #11370) * A wide variety of miscellaneous bug-fixes have also been merged. Given how much progress has been made, we approaching the point where we can begin planning the next candidate. At this point the plan is to wait until we have landed George's pattern checker rework, as this has been one of the more visible issues in practice. This may be merged as early as the latter half of next week. If this happens then we hope to have the next release candidate out late next week. In the interim we will continue to work on paring down the bug list, focusing on the highest priority tickets listed on the 8.0.1 status page. In particular, those with owners are most likely to be dealt with soon. If you see a ticket which you would like to see addressed in 8.0.1 but is unowned or lower than "highest" priority, please bring this to our attention. Given the large amount of churn still being felt in the `ghc-8.0` branch (with over 150 comments in the two weeks since -rc1) and the size of some of the patches that have yet to land, it's unclear whether two release candidates will be sufficient to ensure the stability of the branch. Consequently, unless -rc2 yields a particularly small number of new tickets it's likely that it will be followed by a third candidate before the release. As always, let us know if you have any questions or concerns about the release process. Thanks to everyone who has contributed to -rc1! Cheers, - Ben

2 2