- ghc-devs - Haskell.org

Converting unboxed sum types in StgCmm
by Johan Tibell 22 Sep '15

22 Sep '15

Hi! The original idea for implementing the backend part of the unboxed sums proposal was to convert from the core representation to the actual data representation (i.e. a tag followed by some pointer and non-pointer fields) in the unarise stg-to-stg <https://github.com/ghc/ghc/blob/master/compiler/simplStg/UnariseStg.hs> pass. I have now realized that this won't work. The problem is that stg is too strongly typed. When we "desugar" sum types we need to convert functions receiving a value e.g. from f :: (# Bool | Char #) -> ... to f :: NonPointer {-# tag#-} -> Pointer {-# Bool or Char #-} -> ... Since stg is still typed with normal Haskell types (e.g. Bool, Char, etc), this is not possible, as we cannot represent an argument which has two different types. It seems to me that we will have to do the conversion in the stg-to-cmm <https://github.com/ghc/ghc/blob/master/compiler/codeGen/StgCmm.hs> pass, which is quite a bit more involved. For example, StgCmmEnv.idToReg function will have to change from idToReg :: DynFlags -> NonVoid Id -> LocalReg to idToReg :: DynFlags -> NonVoid Id -> [LocalReg] to accommodate the fact that we might need more than one register to store a binder. Any ideas for a better solution? -- Johan

5 12

Proposal: Associating pattern synonyms with types
by Matthew Pickering 19 Sep '15

19 Sep '15

Dear devs, There has been a bit of discussion on #10653 about extending exports to allow users to associate pattern synonyms with types (just like ordinary data constructors). There is a section on the wiki which describes the problem and the proposed behaviour. https://ghc.haskell.org/trac/ghc/wiki/PatternSynonyms#Associatingsynonymswi… Any comments would be appreciated before I start the implementation. Please read all the way to the bottom past the examples as their are a few sections of clarification which might answer your question. Matt

1 0

Haskell Error Messages
by Alan & Kim Zimmerman 18 Sep '15

18 Sep '15

Is there currently any planned work around making the haskell error messages able to support something like the ones in IDRIS, as shown in David Christianson's talk "A Pretty printer that says what it means" at HIW? https://www.youtube.com/watch?v=m7BBCcIDXSg&list=PLnqUlCo055hVfNkQHP7z43r10… Alan

4 5

Renaming InstalledPackageId
by Edward Z. Yang 18 Sep '15

18 Sep '15

Hello friends, During discussions with many people about Nix-like Cabal, it has emerged that InstalledPackageId is /really/ bad name. Consider: the commonly accepted definition of an InstalledPackageId in Nix is that it is morally a hash of all the inputs to compilation: the source code, the dependencies of the package, and the build configuration. However, a Cabal package can have *multiple* components (e.g. the library, the test suite, etc), each of which has its own 'build-depends' field. The concept of the "dependencies of a package" is simply not well-defined! The "simplification" that Cabal has adopted for a long time is to say that the installed package ID always refers to the library component of a package. [1] But the name InstalledPackageId has caused countless misunderstandings about how dependency resolution is done, even in Cabal's code. [2] I propose that we change the name of InstalledPackageId. The new name should have the following properties: 1. It should not say anything about packages, because it is not well-defined for a package, e.g. it should be something like "ComponentId". 2. It should not say anything about installation, because it is well-defined even before a package is even built. 3. It should some how communicate that it is a hash of the transitive source code (e.g. including dependencies) as well as build parameters. SPJ likes "SourceHash" because it's evocative of this (though I don't like it as much); there may also be a Nix-y term like "Derivation" we can use here. My proposed new name is "ComponentBuildHash", however I am open to other suggestions. I might also be convinced by "InstalledComponentId" (which runs aground (2) but is fairly similar to the old name, and gains points for familiarity.) However, I would like to hear your comments: have a better name? Think this is unnecessary? Please let me know. Edward P.S. With Backpack, the ComponentBuildHash won't even be the primary key into the installed package (to be renamed to a component/unit) database, because a single ComponentBuildHash can be rebuilt multiple times with different instantiations of its holes. So GHC will have some identifier, which we will probably continue to call the 'UnitKey', which is the ComponentBuildHash (entirely Cabal generated) plus extra information about how holes are instantiated (entirely GHC generated). [1] Except when it doesn't: cabal-install currently merges all the dependencies of all components that are being built for a package together and treats that as the sum total dependencies of the package. This causes problems when the test suite for text depends on a testing library which in turn depends on text, c.f. https://github.com/haskell/cabal/issues/960 [2] Here are some bugs caused by confusion of package dependencies versus component dependency: https://github.com/haskell/cabal/issues/2802 Specify components when configuring, not building https://github.com/haskell/cabal/issues/2623 `-j` should build package components in parallel https://github.com/haskell/cabal/issues/1893 Use per-component cabal_macros.h https://github.com/haskell/cabal/issues/1575 Do dependency resolution on a per component basis https://github.com/haskell/cabal/issues/1768 The "benchmark" target dependencies conflict with "executable" targets https://github.com/haskell/cabal/issues/960 Can't build with --enable-tests in presence of circular dependencies

2 1

Making compilation results deterministic (#4012)
by Bartosz Nitka 18 Sep '15

18 Sep '15

Hello, For the past couple of weeks I've been working on making compilation results deterministic. What I'm focusing on right now is the interface file determinism, I don't care about binaries being deterministic. I'd like to give a status update and ask for some advice, since I'm running into issues that I don't have a good way of solving. The first question anyone might ask is how did nondeterminism creep into the compiler. If we're compiling with a single thread there's no reason for the computation to proceed in non deterministic way. I'm fairly certain that the issue originates from lazy loading of interface files. Relevant function: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/typech…. What happens is that if you already have an interface file for a target you're trying to build the computation will proceed differently. Why does lazy loading matter? As you load the interface file it needs to get type-checked and that means it needs to pull some Uniques from a global UniqSupply. It does that in different order resulting in different Unique assignment. As far as I can tell, lazy loading is required for performance, so I abandoned the idea of fixing it. I haven't looked at parallel compilation yet, but I'd expect it to result in different Unique assignment as well. I believe up to this point we're ok. Uniques are non-deterministic, but it shouldn't be a big deal. Uniques should be opaque enough to not affect the order of computation, for example the order of binds considered. But they aren't. Uniques are used in different ways throughout the compiler and they end up reordering things: 1) They have an `Ord` instance: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/basicT… . So far the places it impacts the most are places that use `stronglyConnCompFromEdgedVertices`, because Unique is used as a Node key and the result depends on the order of Nodes being considered. Some examples: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/simplC… https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/rename… (because Ord for Name uses Unique https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/basicT… ) I've tried to see what removing it would entail and the changes would be far reaching: https://phabricator.haskell.org/P62. 2) VarEnv, NameEnv are implemented in terms of UniqFM, which is just Data.IntMap with keys being the Unique integer values. The way this bites us is that when UniqFM's get converted to a list they end up being sorted on Unique value. This problem is more widespread than the `stronglyConnCompFromEdgedVertices` issue, there's even a place where it's implicitly depended on: https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/native… . I've tried to fix it by making `toList` return the elements in the order of insertion (https://phabricator.haskell.org/P63) but that turned out to have significant cost. My unscientific benchmark on aeson and text showed 10% compilation time increase. It's possible it can be done in less expensive way, I've tried a couple of approaches, but all of them resulted in 10% time increase. I've also considered to split UniqFM to two different types, one that keeps the ordering, and one that can't `toList`, but I suspect that the cut will not be clean, so I haven't tried that. In some cases we got away with ordering things by OccName where needed: ( https://phabricator.haskell.org/D1073, https://phabricator.haskell.org/D1192) but OccName's don't have to be unique in every case and if we try to make them unique that would make them longer and probably result in even greater slowdown. The instance I've recently looked at doesn't look like it can be solved by sorting by OccName. The code that triggers the problem (simplified from haskell-src-exts): data Decl l = Boring l deriving (Eq) data Binds l = BDecls l [Decl l] -- ^ An ordinary binding group | IPBinds l [IPBind l] -- ^ A binding group for implicit parameters deriving (Eq) data IPBind l = Boring2 l deriving (Eq) The end result is: 4449fe3f8368a2c47b2499a1fb033b6a $fEqBinds_$c==$Binds :: Eq l => Binds l -> Binds l -> Bool {- Arity: 1, HasNoCafRefs, Strictness: <L,U(C(C1(U)),A)>, Unfolding: (\ @ l $dEq :: Eq l -> let { $dEq1 :: Eq (Decl l) = $fEqDecl @ l $dEq } in let { $dEq2 :: Eq (IPBind l) = $fEqIPBind @ l $dEq } in \ ds :: Binds l ds1 :: Binds l -> case ds of wild { BDecls a1 a2 -> case ds1 of wild1 { BDecls b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (Decl l) $dEq1 a2 b2 } IPBinds ipv ipv1 -> False } IPBinds a1 a2 -> case ds1 of wild1 { BDecls ipv ipv1 -> False IPBinds b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (IPBind l) $dEq2 a2 b2 } } }) -} vs bb525bf8c0145a5379b3c29e8adb4b18 $fEqBinds_$c==$Binds :: Eq l => Binds l -> Binds l -> Bool {- Arity: 1, HasNoCafRefs, Strictness: <L,U(C(C1(U)),A)>, Unfolding: (\ @ l $dEq :: Eq l -> let { $dEq1 :: Eq (IPBind l) = $fEqIPBind @ l $dEq } in let { $dEq2 :: Eq (Decl l) = $fEqDecl @ l $dEq } in \ ds :: Binds l ds1 :: Binds l -> case ds of wild { BDecls a1 a2 -> case ds1 of wild1 { BDecls b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (Decl l) $dEq2 a2 b2 } IPBinds ipv ipv1 -> False } IPBinds a1 a2 -> case ds1 of wild1 { BDecls ipv ipv1 -> False IPBinds b1 b2 -> case == @ l $dEq a1 b1 of wild2 { False -> False True -> $fEq[]_$c== @ (IPBind l) $dEq1 a2 b2 } } }) -} This happens because when desugaring dictionaries we do an SCC on Uniques that ends up reordering lets ( https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/deSuga… ) Now all of the dictionaries have OccName of Eq. I could probably reach deeper into the term to extract enough information (in this instance, the constructor name) to get deterministic ordering, but this feels very ad-hoc and I don't expect it to scale to the whole codebase. Another problem with fixing things in this ad-hoc manner is keeping them fixed. There's nothing preventing people from introducing nondeterminism. One idea that makes it more testable is to test it with different UniqSupply allocation patterns. I've found it useful to compare against UniqSupply that starts at a big number and allocates in decreasing order. Gray codes could be used to generate non-sequential order. The reason I'm posting this is to get some ideas, because at this point I feel stuck, I don't see a good way of achieving the end goal. I hope someone with more intimate GHC knowledge can point out a wrong assumption I've made or suggest an approach I haven't thought of. Cheers, Bartosz

5 8

GHC Weekly News - 2015/09/17
by Austin Seipp 17 Sep '15

17 Sep '15

(This post is available online at https://ghc.haskell.org/trac/ghc/blog/weekly20150917) Hi *, Welcome for the latest entry in the GHC Weekly News. It's been a little while, but here we are! And your author has finally returned from his 8 week sabbatical, too! So without any futher ado, lets get going... == 8.0.1 release roadmap == So `HEAD` has been steaming along pretty smoothly for the past few months now. After talking with Simon last week, we decided that the best course of action would be to release 8.0.1 (a super-major release) sometime around late February, which were the plans for 7.10 (modulo a few weeks due to the FTP debates). The current schedule is roughly: - November: Fork the new `ghc-8.0` STABLE branch - At this point, `master` development will probably slow as we fix bugs. - This gives us 2 months or so until branch, from Today. - This is nice as the branch is close to the first RC. - December: First release candidate - Mid/late February: Final release. "'''Why call it 8.0.1?'''", you ask? Because we have a lot of excellent features planned! I'm particularly partial to Richard's work for merging types and kinds (Phab:D808). But there's a lot of other stuff. For all the nitty gritty details, be sure to check [wiki:Status/GHC-8.0.1 8.0.1 status page] to keep track of everything - it will be our prime source of information and coordination. And be sure to [https://mail.haskell.org/pipermail/ghc-devs/2015-September/009952.html read my email to `ghc-devs`] for more info. === ... and a 7.10.3 release perhaps? === On top of this, we've been wondering if another release in the 7.10 branch should be done. Ben did the release shortly after I left, and for the most part looked pretty great. But there have been some snags, as usual. So we're asking: [https://mail.haskell.org/pipermail/ghc-devs/2015-September/009953.html who needs GHC 7.10.3?] We'd really like to know of any major showstoppers you've found with 7.10 that are preventing you from using it. Especially if you're stuck or there's no clear workaround. Currently, we're still not 100% committed to this course of action (since the release will take time away from other things). However, we'll keep the polls open for a while - so ''please'' get in touch with us if you need it! (Be sure to read my email for specific information.) == List chatter == (Over the past two weeks) - Bartosz Nitka writes to `ghc-devs` about the ongoing work to try and fix deterministic compilation in GHC (the dreaded ticket #4012). There's a very detailed breakdown of the current problems and issues in play, with responses from others - https://mail.haskell.org/pipermail/ghc-devs/2015-September/009964.html - Richard Eisenberg wants to know - how can I download all of `Hackage` to play with it? GHC developers are surely interested in this, so they can find regressions quickly - https://mail.haskell.org/pipermail/ghc-devs/2015-September/009956.html - I wrote to the list about the upcoming tentative 7.10.3 plans, as I mentioned above. https://mail.haskell.org/pipermail/ghc-devs/2015-September/009953.html - I ''also'' wrote to the list about the tentative 8.0.1 plans, too. https://mail.haskell.org/pipermail/ghc-devs/2015-September/009952.html - Johan Tibell asks about his ongoing work for implementing unboxed sum types - in particular, converting unboxed sum types in `StgCmm`. https://mail.haskell.org/pipermail/ghc-devs/2015-September/009926.html - Ryan Scott wrote a proposal for the automatic derivation of `Lift` through GHC's deriving mechanism, specifically for `template-hasekll` users. The response was positive and the code is going through review now (Phab:D1168). https://mail.haskell.org/pipermail/ghc-devs/2015-September/009838.html - Andrew Gibiansky writes in with his own proposal for a new "Argument Do" syntax - a change which would allow `do` to appear in positions without `($)` or parenthesis, essentially changing the parser to insert parens as needed. The code is up at Phabricator for brave souls (Phab:D1219). https://mail.haskell.org/pipermail/ghc-devs/2015-September/009821.html - Edward Yang started a monstrous thread after some discussions at ICFP about a future for ''unlifted'' data types in GHC. These currently exist as special magic, but the proposals included would allow users to declare their own types as unlifted, and make unlifted values more flexible (allowing `newtype` for example). See wiki:UnliftedDataTypes and Edward's thread for more. https://mail.haskell.org/pipermail/ghc-devs/2015-September/009799.html == Noteworthy commits == (Over the past several weeks) - Commit 374457809de343f409fbeea0a885877947a133a2 - '''Injective Type Families''' - Commit 8ecf6d8f7dfee9e5b1844cd196f83f00f3b6b879 - '''Applicative Do notation''' - Commit 6740d70d95cb81cea3859ff847afc61ec439db4f - Use IP-based CallStack in `error` and `undefined` - Commit 43eb1dc52a4d3cbba9617f5a26177b8251d84b6a - Show `MINIMAL` complete definition in GHCi's `:info` - Commit 296bc70b5ff6c853f2782e9ec5aa47a52110345e - Use a response file for linker command line arguments - Commit 4356dacb4a2ae29dfbd7126b25b72d89bb9db1b0 - Forbid annotations when Safe Haskell is enabled - Commit 7b211b4e5a38efca437d76ea442495370da7cc9a - Upgrade GCC/binutils to 5.2.0 release for Windows (i386/amd64) == Closed tickets == (Over the past two weeks) #10834, #10830, #10047, #9943, #1851, #1477, #8229, #8926, #8614, #10777, #8596, #10788, #9500, #9087, #10157, #10866, #10806, #10836, #10849, #10869, #10682, #10863, #10880, #10883, #10787, #8552, #10884, #7305, #5757, #9389, #8689, #10105, #8168, #9925, #10305, #4438, #9710, #10889, #10885, #10825, #10821, #10790, #10781, #9855, #9912, #10033, #9782, #10035, #9976, #10847, and #10865. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

1 0

HEADS UP: Need 7.10.3?
by Austin Seipp 17 Sep '15

17 Sep '15

Hi *, (This is an email primarily aimed at users reading this list and developers who have any interest). As some of you may know, there's currently a 7.10.3 milestone and status page on our wiki: https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.10.3 The basic summary is best captured on the above page: "We have not yet decided when, or even whether, to release GHC 7.10.3. We will do so if (but only if!) we have documented cases of "show-stoppers" in 7.10.2. Namely, cases from users where - You are unable to use 7.10.2 because of some bug - There is no reasonable workaround, so you are truly stuck - We know how to fix it - The fix is not too disruptive; i.e. does not risk introducing a raft of new bugs" That is, we're currently not fully sold on the need for a release. However, the milestone and issue page serve as a useful guide, and also make it easier to keep track of smaller, point-release worthy issues. So in the wake of the 8.0 roadmap I just sent: If you *need* 7.10.3 because the 7.10.x series has a major regression or problem you can't work around, let us know! - Find or file a bug in Trac - Make sure it's highest priority - Assign it to the 7.10.3 milestone - Follow up on this email if possible, or edit it on the status page text above - it would be nice to get some public feedback in one place about what everyone needs. Currently we have two bugs on the listed page in the 'show stopper category', possibly the same bug, which is a deal-breaker for HERMIT I believe. Knowing of anything else would be very useful. Thanks all! -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

5 5

Is it possible to build ghc with static linkage to its dependencies?
by Hengchu Zhang 17 Sep '15

17 Sep '15

Hi GHC Devs, I'd like to build ghc for an environment (linux x86_64) that I don't fully control, and most importantly libgmp.so on that environment is severely outdated. I was wondering if I can build ghc on another linux x86_64 machine but link in all the dependencies as static libraries instead finding them as dynamic libraries? Thank you! Best, Hengchu

2 1

utils/hp2ps/Main.c:88: possible missing break ?
by David Binderman 15 Sep '15

15 Sep '15

Hello there, [utils/hp2ps/Main.c:88] -> [utils/hp2ps/Main.c:91]: (warning) Variable 'iflag' is reassigned a value before the old one has been used. 'break;' missing? Source code is switch( *(*argv + 1) ) { case '-': iflag = -1; case '+': default: iflag = 1; } Suggest add missing break. Regards David Binderman

2 1

download all of Hackage?
by Richard Eisenberg 15 Sep '15

15 Sep '15

Hi devs, Is there an easy way to download (but not compile) all of Hackage? I know of the hackager package, but that's about compiling. I just want a whole big load of Haskell code to play with. I thought I could find a link on Hackage to do this, but failed. Thanks! Richard

9 10