- ghc-devs - Haskell.org

Re: Proposal: Improving the LLVM backend by packaging it
by George Colpitts 01 Nov '14

01 Nov '14

I think this is a great idea. My understanding from the wiki page is that the full plan involves: 1. We need to *fix compatibility with recent LLVM versions*. This really sucks for users. *Ben Gamari* is working on this, see #9142 <https://ghc.haskell.org/trac/ghc/ticket/9142> and Phab:D155 <https://phabricator.haskell.org/D155> 2. We need to fix up the LLVM IR generation in GHC, and fix the compiler driver to invoke the tools with a better set of optimizations. *Nathan Howell* has been wrangled into looking into this. 3. We need to ship a pre-built version of LLVM for developers and users, and put them in binary distributions for major platforms. *Austin Seipp* could handle this. 4. We can fix other bugs, like LLVM -split-obj support, which opens the way to shipping GHC with LLVM-optimized base libraries. *Reid Barton* expressed some interest in this. I assume that 1. will come with 7.10.1. Is there any way we could do 3. in one of the 7.10 releases before 7.12.1? Basing this question on Austin's words "we *only* need opt and llc, so the distributions can be tiny. " and "it's not actually very complicated to implement or support, I think." :) I guess it might be more difficult than I think but I thought it was worth asking the question as I believe 3. on its own is worthwhile and would hopefully simplify support for LLVM in GHC. Thanks George On Fri, Oct 24, 2014 at 8:52 PM, Austin Seipp <austin(a)well-typed.com> wrote: > Hi *, > > A few days ago a discussion on IRC occurred about the LLVM backend, > its current status, and what we could do to make it a rock solid part > of GHC for all our users. > > Needless to say, the situation right now isn't so hot: we have no > commitment to version support, two major versions are busted, others > are seriously buggy, and yet there are lots of things we could improve > on. > > So I give you a proposal, from a few of us to you all, about improving it: > > https://ghc.haskell.org/trac/ghc/wiki/ImprovedLLVMBackend > > I won't repeat what's on the wiki page too much, but the TL;DR version > is: we should start packaging a version of LLVM, and shipping it with > e.g. binary distributions of GHC. It's just a lot better for everyone. > > I know we're normally fairly hesitant about things like this (shipping > external dependencies), but I think it's the only sane thing to do > here, and the situation is fairly unique in that it's not actually > very complicated to implement or support, I think. > > We'd like to do this for 7.12. I've also wrangled some people to help. > Those people know who they are (because they're CC'd), and I will now > badger them into submission until it is fixed for 7.12. > > Please let me know what you think. > > PS. Joachim, I would be particularly interested in upstream needs for > Debian, as I know of their standard packaging policy to not duplicate > things. > > -- > Regards, > > Austin Seipp, Haskell Consultant > Well-Typed LLP, http://www.well-typed.com/ > _______________________________________________ > ghc-devs mailing list > ghc-devs(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs >

1 0

GHC Weekly News - 2014/10/31 (Halloween Edition)
by Austin Seipp 31 Oct '14

31 Oct '14

Hello *, Welcome to the GHC Weekly news. And it's just in time before you go out and eat lots of candy and food. * David Feuer and Joachim Brietner spent some time this past week talking about more optimizations for Haskell code for fusing code, and creating better consumers and producers. This work includes optimizations of "One shot lambdas" (lambdas used at most once) and Call Arity, which was implemented by Joachim at Microsoft Research. The thread is here - https://www.haskell.org/pipermail/ghc-devs/2014-October/006901.html The current situation is that Call Arity and One shot analysis tends to have good combined results for exploiting more fusion opportunities, but sometimes these backfire. As a result, Joachim and David have worked on improving the situation - particularly by letting programmers help with a new `oneShot` primitive (in Phab:D392 & Phab:D393). * Herbert Valerio Riedel opened up a discussion about the origins of code contributions. In short, we'd like to have some stronger ground to stand on in the face of contributions from contributors - the origin of a change and its legal status is a bit nebulous. The thread is here: https://www.haskell.org/pipermail/ghc-devs/2014-October/006959.html Overall, there's been a lot of separate points made, including CLAs (unlikely), "Developer Certificates of Origin" a la the Linux Kernel, and reworking how we mark up header files, and keep track of GHC's long history of authors. If you work on a big project where some of these concerns are real, we'd like to hear what you have to say! * Gintautas Milauskas has done some fantastic work for GHC on Windows lately, including fixing tests, improving the build, and making things a lot more reasonable to use. With his work, we hope GHC 7.10 will finally ship an updated MinGW compiler (a long requested feature), and have a lot of good fixes for windows. Thank you, Gintautas! * And on that note, the call for Windows developers rages on - it looks like Gintautaus, Austin, Simon, and others will be meeting to discuss the best way to tackle our current woes. Are you a Windows user? Please give us input - having input is a crucial part of the decision making process, so let us know. * Jan Stolarek had a question about core2core - a lot of questions, in fact. What's the difference between demand, strictness, and cardinality analylsis? Does the demand analyzer change things? And what's going on in some of the implementation? A good read if you're interested in deep GHC optimization magic: https://www.haskell.org/pipermail/ghc-devs/2014-October/006968.html * Peter Wortmann has put up the new DWARF generation patches for review, in Phab:D396. This is one of the major components we still plan on landing in 7.10, and with a few weeks to spare, it looks like we can make sure it's merged for the STABLE freeze! * There have been a lot of good changes in the tree this past week: Thanks to Michael Orlitzky, we plan on adding doctest examples to more modules in 7.10, and increase that coverage further. This is *really* important work, but very low fruit - thanks a ton Michael! `Data.Bifunctor` is now inside base! (Phab:D336) `atomicModifyIORef'` has been optimized with excellent speedups (as much as 1.7x to 1.4x, depending on the RTS used), thanks to some older work by Patrick Palka (Phab:D315). GHC's internals have been reworked to unwire `Integer` from GHC, leading not only to a code cleanup, but laying the foundation for further GMP (and non-GMP!) related `Integer` improvements (Phab:D351). David Feuer and Joachim have been relentless in improving fusion opportunities, including the performance of `take`, `isSuffixOf`, and more prelude improvements, spread over nearly half a dozen patches. And this doesn't even include the work on improving `oneShot` or Call Arity! In a slight change to `base` semantics, David Feuer also finally fixed #9236. This is a change that can expose latent bugs in your program (as it did for Haddock), so be sure to test thoroughly with 7.10 (Phab:D327). GHC now has support for a new `__GLASGOW_HASKELL_TH__` macro, primarily useful for testing bootstrap compilers, or compilers which don't support GHCi. And there have been many closed tickets: #9549, #9593, #9720, #9031, #8345, #9439, #9435, #8825, #9006, #9417, #9727, #2104, #9676, #2628, #9510, #9740, #9734, #9367, #9726, #7984, #9230, #9681, #9747, and #9236. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

1 0

cabal error: http://hackage.haskell.org/packages/archive/00-index.tar.gz : ErrorMisc "Error HTTP code: 502"
by George Colpitts 31 Oct '14

31 Oct '14

When I type cabal update I get http://hackage.haskell.org/packages/archive/00-index.tar.gz : ErrorMisc "Error HTTP code: 502" Anybody else seeing this? Thanks George

2 2

git question
by Simon Peyton Jones 31 Oct '14

31 Oct '14

Friends I am working on wip/new-flatten-skolems-Oct14. I have pushed lots of patches up to the main repo. Now I want to rebase to clean up. Can I just do a local rebase and then git push? Nothing special about the push? I know that will confuse anyone who is pulling from that branch, but I've warned them! Simon

3 7

Re: Understanding core2core optimisation pipeline
by Jan Stolarek 31 Oct '14

31 Oct '14

I'm resurrecting this 3-month old thread as I have some more questions about cardinality analysis. 1. I'm still a bit confused about terminology. Demand analysis, strictness analysis, cardinality analysis - do these three terms mean exactly the same thing? If no, then what are the differences? 2. First pass of full laziness is followed by floating in. At that stage we have not yet run the demand analysis and yet the code that does the floating-in checks whether a binder is one-shot (FloatIn.okToFloatInside called by FloatIn.fiExpr AnnLam case). This suggests that cardinality analysis is done earlier (but when?) and that demand analysis is not the same thing as cardinality analysis. 3. Does demand analyser perform any transformations? Or does it only annotate Core with demand information that can be used by subsequent passes? 4. BasicTypes module defines: data OneShotInfo = NoOneShotInfo -- ^ No information | ProbOneShot -- ^ The lambda is probably applied at most once | OneShotLam -- ^ The lambda is applied at most once. Do I understand correctly that `NoOneShotInfo` really means no information, ie. a binding annotated with this might in fact be one shot? If so, then do we have means of saying that a binding is certainly not a one-shot binding? 5. What is the purpose of SetLevels.lvlMFE function? Janek > The wiki page just went live: > > https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/Core2CorePipeline > > It's not yet perfect but it should be a good start. > > > Roughtly, a complete run of the simplifier means "run the simplifier > > repeatedly until nothing further happens". The iterations are the > > successive iterations of this loop. Currently there's a (rather arbitrary) > > limit of four such iterations before we give up and declare victory. > > A limit or a default value for that limit? > > To Ilya: > > If you grep for the "late_dmd_anal" option variable in the compiler/simplCore/SimplCore.lhs > > module, you'll see that it triggers a phase close to the endo of getCoreToDo's tasks, which > > contains, in particular, the "CoreDoStrictness" pass. This is the "late" phase. > > The paper said that the late pass is run to detect single-entry thunks and the reason why it is > run late in the pipeline is that if it were run earlier this information could be invalidated > by the transformations. But in the source code I see that this late pass is followed by the > simplifier, which can invalidate the information. Also, the documentation for -flate-dmd-anal > says: "We found some opportunities for discovering strictness that were not visible earlier; > and optimisations like -fspec-constr can create functions with unused arguments which are > eliminated by late demand analysis". This says nothing about single-netry thunks. So, is the > single-entry thunk optimisation performed by GHC? > > Janek

3 4

RFC: Properly stated origin of code contributions
by Herbert Valerio Riedel 31 Oct '14

31 Oct '14

Hi, GHC's Git history has (mostly) a good track record of having properly attributed authorship information in the recent past; Some time ago I've even augmented the .mailmap file to fix-up some of the pre-Git meta-data which had mangled author/committer meta-data (try 'git shortlog -sn' if you're curious) However, I just noticed that http://git.haskell.org/ghc.git/commitdiff/322810e32cb18d7749e255937437ff2ef… landed recently, which did change a significant amount of code, but at the same time the author looks like a pseudonym to me (and apologies if I'm wrong). Other important projects such as Linux or Samba, just to name two examples, reject contributions w/o a clearly stated origin, and explicitly reject anonymous/pseudonym contributions (as part of their "Developer's Certificate of Origin" policy[1] which involves a bit more than merely stating the real name) I believe the GHC project should consider setting some reasonable ground-rules for contributions to be on the safe side in order to avoid potential copyright (or similiar) issues in the future, as well as giving confidence to commercial users that precautions are taken to avoid such issues. Comments? Cheers, hvr [1]: See http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documen…

11 16

Re: Understanding core2core optimisation pipeline
by David Feuer 31 Oct '14

31 Oct '14

On Thu, Oct 30, 2014 Jan Stolarek wrote: > > 2. First pass of full laziness is followed by floating in. At that stage > we have not yet run the > demand analysis and yet the code that does the floating-in checks whether > a binder is one-shot > (FloatIn.okToFloatInside called by FloatIn.fiExpr AnnLam case). This > suggests that cardinality > analysis is done earlier (but when?) and that demand analysis is not the > same thing as > cardinality analysis. > If you're looking at super-recent code, that could be Joachim Breitner's work. He's exposed the one-shot stuff at the Haskell level with the experimental magic oneShot function, intended primarily for use in the libraries to make foldl-as-foldr and related things be analyzed more reliably. The old GHC arity analysis combined with his Call Arity get almost everything right, but there are occasional corner cases where things go wrong, and when they do the results tend to be extremely bad.

5 5

RE: package hashes
by Simon Peyton Jones 31 Oct '14

31 Oct '14

No, I did not. You can try it yourself! | -----Original Message----- | From: Edward Z. Yang [mailto:ezyang@cs.stanford.edu] | Sent: 30 October 2014 19:50 | To: Simon Peyton Jones | Cc: ghc-devs | Subject: Re: package hashes | | Hmm. I'm not sure, but the right thing might just be to accept this | new output. But I guess I'm a bit confused, since you probably didn't | change this test-case at all, did you? | | Edward | | Excerpts from Simon Peyton Jones's message of 2014-10-30 08:26:45 - | 0700: | > Edward | > | > On branch wip/new-flatten-skolems-Oct14, I'm getting this test | failure repeatably on test safePkgO1. I'm pretty sure I have done | nothing to mess with package hashes! | > | > Any ideas? | > | > Simon | > | > =====> safePkg01(normal) 108 of 120 [0, 0, 0] | > cd ./check/pkg01 && $MAKE -s --no-print-directory safePkg01 | VANILLA=--enable-library-vanilla PROF=--disable-library-profiling | DYN=--enable-shared </dev/null >safePkg01.run.stdout | 2>safePkg01.run.stderr | > Actual stdout output differs from expected: | > --- ./check/pkg01/safePkg01.stdout 2014-10-29 15:09:16.000000000 | +0000 | > +++ ./check/pkg01/safePkg01.run.stdout 2014-10-30 | 15:25:17.799094762 +0000 | > @@ -29,17 +29,17 @@ | > require own pkg trusted: True | > M_SafePkg6 | > -package dependencies: array-0.5.0.1@array_<HASH> | > +package dependencies: array-0.5.0.1 base-4.8.0.0* | > trusted: trustworthy | > require own pkg trusted: False | > M_SafePkg7 | > -package dependencies: array-0.5.0.1@array_<HASH> | > +package dependencies: array-0.5.0.1 base-4.8.0.0* | > trusted: safe | > require own pkg trusted: False | > M_SafePkg8 | > -package dependencies: array-0.5.0.1@array_<HASH> | > +package dependencies: array-0.5.0.1 base-4.8.0.0 | > trusted: trustworthy | > require own pkg trusted: False

1 0

cabal sdist trouble with GHC from head
by Sven Panne 31 Oct '14

31 Oct '14

Does anybody have a clue what's going wrong at the sdist step here? https://travis-ci.org/haskell-opengl/OpenGLRaw/jobs/38707011#L104 This only happens with a GHC from head, a build with GHC 7.8.3 is fine: https://travis-ci.org/haskell-opengl/OpenGLRaw/jobs/38707010 Any help highly appreciated... Cheers, S.

4 7

URL issue
by Simon Peyton Jones 30 Oct '14

30 Oct '14

Herbert, Austin I was in a fairly old repo, but one I have been using daily, and pushing to the central repo. The pushurl was pushurl = ssh://git@ghc.haskell.org/ghc.git That worked two days ago, but silently hangs now. But when I finally realised that it should be "git.haskell.org" it works fine. So something must have changed. I'm certain I was pushing to ghc.haskell.org until a couple of days ago. Strange, and may be useful knowledge for others. Anyway, no problem now. But the "Repositories" pages says nothing about what URL to use for pushing - and it really should! That would be worth fixing. Thanks Simon

2 1