January 2016 - ghc-devs

Improving trac notifications
by Matthew Pickering 12 Feb '17

12 Feb '17

I subscribe to the ghc-tickets[1] mailing list which provides updates to all trac tickets. This is very useful, however when a ticket description is updated the whole description is included in the email which makes it hard to see what actually changed. The web interface shows a nice diff[2] of the changes, I think it would be good if emails could also include a diff rather than the current quite useless output. After a bit of investigation, it appears that the easiest way to achieve this is apply a simple patch to our copy of trac. The current format is hard coded on line 558 in this module, it seems easy to modify this section to instead provide a diff. http://www.edgewall.org/docs/trac-trunk/epydoc/trac.ticket.notification-pys… There is also already a module which has the diff operations implemented which could be used. http://www.edgewall.org/docs/trac-trunk/epydoc/trac.versioncontrol.diff-mod… Another option would be to use something like the `AnnouncerPlugin`[3] but that seems like much more work and would still require someone to write a custom formatter. Any thoughts? Matt [1]: https://mail.haskell.org/mailman/listinfo/ghc-tickets [2]: https://ghc.haskell.org/trac/ghc/ticket/10873?action=diff&version=6 [3]: https://trac-hacks.org/wiki/AnnouncerPlugin

3 8

RE: vectorisation code?
by Simon Peyton Jones 12 Sep '16

12 Sep '16

Austin, (or anyone else) Manuel says: | > Would it be ok if we left it in the repo, but CPP'd it out so that | we | > didn't compile everything? (The DPH library is in the same state at | > the moment.) | > | > It might suffer bit-rot, but it’d still be there for resurrection. | | Sure, that’s ok. Could you action this? Just avoid compiling anything in 'vectorise/', using (I suppose) cpp to create a stub where necessary. Leave enough comments to explain! Simon | | I hope everything is fine in Cambridge! | Manuel | | > | -----Original Message----- | > | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | > | Manuel M T Chakravarty | > | Sent: 16 January 2015 02:58 | > | To: Richard Eisenberg | > | Cc: ghc-devs(a)haskell.org Devs | > | Subject: Re: vectorisation code? | > | | > | [Sorry, sent from the wrong account at first.] | > | | > | We currently don’t have the resources to work on DPH. I would | > | obviously prefer to leave the code in, in the hope that we will be | > | able to return to it. | > | | > | Manuel | > | | > | > Richard Eisenberg <eir(a)cis.upenn.edu>: | > | > | > | > Hi devs, | > | > | > | > There's a sizable number of modules in the `vectorise` | > | subdirectory of GHC. I'm sure these do all sorts of wonderful | > | things. But what, exactly? And, does anyone make use of these | wonderful things? | > | > | > | > A quick poking through the code shows a tiny link between the | > | vectorise code and the rest of GHC -- the function `vectorise` | > | exported from the module `Vectorise`, which is named in exactly | one | > | place from SimplCore. From what I can tell, the function will be | > | called only when `-fvectorise` is specified, and then it seems to | > | interact with a {-# VECTORISE #-} pragma. However, `{-# VECTORISE | > | #-}` doesn't appear in the manual at all, and `-fvectorise` is | > | given only a cursory explanation. It seems these work with DPH... | > | which has been disabled, no? Searching online finds several hits, | > | but nothing more recent than 2012. | > | > | > | > I hope this question doesn't offend -- it seems that | > | vectorisation probably has amazing performance gains. Yet, the | > | feature also seems unloved. In the meantime, compiling (and | > | recompiling, and | > | recompiling...) the modules takes time, as does going through | them | > | to propagate changes from elsewhere. If this feature is truly | > | orphaned, unloved, and unused at the moment, is it reasonable to | > | consider putting it on furlough? | > | > | > | > Thanks, | > | > Richard | > | > _______________________________________________ | > | > ghc-devs mailing list | > | > ghc-devs(a)haskell.org | > | > http://www.haskell.org/mailman/listinfo/ghc-devs | > | | > | _______________________________________________ | > | ghc-devs mailing list | > | ghc-devs(a)haskell.org | > | http://www.haskell.org/mailman/listinfo/ghc-devs

13 35

Re: Questions concerning the LLVM backend - i.e. 'proc point splitting'
by David Spitzenberg 04 Mar '16

04 Mar '16

Simon > | Could you explain the need of further info-tables for 'inner' > | proc-points (those, which are not the entry-block of a function) to > | me, please? > > The main purpose of inner info tables is this. Given (case e of blah) we push a return address (for 'blah') on the stack and evaluate 'e'. During evaluation of 'e', garbage collection may take place. So the garbage collector needs to walk the stack to follow live pointers. Who knows what pointers are live? Answer: the code for 'blah' clearly knows what pointers are alive in the current stack frame, because it is going to access them. So the return address gets an info table that makes that implicit knowledge explicit, as a bitmap describing the liveness/pointer-hood of the stack frame. > > Addresses that are only jumped to don’t need info tables. > > It would be great if, as you learn about this stuff, you could add to the Commentary > https://ghc.haskell.org/trac/ghc/wiki/Commentary > to capture what you have learned. Thank you very much! In cases I find the current description imprecise or incomplete I can add what I've learned to the Commentary of course. > | The documentation states, that proc-point splitting is only required > | by the LLVM backend (see [2]). Does the cmm-pipeline perform proc- > | point splitting even when using the native code generator? > > I'm not certain, but I don't think so. > > | As stated above, the major problem of LLVM seems to be inferring > | control flow when it comes to call proc-points. As it's hard to avoid > | proc-point splitting, we leave the cmm-pipeline 'as-is'. We modify the > > I regard the whole proc-point thing as a major wart. I think we should work hard to eliminate it. So: another line of thought might be: how could we influence LLVM so that we didn't need proc-point splitting? I totally agree with you here. However, determining how LLVM needs to be influenced to eliminate proc-point splitting is a thing I won't be capable of doing mid term because of my current knowledge and lack of time. > | To make control flow more apparent to LLVM, they came up with the > | following idea. Remark before showing the idea: I know that it greatly > > I'm sorry to say that I didn't really understand the idea. Maybe someone else did? David Terei? My apologies for being imprecise. I hope to find the time to work out a little demonstration during the upcoming holidays. I'm currently fully occupied by university because of upcoming exams this week. > If you are seriously about investing effort in the back end, I'd be happy to help; in that case a Skype call is probably the best way. That is a great idea. Due to the above mentioned time constraints I propose that we follow up in January. Regards, David > Simon > > | interferes with CPS. It leaves parts of the continuation-handling to > | the implicit LLVM call-stack. Further, this does not solve the problem > | of proc-point splitting at all. It is more a workaround than a > | solution. > | But it would greatly increase LLVM's possibilities to optimize code > | later on. That's why I found this idea worth mentioning. Especially > | because after reading your answer, Ben, I'm somewhat convinced that > | solving the problem of proc-point splitting is nearly impossible with > | the capabilities of LLVM IR available at the moment. Now back to the > | idea. > | > | As stated above, the major problem of LLVM seems to be inferring > | control flow when it comes to call proc-points. As it's hard to avoid > | proc-point splitting, we leave the cmm-pipeline 'as-is'. We modify the > | code, llvmGen produces when it comes to call proc-points instead. All > | other types of proc-points remain unchanged. > | > | Initial situation in pseudo cmm: > | > | foo() { > | ... > | call bar() returns to cont // bar defined externally > | > | cont: > | ... > | } > | > | After proc-point splitting, this is converted into LLVM IR similar to > | the following: > | > | @foo() { > | ... > | store @cont(), %Sp ; push continuation on the stack tail call @bar() ; > | @bar defined externally } > | > | @cont() { > | ... > | } > | > | To fix the above issue, we introduce a method, which restores the > | pointers Sp, Hp and the register R1 (among everything else, what has > | to be restored) and 'fakes' the continuation. Note, that '@restore' > | returns without calling into the continuation. This way, the call into > | the continuation can be made direct. The above code would be > | transformed into something similar to the following: > | > | @foo() { > | ... > | store @restore(), %Sp ; 'fake' continuation call @bar() ; br label > | restore > | > | restore: > | %Sp = load (getelementptr @environment 0, 0) ; restore what has to be > | ... ; restored tail call @cont() } > | > | @cont() { > | ... > | } > | > | @restore(%Sp, %Hp, %R1, ...) { > | store %Sp, (getelementptr @environment 0, 0) ; store everything into > | ... ; global variable ret void } > | > | @environment ; structure of type similar to {i64*, i64*, i64*, i64, > | i64, ..} > | > | As an alternative, the above transformation could be done by a LLVM > | pass. llvmGen could then annotate the continuation of calls call cite. > | This would greatly simplify the detection of the correct continuation. > | > | What do you think about this idea? Especially when thinking about the > | interference with CPS? > | > | Regards, > | > | David > | > | > | > | > | [1] > | https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/GeneratedCod > | e#Importantconceptsinthemachine > | > | [2] > | https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/CodeGen#Seco > | ndstage:theCmmpipeline > | > | > | > | > | On 11/21/2015 12:00 AM, Ben Gamari wrote: > | > Simon Peyton Jones <simonpj(a)microsoft.com> writes: > | > > | >> David > | >> > | >> All this stuff is terribly paged-out for me. But I'd love someone > | to > | >> pay attention to some of this back-end stuff, so if you'd like to > | >> work there, I'd like to help you. > | >> > | >> David Terei was also thinking of getting rid of proc point > | splitting > | >> for LLVM (see attached email and the notes attached to it) > | >> > | >> Simon Marlow also was thinking about this; see his attached email. > | >> > | >> The main *reason* for establishing a proc-point is so that, when > | >> generating C code (which we don't do any more) you can take its > | >> address. E.g. > | >> > | >> foo() { ... Push &bar onto the stack (effectively a return address) > | >> Jump to thingumpy } > | >> > | >> bar() { ... } > | >> > | >> Really bar is part of foo, but we have make it a separate top level > | >> thing so that we can take the address of bar and push it on the > | stack. > | >> > | >> The trouble is that if there is a label inside bar that foo wants > | to > | >> jump to (i.e. without pushing &bar etc) then we have to make that > | >> label too into a proc-point, so that both foo and bar can see it. > | >> Urgh. > | >> > | >> In LLVM this probably is not true; presumably you can take the > | >> address of any label? > | >> > | > This is true. While labels themselves have function-local scope in > | > LLVM, there is an expression, `blockaddress`, which allows you to > | take > | > an address to a label in another function [1]. That being said, in > | > reading through the documentation it's not at all clear to me that > | it > | > would be safe to jump to such an address. In fact, given that the > | > instruction that this expression is supposed to be used with, > | > `indirectbr`, can only be used for local blocks, I suspect it is > | not. > | > More information about this feature can be found here [2]. > | > > | > The jump issue aside, I don't know how you would deal with > | > tables-next-to-code. The prefix data support that currently > | available > | > in LLVM is attached to functions and I unfortunately don't see that > | > changing any time soon. > | > > | > Ultimately it seems that trying to refer to labels defined in other > | > functions is using LLVM against the way it was intended. One > | > alternative would be to teach llvmGen to merge mutually recusive > | > top-level functions into a single LLVM function during code > | > generation. Otherwise I'm afraid you are stuck with either the > | status > | > quo or attempting to improve on LLVM's own cross-procedure > | optimization ability. > | > > | > That being said, it sounds as though eliminating proc-point > | splitting > | > would make for quite a nice project in the native code generator. > | > > | > Cheers, > | > > | > - Ben > | > > | > > | > [1] > | > > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fllvm.o > | > rg%2fdocs%2fLangRef.html%23addresses-of-basic- > | blocks&data=01%7c01%7csi > | > > | monpj%40064d.mgd.microsoft.com%7c7fdbfda0b4514a098c4d08d2f74448f1%7c72 > | > > | f988bf86f141af91ab2d7cd011db47%7c1&sdata=nKVLgmZezUdEYL%2fma1exMMPLbcT > | > RkaE1JrvVaR1EesY%3d [2] > | > > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fblog.l > | > lvm.org%2f2010%2f01%2faddress-of-label-and-indirect- > | branches.html&data > | > > | =01%7c01%7csimonpj%40064d.mgd.microsoft.com%7c7fdbfda0b4514a098c4d08d2 > | > > | f74448f1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BTMdZESwYS4tZa5W > | > sOAqyFutoI5xNNFDe0b%2bdKC3ODs%3d [3] > | > > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fllvm.o > | > rg%2fdocs%2fLangRef.html%23prefix- > | data&data=01%7c01%7csimonpj%40064d.m > | > > | gd.microsoft.com%7c7fdbfda0b4514a098c4d08d2f74448f1%7c72f988bf86f141af > | > > | 91ab2d7cd011db47%7c1&sdata=rdo4ZLjjE%2bWxNykVL%2bSuToWioY6nzQpflSk296E > | > 8W70%3d > | > >

3 4

"Excuse me, I think this i my stop..." - Resigning from the Platform
by Mark Lentczner 22 Feb '16

22 Feb '16

I think this is the right time for me to exit: The truth is, I still can't bring myself to use a version of Haskell post the Foldable-Traversable-aPocalypse, let alone some future Haskell after the changes now in the works. My personal machines are all still 7.8. My personal projects are all pre-FTP. The Haskell I love to code in, the Haskell I'm passionate about, the Haskell I've advocated for real world use, and the Haskell I like to teach, is 7.8, pre-FTP. It's not that I'm dead set against change and evolution in a language, or even breaking changes. But FTP and beyond are changes that have lost the balance that Haskell had between abstraction and clarity, between elaborate and practical engineering. I don't see any restraint going forward, so I'm getting off the train. This puts me in an odd position with respect to Haskell Platform: I find myself building the Platform for a version of Haskell that I don't use. This isn't healthy for either the Platform or me. Hence, I'm resigning as release manager. I am sad because I believed that Haskell's path to much wider adoption was within reach. Now, especially with the ramping up of the Haskell Prime committee, which seems preordained to codify FTP and beyond into standard, we are approaching our Algol 68 moment: Building a major language revision with less opportunity than it's predecessor. I'll still see you 'round at meet-ups and conferences. I'll just be coding with an older accent. - Mark "mzero" Lentczner

7 8

Fwd: Compiling Core Haskell using GHC API
by Marek Wawrzos 20 Feb '16

20 Feb '16

Hello, I'm trying to compile some module with GHC API. I'm going to feed GHC with Core generated by me. So far is noticed, that function compileCoreToObj ends with an runtime error on each call. In spide of this function produces some *.o and *.hi files. However, produced interface file does not contain any item in exports field, what makes compiled module unusable. Because of this problem, I tried to achieve my goal by using steps described here: https://wiki.haskell.org/GHC/As_a_library (the second example) I was able to print out Core produced form file. If I'm not wrong, modifying structures passing on between sequent calls does not effect on produced *.hi and *.o files. Both files are generated by `load LoadAllTargets' call, and further calls (parseModule, loadModule, ...) does not effect on generated files. Could You give me any advices in that topic? Am I right about compileCoreToObj and functions mentioned above? Best Regards, Marek

2 3

GHC build time graphs
by Edward Z. Yang 13 Feb '16

13 Feb '16

Joachim, Anthony Cowley was recently complaining on Twitter that building GHC takes 39.5% longer now than it did for 7.10.1: https://twitter.com/a_cowley/status/692481516744634368 Do we have graphs for these metrics, e.g. can we see the breakdown of which commits increased the build time? Thanks, Edward

6 9

Specializing expressions beyond names?
by Conal Elliott 04 Feb '16

04 Feb '16

It seems to be the case that SPECIALIZE pragmas are syntactically restricted to type specializations of a *name* (identifier) rather than a general expression. Is my understanding correct here? If so, is there any reason for this restriction? I ask because I’m reifying Core code (into code that constructs a corresponding run-time representation for further processing), and I’m looking for a clean way to integrate that process with GHC, to support separate compilation and to avoid interfering with GHC’s regular flow. It occurred to me that I could enable separate compilation via a pragma of the form “{-# SPECIALIZE reify foo ∷ E t #-}” for some t, where E t is a reified form of values of type t. Type checking would infer the specialized type of foo, and the usual specialization phase would do its usual thing on that specialization, leaving “reify foo = reify specialized_foo”, and then the reification compiler plugin would transform the right-hand side, pushing the reify inward. Some reify calls may remain (e.g., due to polymorphism), triggering future rule applications. As much as possible of the fully-reified version would be factored out of the generated rule’s RHS for cheap reuse. Thanks, - Conal

2 3

CallStack naming
by Richard Eisenberg 03 Feb '16

03 Feb '16

Hi devs, I'm sure there's an easy answer to this, but I'm wondering: why is the CallStack feature implemented with implicit parameters instead of just a magical constraint? Whenever I use this feature, I don't want to have to enable -XImplicitParams and then make sure I get the name right. What would be wrong with, e.g., > undefined :: AppendsCallStack => a Seems simpler. Is it problems with a nullary class? Thanks, Richard

6 18

Re: Adding a "release" setting in build.mk.sample (and some other build system questions)
by Harry . 03 Feb '16

03 Feb '16

> https://ghc.haskell.org/trac/ghc/wiki/MakingReleases#Makingthebinarybuilds > https://ghc.haskell.org/trac/ghc/wiki/Building/Using#Buildconfiguration So the mysterious SRC_HC_OPTS = -O -H64m which appears in every build flavour isn't used for the release build?

3 3

Best practices for merging?
by Geoffrey Mainland 02 Feb '16

02 Feb '16

I've been away from GHC for a while, and it's not clear to me what the best practices for merging are now. My usual git workflow is to work on a feature branch, get a nice clean set of patches, each of which implements a discrete bit of functionality, rebase onto master, and then merge with an empty merge commit, i..e, *not* fast-forward. This lets me look at the repo history and easily see where a feature branch landed and get a good idea of what changes it contains. I have used this approach in the past with GHC, e.g., SIMD support and typed TH. For smaller sets of changes, I might skip the merge commit, but I still break up my changes into discreet bits, which helps anyone reading the change set to see what's going on. It seems the current policy is to submit changes via Phab and then, after approval, squash everything into a single patch and apply it with arc. At least that is my reading of https://ghc.haskell.org/trac/ghc/wiki/Phabricator/Extras#Landingreviews. Is that what I should be doing? For feature branches, is that also what is advised? Thanks, Geoff

10 19