October 2015 - ghc-devs

RE: vectorisation code?
by Simon Peyton Jones 12 Sep '16

12 Sep '16

Austin, (or anyone else) Manuel says: | > Would it be ok if we left it in the repo, but CPP'd it out so that | we | > didn't compile everything? (The DPH library is in the same state at | > the moment.) | > | > It might suffer bit-rot, but it’d still be there for resurrection. | | Sure, that’s ok. Could you action this? Just avoid compiling anything in 'vectorise/', using (I suppose) cpp to create a stub where necessary. Leave enough comments to explain! Simon | | I hope everything is fine in Cambridge! | Manuel | | > | -----Original Message----- | > | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | > | Manuel M T Chakravarty | > | Sent: 16 January 2015 02:58 | > | To: Richard Eisenberg | > | Cc: ghc-devs(a)haskell.org Devs | > | Subject: Re: vectorisation code? | > | | > | [Sorry, sent from the wrong account at first.] | > | | > | We currently don’t have the resources to work on DPH. I would | > | obviously prefer to leave the code in, in the hope that we will be | > | able to return to it. | > | | > | Manuel | > | | > | > Richard Eisenberg <eir(a)cis.upenn.edu>: | > | > | > | > Hi devs, | > | > | > | > There's a sizable number of modules in the `vectorise` | > | subdirectory of GHC. I'm sure these do all sorts of wonderful | > | things. But what, exactly? And, does anyone make use of these | wonderful things? | > | > | > | > A quick poking through the code shows a tiny link between the | > | vectorise code and the rest of GHC -- the function `vectorise` | > | exported from the module `Vectorise`, which is named in exactly | one | > | place from SimplCore. From what I can tell, the function will be | > | called only when `-fvectorise` is specified, and then it seems to | > | interact with a {-# VECTORISE #-} pragma. However, `{-# VECTORISE | > | #-}` doesn't appear in the manual at all, and `-fvectorise` is | > | given only a cursory explanation. It seems these work with DPH... | > | which has been disabled, no? Searching online finds several hits, | > | but nothing more recent than 2012. | > | > | > | > I hope this question doesn't offend -- it seems that | > | vectorisation probably has amazing performance gains. Yet, the | > | feature also seems unloved. In the meantime, compiling (and | > | recompiling, and | > | recompiling...) the modules takes time, as does going through | them | > | to propagate changes from elsewhere. If this feature is truly | > | orphaned, unloved, and unused at the moment, is it reasonable to | > | consider putting it on furlough? | > | > | > | > Thanks, | > | > Richard | > | > _______________________________________________ | > | > ghc-devs mailing list | > | > ghc-devs(a)haskell.org | > | > http://www.haskell.org/mailman/listinfo/ghc-devs | > | | > | _______________________________________________ | > | ghc-devs mailing list | > | ghc-devs(a)haskell.org | > | http://www.haskell.org/mailman/listinfo/ghc-devs

13 35

"Excuse me, I think this i my stop..." - Resigning from the Platform
by Mark Lentczner 22 Feb '16

22 Feb '16

I think this is the right time for me to exit: The truth is, I still can't bring myself to use a version of Haskell post the Foldable-Traversable-aPocalypse, let alone some future Haskell after the changes now in the works. My personal machines are all still 7.8. My personal projects are all pre-FTP. The Haskell I love to code in, the Haskell I'm passionate about, the Haskell I've advocated for real world use, and the Haskell I like to teach, is 7.8, pre-FTP. It's not that I'm dead set against change and evolution in a language, or even breaking changes. But FTP and beyond are changes that have lost the balance that Haskell had between abstraction and clarity, between elaborate and practical engineering. I don't see any restraint going forward, so I'm getting off the train. This puts me in an odd position with respect to Haskell Platform: I find myself building the Platform for a version of Haskell that I don't use. This isn't healthy for either the Platform or me. Hence, I'm resigning as release manager. I am sad because I believed that Haskell's path to much wider adoption was within reach. Now, especially with the ramping up of the Haskell Prime committee, which seems preordained to codify FTP and beyond into standard, we are approaching our Algol 68 moment: Building a major language revision with less opportunity than it's predecessor. I'll still see you 'round at meet-ups and conferences. I'll just be coding with an older accent. - Mark "mzero" Lentczner

7 8

Fwd: Compiling Core Haskell using GHC API
by Marek Wawrzos 20 Feb '16

20 Feb '16

Hello, I'm trying to compile some module with GHC API. I'm going to feed GHC with Core generated by me. So far is noticed, that function compileCoreToObj ends with an runtime error on each call. In spide of this function produces some *.o and *.hi files. However, produced interface file does not contain any item in exports field, what makes compiled module unusable. Because of this problem, I tried to achieve my goal by using steps described here: https://wiki.haskell.org/GHC/As_a_library (the second example) I was able to print out Core produced form file. If I'm not wrong, modifying structures passing on between sequent calls does not effect on produced *.hi and *.o files. Both files are generated by `load LoadAllTargets' call, and further calls (parseModule, loadModule, ...) does not effect on generated files. Could You give me any advices in that topic? Am I right about compileCoreToObj and functions mentioned above? Best Regards, Marek

2 3

Unlifted data types
by Edward Z. Yang 29 Dec '15

29 Dec '15

Hello friends, After many discussions and beers at ICFP, I've written up my current best understanding of the unlifted data types proposal: https://ghc.haskell.org/trac/ghc/wiki/UnliftedDataTypes Many thanks to Richard, Iavor, Ryan, Simon, Duncan, George, Paul, Edward Kmett, and any others who I may have forgotten for crystallizing this proposal. Cheers, Edward

12 67

arc land changes author
by Joachim Breitner 19 Dec '15

19 Dec '15

Hi, I was just about to apply a DR by David. First I ran $ arc patch D390 which put me on a feature branch. Then I reworded the commit with $ git commit --amend and then I tried to land it with $ arc land Luckily I also passed --hold... It even asked me if I want to land the commit although I am not the author: This branch has revision 'D390: Reorder GHC.List; fix performance regressions' but you are not the author. Land this revision by dfeuer? [y/N] y But it still put me in the author field: $ git show HEAD commit a387c359b99667ce55016de3816b5d9873a155db Author: Joachim Breitner <mail(a)joachim-breitner.de> Date: Wed Oct 29 08:17:05 2014 +0100 Reorder GHC.List; fix performance regressions Summary: Rearrange some oddly placed code. (and undid my commit message changes....) Did I do something wrong? Anyways, it said something along the lines of Cleaning up feature branch... (Use `git checkout -b 'arcpatch-D390' 'a3ceb0e7f44d53f728fb8bd0dfb4c97297818029'` if you want it back.) so with $ git commit --amend -C a3ceb0e7f44d53f728fb8bd0dfb4c97297818029 I was able to get the edited commit message with the right author back. Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata(a)debian.org

2 2

A GHC Library
by Richard Eisenberg 08 Dec '15

08 Dec '15

Hi devs, It has struck me that we have a vast trove of documentation about GHC in the form of academic papers, but little organization to this trove. The papers are scattered across authors' websites, and sometimes it is hard to remember what a paper is titled when searching for it. I have thus started a wiki page here: https://ghc.haskell.org/trac/ghc/wiki/Library I would love for this page to be the Definitive List of GHC-related papers. The list I've included is, of course, woefully short, but it is a starting point. Please add to it! Have you written a GHC paper? If so, make sure it's included. Thanks, Richard

6 12

Re: Proposal: accept pull requests on GitHub
by Thomas Miedema 20 Nov '15

20 Nov '15

> > The real hint is that "the number of contributions will go up". That's > a noble goal and I think it's at the heart of this proposal. > It's not. What's at the heart of my proposal is that `arc` sucks. Most of those quotes I posted are from regular contributors (here's another one: "arcanist kinda makes stuff even more confusing than Git by itself"). Newcomers will give it their best shot, thinking it's just another thing they need to learn, thinking it's their fault for running into problems, thinking they'll get the hang of it eventually. Except they won't, or at least I haven't, after using it for over a year. Maybe the fundamental problem with Phabricator is that it doesn't understand Git well, and the problems I posted on https://ghc.haskell.org/trac/ghc/wiki/WhyNotPhabricator are just symptoms of it. I'm having trouble putting this into words though (something about branches and submodules). Perhaps someone else can? In my opinion it's is a waste of our time trying to improve `arc` (it is 34000 lines of PHP btw + another 70000 LOC for libphutil), when `pull requests` are an obvious alternative that most of the Haskell community already uses. When you're going to require contributors to use a non-standard tool to get patches to your code review system, it better just work. `arc` is clearly failing us here, and I'm saying enough is enough. I need to think about your other points. Thank you for the thorough reply. > Here's the meat of it question: what is the cost of achieving this > goal? That is, what amount of work is sufficient to make this goal > realizable, and finally - why is GitHub *the best use of our time for > achieving this?* That's one aspect of the cost - that it's the best > use of the time. I feel like this is fundamentally why I always seem > to never 'get' this argument, and I'm sure it's very frustrating on > behalf of the people who have talked to me about it and like GitHub. > But I feel like I've never gotten a straight answer for GHC. > > If the goal is actually "make more people contribute", that's pretty > broad. I can make that very easy: give everyone who ever submits a > patch push access. This is a legitimate way to run large projects that > has worked. People will almost certainly be more willing to commit, > especially when overhead on patch submission is reduced so much. Why > not just do that instead? It's not like we even mandate code review, > although we could. You could reasonably trust CI to catch and revert > things a lot of the time for people who commit directly to master. We > all do it sometimes. > > I'm being serious about this. I can start doing that tomorrow because > the *cost is low*, both now and reasonably speaking into some > foreseeable future. It is one of many solutions to raw heart of the > proposal. GitHub is not a low cost move, but also, it is a *long term > cost* because of the technical deficiencies it won't aim to address > (merge commits are ugly, branch reviews are weak, ticket/PR namespace > overlaps with Trac, etc etc) or that we'll have to work around. > > That means that if we want GitHub to fix the "give us more > contributors" problem, and it has a high cost, it not only has _to fix > the problem_, it also has to do that well enough to offset its cost. I > don't think it's clear that is the case right now, among a lot of > other solutions. > > I don't think the root issue is "We _need_ GitHub to get more > contributors". It sounds like the complaint is more "I don't like how > Phabricator works right now". That's an important distinction, because > the latter is not only more specific, it's more actionable: > > - Things like Arcanist can be tracked as a Git submodule. There is > little to no pain in this, it's low cost, and it can always be > synchronized with Phabricator. This eliminates the "Must clone > arcanist" and "need to upgrade arcanist" points. > > - Similarly when Phabricator sometimes kills a lot of builds, it's > because I do an upgrade. That's mostly an error on my part and I can > simply schedule upgrades regularly, barring hotfixes or somesuch. That > should basically eliminate these. The other build issues are from > picking the wrong base commit from the revision, I think, which I > believe should be fixable upstream (I need to get a solid example of > one that isn't a mega ultra patch.) > > - If Harbormaster is not building dependent patches as mentioned in > WhyNotPhabricator, that is a bug, and I have not been aware of it. > Please make me aware of it so I can file bugs! I seriously don't look > at _every_ patch, I need to know this. That could have probably been > fixed ASAP otherwise. > > - We can get rid of the awkwardness of squashes etc by using > Phabricator's "immutable" history, although it introduces merge > commits. Whether this is acceptable is up to debate (I dislike merge > commits, but could live with it). > > - I do not understand point #3, about answering questions. Here's > the reality: every single one of those cases is *almost always an > error*. That's not a joke. Forgetting to commit a file, amending > changes in the working tree, and specifying a reviewer are all total > errors as it stands today. Why is this a minus? It catches a useful > class of 'interaction bugs'. If it's because sometimes Phabricator > yells about build arifacts in the tree, those should be .gitignore'd. > If it's because you have to 'git stash' sometimes, this is fairly > trivial IMO. Finally, specifying reviewers IS inconvenient, but > currently needed. We could easily assign a '#reviewers' tag that would > add default reviewers. > - In the future, Phabricator will hopefully be able to > automatically assign the right reviewers to every single incoming > patch, based on the source file paths in the tree, using the Owners > tool. Technically, we could do that today if we wanted, it's just a > little more effort to add more Herald rules. This will be far, far > more robust than anything GitHub can offer, and eliminates point #3. > > - Styling, linting etc errors being included, because reviews are > hard to create: This is tangential IMO. We need to just bite the > bullet on this and settle on some lint and coding styles, and apply > them to the tree uniformly. The reality is *nobody ever does style > changes on their own*, and they are always accompanied by a diff, and > they always have to redo the work of pulling them out, Phab or not. > Literally 99% of the time we ask for this, it happens this way. > Perhaps instead we should just eliminate this class of work by just > running linters over all of the source code at once, and being happy > with it. > > Doing this in fact has other benefits: like `arc lint` will always > _correctly_ report when linting errors are violated. And we can reject > patches that violate them, because they will always be accurate. > > - As for some of the quotes, some of them are funny, but the real > message lies in the context. :) In particular, there have been several > cases (such as the DWARF work) where the idea was "write 30 commits > and put them on Phabricator". News flash: *this is bad*, no matter > whether you're using Phabricator or not, because it makes reviewing > the whole thing immensely difficult from a reviewer perspective. The > point here is that we can clear this up by being more communicative > about what we expect of authors of large patches, and communicating > your intent ASAP so we can get patches in as fast as possible. Writing > a patch is the easiest part of the work. > > And more: > > - Clean up the documentation, it's a mess. It feels nice that > everything has clear, lucid explanations on the wiki, but the wiki is > ridiculously massive and we have a tendancy for 'link creep' where we > spread things out. The contributors docs could probably stand to be > streamlined. We would have to do this anyway, moving to GitHub or not. > > - Improve the homepage, directly linking to this aforementioned page. > > - Make it clear what we expect of contributors. I feel like a lot of > this could be explained by having a 5 minute drive-by guide for > patches, and then a longer 10-minute guide about A) How to style > things, B) How to format your patches if you're going to contribute > regularly, C) Why it is this way, and D) finally links to all the > other things you need to know. People going into Phabricator expecting > it to behave like GitHub is a problem (more a cultural problem IMO but > that's another story), and if this can't be directly fixed, the best > thing to do is make it clear why it isn't. > > Those are just some of the things OTTOMH, but this email is already > way too long. This is what I mean though: fixing most of these is > going to have *seriously smaller cost* than moving to GitHub. It does > not account for "The GitHub factor" of people contributing "just > because it's on GitHub", but again, that value has to outweigh the > other costs. I'm not seriously convinced it does. > > I know it's work to fix these things. But GitHub doesn't really > magically make a lot of our needs go away, and it's not going to > magically fix things like style or lint errors, the fact Travis-CI is > still pretty insufficient for us in the long term (and Harbormaster is > faster, on our own hardware, too), or that it will cause needlessly > higher amounts of spam through Trac and GitHub itself. I don't think > settling on it as - what seems to be - a first resort, is a really > good idea. > > > On Wed, Sep 2, 2015 at 4:09 PM, Niklas Hambüchen <mail(a)nh2.me> wrote: > > On 02/09/15 22:42, Kosyrev Serge wrote: > >> As a wild idea -- did anyone look at /Gitlab/ instead? > > > > Hi, yes. It does not currently have a sufficient review functionality > > (cannot handle multiple revisions easily). > > > > On 02/09/15 20:51, Simon Marlow wrote: > >> It might feel better > >> for the author, but discovering what changed between two branches of > >> multiple commits on github is almost impossible. > > > > I disagree with the first part of this: When the UI of the review tool > > is good, it is easy to follow. But there's no open-source implementation > > of that around. > > > > I agree that it is not easy to follow on Github. > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs(a)haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > > -- > Regards, > > Austin Seipp, Haskell Consultant > Well-Typed LLP, http://www.well-typed.com/ > _______________________________________________ > ghc-devs mailing list > ghc-devs(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >

4 4

Better calling conventions for strict functions (bang patterns)?
by Ryan Newton 02 Nov '15

02 Nov '15

Hi all, With module-level Strict and StrictData pragmas coming soon, one obvious question is what kind of the code quality GHC can achieve for strict programs. When it came up in discussion in our research group we realized we didn't actually know whether the bang patterns, `f !x`, on function arguments were enforced by caller or callee. Here's a Gist that shows the compilation of a trivial function: foo :: Maybe Int -> Intfoo !x = case x of Just y -> y https://gist.github.com/rrnewton/1ac722189c65f26fe9ac If that function is compiled to *assume* its input is in WHNF, it should be just as efficient as the isomorphic MLton/OCaml code, right? It only needs to branch on the tag, do a field dereference, and return. But as you can see from the STG and CMM generated, foo *does indeed* enter the thunk, adding an extra indirect jump. Here's the body: c3aY: if ((Sp + -8) < SpLim) goto c3aZ; else goto c3b0; c3aZ: // nop R1 = PicBaseReg + foo_closure; call (I64[BaseReg - 8])(R2, R1) args: 8, res: 0, upd: 8; c3b0: I64[Sp - 8] = PicBaseReg + block_c3aO_info; R1 = R2; Sp = Sp - 8; if (R1 & 7 != 0) goto c3aO; else goto c3aP; c3aP: call (I64[R1])(R1) returns to c3aO, args: 8, res: 8, upd: 8; c3aO: if (R1 & 7 >= 2) goto c3aW; else goto c3aX; c3aW: R1 = P64[R1 + 6] & (-8); Sp = Sp + 8; call (I64[R1])(R1) args: 8, res: 0, upd: 8; c3aX: R1 = PicBaseReg + lvl_r39S_closure; Sp = Sp + 8; call (I64[R1])(R1) args: 8, res: 0, upd: 8; The call inside c3aP is entering "x" as a thunk, which also incurs all of the stack limit check code. I believe that IF the input could be assumed to be in WHNF, everything above the label "c3aO" could be omitted. So... if GHC is going to be a fabulous pure *and* imperative language, and a fabulous lazy *and* strict compiler/runtime.. is there some work we can do here to improve this situation? Would the following make sense: - Put together a benchmark suite of all-strict programs with Strict/StrictData (compare a few benchmark's generated code to MLton, if time allows) - Modify GHC to change calling conventions for bang patterns -- caller enforces WHNF rather than callee. Existing strictness/demand/cardinality analysis would stay the same. Unless there's something I'm really missing here, the result should be that you can have a whole chain of strict function calls, each of which knows its arguments and the arguments it passes to its callees are all in WHNF, without ever generating thunk-entry sequences. Thanks for your time, -Ryan

6 12

Proposal: accept pull requests on GitHub
by Thomas Miedema 02 Nov '15

02 Nov '15

Hello all, my arguments against Phabricator are here: https://ghc.haskell.org/trac/ghc/wiki/WhyNotPhabricator. Some quotes from #ghc to pique your curiosity (there are some 50 more): * "is arc broken today?" * "arc is a frickin' mystery." * "i have a theory that i've managed to create a revision that phab can't handle." * "Diffs just seem to be too expensive to create ... I can't blame contributors for not wanting to do this for every atomic change" * "but seriously, we can't require this for contributing to GHC... the entry barrier is already high enough" GitHub has side-by-side diffs <https://github.com/blog/1884-introducing-split-diffs> nowadays, and Travis-CI can run `./validate --fast` comfortably <https://travis-ci.org/ghc/ghc/builds>. *Proposal: accept pull requests from contributors on https://github.com/ghc/ghc <https://github.com/ghc/ghc>.* Details: * use Travis-CI to validate pull requests. * keep using the Trac issue tracker (contributors are encouraged to put a link to their pull-request in the 'Differential Revisions' field). * keep using the Trac wiki. * in discussions on GitHub, use https://ghc.haskell.org/ticket/1234 to refer to Trac ticket 1234. The shortcut #1234 only works on Trac itself. * keep pushing to git.haskell.org, where the existing Git receive hooks can do their job keeping tabs, trailing whitespace and dangling submodule references out, notify Trac and send emails. Committers close pull-requests manually, just like they do Trac tickets. * keep running Phabricator for as long as necessary. * mention that pull requests are accepted on https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/FixingBugs. My expectation is that the majority of patches will start coming in via pull requests, the number of contributions will go up, commits will be smaller, and there will be more of them per pull request (contributors will be able to put style changes and refactorings into separate commits, without jumping through a bunch of hoops). Reviewers will get many more emails. Other arguments against GitHub are here: https://ghc.haskell.org/trac/ghc/wiki/WhyNotGitHub. I probably missed a few things, so fire away. Thanks, Thomas

16 20

Re: [commit: ghc] master: Kill redundant import (da58d15)
by Herbert Valerio Riedel 30 Oct '15

30 Oct '15

originally, http://git.haskell.org/ghc.git/commitdiff/7eb5e29b4a7b6fef55512bc7bf3308e71… the import looked like this: import qualified HsSyn -- hack as we want to reexport the whole module import HsSyn hiding ((<.>)) so it wasn't redundant back then at least. > commit da58d1517ce9c57514f7b2bd7d444c9e3051a717 > Author: Simon Peyton Jones <simonpj(a)microsoft.com> > Date: Fri Oct 30 09:45:16 2015 +0000 > > Kill redundant import > > I don't really understand this. GHC.hs had: > > module GHC( ..., module HsSyn, ... ) where > import HsSyn > import qualified HsSyn -- hack as we want to reexport the whole module > > GHC now reports the 'import qualified' line as redundant; and it certainly > is, because 'import HsSyn' brings everything into scope qualified *and* > unqualified. I have no idea why the "hack" was necessary before, but > following my refactoring of tcg_used_gres (previous commit), it's reported > as redundant, so I've deleted it. > > I'm making it a separate commit because it seems a bit mysterious and I > wanted to draw attention to it.

1 0