On 13/12/12 09:54, Yuras Shumovich wrote:

On Thu, 2012-12-13 at 09:41 +0000, Chris Nicholls wrote:

...

What's the best way to get started? Bug fixes? Writing a toy plugin? I don't have a huge amount of time to offer, but I would like to learn to help!

GHC bug sweep is the way I'm trying to start with: http://hackage.haskell.org/trac/ghc/wiki/BugSweep (have no idea whether it is the best way or not :) )

The BugSweep was a great idea at the time, but I think it stalled. Do feel free to carry on though! I think it's a great way to learn about GHC, because it will send you off in random directions investigating things. While there are lots of old bugs that are now fixed, or have duplicates, or aren't relevant, etc. there are also lots of old bugs that are still around because they aren't fixed because they aren't worth the effort. Still, making even *some* progress on any old bug is worthwhile, even if it is to point out a workaround that didn't exist before, or update the test case. This will generate an email, and since there are people that read all the Trac emails, occasionally this prods someone else to make further progress or just close the ticket. Cheers, Simon

...

And this is where I'm a little stuck too. I have a reasonably good picture of GHC's pipeline, having worked on it for my master's thesis, and I'd like to get more involved and possibly even contribute something useful. But I'm not sure where to start. Well, that's a bit discouraging - if you worked on it in master's thesis and still don't know where to start then it this is probably even harder than I suspected.

Sorry! That wasn't what I intended. I meant more that I have a reasonably, good idea of *how* to work on ghc but not much of idea of *what* I can usefully do. The work in my thesis turned out not to be that useful, and I'm not sure what to do next. It sounds like you've already got some ideas there though. Perhaps I shouldn't have hijacked your question with a 'me too' but I suspect there are quite a lot of people who want to contribute, but don't really know where to start. Don't be discouraged, its absolutely possible to get your hands dirty without having to understand all the theory and the community is incredibly friendly. As for getting an overview of the pipeline, I found the videos of the talks the Simons gave at the 2006 Hackathon helpful. Perhaps someone with more knowledge than me can comment on how relevant the content is after six years?

Hi Jan, On Thu, Dec 13, 2012 at 01:56:13PM +0100, Jan Stolarek wrote:

That's what I'm trying to figure out right now. GHC build system looks very complicated and it seems I'll be spending weeks on understanding it before I can get to the compiler code itself.

I think it's easier to have a fix point. Look for something you would like to add to GHC, for a bug you would like to fix. Starting at this fix point you can spread out and learn all the necessary things to be able to implement it. Sure, you will never know if you know enough to implement it in the most fitting way for GHC. But I would consider this as part of the learning process and the GHC developers will give you appropriate feedback. I think that it's quite hard to learn such a big system like GHC in a top to bottom manner and for myself it's easier to stay motivated if I've a concrete task, the learning has a concrete reason. Greetings, Daniel

Hi Chris, On Thu, Dec 13, 2012 at 02:52:37PM +0000, Chris Nicholls wrote:

I worked on my own, but the GHC folk were very helpful when I got stuck. That was quite a slow tactic in retrospect, I reckon I could have got much further if I'd had the bottle to ask more questions. One problem with the Haskell community is that everyone else seems so much smarter...

Smart people won't think of you to be dumb because you know less, because they know how little they know themself. ;) Fear is such a drawback and the more reserved guys get easily intimidated by the cocky guys, because they think that the cocky guys really know what they're doing. Greetings, Daniel

On 12/14/12 9:44 AM, Simon Peyton-Jones wrote:

This thread has made it clear that we should do more to help people find a "way in" to GHC. Here is what I have done:

·Started a GHC Reading List page http://hackage.haskell.org/trac/ghc/wiki/ReadingList, giving background reading. It's just a start; there are many gaps. I would love it if you would all help fill it out. It's a wiki.

·Amplified the Working on GHC http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions page. Again, please help make it more useful. It's a wiki.

What else would help? If the answer is X, can anyone help do X?

Something else that I think would be useful is a long-term 'wishlist' for ghc, consisting of modest goals from an engineering standpoint. My sense is that these involve factoring subsystems to make them more independent and modular, as well as moving towards parallelizable code, and perhaps the notion of ghc-as-a-service so that one could have a long-running compiler process able to perform individual parts of a pipeline on-demand. The scheme of what needs to change and overall architecture needs to be determined by people with very good knowledge of the compiler as a whole. But individual tasks and areas could be tackled by others wishing to contribute. I'd also like more of a sense of what work remains to be done to decouple ghci from ghc, such that we could pleasantly produce other ghci-like environments using the same API, and perhaps ultimately treat desktop ghci as one means among many of interacting with ghc-as-a-library. (i.e. how can we get to 'cabal install ghci'). I'm also curious about the state of the plugins pipeline, how comfortable we are with it, and what could potentially be improved (and how that relates to having ghc potentially process externally generated or transformed core). This is to say, outside of the list of bugs on trac, and outside of potentially researchy new features, what are the sorts of things we would like to make GHC a more dependable, efficient, maintainable, flexible piece of software? The things I'm suggesting a focus on are things that I remember discussion of along these lines. They're perhaps not where core GHC developers feel effort should be focused. But where then *should* it be? A sense of this might help interested contributors to decide where to direct their efforts. Thanks, Gershom

Dnia piątek, 14 grudnia 2012, Simon Peyton-Jones napisał:

This thread has made it clear that we should do more to help people find a "way in" to GHC. Thanks for taking intrest in our efforts Simon :)

* Started a GHC Reading List pagehttp://hackage.haskell.org/trac/ghc/wiki/ReadingList, giving background reading. It's just a start; there are many gaps. I would love it if you would all help fill it out. It's a wiki. I read some of the listed papers and while they definitely gave me a nice overview of the compiler I still feel a bit helpless when faced with the actual code. Simon, you listed your books in "General background" section. I don't doubt they are interesting (I only started reading fisrt of the books) and if one wants to impement functional languages they are a must. But they are over 700 pages in total - is reading them an optimal way of getting into GHC development? Well, if the answer is 'yes' then I'm off to reading, but perhaps there are easier approaches? Anyway, it's a great idea to have a list of papers and books about architecture of GHC, so I'm going to fill out DPH section. BTW. GpH page has a nicely prepared list of papers - GHC deserves

After three days of reading through the wiki I came up with some conclusions. The thing that seems to be most difficult for me is that wiki has a tree structure: every page links to many other pages. For an unexperienced person it is hard to prioritize information. What do I need to know first and what can be left out for later? It would be good if this knowledge was somehow linearized. I found Hackathon lectures to be slightly helpful. Low quality of sound and video makes it difficult to learn from these particular videos but I think the idea itself is great. Perhaps it would be possible to create some introductory videos & presentations that would introduce GHC's build system and architecture? I know this would be a lot of effort but perhaps it would pay off? I'm thinking something like Coursera class. I would love to see such a thing! Another thing I found difficult is that wiki gives different information on getting the sources. This page: http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSources gives one set of commands one has to issue to check out the source code, but this page: http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Git gives a different set of instructions. For a person that does not know the structure of repositories and the build system itself this is confusing. I think that at this point people expect - at least I expected - a list of commands to run in order to get the sources. Having two different possibilities is confusing as one needs to figure out what these commands actually do. With my current knowledge - or lack of thereof - I'm unable to correct this on the wiki the same. To sum up: I think that wiki has almost all the knowledge needed to get started, but that knowledge has to be presented to the beginners in a form of some tutorial, presentations or anything that is linear and guides people by hand. I hope that was constructive. Janek

* Amplified the Working on GHChttp://hackage.haskell.org/trac/ghc/wiki/WorkingConventions page. Again, please help make it more useful. It's a wiki.

What else would help? If the answer is X, can anyone help do X?

Thanks

Simon

From: glasgow-haskell-users-bounces@haskell.org [mailto:glasgow-haskell-users-bounces@haskell.org] On Behalf Of Chris Nicholls Sent: 13 December 2012 12:00 To: Jan Stolarek Cc: GHC Users List Subject: Re: How to start with GHC development?

...

And this is where I'm a little stuck too. I have a reasonably good picture of GHC's pipeline, having worked on it for my master's thesis, and I'd like to get more involved and possibly even contribute something useful. But I'm not sure where to start.

Well, that's a bit discouraging - if you worked on it in master's thesis and still don't know where to start then it this is probably even harder than I suspected.

Sorry! That wasn't what I intended. I meant more that I have a reasonably, good idea of *how* to work on ghc but not much of idea of *what* I can usefully do. The work in my thesis turned out not to be that useful, and I'm not sure what to do next. It sounds like you've already got some ideas there though.

Perhaps I shouldn't have hijacked your question with a 'me too' but I suspect there are quite a lot of people who want to contribute, but don't really know where to start.

Don't be discouraged, its absolutely possible to get your hands dirty without having to understand all the theory and the community is incredibly friendly. As for getting an overview of the pipeline, I found the videos of the talks the Simons gave at the 2006 Hackathon helpful. Perhaps someone with more knowledge than me can comment on how relevant the content is after six years?

Dnia piątek, 14 grudnia 2012, Ian Lynagh napisał:

I think the main problem is that it's a very broad question. The answer to "how should I get started" would be completely different for if you wanted to implement a type system extension, port GHC to a new platform, or fix a bug in ghci All three task have one thing in common though - the GHC build system. Project sources are over 500MB, there are many subrepositories, build scripts etc - this is really intimidating at first and I guess that seeing all that was a bit discouraging. Now that I know the compiler itself is about 9MB it doesn't look that bad :) Wiki indeed seems to have all the information required. It's just that it would be good to give beginners a _precise_ list of things they should read to begin hacking.

Janek

OK, so how can we improve it? First of all I think that materials in the wiki are well organized for people who already have some knowledge and only need to learn about particular things they don't yet know. In this case I think it is fine to have wiki pages connected between one another so they form cycles and don't have a particular ordering.

However as a beginner I'd expect that pages are ordered in a linear way (sorry for repeating myself, but I think this is important), that every concept is explained in only one place and that there are no links in the text (even at the end - if the text is structured lineary than I don't need to decide what to read next, I just go to next section/chapter). This is of course a perfect situation that seems to be achievable in a book, but probably not in a wiki (wikibook could be a middle ground?). Right now there are two pages (I know of...) that instruct how to get the sources: http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSources http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Git They list two different places containing the sources: github and darcs.haskell.org. Are these equivalent? A couple of months ago I played with GHC sources: the ones from github were about 17MB, the ones from haskell.org were much larger (85MB IIRC) so I guess that these might not be exact mirrors. Second page gives a nice description of how to set up build trees, but it duplicates information from the first page (checking out the sources). First page also tells to make a local clone of the repo but it does not explain why such a clone should be made (the second page does). I think that merging these two pages into one might be a good idea. I would definitely like more details on the git workflow, especially the branches. Right now I see about 40 branches - why so many? Am I required to create my own branch when I start hacking or should I just go with master? It seems that many informations in the wiki are duplicated. There are two pages about repositories: http://hackage.haskell.org/trac/ghc/wiki/Repositories http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositories (after reading the first one source tree started to make much more sense - this is one of the informations *I* would like to get first). In general I think that for beginers it would be good if the wiki had a form of a book divided into chapters. I only wonder if it is possible to adjust the wiki for the newbies and at the same time keep it useful for more experienced developers. Janek

| > It seems that many informations in the wiki are duplicated. There are | > two pages about | > repositories: | > http://hackage.haskell.org/trac/ghc/wiki/Repositories | > http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositori | > es (after reading the first one source tree started to make much more | > sense - this is one of the informations *I* would like to get first). | | The first page lists the repositories and where the upstreams and | mirrors are. The second page contains the conventions for working on | other repositories (which is why it's under WorkingConventions). Simon, I don't find that a clear distinction. Looking at the two, I'm a bit confused too! * The lists on WorkingConventions/Repositories duplicates the table in Repositories. * I believe that perhaps WorkingConventions/Repositories is solely concerned with how to *modify* a library; it opening para says as much. Fine; but it shouldn't duplicate the info. Maybe the table could do with a column saying "GHC" or "Upstream" to specify the "how to modify" convention? (I wish the table could somehow be narrower. And that the library name was the first column.) Perhaps the master table can look like this: What GHC repo location Upstream repo exists? http://darcs.haskell.org/ GHC ghc.git ghc-tarballs ghc-tarballs.git ...etc... binary binary.git YES ...etc... Then we can deal with the complexities of upstream repos in another page. I think that might put the info in a way that's easier to grok. I can do it if Simon and Ian agree; or Ian could. Simon

So the first page here tells you how to get a single source tree so that you can build it. The second page tells you how to create a 2-tree setup for use with GHC's validate; the latter is aimed at people doing lots of GHC development (which is why it's under WorkingConventions). I think that wiki doesn't make distinction between (setup for beginners)/(setup for regular developers) clear - see below. It would be good to give some motivation behind each of these approaches as right now it seems there are two different instructions to do the same thing.

(...) I think it's more important to deal with the simple case first which is why it is right near the start of the Building Guide. Please notice that one of the first things one reads on "Getting the GHC Sources" page is:

"The first thing to do is install git. And then read Git Working Conventions for instructions on how to use Git with GHC development." It suggests quite explicitly to read the conventions before proceeding with the rest of the page! Personally I find information about dual-tree setup very useful. Assuming one knows how to use git it is not as overcomplicated setup.

The first page lists the repositories and where the upstreams and mirrors are. The second page contains the conventions for working on other repositories (which is why it's under WorkingConventions). I would find it more intuitive to have one page with couple of sections: 1) Structure of repositories; 2) Working with external repositories; - or something like that.

The nice thing about a wiki is that you don't have to move content around, you can just make new contents pages that contain whatever organisation you want. So maybe what you want is a separate page that links to things to read in a particular order? No, I think not - the main page of "Building Guide" is already such a page. The list of topics is very nice and I don't feel it needs improving. The problems begin when one start reading through particular topics.

I must add that despite my endless comments on how Commentary could be improved I consider GHC to be the best documented project I've seen so far. You did a really fantastic work with putting your knowledge on the wiki! Janek

Try this:

http://hackage.haskell.org/trac/ghc/wiki/Repositories Very nice! WorkingConventions/Repositories still duplicates some information but that's not a big problem since neither Repositories nor Repositories/Upstream link to it. Perhaps it would be good to give a short explanation about darcs mirrors? They are in the diagram in Repositories/Upstream. I think there was some explanation on the previous version of these pages.

Janek