On 29/05/2013, at 1:11 AM, Edward Kmett wrote:

This unfortunately means, that we can't really show the unaccepted proposals with information about how to avoid getting your proposal rejected.

You can if you rewrite the key points of proposal to retain the overall message, but remove identifying information. I think it would be helpful to write up some of the general reasons for projects being rejected. I tried to do this for Haskell experience reports, on the Haskell Symposium experience report advice page. http://www.haskell.org/haskellwiki/HaskellSymposium/ExperienceReports I'd imagine you could write up some common proposal / rejection / advice tuples like: Proposal: I want to write a MMORPG in Haskell, because this would be a good demonstration for Haskell in a large real world project. We can use this as a platform to develop the networking library infrastructure. Rejection: This project is much too big, and the production of a MMORPG wouldn't benefit the community as a whole. Advice: If you know of specific problems in the networking library infrastructure, then focus on those, using specific examples of where people have tried to do something and failed. Ben.

Hi Edward, Thanks for this comprehensive answer (and also thanks to participants in the follow-up dissuasion). How is the "public good" determined? (sounds rather Benthamite). I would have been disappointed if "charts using diagrams" had not been selected yet I don't recall being canvassed. Sorry to sound picky. I think from what you say that in this particular year it was obvious which projects should be selected; in future it may not be. I think an acceptable reason would be "there was only one user who wanted it". Maybe we should use something like: https://www.uservoice.com. Sadly it seems this requires payment but there may be a free equivalent Dominic Steinitz dominic@steinitz.org http://idontgetoutmuch.wordpress.com On 28 May 2013, at 16:11, Edward Kmett wrote:

Hi Dominic,

The proposal is admittedly rather unfortunately opaque.

The parts I can shed light on:

Students come up with proposals with the help of the community and then submit them to google-melange.com.

A bunch of folks from the haskell community sign up as potential mentors, vote on and discuss the proposals. (We had ~25 candidate mentors and ~20 proposals this year).

The student application template contains a number of desirable criteria for a successful summer of code application, which is shown on the google-melange website under our organization -- an old version is available http://hackage.haskell.org/trac/summer-of-code/wiki/StudApply2012 contains

Once we have the proposals in hand, and some initial ranking, we ask google for slots. Allocation is based on past performance, arcane community parameters that only they know, mentor ratio, etc. This should be our largest year in the program, despite the fact that in general organizations have been getting fewer slots as more organizations join, so we're apparently doing rather well.

In general we do try to select projects that maximize the public good. Most of the time this can almost be done by just straight cut off based on the average score. There is some special casing for duplicate applications between different students and where students have submitted multiple applications we can have some flexibility in how to apply them.

This year we also received an extra couple of special-purpose darcs slots from Google in exchange for continuing to act as an umbrella organization over darcs at the request of the administrator of the program at Google. In previous years I had requested an extra slot for them, this year the request came in the other direction.

We do inevitably get more good proposals than we get slots. This year we could have easily used another 3-4 slots to good effect.

The main part I can't shed light on:

Google requests that the final vote tallies remain private. This is done so that students who put in proposals to a high volume orgs and don't get accepted, or who are new to the process and don't quite catch all the rules, don't wind up with any sort of publicly visible black mark. This unfortunately means, that we can't really show the unaccepted proposals with information about how to avoid getting your proposal rejected.

I hope that helps. If you have any more questions or if my answer didn't suffice please feel free to follow up!

-Edward Kmett

On Tue, May 28, 2013 at 6:52 AM, Dominic Steinitz wrote: Hi Edward,

Although the project I am interested in (as a user) has been accepted :-), I can't help feeling the selection process is a bit opaque. Is it documented somewhere and I just missed it? Apologies if I did.

BTW I appreciate all the hard work that goes into the selection process.

Dominic Steinitz dominic@steinitz.org http://idontgetoutmuch.wordpress.com

I will certainly volunteer (to mentor) next year if I feel I can add value. Dominic Steinitz dominic@steinitz.org http://idontgetoutmuch.wordpress.com On 2 Jun 2013, at 17:23, Edward Kmett wrote:

Public good is a nebulous concept, but it is something that each of the folks who sign up as mentors judges independently when they are rating the projects and talking about them.

Most of the folks who are offering to mentor have been involved in the community for quite some time and have a pretty good overview of what is going on, and what are currently active pain points.

With 25 mentors we get a pretty good cross section of the community. We aren't really able to canvas outside of the mentor group during the approval process by google's guidelines, since we shouldn't leak information about unaccepted projects.

Something like that uservoice site might be used to gauge public opinion of general ideas before the proposals start coming in, but in the end students write the proposals we get, so the things we would have polled about are inevitably not quite what we're rating anyways. We rarely get something that is just cut and pasted from the ideas list. Consequently a generic rating that doesn't take into consideration the actual proposal isn't worth a whole lot, beyond giving students an idea of what might be a successful proposal. There is a lot of variability in the ratings for projects based simply on what we know about the student, how clear the proposal is, and how achievable his or her particular goals are.

In practice, we've been able to make sure that a couple of slots go to separable tasks in projects like cabal, haddock, and ghc that benefit everyone and that exceptional one-off projects don't get shut out completely just by asking each mentor to rate all of the projects, even the ones they aren't interested in mentoring, and from the discussions between the mentors and between the mentors and students that ensue within melange.

My main advice is that if you want to get involved in the process, the easiest way to peel back the curtain is to volunteer to mentor! We're generally quite open to adding new voices to the discussion.

-Edward

On Sun, Jun 2, 2013 at 10:14 AM, Dominic Steinitz wrote: Hi Edward,

Thanks for this comprehensive answer (and also thanks to participants in the follow-up dissuasion).

How is the "public good" determined? (sounds rather Benthamite). I would have been disappointed if "charts using diagrams" had not been selected yet I don't recall being canvassed.

Sorry to sound picky. I think from what you say that in this particular year it was obvious which projects should be selected; in future it may not be. I think an acceptable reason would be "there was only one user who wanted it". Maybe we should use something like: https://www.uservoice.com. Sadly it seems this requires payment but there may be a free equivalent

Dominic Steinitz dominic@steinitz.org http://idontgetoutmuch.wordpress.com

On 28 May 2013, at 16:11, Edward Kmett wrote:

...

Hi Dominic,

The proposal is admittedly rather unfortunately opaque.

The parts I can shed light on:

Students come up with proposals with the help of the community and then submit them to google-melange.com.

A bunch of folks from the haskell community sign up as potential mentors, vote on and discuss the proposals. (We had ~25 candidate mentors and ~20 proposals this year).

The student application template contains a number of desirable criteria for a successful summer of code application, which is shown on the google-melange website under our organization -- an old version is available http://hackage.haskell.org/trac/summer-of-code/wiki/StudApply2012 contains

Once we have the proposals in hand, and some initial ranking, we ask google for slots. Allocation is based on past performance, arcane community parameters that only they know, mentor ratio, etc. This should be our largest year in the program, despite the fact that in general organizations have been getting fewer slots as more organizations join, so we're apparently doing rather well.

In general we do try to select projects that maximize the public good. Most of the time this can almost be done by just straight cut off based on the average score. There is some special casing for duplicate applications between different students and where students have submitted multiple applications we can have some flexibility in how to apply them.

This year we also received an extra couple of special-purpose darcs slots from Google in exchange for continuing to act as an umbrella organization over darcs at the request of the administrator of the program at Google. In previous years I had requested an extra slot for them, this year the request came in the other direction.

We do inevitably get more good proposals than we get slots. This year we could have easily used another 3-4 slots to good effect.

The main part I can't shed light on:

Google requests that the final vote tallies remain private. This is done so that students who put in proposals to a high volume orgs and don't get accepted, or who are new to the process and don't quite catch all the rules, don't wind up with any sort of publicly visible black mark. This unfortunately means, that we can't really show the unaccepted proposals with information about how to avoid getting your proposal rejected.

I hope that helps. If you have any more questions or if my answer didn't suffice please feel free to follow up!

-Edward Kmett

On Tue, May 28, 2013 at 6:52 AM, Dominic Steinitz wrote: Hi Edward,

Although the project I am interested in (as a user) has been accepted :-), I can't help feeling the selection process is a bit opaque. Is it documented somewhere and I just missed it? Apologies if I did.

BTW I appreciate all the hard work that goes into the selection process.

Dominic Steinitz dominic@steinitz.org http://idontgetoutmuch.wordpress.com