Re: [ghc-steering-committee] Statistical analysis of GHC proposals

Dear Andrew
Thank you -- that's very interesting.
GHC SC: do look below!
I'm not quite sure what, if anything, we should do in the light of this
information.
Simon
On Sun, 27 Oct 2024 at 23:06, Andrew Lelechenko
Hi Simon,
I’ve been exploring GitHub API capabilities by gathering data on GHC proposals. I wonder if any of the numbers below might be interesting to reflect on for GHC SC.
Total number of GHC proposals: 441 Rate of proposals: 5 per month Approved proposals: 149 Need revision: 30 Declined proposals: 16
Median time from creation to decision: 101 days Average time from creation to decision: 181 days Median time from creation to approval: 100 days Average time from creation to approval: 165 days Fastest approval: 8 days for "Adjust warning category syntax" 2nd fastest approval: 11 days for "Remove dot from characters allowed in overloaded labels" 2nd slowest approval: 980 days for "Fine-grained unused warnings" Slowest approval: 1112 days for "Decorate exceptions with backtrace information"
Total activity: 15999 comments Median activity per proposal: 23 comments Average activity per proposal: 36 comments Median activity per approved proposal: 31 comments Average activity per approved proposal: 51 comments Least active approved proposal: 1 comment for "Amend the Higher Order Patterns proposal" 2nd least active approved proposal: 2 comments for "Amend 281 visible forall to work without ScopedTypeVariables" 2nd most active: 313 comments for "Lambda expressions with guards and multiple clauses (was: `\\ of`, -XMultiWayLambda)" Most active: 519 comments for "RecordDotSyntax language extension proposal"
Open proposals: 117 Median age for open proposals: 1105 days Average age for open proposals: 1215 days Newest open proposal: 12 days for "Clarify CRLF behavior in multiline strings" Oldest open proposal: 2960 days for "Mutable constructor fields"
Best regards, Andrew

This is indeed interesting, thanks Andrew. We have some impressively mature proposals! Something I've been mulling over for a while is that our process tends not to explicitly reject many proposals, but it is not unusual for proposals to stall in one of two states: 1. There is some initial discussion, but no final decision, and the original author stops actively driving the proposal forward (e.g. because they no longer have time or enthusiasm). In some cases the discussion may have been leaning towards rejection, but in others there are good ideas lingering that need someone to pick them up. 2. The proposal is accepted, but then lingers awaiting implementation. This is difficult to address in a world of limited resources for GHC development (both volunteer effort and paid maintainer time). At the moment I usually wait until a proposal author explicitly submits the proposal for committee consideration before assigning a shepherd. It is then the shepherd's responsibility to make progress in a reasonable time. One change I've been wondering about is that we might try to identify shepherds from the committee at an earlier stage, and/or encourage shepherds to volunteer when they see an interesting proposal. The shepherd could then help the original author respond to the discussion and move through the process (perhaps including taking over the proposal if the original author is no longer engaging). For example I'm effectively trying to do this on #668. This might result in fewer proposals getting blocked indefinitely, at the cost of more shepherding work for the committee. What do you all think? Adam On 28/10/2024 08:13, Simon Peyton Jones wrote:
Dear Andrew
Thank you -- that's very interesting.
GHC SC: do look below!
I'm not quite sure what, if anything, we should do in the light of this information.
Simon
On Sun, 27 Oct 2024 at 23:06, Andrew Lelechenko
mailto:andrew.lelechenko@gmail.com> wrote: Hi Simon,
I’ve been exploring GitHub API capabilities by gathering data on GHC proposals. I wonder if any of the numbers below might be interesting to reflect on for GHC SC.
Total number of GHC proposals: 441 Rate of proposals: 5 per month Approved proposals: 149 Need revision: 30 Declined proposals: 16
Median time from creation to decision: 101 days Average time from creation to decision: 181 days Median time from creation to approval: 100 days Average time from creation to approval: 165 days Fastest approval: 8 days for "Adjust warning category syntax" 2nd fastest approval: 11 days for "Remove dot from characters allowed in overloaded labels" 2nd slowest approval: 980 days for "Fine-grained unused warnings" Slowest approval: 1112 days for "Decorate exceptions with backtrace information"
Total activity: 15999 comments Median activity per proposal: 23 comments Average activity per proposal: 36 comments Median activity per approved proposal: 31 comments Average activity per approved proposal: 51 comments Least active approved proposal: 1 comment for "Amend the Higher Order Patterns proposal" 2nd least active approved proposal: 2 comments for "Amend 281 visible forall to work without ScopedTypeVariables" 2nd most active: 313 comments for "Lambda expressions with guards and multiple clauses (was: `\\ of`, -XMultiWayLambda)" Most active: 519 comments for "RecordDotSyntax language extension proposal"
Open proposals: 117 Median age for open proposals: 1105 days Average age for open proposals: 1215 days Newest open proposal: 12 days for "Clarify CRLF behavior in multiline strings" Oldest open proposal: 2960 days for "Mutable constructor fields"
Best regards, Andrew
-- Adam Gundry, Haskell Consultant Well-Typed LLP, https://www.well-typed.com/ Registered in England & Wales, OC335890 27 Old Gloucester Street, London WC1N 3AX, England

On 2024-10-29 20:35, Adam Gundry wrote:
At the moment I usually wait until a proposal author explicitly submits the proposal for committee consideration before assigning a shepherd. It is then the shepherd's responsibility to make progress in a reasonable time.
One change I've been wondering about is that we might try to identify shepherds from the committee at an earlier stage, and/or encourage shepherds to volunteer when they see an interesting proposal. The shepherd could then help the original author respond to the discussion and move through the process (perhaps including taking over the proposal if the original author is no longer engaging). For example I'm effectively trying to do this on #668. This might result in fewer proposals getting blocked indefinitely, at the cost of more shepherding work for the committee. What do you all think?
Adam
Thank you Adam. I think this is a great idea. The proposal process is perceived daunting by many so we should try to be excellent and welcoming hosts. (Which does not mean compromising in rigor.) Main argument in favor is in my opinion, that in the last year the (felt) proposal throughput has decreased will the backlog of work for the committee is mostly empty. That is actually partially an accomplishment of the committee, but it also shows that we can probably do more to foster progress in GHC. The only problem I see - and it is the obvious one - is that we have historically had sheperds on the committee which were very unresponsive, for various (probably quite understandable reasons). With more responsibilities the gap between expectation and delivery would likely increase. So we should probably only do this if a clear majority of the committee is in favor. (Or more generally if we can still find enough volunteers to commit to doing this.) Otherwise we would promise something we can’t deliver. Now that I am thinking about it we would then need a hopefully clear description of the responsibilities of the sheperd before submission. I think it is sometimes kind of problematic, that there is supposed to be a community discussion during preparation of the proposal, but then the committee is only formally asked to look at it after that discussion happened. We should be part of the discussion. Simon has at times pointed us at proposals in the discussion phase, but maybe there is a slightly more structured way to do this. Best Malte

Now that I am thinking about it we would then need a hopefully clear description
of the responsibilities of the sheperd before submission
Our current statement of those responsibilities is here:
https://github.com/ghc-proposals/ghc-proposals#committee-process-for-respond...
I wonder if we should make it clearer or more prominent. It says:
- Shepherd should make a recommendation within weeks.
- Committee should decide within a month after that
I think it is helpful to have a clear moment at which the proposal is
submitted to the committee. I agree that the involvement of a shepherd
prior to that point would be helpful. But ideally the community at large
engages in discussion with the author to develop and refine their proposal.
I'm cautious about adding an obligation to proactively engage with all
proposals. But *perhaps an author can request a shepherd to consult*, in
advance of submitting the proposal. That puts the initiative on the
author, to which we can (and should) be responsive. Would that work?
Simon
On Tue, 29 Oct 2024 at 22:11, Malte Ott
At the moment I usually wait until a proposal author explicitly submits
proposal for committee consideration before assigning a shepherd. It is
the shepherd's responsibility to make progress in a reasonable time.
One change I've been wondering about is that we might try to identify shepherds from the committee at an earlier stage, and/or encourage shepherds to volunteer when they see an interesting proposal. The shepherd could
help the original author respond to the discussion and move through the process (perhaps including taking over the proposal if the original author is no longer engaging). For example I'm effectively trying to do this on #668. This might result in fewer proposals getting blocked indefinitely, at the cost of more shepherding work for the committee. What do you all
On 2024-10-29 20:35, Adam Gundry wrote: the then then think?
Adam
Thank you Adam. I think this is a great idea. The proposal process is perceived daunting by many so we should try to be excellent and welcoming hosts. (Which does not mean compromising in rigor.)
Main argument in favor is in my opinion, that in the last year the (felt) proposal throughput has decreased will the backlog of work for the committee is mostly empty. That is actually partially an accomplishment of the committee, but it also shows that we can probably do more to foster progress in GHC.
The only problem I see - and it is the obvious one - is that we have historically had sheperds on the committee which were very unresponsive, for various (probably quite understandable reasons). With more responsibilities the gap between expectation and delivery would likely increase.
So we should probably only do this if a clear majority of the committee is in favor. (Or more generally if we can still find enough volunteers to commit to doing this.) Otherwise we would promise something we can’t deliver.
Now that I am thinking about it we would then need a hopefully clear description of the responsibilities of the sheperd before submission. I think it is sometimes kind of problematic, that there is supposed to be a community discussion during preparation of the proposal, but then the committee is only formally asked to look at it after that discussion happened. We should be part of the discussion. Simon has at times pointed us at proposals in the discussion phase, but maybe there is a slightly more structured way to do this.
Best Malte _______________________________________________ ghc-steering-committee mailing list ghc-steering-committee@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
participants (3)
-
Adam Gundry
-
Malte Ott
-
Simon Peyton Jones