Haskell Platform package additions: decision time!
All, About a month ago we came up with a recommendation for a procedure for adding new packages to the Haskell Platform. Initial announcement: http://haskell.org/pipermail/libraries/2009-August/012397.html Proposed policy: http://trac.haskell.org/haskell-platform/wiki/AddingPackages If you like the video format then see 7min:15sec into this talk (from the recent Haskell implementers' workshop) http://www.vimeo.com/6572504 We would like to get this wrapped up so that we can move on to discussing more guidelines/requirements on packages and indeed to actually proposing some packages. So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes". If you were following the original thread you'll remember that Ian has a few concerns that we've not yet fully resolved. I have suggestions for how to move forward. Feedback and opinions from other people would probably help. I'll set out what the concerns are and what I suggest. Two of the complaints we can deal with fairly easily, the third would be a more radical change from what we're proposing, but we can move a bit closer. First the two simple ones: Concern 1: "The policy document itself is too long and too detailed." Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no. * Advantages: policy document looks shorter, less intimidating * Disadvantages: slower to jump back and forth between the policy and the rationale (the two are fully cross-referenced) If we split it, then the main page would look like this: http://trac.haskell.org/haskell-platform/wiki/AddingPackagesCore and the "[rationale-x.y]" links would point to a separate page. It really doesn't matter which we pick, we just need to decide and get on with it! Concern 2: "Documentation written for the proposal will get lost." The concern here is that if people write some new intro tutorial or something for a package proposal that this might never get integrated into the package's documentation. Ian's suggested remedy is to force proposals to link to existing documentation rather than letting proposal authors make something new or make a customised variation on existing docs. The principle that the steering committee seemed to agree on is that the proposal author be given the freedom to present their argument in whatever way they see fit. The alternative that I am proposing is this: in the case that completely new documentation is written for a proposal, there be a presumption that a condition of acceptance is that the new material be republished in the appropriate place and form. For example this might mean the reviewers require that new intro docs be integrated into the haddock docs or perhaps as a separate tutorial or user guide on the package's homepage. The point is that by attaching it as a condition of the package being accepted, that we ensure that it does actually get done rather than nice docs languishing where no users will find them. The policy already has the facility for conditional acceptance so this just adds a presumed condition. Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist. Some checklist items would be objective, some subjective, but passing them all would be sufficient for a package to be accepted. With this approach the proposal would also essentially be a checklist, noting that each one is met (with explanation as appropriate). By contrast, the criteria that the steering committee proposes is that a package is accepted if the reviewers reach consensus. That is, ultimately it is up to the judgement of the reviewers. This approach is augmented with a list of minimum package requirements to make it clearer what level package authors have to aim for. In the limit the two approaches are the same. By adding more and more package requirements and guidelines the grey area left up to the judgement of reviewers is reduced. Also, the checklist approach allows some items to be essentially "do the reviewers agree that ...", so it's still a judgement and a consensus but broken up into categories. Related to this issue is the concern that we will find it impossible to agree on any packages before we have agreed a more comprehensive set of package requirements. My argument is that we can make progress on both simultaneously, that is we can start discussing individual package proposals while also discussing general package requirements. Indeed I think the two discussions might inform each other. In the worst case we get blocked discussing a package because we do not agree on the level of some quality requirement, in which case we move to discussing general package requirements and record the decision in the policy. So my suggestion on this complaint is that we go with the policy that the steering committee has proposed and that we move quickly afterwards to discuss more comprehensive package requirements and guidelines. Duncan
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Yes.
Concern 1: "The policy document itself is too long and too detailed."
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
What you asked was not a yes-no question :-p But I say split it to a separate page.
Concern 2: "Documentation written for the proposal will get lost." [...]
The alternative that I am proposing is this: in the case that completely new documentation is written for a proposal, there be a presumption that a condition of acceptance is that the new material be republished in the appropriate place and form.
I agree with this view. Having documentation in the proposal itself is nicer than just links, and your proposal deals with the issue that arises.
Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist. Some checklist items would be objective, some subjective, but passing them all would be sufficient for a package to be accepted. With this approach the proposal would also essentially be a checklist, noting that each one is met (with explanation as appropriate).
By contrast, the criteria that the steering committee proposes is that a package is accepted if the reviewers reach consensus. That is, ultimately it is up to the judgement of the reviewers. This approach is augmented with a list of minimum package requirements to make it clearer what level package authors have to aim for.
I'm with the steering committee on this one. We simply don't have enough experience to draw on to be able to come up with an all-encompassing checklist. It may be that such a checklist will de facto emerge when we get the process rolling, but let's not try to fixate one initially. Cheers, /Niklas
On 14/09/2009 23:36, Niklas Broberg wrote:
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Yes.
Concern 1: "The policy document itself is too long and too detailed."
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
What you asked was not a yes-no question :-p But I say split it to a separate page.
Concern 2: "Documentation written for the proposal will get lost." [...]
The alternative that I am proposing is this: in the case that completely new documentation is written for a proposal, there be a presumption that a condition of acceptance is that the new material be republished in the appropriate place and form.
I agree with this view. Having documentation in the proposal itself is nicer than just links, and your proposal deals with the issue that arises.
Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist. Some checklist items would be objective, some subjective, but passing them all would be sufficient for a package to be accepted. With this approach the proposal would also essentially be a checklist, noting that each one is met (with explanation as appropriate).
By contrast, the criteria that the steering committee proposes is that a package is accepted if the reviewers reach consensus. That is, ultimately it is up to the judgement of the reviewers. This approach is augmented with a list of minimum package requirements to make it clearer what level package authors have to aim for.
I'm with the steering committee on this one. We simply don't have enough experience to draw on to be able to come up with an all-encompassing checklist. It may be that such a checklist will de facto emerge when we get the process rolling, but let's not try to fixate one initially.
Well, a first cut at the checklist could be * Do the reviewers agree that the package is suitable for inclusion in the platform? and then we can refine it later :) I basically agree with Duncan's points. The main advantages of a checklist are that it helps organise the discussion and reminds people what points are yet to be agreed upon. The problem with trying to come up with a checklist first is that we don't have a clear idea what should be in it, and we're in danger of spending too much time figuring out what the requirements should be without considering any actual proposals. So let's get started with reviewing package proposals and simultaneously refine the list of requirements. Cheers, Simon
On Tue, 2009-09-15 at 11:33 +0100, Simon Marlow wrote:
I basically agree with Duncan's points. The main advantages of a checklist are that it helps organise the discussion and reminds people what points are yet to be agreed upon.
Right, and we can have a "you might like to think about this" list to remind reviewers when they are making their judgements. Duncan
duncan.coutts:
We would like to get this wrapped up so that we can move on to discussing more guidelines/requirements on packages and indeed to actually proposing some packages.
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Yes. Let's do this.
Concern 1: "The policy document itself is too long and too detailed."
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
* Advantages: policy document looks shorter, less intimidating * Disadvantages: slower to jump back and forth between the policy and the rationale (the two are fully cross-referenced)
If we split it, then the main page would look like this: http://trac.haskell.org/haskell-platform/wiki/AddingPackagesCore and the "[rationale-x.y]" links would point to a separate page.
It really doesn't matter which we pick, we just need to decide and get on with it!
Doesn't matter. We can proceed anyway.
Concern 2: "Documentation written for the proposal will get lost."
The concern here is that if people write some new intro tutorial or something for a package proposal that this might never get integrated into the package's documentation.
Ian's suggested remedy is to force proposals to link to existing documentation rather than letting proposal authors make something new or make a customised variation on existing docs.
The principle that the steering committee seemed to agree on is that the proposal author be given the freedom to present their argument in whatever way they see fit.
The alternative that I am proposing is this: in the case that completely new documentation is written for a proposal, there be a presumption that a condition of acceptance is that the new material be republished in the appropriate place and form. For example this might mean the reviewers require that new intro docs be integrated into the haddock docs or perhaps as a separate tutorial or user guide on the package's homepage. The point is that by attaching it as a condition of the package being accepted, that we ensure that it does actually get done rather than nice docs languishing where no users will find them. The policy already has the facility for conditional acceptance so this just adds a presumed condition.
My bet is that people will polish the documentation and do new releases once under review. This will improve the package. I don't mind if it isn't required.
Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist. Some checklist items would be objective, some subjective, but passing them all would be sufficient for a package to be accepted. With this approach the proposal would also essentially be a checklist, noting that each one is met (with explanation as appropriate).
Acting will reveal to what degree we need a checklist. Let's act, and work this out later. -- Don
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Basically, yes.
Concern 1: "The policy document itself is too long and too detailed."
I must admit, I too did not read it fully, because it was intimidatingly long. In fact, it made me think "phew, thank goodness I didn't want to propose any of my packages for the platform!"
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
Definitely split off the rationale, and similarly, the "how to". (Actually, the latter seems to have already been split out, but there are remaining dangling pointers to the old in-page location.)
* Disadvantages: slower to jump back and forth between the policy and the rationale (the two are fully cross-referenced)
Suggestion: can the rationale be coded to come up in a separate browser window? (Especially if the links ensure that different rationale links come through to the _same_ target window.) I would find it helpful to be able to see policy + rationale side-by-side (although not intermingled).
Concern 2: "Documentation written for the proposal will get lost." ... by attaching it as a condition of the package being accepted, that we ensure that it does actually get done rather than nice docs languishing where no users will find them.
My question is who enforces this, and how?
Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist.
So my suggestion on this complaint is that we go with the policy that the steering committee has proposed and that we move quickly afterwards to discuss more comprehensive package requirements and guidelines.
If the aim is gradually to formalise a checklist, by first examining the packages we want in the platform, then abstracting the characteristics that make them desirable, then it sounds fine. Perhaps the _goal_ of forming an incomplete checklist should be explicit in the first few rounds of package consideration. Ultimately, I think package proposers will benefit from having a clear statement of what they must do to improve their package's "fitness". Regards, Malcolm
On Tue, 2009-09-15 at 09:34 +0100, Malcolm Wallace wrote:
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Basically, yes.
Concern 1: "The policy document itself is too long and too detailed."
I must admit, I too did not read it fully, because it was intimidatingly long. In fact, it made me think "phew, thank goodness I didn't want to propose any of my packages for the platform!"
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
Definitely split off the rationale,
Seems like this is the majority opinion. If it is actually putting people off then that's certainly a good reason to split. The procedure is not very complicated, the policy is just fairly detailed and, I hope, fairly clear. We were trying to anticipate questions and corner cases.
and similarly, the "how to". (Actually, the latter seems to have already been split out, but there are remaining dangling pointers to the old in-page location.)
Fixed the broken link, thanks.
* Disadvantages: slower to jump back and forth between the policy and the rationale (the two are fully cross-referenced)
Suggestion: can the rationale be coded to come up in a separate browser window? (Especially if the links ensure that different rationale links come through to the _same_ target window.) I would find it helpful to be able to see policy + rationale side-by-side (although not intermingled).
I've no idea how to do that in the trac wiki markup. If anyone knows please say so.
Concern 2: "Documentation written for the proposal will get lost." ... by attaching it as a condition of the package being accepted, that we ensure that it does actually get done rather than nice docs languishing where no users will find them.
My question is who enforces this, and how?
The how is by making it a condition of acceptance for the package. The policy describes conditional acceptance: A package is considered as conditionally accepted if, by the discussion deadline, the libraries mailing list reaches consensus to accept it on condition that further minor changes are made. The agreed changes must be made before the package is included in any release. If these changes are made in time for the normal package freeze dates (as set by the release team) then the package is considered as accepted. If the changes cannot be made in time for the immediate major release but are made in time for the subsequent major release then the package is considered as accepted for that subsequent major release and does not need to be re-reviewed. As for who would suggest making it a condition in any particular case, well any reviewer or member of the steering committee. If people complain that it's an unnecessary condition then we point to the section of the policy (as yet unwritten) that says it's a presumed condition. Sound like enough?
Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist.
So my suggestion on this complaint is that we go with the policy that the steering committee has proposed and that we move quickly afterwards to discuss more comprehensive package requirements and guidelines.
If the aim is gradually to formalise a checklist, by first examining the packages we want in the platform, then abstracting the characteristics that make them desirable, then it sounds fine. Perhaps the _goal_ of forming an incomplete checklist should be explicit in the first few rounds of package consideration.
Sure, we can make that clear.
Ultimately, I think package proposers will benefit from having a clear statement of what they must do to improve their package's "fitness".
Indeed. We want to get to the stage where maintainers can do much of the assessment themselves before even getting to the review stage. Duncan
On 15/09/2009 14:44, Duncan Coutts wrote:
On Tue, 2009-09-15 at 09:34 +0100, Malcolm Wallace wrote:
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Basically, yes.
Concern 1: "The policy document itself is too long and too detailed."
I must admit, I too did not read it fully, because it was intimidatingly long. In fact, it made me think "phew, thank goodness I didn't want to propose any of my packages for the platform!"
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
Definitely split off the rationale,
Seems like this is the majority opinion. If it is actually putting people off then that's certainly a good reason to split. The procedure is not very complicated, the policy is just fairly detailed and, I hope, fairly clear. We were trying to anticipate questions and corner cases.
I'd like to chime in in support of the current way it's done. Having it on the same page is better than separate pages because it's easier to navigate. Separate panes or similar would be even better, but that's essentially impossible without deep Trac magic, if at all. If people are worried by the length of the page, then perhaps it could be made clearer at the top of the page that only the first X% of the page is actual policy, the rest is rationale. With a diagram, even :) Cheers, Simon
On Wed, 2009-09-16 at 11:39 +0100, Simon Marlow wrote:
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
Definitely split off the rationale,
Seems like this is the majority opinion. If it is actually putting people off then that's certainly a good reason to split. The procedure is not very complicated, the policy is just fairly detailed and, I hope, fairly clear. We were trying to anticipate questions and corner cases.
I'd like to chime in in support of the current way it's done. Having it on the same page is better than separate pages because it's easier to navigate.
I think we're in the minority however.
Separate panes or similar would be even better, but that's essentially impossible without deep Trac magic, if at all.
I played with inserting html and it works except that trac always wraps it in a div, so it's useless for links.
If people are worried by the length of the page, then perhaps it could be made clearer at the top of the page that only the first X% of the page is actual policy, the rest is rationale. With a diagram, even :)
It does already :-) Who should read this: * People who want to propose a new package; * People reviewing proposed packages; * The Haskell Platform release team Only the first half of this page is essential reading. Duncan
Hello,
On Tue, Sep 15, 2009 at 6:44 AM, Duncan Coutts
On Tue, 2009-09-15 at 09:34 +0100, Malcolm Wallace wrote:
Suggestion: can the rationale be coded to come up in a separate browser window? (Especially if the links ensure that different rationale links come through to the _same_ target window.) I would find it helpful to be able to see policy + rationale side-by-side (although not intermingled).
I've no idea how to do that in the trac wiki markup. If anyone knows please say so.
How about putting the rationale on a separate page, in the usual way, and if people want to view it in a new window or a separate tab, then they can use the browser feature (e.g., right mouse button in Firefox). Not a perfect but simple. :-) -Iavor
On Mon, 2009-09-14 at 22:54 +0100, Duncan Coutts wrote:
We would like to get this wrapped up so that we can move on to discussing more guidelines/requirements on packages and indeed to actually proposing some packages.
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Thanks to the people who responded. I hope we're done now. I've made a couple changes based on the feedback which I explain below. Are there any unresolved concerns? If not then we'll declare the policy adopted and we can move on to discuss package proposals and extra package requirements/guidelines. A quick review of the points I asked people to consider.
Concern 1: "The policy document itself is too long and too detailed."
Quick poll: should we split the rationale into a separate page or keep it on the same page? Yes or no.
Most people thought yes split it. That's now done. http://trac.haskell.org/haskell-platform/wiki/AddingPackages http://trac.haskell.org/haskell-platform/wiki/AddingPackages/Rationale
Concern 2: "Documentation written for the proposal will get lost."
I added a paragraph to the "proposal content" section: If new documentation is written for the proposal that would be useful in future for users of the package (e.g. an introduction or tutorial) then we wish to ensure that it is indeed made available to users. In the case that useful new documentation is written, reviewers are strongly encouraged to make it a condition of acceptance that the new material be republished in the appropriate place and form (e.g. on a package's homepage or integrated into API reference documentation).
Concern 3. The more major and general complaint Ian has is that he believes the criteria for acceptance should be essentially a checklist.
So my suggestion on this complaint is that we go with the policy that the steering committee has proposed and that we move quickly afterwards to discuss more comprehensive package requirements and guidelines.
I've not done anything here. Most people who responded seemed to agree with this suggestion. Duncan
On Thu, 2009-09-17 at 20:35 +0100, Duncan Coutts wrote:
On Mon, 2009-09-14 at 22:54 +0100, Duncan Coutts wrote:
We would like to get this wrapped up so that we can move on to discussing more guidelines/requirements on packages and indeed to actually proposing some packages.
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
Thanks to the people who responded. I hope we're done now. I've made a couple changes based on the feedback which I explain below.
Are there any unresolved concerns?
If not then we'll declare the policy adopted and we can move on to discuss package proposals and extra package requirements/guidelines.
It's been over 10 days since the call for consensus, we've had no objections and several people have been broadly supportive. I think we can declare this adopted. http://trac.haskell.org/haskell-platform/wiki/AddingPackages Duncan
I'm generally in favour, but one thing puzzled me: if some documentation (e.g. major design decisions and examples of API use) are required to be integrated before final acceptance, and are a good idea anyway, why not expect them to be integrated when the package is proposed?
ross:
I'm generally in favour, but one thing puzzled me: if some documentation (e.g. major design decisions and examples of API use) are required to be integrated before final acceptance, and are a good idea anyway, why not expect them to be integrated when the package is proposed?
We expect review will lead to additional documentation generation.
On Thu, Sep 17, 2009 at 02:22:34PM -0700, Don Stewart wrote:
ross:
I'm generally in favour, but one thing puzzled me: if some documentation (e.g. major design decisions and examples of API use) are required to be integrated before final acceptance, and are a good idea anyway, why not expect them to be integrated when the package is proposed?
We expect review will lead to additional documentation generation.
Sorry, I meant that the proposal content lists "major design decisions should be described and motivated" and "For library packages, an example of how the API is intended to be used should be given". I would have expected that these things should already be integrated into the package before it is proposed.
On Mon, Sep 14, 2009 at 2:54 PM, Duncan Coutts
So far we have had a few people send in comments (thanks particularly to Ian and Simon) but a few more would not go amiss, even if it's just "yes".
"Yes" :-) Thanks for reworking the pages to be shorter. That was part of what kept me from reading them. Concern 3. The more major and general complaint Ian has is that he
believes the criteria for acceptance should be essentially a checklist. Some checklist items would be objective, some subjective, but passing them all would be sufficient for a package to be accepted.
A checklist divided into "MUST"/"SHOULD" sections is very valuable, independent of other considerations. That's what we use for Fedora package reviews, and it very much helps us to ensure that we don't skip anything.
participants (8)
-
Bryan O'Sullivan -
Don Stewart -
Duncan Coutts -
Iavor Diatchki -
Malcolm Wallace -
Niklas Broberg -
Ross Paterson -
Simon Marlow