On Wed, Jun 27, 2012 at 11:42:24AM -0700, Johan Tibell wrote:

On Wed, Jun 27, 2012 at 8:12 AM, Ian Lynagh wrote:

...

On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote:

...

I just want to see things changed. :)

We're happy to try to improve things, but I'm not sure what change you want exactly.

I want GHC to stop releasing other people's code, unless they've explicitly asked GHC to do so. Other than that, you can do what you want.

Here's an honest question: If a GHC release needs an unreleased change in one of the libraries, and the maintainer (for whatever reason) is not responding to e-mails, should the GHC release be held up indefinitely? If the answer is "yes", then perhaps we should go back to community maintainership for all the libraries that GHC ships with. As well as being entities in their own right, those libraries are also part of larger systems (ghc, and perhaps also other Haskell implementations). If the answer is "no", then there are going to be times when we need to ship GHC with a version of a library that is not yet released. With the best will in the world, there are always going to be people who are swamped by real life, people on vacation, or even people who unbeknownst to us have died. But all that is really tangential to the main issue: even if the answer to the above question is "no", that does not mean that we need to routinely release libraries maintained by active upstreams. If upstream is responsive, then we can discuss with them what code to use and what releases need to be made. The original e-mail was intended to be the first in that discussion. Perhaps we phrased it badly, or perhaps you have bad memories of previous mistakes or of previous systems of releasing, but all we were trying to do is to find out what code we should set up the new stable branch to use. We're happy to discuss concrete changes to the way things work. But (a) I don't think any change is necessary if your goal is for us to not make containers releases, and (b) GHC is a complex beast, and it already take lots of work over a period of weeks (or sometimes months) to get a release out. I'm keen to keep the process as lightweight as possible. Thanks Ian

On 6/27/12 6:06 PM, Johan Tibell wrote:

This is not a theoretical issue. We have had all of the following problems happen in the past due to the current process:

* patches never making it upstream * releases of libraries without knowledge of the maintainer (who finds out by finding a new version of his/her package on Hackage.) * packages being released by GHC never ending up on Hackage, causing build breakages for people who use older GHCs and can't install the packages as they aren't available on Hackage.

At the almost certain risk of stepping into a discussion I don't fully understand, let me step into a discussion I almost certainly don't fully understand :-) It seems to me that all these issues could be solved by having a member of the GHC team an assistant co-maintainer on packages that GHC depends on, and acting as such in a responsible manner, and in addition, having all packages bundled with GHC releases drawn from hackage releases. This is to say, that ghc-originated patches necessarily get committed to the upstream repo, because they must be there to be released on hackage, that ghc-originated patches necessarily get released to hackage because they must be there for GHC releases to draw on them, and maintainers necessarily know what gets released to hackage because they communicate well with co-maintainers. This is different than community ownership -- packages are still owned and maintained by individuals. However, by having a ghc assistant co-maintainer, there's a specified conduit for collaboration. This is also different from the current situation, because a co-maintainer may only work on issues for GHC release compatibility, but they are acting as someone with direct responsibility for the package and as part of the team that "owns" the package. Problems of collaboration aren't magiced away by this sort of change of titles, of course, but when there are problems of communication and collaboration, they can now be understood as and treated as problems between primary and secondary package maintainers. I hope this makes some semblance of sense. Cheers, Gershom

| > Has maintainer's not being responsive been a problem for GHC in the | > past? | | Yes. Some of the upstreams respond so fast that it makes my head spin, | while others often either don't respond or continually promise to get to | things soon. (again, these are good, well-meaning people, who do a lot | for the community). The obvious solution, as someone else pointed out, is for someone at GHC HQ (perhaps Ian) to be a co-maintainer of these critical dependencies -- except perhaps for libraries whose maintainers are the "make your head spin" responsive kind. This would of course need the maintainer to trust the GHC person to push patches and make releases in sync with GHC. But my guess is they'd be willing. After all, we're all on the same side here! Would that cut the Geordian knot? Simon

Hi Ian, On Thu, Jun 28, 2012 at 6:26 AM, Ian Lynagh wrote:

You didn't give a clear answer to my question. Am I right in thinking that your answer would be "Yes, the GHC release should be delayed indefinitely"?

I did answer it, just not with a "yes" or "no" as it's a false dichotomy. I gave you the 4 options that I think are reasonable. Perhaps someone could think of others, but those are the standard ones typically used by open source communities.

(or at least, for long enough for the maintainer to be declared MIA)

Something reasonable. Perhaps a few weeks to a month. Since this hasn't actually ever been a problem from what I can tell it doesn't matter much at this point. I had a quick look at the list of packages in question (http://hackage.haskell.org/trac/ghc/wiki/Repositories) and to my knowledge all these maintainers are around (and typically well-known contributors in the community.)

...

I think the problem is one of misunderstanding how the process of managing dependencies ought to work (and how it works elsewhere.) "We must release a new version of so-and-so lib because we made such-and-such change" is wrong. Upstream changes (i.e. to GHC deps) ought to happen before downstream releases of dependent code (i.e. GHC.)

This is actually the main reason that the situation between GHC and the libraries it uses is different to most other packages, both within Haskell and without:

It is true that ghc depends on (for example) containers; but containers also depends on base, and base/ghc are so intertwined that they are essentially the same package (at least, I don't think you're suggesting that we should make separate base and ghc releases).

That is what I mean by them being part of the same system.

For example, I recently removed the 'catch' export from Prelude, and this required corresponding changes in Cabal, Win32 and haskeline. It's not possible to make the change in the base library without making the corresponding changes, or the GHC build would break, and there's no reason the maintainers of the other packages would make the change if I didn't ask them to.

A more mundane example is library dependencies. If we make a change in filepath that requires bumping its major version, then we need libraries such as Cabal to relax their dependency on filepath or, again, the GHC build would break.

Go ahead and make those changes to your local clones. That's reasonable. That in no way forces you to make releases of those packages. You have the time from you make the changes to the next GHC release to get the changes pushed upstream and released.

Yes. Some of the upstreams respond so fast that it makes my head spin, while others often either don't respond or continually promise to get to things soon. (again, these are good, well-meaning people, who do a lot for the community).

Have you tried pointing out that this is a problem for GHC and perhaps suggest that they let you make a release on their behalf?

...

I believe this is the first time I've seen an email of this kind from GHC HQ.

Generally these mails are all directly to maintainers. They're generally longer than this, but in essence it normally goes something like    mail 1: "Could you take a look at this patch please?"    mail 2: "Did you have a minute to look at that patch?"    mail 3: "I think the patch is good. Would it help if I pushed it for you?"    mail 4: "This is blocking other things, so I'd like to push it.             Please let me know within a week if you object" (there may be multiple mail 2s, and mail 3 sometimes gets an affirmative response).

Once this has happened a few times, we tend to suggest switching to a system where we just push by default, without the need for the mails and the delay (in fact, more-or-less what Gershom suggested).

If the maintainer is fine with such a solution this is of course fine. But what if they're not? Note that this has nothing to do with releasing from HEAD, which GHC has typically done in the past. Such patches typically only require a patch release. P.S. The Haskell Platform is most likely moving to the complete opposite approach of what GHC uses; a call for version bumps will go out some time before the release and if maintainers don't ask to have their package version bumped, it will stay at the same version used for the last release. Cheers, Johan