Yeah there are many interlocking issues here. I've been thinking about writing one of those ecosystem proposals about this. Here's some observations about the status quo for now. - Frequent releases are a really nice for feature for the community. I know I, for one, would be less motivated to work on GHC if it would take a year in some case for my contributions to trickle back to my day job. Rust releases every 6 *weeks*. I see no reason why we shouldn't aim for that. - The GitLab move and switch to CI have made GHC development more accessible and, temporary Marge Bot issues aside, better scaling. I am immensely grateful for those that put in the work for that every MR I write. Going back to annual releases would feel like a regression, cancelling out some of those gains. - Base really is a PVP mess. Only the messy guts actually make sense to pair with GHC releases, a breaking change every time. The "pure" bits would naturally have *fewer* breaking releases, but *more* non-breaking releases. - AMP-like proposals could probably move faster if they weren't tied to GHC releases. I.e. if multiple GHC releases supported multiple class hierarchies, people could move at their own pace to some extent, and we could support faster *and* slower migrations. - I hope GHC can move in a more modular direction, which by definition means farming off work to libraries. In other words, I think GHC should ultimately have the same degree of code reuse as any other Haskell project. But the current experience with submodules is excruciating, so no one dealing with GHC every day rightfully has any appetite for that. It's a pity because once it's done I think it would be *less* work for the core GHC team, there's just the awkward transition out of today's monolithic local maximum. So all this needs leads me to conclude a few things - Base must be broken up, so it can be released more often than GHC. Base would continue to exist reexport code in other libraries exclusively. New users, executables, and legacy code can continue to depend on base as before, but libraries are encouraged to adopt more fine-grained dependencies on the underlying libraries. - the kicker: Any library that wishes to be included in GHC / base *must share maintainership* with the GHC release team / libraries committee, respectively/./ That means those groups can release things whenever they want to unblock themselves. // It's fairly clear-cut to me: having your library being included in GHC or base is a big honor, and so you should not be able to hold up the release process of these important projects unilaterally. If you want to keep full custody of your baby, that's fine, but it's worth not the precious time of the GHC/base developers to use it then. [Aim for, say, Haskell Platform inclusion rather than base inclusion.] Granted, GHC, unlike Base, there is the alternative to use Cabal's new support for multiple versions of a package in the same package DB. But until will have private dependencies that would place some probably unpopular constraints on consumers of GHC-the-library. If GHC does migrate from ~10 to ~100 dependencies, maybe it might make sense to float that as an alternative, but seeing that is many GHC releases away no matter what, I think it's best to start with the shared maintainership policy.  John On 7/2/19 8:36 AM, Henning Thielemann wrote:

On Tue, 2 Jul 2019, Simon Peyton Jones via Libraries wrote:

...

 1. What should GHC’s release cadence be? For example, would annual be     better than semi-annual?  Herbert and     Michael both think so, and they know a thing or two.   I’d love to     hear from more people on this question.

Annual is better for me. I do not need the latest and greatest GHC features immediately. I am happy to get bugfixes quickly and easily via hvr's ghc ppa.

However, I'd prefer that libraries are separated from GHC and 'base' would be split.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

The biggest problems are for packages like containers, that are not only used by GHC but also exposed to users through the GHC API. These libraries aren't part of GHC or base, but pretty much have to move in lock step. On Tue, Jul 2, 2019, 12:19 PM Simon Peyton Jones via Libraries < libraries@haskell.org> wrote:

| - the kicker: Any library that wishes to be included in GHC / base *must | share maintainership* with the GHC release team / libraries committee, | respectively/./ That means those groups can release things whenever they | want to unblock themselves.

That seems very plausible to me. But "sharing maintainership" could be limited to:

The GHC release team is free to make a minor (patch-level) release of an existing released version of the library L, embodying any changes necessary to make the library work with the new version of GHC

I think that is all that's needed. In general GHC should not rely on a bleeding-edge release of a library. But it's not uncommon for tiny changes to be needed, and by sharing maintainership, the library author would not need to be bothered about making and releasing those changes.

It's worth noting that it's not so much "libraries that want to be included in GHC", but rather "libraries that GHC's authors would like to import in GHC's own source code, rather than re-implementing that functionality (which would be silly)".

Simon

| -----Original Message----- | From: John Ericson | Sent: 02 July 2019 17:00 | To: Henning Thielemann ; Simon Peyton Jones | | Cc: ghc-devops-group@haskell.org; Michael Snoyman < michael@fpcomplete.com>; | David Terei ; Ashley Yakeley ; | Herbert Valerio Riedel ; Haskell Libraries | (libraries@haskell.org) | Subject: Re: [GHC DevOps Group] Making GHC's fast release cadence work | | Yeah there are many interlocking issues here. I've been thinking about | writing one of those ecosystem proposals about this. Here's some | observations about the status quo for now. | | - Frequent releases are a really nice for feature for the community. I know | I, for one, would be less motivated to work on GHC if it would take a year | in some case for my contributions to trickle back to my day job. | Rust releases every 6 *weeks*. I see no reason why we shouldn't aim for | that. | | - The GitLab move and switch to CI have made GHC development more | accessible and, temporary Marge Bot issues aside, better scaling. I am | immensely grateful for those that put in the work for that every MR I | write. Going back to annual releases would feel like a regression, | cancelling out some of those gains. | | - Base really is a PVP mess. Only the messy guts actually make sense to | pair with GHC releases, a breaking change every time. The "pure" bits would | naturally have *fewer* breaking releases, but *more* non-breaking releases. | | - AMP-like proposals could probably move faster if they weren't tied to GHC | releases. I.e. if multiple GHC releases supported multiple class | hierarchies, people could move at their own pace to some extent, and we | could support faster *and* slower migrations. | | - I hope GHC can move in a more modular direction, which by definition | means farming off work to libraries. In other words, I think GHC should | ultimately have the same degree of code reuse as any other Haskell project. | But the current experience with submodules is excruciating, so no one | dealing with GHC every day rightfully has any appetite for that. | It's a pity because once it's done I think it would be *less* work for the | core GHC team, there's just the awkward transition out of today's | monolithic local maximum. | | So all this needs leads me to conclude a few things | | - Base must be broken up, so it can be released more often than GHC. | Base would continue to exist reexport code in other libraries exclusively. | New users, executables, and legacy code can continue to depend on base as | before, but libraries are encouraged to adopt more fine-grained | dependencies on the underlying libraries. | | - the kicker: Any library that wishes to be included in GHC / base *must | share maintainership* with the GHC release team / libraries committee, | respectively/./ That means those groups can release things whenever they | want to unblock themselves. | // | | It's fairly clear-cut to me: having your library being included in GHC or | base is a big honor, and so you should not be able to hold up the release | process of these important projects unilaterally. If you want to keep full | custody of your baby, that's fine, but it's worth not the precious time of | the GHC/base developers to use it then. [Aim for, say, Haskell Platform | inclusion rather than base inclusion.] | | Granted, GHC, unlike Base, there is the alternative to use Cabal's new | support for multiple versions of a package in the same package DB. But | until will have private dependencies that would place some probably | unpopular constraints on consumers of GHC-the-library. If GHC does migrate | from ~10 to ~100 dependencies, maybe it might make sense to float that as | an alternative, but seeing that is many GHC releases away no matter what, I | think it's best to start with the shared maintainership policy. | | John | | On 7/2/19 8:36 AM, Henning Thielemann wrote: | > | > On Tue, 2 Jul 2019, Simon Peyton Jones via Libraries wrote: | > | >> 1. What should GHC’s release cadence be? For example, would annual be | >> better than semi-annual? Herbert and | >> Michael both think so, and they know a thing or two. I’d love to | >> hear from more people on this question. | > | > Annual is better for me. I do not need the latest and greatest GHC | > features immediately. I am happy to get bugfixes quickly and easily | > via hvr's ghc ppa. | > | > However, I'd prefer that libraries are separated from GHC and 'base' | > would be split. | > | > _______________________________________________ | > Libraries mailing list | > Libraries@haskell.org | > | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haske | ll.org%2Fcgi- | bin%2Fmailman%2Flistinfo%2Flibraries&data=02%7C01%7Csimonpj%40microsoft | .com%7C6ae01104c4b345984a0208d6ff06464e%7C72f988bf86f141af91ab2d7cd011db47% | 7C1%7C1%7C636976799775801107&sdata=YEf6lczTdY9Mp%2B6lFHzCdwcagiQWZ3VgxI | QTfqH5ZXk%3D&reserved=0 _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

It seems to mostly be an issue for stack, which wants a fully consistent package set. On Tue, Jul 2, 2019, 12:54 PM Ben Gamari wrote:

David Feuer writes:

...

The biggest problems are for packages like containers, that are not only used by GHC but also exposed to users through the GHC API. These libraries aren't part of GHC or base, but pretty much have to move in lock step.

I'm not sure I understand why this is so. Yes, install plans involving the GHC library are forced to use the same version of containers that GHC uses, but I would think that this is not the common case.

Assuming most people aren't linking against the GHC library then I don't see the harm in GHC staying a bit behind upstreams like containers.

Cheers,

- Ben

The biggest problems are for packages like containers, that are not only used by GHC but also exposed to users through the GHC API. These libraries aren't part of GHC or base, but pretty much have to move in lock step.

fwiw, this what https://mail.haskell.org/pipermail/ghc-devs/2017-July/014424.html aims to address and would help to reduce this artificial boot-library version pinning for consumers. Moritz has been pushing this effort further via https://gitlab.haskell.org/ghc/ghc/merge_requests/490 but it didn't make it into GHC 8.8 yet.

Ashley Yakeley writes:

For the time library, I would rather the GHC team make a pull request first, and let me do the release. At the very least I want to know what's going on and what the needs are.

As long as the review turnaround is reasonably fast (which it always has been in the case of time; thank you!), I am perfectly content with this. Cheers, - Ben

The GHC release team is free to make a minor (patch-level) release of an existing released version of the library L, embodying any changes necessary to make the library work with the new version of GHC

Yeah I like that limitation. It is good to make my proposal less draconian :) Happy to hear that's what you wanted all along Ben! Ashley:

For the time library, I would rather the GHC team make a pull request first, and let me do the release. At the very least I want to know what's going on and what the needs are.

Adding to what Ben said, I think the way submodules work today mean that unmerged (e.g. just in the GHC fork) changes to a submodule block a PR? So only in repos where the PR happens to live in the same repo as master can GHC use the PR branch and still pass CI, and the usual case is the PR must be merged before GHC relies on it. I hope in the majority of cases these GHC-driven releases would just take stuff that is on master but unreleased, and therefore already reviewed the normal way. David:

It seems to mostly be an issue for stack, which wants a fully consistent package set.

Stackage should be comfortable building GHC itself with a newer containers then, I think. (And likewise for anyone that needs a newer containers in their build of the GHC library.) John On 7/2/19 12:46 PM, Ben Gamari wrote:

Simon Peyton Jones via Libraries writes:

...

| - the kicker: Any library that wishes to be included in GHC / base *must | share maintainership* with the GHC release team / libraries committee, | respectively/./ That means those groups can release things whenever they | want to unblock themselves.

That seems very plausible to me. But "sharing maintainership" could be limited to:

I think that is all that's needed. In general GHC should not rely on a bleeding-edge release of a library. But it's not uncommon for tiny changes to be needed, and by sharing maintainership, the library author would not need to be bothered about making and releasing those changes.

Yes, this is precisely what I was trying to articulate in my original email (although failed to do so clearly). We would not want to be responsible for managing major releases; merely incremental patch-level releases to patch things up to be shippable with GHC.

Cheers,

- Ben

For what it's worth, we do provide nightly builds as a natural result of our CI infrastructure. Perhaps we should draw more attention to this fact. Cheers, Ben On July 6, 2019 2:21:33 AM EDT, Henning Thielemann wrote:

On Tue, 2 Jul 2019, John Ericson wrote:

...

Yeah there are many interlocking issues here. I've been thinking about writing one of those ecosystem proposals about this. Here's some observations about the status quo for now.

- Frequent releases are a really nice for feature for the community. I know I, for one, would be less motivated to work on GHC if it would take a year in some case for my contributions to trickle back to my day job. Rust releases every 6 *weeks*. I see no reason why we shouldn't aim for that.

In the past we had nightly builds. They are not quite releases but if provided as, say, Debian packages it would be easy to install and upgrade them. _______________________________________________ Ghc-devops-group mailing list Ghc-devops-group@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.