It's not obvious to me that this is a good idea. Users already complain that "upgrading GHC broke my code", when in practice this frequently means "when upgrading GHC I used new boot library versions with incompatible APIs that broke my code". So having a policy of introducing more frequent API changes (and making it harder to switch between GHC releases in a single release series) needs clear justification IMHO. (Of course in the long term we should make it easier for users to pin boot library APIs while upgrading GHC independently, but we're not there yet.) I think that by default, it makes sense for a new minor GHC release to keep the same API versions of boot libraries, but bump to the latest super-minor component (i.e. for an A.B.C.D version, keep A.B.C the same but bump D). If a boot library fixes a bug that is critical enough to require bumping the version distributed with GHC, ideally there would be an API-compatible release of the boot library. Of course that might not always be feasible, and it is ultimately a judgement call for the GHC maintainers as to which boot library versions are best to ship. Cheers, Adam On 26/11/2024 22:27, Hécate via ghc-devs wrote:

From the cabal perspective this great news, as this will allow the feedback cycle between our two projects to be shorter. Glad to see the GHC team is considering this!

Cheers, Hécate

Le 26/11/2024 à 21:14, Ben Gamari a écrit :

...

tl;dr. We propose that GHC more aggressively bump its boot library         dependencies.

Hello all,

Historically, GHC minor releases have been quite conservative in bumping boot library versions, generally defaulting to retaining the versions used in the GHC-x.y.1 release unless

  * a request is made by a boot library maintainer, or   * we discover an issue which warrants a bump

The motivation for this policy is both to

  * minimize the potential for regressions in correctness and     performance, and

  * reduce the potential for end-user breakage due to interface changes     as even minor releases can result in considerable impact (e.g.     #25411, #22633)

While this policy has served us well, its conservative nature demands the vigilence of both GHC and upstream maintainers to ensure that bumps that are truly necessary do not fall through the cracks.

As the flaws of this process have been more apparent recently (e.g. #24597), we have done a bit of reflection on how we might improve the status quo. Concretely, I would like feedback on the adopting the following policy going forward:

...

Unless guidance is provided otherwise by a library maintainer, a GHC x.y.z release will attempt to ship the newest minor version of each boot libray in the same major series shipped with GHC x.y.1. I believe this policy would leave less room for human error and open opportunities for automated checking. On the other hand, the more aggressive bumping of submodules may mean that users are hit with more (usually minor) compatibility issues when moving between minor GHC releases.

How do others feel about this? We are particularly interested to hear from boot library maintainers but feedback from packagers and users is also very much welcome.

Cheers,

- Ben

-- Adam Gundry, Haskell Consultant Well-Typed LLP, https://www.well-typed.com/ Registered in England & Wales, OC335890 27 Old Gloucester Street, London WC1N 3AX, England

Hi, Do we have concrete evidence that minor bumps in a boot library have caused breakage in the past? (That would of course be a bug in the boot library according to the PVP.) I would think that such breakage is even less of an issue for a new GHC minor release (say GHC 9.8.4). The hackage ecosystem is likely prepared to compile with the GHC 9.8 series at this point, so it should be possible to detect such breakages by compiling stackage/hackage and running testsuites. That is, I would think it is harder to anticipate unforeseen boot library breakage for a new major release of GHC than it is to anticipate breakage for a minor GHC release. Besides, the only reliable way to prevent such breakage is not to bump any boot library *at all*. If a boot library breaks on a minor bump, it is just as likely that it breaks on a super-minor bump. So bumping minor versions of boot library sounds reasonable to me, barring concrete evidence that minor bumps have caused problems in the past (which again means there was a bug in a boot library). Sebastian ------ Originalnachricht ------ Von "Tom Ellis" An ghc-devs@haskell.org Datum 27.11.2024 10:57:57 Betreff Re: Feedback on potential change in boot library

Thanks Adam,

I'm not too familiar with the GHC-side requirements here, but I wonder if we can approach this issue from a different direction: is there something we can do to help boot libraries avoid breaking changes?

Tom

On Wed, Nov 27, 2024 at 08:02:10AM +0000, Adam Gundry wrote:

...

It's not obvious to me that this is a good idea. Users already complain that "upgrading GHC broke my code", when in practice this frequently means "when upgrading GHC I used new boot library versions with incompatible APIs that broke my code". So having a policy of introducing more frequent API changes (and making it harder to switch between GHC releases in a single release series) needs clear justification IMHO.

(Of course in the long term we should make it easier for users to pin boot library APIs while upgrading GHC independently, but we're not there yet.)

I think that by default, it makes sense for a new minor GHC release to keep the same API versions of boot libraries, but bump to the latest super-minor component (i.e. for an A.B.C.D version, keep A.B.C the same but bump D). If a boot library fixes a bug that is critical enough to require bumping the version distributed with GHC, ideally there would be an API-compatible release of the boot library. Of course that might not always be feasible, and it is ultimately a judgement call for the GHC maintainers as to which boot library versions are best to ship.

On 26/11/2024 22:27, Hécate via ghc-devs wrote:

...

From the cabal perspective this great news, as this will allow the feedback cycle between our two projects to be shorter. Glad to see the GHC team is considering this!

Cheers, Hécate

Le 26/11/2024 à 21:14, Ben Gamari a écrit :

...

tl;dr. We propose that GHC more aggressively bump its boot library dependencies.

Hello all,

Historically, GHC minor releases have been quite conservative in bumping boot library versions, generally defaulting to retaining the versions used in the GHC-x.y.1 release unless

* a request is made by a boot library maintainer, or * we discover an issue which warrants a bump

The motivation for this policy is both to

* minimize the potential for regressions in correctness and performance, and

* reduce the potential for end-user breakage due to interface changes as even minor releases can result in considerable impact (e.g. #25411, #22633)

While this policy has served us well, its conservative nature demands the vigilence of both GHC and upstream maintainers to ensure that bumps that are truly necessary do not fall through the cracks.

As the flaws of this process have been more apparent recently (e.g. #24597), we have done a bit of reflection on how we might improve the status quo. Concretely, I would like feedback on the adopting the following policy going forward:

...

Unless guidance is provided otherwise by a library maintainer, a GHC x.y.z release will attempt to ship the newest minor version of each boot libray in the same major series shipped with GHC x.y.1. I believe this policy would leave less room for human error and open opportunities for automated checking. On the other hand, the more aggressive bumping of submodules may mean that users are hit with more (usually minor) compatibility issues when moving between minor GHC releases.

How do others feel about this? We are particularly interested to hear from boot library maintainers but feedback from packagers and users is also very much welcome.

---

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

On 27.11.2024 20.21, Ben Gamari wrote:

"Sebastian Graf" writes:

...

Hi,

Do we have concrete evidence that minor bumps in a boot library have caused breakage in the past? (That would of course be a bug in the boot library according to the PVP.)

Yes, I cited two such examples that came to mind in the original message. These weren't hard to find, despite boot library bumps in GHC minor releases being historically rather rare.

Contrary to what one might be tempted to conclude, PVP-compliant, non-buggy minor bumps can cause breakage for downstream users. The reason for this is typically the introduction of new exports. For instance, in #25411 we found that the bump of `deepseq` in 9.8.3 introduced a new export (`Unit`) from `Control.DeepSeq`. This is, of course, a minor change under the PVP but renders previously working modules unbuildable due to ambiguous occurrences.

Perhaps in an ideal world all `import`s would be either `qualified` or include an explicit import list but, sadly, this is not the world in

PVP suggest to use qualified or explicit import lists. https://wiki.haskell.org/Import_modules_properly Only then "A client MAY specify that they are insensitive to additions to the API by allowing a range of C values, e.g. build-depends: base >= 2.1.1 && < 2.2." That's right where in the PVP: https://pvp.haskell.org/ point 5.

which we live. Consequently, the principle difference between a major and minor bump is that the breakage from the latter is usually "easy" to fix. However, it is breakage nevertheless and even easy breakage can cost significant time when migrating large code-bases.

No. PVP clearly says which kind of changes MUST be major bumps, which MUST be at least minor bumps etc. There is very little of "interpretation". But in addition to that, PVP is contract between library authors and consumers. It also says how consumer should specify their dependencies. BOTH have to do their parts, if consumers don't follow PVP, it's irrelevant what library authors do. "The goal of a versioning system is to inform clients of a package of changes to that package that might affect them, and to provide a way for clients to specify a particular version or range of versions of a dependency that they are compatible with." Otherwise we can simply treat all changes as breaking changes and throw any semantic* version policy into a garbage bin. Make people read the changelogs. * Semantic as in "assigns semantics to version number changes". To make this even more concrete: Minor GHC releases couldn't bump minor `base` either. If GHC-9.12.1 ships with `base-4.21.0.0`, then will GHC-9.12.2 and later too. Significant amount of ecosystem does `base <5`, so it's arguable that we cannot introduce any (breaking) change, like removing modules from `base`, without going past version 5 (and before people start to do `base < 1000` bounds, which I think i have seen). So to repeat: if clients don't care to specify their compatibility correctly, have courage to say it's their fault their code broke.

...

I would think that such breakage is even less of an issue for a new GHC minor release (say GHC 9.8.4). The hackage ecosystem is likely prepared to compile with the GHC 9.8 series at this point, so it should be possible to detect such breakages by compiling stackage/hackage and running testsuites.

Indeed head.hackage identified the breakage in 9.8.3. However, we decided to move ahead regardless since the maintainer requested it. The policy proposed in the head of thread will make this sort of breakage considerably more likely to appear in future releases.

...

Besides, the only reliable way to prevent such breakage is not to bump any boot library *at all*. If a boot library breaks on a minor bump, it is just as likely that it breaks on a super-minor bump.

I am not sure I agree. While the PVP is silent on this matter, IMHO patch-level bumps should not touch external interfaces at all, only implementation and packaging. This makes it much less likely for breakage to manifest.

Cheers,

- Ben

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

On Wed, Nov 27, 2024 at 08:59:51PM +0200, Oleg Grenrus wrote:

On 27.11.2024 20.49, Oleg Grenrus wrote:

...

PVP suggest to use qualified or explicit import lists. https://wiki.haskell.org/Import_modules_properly

Only then "A client MAY specify that they are insensitive to additions to the API by allowing a range of C values, e.g. build-depends: base >= 2.1.1 && < 2.2."

That's right where in the PVP: https://pvp.haskell.org/ point 5.

I forgot to say, that if clients are not insensitive to additions to the API, i.e. they do use open imports, the PVP dictates to use strict bounds:

4. Client specification. Hence A.B.C uniquely identifies the API. A client that wants to specify that they depend on a particular version of the API can specify a particular A.B.C and be sure of getting that API only. For example, build-depends: mypkg >= 2.1.1 && < 2.1.2.

which no-one does, but that's their fault.

I'd argue that a design mistake in Haskell. We should not have open imports, nor implicit Prelude. The default thing (import Foo) does the wrong thing. Compare to Rust where "use foo" isn't a wildcard import. https://www.lurklurk.org/effective-rust/wildcard.html

I agree with everything you've said in this thread, Oleg. Digressing slightly, I think there's scope for a new language extension ("ImportQualifiedDefault"?) that makes import Foo import Bar unqualifed mean what import Foo qualified import Bar currently means. Tom

On Wed, 27 Nov 2024 at 11:58, Tom Ellis < tom-lists-haskell-cafe-2023@jaguarpaw.co.uk> wrote:

Thanks Adam,

I'm not too familiar with the GHC-side requirements here, but I wonder if we can approach this issue from a different direction: is there something we can do to help boot libraries avoid breaking changes?

Tom

Breaking changes caused by new exports could be mitigated with a library package or a GHC plugin with such auto refactoring functionality : * Uniquely qualify (i.e., leave no different modules similarly qualified in the same module) each imported module and each imported symbol. This refactoring should be possible in code which already compiles. It would not require syntax / grammar changes. Something similar to smuggler2 package.

I'd like to add a third option to Andreas' list. It seems the current policy is both too lax and too strict: Minor versions are never incorporated, but when they are, it is despite the fact they are known to create breakage. This is what I've gathered from Ben's comment that head.hackage caught the 9.8.3 breakage. So the third option: ## Always bump minor versions, unless they are known to cause breakage. (1) "Known to cause breakage" is subject to further specification, but at the very least, head.hackage can already be used for this purpose. Tangentially, regardless of hard-and-fast rules, exceptions should be allowed if justified. This is also subject to further specification, but again at the very least, justification should be a mix of release manager's discretion, user needs, and library maintainer wishes. This might be stating the obvious, but it's only obvious if it's actually stated. On Thu, 28 Nov 2024 at 14:51, Andreas Klebinger via ghc-devs < ghc-devs@haskell.org> wrote:

...

Breaking changes caused by new exports could be mitigated with a library package or a GHC plugin with such auto refactoring functionality :

All of this is true. But from my side the worry here is less the effort required to fix such issues. Rather it's that users need to take any action at all when upgrading from ghc-9.X.Y to ghc-9.X.Y+1 in order for their projects to continue to compile.

Our goals here are: * Ensure bug fix releases of libraries are shipped with GHC without requiring maintainer attention. * Try to keep breakage for users minimal.

In concept the idea of only bumping sumperminor versions makes sense, as those typically don't contain any breaking changes. But after looking through some of the changelogs of boot libraries it seems like *many* bug fixes are currently released as minor releases. As a consequence I don't see applying this policy only to superminor versions as productive unless the release practices of boot libraries change significantly which seems unlikely.

Similarly in theory we can disregard concerns about breakage with a reference to best practices or paint users as responsible for open imports. But that won't change the fact that it's common practice to use open imports and as a result there will be breakage from such changes.

So this leaves us with:

## Always bump minor versions [unless there are objections].

This ensures bug fixes make it into releases reliably with little maintainer burden, but will result in breakage for some projects when upgrading from ghc-X.Y.Z to ghc-X.Y.Z+1.

Breakage from such issues tends to be immediately observed as tied to a ghc upgrade, and as such we tend to more reliably hear of it.

## Never bump minor versions [unless explicitly requested by maintainers of the library, or deemed necessary by ghc maintainers].

This reduces breakage from added imports, but means ghc might ship at times with libraries which contain bugs for which a fix already existed. It is hard to accurately judge the cost/benefit of this approach. As such bugs typically are discovered once applications are deployed and as ghc devs we might never hear of issues caused this way.

Some projects are also locked into boot library versions, and might want to use newer versions for non-bugfix reasons.

-----------------

Personally I think we should bump minor versions by default despite it occasionally causing breakage. However I think we should also be willing to avoid a bump if it causes a large amount of known breakage while not providing any bug fixes.

For example we should probably not upgrade point releases from text-2.1.1 to text-2.1.2 as the addition of `show` causes breakage and there seem to be no bug fixes in the release, unless the library authors request us to do so.

Am 28/11/2024 um 08:44 schrieb Imants Cekusins:

On Wed, 27 Nov 2024 at 11:58, Tom Ellis < tom-lists-haskell-cafe-2023@jaguarpaw.co.uk> wrote:

...

Thanks Adam,

I'm not too familiar with the GHC-side requirements here, but I wonder if we can approach this issue from a different direction: is there something we can do to help boot libraries avoid breaking changes?

Tom

Breaking changes caused by new exports could be mitigated with a library package or a GHC plugin with such auto refactoring functionality :

* Uniquely qualify (i.e., leave no different modules similarly qualified in the same module) each imported module and each imported symbol.

This refactoring should be possible in code which already compiles. It would not require syntax / grammar changes.

Something similar to smuggler2 package.

_______________________________________________ ghc-devs mailing listghc-devs@haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs