PVP proposal: no upper bounds on non-upgradeable packages

Michael Snoyman

9 Apr 2014 9 Apr '14

3:24 p.m.

At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is: Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code. The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change: * Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently. Downsides I'm aware of: * If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build. [1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

Attachments:

attachment.html (text/html — 1.7 KB)

Show replies by date

Antoine Latter

9 Apr 9 Apr

3:52 p.m.

I haven't tested this with the new cabal-install solver, but with the old one the user would get much worse error messages if 'template-haskell' had bounds - cabal-install would start trying to install the old version and fail, sending the user off on a wild goose chase trying to figure out how to make that install work. If you have no bounds then the error is much easier to diagnose - the library needs to be modified to work with the version of template-haskell currently installed. I'm +1 on this policy, even though I would prefer the fix be in cabal-install itself. On Wed, Apr 9, 2014 at 3:24 PM, Michael Snoyman wrote:

...

At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Erik Hesselink

4:08 p.m.

The only constraint that actually makes sense for template-haskell is 'installed', which I have in my cabal.config. I think this should perhaps be built into cabal. This can then be combined with a constraint on the known versions of TH a package works with. Erik On Wed, Apr 9, 2014 at 9:52 PM, Antoine Latter wrote:

...

I haven't tested this with the new cabal-install solver, but with the old one the user would get much worse error messages if 'template-haskell' had bounds - cabal-install would start trying to install the old version and fail, sending the user off on a wild goose chase trying to figure out how to make that install work.

If you have no bounds then the error is much easier to diagnose - the library needs to be modified to work with the version of template-haskell currently installed.

I'm +1 on this policy, even though I would prefer the fix be in cabal-install itself.

On Wed, Apr 9, 2014 at 3:24 PM, Michael Snoyman wrote:

...
At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Johan Tibell

4:20 p.m.

On Wed, Apr 9, 2014 at 10:08 PM, Erik Hesselink wrote:

...

The only constraint that actually makes sense for template-haskell is 'installed', which I have in my cabal.config. I think this should perhaps be built into cabal. This can then be combined with a constraint on the known versions of TH a package works with.

Adding "installed" to the user supplied constraints for base, template-haskell, and ghc-prim sounds sensible.

Carter Schonwald

4:37 p.m.

Yeah. I think the real solution lies with cabal better knowing what libs ghc was baked/built with vs which can safely usually have a conflicting version in the user package db. This would be a nice use case for cabal / ghc-pkg better understanding private vs public dependencies too! (Eg user ByteString vs ghc ByteString ) This is starting to sound like a bunch of good ideas to try on cabal! :-) On Wednesday, April 9, 2014, Johan Tibell wrote:

...

On Wed, Apr 9, 2014 at 10:08 PM, Erik Hesselink javascript:_e(%7B%7D,'cvml','hesselink@gmail.com');

...
wrote:

...
The only constraint that actually makes sense for template-haskell is 'installed', which I have in my cabal.config. I think this should perhaps be built into cabal. This can then be combined with a constraint on the known versions of TH a package works with.

Adding "installed" to the user supplied constraints for base, template-haskell, and ghc-prim sounds sensible.

Johan Tibell

4:40 p.m.

On Wed, Apr 9, 2014 at 10:37 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Yeah. I think the real solution lies with cabal better knowing what libs ghc was baked/built with vs which can safely usually have a conflicting version in the user package db.

Aside: cabal already knows something about GHC packages. More precisely it knows that base and ghc-prim are not upgradable and will thus never try to upgrade those.

...

This would be a nice use case for cabal / ghc-pkg better understanding private vs public dependencies too! (Eg user ByteString vs ghc ByteString )

This is starting to sound like a bunch of good ideas to try on cabal! :-)

There are lots of good ideas to try, especially when it comes to better tooling. The problem is to actually get someone to write the code. -- Johan

Henning Thielemann

27 Apr 27 Apr

5:59 a.m.

Am 09.04.2014 22:08, schrieb Erik Hesselink:

...

The only constraint that actually makes sense for template-haskell is 'installed', which I have in my cabal.config.

This is clever. Maybe Cabal can write this into a default cabal-config file?

...

This can then be combined with a constraint on the known versions of TH a package works with.

A package could specify version bounds in Build-Depends: template-haskell

Edward Kmett

10:45 a.m.

This has already been done for cabal 1.20. -Edward On Sun, Apr 27, 2014 at 5:59 AM, Henning Thielemann < schlepptop@henning-thielemann.de> wrote:

...

Am 09.04.2014 22:08, schrieb Erik Hesselink:

The only constraint that actually makes sense for template-haskell is

...
'installed', which I have in my cabal.config.

This is clever. Maybe Cabal can write this into a default cabal-config file?

This can then be combined with a

...
constraint on the known versions of TH a package works with.

A package could specify version bounds in Build-Depends: template-haskell

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Ganesh Sittampalam

9 Apr 9 Apr

6:57 p.m.

-1 (because of the listed downsides) On 09/04/2014 20:24, Michael Snoyman wrote:

...

At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Adam Bergmark

9:33 p.m.

...

Upper bounds should not be included on non-upgradeable packages, such as

...

Alternatively, we should establish some accepted upper bound on these

...

* Some build plans that would have previously been impossible are now

I agree with Erik that non-upgradeable packages should be constrained to installed in Cabal. I have this in my cabal config as well. Better error messages for this would be gravy. base and template-haskell (are there others?). Removing these upper bounds throws away what I consider vital information: What version of GHC was this package successfully built against? As was discussed elsewhere, perhaps base and other core packages should have fewer breaking changes instead. packages, e.g. many people place base < 5 on their code. I would be okay with this if we knew what would change up until base 5, but we don't so I think the effect would be the same as if the upper bound was dropped. possible, without resorting to the nuclear option of --allow-newers. I've seen this argument about making more builds plans possible several times and I don't get it. Yes it's great if a package still works when one of its dependencies goes through a breaking change, but the point is that the maintainer (generally) does not know this until that version is released. An upper bound gives the maintainer some time to test the new version, if it's just a matter of bumping the upper bound then this can be done really fast, if bigger changes need to be done then the lack of an upper bound wouldn't have helped since the package was broken with that version anyway. If we restrict this to the non-upgradeable packages, maintainers have time to test on a new version of GHC before the next haskell-platform release, as is the case right now. You can probably tell that I'm -1 on this :) On Thu, Apr 10, 2014 at 12:57 AM, Ganesh Sittampalam wrote:

...

-1 (because of the listed downsides)

On 09/04/2014 20:24, Michael Snoyman wrote:

...
At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Michael Snoyman

10 Apr 10 Apr

12:25 a.m.

On Thu, Apr 10, 2014 at 4:33 AM, Adam Bergmark wrote:

...

I agree with Erik that non-upgradeable packages should be constrained to installed in Cabal. I have this in my cabal config as well. Better error messages for this would be gravy.

...
Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Removing these upper bounds throws away what I consider vital information: What version of GHC was this package successfully built against? As was discussed elsewhere, perhaps base and other core packages should have fewer breaking changes instead.

Are you aware that there's a tested-with field in cabal for exactly that purpose? I'm not sure why you'd prefer a proxy variable with side-effects to the real thing.

...

...
Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code. I would be okay with this if we knew what would change up until base 5, but we don't so I think the effect would be the same as if the upper bound was dropped.

...
* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers.

I've seen this argument about making more builds plans possible several times and I don't get it. Yes it's great if a package still works when one of its dependencies goes through a breaking change, but the point is that the maintainer (generally) does not know this until that version is released. An upper bound gives the maintainer some time to test the new version, if it's just a matter of bumping the upper bound then this can be done really fast, if bigger changes need to be done then the lack of an upper bound wouldn't have helped since the package was broken with that version anyway. If we restrict this to the non-upgradeable packages, maintainers have time to test on a new version of GHC before the next haskell-platform release, as is the case right now.

I don't understand what you're saying here. How do maintainers get extra time by having an upper bound on base? Either way, beta testers of GHC will be sending reports to please support the new GHC, this just makes it slightly easier for those beta testers to get useful reports to maintainers. Michael

...

You can probably tell that I'm -1 on this :)

On Thu, Apr 10, 2014 at 12:57 AM, Ganesh Sittampalam wrote:

...
-1 (because of the listed downsides)

On 09/04/2014 20:24, Michael Snoyman wrote:

...
At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Herbert Valerio Riedel

5:43 a.m.

On 2014-04-10 at 06:25:24 +0200, Michael Snoyman wrote: [...]

...

...
What version of GHC was this package successfully built against? As was discussed elsewhere, perhaps base and other core packages should have fewer breaking changes instead.

...

Are you aware that there's a tested-with field in cabal for exactly that purpose? I'm not sure why you'd prefer a proxy variable with side-effects to the real thing.

This is a cabal field I've been wondering about already. What is this 'tested-with' field actually used for (or what was its original motivation)? Are there tools actually interpreting it?

Michael Snoyman

6:58 a.m.

On Thu, Apr 10, 2014 at 12:43 PM, Herbert Valerio Riedel wrote:

...

On 2014-04-10 at 06:25:24 +0200, Michael Snoyman wrote:

[...]

...
...
What version of GHC was this package successfully built against? As was discussed elsewhere, perhaps base and other core packages should have fewer breaking changes instead.

...
Are you aware that there's a tested-with field in cabal for exactly that purpose? I'm not sure why you'd prefer a proxy variable with side-effects to the real thing.

This is a cabal field I've been wondering about already. What is this 'tested-with' field actually used for (or what was its original motivation)? Are there tools actually interpreting it?

I'm not aware of it being used for anything besides display purposes. Michael

Adam Bergmark

9:33 a.m.

...

Are you aware that there's a tested-with field in cabal for exactly that purpose? I'm not sure why you'd prefer a proxy variable with side-effects to the real thing.

I did not consider tested-with! I would be okay with using this instead of upper bounds if it implied constraints on non-upgradeable packages. To not break all of hackage the omission of this field should mean "no version bounds" (and some packages don't depend on base). From what I've seen usage of this field is very rare, which might change if cabal-install actually does something with it. Then we can gradually shift towards not allowing this field to be omitted. --allow-newer needs to be able to override this as well.

...

I don't understand what you're saying here. How do maintainers get extra time by having an upper bound on base? Either way, beta testers of GHC will be sending reports to please support the new GHC, this just makes it slightly easier for those beta testers to get useful reports to maintainers.

I think I phrased this poorly, sorry, please disregard that part. The upper bound signals what the maintainer knows about the package (ie. it has not been tested with higher versions) and beta testers will then see this, l think it's useful for them to know whether the maintainer is ahead of them. I do think --allow-newer is the proper solution here, I consider being a beta tester more nuclear than using --allow-newer :-)

...

...
On Thu, Apr 10, 2014 at 12:57 AM, Ganesh Sittampalam wrote:

...
-1 (because of the listed downsides)

...
At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are

On 09/04/2014 20:24, Michael Snoyman wrote: purely

...
about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Johan Tibell

1:27 a.m.

On Apr 10, 2014 3:33 AM, "Adam Bergmark" wrote:

...

I agree with Erik that non-upgradeable packages should be constrained to

installed in Cabal. I have this in my cabal config as well. Better error messages for this would be gravy. Now fixed in Cabal HEAD. Base and ghc-prim already had an installed constraint, but we added template-haskell and integer-gmp/simple.

Edward Kmett

6:27 p.m.

For this particular issue, in general I'm +1. The main quibble I have with it the base bound was needed for the 'default to base 3 if no bound is specified' hack, though. The base < 5 convention is a by-blow of that era. As for the others, bounds in third party packages on template-haskell cause an unreasonably large fraction of the user complaints I receive. An even worse situation is where they _do_ wind up "upgrading" template-haskell and now it doesn't work at all, silently several packages later. That said, this should be rather mitigated going forward by: https://github.com/hvr/cabal/commit/65e9b88bc849e76040ed7947591ea7172cd02db5 which closes out https://github.com/haskell/cabal/issues/1444 and https://github.com/haskell/cabal/issues/667 So in essence, it _is_ being fixed in cabal. -Edward On Wed, Apr 9, 2014 at 3:24 PM, Michael Snoyman wrote:

...

At Herbert's request, I'm splitting off this part of the PVP proposal I made[1] to its own separate proposal. Same discussion period of three weeks. The proposal is:

Upper bounds should not be included on non-upgradeable packages, such as base and template-haskell (are there others?). Alternatively, we should establish some accepted upper bound on these packages, e.g. many people place base < 5 on their code.

The purpose (elaborated in my blog post[2]) is that these upper bounds virtually never provide for a successful build. Instead, they are purely about what error messages the user receives. With this change:

* Some build plans that would have previously been impossible are now possible, without resorting to the nuclear option of --allow-newers. * Instead of cabal version bound error messages, users get GHC compiler errors that can be fed back upstream to help get packages fixed more expediently.

Downsides I'm aware of:

* If you consider the cabal error messages more user friendly, then the error message quality goes down. * Users may find out later than they do right now about a failing build.

[1] http://www.haskell.org/pipermail/libraries/2014-April/022529.html [2] http://www.yesodweb.com/blog/2014/04/proposal-changes-pvp

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Herbert Valerio Riedel

11 Apr 11 Apr

3:30 a.m.

On 2014-04-11 at 00:27:31 +0200, Edward Kmett wrote: [...]

...

That said, this should be rather mitigated going forward by:

https://github.com/hvr/cabal/commit/65e9b88bc849e76040ed7947591ea7172cd02db5

which closes out

https://github.com/haskell/cabal/issues/1444

and

https://github.com/haskell/cabal/issues/667

So in essence, it _is_ being fixed in cabal.

Btw, the case of 'template-haskell' could be considered another data-point in the upper-bounds debate: Cabal would have been able to do a better job, if 'template-haskell' would have set tighter version-bounds :-) If you look at the previous versions, on http://hackage.haskell.org/package/template-haskell 'template-haskell' had the obviously incorrect constraint build-depends: base >= 4.2 && < 5, pretty, containers from version 2.4.0.1 till 2.8.0.0; only starting with 2.9.0.0 it got a tighter one, specifically build-depends: base == 4.7.*, containers == 0.5.*, pretty == 1.1.* that will keep Cabal from even attempting to install template-haskell-2.8.0 on GHCs prior to GHC 7.8.1, *but* it wouldn't keep GHC 7.8.1 from installing versions of 'template-haskell' prior to 2.8.0.0 So we have to resort to declare 'template-haskell' explicitly in cabal-install's solvers as being a non-upgradable package as a hack, instead of being able to rely on the .cabal meta-data to direct Cabal's solver. PS: Strictly speaking, it seems to me 'template-haskell' can actually be upgraded in theory, GHC will use the latest installed version as its wired-in package (if understand the code correctly), but there was no need for updating 'template-haskell' between GHC releases in the past.

4043

Age (days ago)

4061

Last active (days ago)

List overview

Download

16 comments

10 participants

participants (10)

Adam Bergmark
Antoine Latter
Carter Schonwald
Edward Kmett
Erik Hesselink
Ganesh Sittampalam
Henning Thielemann
Herbert Valerio Riedel
Johan Tibell
Michael Snoyman