On Wed, Sep 05, 2012 at 05:43:04PM -0700, Michael Sloan wrote:

While I suppose it could break proper dependency resolution, a big reason for upper bounds is preventing a cascade of confusing errors. With orphan instances the errors are very finite.

I don't think the type of errors was a factor in designing the PvP. The primary (perhaps even only) goal was to design packages in such a way that if "cabal install foo" would have worked yesterday, then it will also work today. That means that if foo depends on bar, and there is a suitable newer version of bar available today, then foo had better work with that version too. The person installing foo isn't interested in developing foo or bar, so doesn't want any error at all! Thanks Ian

On Thu, Sep 6, 2012 at 7:01 AM, Michael Sloan wrote:

Thing is, that if this is the design goal of the PVP, then _any_ addition to the exports of a module should necessitate a major version bump, because of the potential for clashing unqualified, non-explicit imports. The reasoning for this seems to be that the chances are low (very context dependent) and unqualified, non-explicit imports are discouraged and bad style. Orphan instances are also discouraged, so for consistency with the policy for regular additions, it would make sense for libraries adding instances to only necessitate a minor version bump.

No, the PVP explicitly mentions this. An added export needs only a minor version bump. A user of the package can then: * Depend on only the range of bugfix releases (A.B.C.*), thus guarding against additions. Or, * Depend on a range of minor versions (A.B.*), and use only explicit or qualified imports, again guarding against additions. Erik

On Thu, Sep 06, 2012 at 06:01:44AM +0100, Michael Sloan wrote:

Thing is, that if this is the design goal of the PVP, then _any_ addition to the exports of a module should necessitate a major version bump, because of the potential for clashing unqualified, non-explicit imports.

You can make you code proof against additional exports by using explicit imports, which is why they only need a minor version bump. But there's no way to control the import of instances. The typical issue is that you add a type or class, and your users want an instance you didn't provide, so they define one, which will be broken when you add the instance in the next version. You may blame the users for defining the orphan instance, but they had little choice. The principle should be: - code that built against a particular version of a package will also build against any release with the same major version. - code using explicit imports that built against a particular version of a package will also build against any release with the same minor version.

* Ross Paterson [2012-09-06 08:34:20+0100]

The principle should be: - code that built against a particular version of a package will also build against any release with the same major version. - code using explicit imports that built against a particular version of a package will also build against any release with the same minor version.

I think you swapped 'major' and 'minor' in these two sentences. -- Roman I. Cheplyaka :: http://ro-che.info/

Interesting ideas! One way to make explicit re-export pragmas less necessary would be to ignore package-local re-exports. This might seem funky initially, there's precedent for involving packages with import semantics (package-qualified imports), and this is a sensible granularity for such judgments because it's reasonable to assume "global" knowledge of instances inside a package. To minimize need for explicit pragmas, we could have another variety of module export that specifies which imports provide instances that should be explicitly re-exported. While the explicit pragma might still be desirable (seems fiddly, though..), I think that this would cover a great deal of the problems. The syntax might look like: module Foo(..., module instances Bar) where import Bar Could also think about "filtering" based on partial specifications - "Only re-export Monad instances", "Only re-export things involving Maybe", "Hide everything involving Maybe", etc etc. This gets a bit more complicated: module Foo(..., module instances Bar(Monad, Maybe)) where import Foo I'm not at all sold on this more complicated one, but something along this line might be good. -mgsloan On Thu, Sep 6, 2012 at 2:10 AM, Henning Thielemann wrote:

Am 06.09.2012 02:43, schrieb Michael Sloan:

...

I agree that it's quite unfortunate that adding instances causes a major bump. I think that instead it would make sense to only bump your major version whenever you add *orphan* instances. This is because it is known to be dangerous to create orphan instances, and the errors are relatively transparent. While I suppose it could break proper dependency resolution, a big reason for upper bounds is preventing a cascade of confusing errors. With orphan instances the errors are very finite.

In general here are a few different things that would help with the orphans problem:

There have been some ideas about explicit import of instances.

I also had some thoughts about how GHC can help managing orphan instances when they are necessary: http://www.haskell.org/pipermail/haskell-cafe/2011-July/094014.html

Glad to hear that you agree that it's consistent! This would make the PVP simpler. I've been thinking, there's still a way to retain the property that you can specify version bounds which retain compilation, even with orphan instances. If your package really needs to add an instance to a datatype / class exported by some dependency, then you can put an upper limit on the minor version. Harsh, I know, but orphans muck things up any way you slice it. This means that any orphans you use, you'd need to explicitly depend on an A.B.C upper-bounded version of the packages that define the datatypes (and classes, if you're being entirely correct), and not consume them through something that re-exports them. Ideally, this could be mechanically checked. The maintenance of this kind of stuff can be reduced by centralizing stuff in orphans package (does Hackage 2.0 provide a way to email-subscribe to a package? might also be good for maintaining these sorts of things). If a central orphans package isn't acceptable, then you're probably not writing a library for general consumption anyway, so upper limits on minor bounds wouldn't be problematic. -Michael On Thu, Sep 6, 2012 at 4:24 AM, Simon Marlow wrote:

On 06/09/2012 01:16, Johan Tibell wrote:

...

Hi all,

The PVP says:

"A.B is known as the major version number, and C the minor version number. When a package is updated, the following rules govern how the version number must change relative to the previous version:

1. If any entity was removed, or the types of any entities or the definitions of datatypes or classes were changed, or instances were added or removed, then the new A.B must be greater than the previous A.B. Note that modifying imports or depending on a newer version of another package may cause extra instances to be exported and thus force a major version change."

The part about adding instances and the one about modifying imports makes it hard to follow the PVP. Bumping the major version number is a quite disruptive change for your users if they use upper bounds on their dependencies. Minor version bumps are not nearly as disruptive as you can you can depend on x.y.* as long as you only use qualified imports and/or explicit import lists.

Assuming no one uses orphan instances, adding instances is always safe because you can only add instances for types from packages you depend on, which means that they can't be depending on your package in turn and have defined a non-orphan instance for a type or class defined in your package.

I suggest that the rule be changed to not require a major version bump if instances are added.

Yes, I think this is reasonable. The client already has some obligations if they want to be independent of minor versions: they have to use explicit import lists. So adding another obligation, no orphan instances, is consistent with this and shouldn't cause problems in the majority of cases.

Cheers, Simon

...

P.S. I believe the same reasoning can be applied to the import part of the rule above.

-- Johan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Tue, Sep 11, 2012 at 5:32 AM, Ben Millwood wrote:

On Tue, Sep 11, 2012 at 6:14 AM, Michael Sloan wrote:

...

Glad to hear that you agree that it's consistent! This would make the PVP simpler.

I've been thinking, there's still a way to retain the property that you can specify version bounds which retain compilation, even with orphan instances. If your package really needs to add an instance to a datatype / class exported by some dependency, then you can put an upper limit on the minor version. Harsh, I know, but orphans muck things up any way you slice it.

This means that any orphans you use, you'd need to explicitly depend on an A.B.C upper-bounded version of the packages that define the datatypes (and classes, if you're being entirely correct), and not consume them through something that re-exports them. Ideally, this could be mechanically checked.

This seems sensible. However, in case this wasn't already clear, it means when you /add/ an orphan instance, you still need a major version bump: suppose I depend on package A and package B and have an orphan for a type in package B, then you suggest I depend on package-B-minor and package-A-major – so if package A adds an orphan instance, they need to make their version bump major to stop me from breaking.

I actually didn't write down what I meant to - instead of "any orphans you use" --> "any orphans you define". Using orphans does not cause finer grained upper bounds. Hmm, I hadn't thought of what you're pointing out! So, the way to get the properties I was going for, even with orphans, is that as soon as you define an orphan instance, you must specify minor bounds on every package that could realistically want to define their own orphans. Definitely not ideal! I guess the alternative is the "instance manifest" solution, and have Cabal use them for resolution (also hairy and troublesome)..

...

The maintenance of this kind of stuff can be reduced by centralizing stuff in orphans package (does Hackage 2.0 provide a way to email-subscribe to a package? might also be good for maintaining these sorts of things). If a central orphans package isn't acceptable, then you're probably not writing a library for general consumption anyway, so upper limits on minor bounds wouldn't be problematic.

Whether or not centralised orphans are appropriate really depends on why the orphans exist in the first place. If they are just because the original package deemed those instances not necessary, then fine (although they should really be put in the original package). But frequently they are because there is a class in module A and a type in module B that have a sensible instance, but neither the package containing A nor B know about each other (or there is no natural direction for the dependency to go). In that case, you really want a package per A,B pair, since otherwise you're going to bring in irrelevant dependencies.

Yup, I can agree to that! A naming + namespace convention for this variety of package might be good, for discoverability. -Michael

...

-Michael

On Thu, Sep 6, 2012 at 4:24 AM, Simon Marlow wrote:

...

On 06/09/2012 01:16, Johan Tibell wrote:

...

Hi all,

The PVP says:

"A.B is known as the major version number, and C the minor version number. When a package is updated, the following rules govern how the version number must change relative to the previous version:

1. If any entity was removed, or the types of any entities or the definitions of datatypes or classes were changed, or instances were added or removed, then the new A.B must be greater than the previous A.B. Note that modifying imports or depending on a newer version of another package may cause extra instances to be exported and thus force a major version change."

The part about adding instances and the one about modifying imports makes it hard to follow the PVP. Bumping the major version number is a quite disruptive change for your users if they use upper bounds on their dependencies. Minor version bumps are not nearly as disruptive as you can you can depend on x.y.* as long as you only use qualified imports and/or explicit import lists.

Assuming no one uses orphan instances, adding instances is always safe because you can only add instances for types from packages you depend on, which means that they can't be depending on your package in turn and have defined a non-orphan instance for a type or class defined in your package.

I suggest that the rule be changed to not require a major version bump if instances are added.

Yes, I think this is reasonable. The client already has some obligations if they want to be independent of minor versions: they have to use explicit import lists. So adding another obligation, no orphan instances, is consistent with this and shouldn't cause problems in the majority of cases.

Cheers, Simon

...

P.S. I believe the same reasoning can be applied to the import part of the rule above.

-- Johan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Tue, 11 Sep 2012, Ganesh Sittampalam wrote:

What's the problem with forcing a major version bump in every release that adds an orphan instance?

+1 This would treat orphan instances like all new instances are handled today by the PVP.

On Tue, Sep 11, 2012 at 6:04 PM, Michael Sloan wrote:

On Tue, Sep 11, 2012 at 5:32 AM, Ben Millwood wrote:

...

On Tue, Sep 11, 2012 at 6:14 AM, Michael Sloan wrote:

...

Glad to hear that you agree that it's consistent! This would make the PVP simpler.

I've been thinking, there's still a way to retain the property that you can specify version bounds which retain compilation, even with orphan instances. If your package really needs to add an instance to a datatype / class exported by some dependency, then you can put an upper limit on the minor version. Harsh, I know, but orphans muck things up any way you slice it.

This means that any orphans you use, you'd need to explicitly depend on an A.B.C upper-bounded version of the packages that define the datatypes (and classes, if you're being entirely correct), and not consume them through something that re-exports them. Ideally, this could be mechanically checked.

This seems sensible. However, in case this wasn't already clear, it means when you /add/ an orphan instance, you still need a major version bump: suppose I depend on package A and package B and have an orphan for a type in package B, then you suggest I depend on package-B-minor and package-A-major – so if package A adds an orphan instance, they need to make their version bump major to stop me from breaking.

I actually didn't write down what I meant to - instead of "any orphans you use" --> "any orphans you define". [...]

I /think/ that's how I read it, but I might have misunderstood.

Hmm, I hadn't thought of what you're pointing out! So, the way to get the properties I was going for, even with orphans, is that as soon as you define an orphan instance, you must specify minor bounds on every package that could realistically want to define their own orphans. Definitely not ideal! I guess the alternative is the "instance manifest" solution, and have Cabal use them for resolution (also hairy and troublesome)..

No, you only have to specify minor bounds on anyone who could add a instance /without/ it being an orphan - i.e. the owners of the class and type. If any of the other packages declared an orphan, they'd major-bump. So, in summary: - adding an instance of a type or class that you control is a minor (x.y.z) version bump - adding an orphan instance requires a major version bump, and also makes you fragile to minor version bumps in the package that defines the class or type. - removing an instance ordinary or otherwise still requires a major version bump, so when an orphan transitions to the original package, the original package minor-bumps and the orphan package major-bumps, and code with only a major constraint on both will not break (because the constraints of the orphan package mean you can either build with the old version of both or the new version of both).

+1! This simple change would dramatically decrease my workload. I often add instances for my own classes to my own types, and the need for a major version bump can ripple through 20+ packages and then out into 3rd party packages. -Edward On Thu, Sep 6, 2012 at 7:24 AM, Simon Marlow wrote:

On 06/09/2012 01:16, Johan Tibell wrote:

...

Hi all,

The PVP says:

"A.B is known as the major version number, and C the minor version number. When a package is updated, the following rules govern how the version number must change relative to the previous version:

1. If any entity was removed, or the types of any entities or the definitions of datatypes or classes were changed, or instances were added or removed, then the new A.B must be greater than the previous A.B. Note that modifying imports or depending on a newer version of another package may cause extra instances to be exported and thus force a major version change."

The part about adding instances and the one about modifying imports makes it hard to follow the PVP. Bumping the major version number is a quite disruptive change for your users if they use upper bounds on their dependencies. Minor version bumps are not nearly as disruptive as you can you can depend on x.y.* as long as you only use qualified imports and/or explicit import lists.

Assuming no one uses orphan instances, adding instances is always safe because you can only add instances for types from packages you depend on, which means that they can't be depending on your package in turn and have defined a non-orphan instance for a type or class defined in your package.

I suggest that the rule be changed to not require a major version bump if instances are added.

Yes, I think this is reasonable. The client already has some obligations if they want to be independent of minor versions: they have to use explicit import lists. So adding another obligation, no orphan instances, is consistent with this and shouldn't cause problems in the majority of cases.

Cheers, Simon

P.S. I believe the same reasoning can be applied to the import part of

...

the rule above.

-- Johan

______________________________**_________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/**mailman/listinfo/librarieshttp://www.haskell.org/mailman/listinfo/libraries

______________________________**_________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/**mailman/listinfo/librarieshttp://www.haskell.org/mailman/listinfo/libraries

On Tue, 11 Sep 2012, Ian Lynagh wrote:

On Tue, Sep 11, 2012 at 02:11:36PM -0400, Edward Kmett wrote:

...

I often add instances for my own classes to my own types, and the need for a major version bump can ripple through 20+ packages and then out into 3rd party packages.

If you're adding the instance at the same time as adding either the type or the class, then a major bump shouldn't be needed.

However, if you often find that you need to add an instance for an existing class+type, then isn't it plausible that the authors of these 3rd party packages would find they need the instances first, so add orphan instances?

If a package user needs a new instance he should propose to add the instance the package author. Sometimes the package user and the package author find out this way that there is no unique natural instance. If this is the case or the package author does not respond fast enough then the package user should create a newtype wrapper. Adding an orphan instance is not a solution since it may not only clash with new instances in the original package but also with orphan instances from other packages. Orphan instances are really only acceptable if there is an agreed natural instance but there is no package where to put it into naturally.

A "no orphan instance" rule could work, but note that not only could you not use orphan instances yourself, but you also couldn't use any packages that contain orphan instances. I think that to make that work in practice we'd have to make orphan instances an error, unless either the type or the class is in the current package.

No rule of the PVP is currently enforced by the compiler, so why should orphan instances be an error? GHC warns about them, for me this is enough.