Re: [Haskell] Re: Trying to install binary-0.4
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often. So rather than keep replying to individual points, I'd like to make some concrete proposals so we can make progress. 1. Document the version numbering policy. We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where: x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical further sub-versions may be added after the x.y, their meaning is package-defined. Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest. 2. Precise dependencies. As suggested by various people in this thread: we change the convention so that dependencies must specify a single x.y API version, or a range of versions with an upper bound. Cabal or Hackage can refuse to accept packages that don't follow this convention (perhaps Hackage is a better place to enforce it, and Cabal should just warn, I'm not sure). Yes, earlier I argued that not specifying precise dependencies allows some packages to continue working even when dependencies change, and that having precise dependencies means that all packages are guaranteed to break when base is updated. However, I agree that specifying precise dependencies is ultimately the right thing, we'll get better errors when things break, There's lots more to discuss, but I think the above 2 proposals are a step in the right direction, agreed? Cheers, Simon
On Tue, Oct 16, 2007 at 01:08:49PM +0100, Simon Marlow wrote:
So rather than keep replying to individual points, I'd like to make some concrete proposals so we can make progress.
1. Document the version numbering policy.
We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where:
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
further sub-versions may be added after the x.y, their meaning is package-defined.
This should be required for at least the GHC boot packages (and encouraged for others). I would make "API extended only" a bit more precise: any module that uses explicit import lists will not be affected by the changes. So one can add classes, types and functions, but not instances (except where either the class or the type is new). You probably can't add data constructors or fields, and have to be careful with new methods. I'd also prefer that major versions used two numbers, because that's common now, it supports the experimental versions 0.x apfelmus mentioned, and it makes it easier to leave room for development versions (possibly using an odd-even scheme). If you make your development repository available, and it contains API changes, you'll want its version number to have a larger major number.
Ross Paterson wrote:
I would make "API extended only" a bit more precise: any module that uses explicit import lists will not be affected by the changes. So one can add classes, types and functions, but not instances (except where either the class or the type is new).
okay
You probably can't add data constructors or fields, and have to be careful with new methods.
If they're exported and new members of existing classes/datatypes, then you can't add them, because they might be imported with "class/typename(..)". (right?) What about semantic changes to the API? Including, adding a default to a class method changes the default from 'undefined', which someone might have relied on as the default (although it seems unlikely). Isaac
I've written down the proposed policy for versioning here: http://haskell.org/haskellwiki/Package_versioning_policy It turned out there was a previous page written by Bulat that contained essentially this policy, but it wasn't linked from anywhere which explains why it was overlooked. I took the liberty of rewriting the text. I took into account Ross's suggestions that the major version should have two components, and that we need to be more precise about what it means to extend an API. After a round of editing, we can start to link to this page from everywhere, and start migrating packages to this scheme where necessary. Cheers, Simon
On Wed, Oct 17, 2007 at 12:54:12PM +0100, Simon Marlow wrote:
I've written down the proposed policy for versioning here:
This says: If [...] instances were added or removed, then the new A.B must be greater than the previous A.B. This presumably includes changing module imports, or depending on a newer version of a package, which results in the visible instances changing? I think this should be spelt out in the policy. The example: build-depends: mypkg == 2.1.1 should be: build-depends: mypkg >= 2.1.1, mypkg < 2.1.2 with the current dependency syntax/semantics. Thanks Ian
On Wed, Oct 17, 2007 at 12:54:12PM +0100, Simon Marlow wrote:
I've written down the proposed policy for versioning here:
http://haskell.org/haskellwiki/Package_versioning_policy
It turned out there was a previous page written by Bulat that contained essentially this policy, but it wasn't linked from anywhere which explains why it was overlooked. I took the liberty of rewriting the text.
You wrote:
A client that wants to specify that they depend on a particular version of the API can specify a particular A.B.C and be sure of getting that API only. For example, build-depends: mypkg-2.1.1
Are you proposing an extension along the lines of that proposed by Thomas (and Bulat, and others), i.e. this would be equivalent to build-depends: mypkg >= 2.1.1 && < 2.1.2 ? The current syntax of mypkg == 2.1.1 would match the initial release but not subsequent patch releases.
Ross Paterson wrote:
On Wed, Oct 17, 2007 at 12:54:12PM +0100, Simon Marlow wrote:
I've written down the proposed policy for versioning here:
http://haskell.org/haskellwiki/Package_versioning_policy
It turned out there was a previous page written by Bulat that contained essentially this policy, but it wasn't linked from anywhere which explains why it was overlooked. I took the liberty of rewriting the text.
You wrote:
A client that wants to specify that they depend on a particular version of the API can specify a particular A.B.C and be sure of getting that API only. For example, build-depends: mypkg-2.1.1
Are you proposing an extension along the lines of that proposed by Thomas (and Bulat, and others), i.e. this would be equivalent to build-depends: mypkg >= 2.1.1 && < 2.1.2 ?
Yes, I should have mentioned that, thanks. I'll update the page shortly with this suggestion and others. Cheers, Simon
simonmarhaskell:
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often.
So rather than keep replying to individual points, I'd like to make some concrete proposals so we can make progress.
1. Document the version numbering policy.
We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where:
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
further sub-versions may be added after the x.y, their meaning is package-defined. Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest.
2. Precise dependencies.
As suggested by various people in this thread: we change the convention so that dependencies must specify a single x.y API version, or a range of versions with an upper bound. Cabal or Hackage can refuse to accept packages that don't follow this convention (perhaps Hackage is a better place to enforce it, and Cabal should just warn, I'm not sure).
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust. -- Don
Hi
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust.
In general, if it compiles and type checks, it will work. It is rare that an interface stays sufficiently similar that the thing compiles, but then crashes at runtime. Given that, shouldn't the tested versions be something a machine figures out - rather than something each library author has to tend to with every new release of every other library in hackage? Thanks Neil
Neil Mitchell wrote:
Hi
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust.
In general, if it compiles and type checks, it will work. It is rare that an interface stays sufficiently similar that the thing compiles, but then crashes at runtime.
True.. GoboLinux's package system records the exact set of versions something compiles with (just for reference), and uses min version bounds (and max bounds where needed) for dependencies. It's always possible for Haskell library implementation-bug-fixes to change relied-on behavior, as discussed in the original ECT description. I agree that compiling and type-checking is a pretty good sign of working. Passing tests (e.g. QuickCheck) could be tested too, where available. If optimizations and unsafePerformIO interact differently, different compiler versions could also affect whether something works correctly, but still compiles... But, the issue here is much more limited: we assume that there were some set of versions of these libraries that DID work, and, that every version of each library, on its own (or with only the libraries it depends on), works. So it might be valuable to record subjectively-working exact version sets, somewhere. Isaac
Neil Mitchell wrote:
Hi
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust.
In general, if it compiles and type checks, it will work. It is rare that an interface stays sufficiently similar that the thing compiles, but then crashes at runtime. Given that, shouldn't the tested versions be something a machine figures out - rather than something each library author has to tend to with every new release of every other library in hackage?
The only reasonable way we have to test whether a package compiles with a new version of a dependency is to try compiling it. To do anything else would be duplicating what the compiler does, and risks getting it wrong. But you're right that tools could help a lot: for example, after a base version bump, Hackage could try to build all its packages against the new base to figure out which ones need source code modifications and which can probably just have their .cabal files tweaked to allow the new version. Hackage could tentatively fix the .cabal files itself and/or contact the maintainer. We'll really need some tool to analyse API changes too, otherwise API versioning is too error-prone. Anyone like to tackle this? It shouldn't be too hard using the GHC API.. Cheers, Simon
[would it be possible to pick a single list to discuss this on please, so there is no danger of some people missing some subthreads if they aren't on all the lists, or getting messages 3 times if they are?] On Tue, Oct 16, 2007 at 01:08:49PM +0100, Simon Marlow wrote:
2. Precise dependencies.
While not directly related to this, I have the impression some people want precise dependencies so that things work properly when multiple versions of a library are installed. Personally I'm not a fan of that, as if I have package foo: module Foo where data T package bar: module Bar where bar :: T package baz: module Baz where baz :: T -> () then baz bar might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions. Thanks Ian
On Wed, 2007-10-17 at 00:35 +0100, Ian Lynagh wrote:
[would it be possible to pick a single list to discuss this on please, so there is no danger of some people missing some subthreads if they aren't on all the lists, or getting messages 3 times if they are?]
On Tue, Oct 16, 2007 at 01:08:49PM +0100, Simon Marlow wrote:
2. Precise dependencies.
While not directly related to this, I have the impression some people want precise dependencies so that things work properly when multiple versions of a library are installed.
Personally I'm not a fan of that, as if I have
package foo: module Foo where data T
package bar: module Bar where bar :: T
package baz: module Baz where baz :: T -> ()
then baz bar might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions.
Sure, those are not compatible. But we can detect that just by looking at the dependencies of each package. The solution is to distinguish private and public dependencies and to slot packages on their own version numbers and all version numbers of the packages they publicly depend upon. Then in your example we'd just compile another copy of bar or baz using a compatible version of foo. This is what Nix does. Duncan
On Wed, Oct 17, 2007 at 01:28:45AM +0100, Duncan Coutts wrote:
On Wed, 2007-10-17 at 00:35 +0100, Ian Lynagh wrote:
package foo: module Foo where data T
package bar: module Bar where bar :: T
package baz: module Baz where baz :: T -> ()
then baz bar might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions.
Sure, those are not compatible. But we can detect that just by looking at the dependencies of each package.
In general it is possible to have 2 different versions of a package in a single program, though, e.g. bar and baz could internally use different versions of a binary-tree package. Thanks Ian
On Wed, 2007-10-17 at 20:28 +0100, Ian Lynagh wrote:
On Wed, Oct 17, 2007 at 01:28:45AM +0100, Duncan Coutts wrote:
On Wed, 2007-10-17 at 00:35 +0100, Ian Lynagh wrote:
might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions.
Sure, those are not compatible. But we can detect that just by looking at the dependencies of each package.
In general it is possible to have 2 different versions of a package in a single program, though, e.g. bar and baz could internally use different versions of a binary-tree package.
Right, but only if those packages are used privately, ie such that types from the dependent packages do not leak into their public api, since that would allow the type error you originally pointed out. If we distinguish private use of a dependent package then we know it does not matter if we end up using two versions of it in a single program. Duncan
participants (7)
-
Don Stewart -
Duncan Coutts -
Ian Lynagh -
Isaac Dupree -
Neil Mitchell -
Ross Paterson -
Simon Marlow