Re: [Haskell] Re: Trying to install binary-0.4
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often. So rather than keep replying to individual points, I'd like to make some concrete proposals so we can make progress. 1. Document the version numbering policy. We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where: x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical further sub-versions may be added after the x.y, their meaning is package-defined. Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest. 2. Precise dependencies. As suggested by various people in this thread: we change the convention so that dependencies must specify a single x.y API version, or a range of versions with an upper bound. Cabal or Hackage can refuse to accept packages that don't follow this convention (perhaps Hackage is a better place to enforce it, and Cabal should just warn, I'm not sure). Yes, earlier I argued that not specifying precise dependencies allows some packages to continue working even when dependencies change, and that having precise dependencies means that all packages are guaranteed to break when base is updated. However, I agree that specifying precise dependencies is ultimately the right thing, we'll get better errors when things break, There's lots more to discuss, but I think the above 2 proposals are a step in the right direction, agreed? Cheers, Simon
From: libraries-bounces@haskell.org [mailto:libraries-bounces@haskell.org] On Behalf Of Simon Marlow
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest.
Just a minor point, but would mind explaining exactly what lexicographic ordering implies? It appears to me that e.g. version 9.3 of a package would be preferred over version 10.0. That strikes me as counter-intuitive. Alistair ***************************************************************** Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person(s) or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer. *****************************************************************
Bayley, Alistair wrote:
From: libraries-bounces@haskell.org [mailto:libraries-bounces@haskell.org] On Behalf Of Simon Marlow
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest.
Just a minor point, but would mind explaining exactly what lexicographic ordering implies? It appears to me that e.g. version 9.3 of a package would be preferred over version 10.0. That strikes me as counter-intuitive.
The lexicographical ordering would make 10.0 > 9.3. In general, A.B > C.D iff A > C or A == C && B > D. When we say the "latest" version we mean "greatest", implying that version numbers increase with time. Does that help? Cheers, Simon
From: Simon Marlow [mailto:simonmarhaskell@gmail.com]
The lexicographical ordering would make 10.0 > 9.3. In general, A.B > C.D iff A > C or A == C && B > D. When we say the "latest" version we mean "greatest", implying that version numbers increase with time. Does that help?
Sort of. It's what I'd expect from a sensible version comparison. It's just not something I'd ever choose to call lexicographic ordering. IMO, lexicographgic ordering is a basic string comparision so e.g. max "10.0" "9.3" = "9.3" I'd call what you're doing numeric ordering. Does it have a better name, like version-number-ordering, or section-number-ordering (e.g. Section 3.2.5, Section 3.2.6)? Alistair ***************************************************************** Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person(s) or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer. *****************************************************************
Bayley, Alistair wrote:
From: Simon Marlow [mailto:simonmarhaskell@gmail.com]
The lexicographical ordering would make 10.0 > 9.3. In general, A.B > C.D iff A > C or A == C && B > D. When we say the "latest" version we mean "greatest", implying that version numbers increase with time. Does that help?
Sort of. It's what I'd expect from a sensible version comparison. It's just not something I'd ever choose to call lexicographic ordering. IMO, lexicographgic ordering is a basic string comparision so e.g.
max "10.0" "9.3" = "9.3"
I'd call what you're doing numeric ordering. Does it have a better name, like version-number-ordering, or section-number-ordering (e.g. Section 3.2.5, Section 3.2.6)?
I've heard it called lexicographical ordering before, but I'm happy to call it by whatever name induces the least confusion! Cheers, Simon
On Tue, 2007-10-16 at 14:01 +0100, Bayley, Alistair wrote:
From: Simon Marlow [mailto:simonmarhaskell@gmail.com]
The lexicographical ordering would make 10.0 > 9.3. In general, A.B > C.D iff A > C or A == C && B > D. When we say the "latest" version we mean "greatest", implying that version numbers increase with time. Does that help?
Sort of. It's what I'd expect from a sensible version comparison. It's just not something I'd ever choose to call lexicographic ordering. IMO, lexicographgic ordering is a basic string comparision so e.g.
max "10.0" "9.3" = "9.3"
I'd call what you're doing numeric ordering. Does it have a better name, like version-number-ordering, or section-number-ordering (e.g. Section 3.2.5, Section 3.2.6)?
It's lexicographic ordering on the list of numbers, not on the string representation. ie it's [10, 0] > [9, 3] not "10.0" > "9.3" Internally we represent version numbers as lists of integers and use the default Ord instance. Duncan
Simon Marlow wrote:
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often.
I have another concrete proposal to avoid things breaking so often. Let us steal from something that works: shared library versioning on unixy systems. On Max OS X, I note that I have, in /usr/lib:
lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.2.dylib -> libcurl.3.dylib lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.3.0.0.dylib -> libcurl.3.dylib -rwxr-xr-x 1 root wheel 201156 Aug 17 17:14 libcurl.3.dylib lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.dylib -> libcurl.3.dylib
The above declaratively expresses that libcurl-3.3.0 provides the version 3 API and the version 2 API. This is the capability that should be added to Haskell library packages. Right now a library can only declare a single version number. So if I update hsFoo from 2.1.1 to 3.0.0 then I cannot express whether or not the version 3 API is a superset of (backward compatible with) the version 2 API. Once it is possible to have cabal register the hsFoo-3.0.0 also as hsFoo-2 it will be easy to upgrade to hsFoo. No old programs will fail to compile. Who here knows enough about the ghc-pkg database to say how easy or hard this would be? -- Chris
ChrisK wrote:
Simon Marlow wrote:
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often.
I have another concrete proposal to avoid things breaking so often. Let us steal from something that works: shared library versioning on unixy systems.
On Max OS X, I note that I have, in /usr/lib:
lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.2.dylib -> libcurl.3.dylib lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.3.0.0.dylib -> libcurl.3.dylib -rwxr-xr-x 1 root wheel 201156 Aug 17 17:14 libcurl.3.dylib lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.dylib -> libcurl.3.dylib
The above declaratively expresses that libcurl-3.3.0 provides the version 3 API and the version 2 API.
This is the capability that should be added to Haskell library packages.
Right now a library can only declare a single version number. So if I update hsFoo from 2.1.1 to 3.0.0 then I cannot express whether or not the version 3 API is a superset of (backward compatible with) the version 2 API.
Certainly, this is something we want to support. However, there's an important difference between shared-library linking and Haskell: in Haskell, a superset of an API is not backwards-compatible, because it has the potential to cause new name clashes.
Once it is possible to have cabal register the hsFoo-3.0.0 also as hsFoo-2 it will be easy to upgrade to hsFoo. No old programs will fail to compile.
Who here knows enough about the ghc-pkg database to say how easy or hard this would be?
It could be done using the tricks that Claus just posted and I followed up on. You'd need a separate package for hsFoo-2 that specifies exactly which bits of hsFoo-3 are re-exported. Given some Cabal support and a little extension in GHC, this could be made relatively painless for the library maintainer. Cheers, Simon
On Tue, Oct 16, 2007 at 01:57:01PM +0100, Simon Marlow wrote:
Certainly, this is something we want to support. However, there's an important difference between shared-library linking and Haskell: in Haskell, a superset of an API is not backwards-compatible, because it has the potential to cause new name clashes.
This is the case on Unixy .so systems too, because the namespace is flat. If libreadline suddenly starts exporting a symbol named SDL_init, programs which use both readline and sdl will break. I have not seen this happen in practice. (Which might have something to do with the aforementioned name mangling :)) Stefan
I have another concrete proposal to avoid things breaking so often. Let us steal from something that works: shared library versioning on unixy systems.
indeed!-) there are established workarounds that are needed to make that system work as it does, so it is a good idea to check whether cabal has the means to cover those situations.
The above declaratively expresses that libcurl-3.3.0 provides the version 3 API and the version 2 API.
This is the capability that should be added to Haskell library packages.
Right now a library can only declare a single version number. So if I update hsFoo from 2.1.1 to 3.0.0 then I cannot express whether or not the version 3 API is a superset of (backward compatible with) the version 2 API.
Certainly, this is something we want to support. However, there's an important difference between shared-library linking and Haskell: in Haskell, a superset of an API is not backwards-compatible, because it has the potential to cause new name clashes.
yes, one would need to define what it means for one api to be compatible with another. even so, i think that permitting a single package to act as a provider for multiple versions of an api is a necessary feature, even more so if loose dependency specs like 'base', or 'base >= 1.0' are going to be discouraged.
It could be done using the tricks that Claus just posted and I followed up on. You'd need a separate package for hsFoo-2 that specifies exactly which bits of hsFoo-3 are re-exported. Given some Cabal support and a little extension in GHC, this could be made relatively painless for the library maintainer.
are those tricks necessary in this specific case? couldn't we have a list/range of versions in the version: field, and let cabal handle the details? aside: what happens if we try to combine two modules M and N that use the same api A, but provided by two different packages P1 and P2? say, M was built when P1 was still around, but when N was built, P2 had replaced P1, still supporting A, but not necessarily with the same internal representation as used in P1. claus
Claus Reinke wrote:
It could be done using the tricks that Claus just posted and I followed up on. You'd need a separate package for hsFoo-2 that specifies exactly which bits of hsFoo-3 are re-exported. Given some Cabal support and a little extension in GHC, this could be made relatively painless for the library maintainer.
are those tricks necessary in this specific case? couldn't we have a list/range of versions in the version: field, and let cabal handle the details?
I don't understand what you're proposing here. Surely just writing version: 1.0, 2.0 isn't enough - you need to say what the 1.0 and 2.0 APIs actually *are*, and then wouldn't that require more syntax? I don't yet see a good reason to do this in a single .cabal file instead of two separate packages. The two-package way seems to require fewer extensions to Cabal.
aside: what happens if we try to combine two modules M and N that use the same api A, but provided by two different packages P1 and P2? say, M was built when P1 was still around, but when N was built, P2 had replaced P1, still supporting A, but not necessarily with the same internal representation as used in P1.
Not sure what you mean by "try to combine". A concrete example? Cheers, Simon
are those tricks necessary in this specific case? couldn't we have a list/range of versions in the version: field, and let cabal handle the details?
I don't understand what you're proposing here. Surely just writing
version: 1.0, 2.0
isn't enough - you need to say what the 1.0 and 2.0 APIs actually *are*, and then wouldn't that require more syntax? I don't yet see a good reason to do this in a single .cabal file instead of two separate packages. The two-package way seems to require fewer extensions to Cabal.
yes, and no. cabal is currently not symmetric in this: providers specify apis (at the level of exposed modules), clients only specify api numbers as dependencies. the idea was for the cabal file to specify a single provided api, but to register that as sufficient for a list of dependency numbers. so the package would implement the latest api, but could be used by clients expecting either the old or the new api.
aside: what happens if we try to combine two modules M and N that use the same api A, but provided by two different packages P1 and P2? say, M was built when P1 was still around, but when N was built, P2 had replaced P1, still supporting A, but not necessarily with the same internal representation as used in P1.
Not sure what you mean by "try to combine". A concrete example?
lets see - how about this: -- package P-1, Name: P, Version: 0.1 module A(L,f,g) where newtype L a = L [a] f a (L as) = elem a as g as = L as -- package P-2, Name: P, Version: 0.2 module A(L,f,g) where newtype L a = L (a->Bool) f a (L as) = as a g as = L (`elem` as) if i got this right, both P-1 and P-2 support the same api A, right down to types. but while P-1's A and P-2's A are each internally consistent, they can't be mixed. now, consider module M where import A m = g [1,2,3] module N where import A n :: Integer -> A.L Integer -> Bool n = f so, if i install P-1, then build M, then install P-2, then build N, wouldn't N pick up the "newer" P-2, while M would use the "older" P-1? and if so, what happens if we then add module Main where import M import N main = print (n 0 m) i don't seem to be able to predict the result, without actually trying it out. can you?-) i suspect it won't be pretty, though. claus
Simon Marlow wrote:
The proposed policy, for the sake of completeness is: x.y where:
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical [..] dependencies must specify a single x.y API version, or a range of versions with an upper bound.
What about x == 0? I mean, the intention of 0 is that 0.1, 0.2 and so on may well overthrow the API. But that's a bikeshed issue. ChrisK wrote:
The above declaratively expresses that libcurl-3.3.0 provides the version 3 API and the version 2 API.
Good idea. In the end, both proposals are based on the same observation: the dependency knows best which of its versions are compatible and which aren't. It's *not* a responsibility of the dependent package to know that. It only has to know one particular version it compiles against. So, it would just specify "I do compile with foobar 1.2" and let foobar 1.3 decide whether it's compatible with 1.2 or not. Given Simon's proposed policy, the answer would be "yes", if it were not for possible name clashes:
However, there's an important difference between shared-library linking and Haskell: in Haskell, a superset of an API is not backwards-compatible, because it has the potential to cause new name clashes.
Solution? explicit imports? explicit module signatures with subtyping? implicit signature conversion foobar 1.3 -> foobar 1.2 on the package level? Regards, apfelmus
ChrisK
Once it is possible to have cabal register the hsFoo-3.0.0 also as hsFoo-2 it will be easy to upgrade to hsFoo. No old programs will fail to compile.
Who here knows enough about the ghc-pkg database to say how easy or hard this would be?
Ignoring disk space, I suppose the motivation is that it will ease the user experience by only having to download, compile and install a single package? And perhaps ease the maintenance a bit for the library author, too. One way to do this would be to have multiple .cabal files in the package, with small differences like different version numbering. You can use a Makefile or other hack to automate switching. -k -- If I haven't seen further, it is by standing in the footprints of giants
On Wednesday 17 October 2007 01:32, ChrisK wrote:
Simon Marlow wrote:
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often.
I have another concrete proposal to avoid things breaking so often. Let us steal from something that works: shared library versioning on unixy systems.
On Max OS X, I note that I have, in /usr/lib:
lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.2.dylib -> libcurl.3.dylib lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.3.0.0.dylib -> libcurl.3.dylib -rwxr-xr-x 1 root wheel 201156 Aug 17 17:14 libcurl.3.dylib lrwxr-xr-x 1 root wheel 15 Jul 24 2005 libcurl.dylib -> libcurl.3.dylib
The above declaratively expresses that libcurl-3.3.0 provides the version 3 API and the version 2 API.
This is the capability that should be added to Haskell library packages.
Right now a library can only declare a single version number. So if I update hsFoo from 2.1.1 to 3.0.0 then I cannot express whether or not the version 3 API is a superset of (backward compatible with) the version 2 API.
If 3.0.0 is a superset of 2.1.1 why was it necessary to bump to 3.0.0? Why not 2.2.0?
1. Document the version numbering policy.
agreed. just making everybody's interpretation explicit has already exposed subtle differences, so documenting common ground will help.
We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where:
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
further sub-versions may be added after the x.y, their meaning is package-defined. Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest.
referring to a haskell function to compute ordering, or to parse version strings into lists of numbers, might remove ambiguities here. for instance, some people use patch-levels as sub-versions, some use dates. also, compare Simon's (S) with Daniel's (D) version: | If the convention for modifying package versions of form x.y.z is: | - increment z for bugfixes/changes that don't alter the interface | - increment y for changes that consist solely of additions to the interface, | parts of the interface may be marked as deprecated | - increment x for changes that include removal of deprecated parts of the | interface version D gives us strictly more information from a version number: just from number differences, we can tell what kind of changes happened to the api. i like that. version S is closer to current practice, which is less informative but psychologically motivated:-) if one does a substantial rewrite without changing the api, or if one adds fundamentally new features without breaking backwards compatibility, one likes to bump the leading number (that is no doubt inspired by commercialism: paying customers are said to prefer higher version numbers, and to focus on new features). corollary: after fixing the version numbering policy (policies?), the implications on usage need to be investigated (sorting wrt dates? does a version number tell us anything about which version can stand in for which dependency?).
2. Precise dependencies.
As suggested by various people in this thread: we change the convention so that dependencies must specify a single x.y API version, or a range of versions with an upper bound. Cabal or Hackage can refuse to accept packages that don't follow this convention (perhaps Hackage is a better place to enforce it, and Cabal should just warn, I'm not sure).
Yes, earlier I argued that not specifying precise dependencies allows some packages to continue working even when dependencies change, and that having precise dependencies means that all packages are guaranteed to break when base is updated. However, I agree that specifying precise dependencies is ultimately the right thing, we'll get better errors when things break,
agreed. please note, however, that this is likely to flush out issues that have so far been swiped under the carpet. this is a good thing, as it will lead to proposals for making cabal deal with these issues properly (replacing unspecified user complaints with concrete bugs and fixes). but it will increase the noise!-) claus
On Tue, Oct 16, 2007 at 01:08:49PM +0100, Simon Marlow wrote:
So rather than keep replying to individual points, I'd like to make some concrete proposals so we can make progress.
1. Document the version numbering policy.
We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where:
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
further sub-versions may be added after the x.y, their meaning is package-defined.
This should be required for at least the GHC boot packages (and encouraged for others). I would make "API extended only" a bit more precise: any module that uses explicit import lists will not be affected by the changes. So one can add classes, types and functions, but not instances (except where either the class or the type is new). You probably can't add data constructors or fields, and have to be careful with new methods. I'd also prefer that major versions used two numbers, because that's common now, it supports the experimental versions 0.x apfelmus mentioned, and it makes it easier to leave room for development versions (possibly using an odd-even scheme). If you make your development repository available, and it contains API changes, you'll want its version number to have a larger major number.
Ross Paterson wrote:
I would make "API extended only" a bit more precise: any module that uses explicit import lists will not be affected by the changes. So one can add classes, types and functions, but not instances (except where either the class or the type is new).
okay
You probably can't add data constructors or fields, and have to be careful with new methods.
If they're exported and new members of existing classes/datatypes, then you can't add them, because they might be imported with "class/typename(..)". (right?) What about semantic changes to the API? Including, adding a default to a class method changes the default from 'undefined', which someone might have relied on as the default (although it seems unlikely). Isaac
I've written down the proposed policy for versioning here: http://haskell.org/haskellwiki/Package_versioning_policy It turned out there was a previous page written by Bulat that contained essentially this policy, but it wasn't linked from anywhere which explains why it was overlooked. I took the liberty of rewriting the text. I took into account Ross's suggestions that the major version should have two components, and that we need to be more precise about what it means to extend an API. After a round of editing, we can start to link to this page from everywhere, and start migrating packages to this scheme where necessary. Cheers, Simon
On Wed, Oct 17, 2007 at 12:54:12PM +0100, Simon Marlow wrote:
I've written down the proposed policy for versioning here:
This says: If [...] instances were added or removed, then the new A.B must be greater than the previous A.B. This presumably includes changing module imports, or depending on a newer version of a package, which results in the visible instances changing? I think this should be spelt out in the policy. The example: build-depends: mypkg == 2.1.1 should be: build-depends: mypkg >= 2.1.1, mypkg < 2.1.2 with the current dependency syntax/semantics. Thanks Ian
On Wed, Oct 17, 2007 at 12:54:12PM +0100, Simon Marlow wrote:
I've written down the proposed policy for versioning here:
http://haskell.org/haskellwiki/Package_versioning_policy
It turned out there was a previous page written by Bulat that contained essentially this policy, but it wasn't linked from anywhere which explains why it was overlooked. I took the liberty of rewriting the text.
You wrote:
A client that wants to specify that they depend on a particular version of the API can specify a particular A.B.C and be sure of getting that API only. For example, build-depends: mypkg-2.1.1
Are you proposing an extension along the lines of that proposed by Thomas (and Bulat, and others), i.e. this would be equivalent to build-depends: mypkg >= 2.1.1 && < 2.1.2 ? The current syntax of mypkg == 2.1.1 would match the initial release but not subsequent patch releases.
Ross Paterson wrote:
On Wed, Oct 17, 2007 at 12:54:12PM +0100, Simon Marlow wrote:
I've written down the proposed policy for versioning here:
http://haskell.org/haskellwiki/Package_versioning_policy
It turned out there was a previous page written by Bulat that contained essentially this policy, but it wasn't linked from anywhere which explains why it was overlooked. I took the liberty of rewriting the text.
You wrote:
A client that wants to specify that they depend on a particular version of the API can specify a particular A.B.C and be sure of getting that API only. For example, build-depends: mypkg-2.1.1
Are you proposing an extension along the lines of that proposed by Thomas (and Bulat, and others), i.e. this would be equivalent to build-depends: mypkg >= 2.1.1 && < 2.1.2 ?
Yes, I should have mentioned that, thanks. I'll update the page shortly with this suggestion and others. Cheers, Simon
simonmarhaskell:
Several good points have been raised in this thread, and while I might not agree with everything, I think we can all agree on the goal: things shouldn't break so often.
So rather than keep replying to individual points, I'd like to make some concrete proposals so we can make progress.
1. Document the version numbering policy.
We should have done this earlier, but we didn't. The proposed policy, for the sake of completeness is: x.y where:
x changes ==> API changed x constant but y changes ==> API extended only x and y constant ==> API is identical
further sub-versions may be added after the x.y, their meaning is package-defined. Ordering on versions is lexicographic, given multiple versions that satisfy a dependency Cabal will pick the latest.
2. Precise dependencies.
As suggested by various people in this thread: we change the convention so that dependencies must specify a single x.y API version, or a range of versions with an upper bound. Cabal or Hackage can refuse to accept packages that don't follow this convention (perhaps Hackage is a better place to enforce it, and Cabal should just warn, I'm not sure).
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust. -- Don
Hi
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust.
In general, if it compiles and type checks, it will work. It is rare that an interface stays sufficiently similar that the thing compiles, but then crashes at runtime. Given that, shouldn't the tested versions be something a machine figures out - rather than something each library author has to tend to with every new release of every other library in hackage? Thanks Neil
Neil Mitchell wrote:
Hi
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust.
In general, if it compiles and type checks, it will work. It is rare that an interface stays sufficiently similar that the thing compiles, but then crashes at runtime.
True.. GoboLinux's package system records the exact set of versions something compiles with (just for reference), and uses min version bounds (and max bounds where needed) for dependencies. It's always possible for Haskell library implementation-bug-fixes to change relied-on behavior, as discussed in the original ECT description. I agree that compiling and type-checking is a pretty good sign of working. Passing tests (e.g. QuickCheck) could be tested too, where available. If optimizations and unsafePerformIO interact differently, different compiler versions could also affect whether something works correctly, but still compiles... But, the issue here is much more limited: we assume that there were some set of versions of these libraries that DID work, and, that every version of each library, on its own (or with only the libraries it depends on), works. So it might be valuable to record subjectively-working exact version sets, somewhere. Isaac
Neil Mitchell wrote:
Hi
I agree. >= 1.0 isn't viable in the long term. Rather, a specific list, or bounded range of tested versions seems likely to be more robust.
In general, if it compiles and type checks, it will work. It is rare that an interface stays sufficiently similar that the thing compiles, but then crashes at runtime. Given that, shouldn't the tested versions be something a machine figures out - rather than something each library author has to tend to with every new release of every other library in hackage?
The only reasonable way we have to test whether a package compiles with a new version of a dependency is to try compiling it. To do anything else would be duplicating what the compiler does, and risks getting it wrong. But you're right that tools could help a lot: for example, after a base version bump, Hackage could try to build all its packages against the new base to figure out which ones need source code modifications and which can probably just have their .cabal files tweaked to allow the new version. Hackage could tentatively fix the .cabal files itself and/or contact the maintainer. We'll really need some tool to analyse API changes too, otherwise API versioning is too error-prone. Anyone like to tackle this? It shouldn't be too hard using the GHC API.. Cheers, Simon
Following is a summary of my thoughts on the matter, in large part so I can figure out what I'm thinking... apologies if it's a bit of a ramble. All comments welcome. Basically - version numbering which differs from Simon's proposal - precise dependencies, I think the same as Simon is proposing - 'permanent' availability of compatible package versions - never a need to update working cabal files - a cabal file installs exactly one version of a package 1) Package version numbers are of the form x.y.z 2) There are version-segment ordering functions cmpX, cmpY, and cmpZ. cmpX and cmpY are globally defined and operate over non-negative integers. Perhaps cmpZ is globally defined, or could be defined per package, or be lexicographic, or... something else. cmpZ could even be a partial ordering I suppose. 3) A cabal file specifies how to build a single version of a package. name: foo version: 2.12.5 This cabal file will build version 2.12.5 of package foo. 4) The dependencies in a cabal file define baseline versions of required packages. depends: bar [3.4] baz [1.2.6, 3] Version 2.12.5 of foo requires a version of bar that is API-compatible with 3.4.0 and a version of baz that is API-compatible with 1.2.6 _or_ API-compatible with 3.0.0. Note that this doesn't imply that baz 3.0.0 is API-compatible with baz 1.2.6 (by definition it is not), it implies that foo is using a subset of the intersection of those two baz APIs. Note that baz 2.y.z would not satisfy the dependency. Perhaps a function was removed with the bump to 2 and restored only with the bump to 3. 5) Package version numbers encode whether one version of a package is API-compatible with another version of the package. Given two versions x.y.z and i.j.k of a package: - x == i && y == j ==> x.y.z is API-identical (hence API-compatible) with i.j.k, cmpZ can be used to determine preferred version - x == i && y > j ==> x.y.z is API-compatible with i.j.k, it has undergone compatibility-preserving changes, x.y.z is preferred to i.j.k - x > i ==> x.y.z is not API-compatible with i.j.k, it has undergone non-compatibility-preserving changes - otherwise ==> x.y.z is not API-compatible with i.j.k, it is a lower version that has less functionality 6) A compatibility-preserving change is generally a change which just adds to the API. Ross Paterson points out adding extra data constructors or instances may not be compatibility-preserving. A non-compatibility-preserving change is generally a change which includes the removal of some part of the API. It might also include changes which leave the API unmodified but significantly degrade usability, e.g. worse time or space performance. 7) Once a version of a package is building successfully it remains available for a 'long time'. If sufficient versions of a package remain available then API-compatible versions of required packages are always available, so the building of packages should never break. An uploaded cabal file should never need to be changed, regardless of what happens to the packages it depends upon. 8) If a version of a package is discovered to have security flaws or serious bugs it should remain available in a quarantined state until a fixed API-compatible version is available. 9) Something (hackage?) could enforce adherence to version numbering policy. At the least any new version uploaded that claims to be API-compatible can be test compiled against packages which depend on it. Something (hackage?) could assist package maintainers in releasing a new version of their package with updated dependency information. Hackage could attempt to compile against non API-compatible versions and report the outcome, for example foo 2.12.5 compiles with the new baz 3.0.0 but not the latest baz 2.y.z Dan
[would it be possible to pick a single list to discuss this on please, so there is no danger of some people missing some subthreads if they aren't on all the lists, or getting messages 3 times if they are?] On Tue, Oct 16, 2007 at 01:08:49PM +0100, Simon Marlow wrote:
2. Precise dependencies.
While not directly related to this, I have the impression some people want precise dependencies so that things work properly when multiple versions of a library are installed. Personally I'm not a fan of that, as if I have package foo: module Foo where data T package bar: module Bar where bar :: T package baz: module Baz where baz :: T -> () then baz bar might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions. Thanks Ian
On Wed, 2007-10-17 at 00:35 +0100, Ian Lynagh wrote:
[would it be possible to pick a single list to discuss this on please, so there is no danger of some people missing some subthreads if they aren't on all the lists, or getting messages 3 times if they are?]
On Tue, Oct 16, 2007 at 01:08:49PM +0100, Simon Marlow wrote:
2. Precise dependencies.
While not directly related to this, I have the impression some people want precise dependencies so that things work properly when multiple versions of a library are installed.
Personally I'm not a fan of that, as if I have
package foo: module Foo where data T
package bar: module Bar where bar :: T
package baz: module Baz where baz :: T -> ()
then baz bar might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions.
Sure, those are not compatible. But we can detect that just by looking at the dependencies of each package. The solution is to distinguish private and public dependencies and to slot packages on their own version numbers and all version numbers of the packages they publicly depend upon. Then in your example we'd just compile another copy of bar or baz using a compatible version of foo. This is what Nix does. Duncan
On Wed, Oct 17, 2007 at 01:28:45AM +0100, Duncan Coutts wrote:
On Wed, 2007-10-17 at 00:35 +0100, Ian Lynagh wrote:
package foo: module Foo where data T
package bar: module Bar where bar :: T
package baz: module Baz where baz :: T -> ()
then baz bar might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions.
Sure, those are not compatible. But we can detect that just by looking at the dependencies of each package.
In general it is possible to have 2 different versions of a package in a single program, though, e.g. bar and baz could internally use different versions of a binary-tree package. Thanks Ian
On Wed, 2007-10-17 at 20:28 +0100, Ian Lynagh wrote:
On Wed, Oct 17, 2007 at 01:28:45AM +0100, Duncan Coutts wrote:
On Wed, 2007-10-17 at 00:35 +0100, Ian Lynagh wrote:
might be a type error if I have multiple versions of foo installed and bar and baz have been compiled against different versions.
Sure, those are not compatible. But we can detect that just by looking at the dependencies of each package.
In general it is possible to have 2 different versions of a package in a single program, though, e.g. bar and baz could internally use different versions of a binary-tree package.
Right, but only if those packages are used privately, ie such that types from the dependent packages do not leak into their public api, since that would allow the type error you originally pointed out. If we distinguish private use of a dependent package then we know it does not matter if we end up using two versions of it in a single program. Duncan
participants (14)
-
apfelmus -
Bayley, Alistair -
ChrisK -
Claus Reinke -
Daniel McAllansmith -
Don Stewart -
Duncan Coutts -
Ian Lynagh -
Isaac Dupree -
Ketil Malde -
Neil Mitchell -
Ross Paterson -
Simon Marlow -
Stefan O'Rear