Dependencies/backwards compatibility in Hackage

Hi everyone, While trying out cabal-install today, I stumbled across the following: I wanted to install XmlRpc which depends on HaXml and works with HaXml-1.13.2. However, HaXml-1.17 which has also been uploaded to hackageDB changed an exposed module's name and XmlRpc doesn't work with it anymore. I'm wondering how to avoid such situations. I appears to me that expecting the maintainer of every package depending on HaXml to update their dependencies "after the fact" doesn't make sense because it leaves an arbitrarily long window where the package won't build straight out of hackageDB. Wouldn't it be sensible for the author of HaXml to specify in the .cabal file for version 1.17 whether it is backwards-compatible or not? I'd imagine a field like "compatible:" or something to name any older versions that this version is backwards-compatible to, e.g. name: base version: 2.0 compatible: 1.0 ... The field would default to "not backwards-compatible to anything" so it would fail safe in the case where it's not given. There would still be the possibility of the maintainer erroneously stating compatibility where there is none. For example, she could set the field at some point and forget to update it. There might be a clever way to avoid this pitfall, though. What's the general opinion, should something like the above be added? (I'd be willing to volunteer for the code work.) On a side note, it probably wouldn't be too hard with existing tools to build a "gatekeeper" program to detect obvious kinds of flux in the API when uploading a new package to hackageDB: Check that all old modules are there, export all old functions, and maybe whether all the old functions still have the same type signatures (otherwise warn and ask for confirmation...), etc. Regards, Sven Moritz

On Thu, Feb 01, 2007 at 12:57:01PM +0100, Sven Moritz Hallberg wrote:
While trying out cabal-install today, I stumbled across the following: I wanted to install XmlRpc which depends on HaXml and works with HaXml-1.13.2. However, HaXml-1.17 which has also been uploaded to hackageDB changed an exposed module's name and XmlRpc doesn't work with it anymore.
I'm wondering how to avoid such situations. I appears to me that expecting the maintainer of every package depending on HaXml to update their dependencies "after the fact" doesn't make sense because it leaves an arbitrarily long window where the package won't build straight out of hackageDB.
We could decide on a standard interpretation of version numbers, e.g. major.minor.patch. To support this, we'd want wildcards like 1.13.* in version ranges. In the meantime, I'll remove HaXml-1.17 from HackageDB.

Ross Paterson wrote:
On Thu, Feb 01, 2007 at 12:57:01PM +0100, Sven Moritz Hallberg wrote:
While trying out cabal-install today, I stumbled across the following: I wanted to install XmlRpc which depends on HaXml and works with HaXml-1.13.2. However, HaXml-1.17 which has also been uploaded to hackageDB changed an exposed module's name and XmlRpc doesn't work with it anymore.
I'm wondering how to avoid such situations. I appears to me that expecting the maintainer of every package depending on HaXml to update their dependencies "after the fact" doesn't make sense because it leaves an arbitrarily long window where the package won't build straight out of hackageDB.
We could decide on a standard interpretation of version numbers, e.g. major.minor.patch. To support this, we'd want wildcards like 1.13.* in version ranges.
In the meantime, I'll remove HaXml-1.17 from HackageDB.
I think that the correct solution to this problem would be to make a new release of haxr (which used to be XmlRpc) that works with HaXml 1.17. On a side note, I think that XmlRpc should be removed, or there should be a way for one package to obsolete another. The XmlRpc package has been renamed to haxr, to have a more uniform naming within the package. /Björn

On Thu, Feb 01, 2007 at 03:36:00PM +0100, Björn Bringert wrote:
I think that the correct solution to this problem would be to make a new release of haxr (which used to be XmlRpc) that works with HaXml 1.17.
rss also uses HaXml. I'm not sure if Malcolm intends HaXml 1.17 to be released yet.
On a side note, I think that XmlRpc should be removed, or there should be a way for one package to obsolete another. The XmlRpc package has been renamed to haxr, to have a more uniform naming within the package.
It is gone.

Björn Bringert
Ross Paterson wrote: I think that the correct solution to this problem would be to make a new release of haxr (which used to be XmlRpc) that works with HaXml 1.17.
But then what is your answer to the problem of things breaking in the time between the release of HaXml 1.17 and making a new release of haxr? -Sven

Ross Paterson wrote:
We could decide on a standard interpretation of version numbers, e.g. major.minor.patch. To support this, we'd want wildcards like 1.13.* in version ranges. [...]
Yes, I think a solution like this (major: distruptive change, minor: backward compatible change, patch: no api change) would be very nice. Then one would be able to have many different incompatible versions encoding the major version name into the path of the library (or library name). OSX Frameworks for example use this solution (with a symlink to the latest version that is used for normal builds, and last used major version for build objects). Fawzi

"Sven Moritz Hallberg"
Ross Paterson
, 2007-02-01 12.14 +0000: We could decide on a standard interpretation of version numbers, e.g. major.minor.patch. To support this, we'd want wildcards like 1.13.* in version ranges.
Sounds compelling, but will everyone want/agree to follow that scheme?
I think we should just declare such a standard. Very few people care about version numbers outside of ghc-pkg, hackage, and cabal. If we pick something that works for us, I suspect that the majority of people will just go along, as long as it's reasonable. Does anyone want to declare such a standard? :) peace, isaac

Isaac Jones wrote:
"Sven Moritz Hallberg"
writes: Ross Paterson
, 2007-02-01 12.14 +0000: We could decide on a standard interpretation of version numbers, e.g. major.minor.patch. To support this, we'd want wildcards like 1.13.* in version ranges.
It would be a bit more tedious to type it every time, but wouldn't "HaXml >= 1.13 && < 1.14" be equivalent to "HaXml == 1.13.*"?
Sounds compelling, but will everyone want/agree to follow that scheme?
I think we should just declare such a standard. Very few people care about version numbers outside of ghc-pkg, hackage, and cabal. If we pick something that works for us, I suspect that the majority of people will just go along, as long as it's reasonable.
Does anyone want to declare such a standard? :)
Here's a draft based on Ross's proposal above. Please rip it apart. Executive summary: Library package version numbers should be on the format major.minor.revision, where major.minor must change whenever the API changes in a backwards-incompatible way. The revision identifier can have any number of segments, allowing snapshot releases etc. Background: Cabal package version numbers are non-empty sequences of natural numbers, separated by dots. Version numbers are ordered lexicographically. This is a proposed policy for how to assign such version numbers to library releases. Assigning version numbers: Library packages should have version numbers on the format major.minor.revision, where major and minor are natural numbers and revision is a sequence of natural numbers separated by dots. If the revision is the empty sequence, the version number has the format major.minor. Version numbers should be assigned so that software which uses the public API of the library should not be broken by an upgrade to a higher version with the same major.minor combination. Declaring dependencies: Package dependencies should be on a ranges of version numbers such that for each range, the smallest accepted version number is one that the package works with, and the largest accepted number has the same major.minor combination as a version that the package has been determined to work with. For example, if a package has been found to work with foo 1.2.5 (but not 1.2.4) and foo 1.3, the dependency should be declared as "foo >= 1.2.5 && < 1.4". Consequences: - you must change the major.minor number if you release a version where: - you change any exported names, or if you change the type of any exported entity to a type of which the old type is not an instance. - you export an additional name or module. - for each release that needs a new major.minor combinaton, you are free to choose whether to increment the major or the minor number. It is suggested that the major number is incremented when the changes are major. - you are free to choose the revision number scheme. Suggestions for how to assign revision numbers: - Revision numbers could be chosen sequentially, for example using "" (or ".0"), ".1", ".2" etc. - Another level could added to the revision number for snapshot releases, e.g. ".0.20070205", ".0.20070206". - To avoid confusion, it could be a good idea to not use "x.y" and "x.y.0" for different versions. Problems: - Packages will often have their dependencies too tightly specified, since many API changes don't all depending packages. However, having multiple versions of a package installed is probably better than random breakage. - Snapshot releases can't be required to change the major.minor number every time the API changes. Packages could possibly use an odd/even scheme to indicate stability. - Does "the public API" include all exported modules? Or does it exclude semi-public Internals modules etc? - Does "break" include fixing bugs (undocumented behavior) that a depending library relied on? /Björn

[...] Here's a draft based on Ross's proposal above. Please rip it apart.
Executive summary:
Library package version numbers should be on the format major.minor.revision, where major.minor must change whenever the API changes in a backwards-incompatible way. The revision identifier can have any number of segments, allowing snapshot releases etc.
As I said I really like the method used for frameworks on MacOSX (or for some libraries on linux) which is described here http://developer.apple.com/documentation/MacOSX/Conceptual/ BPFrameworks/Concepts/VersionInformation.html major: The major number means non backward compatible change, and is part of the name (on linux you see for example libreadline.5.so where 5 is the major revision). It is possible to have more than one major revision installed (very useful during transitions) minor: backward compatible change (additions to the api, bug-fix,...) revision: bug-fixes, development (can be absent), I would like to have this a lexicographically ordered string, so that one could use a date YYYYMMDDTHHMM plus some other "snapshot defining" string, or add a fourth non ordered string value now when a modules says that it needs 3.4.5 it means major=3, minor>=4, (if minor==4 rev>=5). I like the idea of having a tool that checkes the interface changes and automatically says if one should change minor or major version (it would work most of the time I think, and be very useful).
[...]Declaring dependencies:
Package dependencies should be on a ranges of version numbers such that for each range, the smallest accepted version number is one that the package works with, and the largest accepted number has the same major.minor combination as a version that the package has been determined to work with. For example, if a package has been found to work with foo 1.2.5 (but not 1.2.4) and foo 1.3, the dependency should be declared as "foo >= 1.2.5 && < 1.4".
a problem with this is that how much it works with future releases is difficult to predict. I major version change specifies a non backward compatible change there is a meaningful default: if not specified the package will not work with different major vesions, but will work with (higher) minor versions.
- for each release that needs a new major.minor combinaton, you are free to choose whether to increment the major or the minor number. It is suggested that the major number is incremented when the changes are major.
for the resons utlined before unless other feel that it is a too big ingerence in the version numbering I think that non backward comaptible changes should always increment the major number.
Suggestions for how to assign revision numbers:
- Revision numbers could be chosen sequentially, for example using "" (or ".0"), ".1", ".2" etc.
- Another level could added to the revision number for snapshot releases, e.g. ".0.20070205", ".0.20070206".
- To avoid confusion, it could be a good idea to not use "x.y" and "x.y.0" for different versions.
Problems:
- Packages will often have their dependencies too tightly specified, since many API changes don't all depending packages. However, having multiple versions of a package installed is probably better than random breakage.
- Snapshot releases can't be required to change the major.minor number every time the API changes. Packages could possibly use an odd/even scheme to indicate stability.
maybe one could add a stability flag, or add unstable to the revision string
- Does "the public API" include all exported modules? Or does it exclude semi-public Internals modules etc?
Exported modules has the advantage of being easily checkable, but I would allow for extensions of the private api to have the same minor number, otherwise development could be more difficult. On the other hand a hard approach to it might have advantages, published modules should respect the rules. Maybe here a stability flag could be useful.
- Does "break" include fixing bugs (undocumented behavior) that a depending library relied on?
I would say no, unless it is something that is expected to give major problems. If problems arise in a package one should modify its compatibility range. Fawzi
participants (5)
-
Björn Bringert
-
Fawzi Mohamed
-
Isaac Jones
-
Ross Paterson
-
Sven Moritz Hallberg