Re: Dependencies/backwards compatibility in Hackage

Sven Moritz Hallberg wrote:
Björn Bringert
, 2007-02-01 15.36 +0100: Ross Paterson wrote: I think that the correct solution to this problem would be to make a new release of haxr (which used to be XmlRpc) that works with HaXml 1.17.
But then what is your answer to the problem of things breaking in the time between the release of HaXml 1.17 and making a new release of haxr?
-Sven
I don't really have one, expect fast development :-). One possibility would be to have a convention about version number changes when libraries make backwards-incompatible API changes. E.g. HaXml should change major version whenever a change would break something that depends on it. Then all libraries which depend on HaXml could specify: Build-depends: HaXml >= 1.13 && < 2 or something like that. The only problem would be that it would be overly restrictive if a HaXml 2.0 comes out that the current version of the depending library actually works with. /Björn

Björn Bringert wrote:
Sven Moritz Hallberg wrote:
Björn Bringert
, 2007-02-01 15.36 +0100: Ross Paterson wrote: I think that the correct solution to this problem would be to make a new release of haxr (which used to be XmlRpc) that works with HaXml 1.17.
But then what is your answer to the problem of things breaking in the time between the release of HaXml 1.17 and making a new release of haxr?
-Sven
I don't really have one, expect fast development :-).
One possibility would be to have a convention about version number changes when libraries make backwards-incompatible API changes. E.g. HaXml should change major version whenever a change would break something that depends on it. Then all libraries which depend on HaXml could specify:
Build-depends: HaXml >= 1.13 && < 2
or something like that. The only problem would be that it would be overly restrictive if a HaXml 2.0 comes out that the current version of the depending library actually works with.
Eh that is what Ross already said. Sorry about that. This is the approach taken by Unix shared libraries by the way. /Björn

Hi
We're all assuming here that HaXml deliberately decided to change the
interface. Is that really true? (only Malcolm can answer)
Most of the time I guess that the interface will change by accident,
without people spotting. Perhaps we are trying to solve the wrong
problem...
Thanks
Neil
On 2/1/07, Björn Bringert
Björn Bringert wrote:
Sven Moritz Hallberg wrote:
Björn Bringert
, 2007-02-01 15.36 +0100: Ross Paterson wrote: I think that the correct solution to this problem would be to make a new release of haxr (which used to be XmlRpc) that works with HaXml 1.17.
But then what is your answer to the problem of things breaking in the time between the release of HaXml 1.17 and making a new release of haxr?
-Sven
I don't really have one, expect fast development :-).
One possibility would be to have a convention about version number changes when libraries make backwards-incompatible API changes. E.g. HaXml should change major version whenever a change would break something that depends on it. Then all libraries which depend on HaXml could specify:
Build-depends: HaXml >= 1.13 && < 2
or something like that. The only problem would be that it would be overly restrictive if a HaXml 2.0 comes out that the current version of the depending library actually works with.
Eh that is what Ross already said. Sorry about that. This is the approach taken by Unix shared libraries by the way.
/Björn _______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://www.haskell.org/mailman/listinfo/cabal-devel

Neil Mitchell wrote:
We're all assuming here that HaXml deliberately decided to change the interface. Is that really true? (only Malcolm can answer)
Yes, it's true. Even a non-Malcolm such as myself can answer this, because of the magnitude of the changes from 1.13 to 1.17. HaXml is somewhere in the process of a big spring cleaning. The real question that only Malcolm can answer is how far through the process of tidying up it is. That bears on whether it's worth bringing haxr up to date; if there's more churn ahead, the answer is probably "not yet".

Bryan O'Sullivan wrote:
Neil Mitchell wrote:
We're all assuming here that HaXml deliberately decided to change the interface. Is that really true? (only Malcolm can answer)
Yes, it's true. Even a non-Malcolm such as myself can answer this, because of the magnitude of the changes from 1.13 to 1.17. HaXml is somewhere in the process of a big spring cleaning.
The real question that only Malcolm can answer is how far through the process of tidying up it is. That bears on whether it's worth bringing haxr up to date; if there's more churn ahead, the answer is probably "not yet".
If an experimental HaXml version was added to Hackage, it should be tagged as such in some way. A cheapo way to do that is to call the package "HaXml-experimental" or some such. /Björn

Neil Mitchell
We're all assuming here that HaXml deliberately decided to change the interface. Is that really true? (only Malcolm can answer)
Yes, I had a glance at it and AFAICT he changed the name of the submodule Text.XML.HaXml.Xml2Haskell to something more suitable.
Most of the time I guess that the interface will change by accident, without people spotting. Perhaps we are trying to solve the wrong problem...
You have a point, interface change by accident is a different problem. But it doesn't make the other one wrong. Clearly not all interface change is by accident; see above. But without a way for the author to declare it, we're effectively treating all these instances as change by accident as well, which means more maintenance. -Sven

Björn Bringert
But then what is your answer to the problem of things breaking in the time between the release of HaXml 1.17 and making a new release of haxr?
-Sven
I don't really have one, expect fast development :-).
Okay, but that's unreliable. The way I see it, the upshot of it would be that cabal-install would only actually work half the time. No fun!
One possibility would be to have a convention about version number changes when libraries make backwards-incompatible API changes. E.g. HaXml should change major version whenever a change would break something that depends on it. Then all libraries which depend on HaXml could specify:
Build-depends: HaXml >= 1.13 && < 2
or something like that. The only problem would be that it would be overly restrictive if a HaXml 2.0 comes out that the current version of the depending library actually works with.
What's wrong with the solution I proposed, i.e. to add an explicit field to the package description which states whether we broke compatibility or not? It's effectively the same as a version numbering convention without having to tell everyone how to assign their numbers. Don't get me wrong, I'd be all for a consistent numbering scheme. Is everyone else also? In the end, I just want two things: - As a library author I want to be able to freely change my interface around without having to fear tons of programs breaking. Of course interface stability is good, but not everyone has an Architecture Review Board behind them that makes sure the interface actually makes sense. If I discover that it doesn't, I want to be able to change it, not stagnate on the old crud because of backwards-compatibility. - As a user, I just want cabal-install to work reliably. That can only happen if incompatibility is somehow avoided. Let's note that one solution lingering around is the Debian-like stable/testing/unstable model, i.e. every new package would go into unstable, break possibly tons of stuff there, which possibly tons of people will have to fix up, taking considerable time. Some things might not get fixed at all -> old programs become unusable. After some time things would propagate to testing, hopefully not breaking too much, and finally appear in stable in all their glory. The (IMHO) undesirable consequence of this would be, however, that you would get the Debian-like tradeoff between stability and having access to the latest developments. With the ability to declare your incompatibility, ideally combined with some automatic checks, unstable could be removed. We might still want to have testing because you never know, though. -Sven

On Thu, Feb 01, 2007 at 05:29:44PM +0100, Sven Moritz Hallberg wrote:
What's wrong with the solution I proposed, i.e. to add an explicit field to the package description which states whether we broke compatibility or not? It's effectively the same as a version numbering convention without having to tell everyone how to assign their numbers. Don't get me wrong, I'd be all for a consistent numbering scheme. Is everyone else also?
With version numbers, it suffices that each author has a rule of incrementing at a certain level if compatibility is broken. It need not be the same level for different packages (though that would be less confusing). They might have two levels of compatibility: merely adding functions, types or classes will not break any clients that use explicit imports.

Hi
With version numbers, it suffices that each author has a rule of incrementing at a certain level if compatibility is broken. It need not be the same level for different packages (though that would be less confusing). They might have two levels of compatibility: merely adding functions, types or classes will not break any clients that use explicit imports.
We can of course automate this with haddock --hoogle and diff quite easily. It doesn't get it perfectly - if the author changes the semantics or introduces a type alias it won't give the right result, but it should get the answer most of the time. Thanks Neil

Ross Paterson
With version numbers, it suffices that each author has a rule of incrementing at a certain level if compatibility is broken. It need not be the same level for different packages (though that would be less confusing).
They might have two levels of compatibility: merely adding functions, types or classes will not break any clients that use explicit imports.
Yes, good point. In fact, I didn't think of the fact that adding functions etc. can also break an importer. :/ It might indeed be a good idea to be able to distinguish these two levels. Already requiring old versions when just an export is added might not offer much benefit to treating all versions incompatible to each other in the first place... Actually, upon pondering the fact that it's impossible to enforce the usage of a particular numbering scheme, it occured to me: We could just make version numbers following the standard scheme (whichever exactly) syntactically distinguishable from the current form. Then cabal-install could work happily with its new information and we don't have to /impose/ a mandatory scheme on everyone. (We could, however, apply some leverage towards "persuading" authors to use the new scheme by emitting a warning from Cabal when it's not used.) Is this a good idea? Suggestions for what "standard-scheme" version numbers could look like? (Coolest look wins!) Regards, Sven

The way I propose to solve this version incompatibility problem in the short term is to use the "testing" hackage repository to collect packages that are known to work together. For instance, we could put haxml 1.17 in and only allow things that compile with that version. unstable can still have stuff that only compiles w/ 1.13, but if an author wants their tool to be in "testing", then they have to fix it up to be compatible. (Or we might decide that 1.13 is a better version to have in testing.) This will encouarage people to update their packages with a collection of dependencies that are somewhat "blessed", even if that blessing is a little arbitrary. We can ourselves modify packages so that their version dependency is more accurate than upstream knows them to be :) peace, isaac

On Sat, 2007-02-03 at 12:37 -0800, Isaac Jones wrote:
The way I propose to solve this version incompatibility problem in the short term is to use the "testing" hackage repository to collect packages that are known to work together. For instance, we could put haxml 1.17 in and only allow things that compile with that version. unstable can still have stuff that only compiles w/ 1.13, but if an author wants their tool to be in "testing", then they have to fix it up to be compatible. (Or we might decide that 1.13 is a better version to have in testing.)
This will encouarage people to update their packages with a collection of dependencies that are somewhat "blessed", even if that blessing is a little arbitrary.
We can ourselves modify packages so that their version dependency is more accurate than upstream knows them to be :)
It's for issues like this (ie creating a distribution) that having the .cabal file outside the .tar.gz file is a bonus. Let me compare to Gentoo: we provide a distribution of Haskell packages that we test as a group and know that they work together. We test stuff and make sure that the dependencies are sufficiently tight so that things don't break on users' systems. We have an .ebuild file for each package. For a haskell package this obviously duplicates most of the information in the .cabal file, however it gives us an opportunity to make slight corrections. The analogy in hackage is that the .cabal file outside the .tar.gz package can serve as the place where the distributors can 'fix' things. One example of this is the "tested-with:" field, another might be changing "build-depends:" to more tightly constrain version numbers. Duncan

Duncan Coutts
We can ourselves modify packages so that their version dependency is more accurate than upstream knows them to be :)
It's for issues like this (ie creating a distribution) that having the .cabal file outside the .tar.gz file is a bonus.
Let me compare to Gentoo: we provide a distribution of Haskell packages that we test as a group and know that they work together. We test stuff and make sure that the dependencies are sufficiently tight so that things don't break on users' systems. We have an .ebuild file for each package. For a haskell package this obviously duplicates most of the information in the .cabal file, however it gives us an opportunity to make slight corrections.
The analogy in hackage is that the .cabal file outside the .tar.gz package can serve as the place where the distributors can 'fix' things.
One example of this is the "tested-with:" field, another might be changing "build-depends:" to more tightly constrain version numbers.
Debian maintains diffs between the upstream version and the Debian version. It would be nice to do this with darcs instead. What I just did for HAppS, for instance, was update the .cabal file with a description and such, then did a darcs record and forwarded the .cabal file upstream, as well as modifying it within the .tarball that I intend to upload. What would be nice is if this were somehow integrated with darcs, so that I can do a "darcs send" such changes to hackage, which would relay it upstream, as well as keep track of the differences between the upstream version and the Hackage version. peace, isaac

On Feb 3, 2007, at 21:37 , Isaac Jones wrote:
The way I propose to solve this version incompatibility problem in the short term is to use the "testing" hackage repository to collect packages that are known to work together. For instance, we could put haxml 1.17 in and only allow things that compile with that version. unstable can still have stuff that only compiles w/ 1.13, but if an author wants their tool to be in "testing", then they have to fix it up to be compatible. (Or we might decide that 1.13 is a better version to have in testing.)
This will encouarage people to update their packages with a collection of dependencies that are somewhat "blessed", even if that blessing is a little arbitrary.
We can ourselves modify packages so that their version dependency is more accurate than upstream knows them to be :)
Is there a way to get that working smoothly with the web interface? I guess we could have a separate web interface installation for each "distribution", but maybe some built-in support could make it nicer to use? /Björn
participants (8)
-
Bjorn Bringert
-
Björn Bringert
-
Bryan O'Sullivan
-
Duncan Coutts
-
Isaac Jones
-
Neil Mitchell
-
Ross Paterson
-
Sven Moritz Hallberg