Hi!
First, sorry for some confusion - I wanted to post further details
needed for the analysis in the mail I lost, and I had excluded them
from my summary to keep it short.
On Sun, Sep 12, 2010 at 15:26, Tillmann Rendel
Hi Paolo,
Paolo Giarrusso wrote:
- when recompiling a package with ABI changes, does cabal always update dependent packages?
It never recompiles them. Recompilation should not be needed, because different versions of packages exports different symbols, so a package can never be linked against the wrong version of its dependency.
= the one they needed. An "Hello World program", using no recent functions, could thus be backportable even to an earlier version of
My problem was with reinstalling the same version of a package. Again,
sorry for the confusion (see above). cabal should then recompile (not
"update", strictly speaking) dependent packages:
Recompiling the same version of a package does not always yield the
same result, ABI-wise.
The problem was an ABI difference between two compilation results for
old-time-1.0.0.2, one linked against old-locale-1.0.0.1, the other
against version 1.0.0.2 of the same package. The ABI difference was
_probably_ due to different cross-module inlining decisions: the
disappeared symbols were:
oldzmtimezm1zi0zi0zi2_SystemziTime_a149_{closure,info},
i.e. (I assume) old-time-1.0.0.2-System.Time.a149. That name is
generated by GHC during optimization, and my educated guess is that it
is exported to allow cross-module inlining. In particular, a149 is
mentioned in the .hi interface of old-time's System.Time - <libs
dir>/old-time-1.0.0.2/System/Time.hi
It thus _seems_ that the ABI of a module is a (pure) function of the
versions of all its (transitive) dependencies, and their compilation
options - in particular, the optimization options and the used
compiler version.
More formally, my conjecture is:
- let us represent "package a depends on b" as "a =>> b", where a, b
are package names tagged with a version
- let =>>* be the transitive-reflexive closure of =>>
we need to compute:
DEPS(p) = {q | p =>>* q }
TAGGED_DEPS(p) = { (q, compilation_opts(q)) | q \in DEPS(p) }
where compilation_opts(q) are the compilation options used to compile package q.
Then, the ABI of a module is a pure function of TAGGED_DEPS(p), not
just of p. My experience proves at least that the ABI does not depend
just on p.
And since, after the discussion up-to-now, it turns out that this is
unexpected, I reported this as a bug:
http://hackage.haskell.org/trac/hackage/ticket/738
Another result of the above is that a mere "cabal install --reinstall
-O2 FooPackage", recompiling the same version of FooPackage with -O2
instead of the default (-O1), requires recompiling all packages which
depends on (i.e. use symbols from) FooPackage; recompiling them
implies recompiling packages depending on them, and so on, in a
"transitively closed" way.
However, see the following tickets:
http://hackage.haskell.org/trac/hackage/ticket/701 http://hackage.haskell.org/trac/hackage/ticket/704
Had a look, thanks - but they do not apply here, the problem is with Haskell symbols.
Interestingly, HTTP, directory, process, zip-archive were not reinstalled, which confirms that Cabal had reinstalled them before just because of an upgrade to the dependencies.
I think you are misinterpreting this.
When you asked cabal-install to install pandoc, it tried to make a consistent install plan for all its transitive dependencies. cabal-install will not touch a package which is not a transitive dependency of the package you request to be installed. Therefore, cabal-install will not touch Cabal if you ask it to install pandoc.
Thank you very much for the explanation. Note that the policy you describe is not necessarily "the right one", it is just a choice which can work, with the fixes you propose (read on for alternatives).
To make a consistent install plan, cabal-install has to pick exactly one version number for each of the transitive dependencies, so that all version constraints of all transitive dependencies are fullfilled. For some reason, cabal-install picked old-locale-1.0.0.2 instead of the already installed old-locale-1.0.0.1, and newer versions of HTTP, directory etc. too.
My case was slightly different, old-locale-1.0.0.2 (user-level) and .1 (system level) were already present. old-time-1.0.0.2 was installed at the system level, and linked against the older, system-level old-locale. So, while no dependency was broken, old-locale had been upgraded, by me or by cabal on its own. Even if it was my fault, only "cabal upgrade" gives a warning, while "cabal install --help" doesn't mention the issue, which is "suboptimal" :-D. And of course, cabal should prevent a user from shooting himself in his foot, but that's a more advanced feature. A consistent plan for installing pandoc would have been to either compile it using the older old-locale, or to recompile old-time against the new version. Sadly, cabal chose the latter alternative, since it is not obviously wrong, and this broke everything, except packages on which pandoc depended: they were recompiled because of the changed dep. I later forced the other install plan, with some pain, but it works. So cabal behavior is inconsistent: it knows that recompiling a package means recompiling its dependencies, but it should either do so also for packages unrelated to the one being installed, or it should not recompile any package (as you suggest).
I think this is the bug: cabal-install should not be allowed to install old-locale, because doing so apparantly causes havoc.
Lacking a proper dependency handling (which _is_ not trivial), yes. And it's the best current choice. But one can recompile and upgrade on a running system, without downtime nor glitches, both the C standard library and libraries without stable ABIs (see Gentoo Linux). Thus, upgrading old-locale and automatically rebuilding its dependencies would be a sensible feature. It just should not happen unless the user requests a full upgrade. Finally, while old-locale is a core package and one can restrict its upgrades, if I recompile non-core package A, and non-core package B breaks, that's still "havoc" - the only difference is that B is not cabal, and the bug is thus less severe and you can recompile manually B. OTOH, fixing this less severe and more complicated bug, via transitively closed dependency tracking as described above, helps supporting complete upgrades.
Looking at the inter-dependencies of pandoc's transitive dependencies, I do not see a reason to install a new version of a package instead of keeping the old.
Maybe it's somehow related to the transition from base-3 to base-4? The newer old-locale was already installed - see above my description of the two possible install plans.
For the transition, I had added: preference: base >= 4 based on a suggestion from somebody on haskell-cafe and the resulting discussion. -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/