Re: nice little Cabal gotcha
On Thu, 2008-04-10 at 11:29 +0700, Peter Gammie wrote:
On 10/04/2008, at 3:57 AM, Duncan Coutts wrote:
On Mon, 2008-04-07 at 18:37 +0700, Peter Gammie wrote:
Imagine this: you're hacking a library, and occasionally you put a "- p" on your config line and sometimes you don't.
Yep. And Cabal doesn't track if the profiling versions of dependencies are installed.
(My cycle is occasional-clean-setup then build, which I grant might be a bit paranoid. I get impatient and fiddle with the profiling and optimisation options.)
Now, if one compiles a library with "-p", then without it, the old profiling library persists in the installed location, leading to some head scratching when a later compilation with a "-p" config option mysteriously fails.
You mean because the old version of the profiling lib is installed but it does not match the newer non-profiling version? So building a dependent lib with -p fails with weird complication errors?
Just to clarify slightly: "install" is just additive, so any old cruft in the install directories persists if not overwritten. If the package interface hasn't changed, the later "-p" compilation of something else may succeed, but misbehave relative to expectations.
Ah, so same interface but changed semantics.
My proposal is that if cabal is going to install something in, e.g.:
~/lib/package/ghc-6.6.1/
that it say:
rm -rf ~/lib/package/ghc-6.6.1/*
first. Surely it is reasonable to say Cabal "owns" that particular directory?
I don't think so, no.
I also feel the same way about documentation.
Now tracking dependencies a partial solution to this: imagine I remove a module from my .cabal file, then "clean" "build" "install". Will Cabal delete the removed-module's .hi (etc.) file?
No.
Can you expand on your squeamishness?
We should not delete files we don't know that we own. We only know we own files we're installing. So we cannot delete anything. Only a package manager can do that because it knows what files belong to the package we're replacing. Of course cabal-install is a package manager of sorts so if it tracked what files belonged to packages it installed then it'd know what files to remove when it replaced packages.
I can feel a proposal for a "--force" relative coming on, something like "--purge-old-crap". :-)
The other problem is that we do not necessarily install all files into one directory. In fact it's not even the default on unix. The various different kinds of files go in different dirs and the user can change any of them.
I'll write up a ticket after your response.
Perhaps I can convince you that the better solution is to get Cabal to provide a way track installed files and for cabal-install to actually do so. Duncan
On 14/04/2008, at 6:36 AM, Duncan Coutts wrote:
On Thu, 2008-04-10 at 11:29 +0700, Peter Gammie wrote:
On 10/04/2008, at 3:57 AM, Duncan Coutts wrote:
On Mon, 2008-04-07 at 18:37 +0700, Peter Gammie wrote:
Just to clarify slightly: "install" is just additive, so any old cruft in the install directories persists if not overwritten. If the package interface hasn't changed, the later "-p" compilation of something else may succeed, but misbehave relative to expectations.
Ah, so same interface but changed semantics.
Yes.
My proposal is that if cabal is going to install something in, e.g.:
~/lib/package/ghc-6.6.1/
that it say:
rm -rf ~/lib/package/ghc-6.6.1/*
first. Surely it is reasonable to say Cabal "owns" that particular directory?
I don't think so, no.
I also feel the same way about documentation.
Now tracking dependencies a partial solution to this: imagine I remove a module from my .cabal file, then "clean" "build" "install". Will Cabal delete the removed-module's .hi (etc.) file?
No.
Can you expand on your squeamishness?
We should not delete files we don't know that we own. We only know we own files we're installing. So we cannot delete anything.
As I said before, I disagree. Directories like ~/lib/package/ghc-6.6.1
are clearly Cabal-directories. (Please insert some $'s to make that
suitably relative). All sorts of gunk piles up in there while
developing. When hacking libs, one really wants "install" to mirror
what one has just built.
Consider this sequence:
Step 1:
configure -p
build
install
Step 2:
<change implementation but not API>
clean
configure
build
install
Now the profiling libs are out of date, Cabal "owned" them before but
not now...
Also consider:
Only a package manager can do that because it knows what files belong to the package we're replacing. Of course cabal-install is a package manager of sorts so if it tracked what files belonged to packages it installed then it'd know what files to remove when it replaced packages.
This is a non-solution: I've *changed* what is owned by the package. Is anyone really going to expect Cabal et al not to clobber the library install directory? What if I've already carefully installed a .a file of my own before "install"? I claim this is all so arbitrary that it is better for Cabal to simply throw everything out... the alternative is endless prompts: "do you really want me to clobber this file you don't care about?" or dodgy behaviour in corner cases.
I can feel a proposal for a "--force" relative coming on, something like "--purge-old-crap". :-)
The other problem is that we do not necessarily install all files into one directory. In fact it's not even the default on unix. The various different kinds of files go in different dirs and the user can change any of them.
Well, yes and no. I expect all "hi" files to end up in one place, and .a's and so forth. BTW can I really ask Cabal to install a lib into: ~/ and have all files end up there? (i.e. no ghc-6.6.1 etc. subdirs?) If not, we are arguing past each other.
I'll write up a ticket after your response.
Perhaps I can convince you that the better solution is to get Cabal to provide a way track installed files and for cabal-install to actually do so.
Would be nice. You need debian-style "remove using old .cabal" "install using new .cabal" persistence though, in case the .cabal has changed. cheers peter
On Mon, 2008-04-14 at 16:08 +0700, Peter Gammie wrote:
Can you expand on your squeamishness?
We should not delete files we don't know that we own. We only know we own files we're installing. So we cannot delete anything.
As I said before, I disagree. Directories like ~/lib/package/ghc-6.6.1 are clearly Cabal-directories. (Please insert some $'s to make that suitably relative). All sorts of gunk piles up in there while developing. When hacking libs, one really wants "install" to mirror what one has just built.
It may look that way but what if someone configures to use ~/lib directly? The directory structure we use is just a default, it's all completely configurable. We cannot rely on 'owning' anything.
Consider this sequence:
Step 1:
configure -p build install
Step 2:
<change implementation but not API> clean configure build install
Now the profiling libs are out of date, Cabal "owned" them before but not now...
Sure I see the problem.
Also consider:
<delete some modules from blah.cabal> configure build install and now we may have some .hi files orphaned by Cabal.
Ditto for documentation.
Yep.
Only a package manager can do that because it knows what files belong to the package we're replacing. Of course cabal-install is a package manager of sorts so if it tracked what files belonged to packages it installed then it'd know what files to remove when it replaced packages.
This is a non-solution: I've *changed* what is owned by the package.
Exactly! That's just why we need a package manager. It remembers what files belonged to the Foo package before and now that we're installing Foo again it knows that we're *replacing* Foo, so it's safe to delete all the old files that belonged to the Foo package and install all the new Foo files (and remember them for later).
I claim this is all so arbitrary that it is better for Cabal to simply throw everything out... the alternative is endless prompts: "do you really want me to clobber this file you don't care about?" or dodgy behaviour in corner cases.
This behaviour is completely consistent with all other build systems I know of. We really need package managers to solve the problem.
BTW can I really ask Cabal to install a lib into:
~/
and have all files end up there? (i.e. no ghc-6.6.1 etc. subdirs?)
Yes you can do that.
If not, we are arguing past each other.
Would be nice. You need debian-style "remove using old .cabal" "install using new .cabal" persistence though, in case the .cabal has changed.
Well, as I said, the package manager just needs to remember the set of files that belong to each installed package. A related problem is that we need to solve this problem at least within the ./dist/ directory. One of your examples was changing a .cabal file to add/remove a module. Currently we would not delete any existing .hi or .o files within ./dist/. As it happens for .o and .hi we copy out exactly the ones we need, but that's not the case for the documentation for example so we might end up copying stale .html files. To do that properly we'd need to track generated files much more closely. Duncan
participants (2)
-
Duncan Coutts -
Peter Gammie