2008/5/21 Galchin, Vasili :

Hi Dan,

I am still looking into this myself. I just stumbled across the URL below. i would suggest keeping an eye on this URL for more news.

I quit using the ubuntu debs. I've just been downloading the linux binaries and runing them in some odd subdir, and using --prefix=/path/to/my/ghc with cabal, and much happier for it. Is there some value with the official debs that I'm missing? -- Darrin

...

https://bugs.launchpad.net/ubuntu/+source/gtk2hs/+bug/229489 .... this is almost identical to my problem. I am just trying to help others on this list who are using Ubuntu Linux to avoid my predicament!

I had that problem upgrading too.

I posted a workaround:

http://bugs.launchpad.net/ubuntu/+source/gtk2hs/+bug/229489/comments/9

|What's going on: |-------------------------- |Every Haskell library gets registered with the Haskell compiler ghc. |The ghc-pkg program is used to do that. Packages get registered |(ghc-pkg register) when they're installed and unregistered (ghc-pkg |unregister) when they get uninstalled or upgraded. To be unregistered |they have to be registered in the first pace. |So what's gone wrong is that ghc itself has been upgraded before the |other libs (gtk2hs, mtl etc) were unregistered.When ghc gets upgraded |it includes an empty package database. So now when it comes time to |upgrade the other Haskell libs they are of course no longer registered |(because they were registered with the older version of the compiler |and not the new one). The uninstall script then fails because ghc-pkg |unregister fails, since the package is not registered. this sounds as if the uninstall simply needs an option deciding between uninstall-or-fail and achieve-uninstalled-state? but then again, uninstall should know from which compiler version/package database to uninstall, and uninstalling ghc should not disappear things without notifying other (un-)installation tools. i'll abuse this real-life opportunity to point once again at a serious flaw in the "throw-away installations" idea in common use for haskell compilers and libraries: there is no support for upgrading or cross-version consistency! throw-away installation: - if you install a library or a compiler version or tool, it puts itself in a separate place, free from interference with other libraries or compiler versions - usual practice then cleans away/forgets about the source for the installation, so the installation is not repeatable; it also forgets about the other dependencies that were used to build this installation (specific package or compiler versions needed to build this library/compiler/tool) - if you uninstall a library or a compiler version, it just removes itself (and even that isn't all that well supported, beyond unregistration..) there are so many ways this can go wrong that it is surprising that the real-life bugs have taken so long to show up.here are some examples (many of which have been discussed earlier, but have always been tracked as low priority/nice to have..): - if i update my ghc, i want to take all my libraries with me (especially since binary incompatibility means that i can't just reuse the existing installations..) - if i uninstall a version of ghc, i want to be reminded of which libraries are installed only for this ghc version (the libraries i'd lose by uninstalling, as opposed to libraries also installed for later ghc versions), and which installed tools were built using this ghc version (in case upgrading them still requires that old version..) - if i install a new library, i don't want version conflicts because an older, incompatible library version is in use by other packages - if i uninstall a library, i don't want dangling references to it in the package database - i don't want to have to remove anything explicitly, because that would mean bypassing the haskell installation managers - i would want to see a single haskell installation manager for each system, covering compiler versions as well, not to mention collaboration between all cabal/ghc-pkg versions currently installed; so that i can ask about all the package databases/compiler versions/packages; this might also help as a coordination point for upgrades - i would like to be able to determine, for each installed tool, library and compiler, which package and compiler versions were involved in building it, by querying that haskell installation manager - .. it just isn't sufficient not to interfere with existing stuff, it is important to be aware of relationships with existing stuff, and to adjust actions and practices accordingly (to guard against inconsistencies/disappearing dependencies, and to support upgrades/reproducable installations). perhaps the existing tickets in this direction could be given higher priorities, and the various groups and tools concerned with packaging could coordinate on this? claus

Simon Marlow wrote:

Claus Reinke wrote:

...

- i don't want to have to remove anything explicitly, because that would mean bypassing the haskell installation managers - i would want to see a single haskell installation manager for each system,

I think that's fundamentally the wrong approach. We shouldn't have to build a "Haskell installation manager". Would you also want installation managers for Perl, Python, Ruby, C, C++, etc. each with their own different user interfaces and feature sets? I think not - you want a single package manager for the whole system from which you can install/uninstall libraries for any language.

This is something that Windows gets completely wrong. Why do I have twelve icons in my status bar all representing little programs that are running in the background checking for updates to their own bits of software? Why on earth do I have a Printer Driver Update Manager? And I'd be wondering the same thing about a "Haskell installation manager": installation and dependencies are not something specific to Haskell.

Well, then there are developers who don't want to do .ebuilds, .rpms for 20 distributions, .debs for 20 distributions, .cabs... Meaning that if you have a project with 5 developers using 3 1/2 distributions, you will have a hard time installing. Haskell code tends to be platform unspecific, one shouldn't have to write platform-specific installation code just to make users happy. You have a point, though, and I wouldn't mind at all cabal-install being integrated into portage, that is, make portage _understand_ .cabal files and introduce another field in them that specifies non-haskell (e.g. gtk) dependencies. That is: I'd like to see a cabal-install for every system, using native package management where possible. Aren't there any usable third-party package managers for windoze? Maybe we can hook properly into cygwin, though it's surely not meant to support non-binary packages. Maybe gentoo should start to do binary releases, too, superseding debian and any other distribution. On another approach vector, I wouldn't want to update my Eve client using portage. It has to check whether there's a patch available before it connects to the server, anyway, so it can as well update. -- (c) this sig last receiving data processing entity. Inspect headers for past copyright information. All rights reserved. Unauthorised copying, hiring, renting, public performance and/or broadcasting of this signature prohibited.

On Wed, 2008-06-04 at 14:54 +0200, Ketil Malde wrote:

...

You have a point, though, and I wouldn't mind at all cabal-install being integrated into portage,

I'm not too familiar with portage, but I think a better solution is to provide tools to automatically generate packages for the various systems.

Yeah, we've got one for gentoo/portage called hackport. That's why we have so many packages in the haskell overlay. We didn't do them all manually :-)

How would you specify dependencies on non-haskell components in a portable way?

Unfortunately the same packages get named differently on different package systems, eg "zlib", "libzlib", "libzlib-dev" etc. It's probably possible to use reasonably standard names and map to the distro-names. Doing that reliably is not going to be easy though. Duncan

Ketil Malde wrote:

Achim Schneider writes:

Caveat: I have only a vague grasp on what exactly is being criticized here - using a modern Linux distribution, tons of packages are available, and almost all issues Claus point out seem to be taken care of - at least as far as I can see.

...

Well, then there are developers who don't want to do .ebuilds, .rpms for 20 distributions, .debs for 20 distributions, .cabs... Meaning that if you have a project with 5 developers using 3 1/2 distributions, you will have a hard time installing.

I think you should either require your developers to use the system that is provided to them, or be able and willing to maintain their own system. Most large Linux distributions seem to come with lots of Haskell-related stuff nowadays - 139 packages on my Ubuntu install (divide by something close to 3, as most library stuff comes in -dev, -doc and -prof variants).

Well, you have a point but still don't have one. Many of gentoo's haskell .ebuilds are seriously outdated, eg. wxhaskell still depends on ghc 6.4. See "Damnit, we need a CPAN" The haskell overlay features about 240 packages from alex to yi, hackage currently lists 596 packages. There are always things that a distribution doesn't include, especially sparsely used special purpose software. Compiling a LADSPA plugin by hand isn't that much of an issue, but you'll get into problems as soon as you want your programs to find it without touching paths that only your system's package manager should touch. I'm proud to say that my current gentoo installation is still the first one, surviving several world updates and at least 4 years of hacking around, using a lot of unstable and masked packages.

...

You have a point, though, and I wouldn't mind at all cabal-install being integrated into portage,

I'm not too familiar with portage, but I think a better solution is to provide tools to automatically generate packages for the various systems. How would you specify dependencies on non-haskell components in a portable way?

By using portage ;) Seriously: Gentoo isn't a distribution, but a meta-distribution. It wouldn't make much sense to support the generation of alien binary packages, though, as dependency names will surely differ, and if you have to generate the whole distribution, you can equally well just use portage to install it. Regarding non-haskell dependencies: It's already a problem from distribution to distribution. In portage, you would have to depend on e.g. either gtk+ or emul-linux-x86-gtklibs (if you want to build 32bit software...), in debian on gtk-dev. Each distribution would have to make a list on how cabal's non-haskell dependencies map to their own package names, which is seriously less work than figuring these out by hand. -- (c) this sig last receiving data processing entity. Inspect headers for past copyright information. All rights reserved. Unauthorised copying, hiring, renting, public performance and/or broadcasting of this signature prohibited.

Ketil Malde wrote:

The most usable one I've seen is Steam from Valve, IIRC. It'd be cool if Haskell packages were provided this way.

Cygwin's installer seems to work pretty well, too. Cheers, Simon

On Wed, 2008-06-04 at 11:33 +0100, Simon Marlow wrote:

Claus Reinke wrote:

...

- i don't want to have to remove anything explicitly, because that would mean bypassing the haskell installation managers - i would want to see a single haskell installation manager for each system,

I think that's fundamentally the wrong approach. We shouldn't have to build a "Haskell installation manager". Would you also want installation managers for Perl, Python, Ruby, C, C++, etc. each with their own different user interfaces and feature sets? I think not - you want a single package manager for the whole system from which you can install/uninstall libraries for any language.

As I see it we need both. We need to make it easy to translate cabal packages into distro packages. We do have tools to do that at the moment for Gentoo, Debian and Fedora. I'm sure they could be improved. However we cannot expect all distros (esp Windows) to have all packages that are on hackage at all times. That's where it makes sense to have a tool like cabal-install as a secondary package manager. There's also the fact that most distro package managers do not handle unprivileged per-user installations very well. A further issue is that the dependencies that Haskell packages have are pretty complex and more so than can actually be expressed in some distro package systems. In particular Gentoo portage. Binary distros are better off in that respect since the deps of binary haskell packages are considerably simpler. Duncan

Duncan Coutts wrote:

On Wed, 2008-06-04 at 11:33 +0100, Simon Marlow wrote:

...

Claus Reinke wrote:

...

- i don't want to have to remove anything explicitly, because that would mean bypassing the haskell installation managers - i would want to see a single haskell installation manager for each system, I think that's fundamentally the wrong approach. We shouldn't have to build a "Haskell installation manager". Would you also want installation managers for Perl, Python, Ruby, C, C++, etc. each with their own different user interfaces and feature sets? I think not - you want a single package manager for the whole system from which you can install/uninstall libraries for any language.

As I see it we need both. We need to make it easy to translate cabal packages into distro packages. We do have tools to do that at the moment for Gentoo, Debian and Fedora. I'm sure they could be improved.

However we cannot expect all distros (esp Windows) to have all packages that are on hackage at all times. That's where it makes sense to have a tool like cabal-install as a secondary package manager. There's also the fact that most distro package managers do not handle unprivileged per-user installations very well.

Well, that's true. I guess what I'm really objecting to in Claus's message is the implication that we should always use a Haskell Installation Manager, even on systems with good built-in package management. Yes, I agree we need good support for managing packages for the other scenarios: no system package manager, home-directory installs, no pre-prepared system package. I just don't want whatever provision we make for these cases to replace the system package manager for global package installs on systems where that is well supported. Cheers, Simon

...

Well, that's true. I guess what I'm really objecting to in Claus's message is the implication that we should always use a Haskell Installation Manager, even on systems with good built-in package management.

what was implied was that haskell installation manager (HIM) and native package managers (NPM) (where they exist) should collaborate, so that NPMs know how to extract dependencies from haskell packages, and HIM knows how to extract dependencies (haskell or otherwise) from NPMs. for the NPM-calls-HIM direction, that could be dynamically or, as in current practice, in a separate phase converting HIM to NPM packages. for the HIM-calls-NPM direction, that would simply present a haskell view on the native software (ie, cabal install/list/.. negotiating with NPM). whereas, currently, if i understand correctly, systems with NPM will have many haskell packages under NPM control and some haskell packages under HIM control and some haskell packages under no control whatsoever (eg, installing tools or compilers leaves no record, necessitating use of autoconf & co). i'm not saying "reimplement and ignore NPMs", i'm saying "NPMs and HIM should collaborate". if anything, that would reduce the number of exceptions where haskell packages are installed without the NPM knowing about them. and i'd like to see fewer cases where HIM forgets about what it or NPM have installed. so, on systems with NPM, users have a choice of interface, but nothing bad will happen no matter whether they choose NPM directly or HIM as an interface to it. and haskell book and tutorial authors only need to explain HIM, and its commands will just work - independent of the variant or lack of NPM on the reader's system. is that still objectionable?-) claus ps. it is a lot like using fmap, no matter what the Functor is, or using something like System.Process, no matter whether or not the OS supports that easily.

...

Yes, I agree we need good support for managing packages for the other scenarios: no system package manager, home-directory installs, no pre-prepared system package. I just don't want whatever provision we make for these cases to replace the system package manager for global package installs on systems where that is well supported.

Indeed. I wholly agree.

Duncan

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Wed, 2008-06-04 at 15:14 +0100, Claus Reinke wrote:

...

I think that's fundamentally the wrong approach. We shouldn't have to build a "Haskell installation manager". Would you also want installation managers for Perl, Python, Ruby, C, C++, etc. each with their own different user interfaces and feature sets? I think not - you want a single package manager for the whole system from which you can install/uninstall libraries for any language.

This is something that Windows gets completely wrong. Why do I have twelve icons in my status bar all representing little programs that are running in the background checking for updates to their own bits of software? Why on earth do I have a Printer Driver Update Manager? And I'd be wondering the same thing about a "Haskell installation manager": installation and dependencies are not something specific to Haskell.

why then do we have ghc-pkg, or cabal? surely the native package managers should handle all that?

there are (at least) two dimensions of uniformity: across different kind of software on a single kind of system, and with a single kind of software across different kinds of system. platform-specific package managers hide the software-specific notions of package dependency maintainence, haskell-specific package managers hide the platform-specific notions of package dependency maintainence.

there is no need for platform- and haskell-specific tools to be entirely separate or in conflict with each other: where both exist, one can be a view on the other (if you are on linux-of-the-day, you can use its package manage, independent of whether your packages are haskell or lisp; and if you are using haskell, you should be able to use its package manager, independent of whether you are on unix-variant-of-today or on unix-variant-of-yesterday).

there seems to be a lot of confusion here, some of us not understanding the issues because we happen to be using systems where "everything just works", others among us not understanding the issues because we happen to be using systems where "such things would never work anyway", and yet others insisting on "i'll do it my way, so i know what works" (and then, of course, there are those who are actively working on improving the situation who will see my criticism as constructive, i hope!-).

1. there are no systems where "packages just work"! there are systems where a few people ensure that many people can live in such an illusion, though.

Yes indeed! :-)

2. systems with native package manager software still need help from haskell-specific toolchains (unless you want the human package managers on those systems to code all haskell-specific dependencies by hand).

Yes. As an illustration: gentoo has an "haskell-cabal.eclass" that interfaces between ebuilds and cabal as the build manager and there is a tool to generate ebuilds that use the haskell-cabal.eclass from .cabal descriptions (so we get correct deps automatically).

3. systems without native package managers (or perhaps i should say: systems on which users with unix background traditionally avoid getting acquainted with the details and usage of whatever might pass as installation management on those systems) are still in *very* wide-spread use, and if haskell users on those systems are left out in the rain, haskell developers will not be able to support those systems. this limits the user and application base of haskell on those systems, making haskell less relevant than it could be.

Eg Windows, OSX, Solaris.

4. haskell enables programming at a very high level of abstraction, with fairly good support for mostly platform independent code. but that code needs to be installed, and integrated with dependencies and dependents, and the integrated haskell installations needs to be maintained. and that should "just work", even if the developer is on (1;2) and the user is on (3), or vice versa, or if developers and users are on different flavours of (1;2) or (3).

with these clarifications out of the way, my interpretation of cabal was that it set out to provide two things

(A) a uniform platform-independent interface to such a haskell package installation manager. (B) a uniform platform-independent toolchain to support such a haskell package installation manager.

I guess so.

on systems in the (1;2) class, human package managers would use (B) to integrate haskell packages into the native package management software, so users might never even encounter cabal.

As in my example with the gentoo haskell packages above.

even so, (A) might offer a haskell-specific view on the general platform package management (when you want to see the haskell gui libs rather than all gui libs).

on systems in the (3) class, users and developers would interface with (A/B) directly, for lack of a better alternative.

and developers/users in the (4) class would simply use (A/B), without having to check whether they are "real" or just a view on the platform-specific software. it is this cross platform view of software development that i'm most interested in (i'm one of those who use bash, vim, opera, and haskell, no matter whether i'm on windows, solaris, or whatever, and the cross-platform availability of those tools has saved me many a headache;-).

returning to my earlier message, it seems that my concerns were mainly these:

- it isn't sufficient to worry about installation management, one has to worry about integration, lifetime and uninstall management as well. in short, maintain the dependency graphs over any of "install"/"upgrade"/"uninstall".

For users in (1;2) class we would expect the native package manager to do this.

- for this to work, cabal needs to maintain not only libraries as packages, but tools and compilers as well. without this, some dependencies are not recorded (this haddock depends on that ghc; to build this package i need that tool; that tool was built with this ghc version, from those sources, etc).

and if the dependencies are not even recorded, they are likely to be broken if one does install/upgrade or uninstall any haskell software, be it library, tool, or compiler.

Right. Currently cabal-install in its roll as a secondary package manager is very weak on that front. It does not record anything about what it installs. There's clearly plenty of work to be done there.

ps. i'd cc to libraries or cabal, but i don't know which would be most appropriate (perhaps someone could point readers there to this thread if that sounds relevant?).

You can cc both. Duncan

On Wed, 2008-06-04 at 17:22 +0200, Achim Schneider wrote:

The question, IMHO, seems to be

"How would a package manager for a posix-compilant kinetic look like?"

http://nixos.org/index.html Duncan

Darrin Thompson wrote:

On Wed, Jun 4, 2008 at 7:48 PM, Duncan Coutts wrote:

...

On Wed, 2008-06-04 at 17:22 +0200, Achim Schneider wrote:

...

The question, IMHO, seems to be

"How would a package manager for a posix-compilant kinetic look like?"

http://nixos.org/index.html

How well does that coexist with other OS's? Could I have it manage only specific applications in it's store and trust the OS for the rest?

Yes. You can install nix (the package manager) under linux (and, I think, other posix-compiant OS) and use it in parallel with native packages (.deb, .rpm, whatever). This means you'll get some duplication, since the nix store contains everything needed to build and run the installed packages. The nix /package database/ contains not only nix package descriptions but relies essentially on build-scripts, which are mostly written in shell. So to use that, you'd have to have a posix-shell on your system, too. Cheers Ben

"Claus Reinke" writes:

1. there are no systems where "packages just work"! there are systems where a few people ensure that many people can live in such an illusion, though.

Exactly. Integrating Cabal packages into the system package manager is still non-trivial, and a package maintainer is necessary for this to work. Which is why it's rather utopian to hope that all of Hackage can be found in even the most widely used distributions, since many of those are way to narrow in scope for somebody to take on the work of providing package management. I bet some aren't even maintained, and possibly don't work with current GHC or Cabal. But I digress.

- it isn't sufficient to worry about installation management, one has to worry about integration, lifetime and uninstall management as well. in short, maintain the dependency graphs over any of "install"/"upgrade"/"uninstall".

There is an important difference between Hackage and, say, Ubuntu, and that is that the latter does releases. If I use a particular Ubuntu version, I get a particular GHC version, and a particular version of some dozens of libraries and applications. They are all compiled with that GHC version, tested, and can to a reasonable degree, be expected to work together. Sometimes they don't, but the good news is that a zillion other users are in the same boat (well, even for Haskell, it's probably dozens, at least), and chances are the bug will be fixed. If I upgrade to the next version of Ubuntu, I can reasonably expect all packages to be updated to current versions, and some to be replaced. In contrast, for my packages on Hackage, I'm reasonably certain it worked on my particular system at the time I uploaded it. And dependencies are none too specific - the package might work with different versions than I compiled with or not. I think for Cabal to provide the convenience currently afforded by system package managers, you'd need to do Hackage-wide releases, and test everything together. This is a lot of work, but not impossible, it's what Gnome and KDE does, for instance. It would probably make things easier for system package maintainers, too. Alternatives, like pessimistic dependencies: only exact versions in cabal files, will probably make you end up with installing every version of every package, and still suffer dependency hell. Exhaustive testing of all version combinations, which will require a lot of computational resources, even if it could be done automatically. -- I generally install most libraries from the distribution's repository, and let the system worry about updates etc. I think it's also an advantage if my programs avoid depending on stuff that's not in the repo, since it's likely to be is less widely used and maintained. I install manually the stuff I need the distribution doesn't provide, and also when I want to look at the code, or just have the latest and greatest. I tend to get the darcs version, rather than the Hackage one, though. The downside is that I need to reinstall these when I do a system upgrade, but it's rare enough that it doesn't bother me much, and chances are that more libraries are provided with my distribution for every release, so a cleanup is probably a good idea anyway. Anyway, this works pretty well for me - if I'm living on an illusion, it's at least a good one. :-) -k -- If I haven't seen further, it is by standing in the footprints of giants