Issues with cabal2arch/AUR packages

I guess I am creating this thread due to the frustrations of using cabal2arch/AUR for haskell packages (instead of using the standard cabal install). I previously had issues, mainly with having to manually update AUR installed cabal packages due to them not updating (which was recently fixed by forcing the update through ghc) This however seemed didn't seem to solve the issue completely, although it will update fine through a yaourt -Syu --aur (or clyde -Syu --aur or whatever package helper you use), it still ends up breaking GHC because all the hashes for the packages change and it was a forced update I unfortunately don't have the output message, since I was in a rush to get things working (due to my job), I just ended up doing a clyde -R --cascade ghc and just installing my package through cabal). The package in question was basically yesod (which pushes around 20 or 30 packages on a fresh install) and the updates had to do with some components of yesod being updated In fact I think that yesod is a perfect example in showing the issues when trying to integrate archlinux package with cabal package management, since there are a lot of packages in yesod (like authenticate/cookie) which cannot use the latest version of the package on AUR, where as there are upgrades with the other packages. As I mentioned earlier, I just ended up giving up. Right now I only have ghc installed (through official arch repository) and any package I need to install, I do so with cabal. The problems with this are obvious however, but right now its not an issue. The main issue being installing haskell binaries that are on AUR which has the issue of duplication if that binary depends on a library that is already installed through cabal install. In my opinion the main issue seems to be that the granularity of cabal package system is a lot finer then Arch's (cabal was purpously designed for this, and its great for haskell packages), and unlike with other languages, the situation where you have a single package that depends on a lot of both updated and outdated packages is very common, along with GHC's runtime system updates forcing recompilation of libraries every haskell-platform release. This isn't so much of an issue now regarding binaries (because to be honest, we don't have that many heavily used haskell based binaries apart from xmonad), but in the future should haskell "break through" and become a much more widely used language, this can create a pain, especially for haskell-binaries (and its already painful with haskell-libraries) Of course, trying to force archlinux package management to have the same granularity as arch's is out of the question, which I guess made me come with my second idea. Wouldn't it just be smarter to have the archlinux haskell environment set up like this. Archlinux's package management is one that makes some sacrifices (in regards to having a rolling release system thats always up to date at the cost of granularity in certain areas) - ghc is an official package in the official repository - haskell-platform will be the same as it is currently (uses GHC, and cabal2arch packages from both the official repository and AUR). This is fine since the point of haskell-platform is the standard packages it uses are frozen for half a year. So the only haskell libraries on AUR would be the ones in haskell-platform (and arguably these should be in the official repository, as well as the base haskell-platform package) - All other libraries users would just install/update with cabal install and not through AUR/cabal2arch/bauerbill etc etc. - Haskell packages that provide binaries would check for their dependencies using cabal/ghc-pkg, and if a haskell binary can't find a package it can install it in a temp directory/and or install it automatically through cabal install (depends if its an updated/outdated/etc etc). If the library will be built to a temp section, then it will just be linked to GHC manually when building that binary (the PKGBUILD will cover this). This also allows building of packages that use outdated packages (such as leksah as an example, its not possible to build this package if you have newest haskell-gtk install, you have to manually do dependency tracking). Just like with cabal2arch PKGBUILD generator, someone can easily create a program which will create PKGBUILDS out of hackagedb binaries, the actual PKBUILD will search for dependancies through cabal (and not the PKGBUILD "depends"). Binaries that statically link libraries (which is the case for almost all haskell binaries at this point) do not actually require the libraries they depend on to build to be installed after the executable has been built, so doing temp library installs to build a binary will not be an issue (C/C++ binaries already do this in circumstances, and its something that can easily be automated through hackagedb) I am just saying this, because my experience of using cabal2arch/AUR hackagedb packages have just been a massive pain for years (I was basically forced to do manual package dependency management), and as my haskell colleges/friends have (rightly) stated, just use cabal. This of course isn't really an issue if you don't need to install haskell binaries through AUR, of which that many currently don't really exist (or just the occasional one like xmonad), and using cabal just *works*, even if it feels dirty ;) Of course this suggestion may have issues with shared libraries, which will probably need to be managed in their own way Thanks

My understanding of current arch-haskell is to support the largest subset of
packages that can be installed together. This may mean some dependencies
are not always at the latest versions.
Haskell Platform should be directly installable from the [extra] Arch Linux
repository. Many others should be installable from AUR, from HABS[1], or
more conveniently from its binary repo. I'll add yesod back in there in a
couple days, unless someone else gets to it first.
If there's issues with existing packages or packages you'd like to see
added, please let us know.
Arch Haskell library packages are being hard-coded to specific versions of
their dependencies these days. This, along with bumping the PKGREL of
packages when one of their decencies gets changed, should cause an update to
be installed by your package management tool (i.e. yaourt).
Packages that are only an executable and not a library should just have
their decencies listed as makedepends, not as run-time depends (being that
they're statically linked at present). The set of these that don't build
against the current set of package versions in HABS should hopefully be
rather small. If you're using Arch packages to manage your cabal packages
(instead of using cabal by itself), and want to use a package like this that
won't build against the versions in HABS, there might be an older binary
package built in HABS you could install.
I've been using yaourt to manage all my Haskell packages for a while now.
Having to re-install all my Haskell packages when upgrading GHC is
annoying, but I'm used to it. It sometimes requires more messing-with than
cabal-install, and doesn't support multiple versions of a package being
installed at once, but the binary repo of pre-compiled packages is quite
convenient, and having multiple package versions installed can present its
own difficulties...
-Leif Warner
[1] http://github.com/archhaskell/habs
On Mon, Mar 28, 2011 at 2:27 AM, Mathew de Detrich
I guess I am creating this thread due to the frustrations of using cabal2arch/AUR for haskell packages (instead of using the standard cabal install). I previously had issues, mainly with having to manually update AUR installed cabal packages due to them not updating (which was recently fixed by forcing the update through ghc)
This however seemed didn't seem to solve the issue completely, although it will update fine through a yaourt -Syu --aur (or clyde -Syu --aur or whatever package helper you use), it still ends up breaking GHC because all the hashes for the packages change and it was a forced update
I unfortunately don't have the output message, since I was in a rush to get things working (due to my job), I just ended up doing a clyde -R --cascade ghc and just installing my package through cabal). The package in question was basically yesod (which pushes around 20 or 30 packages on a fresh install) and the updates had to do with some components of yesod being updated
In fact I think that yesod is a perfect example in showing the issues when trying to integrate archlinux package with cabal package management, since there are a lot of packages in yesod (like authenticate/cookie) which cannot use the latest version of the package on AUR, where as there are upgrades with the other packages.
As I mentioned earlier, I just ended up giving up. Right now I only have ghc installed (through official arch repository) and any package I need to install, I do so with cabal. The problems with this are obvious however, but right now its not an issue. The main issue being installing haskell binaries that are on AUR which has the issue of duplication if that binary depends on a library that is already installed through cabal install.
In my opinion the main issue seems to be that the granularity of cabal package system is a lot finer then Arch's (cabal was purpously designed for this, and its great for haskell packages), and unlike with other languages, the situation where you have a single package that depends on a lot of both updated and outdated packages is very common, along with GHC's runtime system updates forcing recompilation of libraries every haskell-platform release. This isn't so much of an issue now regarding binaries (because to be honest, we don't have that many heavily used haskell based binaries apart from xmonad), but in the future should haskell "break through" and become a much more widely used language, this can create a pain, especially for haskell-binaries (and its already painful with haskell-libraries)
Of course, trying to force archlinux package management to have the same granularity as arch's is out of the question, which I guess made me come with my second idea. Wouldn't it just be smarter to have the archlinux haskell environment set up like this. Archlinux's package management is one that makes some sacrifices (in regards to having a rolling release system thats always up to date at the cost of granularity in certain areas)
- ghc is an official package in the official repository - haskell-platform will be the same as it is currently (uses GHC, and cabal2arch packages from both the official repository and AUR). This is fine since the point of haskell-platform is the standard packages it uses are frozen for half a year. So the only haskell libraries on AUR would be the ones in haskell-platform (and arguably these should be in the official repository, as well as the base haskell-platform package) - All other libraries users would just install/update with cabal install and not through AUR/cabal2arch/bauerbill etc etc. - Haskell packages that provide binaries would check for their dependencies using cabal/ghc-pkg, and if a haskell binary can't find a package it can install it in a temp directory/and or install it automatically through cabal install (depends if its an updated/outdated/etc etc). If the library will be built to a temp section, then it will just be linked to GHC manually when building that binary (the PKGBUILD will cover this). This also allows building of packages that use outdated packages (such as leksah as an example, its not possible to build this package if you have newest haskell-gtk install, you have to manually do dependency tracking). Just like with cabal2arch PKGBUILD generator, someone can easily create a program which will create PKGBUILDS out of hackagedb binaries, the actual PKBUILD will search for dependancies through cabal (and not the PKGBUILD "depends"). Binaries that statically link libraries (which is the case for almost all haskell binaries at this point) do not actually require the libraries they depend on to build to be installed after the executable has been built, so doing temp library installs to build a binary will not be an issue (C/C++ binaries already do this in circumstances, and its something that can easily be automated through hackagedb)
I am just saying this, because my experience of using cabal2arch/AUR hackagedb packages have just been a massive pain for years (I was basically forced to do manual package dependency management), and as my haskell colleges/friends have (rightly) stated, just use cabal. This of course isn't really an issue if you don't need to install haskell binaries through AUR, of which that many currently don't really exist (or just the occasional one like xmonad), and using cabal just *works*, even if it feels dirty ;)
Of course this suggestion may have issues with shared libraries, which will probably need to be managed in their own way
Thanks
_______________________________________________ arch-haskell mailing list arch-haskell@haskell.org http://www.haskell.org/mailman/listinfo/arch-haskell

2011/3/28 Mathew de Detrich
Of course, trying to force archlinux package management to have the same granularity as arch's is out of the question, which I guess made me come with my second idea. Wouldn't it just be smarter to have the archlinux haskell environment set up like this. Archlinux's package management is one that makes some sacrifices (in regards to having a rolling release system thats always up to date at the cost of granularity in certain areas) - ghc is an official package in the official repository - haskell-platform will be the same as it is currently (uses GHC, and cabal2arch packages from both the official repository and AUR). This is fine since the point of haskell-platform is the standard packages it uses are frozen for half a year. So the only haskell libraries on AUR would be the ones in haskell-platform (and arguably these should be in the official repository, as well as the base haskell-platform package) - All other libraries users would just install/update with cabal install and not through AUR/cabal2arch/bauerbill etc etc. - Haskell packages that provide binaries would check for their dependencies using cabal/ghc-pkg, ...
I don't really understand what you mean. Nothing prevents you from using cabal to install packages. The point of the binary repositories is to speed up the task by providing a set of pre-compiled packages for common libraries. We do not intend to replace cabal as a package management system, since obviously Archlinux package system is not strong enough to express the full meaning the cabal dependencies. My piont of view on this is - archlinux provides GHC, haskell-platform (minus the OpenGL packages), darcs and xmonad, + dependencies of common programs - arch-haskell provides a suitably chosen version of several additional libraries as binary packages or PKGBUILDs on AUR, and having more libraries can't be a bad thing - other packages (older versions of packages, etc.), that are problematic for package management are really meant to be installed through cabal. And this is basically what you describe. And actually people can use any tool they prefer to manage packages. If an older version of a package (e.g. parsec 2) happens to be very common and very needed, it is perfectly possible to provide a binary package or a PKGBUILD for it with the correct "provides" fields so that it can be used in a transparent manner by PKGBUILDs that need it. All we have to do is to find a correct balance between no binary package and too many binary packages. The current situation looks good. Rémy.

Hi Mathew, thank you for sharing your thoughts and insights.
I guess I am creating this thread due to the frustrations of using cabal2arch/AUR for haskell packages (instead of using the standard cabal install).
Your frustration is quite understandable. A number of PKGBUILD files published on AUR don't actually work. Others do work, but they're significantly out-of-date, which effectively prevents other packages from being installed. Haskell support on ArchLinux is far from perfect.
I previously had issues, mainly with having to manually update AUR installed cabal packages due to them not updating (which was recently fixed by forcing the update through ghc)
We have recently picked up the habit of bumping the $pkgrel field of every package that needs to be re-built because of an update of one of its dependencies, so this particular problem should be a thing of the past. The repository http://github.com/archhaskell/habs/ contains a set of packages that can be expected to be compile fine. Those packages are also fairly up-to-date, and we publish those files on AUR, too. Haskell packages outside of HABS, however, aren't maintained much, and they're probably not reliable. Is there any specific package currently not included in HABS that you would like to see supported?
[The] main issue seems to be that the granularity of cabal package system is a lot finer then Arch's.
Yes, exactly. In Cabal, a package can be installed in any number of different versions, and these just co-exist. Pacman, however, wasn't designed to do this. It does distinguish $pkgname and $pkgver, but only one version of $pkgname can be installed at a time. When we maintain packages in ArchLinux, we perform two tasks: 1) We convert Cabal to PKGBUILD, and we 2) figure out a conflict-free set of versions that can be installed at the same time. Unfortunately, part (2) is way more difficult than (1). :-)
Wouldn't it just be smarter to have the archlinux haskell environment set up like this.
- ghc is an official package in the official repository
- haskell-platform will be the same as it is currently [...].
- All other libraries users would just install/update with cabal install and not through AUR/cabal2arch/bauerbill etc etc.
Yes, I agree, and I think what we're actually doing is pretty close to your description. The complete Haskell Platform is part of [extra], and everything else can be installed through Cabal. Popular applications written in Haskell, like darcs, are also available from [extra], and these packages usually don't depend on GHC or any other Haskell package. On top of this, there is a community-driven binary repository accessible by adding the lines [haskell] Server = http://andromeda.kiwilight.com/$repo/$arch to /etc/pacman.conf, which adds another ~200 popular Haskell packages.
- Haskell packages that provide binaries [should build their dependencies on-the-fly via Cabal]. This allows building of packages that use outdated packages (such as leksah ...)
That is a great idea. As far as I know, no-one has ever tried doing that before. It would be interesting to know whether this approach works well in practice. One source of trouble could be that all dependencies have to be $makedepends, but some libraries cannot be $makedepends, because they need auxiliary data files installed at run-time. Problems such as this can be worked-around, no doubt, but it's likely to be some effort. Another drawback is that this scheme may perform redundant compilations, because intermediate packages aren't re-used in any way.
[My] experience of using cabal2arch/AUR hackagedb packages have just been a massive pain for years (I was basically forced to do manual package dependency management), and as my haskell colleges/friends have (rightly) stated, just use cabal.
Yes, indeed, that is good advice. The installation can be managed with cabal-install just fine. For some users, Cabal is probably the superior choice, especially for those who have a deep understanding of the problems involved in dependency resolution, package management, etc. For other users, Cabal is not an option because their needs are different. For instance, large-scale binary package distribution cannot be done with Cabal, but Pacman can do it just fine. ;-) Take care, Peter

On Wed, Mar 30, 2011 at 6:24 AM, Peter Simons
The following message is a courtesy copy of an article that has been posted to gmane.comp.lang.haskell.arch-linux as well.
Hi Mathew,
thank you for sharing your thoughts and insights.
I guess I am creating this thread due to the frustrations of using cabal2arch/AUR for haskell packages (instead of using the standard cabal install).
Your frustration is quite understandable. A number of PKGBUILD files published on AUR don't actually work. Others do work, but they're significantly out-of-date, which effectively prevents other packages from being installed. Haskell support on ArchLinux is far from perfect.
I previously had issues, mainly with having to manually update AUR installed cabal packages due to them not updating (which was recently fixed by forcing the update through ghc)
We have recently picked up the habit of bumping the $pkgrel field of every package that needs to be re-built because of an update of one of its dependencies, so this particular problem should be a thing of the past. The repository http://github.com/archhaskell/habs/ contains a set of packages that can be expected to be compile fine. Those packages are also fairly up-to-date, and we publish those files on AUR, too.
Haskell packages outside of HABS, however, aren't maintained much, and they're probably not reliable. Is there any specific package currently not included in HABS that you would like to see supported?
My main issue was basically with packages such as yesod, which rely on a lot of other packages (and by a lot, I mean a lot, I think a fresh install does around 30 package installations) and because yesod has so many packages you have some that are updated and others that are out of date (which describes the problem I was describing earlier)
[The] main issue seems to be that the granularity of cabal package system is a lot finer then Arch's.
Yes, exactly. In Cabal, a package can be installed in any number of different versions, and these just co-exist. Pacman, however, wasn't designed to do this. It does distinguish $pkgname and $pkgver, but only one version of $pkgname can be installed at a time. When we maintain packages in ArchLinux, we perform two tasks:
1) We convert Cabal to PKGBUILD, and we
2) figure out a conflict-free set of versions that can be installed at the same time.
Unfortunately, part (2) is way more difficult than (1). :-)
Wouldn't it just be smarter to have the archlinux haskell environment set up like this.
- ghc is an official package in the official repository
- haskell-platform will be the same as it is currently [...].
- All other libraries users would just install/update with cabal install and not through AUR/cabal2arch/bauerbill etc etc.
Yes, I agree, and I think what we're actually doing is pretty close to your description. The complete Haskell Platform is part of [extra], and everything else can be installed through Cabal. Popular applications written in Haskell, like darcs, are also available from [extra], and these packages usually don't depend on GHC or any other Haskell package. On top of this, there is a community-driven binary repository accessible by adding the lines
[haskell] Server = http://andromeda.kiwilight.com/$repo/$arch
to /etc/pacman.conf, which adds another ~200 popular Haskell packages.
- Haskell packages that provide binaries [should build their dependencies on-the-fly via Cabal]. This allows building of packages that use outdated packages (such as leksah ...)
That is a great idea. As far as I know, no-one has ever tried doing that before. It would be interesting to know whether this approach works well in practice. One source of trouble could be that all dependencies have to be $makedepends, but some libraries cannot be $makedepends, because they need auxiliary data files installed at run-time. Problems such as this can be worked-around, no doubt, but it's likely to be some effort.
I will likely make an attempt in a couple of weeks when I have holidays (it would be a tool like cabal2arch that would convert haskell packages that create binaries into PKGBUILDS). There were other issues with the past, but I think you get the general idea of whats happening ;) Also is there any reason the libraries have to be $makedepends instead of just having the dependencies tracked by cabal via the PKGBUILD?
Another drawback is that this scheme may perform redundant compilations, because intermediate packages aren't re-used in any way.
Yes, I already had some sought of an idea as a solution. If PKGBUILD finds
a dependency that it can install through cabal without conflicts, then it would do a proper cabal installation of that package (so future installations would just use that install). This however would cause issues regarding fakeroot, by default cabal install packages in the users home directory and if we do this we don't want another cabal installation of libraries in root (we could alternately make cabal install libraries by default in /usr/lib or something like that, but this creates many other issues) The other problem with this method is that it probably breaks a 'rule' in package management, that is, its making changes to your system outside of installing the actual package (which are only required to build the package), which in terms of general package management, and are not tracked by arch's package manager The real beauty of this method though, is there is nothing stopping from PKGBUILD detecting library installs from AUR, as well as library installs from cabal install
[My] experience of using cabal2arch/AUR hackagedb packages have just been a massive pain for years (I was basically forced to do manual package dependency management), and as my haskell colleges/friends have (rightly) stated, just use cabal.
Yes, indeed, that is good advice. The installation can be managed with cabal-install just fine. For some users, Cabal is probably the superior choice, especially for those who have a deep understanding of the problems involved in dependency resolution, package management, etc.
For other users, Cabal is not an option because their needs are different. For instance, large-scale binary package distribution cannot be done with Cabal, but Pacman can do it just fine. ;-)
I think we also need to take into account that haskell's programming 'environment' is different from other languages. With GHC, its a statically compiled language which means that libraries are compiled into object files. Unlike C/C++ however, GHC's runtime basically changes every time a new haskell-platform is released, where as with C/C++ relies on glibc, which changes once every few years. Old C/C++ libraries will generally with with gcc for years without needing recompilation Also haskell programs/libraries tend to follow the unix philosophy, (do one thing and do it well), which means that haskell packages tend to have a massive number of small dependencies instead of a few large ones (that you typically find in C/C++, boost as an example). This isn't a bad thing about Haskell (in fact its a great redeeming feature), it just means we have to deal with a language that does heavy static compilation/recompilation and typically has very large dependency trees on packages. Dealing with dependency issues is a lot less trivial if you have a trivial amount of dependencies in the first place If myself (or someone else) ends up making a tool that for converting haskelldb packages that provide binaries into AUR PKGBUILDS that uses cabal for dependency tracking, we could get into a situation where it creates a conflicting impression on new archlinux users onto how exactly to set up their haskell environment in arch. Would there be any issue in making an 'official' arch-haskell tool similar to baeurbill that would just automatically create PKGBUILDS out of cabal2arch for the users that wish to install haskell libraries through AUR and that all current AUR packages that are not part of the haskell-platform would be removed (and actually moved into the official repository)?. Then haskell packages that provide binaries (aka programs) would be able to track dependencies regardless if they are installed via the current method (of cabal2arch PKGBUILDS) or by cabal install Im just thinking about future scenarios here
Take care, Peter

Hi Mathew,
Is there any specific package currently not included in HABS that you would like to see supported?
My main issue was basically with packages such as yesod [...].
indeed, yesod currently doesn't compile. If you happen to have any fixes or patches you could share to remedy that situation, then it would be very much appreciated. Take care, Peter
participants (4)
-
Leif Warner
-
Mathew de Detrich
-
Peter Simons
-
Rémy Oudompheng