This is wondeful work Xyne!! You're really making things work. I'll follow up with detailed comments on the cabal2arch changes shortly. Well done, again. -- Don xyne:

Hi,

I've added preliminary support for Hackage to Bauerbill. The following now work:

"bauerbill -S --hackage [pkgs]" : build [pkgs] directly from Hackage* "bauerbill -Si --hackage [pkgs]" : look up information about [pkgs] "bauerbill -Ss --hackage [args]" : search for [args] "bauerbill -Qu --hackage" : list upgradable Hackage packages

* with dependency resolution and build dependency tracking and removal

I've only run a few simple tests so far and I haven't announced this on my site or on the forum yet. I'm waiting because I still have some issues with cabal2arch which I hope can be solved before I push ahead (see below, although I suppose what follows mostly concerns Don).

Btw, for now at least, if you enable AUR support as well (--aur) then AUR packages take precedence over Hackage packages. The reasoning is that AUR packages are more likely to be consistent as they're managed by arch-haskell and thus have some human oversight. This may change later if it's deemed move useful to give precedence to Hackage when both are enabled.

Feedback and bug reports are always welcome.

Don,

I've updated to the latest version of cabal2arch and I've noticed that most of the clutter is gone. I appreciate the changes and consider them to be a great improvement, but it still creates an archive of the build directory which I neither need nor want.

I would like to make the following suggestions:

*) add an option to create the archive of the build directory and suppress it by default so that cabal2arch only create the build dir with the PKGBUILD and install file when run with no options

*) add an option to create the PKGBUILD and install file in the current working directory, and create nothing else

*) enable batching of multiple cabal files, so that it can accept multiple URLs and paths on the command line (including paths to directories which contain multiple cabal files)

I've also noticed that it downloads source files from Hackage. Is that just to get the checksums for the archive? If so, it seems rather wasteful. Would it not be possible to provide that information in the cabal file or via a web API on Hackage?

If it is not possible to create the PKGBUILD without the source archive, please add an option to first check if the source archive exists in the build directory and mirror it if it doesn't. This would prevent redundant downloads when building packages (which saves time and bandwidth for all involved). Bauerbill in particular would benefit from this by being able to handle the downloads itself (e.g. in parallel). It could also preserve them between build sessions and mirror them to only update when necessary.

Apropos the availability of information from Hackage, the Hackage backend of Bauerbill currently uses the log and pkg-list.html (in /packages/archives/) to create a plain-text database of all packages on Hackage. You can view the "database" file created by bauerbill (/tmp/bauerbill/hackage/hackage_db by default). This file is smaller than both the log and pkg-list.html and follows the line format <name> <version> <category> <type>: <synopsis>

Would it be possible to generate this file directly on Hackage? Again, this would save bandwidth and it would also enable control over the versions built by Bauerbill. Currently it tracks the latest versions by simple parsing of the log. If the database file were instead provided then it would use whichever versions it contained. This would make it possible to specify which versions should be built. I don't know if this would be an issue but version compatibility does seem to come up quite often.

Of course even if it isn't an issue, it would still be beneficial to provide such a plain-text file which applications such as Bauerbill could use.

One final niggle: cabal2arch doesn't understand relative paths to parent directories, e.g. "../cabal/foo.cabal".

Sorry for the long message with several suggestions/requests. I hope they're considered reasonable and not too difficult to implement. I would particularly appreciate the option to use local source files to avoid redundant downloads and the option to suppress creation of the build dir archive. Many thanks if you can do this.

Regards, Xyne

p.s. Would it be useful to be able to rebuild all Hackage packages with a simple command? If cabal2arch placed Hackage packages in a group, I could easily add a "--rebuild-group" option to Bauerbill. I'm asking because I don't actually know if this would be useful

xyne:

Hi,

I've added preliminary support for Hackage to Bauerbill. The following now work:

Btw, for now at least, if you enable AUR support as well (--aur) then AUR packages take precedence over Hackage packages. The reasoning is that AUR packages are more likely to be consistent as they're managed by arch-haskell and thus have some human oversight. This may change later if it's deemed move useful to give precedence to Hackage when both are enabled.

I think it is reasonable to prefer, in order, [extra] [community] AUR Hackage in particular, AUR is likely to have any special cases handled that cabal2arch couldn't handle (which sometimes happens, though rarely).

I've updated to the latest version of cabal2arch and I've noticed that most of the clutter is gone. I appreciate the changes and consider them to be a great improvement, but it still creates an archive of the build directory which I neither need nor want.

I would like to make the following suggestions:

*) add an option to create the archive of the build directory and suppress it by default so that cabal2arch only create the build dir with the PKGBUILD and install file when run with no options

*) add an option to create the PKGBUILD and install file in the current working directory, and create nothing else

*) enable batching of multiple cabal files, so that it can accept multiple URLs and paths on the command line (including paths to directories which contain multiple cabal files)

These all look reasonable. I've added them to my todo.

I've also noticed that it downloads source files from Hackage. Is that just to get the checksums for the archive? If so, it seems rather wasteful. Would it not be possible to provide that information in the cabal file or via a web API on Hackage?

It is indeed only for the md5. We could store it on Hackage. I've filed a ticket for Hackage to do this. http://hackage.haskell.org/trac/hackage/ticket/635 We could also just compute our own list of hashes once and store them (I could add support for such a thing to cabal2arch I suppose).

If it is not possible to create the PKGBUILD without the source archive, please add an option to first check if the source archive exists in the build directory and mirror it if it doesn't. This would

That's a good idea. Currently cabal2arch checks for a .cabal file in the current directory, not the .tar.gz that it would download.

prevent redundant downloads when building packages (which saves time and bandwidth for all involved). Bauerbill in particular would benefit from this by being able to handle the downloads itself (e.g. in parallel). It could also preserve them between build sessions and mirror them to only update when necessary.

Apropos the availability of information from Hackage, the Hackage backend of Bauerbill currently uses the log and pkg-list.html (in /packages/archives/) to create a plain-text database of all packages on Hackage. You can view the "database" file created by bauerbill (/tmp/bauerbill/hackage/hackage_db by default). This file is smaller than both the log and pkg-list.html and follows the line format <name> <version> <category> <type>: <synopsis>

Would it be possible to generate this file directly on Hackage? Again, this would save bandwidth and it would also enable control over the versions built by Bauerbill. Currently it tracks the latest versions by simple parsing of the log. If the database file were instead provided then it would use whichever versions it contained. This would make it possible to specify which versions should be built. I don't know if this would be an issue but version compatibility does seem to come up quite often.

There's a much easier way -- since the database is already mirroed locally by cabal. The cabal-install tool holds this database, and can list it for you: $ cabal list And we have a programmatic binding to it in the archlinux package: http://hackage.haskell.org/packages/archive/archlinux/0.3/doc/html/Distribut... which returns a table of all packages and their latest versions.

One final niggle: cabal2arch doesn't understand relative paths to parent directories, e.g. "../cabal/foo.cabal".

Added.

p.s. Would it be useful to be able to rebuild all Hackage packages with a simple command? If cabal2arch placed Hackage packages in a group, I could easily add a "--rebuild-group" option to Bauerbill. I'm asking because I don't actually know if this would be useful

Mmmm. Currently I just loop over all the packages (computed as follows, based on the AUR maintainer): import Distribution.ArchLinux.AUR import Control.Monad me = "arch-haskell" main = do packages <- maintainer me forM_ packages $ putStrLn . packageName Hackage itself can be built with the cabal plan, assuming you have a coherent set of packages. We use this for regression testing against GHC. Thanks for all the feature requests. I've taken action on some, and added todos for others. -- Don