Re: [arch-haskell] What to do now?

On 09/10/10 23:17, Rémy Oudompheng wrote:
Magnus Therning
wrote: On 09/10/10 09:43, Rémy Oudompheng wrote: [...]
- a darcs repo for the PKGBUILDs
1. Are you suggesting we keep binary versions of *all* of hackage in a repo, or 2. we keep PKGBUILDs for all of hackage in an ABS tree, and only provide binaries for a subset of packages?
I suggest we keep PKGBUILDs for as many packages as we want, and provide binary packages for either each of them, or just a subset, for example the set of packages currently on AUR, and add packages to that list on demand.
The set of packages currently on AUR is *huge*, I think Don recently mentioned something in the order of 2000 haskell packages on AUR, and that is about 10% of AUR. I would suggest starting somewhat smaller than that :-) Maybe starting from Haskell Platform and growing on demand from that?
* one dir per package, and subdirs $package/trunk, $package/repo (holding the current WIP version of the PKGBUILD and one corresponding to the binary package in repo) * people are supposed to do only a partial checkout of the darcs repo, I know Git can do that, but that said, a full working copy is only a few thousand files. Is darcs as efficient as Git for storage ? I expect the transfer size for a full cloning to be less than 5MB.
It's worth clarifying here that while git does support partial checkout it doesn't support partial cloning. darcs supports lazy cloning, and I think git does as well.
Just to get some numbers I downloaded the cabal files for for the latest version of all packages on Hackage. Then I ran cabal2arch on it all. After that I attempted to put the results in darcs and git.
Adding all files, 100 at a time:
* darcs: 522.18s user 5.49s system 99% cpu 8:48.96 total * git: 1.90s user 0.77s system 97% cpu 2.726 total
Record/commit of initial changeset:
* darcs: NA, it seems frozen with: 5548 done, 5480 queued * git: 0.33s user 0.32s system 57% cpu 1.128 total
There are a total of 4528 files in the (git) repo and 'du -sh' says that it takes 57M.
Thank you. What is the size of the git repo itself (I mean the .git subdir) ? It is probably only a few megabytes, so it can probably be hosted anywhere.
The size of the .git dir is: % du -sh .git 30M .git For comparison I changed strategy for darcs, instead I recorded each package in its own changeset: Total time to add and record: 7638.14s user 1108.26s system 99% cpu 2:26:04.82 total Size of the whole work area: 436M Size of _darcs: 409M /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus@therning.org Jabber: magnus@therning.org http://therning.org/magnus identi.ca|twitter: magthe

On Sun, 10 Oct 2010 17:55:47 +0100, Magnus Therning
On 09/10/10 23:17, Rémy Oudompheng wrote:
Magnus Therning
wrote: On 09/10/10 09:43, Rémy Oudompheng wrote: [...]
- a darcs repo for the PKGBUILDs
1. Are you suggesting we keep binary versions of *all* of hackage in a repo, or 2. we keep PKGBUILDs for all of hackage in an ABS tree, and only provide binaries for a subset of packages?
I suggest we keep PKGBUILDs for as many packages as we want, and provide binary packages for either each of them, or just a subset, for example the set of packages currently on AUR, and add packages to that list on demand.
The set of packages currently on AUR is *huge*, I think Don recently mentioned something in the order of 2000 haskell packages on AUR, and that is about 10% of AUR. I would suggest starting somewhat smaller than that :-)
Maybe starting from Haskell Platform and growing on demand from that?
Sure we should take care that the Haskell Platform works nicely. Basically the hard ones are those depending on external libraries, we have to manually take care of them. Then packages only made of pure Haskell code on top of that should be automatically built. Maybe one can start with recent uploads to Hackage and transitively build/package their dependencies. [...]
The size of the .git dir is:
% du -sh .git 30M .git
For comparison I changed strategy for darcs, instead I recorded each package in its own changeset:
Total time to add and record: 7638.14s user 1108.26s system 99% cpu 2:26:04.82 total Size of the whole work area: 436M Size of _darcs: 409M
Let's forget darcs for this kind of task. -- Nicolas Pouillard http://nicolaspouillard.fr

On 2010/10/11, Nicolas Pouillard
On Sun, 10 Oct 2010 17:55:47 +0100, Magnus Therning
wrote: The set of packages currently on AUR is *huge*, I think Don recently mentioned something in the order of 2000 haskell packages on AUR, and that is about 10% of AUR. I would suggest starting somewhat smaller than that :-)
Maybe starting from Haskell Platform and growing on demand from that?
Sure we should take care that the Haskell Platform works nicely. Basically the hard ones are those depending on external libraries, we have to manually take care of them. Then packages only made of pure Haskell code on top of that should be automatically built. Maybe one can start with recent uploads to Hackage and transitively build/package their dependencies.
cabal2arch is supposed to handle that automatically. Problems are either an upstream problem or a bug in cabal2arch :) For example, cabal2arch gives me a haskell-pango PKGBUILD depending on pangocairo, which is wrong, for some unknown reason because it should be translated to pango. Another example is haskell-openal which does not have the obvious dependency on openal, but that dependency is not stated anywhere in the cabal file. My automated build is still running happily, with 633 packages built. There is still a dependency nightmare due to packages needing old versions to run. -- Rémy.

On Mon, 11 Oct 2010 11:19:25 +0200, Rémy Oudompheng
On 2010/10/11, Nicolas Pouillard
wrote: On Sun, 10 Oct 2010 17:55:47 +0100, Magnus Therning
wrote: The set of packages currently on AUR is *huge*, I think Don recently mentioned something in the order of 2000 haskell packages on AUR, and that is about 10% of AUR. I would suggest starting somewhat smaller than that :-)
Maybe starting from Haskell Platform and growing on demand from that?
Sure we should take care that the Haskell Platform works nicely. Basically the hard ones are those depending on external libraries, we have to manually take care of them. Then packages only made of pure Haskell code on top of that should be automatically built. Maybe one can start with recent uploads to Hackage and transitively build/package their dependencies.
cabal2arch is supposed to handle that automatically. Problems are either an upstream problem or a bug in cabal2arch :) For example, cabal2arch gives me a haskell-pango PKGBUILD depending on pangocairo, which is wrong, for some unknown reason because it should be translated to pango. Another example is haskell-openal which does not have the obvious dependency on openal, but that dependency is not stated anywhere in the cabal file.
Right that's in cabal2arch (which holds a good part of the packaging work). What I meant is that if I upload a new package today which depends on some external library then we cannot expect it to be packaged automatically. In this case it would be nice if there was a page (and/or a notification) for such packages, such that one of the team member can say that he takes care of it, which often amount to pushing a cabal2arch patch.
My automated build is still running happily, with 633 packages built. There is still a dependency nightmare due to packages needing old versions to run.
Yes this is another nigthmare, some packages either don't give an upper bound to their dependency so they should fix their code or there dependencies. Some others clearly state an upper bound and so I see three options: * Do not package it * If the pattern is frequent enough (QuickCheck, parsec, base...), one can provide sevral versions of the same package. * Always package the different versions needed. Option 2 seems fine for now and it will push people to upgrade if they want to be packaged. Regards, -- Nicolas Pouillard http://nicolaspouillard.fr

remyoudompheng:
On 2010/10/11, Nicolas Pouillard
wrote: On Sun, 10 Oct 2010 17:55:47 +0100, Magnus Therning
wrote: The set of packages currently on AUR is *huge*, I think Don recently mentioned something in the order of 2000 haskell packages on AUR, and that is about 10% of AUR. I would suggest starting somewhat smaller than that :-)
Maybe starting from Haskell Platform and growing on demand from that?
Sure we should take care that the Haskell Platform works nicely. Basically the hard ones are those depending on external libraries, we have to manually take care of them. Then packages only made of pure Haskell code on top of that should be automatically built. Maybe one can start with recent uploads to Hackage and transitively build/package their dependencies.
cabal2arch is supposed to handle that automatically. Problems are either an upstream problem or a bug in cabal2arch :) For example, cabal2arch gives me a haskell-pango PKGBUILD depending on pangocairo, which is wrong, for some unknown reason because it should be translated to pango. Another example is haskell-openal which does not have the obvious dependency on openal, but that dependency is not stated anywhere in the cabal file.
There's a table (wired in) to cabal2arch, that gives the necessary translations between C libraries that appear in Cabal files (e.g. "pangocairo") and the actual C library name on Arch (e.g. "pango"). This list should be an external, shared file (its not currently). The main job of the cabal2arch maintainer is to keep adding translations to this list. http://haskell.org/haskellwiki/Arch_Linux/Arch_Haskell_Group_Maintainership#...
participants (4)
-
Don Stewart
-
Magnus Therning
-
Nicolas Pouillard
-
Rémy Oudompheng