Starting to help
Hi all, yesterday I submitted a patch to HABS repository just because I met some out-of-date packages in AUR. Then I read the "story" of this group in this mailing list and thought that I can give some help. I think I can take responsibility for a bunch of packages and care for updates, but I need some hints. 1) In order to test for broken packages, do I really need to use the makeworld script? I think it would take too much time! I would prefer to simply compile what I use, also to find if makepkg can resolve dependencies. Launching makeworld in another directory with only my packages would do the job? 2) Past discussions suggest me that the group is centred on a small number of packages to put in a binary repository. It's wonderful to have this, but it takes a very long time. What do you think if some people concentrate in the simplest and fastest task of preparing PKGBUILDs and put them in a place (i.e. AUR) where the core group decide witch package worth an inclusion in the repo? Haskell repository would actually be an expanded and specialised version of Community repo. Anyway I'd prefer the mainter to be the group, not a single. That's only my opinion, of course. 3) For Peter: could you please give some help information on how to use the scripts in https://github.com/peti/arch-haskell/tree/master/scripts ? 4) Why haskell packages in extra depends on ghc-7.0.3 (still in testing)? Tha's all for now. Sorry for my bad English! Cheers Fabio
Hi Fabio,
yesterday I submitted a patch to HABS repository just because I met some out-of-date packages in AUR. Then I read the "story" of this group in this mailing list and thought that I can give some help. I think I can take responsibility for a bunch of packages and care for updates, but I need some hints.
your pull request has been merged. Thank you very much for the contribution!
1) In order to test for broken packages, do I really need to use the makeworld script? I think it would take too much time!
Unfortunately, distributing Haskell packages is a fairly complicated affair, mostly because these packages interact with each other in complex ways. Some problems can be caught without running a test build, but not all of them. My experience is that running 'makeworld' is the only way to be sure that the repository is in a consistent state. The procedure looks intimidating at first, but it's not that complicated, actually. Initially, the script runs for quite a while, because it needs to set-up the chroot build environment, but once that's been accomplished, builds are fairly quick. Note that the script does not build everything from scratch -- rather, it builds only those packages that need to be re-built. So, if at all possible, I'd encourage you to use 'makeworld'. If you don't want to (or can't, for some reason), then it's fine too, though, because others do run it and will catch problems eventually.
2) Past discussions suggest me that the group is centred on a small number of packages to put in a binary repository. It's wonderful to have this, but it takes a very long time. What do you think if some people concentrate in the simplest and fastest task of preparing PKGBUILDs and put them in a place (i.e. AUR) where the core group decide witch package worth an inclusion in the repo?
It is possible to update packages on AUR without building them first. So, yes, we could split this effort into one that updates AUR and one that maintains the binary repository. Past experience suggests, though, that packages that are being published without being built first tend to be very unreliable. This means that users will run into build problems, and then they'll complain on AUR or on this list -- and rightfully so! Dealing with those issues, however, is a lot of effort. I'm not sure whether publishing untested PKGBUILDs is going to be very helpful for the general user base.
Anyway I'd prefer the mainter to be the group, not a single. That's only my opinion, of course.
Yes, I very much agree. This is a community-driven effort that is intended to allow everyone to contribute and to have an immediate impact on the result.
3) For Peter: could you please give some help information on how to use the scripts in https://github.com/peti/arch-haskell/tree/master/scripts ?
I'll try to describe my work-flow briefly: 1) Clone the 'peti/arch-haskell' repository. 2) Run "git submodules init" to set-up the HABS tree. 3) Run "cabal update" to download the latest database from hackage. 4) Run "make" to re-build all files in HABS based on the configuration that's described in the PKGLIST file. The format of that file is: | haskell-aes AES 0.2.8 1 ^^^^^^^^^^^ ^^^ ^^^^^ ^ $pkgname $_hkgname $pkgver $pkgrel 5) Run "git diff" in the habs/ sub-directory to verify the changes. Then commit those changes in habs/. 6) Run "make world" to build the latest version of the binary repository. 7) If all went well, push the updated version of habs/ to Github. Furthermore, you can run ... * "make updates" to get a list of all updates available on Hackage. Edit PKGLIST to pull in those updates that you want, then run "make all" to verify that all version constraints can be fulfilled. This is usually an iterative process, in which one package after another is updated until all updates that can be performed have been found. * If habs/ has been changed by another committer, you can run "scripts/extract-pkglist habs/habs/* >PKGLIST" to synchronize the PKGLIST file with the current state of the repository. * If you have updated packages "foo" and "bar" in habs, then run: make scripts/reverse-dependencies cd habs/habs ../../scripts/reverse-dependencies . foo bar This will show you the list of packages that need to be re-built after that update. If that looks okay, run ../../scripts/bump $(../../scripts/reverse-dependencies . foo bar) to bump the $pkgrel of all those packages. Commit those changes in habs/. Then run scripts/extract-pkglist habs/habs/* >PKGLIST to synchronize the PKGLIST file, and do the usual "make all && make world" routine to verify that everything is okay before pushing the changes to Github. It's a bit of a mess, but it works okay. :-)
4) Why haskell packages in extra depends on ghc-7.0.3 (still in testing)?
There is an update of GHC available in [testing], and unfortunately some packages have been published that were built with that newer GHC version before GHC itself is available. In short, Haskell support is currently broken in ArchLinux. Unfortunately, these things happen, but it's most probably going to be remedied soon. Take care, Peter
On Sun, Jun 26, 2011 at 8:39 AM, Peter Simons
I'll try to describe my work-flow briefly:
1) Clone the 'peti/arch-haskell' repository.
2) Run "git submodules init" to set-up the HABS tree.
3) Run "cabal update" to download the latest database from hackage.
4) Run "make" to re-build all files in HABS based on the configuration that's described in the PKGLIST file. The format of that file is:
| haskell-aes AES 0.2.8 1 ^^^^^^^^^^^ ^^^ ^^^^^ ^ $pkgname $_hkgname $pkgver $pkgrel
5) Run "git diff" in the habs/ sub-directory to verify the changes. Then commit those changes in habs/.
6) Run "make world" to build the latest version of the binary repository.
7) If all went well, push the updated version of habs/ to Github.
Furthermore, you can run ...
* "make updates" to get a list of all updates available on Hackage. Edit PKGLIST to pull in those updates that you want, then run "make all" to verify that all version constraints can be fulfilled. This is usually an iterative process, in which one package after another is updated until all updates that can be performed have been found.
* If habs/ has been changed by another committer, you can run "scripts/extract-pkglist habs/habs/* >PKGLIST" to synchronize the PKGLIST file with the current state of the repository.
* If you have updated packages "foo" and "bar" in habs, then run:
make scripts/reverse-dependencies cd habs/habs ../../scripts/reverse-dependencies . foo bar
This will show you the list of packages that need to be re-built after that update. If that looks okay, run
../../scripts/bump $(../../scripts/reverse-dependencies . foo bar)
to bump the $pkgrel of all those packages. Commit those changes in habs/. Then run
scripts/extract-pkglist habs/habs/* >PKGLIST
to synchronize the PKGLIST file, and do the usual "make all && make world" routine to verify that everything is okay before pushing the changes to Github.
It's a bit of a mess, but it works okay. :-)
Should we put these instructions up on the README of the GitHub project for HABS / ArchHaskell ? I've just been manually mucking about with HABS this whole time.
4) Why haskell packages in extra depends on ghc-7.0.3 (still in testing)?
There is an update of GHC available in [testing], and unfortunately some packages have been published that were built with that newer GHC version before GHC itself is available. In short, Haskell support is currently broken in ArchLinux. Unfortunately, these things happen, but it's most probably going to be remedied soon.
Take care, Peter
_______________________________________________ arch-haskell mailing list arch-haskell@haskell.org http://www.haskell.org/mailman/listinfo/arch-haskell
About that - I installed ghc 7.0.3 and glibc 2.1.4 from testing to work around that. I'm hoping they get moved into extra/core sometime soon. -Leif
Hi Leif,
Should we put these instructions up on the README of the GitHub project for HABS / ArchHaskell ?
you are right, we probably should. I didn't do it at the time, because I had the impression that various contributors were using all kinds of different ways to work on HABS, and I didn't want to convey the impression that the scheme I use is in any way authoritative, or even particularly good. In fact, my setup references HABS as a git sub-module precisely because it's intended do be independent of HABS. I guess that situation has changed somewhat in the mean-time.
I've just been manually mucking about with HABS this whole time.
I should have written more detailed instructions for my tool-chain long ago. I am sorry. :-( Take care, Peter
On Sun, Jun 26, 2011 at 05:39:54PM +0200, Peter Simons wrote:
I'll try to describe my work-flow briefly: [...] It's a bit of a mess, but it works okay. :-)
Isn't this still the same workflow you had when you wrote [1]? /M [1] http://www.haskell.org/pipermail/arch-haskell/2011-January/000865.html -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus I invented the term Object-Oriented, and I can tell you I did not have C++ in mind. -- Alan Kay
On Mon, Jun 27, 2011 at 12:51:18AM +0200, Peter Simons wrote:
Hi Magnus,
Isn't this still the same workflow you had when you wrote [1]?
no, it's not.
Since I'm curious, what improvements have you made to the tools that address the inadequacy you reported back then? Have they just been changes to the scripts found in your personal git repo? /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus Perl is another example of filling a tiny, short-term need, and then being a real problem in the longer term. -- Alan Kay
Hi Peter,
2011/6/26 Peter Simons
> 1) In order to test for broken packages, do I really need to use the > makeworld script? I think it would take too much time!
Unfortunately, distributing Haskell packages is a fairly complicated affair, mostly because these packages interact with each other in complex ways. Some problems can be caught without running a test build, but not all of them. My experience is that running 'makeworld' is the only way to be sure that the repository is in a consistent state.
It seems to me that you're speaking about core packages, needed by many others. I think that these packages should be in the repository, others don't interact much: a web framework with a game, for example. My main fear is spending time in compiling, not the builing process itself.
> 2) Past discussions suggest me that the group is centred on a small > number of packages to put in a binary repository. It's wonderful to > have this, but it takes a very long time. What do you think if some > people concentrate in the simplest and fastest task of preparing > PKGBUILDs and put them in a place (i.e. AUR) where the core group > decide witch package worth an inclusion in the repo?
It is possible to update packages on AUR without building them first. So, yes, we could split this effort into one that updates AUR and one that maintains the binary repository. Past experience suggests, though, that packages that are being published without being built first tend to be very unreliable. This means that users will run into build problems, and then they'll complain on AUR or on this list -- and rightfully so! Dealing with those issues, however, is a lot of effort. I'm not sure whether publishing untested PKGBUILDs is going to be very helpful for the general user base.
I wrote very badly what I was thinking: you're perfectly right: packages must be tested. Before updating AUR I would compile the needed dependencies and only that, faster then makeword (that remember me the old bad times with Gentoo :-) ).
> 3) For Peter: could you please give some help information on how to > use the scripts in > https://github.com/peti/arch-haskell/tree/master/scripts ?
I'll try to describe my work-flow briefly: [...]
The work flow can feet, but know you and me have a compiled repo, hopefully the same. Still someone (I assume you) has to copy it to the [Haskell] repo server and update the AUR. Unless this work is totally automated, it sound like a bad bottle-neck, but I have to understand more of you procedure, I hope I'm very wrong on this.
> 4) Why haskell packages in extra depends on ghc-7.0.3 (still in testing)?
There is an update of GHC available in [testing], and unfortunately some packages have been published that were built with that newer GHC version before GHC itself is available. In short, Haskell support is currently broken in ArchLinux. Unfortunately, these things happen, but it's most probably going to be remedied soon.
That's bad. Now pacman tells me that ghc-7.0.3 is here. So ALL haskell repository must be recompiled, isn't it? If I use your "push" script and add +1 to pkgrel is ok? Maybe it's better if I wait a little. It's a shame, because now I have some time to spend... Well, I'll try your method it, I only need to understand things. Bye Fabio
Hi Fabio,
Packages must be tested. Before updating AUR I would compile the needed dependencies and only that, faster then makeword.
why do you assume that your builds would be faster than makeworld?
I'll try to describe my work-flow briefly: [...]
The work flow can feet, but know you and me have a compiled repo, hopefully the same. Still someone (I assume you) has to copy it to the [Haskell] repo server and update the AUR.
I'm not sure whether I've made myself clear, please excuse me if my previous message was misleading. The point of that work-flow is not so much that it generates a binary repository, that is some kind of extra benefit. The point of compiling the repository is to guarantee that packages *do* compile before publishing them on AUR. Please don't overestimate the importance of the binary repository. That repository is merely a by-product of our quality assurance procedure. It's a great by-product, though.
Unless this work is totally automated, it sound like a bad bottle-neck, but I have to understand more of you procedure.
Well, if you do update HABS and do compile an updated repository (to guarantee that those updated packages really work), then I'm sure that we'll figure out a way how you can update AUR (and the binary repository on Andromeda, too).
pacman tells me that ghc-7.0.3 is here. So ALL haskell repository must be recompiled, isn't it?
Yes, and chances are that we need an update to cabal2arch as well. I'll look into it ASAP. For time being, it's probably best if we wait until that stuff has stabilized before pushing any further updates. Take care, Peter
Hi peter,
2011/6/27 Peter Simons
why do you assume that your builds would be faster than makeworld?
now that I looked into the scripts, I understand that I'm quite wrong about time consuming: after the first time, it should be just the same. Sorry for the silly assumption. I'll wait for making changes. Note that I tried makeworld script and rsync failed to connect to andromeda.kiwilight.com but I can ping the server. Cheers, Fabio
On Sun, Jun 26, 2011 at 04:37:53PM +0200, Fabio Riga wrote:
Hi all,
yesterday I submitted a patch to HABS repository just because I met some out-of-date packages in AUR. Then I read the "story" of this group in this mailing list and thought that I can give some help. I think I can take responsibility for a bunch of packages and care for updates, but I need some hints.
1) In order to test for broken packages, do I really need to use the makeworld script? I think it would take too much time! I would prefer to simply compile what I use, also to find if makepkg can resolve dependencies. Launching makeworld in another directory with only my packages would do the job?
I've written a tool called cblrepo[1] which aims to make it possible to maintain a consistent set of packages without having to rebuild to find broken dependencies. It is currently not used by anyone involved in ArchHaskell, AFAIK, but it is surely possible to use to speed up your ArchHaskell work (I know this from first-hand experience). I should point out that cblrepo can do more than this. For ArchHaskell work I think the following are valuable features: - Maintain a consistent set of Haskell packages, checking dependencies when adding and upgrading packages. - An interface to follow Hackage: - Download the entire set of cabal files for all packages. - Check for updates to packages you maintain in the set. - An interface for adding/upgrading packages, takes a cabal file (filename), or URL for a cabal file, or a package name and version number (grabbed directly from Hackage). - Can show a list of all packages that need to have their revision bumped when a set of packages is upgraded. - Can show a list of all packages that need to be rebuilt when a set of packages are upgraded, the list is printed in a good build order. /M [1] http://hackage.haskell.org/package/cblrepo -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus I invented the term Object-Oriented, and I can tell you I did not have C++ in mind. -- Alan Kay
Hi Magnus,
I will also try your tool. Thanks for the advice.
Cheers
Fabio
2011/6/26 Magnus Therning
I've written a tool called cblrepo[1] which aims to make it possible to maintain a consistent set of packages without having to rebuild to find broken dependencies. It is currently not used by anyone involved in ArchHaskell, AFAIK, but it is surely possible to use to speed up your ArchHaskell work (I know this from first-hand experience). I should point out that cblrepo can do more than this. For ArchHaskell work I think the following are valuable features:
- Maintain a consistent set of Haskell packages, checking dependencies when adding and upgrading packages. - An interface to follow Hackage: - Download the entire set of cabal files for all packages. - Check for updates to packages you maintain in the set. - An interface for adding/upgrading packages, takes a cabal file (filename), or URL for a cabal file, or a package name and version number (grabbed directly from Hackage). - Can show a list of all packages that need to have their revision bumped when a set of packages is upgraded. - Can show a list of all packages that need to be rebuilt when a set of packages are upgraded, the list is printed in a good build order.
/M
[1] http://hackage.haskell.org/package/cblrepo -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus
I invented the term Object-Oriented, and I can tell you I did not have C++ in mind. -- Alan Kay
_______________________________________________ arch-haskell mailing list arch-haskell@haskell.org http://www.haskell.org/mailman/listinfo/arch-haskell
On Mon, Jun 27, 2011 at 12:00:21AM +0200, Fabio Riga wrote:
Hi Magnus,
I will also try your tool. Thanks for the advice.
Please let me know how you get on with it. You can report any bugs straight to my email, as I still haven't gotten around to uploading the source to github. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves. -- Alan Kay
participants (4)
-
Fabio Riga -
Leif Warner -
Magnus Therning -
Peter Simons