Hi Tasha,

Most Haskell packages in [extra] really shouldn't be getting updated in the sense of a version bump (from what I understand) until a new version of the HP comes out or there's an official policy change.

this is true for HP packages, of course. However, [extra] and [community] contain plenty of other code that is not specified by Haskell Platform. Examples would be haskell-text or haskell-hashed-storage. I see no reason why those packages shouldn't be updated in principle. As far as I am concerned, I'd be fine with updating packages that *are* specified by HP, too, if there is a strong technical incentive. Anyway, this issue is beside the point I was trying to make. I am concerned about the lack of communication regarding the maintenance of [extra] and [community]. Whatever is going on in these repositories, the members of this list are not involved in the process even though we are directly affected by changes that occur in these places. This is not an ideal situation, and it already *did* cause a significant amount of trouble in the past. Take care, Peter

Hi Tasha,

I would actually characterize this problem as a lack of people with access to [extra] who care about the Haskell packages or have the free time/resources to get a turnaround on issues consistently to a few days at most.

yes, you are right. I completely agree. It would be quite unreasonable to expect Remy and Vesa to be able to deal with that workload, especially considering the fact that we are all volunteers who have other, more important obligations in life than building Haskell packages for ArchLinux.

Please don't take this as me trying to dismiss your concerns.

Yes, of course not. In the same spirit, I'd like to emphasize that I didn't intend to come across as meaning to criticize anyone. It's true that I am little disappointed about the way this project has turned out for me, but that is not because of other people, it's because I had unreasonable expectations. All things considered, I feel that our efforts haven't been perfect, but still very successful. Your comments seem to confirm that, so ... thanks! :-) Take care, Peter

Hi Magnus,

do you think that our earlier discussion was on target, i.e. did we correctly identify the real pain points [...]?

I'm not quite sure what the problems were that we identified in earlier discussions, but I believe that the points I just explained in my previous posting are fairly accurate: our efforts aren't coordinated enough, we don't communicate enough, and our tools require way too much manual effort to get the job done.

[Does] our suggestions on improving our tools still make sense?

Well, I am not sure what concrete suggestions have been made. I am aware of several patches that Remy has come up with, and those patches are quite valuable because they address concrete deficiencies in cabal2arch. Still, cabal2arch is not even close to generating a working habs tree from Hackage without requiring massive human intervention. We have yet to come up with algorithms and strategies to improve that. I realize that my comments in this thread are only semi-helpful, because all I do is point out problems. What I should be doing is pointing out *solutions*. Unfortunately, I don't know the solutions either. Now, there are lots of smart people on this list. I was hoping that we can figure out the solutions together. Take care, Peter

Hi Magnus, I have the impression that you are avoiding the substance of my original article in this thread. I posted that very long and very detailed message, because I believe that it's vital to understand the problems we have to solve before we try to solve them. I would love to hear your opinion about the issues I raised in a little more detail than "this sort of feedback is invaluable." If you feel that my feedback is invaluable, then please respond it instead of re-starting this thread from scratch, ignoring everything I already wrote. Take care, Peter

Hi Magnus,

I was probably too vague, because I was waiting for you to provide more information.

well, if you have any concrete questions, then please ask me! I cannot possibly know that you are waiting for more information unless you tell me about it.

In your initial email you say things are broken, but in my opinion you don't provide enough information for me to understand WHY you say that.

I am sorry, but it strikes me as fairly unreasonable to summarize my original posting as me saying "thinks are broken". I was providing slightly more information than that. Surely you are exaggerating a bit? After all, you yourself said that my feedback was "invaluable". Now you seem to be saying that my posting wasn't worth much. How is it possible that you have changed your opinion so dramatically?

...

1) A lack of coordination. We have made a couple of attempts to define policies, procedures, and responsibilities for this project, but ultimately we never really got anywhere. The net result is that everyone of us is working on whatever interests him, but the others mostly don't even know about it. Consequently, we are inefficient.

This is true to a large extent. What do you feel is left to define?

Most importantly, we should agree on a common goal that we are trying to achieve. Personally, my goal has been to provide up-to-date ArchLinux build instructions for the Haskell packages published on Hackage. To achieve that goal, I maintained our habs tree. Based on that habs tree, I felt that it's feasible to maintain both AUR and a binary package repository, which was my secondary goal. I have kind-of achieved both of these goals, but only when "Hackage" is defined as "5% of Hackage". So if those goals ought to be achieved completely, then we need a strategy how to deal with the remaining 95% of Hackage. This realization compelled me to start this thread. I notice that you didn't commit much to habs, so clearly you had different goals. I wonder what those were? What exactly is it that you are trying to achieve?

...

We develop patches for the ArchLinux library, and when they're ready, we throw them away, because we realize way too late that we don't like what they do.

I don't think this is that uncommon in FOSS when people work on their own without telling people what they are doing.

Yes, of course you are right. But the point of my remark wasn't that this kind of thing would be uncommon. Rather, I complained about the fact that it is highly inefficient. We are 3(!) people trying to maintain a set of almost 2000 packages. How are we supposed to accomplish that when we waste our time writing patches that are ultimately thrown away? We are volunteers doing this project in our spare time, so it's obviously perfectly alright that each of us does whatever he feels like. I'm not trying to tell anyone what they are supposed to do or don't do. However, unless we coordinate our efforts to some extend, we cannot possibly achieve the goal of providing Hackage to ArchLinux users. Based on the assumption that this is our common goal, it seems obvious to me that we need to coordinate our efforts more than we did in the past.

...

Similarly, we perform updates in [extra], and then we revert them again, because we notice way too late that they break someone else's efforts.

I can only agree on this point, and I'm not sure what can be done to fix it, if anything.

My suggestion would be that we try to establish a practice where haskell-* updates in [extra] are communicated on this mailing list *before* they go on-line, or even before they are being performed at all. Basically, the maintainers of [extra] -- Remy and Vesa -- would post a message to this list, saying: "I'm about to update haskell-xyz to version a.b.c". Is anyone aware of any reasons why this might be a problem? Please let me know!"

...

2) Lack of communication. [...]

This is again about [extra], I can only say I wish it were better, but I can't personally do much about it.

In my humble opinion, communication between the maintainers of habs and [extra] needs to be improved. Obviously, we cannot force anyone to do what we'd like them to do, but surely we can contact the ArchLinux guys, explain the situation, and try to work out a solution together with them? I made a concrete suggestion above. Surely, these are reasonable people who understand the problem and are willing to help us out, especially since it's in the best interest of all ArchLinux users. Maybe Xyne and/or Remy can do something for us in that area?

...

3) Inadequate tools. The cabal2arch utility is a great, but in its current shape it cannot re-generate the habs in a fashion that I'd call "automatic".

Have you raised bugs for the individual issues here, so that we have them documented in a central place?

Yes, of course I did. Didn't you receive notifications from Github?

Please provide info of HOW things are broken, i.e. current behaviour and correct behaviour.

If you have specific questions about the bugs that I filed, could you please attach those to the appropriate issue in Github? Besides, it sounds like you weren't aware of the issues that I filed, so it's probably a good idea to look at them first, to figure out what exactly you feel is missing.

...

Furthermore, we seem to have no functioning tools that help us automate the updating process. My experience so far is that even the tiniest trivial update has a significant potential to break the build of dozens of other packages. Basically, there seem to be trivial updates in Haskell land.

What process are you using? Both when adding completely new packages and when puling in a new version of a package. What tools are you currently using? What actions are manual, and what actions are only partially covered by tools?

Well, as I said, we don't have any functioning tools or procedures that assist in process of updating a package. So I'm not using *any* tools, and *all* actions are manual. It feels like you expect me to provide extensive formal documentation of what I've been doing in the last 3 months. I am sorry, but I don't have the time to submit a report about everything. Instead, I recommend that you just try to do a few updates yourself. Then you'll find out what the problems are. Frankly, I'm surprised that you don't know this already. How exactly did you perform updates in the last 3 months? Take care, Peter

Hi Magnus,

To be very blunt, I said your EXPERIENCE is invaluable. In your initial email you didn't recount that experience in a way that clearly pointed out what wasn't working. That means your initial email wasn't invaluable, because there was nothing in it that I could act on.

yes, you are right. In an ideal world, I would have written my original message differently. I was frustrated, because I had spent quite a lot of time working on this project, and it felt like the quality of the results that I was able to achieve didn't compare favorably to the amount of effort that was necessary to achieve them. So I was disappointed, and that emotion clearly showed in my message. Obviously, my message would have been more helpful if it had been written from a purely objective point of view. However, since I *am* emotionally involved in this project, it is hard for me to be 100% objective. Still, the fact that I'm not 100% objective does not imply that my perception would be entirely wrong or inaccurate. In your response, you ignored the contents of my message, because you felt that it didn't contain any specific information that you could act on. You didn't say that, though. Instead, you said the opposite: you said that my feedback would be invaluable (even though you clearly didn't believe that it was, or you would have responded to it). To me, that looked as if you would dismiss my point of view. Personally, I believe that everyone on this list has nothing but the best intentions, but we are imperfect human beings who are prone to miscommunications. Apparently, we've had a miscommunication. Now that we have cleared that up, I think the way to go is to forget about it, and to move forward. Is that all right with you? Take care, Peter

Hi Magnus,

To conclude, I'm perfectly all right with your comments and questions above, however from your tone I'm getting the feeling that are irritated with me.

personally, I would prefer to describe my impressions as concern. The reasons for my concern are simple. For the longest time, Don Steward has been the driving force behind the ArchHaskell project, and he has done an incredible job at bringing Haskell to ArchLinux. Over the course of the last year, however, it had become increasingly obvious that he alone cannot succeed at maintaining a set of 2,000 packages all on his own. It's just too much work for a single person, no matter how smart or enthusiastic he or she is. Last October, Don stepped down and handed the project over to a small group of people, namely you, Remy, and myself. Now, Remy has stated right from the start that he doesn't see himself maintaining HABS. Instead, he expressed an interest in continuing the development of the ArchLinux library and the cabal2arch utility, and that is exactly what he has done since. Considering that Remy is also our main go-to guy when it comes to [extra], I believe it's fair to say that he has been a major contributor of the project, even though he didn't contribute to HABS itself. Anyway, that leaves HABS firmly in the hands of two people: you and me. Unfortunately, it turned out that you didn't manage to contribute anything to the HABS repository in the entire period between October 22nd and yesterday. No updates, no bugfixes, no commits at all. Please don't get me wrong. I don't want to come across as criticizing you for the fact that you didn't have time to contribute! Still, the reality is that after Don's retirement, I am pretty much the only person who has committed time and resources to the maintenance of HABS. So this leaves us in a situation where the Haskell packages published on AUR are maintained by a single person again, only this time it's not Don, it's me. Duh! I didn't want that to happen, and I don't believe that I can keep 2,000 packages up-to-date any more than Don could. In the last two days, you've suddenly become extremely active again, and I can guess what the reasons for that unexpected burst of motivation were, but I wonder whether it is going to last? Are you sure that you can keep that level of activism up for the next 3 months? Are you certain that you have the time to commit to that?

Here's my "process" for updating packages on AUR:

1. Pull down any changes to my local copy of the HABS tree. 2. Open a clean chroot and bind-mount HABS and some personal tools into it. 3. Get the URL for the package that needs updating. 4. Run `cabal2arch <URL>` in the HABS top-level. 5. Switch to the chroot and build the updated package and it's dependencies (I have a simple shell script that does this) 6. If the build fails, then revert the changes to the package in HABS. 7. Go back to 2 and update another package. 8. When no more packages need updating, build the source packages for all modified packages. 9. Use aurploader to upload the source packages to AUR.

The problem with this is that step 5 can take a VERY long time, and sometimes I end up doing unnecessary work because there's no easy way of ordering the packages I'm updating such that the number of builds are minimised.

In http://github.com/peti/arch-haskell, I have automated that entire procedure so that boils down to running "make world". There is also a target "make updates" that will identify and print out all available updates from Hackage. Then I use "make publish" to upload the newly generated binary repository to the kiwilight.com server. The target "make src" will generate a whole bunch of taurballs that can be published on AUR using the aurupload utility. The set of packages that is being built is determined by this file: http://github.com/peti/arch-haskell/blob/master/PKGLIST I hope this helps! Take care, Peter

On Tue, 11 Jan 2011 12:27:54 +0100, Peter Simons wrote: Hi Peter and others, While I didn't post much on this list I try to carefully follow the evolution since I heavily needs to have a working/"well packaged" environment for Haskell in ArchLinux. I'm in the situation where in order to easily maintain multiple machines I have a personal binary repository for non-official packages that I use from others sources like AUR. Recently I switched from AUR to HABS for my needs in Haskell packages [1], with great success. I don't (yet) use much of the ArchHaskell tools (except cabal2arch) and rely on some simple shell scripts I already have for my non-haskell packages. So while I cannot guarantee any time to spend on HABS, I would glad to contribute from time to time. I have a fork on github. Best regards, [1]: cpphs derive haskell-agda haskell-ansi-terminal haskell-archlinux haskell-attempt haskell-attojson haskell-attoparsec haskell-base-unicode-symbols haskell-binary haskell-boolean haskell-bytestring-lexing haskell-bytestring-nums haskell-bytestring-show haskell-bytestring-trie haskell-cairo haskell-cautious-file haskell-cereal haskell-chart haskell-cmdargs haskell-colorize-haskell haskell-colour haskell-convertible-text haskell-crypto-api haskell-csv haskell-data-accessor haskell-data-accessor-template haskell-data-default haskell-data-object haskell-data-object-json haskell-data-object-yaml haskell-dataenc haskell-deepseq haskell-diff haskell-dlist haskell-dyre haskell-enumerator haskell-executable-path haskell-extensible-exceptions haskell-failure haskell-filemanip haskell-fingertree haskell-frquotes haskell-ghc-mtl haskell-ghc-paths haskell-gio haskell-glib haskell-gtk haskell-haskeline haskell-haskell-lexer haskell-haskell-src haskell-haskell-src-exts haskell-hint haskell-hsh haskell-hslogger haskell-hssyck haskell-html haskell-http haskell-hunit haskell-io-storage haskell-jsonb haskell-memotrie haskell-missingh haskell-mmap haskell-monadcatchio-mtl haskell-monadlib haskell-monads-fd haskell-mtl haskell-mtl2 haskell-network haskell-pango haskell-parallel haskell-parsec haskell-pcre-light haskell-puremd5 haskell-quickcheck haskell-regex-base haskell-regex-compat haskell-regex-posix haskell-regex-tdfa haskell-rosezipper haskell-sha haskell-split haskell-strptime haskell-tabular haskell-tagged haskell-tar haskell-terminfo haskell-testpack haskell-text haskell-transformers haskell-txt-sushi haskell-uniplate haskell-unix-compat haskell-utf8-string haskell-utility-ht haskell-vacuum haskell-vector-space haskell-vty haskell-webkit haskell-x11 haskell-x11-xft haskell-xdg-basedir haskell-xhtml haskell-yaml haskell-zlib xmonad xmonad-contrib -- Nicolas Pouillard http://nicolaspouillard.fr

On Wed, 12 Jan 2011 10:02:09 +0000, Magnus Therning wrote:

On Wed, Jan 12, 2011 at 00:28, Nicolas Pouillard wrote:

...

On Tue, 11 Jan 2011 12:27:54 +0100, Peter Simons wrote:

Hi Peter and others,

While I didn't post much on this list I try to carefully follow the evolution since I heavily needs to have a working/"well packaged" environment for Haskell in ArchLinux. I'm in the situation where in order to easily maintain multiple machines I have a personal binary repository for non-official packages that I use from others sources like AUR.  Recently I switched from AUR to HABS for my needs in Haskell packages [1], with great success.  I don't (yet) use much of the ArchHaskell tools (except cabal2arch) and rely on some simple shell scripts I already have for my non-haskell packages.

So while I cannot guarantee any time to spend on HABS, I would glad to contribute from time to time. I have a fork on github.

Great. Let us know if you are happy to file pull requests or would prefer commit access.

Having commit access would be better, then I will either directly push to master the changes I'm confident with. And push to another branch the other changes. In particular I like to have a branch which can goes at the "speed" I want. Cheers, -- Nicolas Pouillard http://nicolaspouillard.fr

On 09/01/11 17:34, Magnus Therning wrote: [...]

...

For instance, we develop patches for the ArchLinux library, and when they're ready, we throw them away, because we realize way too late that we don't like what they do.

I don't think this is that common in FOSS when people work on their own without telling people what they are doing.

What I meant to write was that I don't think this is UNcommon in FOSS when people work on their own without telling others what they are doing. I'd also like to add that I personally try to avoid this issue by raising bugs/issues for just about anything I'm doing. This is just the way *I* like to work; I find it natural. I'm not religious about it though. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus