so what can we do about it? Some barriers to entry: 1. reporting bugs requires logging in, many people still miss this. 2. bug reports get ignored (which is disheartening to users) because... 3. bug reports do not get reported to this list (like ghc ones get reported to the ghc-bugs list). This also means we get no public discussion going on most bugs, and the people who finally close bugs get no public recognition. 4. there is no cabal hacking guide, a short guide to the source code and style would help (eg explaining the main environments functions operate in, ie BuildInfo, LocalBuildInfo). Simple things like adding fields, commands or new flags. 5. it's not clear if it's possible to set up a local hackage servers to help test & hack on the hackage web code. 6. darcs sending patches get rejected by default since the cabal-devel list is subscriber only. Probably more things. People often complain that the internal Cabal code is hard to understand, which is partly true though it's also related to point 4. cabal-install should be easy to hack and there's not a lot to do to make it really work well. We need to get someone to take an interest in it. It should be a cool program to hack on because it should have a huge number of users eventually. It's also important to get this going now since the hackage web site is working better and better with more and more packages and we want to bundle cabal-install with the upcoming ghc release. Hacking infrastructure should be cool because so many people use it. We need to lower the barriers to entry, get more people involved and reward contributers with greater public recognition. We also need more people to review patches. I'm not always especially quick at responding to the few patches we do get contributed. Then of course there's just doing a bit of evangelism to try and get more contributers, eg blog postings or emails to the libraries and -cafe lists. Sorry for the rant, I've been hacking on cabal for a few days. I think it must have made me bitter or tired or something. Quick hacking report: Thomas committed a series of 57 patches to add the new configurations system. Since then Thomas and I have committed 54 patches and closed several bug reports. We were trying some more radical refactorings, like pulling the compiler support out into a proper Compiler abstraction rather than each bit of code embedding per-compiler code locally. That turned out to be a bit too difficult at the moment. I can give a more detailed explanation if anyone else wants to have a go. Similarly we were considering making some of the common environments into proper monads, eg we have many environments with PackageDescription, LocalBuildInfo and Verbosity. But this impacts on the hooks api, again. Duncan
Duncan Coutts wrote:
so what can we do about it?
BTW, thanks for all the cleanup you did over the weekend, looks great.
Some barriers to entry:
1. reporting bugs requires logging in, many people still miss this. 2. bug reports get ignored (which is disheartening to users) because... 3. bug reports do not get reported to this list (like ghc ones get reported to the ghc-bugs list). This also means we get no public discussion going on most bugs, and the people who finally close bugs get no public recognition.
The Trac configuration is set to auto CC this list with bug updates, but the mailing list config needs to be updated to allow them through. Isaac is the only one with admin permissions on the list at the moment, should we add more people? Volunteers?
4. there is no cabal hacking guide, a short guide to the source code and style would help (eg explaining the main environments functions operate in, ie BuildInfo, LocalBuildInfo). Simple things like adding fields, commands or new flags. 5. it's not clear if it's possible to set up a local hackage servers to help test & hack on the hackage web code. 6. darcs sending patches get rejected by default since the cabal-devel list is subscriber only.
Again, this is a mailing-list setup issue. cabal-devel should do the same as cvs-ghc, and moderate by default (but someone needs to do the moderation).
Probably more things.
People often complain that the internal Cabal code is hard to understand, which is partly true though it's also related to point 4.
...and I think a lot can be done to improve things incrementally. There are abstractions that are not consistently used (eg. Program, but it sounds like you've improved things here), and missing abstractions (compiler-specific stuff is not sufficiently abstracted). There are over-complex abstractions: e.g. hooks. There are not-very abstractions: the "simple" build system is wired-in when it was originally intended to be layered on top of the basic infrastructure. I'm not sure whether we should abandon this layering or try to do it properly. All this has arisen because Cabal has been hacked on by a wide range of people, few of whom really took an interest in the design of the system as a whole. Mostly we were trying to get the job done and make Cabal do what we needed, which to a large extent it does. But this is fertile ground for hackers who enjoy cleaning things up and refactoring...
cabal-install should be easy to hack and there's not a lot to do to make it really work well. We need to get someone to take an interest in it. It should be a cool program to hack on because it should have a huge number of users eventually. It's also important to get this going now since the hackage web site is working better and better with more and more packages and we want to bundle cabal-install with the upcoming ghc release.
Hacking infrastructure should be cool because so many people use it. We need to lower the barriers to entry, get more people involved and reward contributers with greater public recognition. We also need more people to review patches. I'm not always especially quick at responding to the few patches we do get contributed.
Then of course there's just doing a bit of evangelism to try and get more contributers, eg blog postings or emails to the libraries and -cafe lists.
Sorry for the rant, I've been hacking on cabal for a few days. I think it must have made me bitter or tired or something.
thanks for bringing this up, it's definitely important. Cabal is fundamental to the Haskell community now, we should look after it.
Quick hacking report: Thomas committed a series of 57 patches to add the new configurations system. Since then Thomas and I have committed 54 patches and closed several bug reports. We were trying some more radical refactorings, like pulling the compiler support out into a proper Compiler abstraction rather than each bit of code embedding per-compiler code locally. That turned out to be a bit too difficult at the moment. I can give a more detailed explanation if anyone else wants to have a go. Similarly we were considering making some of the common environments into proper monads, eg we have many environments with PackageDescription, LocalBuildInfo and Verbosity. But this impacts on the hooks api, again.
Great! Cheers, Simon
On Mon, 2007-08-06 at 10:16 +0100, Simon Marlow wrote:
Duncan Coutts wrote:
so what can we do about it?
BTW, thanks for all the cleanup you did over the weekend, looks great.
Some barriers to entry:
1. reporting bugs requires logging in, many people still miss this. 2. bug reports get ignored (which is disheartening to users) because... 3. bug reports do not get reported to this list (like ghc ones get reported to the ghc-bugs list). This also means we get no public discussion going on most bugs, and the people who finally close bugs get no public recognition.
The Trac configuration is set to auto CC this list with bug updates, but the mailing list config needs to be updated to allow them through. Isaac is the only one with admin permissions on the list at the moment, should we add more people? Volunteers?
I'm a moderator too at the moment, however the list is subscriber only so I only get notified about over-long posts etc. It was previously open but moderated and we got a huge amount of spam.
4. there is no cabal hacking guide, a short guide to the source code and style would help (eg explaining the main environments functions operate in, ie BuildInfo, LocalBuildInfo). Simple things like adding fields, commands or new flags.
Thomas has offered to write a HACKING file.
5. it's not clear if it's possible to set up a local hackage servers to help test & hack on the hackage web code. 6. darcs sending patches get rejected by default since the cabal-devel list is subscriber only.
Again, this is a mailing-list setup issue. cabal-devel should do the same as cvs-ghc, and moderate by default (but someone needs to do the moderation).
How does cvs-ghc deal with this? Does it go via a spam filter first before the moderator? And can the mailman white-list people so on their first post they can be cleared by the moderator to post from then on (without the person having to subscribe)?
Probably more things.
People often complain that the internal Cabal code is hard to understand, which is partly true though it's also related to point 4.
...and I think a lot can be done to improve things incrementally.
Yes.
There are abstractions that are not consistently used (eg. Program, but it sounds like you've improved things here), and missing abstractions (compiler-specific stuff is not sufficiently abstracted). There are over-complex abstractions: e.g. hooks. There are not-very abstractions: the "simple" build system is wired-in when it was originally intended to be layered on top of the basic infrastructure. I'm not sure whether we should abandon this layering or try to do it properly.
Yes, there's more work to do in improving the Program, PreProcessor and Compiler abstractions.
All this has arisen because Cabal has been hacked on by a wide range of people, few of whom really took an interest in the design of the system as a whole. Mostly we were trying to get the job done and make Cabal do what we needed, which to a large extent it does.
Aye. Cabal/Hackage is fantastically successful, despite the amount we grumble about it's shortcomings.
But this is fertile ground for hackers who enjoy cleaning things up and refactoring...
Certainly. How to recruit such people... I might try and do a 5-min status report at AngloHaskell. "Cabal needs you" or something :-) Duncan
On 8/6/07, Duncan Coutts
Cabal/Hackage is fantastically successful, despite the amount we grumble about it's shortcomings.
But this is fertile ground for hackers who enjoy cleaning things up and refactoring...
Certainly. How to recruit such people...
I might try and do a 5-min status report at AngloHaskell. "Cabal needs you" or something :-)
Some time ago Duncan and I briefly discussed a few things that I could do with Cabal. I forget exactly what they were now, but it had to do with getting Gtk2HS working better with it (or at all, even). I think I got sidetracked by school and other code I've been working on for Gtk2HS, so I never got around to it. I'm interested in helping you guys, but I suppose I should look at the Cabal code some more. One thing that Cabal seems to need is a flexible source code dependency resolution system. I think I'd like to hack something out in that area. Pete
Duncan Coutts wrote:
On Mon, 2007-08-06 at 10:16 +0100, Simon Marlow wrote:
Duncan Coutts wrote:
so what can we do about it? BTW, thanks for all the cleanup you did over the weekend, looks great.
Some barriers to entry:
Out of the 6 (very insightful) barriers to entry you mention, I note that items 1, 2, 3, and 6 all have their root cause in spam problems. Trac got overwhelmed by spam when it was more open, as did the moderation section of the mailing list, as you noted. It's really depressing. Maybe there are some tools out there that could help?
1. reporting bugs requires logging in, many people still miss this. 2. bug reports get ignored (which is disheartening to users) because... 3. bug reports do not get reported to this list (like ghc ones get reported to the ghc-bugs list). This also means we get no public discussion going on most bugs, and the people who finally close bugs get no public recognition.
The Trac configuration is set to auto CC this list with bug updates, but the mailing list config needs to be updated to allow them through. Isaac is the only one with admin permissions on the list at the moment, should we add more people? Volunteers?
I believe Duncan has all the rights on the list that I have, as he mentions and I'd be happy to add more volunteers.
I'm a moderator too at the moment, however the list is subscriber only so I only get notified about over-long posts etc. It was previously open but moderated and we got a huge amount of spam.
:(
4. there is no cabal hacking guide, a short guide to the source code and style would help (eg explaining the main environments functions operate in, ie BuildInfo, LocalBuildInfo). Simple things like adding fields, commands or new flags.
Thomas has offered to write a HACKING file.
That's really great, thanks, Thomas! I'd be glad to proof it and help.
5. it's not clear if it's possible to set up a local hackage servers to help test & hack on the hackage web code. 6. darcs sending patches get rejected by default since the cabal-devel list is subscriber only.
Again, this is a mailing-list setup issue. cabal-devel should do the same as cvs-ghc, and moderate by default (but someone needs to do the moderation).
How does cvs-ghc deal with this? Does it go via a spam filter first before the moderator? And can the mailman white-list people so on their first post they can be cleared by the moderator to post from then on (without the person having to subscribe)?
Probably more things.
People often complain that the internal Cabal code is hard to understand, which is partly true though it's also related to point 4. ...and I think a lot can be done to improve things incrementally.
(Snip. This is interesting when compared to this point:)
All this has arisen because Cabal has been hacked on by a wide range of people, few of whom really took an interest in the design of the system as a whole. Mostly we were trying to get the job done and make Cabal do what we needed, which to a large extent it does.
Although I hear that people do complain, I think it's very telling that Cabal has had so many contributers. Very few Haskell libraries that I can think of have had as many contributers as Cabal has. And please remember that to some extent, there's always going to be complaining when reading someone else's code. I'm not saying that there isn't room for improvement, there is definitely room for improvement.
Aye.
Cabal/Hackage is fantastically successful, despite the amount we grumble about it's shortcomings.
But this is fertile ground for hackers who enjoy cleaning things up and refactoring...
Certainly. How to recruit such people...
I might try and do a 5-min status report at AngloHaskell. "Cabal needs you" or something :-)
That would be great. peace, isaac
On Mon, 2007-08-06 at 10:00 -0700, Isaac Potoczny-Jones wrote:
Duncan Coutts wrote:
On Mon, 2007-08-06 at 10:16 +0100, Simon Marlow wrote:
Duncan Coutts wrote:
so what can we do about it? BTW, thanks for all the cleanup you did over the weekend, looks great.
Some barriers to entry:
Out of the 6 (very insightful) barriers to entry you mention, I note that items 1, 2, 3, and 6 all have their root cause in spam problems. Trac got overwhelmed by spam when it was more open, as did the moderation section of the mailing list, as you noted. It's really depressing. Maybe there are some tools out there that could help?
There is an anti-spam plugin for Trac, it uses the same anti-spam service as we've been using successfully on the Gtk2Hs website for the past couple years. However the anti-spam plugin requires upgrading to Trac 0.10 from our current 0.9 version. We've discussed this at length for many many months and Ian tried to do the upgrade but he found it needed a later version of Python or lots of hacking. It looks like we'd need some support from the sysadmin to get it upgraded. Another alternative that has been discussed is to move the trac instances to the community server which runs more recent software and where we have root.
The Trac configuration is set to auto CC this list with bug updates, but the mailing list config needs to be updated to allow them through. Isaac is the only one with admin permissions on the list at the moment, should we add more people? Volunteers?
I believe Duncan has all the rights on the list that I have, as he mentions and I'd be happy to add more volunteers.
I've emailed you separately about this.
Thomas has offered to write a HACKING file.
That's really great, thanks, Thomas! I'd be glad to proof it and help.
Yes, I'm looking forward to that.
All this has arisen because Cabal has been hacked on by a wide range of people, few of whom really took an interest in the design of the system as a whole. Mostly we were trying to get the job done and make Cabal do what we needed, which to a large extent it does.
Although I hear that people do complain, I think it's very telling that Cabal has had so many contributers. Very few Haskell libraries that I can think of have had as many contributers as Cabal has.
Yes, by many measures it's extremely successful.
And please remember that to some extent, there's always going to be complaining when reading someone else's code.
Of course :-)
I'm not saying that there isn't room for improvement, there is definitely room for improvement.
But this is fertile ground for hackers who enjoy cleaning things up and refactoring...
Certainly. How to recruit such people...
I'm quite pleased that I've been getting a few patches recently from various people, code style/consistency cleanups, bug fixes etc. There has been more talk of refactorings such as making cleaner separations between the cabal library api and the command line user interface, and similarly clear separation between the declarative Cabal info and the Simple build system. Another important one people have talked about is separating the local build config serialisation format from the information that is in the environment for a build. Separating them should allow greater abstraction by allowing us to add function parameters to the build environment rather than everything having to be concrete data for the sake of serialisation. In particular this should allow us to put functions into the Compiler abstraction. There has also been some good discussion of the next big feature, dependency tracking.
I might try and do a 5-min status report at AngloHaskell. "Cabal needs you" or something :-)
That would be great.
I got quite a good response from this. Several people have expressed interest in looking at cabal-install in particular. Duncan
participants (4)
-
Duncan Coutts -
Isaac Potoczny-Jones -
Peter Gavin -
Simon Marlow