The Lisp Curse

Andrew Coppin

19 May 2011 19 May '11

2:33 p.m.

http://www.winestockwebdesign.com/Essays/Lisp_Curse.html Some of you might have seen this. Here's the short version: Lisp is so powerful that it discourages reuse. Why search for and reuse an existing implementation, when it's so trivially easy to reimplement exactly what you want yourself? The net result is a maze of incompatible libraries which each solve a different 80% of the same problem. To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

Show replies by date

Vo Minh Thu

19 May 19 May

2:40 p.m.

2011/5/19 Andrew Coppin :

...

http://www.winestockwebdesign.com/Essays/Lisp_Curse.html

Some of you might have seen this. Here's the short version:

Lisp is so powerful that it discourages reuse. Why search for and reuse an existing implementation, when it's so trivially easy to reimplement exactly what you want yourself? The net result is a maze of incompatible libraries which each solve a different 80% of the same problem.

To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

Hi Andrew, So what exactly is the problem on hackage and what do you propose as a solution? Surely you don't want people to upload a library on hackage only once it is perfect (nor do you think such a perfect, one-size-fits-all library might exist)? I haven't read the provided link but I also guess you don't _really_ mean that the referred packages on hackage were 'so trivially easy to reimplement', nor that it is not possible to use them together... Cheers, Thu

Serguey Zefirov

2:50 p.m.

2011/5/19 Vo Minh Thu :

...

2011/5/19 Andrew Coppin :

...
http://www.winestockwebdesign.com/Essays/Lisp_Curse.html

Some of you might have seen this. Here's the short version:

Lisp is so powerful that it discourages reuse. Why search for and reuse an existing implementation, when it's so trivially easy to reimplement exactly what you want yourself? The net result is a maze of incompatible libraries which each solve a different 80% of the same problem.

To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

So what exactly is the problem on hackage and what do you propose as a solution?

The problem is that you have to try several packages before you get to the stable point. The solution... I think that some ratings, like "used directly by ### packages/projects and indirectly by ###" would be nice, but not much. As for me, I like the diversity of packages. They attack close problems from different fronts. They express different ideas and views. I like all that.

Felipe Almeida Lessa

3:39 p.m.

On Thu, May 19, 2011 at 3:50 PM, Serguey Zefirov wrote:

...

The solution... I think that some ratings, like "used directly by ### packages/projects and indirectly by ###" would be nice, but not much.

As for me, I like the diversity of packages. They attack close problems from different fronts. They express different ideas and views. I like all that.

Regarding the Unicode problem, there is a standard solution today: use the text package. Yes, you may use other libraries, but text is the recommended one. The problem is that no one coming to Hackage discovers that on Hackage itself. You have to dig the Internet to find some blog post telling you what to do. And hope that the blog post isn't old. So we really need a way of raking libraries in Hackage. We already have the Haskell Platform, which is really really nice, but the HP can't embrace everything. For everything outside HP, a ranking system is needed. Cheers! =) -- Felipe.

Andrew Coppin

21 May 21 May

5:13 a.m.

On 19/05/2011 08:39 PM, Felipe Almeida Lessa wrote:

...

Regarding the Unicode problem, there is a standard solution today: use the text package. Yes, you may use other libraries, but text is the recommended one.

Is that true? Last I heard, there was still some debate about the relative performance of the various libraries. (I don't recall hearing much argument about API though, so presumably everybody more or less agrees on that.)

...

The problem is that no one coming to Hackage discovers that on Hackage itself. You have to dig the Internet to find some blog post telling you what to do. And hope that the blog post isn't old.

...I suppose what I just wrote basically confirms this. I remember seeing some traffic about text performance, but I wasn't paying enough attention at the time to remember the precise details. (Or perhaps it's just that this specific subject area isn't generally all that important to me.)

...

So we really need a way of raking libraries in Hackage. We already have the Haskell Platform, which is really really nice, but the HP can't embrace everything. For everything outside HP, a ranking system is needed.

Sounds like a good idea to me. My only minor worry is that when a once-popular library becomes superceeded by a superior competetor, the old library will still have a really good rating, and people might not "find" the new replacement. Then again, how often is that likely to happen?

Roel van Dijk

20 May 20 May

3:28 a.m.

On 19 May 2011 20:50, Serguey Zefirov wrote:

...

The solution... I think that some ratings, like "used directly by ### packages/projects and indirectly by ###" would be nice, but not much.

Maybe my reverse dependencies mirror of hackage could be useful here: http://bifunctor.homelinux.net/~roel/hackage/packages/hackage.html The mirror was intended to show the workings of a patch for the old (current) hackage. It is updated daily. The algorithm needs some attention so don't trust it blindly, but in general it is quite accurate.

Markus Läll

6:46 a.m.

What's stopping it from being put on the official hackage? I use it quite a lot to find well established packages and/or example code, and am quite fond of it. But it is only visible when you know that this exists. On Fri, May 20, 2011 at 10:28 AM, Roel van Dijk wrote:

...

On 19 May 2011 20:50, Serguey Zefirov wrote:

...
The solution... I think that some ratings, like "used directly by ### packages/projects and indirectly by ###" would be nice, but not much.

Maybe my reverse dependencies mirror of hackage could be useful here: http://bifunctor.homelinux.net/~roel/hackage/packages/hackage.html

The mirror was intended to show the workings of a patch for the old (current) hackage. It is updated daily.

The algorithm needs some attention so don't trust it blindly, but in general it is quite accurate.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Markus Läll

Roel van Dijk

7:52 a.m.

On 20 May 2011 12:46, Markus Läll wrote:

...

What's stopping it from being put on the official hackage? I use it quite a lot to find well established packages and/or example code, and am quite fond of it. But it is only visible when you know that this exists.

Poor timing. I wrote the patch when there was significant work being done for the new version of hackage [1]. So the feeling was that the current hackage (which is a bunch of cgi programs) would soon be replaced by the new hackage-server (based on happstack). I also stated in my initial announcement that I would look into writing a version of my reverse dependencies for the new server. I haven't done that yet. Since at least July of 2010 the new server has some support for reverse dependencies. Unfortunately hackage-server seems to have lost a bit of momentum. It is now almost 3 years old. Looking at "darcs show authors" I'd say there are about 4 main contributors, with the main author being Duncan Coutts. The last patches are from December 2010. I think the best solution is to get more people contributing to hackage-server. It is superior to the current hackage in many ways. 1 - http://code.haskell.org/hackage-server/

Gilberto Garcia

19 May 19 May

2:56 p.m.

I think what Andrew meant is that it's not a good idea to have big pile of different implementations of the same library, and all trying to solve the very same problem. I see this kind of problem in the java community. It seems that developers have a need to create everything from scratch more than making existing ones better. Short history, for example, why do we have to have N libraries to read a file? can't we have just one damn good one? Cheers

Daniel Patterson

3:17 p.m.

Correct my ignorance as I'm rather new around here, but I'm not sure if I actually think this happens that much. Different approaches are often put forth, which does mean that there are incompatible libraries that fill the same space for a while, but it seems that once it becomes clear what the best approach is, that becomes pretty well accepted. Are there really example of multiple libraries that use the same approach to solve the same problem that are incompatible and actively used? Right now there are a bunch of iteratee/enumerator/whatever term you want to call it libraries, but those are new ideas, and it isn't clear yet what the best approach is. In an open source community, there will always be multiple approaches to common problems. If as soon as one person came up with a solution, all others had to build upon that, there would be no progress, as no one would be able to try out new ideas. What is important is that there is visibility for packages so that people can see what else is out there and can decide if it makes the most sense to: A. use an existing solution B. improve an existing solution C. start from scratch, because of wanting a fundamentally different approach. I think hackage provides that. So what's the problem?

...

Short history, for example, why do we have to have N libraries to read a file? can't we have just one damn good one?

The only way to get there is to have many solutions and then decide what the best one is! Just my .02 On May 19, 2011, at 2:56 PM, Gilberto Garcia wrote:

...

I think what Andrew meant is that it's not a good idea to have big pile of different implementations of the same library, and all trying to solve the very same problem.

I see this kind of problem in the java community. It seems that developers have a need to create everything from scratch more than making existing ones better.

Cheers

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Andrew Coppin

4:20 p.m.

On 19/05/2011 07:56 PM, Gilberto Garcia wrote:

...

I think what Andrew meant is that it's not a good idea to have big pile of different implementations of the same library, and all trying to solve the very same problem.

I'm glad somebody understood what I was trying to get at. I'm not saying that we shouldn't ever have more than one library tackling the same problem. I'm just saying that when people say "we have multiple libraries competing to solve this problem, and that's *great*"... well, not necessarily, no. Obviously there's room for finding out what the best way to solve the problem is, but ultimately the ideal is to end up with one library that solves the problem well, which everybody can use.

Julian Porter

5:30 p.m.

...

ultimately the ideal is to end up with one library that solves the problem well, which everybody can use.

aditya siram

5:48 p.m.

I wonder if it would be useful to be able to download and use only necessary modules from Hackage. This way if someone writes, say a superior XML parsing API, and someone else has better generating API, the user can pull just those modules , write the glue and have the best of both worlds. On the upside this would: 1. Make for a smaller local cabal repository. 2. Mitigate the problem of having to compile a package that has functionality you don't need. MissingH [1] is a good example. 3. Allow you to continue working even if one of the modules doesn't compile. I recently ran into this with liboleg [2] where I only wanted Control.CCCXe, but couldn't "cabal install" it on my system because one of the other modules failed to compile. -deech [1] http://hackage.haskell.org/package/MissingH-1.1.0.3 [2] http://hackage.haskell.org/package/liboleg On Thu, May 19, 2011 at 4:30 PM, Julian Porter wrote:

...

...
ultimately the ideal is to end up with one library that solves the problem well, which everybody can use.

Nonsense. One library that everyone can use with either end up being so small in functionality that it's actually useless, or so general that either it requires tons and tons of boilerplate just to use it at all, or it's really about eight libraries rolled into one and so impossible to find your way around. Whichever, it's not good. The sweet spot is at the point of maximum tension between generality and simplicity. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Paolo G. Giarrusso

21 May 21 May

11:22 a.m.

Hi, while I'm still a Haskell learner, I wanted to contribute my point of view, which I hope is different enough to be useful. First, of course we don't need to restrict Hackage to contain just 1 library for every purpose, we just need to give incentives for people to collaborate. And if they really have different ideas, they should compare their approaches and their implementation, to some extent. For the unicode example, the question is: why people kept creating new solutions to the same problem, instead of working together? At least, why don't those people document their different goals, if they have some? Andrew Coppin is not trolling. He is calling the community to reflect on why this happens and to find together a solution. To me, it seems that already agreeing that there is a problem would be already a step forward. Daniel Peebles wrote, as an answer:

...

But people generally hate working with other people's code, so I doubt that'll have much of an effect :)

I only hate working on somebody else's code when it's bad/unclear/ badly documented - IMHO code is good when it's a pleasure to work on it. 1) Collaboration and critical mass A library is not just code. It is an ecosystem of code, documentation, experience is using it and refining the design (that is, client software), tutorials and blog posts, and generally any form of shared knowledge. If we split this, none of the projects reach the critical mass. Interestingly, the smaller size of the Haskell community is itself one of the reason of the problem. Also in other languages there are often more solutions for the same problem - I bet every language has tons of interesting JSON libraries, together with not-so-interesting ones. However, I bet that for an interesting domain and a popular language, the best approaches are automatically selected: few libraries reach the critical mass and become popular. Haskell has less developers, so we need more efficient tools to build shared knowledge. It seems that the future Hackage will have comments; I'd also like a guideline such that authors compare their work to the alternatives. Just as papers get rejected when they miss relevant related work, and for good reasons, packages will get downvoted if the author is not aware of another solution. Then, we can encourage the author to either improve his library, or to "deprecate" it if he doesn't care. 2) The Haskell platform - just a partial answer. I understand it was born exactly to address this problem, so it confirms it; I don't think that everything can fit in there, so it doesn't solve the problem for everything - except for what can be included. 3) The advantage of multiple interfaces in 1 library. (Answering to Julian Porter) That is, interoperability. A simpler and a more complex interface might be needed. But why do they have to be into different libraries, written without a coherent design, and without (say) wrapper/converter functions? Probably, in many cases, I want to have 1 library which is general, supplies some standard boilerplate for the common use case, has a single guide which allows me to find my way around. Of course that's a final product, but we need at least to aim for that. This is my answer to Julian Porter's point: [

...

ultimately the ideal is to end up with one library that solves the problem well, which everybody can use.

Nonsense. One library that everyone can use with either end up being so small in functionality that it's actually useless, or so general that either it requires tons and tons of boilerplate just to use it at all, or it's really about eight libraries rolled into one and so impossible to find your way around. Whichever, it's not good. The sweet spot is at the point of maximum tension between generality and simplicity. ] 4) Development level "There are 3-4 versions of iteratee because we are still not sure about the right design". Hmm, this suggests that the concept is still in a research stage and incomplete - so newcomers should either spend lots of effort studying all libraries or avoid using the concept at all because it's too complex. If instead there is consensus that different approaches are useful in different cases, then please compare them, especially because libraries with an advanced type-level construction have a steeper learning curve. One problem here, I think, is also that a lot of us are researchers, and most research produces prototypes which are then often not maintained, leaving a complete implementation to "somebody else". I'm not accusing anybody - I'm a PhD student, and I realize that doing otherwise might be detrimental to my career, at least in its early stages, since I'm supposed to prototype new, innovative ideas instead of complete existing ones. But is that really true for everybody? Or am I mistaken here? On May 19, 10:20 pm, Andrew Coppin wrote:

...

On 19/05/2011 07:56 PM, Gilberto Garcia wrote:

...
I think what Andrew meant is that it's not a good idea to have big pile of different implementations of the same library, and all trying to solve the very same problem.

I'm glad somebody understood what I was trying to get at.

I'm not saying that we shouldn't ever have more than one library tackling the same problem. I'm just saying that when people say "we have multiple libraries competing to solve this problem, and that's *great*"... well, not necessarily, no. Obviously there's room for finding out what the best way to solve the problem is, but ultimately the ideal is to end up with one library that solves the problem well, which everybody can use.

_______________________________________________ Haskell-Cafe mailing list Haskell-C...@haskell.orghttp://www.haskell.org/mailman/listinfo/haskell-cafe

Serguey Zefirov

19 May 19 May

2:45 p.m.

I think this is much less applicable to Haskell than to Lisp. I think that most of intra-incompatibilities of Lisp stem from side effects. The rest is mostly due to (relatively) weak type system which let some errors slip. And remaining percent or two can be attributed to the power of Lisp. ;) 2011/5/19 Andrew Coppin :

...

http://www.winestockwebdesign.com/Essays/Lisp_Curse.html

Some of you might have seen this. Here's the short version:

Lisp is so powerful that it discourages reuse. Why search for and reuse an existing implementation, when it's so trivially easy to reimplement exactly what you want yourself? The net result is a maze of incompatible libraries which each solve a different 80% of the same problem.

To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Stephen Tetley

3:39 p.m.

Och Mr Coppin Lisp is a fine language, but all "Lisp" essays you'll find on the internet except Richard Gabriel's "Worse is Better" are absolute tosh. Read Olin Shiver's introduction to SRE regex notation for an intelligent contribution to the "6 different libraries" problem you seem to be having, rather than some cargo cultist muddy thinking. http://www.scsh.net/docu/post/sre.html By the way, referencing the original article circa half way down - didn't Mark P. Jones write Gofer and the original Hugs by himself - people actually used those rather than Qi.

Don Stewart

3:58 p.m.

This is classic community trolling behavior, Andrew. You post something inflammatory, questioning the core value of our project, without a clear argument about why it article relevant, and then step away to let a monster thread consume everything, as people try to work out what your point was, not trying to argue a point, or other wise participate. Doing this, year after year, is bad for -cafe@ and bad for the community, and why I don't use -cafe@ for problem solving anymore. -- Don I'm glad my mail reader has a "mute" button. On Thu, May 19, 2011 at 12:39 PM, Stephen Tetley wrote:

...

Och Mr Coppin

Lisp is a fine language, but all "Lisp" essays you'll find on the internet except Richard Gabriel's "Worse is Better" are absolute tosh.

Read Olin Shiver's introduction to SRE regex notation for an intelligent contribution to the "6 different libraries" problem you seem to be having, rather than some cargo cultist muddy thinking.

http://www.scsh.net/docu/post/sre.html

By the way, referencing the original article circa half way down - didn't Mark P. Jones write Gofer and the original Hugs by himself - people actually used those rather than Qi.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Andrew Coppin

4:23 p.m.

On 19/05/2011 08:58 PM, Don Stewart wrote:

...

This is classic community trolling behavior, Andrew.

And publicly calling somebody a troll isn't trolling behaviour? I'm going to answer the rest of this off-list. I'm sure nobody else wants to hear it.

Simon Michael

20 May 20 May

12:36 p.m.

Hey now.. maybe so, but this thread is an interesting one.

Juan Daugherty

2:03 p.m.

Prolly noted already but the original presumption is false. The optimal and right thing would be if there was an evolutionary set in each functional area from which you could choose and a common framework in which any selection from that area could work. Otherwise seems likes a purposeless kvetch. -- =============================================== juan@acm.org | Juan.Daugherty on Skype | 01 716 524 0542 =============================================== -------PGP SIGNATURE Attached If Signed-------=

Andrew Coppin

19 May 19 May

4:20 p.m.

On 19/05/2011 08:39 PM, Stephen Tetley wrote:

...

Och Mr Coppin

Lisp is a fine language, but all "Lisp" essays you'll find on the internet except Richard Gabriel's "Worse is Better" are absolute tosh.

This wasn't an attempt to bash Lisp. This is about all those people who think having multiple libraries which only solve half the problem is somehow a "good thing".

vagif.verdi＠gmail.com

4:34 p.m.

Andrew, you are being non constructive. You are saying "We" should. Who "we", Andrew ? Who are you referring to ? The developers who created those six different unicode libraries are not united under any umbrella you can call "we". The reason those six libraries existis is NOT because some mysterious secret "we" commitee decided to have six incompatible libraries. And the maintainers of hackage has better things to do than to police and make arbitrary decisions what should or should not be "allowed" to hackage. Also hackage is NOT a standard library, just like SourceForge or Github is not a standard library. It's just a hosting for haskell libraries. If you want someone tell you what to use, there's an effort in that direction: haskell platform. And their attitue is constructive. They don't demand that "we" (whoever that is) do or do not do something. They compile their own set of libraries and announce about their existence. Go lobby them to include one blessed unicode library. On Thursday, May 19, 2011 01:20:50 PM Andrew Coppin wrote:

...

On 19/05/2011 08:39 PM, Stephen Tetley wrote:

...
Och Mr Coppin

Lisp is a fine language, but all "Lisp" essays you'll find on the internet except Richard Gabriel's "Worse is Better" are absolute tosh.

This wasn't an attempt to bash Lisp.

This is about all those people who think having multiple libraries which only solve half the problem is somehow a "good thing".

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Andrew Coppin

4:56 p.m.

On 19/05/2011 09:34 PM, vagif.verdi@gmail.com wrote:

...

Andrew, you are being non constructive.

It seems I'm being misunderstood. Some people seem to hold the opinion that more libraries = better. I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself. I'm not trying to say "OMG, the way it is now completely sucks!" I'm not trying to say "you must do X right now!" I'm just trying to put forward an opinion. The opinion that having too many libraries can be a problem, which some people don't seem to agree with. (Obviously it isn't *always* bad, I'm just saying that sometimes it can be.) That's all I was trying to say.

Ketil Malde

5:43 p.m.

Andrew Coppin writes:

...

I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I don't think so, the article seems to talk more about social problems among lisp users, which it - at least in part - ascribes to the technical prowess of the language. I don't think the article makes its point very well, though. For instance, it uses object orientation as an example, but CLOS is fairly standard in Lisp, I believe, and the author had to turn to Scheme for his examples. The problem again seems to be more social (the lack of an active committee to do standardizing) than technical.

...

The opinion that having too many libraries can be a problem, which some people don't seem to agree with.

I don't see how the number of available libraries is a problem in itself, but it would be nice if hackage or some other resource provided more help in recommending which library to try first. We do have standardization efforts, committees bringing the language forward and an inclusive and collaborative community. -k -- If I haven't seen further, it is by standing in the footprints of giants

Andrew Coppin

21 May 21 May

5:03 a.m.

On 19/05/2011 10:43 PM, Ketil Malde wrote:

...

Andrew Coppin writes:

...
I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I don't think so, the article seems to talk more about social problems among lisp users, which it - at least in part - ascribes to the technical prowess of the language.

Actually, the author seems to be saying that Lisp is [relatively] unsuccessful because it's so powerful that is discourages cooperation. Personally I don't buy it; I'm sure there are *several* reasons why Lisp never became The Next Big Thing. (There's a dozen dialects of it, for starters...) But hey, I'm not really interested in Lisp. I'm interested in Haskell. What I took away from the article is that if you design a really powerful language, paradoxically this can make it cheaper to reimplement stuff rather than reuse it. You'd think a language with strong support for reuse would encourage reuse, but not necessary. As you point out, people not cooperating is a social rather than technical problem. It seems obvious once you say it out loud, but I hadn't really conciously realised that. Consequently, the language isn't the whole story. If you make the language really powerful, that reduces the cost of implementing things from scratch. On the other hand, if you can do things that reduce the cost of finding existing code that you could reuse, you swing the balance back in the other direction. We have Hackage (and now HP). I have no idea whether Lisp has anything comparable.

...

...
The opinion that having too many libraries can be a problem, which some people don't seem to agree with.

I don't see how the number of available libraries is a problem in itself, but it would be nice if hackage or some other resource provided more help in recommending which library to try first. We do have standardization efforts, committees bringing the language forward and an inclusive and collaborative community.

I certainly don't think having a lot of libraries available is bad. Having lots of libraries available for /the same task/ is different. As some others have pointed out, it varies somewhat depending on what the task in question is. If Haskell still used the old lazy-list I/O approach and Hackage had 3 different monadic I/O libraries implemented on top of that, that would be really bad. Fortunately, once everybody realised that monadic I/O was the way to go, we ended up with one library which everybody uses. It has costs (e.g., you cannot redefine the Monad class to include, say, Data.Set), but it has the huge, huge benefit that nobody will ever utter the words "I'd like to bolt HsLogger onto Yesod - oh wait, I can't, they use incompatible IO monads". On the other hand, there are lots of GUI libraries. Because they bind to different toolkits, or work on different combinations of platforms, or because they wrap low-level bindings with more "functional" interfaces, or whatever. We wouldn't want a massive *explosion* of identical GUI libraries, but I don't think the current set is excessive at all.

Daniel Peebles

19 May 19 May

6 p.m.

The way I understand it, you're saying not that we shouldn't be doing it this way (since it isn't centrally managed, it's the only possible way), but that we shouldn't be "bragging" (for lack of a better word) that we have lots of libraries that do a specific thing. Or if not that, then at least that it isn't a clear win. I agree that from an end-user's perspective it isn't always a clear win, but I do think that having a bunch of libraries (even ones that do the same thing) an indicator of a healthy, active, and enthusiastic community. Sure, it's decentralized and people will often duplicate effort, but different variations on the same idea can also help explore the design space and will reveal to everyone interested what works and what doesn't. But yeah, if you want to do X and you encounter 15 libraries that do X and can't find a clear consensus on what's best, I can understand why that might be frustrating. I don't think there's really a clear solution to that though, other than gently encouraging collaboration and scoping out of existing work before starting new work. But people generally hate working with other people's code, so I doubt that'll have much of an effect :) Dan On Thu, May 19, 2011 at 4:56 PM, Andrew Coppin wrote:

...

On 19/05/2011 09:34 PM, vagif.verdi@gmail.com wrote:

...
Andrew, you are being non constructive.

It seems I'm being misunderstood.

Some people seem to hold the opinion that more libraries = better. I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I'm not trying to say "OMG, the way it is now completely sucks!" I'm not trying to say "you must do X right now!" I'm just trying to put forward an opinion. The opinion that having too many libraries can be a problem, which some people don't seem to agree with. (Obviously it isn't *always* bad, I'm just saying that sometimes it can be.)

That's all I was trying to say.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

austin seipp

6:38 p.m.

I too am not all that concerned about the library proliferation, and I think such work can definitely help find the best design for certain abstractions. There are no less than 3 iteratee libraries - 4 including liboleg's original IterateeM formulation - and a number of FRP implementations as well, but part of that is because we haven't found the best designs for such things yet. Doing that will take time and effort on the part of the community, whether there's 3 competing libraries or 30. In the case of Unicode, I think the community has clearly spoken in its adoption of the `text` package into the Platform, as well as it being one of the most depended on hackage packages. Certainly the platform doesn't recommend a 'blessed' package for every need. JSON is a simple example, and there are many various JSON implementations on Hackage. Outside of the platform, I think there should definitely be a better way to communicate to users what library might be best. I don't know how that would look - Hackage 2 will have commenting and reverse dependencies I think, but when that future hackage will rise is not clear. One thing is for certain: in the Haskell ecosystem, if your code does not exist on Hackage, and *especially* if it does not use Cabal, realistically speaking, it is dead as far as the larger community is concerned (the exception being GHC itself, perhaps. Even gtk2hs - who was one of the biggest outliers for a long time, now is cabalized.) I think I would ultimately rather encourage people to submit code - even if it is small or perhaps duplicate. At least give it and its ideas the chance to survive, be used and talked about, and die if not - rather than resign it to such a fate prematurely. On Thu, May 19, 2011 at 5:00 PM, Daniel Peebles wrote:

...

The way I understand it, you're saying not that we shouldn't be doing it this way (since it isn't centrally managed, it's the only possible way), but that we shouldn't be "bragging" (for lack of a better word) that we have lots of libraries that do a specific thing. Or if not that, then at least that it isn't a clear win. I agree that from an end-user's perspective it isn't always a clear win, but I do think that having a bunch of libraries (even ones that do the same thing) an indicator of a healthy, active, and enthusiastic community. Sure, it's decentralized and people will often duplicate effort, but different variations on the same idea can also help explore the design space and will reveal to everyone interested what works and what doesn't. But yeah, if you want to do X and you encounter 15 libraries that do X and can't find a clear consensus on what's best, I can understand why that might be frustrating. I don't think there's really a clear solution to that though, other than gently encouraging collaboration and scoping out of existing work before starting new work. But people generally hate working with other people's code, so I doubt that'll have much of an effect :) Dan

On Thu, May 19, 2011 at 4:56 PM, Andrew Coppin wrote:

...
On 19/05/2011 09:34 PM, vagif.verdi@gmail.com wrote:

...
Andrew, you are being non constructive.

It seems I'm being misunderstood.

Some people seem to hold the opinion that more libraries = better. I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I'm not trying to say "OMG, the way it is now completely sucks!" I'm not trying to say "you must do X right now!" I'm just trying to put forward an opinion. The opinion that having too many libraries can be a problem, which some people don't seem to agree with. (Obviously it isn't *always* bad, I'm just saying that sometimes it can be.)

That's all I was trying to say.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Regards, Austin

Tillmann Vogt

20 May 20 May

4:44 a.m.

Am 20.05.2011 00:00, schrieb Daniel Peebles:

...

I don't think there's really a clear solution to that though, other than gently encouraging collaboration and scoping out of existing work before starting new work. But people generally hate working with other people's code, so I doubt that'll have much of an effect :)

Hi Daniel, Now that you are mentioning it, how can I gently encourage you to accept my patch to one of your libraries? You didn't answer my mail. Sorry that I have to make this public but what else should I do. Writing again a mail? To show the debugging-capabilities of collada-output I made a video which explains why it is better: http://www.youtube.com/watch?v=ZpJzG28Yv8Y With 3-tuples it is a little bit better. With 4 or 5 tuples it is a lot better (meaning all dimensions increase with the same speed). - Tillmann

Andrew Coppin

21 May 21 May

5:05 a.m.

On 19/05/2011 11:00 PM, Daniel Peebles wrote:

...

I agree that from an end-user's perspective it isn't always a clear win, but I do think that having a bunch of libraries (even ones that do the same thing) an indicator of a healthy, active, and enthusiastic community.

I won't argue with that. ;-)

...

Sure, it's decentralized and people will often duplicate effort, but different variations on the same idea can also help explore the design space and will reveal to everyone interested what works and what doesn't.

Yeah, as some others have said, it depends on the problem domain. For some things, it's really not clear what the correct abstractions are yet, and there's still a lot of design space to explore. (I might pick at random, say, Functional Reactive Programming. As best as I can tell, several people have ideas, but there's no clear "best way" of doing this yet. We're still experimenting.) For stuff like that, it's useful to have lots of people trying different stuff.

...

But yeah, if you want to do X and you encounter 15 libraries that do X and can't find a clear consensus on what's best, I can understand why that might be frustrating.

Moreover, there can be network effects involved. A while back somebody from Google wrote that everybody agrees that String is too slow for high-performance I/O, but each library that involves high-performance I/O tends to end up using a different text representation, which rather impedes compatibility between them.

...

I don't think there's really a clear solution to that though, other than gently encouraging collaboration and scoping out of existing work before starting new work. But people generally hate working with other people's code, so I doubt that'll have much of an effect :)

Yeah, I'm not going to present I have a solution. I merely wanted to point out that it's a problem which does exist. Some of the other folks in this thread seem to be coming up with useful ideas though.

David Leimbach

19 May 19 May

6:42 p.m.

See the Haskell Platform. Sent from my iPhone On May 19, 2011, at 1:56 PM, Andrew Coppin wrote:

...

On 19/05/2011 09:34 PM, vagif.verdi@gmail.com wrote:

...
Andrew, you are being non constructive.

It seems I'm being misunderstood.

Some people seem to hold the opinion that more libraries = better. I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I'm not trying to say "OMG, the way it is now completely sucks!" I'm not trying to say "you must do X right now!" I'm just trying to put forward an opinion. The opinion that having too many libraries can be a problem, which some people don't seem to agree with. (Obviously it isn't *always* bad, I'm just saying that sometimes it can be.)

That's all I was trying to say.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Eric Rasmussen

8:12 p.m.

I only recently started learning Haskell and have had a difficult time convincing other Python hackers to come on board. I see two things that might help: 1) A resource to make informed decisions about different libraries. Something that includes specific criteria like how long a library has been out, how often it's maintained, how many people use it, etc. Ideally you'd be able to see a quick table comparison of features across libraries that perform similar tasks (roughly translated to something like: "this xml library is well established, has great documentation, and works for most parsing tasks, while this other one is much faster but not widely used yet"). 2) Languages like Python make it easy to write fast performing code in a few lines that will read/write files, split strings, and build lists or dictionaries/associative arrays. There are very clever ways of doing all these things Haskell, but it can involve several qualified imports and time researching ByteStrings/Lazy ByteStrings/ByteString.Char8. It would be nice to have a single module that exports some common text operations via ByteStrings without requiring a lot of upfront research time learning to work with ByteStrings, and possibly a limited export of Data.Map features as well. The second one would hold little interest for advanced developers of course, but when someone is faced with a difficult learning task, if you give them a strong starting point that produces results it can help motivate them to keep learning. Is anyone working on either of these things or interested in working on them? I'm not quite ready to produce high quality Haskell code yet, but I'd like to contribute if I can. On Thu, May 19, 2011 at 3:42 PM, David Leimbach wrote:

...

See the Haskell Platform.

Sent from my iPhone

On May 19, 2011, at 1:56 PM, Andrew Coppin wrote:

...
On 19/05/2011 09:34 PM, vagif.verdi@gmail.com wrote:

...
Andrew, you are being non constructive.

It seems I'm being misunderstood.

Some people seem to hold the opinion that more libraries = better. I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I'm not trying to say "OMG, the way it is now completely sucks!" I'm not trying to say "you must do X right now!" I'm just trying to put forward an opinion. The opinion that having too many libraries can be a problem, which some people don't seem to agree with. (Obviously it isn't *always* bad, I'm just saying that sometimes it can be.)

That's all I was trying to say.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Evan Laforge

20 May 20 May

12:23 a.m.

...

2) Languages like Python make it easy to write fast performing code in a few lines that will read/write files, split strings, and build lists or dictionaries/associative arrays. There are very clever ways of doing all these things Haskell, but it can involve several qualified imports and time researching ByteStrings/Lazy ByteStrings/ByteString.Char8. It would be nice to have a single module that exports some common text operations via ByteStrings without requiring a lot of upfront research time learning to work with ByteStrings, and possibly a limited export of Data.Map features as well.

It's basically just Data.Text, Data.ByteString, Data.Map, and various things from the prelude. The problem is that once people learn where those things are they forget their own learning process and start thinking its obvious and everyone should know. Maybe a python <-> haskell equivalents cheatsheet could serve the purpose? And you're probably a good person to do it since you haven't forgotten the learning process yet :) You can just register on the wiki and start adding things. Maybe the page even already exists. E.g.: s = open(fn).read() ==> s <- Text.readFile fn '\n'.join(sorted(s.split('\n'))) ==> Text.unlines . List.sort . Text.lines lineByWord = dict(tuple(line.split(':', 1)) for line in lines) ==> lineByWord = Map.fromList [(word, rest) | line <- lines, let (word, rest) = Text.break (==':') line] now = datetime.datetime.now() print '\n'.join((now - datetime.strptime(..., m)).days for m in re.groups(r'...', line)) ==> you tell me :) Or whatever other things you are doing in python that you wish you could do as easily in haskell. If they're used to the above style of python, then there are pretty direct translations for everything. If they are used to a more imperative style then I think some upfront research time is unavoidable, but it'll be good for them.

Yves Parès

3:47 a.m.

...

it can involve several qualified imports and time researching ByteStrings/Lazy ByteStrings/ByteString.Char8

Evan is right, the right way is to use the text package (plus, it is part of the platform and is simple to use), or at least the utf8-string package (encode/decode functions). I personnaly banned ByteString.Char8, as it is incompatible with UTF8 (the "pack" method truncates Chars). 2011/5/20 Eric Rasmussen

...

I only recently started learning Haskell and have had a difficult time convincing other Python hackers to come on board. I see two things that might help:

1) A resource to make informed decisions about different libraries. Something that includes specific criteria like how long a library has been out, how often it's maintained, how many people use it, etc. Ideally you'd be able to see a quick table comparison of features across libraries that perform similar tasks (roughly translated to something like: "this xml library is well established, has great documentation, and works for most parsing tasks, while this other one is much faster but not widely used yet").

2) Languages like Python make it easy to write fast performing code in a few lines that will read/write files, split strings, and build lists or dictionaries/associative arrays. There are very clever ways of doing all these things Haskell, but it can involve several qualified imports and time researching ByteStrings/Lazy ByteStrings/ByteString.Char8. It would be nice to have a single module that exports some common text operations via ByteStrings without requiring a lot of upfront research time learning to work with ByteStrings, and possibly a limited export of Data.Map features as well.

The second one would hold little interest for advanced developers of course, but when someone is faced with a difficult learning task, if you give them a strong starting point that produces results it can help motivate them to keep learning. Is anyone working on either of these things or interested in working on them? I'm not quite ready to produce high quality Haskell code yet, but I'd like to contribute if I can.

On Thu, May 19, 2011 at 3:42 PM, David Leimbach wrote:

...
See the Haskell Platform.

Sent from my iPhone

On May 19, 2011, at 1:56 PM, Andrew Coppin wrote:

...
On 19/05/2011 09:34 PM, vagif.verdi@gmail.com wrote:

...
Andrew, you are being non constructive.

It seems I'm being misunderstood.

Some people seem to hold the opinion that more libraries = better. I'm trying to voice the opinion that there is such a thing as too many libraries. The article I linked to explains part of why this is the case, in a better way than I've been able to phrase it myself.

I'm not trying to say "OMG, the way it is now completely sucks!" I'm not trying to say "you must do X right now!" I'm just trying to put forward an opinion. The opinion that having too many libraries can be a problem, which some people don't seem to agree with. (Obviously it isn't *always* bad, I'm just saying that sometimes it can be.)

That's all I was trying to say.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Stephen Tetley

19 May 19 May

4:50 p.m.

On 19 May 2011 21:20, Andrew Coppin wrote:

...

This is about all those people who think having multiple libraries which only solve half the problem is somehow a "good thing".

Och (number 2) Those people are the Straw Men - you can wave at them from your car window when you pass them as they stand in farmer's fields, though it is not out of rudeness that they do not wave back. Alas they are really made of straw.

Jesper Louis Andersen

20 May 20 May

7:54 a.m.

On Thu, May 19, 2011 at 20:33, Andrew Coppin wrote: Ok, I'll bite.

...

To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

Here is what usually happens: Every library that is *useful* undergoes an iterative process in which is will gradually be improved until a winner emerges. This is like biological evolution and a signal of a healthy community. For a winner to emerge however (like Text in Unicode) - there need to be a clear problem you are trying to solve and the solution must be fairly easy to use. This doesn't always happen if the initial problem is too vague. This is by the way the same model used for the Linux kernel. At some point there was like 3 wifi stacks and 4 USB stacks in the kernel, until people opted to extend the one that was easiest to extend. Haskell's demon are advanced type-level constructions at the library level. If you build a library which is hard to use because you need to understand a complex type-world, then there will be less adoption compared to a simpler library more people understand. Common Lisps demon are complex macros and that you need discipline in the more dynamic typing environment. Also that there is no enforcing of concern separation. On the other hand, a library with a simple type-world may be too simple and not give the guarantees one would like. Hence, for the same problem, you now have justification for the existence of two libraries. My personal gripe is the construction of libraries for the sake of constructing libraries. I like the tools I use to have been used in anger in real projects. That tends to weed out some problems right away because you had an actual user. It also underpins a mantra I learned from the Erlang community: Build tools, not solutions. The library should support your program and not provide a solution. If that is not the case, you are probably better off without the library in the first place. The horror-case I know is Java Apache XML-RPC. It underwent three iterations. v1.x was useful but cumbersome. v2.x improves the interface and is more useful. v3.x is a mistake. The author decided to use reflection, automatic generation and generate interfaces based on classes. That library is very hard to use in practice because there are many things you want to do you can't. It is also a project with a RequestProcessorFactoryFactory class :) -- J.

Henning Thielemann

23 May 23 May

9:12 a.m.

On Thu, 19 May 2011, Andrew Coppin wrote:

...

To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

Recently I searched for an advanced way of handling command-line arguments, and found several packages on Hackage. Unfortunately they are all in different categories, what makes the categories almost useless. The second thing I checked, is the online Haddock documentation in order to see, whether the API makes sense to me. I like to know, how many extensions are required (I prefer Haskell 98) and how clean the package is written, e.g. I think that something simple as command-line parsing should be possible without an unsafe function. In the end I found none of the packages to fit my needs and I stuck to plain GetOpt. Package ratings by certain criterias or by user votes would not help me much. Instead I like to get the information quickly, that helps me deciding, what package fits my needs best. E.g. Hackage could display the extensions used by a package, but it cannot just list the ones listed in the Cabal file, since modules can switch on extensions individually and also the extensions required by imported packages are needed. Maybe Hackage could allow to list or even rank packages according to criteria given by the user. In contrast to that, universal ratings would not help me much, but may frustrate authors, who's packages get bad ratings, because the expectations of its users differ from the ones of the author.

KC

11:31 a.m.

Librarians have been struggling for years with classifying topics; I don't imagine classifying coding libraries as any easier. :) -- -- Regards, KC

Eric Rasmussen

1:08 p.m.

In terms of making the interface more friendly to beginners, I wonder if this is partially an issue of how to search and how to format the results. I just searched several places for "xml rpc" and found: Hackage: the first few links from the google search are different versions of haxr Hayoo: 0 packages found, but lists functions from packages including haxr Hoogle: No results found Haskell.org: No matches When that happens I can broaden my search (just RPC or just XML), or I can go to google and search "haskell xml rpc" and find results, but without any sense of what I should be clicking on. I'll often start in one place, then hear about something on Haskell-Cafe that's more widely used and never came up in the results. I realize Hayoo and Hoogle are specialized searches (although I imagine people do occasionally use them the way I did in this example), but it would be great if Hackage's search feature could provide its own summary results in a simple table, perhaps like this: Date Released | Last Updated | Downloads | Focus Where "focus" is a one or two liner explaining the intended use or scope of the package, ideally written with comparisons to similar packages in mind. That way if you search "xml rpc" and are immediately given results for a couple of packages, you can get some sense of what's current, what's being used, and the scope or intended use of the package. I'm not sure if this format would work for everyone of course, but I do think some of the ideas coming out of this discussion are promising. And for my part, I'm going to make an effort to participate on the wiki once I wrap a couple projects. Best, Eric On Mon, May 23, 2011 at 8:31 AM, KC wrote:

...

Librarians have been struggling for years with classifying topics; I don't imagine classifying coding libraries as any easier. :)

-- -- Regards, KC

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

5112

Age (days ago)

5116

Last active (days ago)

List overview

Download

37 comments

27 participants

participants (27)

aditya siram
Andrew Coppin
austin seipp
Daniel Patterson
Daniel Peebles
David Leimbach
Don Stewart
Eric Rasmussen
Evan Laforge
Felipe Almeida Lessa
Gilberto Garcia
Henning Thielemann
Jesper Louis Andersen
Juan Daugherty
Julian Porter
KC
Ketil Malde
Markus Läll
Paolo G. Giarrusso
Roel van Dijk
Serguey Zefirov
Simon Michael
Stephen Tetley
Tillmann Vogt
vagif.verdi＠gmail.com
Vo Minh Thu
Yves Parès

The Lisp Curse

Andrew Coppin

Andrew Coppin

Gilberto Garcia

Daniel Patterson

Andrew Coppin

Julian Porter

Andrew Coppin

Juan Daugherty

Andrew Coppin

Andrew Coppin

Andrew Coppin

austin seipp

Andrew Coppin

Eric Rasmussen

Yves Parès

Jesper Louis Andersen

Eric Rasmussen

tags

participants (27)