Gentle Haskellers Haskell has now got a decent, hierarchical module system, supported by GHC, Hugs and NHC, and the library structure is now quite well populated. For example, here are the "base" libraries shipped with GHC (similar libraries come with Hugs and Nhc): http://www.haskell.org/ghc/docs/latest/html/base/index.html Languages flourish when libraries are plentiful, reliable, and well documented. In an open-source community like the Haskell community, plentiful libraries depend on contributions from the community. The difficulty is that it is currently not obvious what Joe Programmer should do to contribute a library to the community. I think this difficulty is a major obstacle to Haskell's progress, perhaps even the largest single obstacle today. One route to getting a library widely distributed is to get it into the "libraries" source tree used by GHC, Hugs, and NHC. That has the merit that the library will be distributed with each of these compilers. But it's too much of a "cathedral" approach. There is no way to distribute libraries between releases, and it makes the distributions ever larger and more complex for the GHC/Hugs/Nhc teams to build. To complement this, there ought to be an easy "bazaar" way for Joe Programmer to implement a library, build distributions, put them up on his web site for others to download, and add a link to some central library index so that people can find it. Sadly this is not an entirely simple task, because (a) the library should work on all Haskell implementations, and (b) should be packaged for binary installation on a variety of platforms (Windows, Linux, OpenBSD, Mac etc). Indeed, since GHC, Hugs and Nhc have different packaging mechanisms, there's a kind of (a)*(b) product involved. The bottom line is that today the obstacles are high enough to discourage all but the most motivated. What is needed is some kind of software infrastructure to automate the common tasks, and to give clear instructions about the non-automatable parts. This infrastructure should not be part of the (complicated) GHC (or Hugs, or Nhc) build systems; indeed, while it will certainly contain implementation-specific parts, its structure should be implementation independent. Simon Marlow emitted a message a while ago about this, but discussion petered out: http://www.haskell.org/pipermail/glasgow-haskell-users/2002-November/thr ead.html#4322 The lack of this library-building infrastructure is a serious problem. Solving it, or even partially solving it, would be a huge step forward. So this message is really to try to kick-start the process again. Is anyone willing to invest effort in making this happen? It's probably a task that is too big for one person. I think what we need is A library-infrastructure Czar, who will a) do the core design and implementation b) coordinate the whole thing to make sure progress happens A bunch of library-infrastructure Supporters, who are willing a) to contribute thoughtfully to the design b) to implement pieces of it, for example pieces specific to one implementation (e.g. GHC) or one target architecture (e.g. PowerPC) Perhaps the first thing to do is to assemble list of people who would be willing to contribute as supporters (in the above sense), hoping that a Czar may then emerge by popular acclaim. Would you like to reply, perhaps to the 'libraries@haskell.org' list, so indicate interest. I really think this is important. If we do not collectively evolve a way to develop libraries in "bazaar" mode, I think Haskell will at best develop more slowly, and at worst wither away. Simon PS1: This is all distinct from the question of library *content* --- that is, what the module name, discussion about the signature of the exported types and functions are --- which does seem be handled OK by the 'libraries' list. This message concerns library-building *infrastructure*. PS2: Another entirely-orthogonal question is where library source code *lives*. One possibility is to use the same CVS repository that GHC/Hugs/Nhc use for libraries, albeit in a separate subtree. Perhaps fptools/user-libs/joe-bloggs/*. Another possibility is SourceForge. Finding backed-up CVS space for source code is not a problem.
"Simon Peyton-Jones"
Languages flourish when libraries are plentiful, reliable, and well documented. In an open-source community like the Haskell community, plentiful libraries depend on contributions from the community.
The difficulty is that it is currently not obvious what Joe Programmer should do to contribute a library to the community.
I think this difficulty is a major obstacle to Haskell's progress, perhaps even the largest single obstacle today.
http://sf.net/projects/haskell-libs/ was created as a place to create, collect and maintain explicitly licensed Open Source libraries for/in Haskell. haskell-libs now has: * irc 'bot ( lambdabot ) * web server with plugins ( hws-wp, derivative of Simon Marlow's hws ) * crypto modules ( Blowfish, RC4, TEA, ISAAC, crc32 ) * rfc2822 parser * rudimentary maildir search engine (lambdabot plugin) * rudimentary bayesian spam filter * chess game * brainfuck interpreter I'd be happy to add haskell developers to the haskell-libs project, just email me or talk to me on the #haskell irc channel.
The lack of this library-building infrastructure is a serious problem. Solving it, or even partially solving it, would be a huge step forward. So this message is really to try to kick-start the process again. Is anyone willing to invest effort in making this happen?
Perhaps the first thing to do is to assemble list of people who would be willing to contribute as supporters (in the above sense), hoping that a Czar may then emerge by popular acclaim. Would you like to reply, perhaps to the 'libraries@haskell.org' list, so indicate interest.
I really think this is important. If we do not collectively evolve a way to develop libraries in "bazaar" mode, I think Haskell will at best develop more slowly, and at worst wither away.
I agree with this, and I'd like to contribute. PS. does anyone know how to contact Jan Skibinski from numeric-quest.com? Jan's haskell sources don't have a license, and I'd like to add them to the haskell-libs CVS tree. -- Shae Matijs Erisson - 2 days older than RFC0226 #haskell on irc.freenode.net - We Put the Funk in Funktion 10 PRINT "HELLO" 20 GOTO 10 ; putStr $ fix ("HELLO\n"++)
I think one way to look at this is to examine similar ideas for other systems that work well. The FreeBSD ports tree is very successful in bringing in a diverse set of libraries and programs. Dependencies and builds are pretty much handled automagically through some (rather complicated) Makefiles. They have also provided a fairly nice web interface for browsing and searching the ports tree. Contributions and updates are handled through CVS. Conceivably, this could handle issues like building and installing libraries for each installed compiler, and eliminate the need to distribute binaries. I think that building and including libraries has to be standardized across the different compilers. One big benefit of using unix is that the C suite is basically consistent across platforms. Other random thoughts... I think that in addition to the Report and Tutorial, a Best Practices document needs to be developed. This would outline the preferable naming conventions, indentation, make targets, and also recommend the use of tools like Haddock of HDoc for generating consistent documentation. I am willing to use my DSP library as a guinea pig for any ideas (it is still a little immature, though), and to help out where I can. -- Matthew Donadio (m.p.donadio@ieee.org)
Don't forget three others: CPAN, CTAN, and SourceForge
Each of these is a distribution model with, I think, CPAN/CTAN being more
accessable than SourceForge because of thier categorization scheme.
Furthermore, CPAN lets you download using the CPAN.pm module which is very
nice: for example, it will download dependent modules as well as requested
ones if you don't have them.
You can further compare those schemes with something like the boost
libraries for C++. These libraries are submitted to a rigourous inclusion
process that weeds out most of the weaker libraries (or people). I actually
would not recommend this approach as it seems to cut down on the actual
usefulness of the library because it often requires the library to be too
general to be useful.
HTH,
Tanton
----- Original Message -----
From: "Matthew Donadio"
I think one way to look at this is to examine similar ideas for other systems that work well.
The FreeBSD ports tree is very successful in bringing in a diverse set of libraries and programs. Dependencies and builds are pretty much handled automagically through some (rather complicated) Makefiles. They have also provided a fairly nice web interface for browsing and searching the ports tree. Contributions and updates are handled through CVS. Conceivably, this could handle issues like building and installing libraries for each installed compiler, and eliminate the need to distribute binaries.
I think that building and including libraries has to be standardized across the different compilers. One big benefit of using unix is that the C suite is basically consistent across platforms.
Other random thoughts...
I think that in addition to the Report and Tutorial, a Best Practices document needs to be developed. This would outline the preferable naming conventions, indentation, make targets, and also recommend the use of tools like Haddock of HDoc for generating consistent documentation.
I am willing to use my DSP library as a guinea pig for any ideas (it is still a little immature, though), and to help out where I can.
-- Matthew Donadio (m.p.donadio@ieee.org) _______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
Tanton Gibbs wrote:
Don't forget three others: CPAN, CTAN, and SourceForge
Yes, they are nice, but they dosn't totally solve the building process (though I could be worng about CPAN; I try to ignore perl :) ). The ports tree handles this. For those unfamiliar with the BSD ports tree, here is how it would be handled in the Haskell world. You would have a set of directories in the same tree, lets say /usr/haskell/ Libraries would live in this tree, and you would group them into rough categories. There would also be a directory with Makefile includes. Each library would include a Makefile to build the library. These Makefiles only define download sites, dependencies, tool requirements, and limitations. They include other Makefiles to do the real work. There would probably be ghc.mk, nhc.mk, and hugs.mk with compiler specific things, and also hdoc.mk, haddock.mk, etc, for tools. There would be a master Makefile that would detect which compilers are installed, and build library versions for each. This master makefile would also handle tasks like downloading the source, applying patches, etc. In the BSD world, this works very well. You can install the minimal OS on a machine, cd to /usr/ports/editors/AbiWord2, type make install, and have it download, build, and install _everything_ needed (GNU make, X11, Gnome, etc) to build AbiWord for Gnome2. Maybe I will create a quick mock up. Another way to do this is to standardize a library format (analygous to Java JARs), and to extend the build process in hmake (or create a hlibtool), so you could do something like hmake -ghc -installlib http://some.site/some/path/somelib.tar.gz To automate everything. Just some caffine inspired thoughts... -- Matthew Donadio (m.p.donadio@ieee.org)
Some comments,
The BSD ports system:
1. Is intended for a completely different purpose and
2. Has far too many bugs and problems.
On #2, I occasionally talk to people who say that the FBSD ports system works for them, but my own experience, and the experience of most of the people I've talked to about it, is that the quality is very variable and many packages simply refuse to install. I did a quick test here and had problems with about 20% of the packages I tried. Other similar systems such as the ports for Net and OpenBSD and emerge all have problems, in varying degrees but in my mind excessive.
Going back to #1, the purpose of systems like BSD ports is to handle the downloading, installation, build, and configuration of packages for an operating system. How much of this is relevant to a language library?
One primary purpose of these systems is to make it possible for less sophisticated users to install and use O/S packages. This certainly does not apply to users of a complex language such as Haskell, and those in the Haskell development community are not novices who need black box tools to hide the complexity of the software infrastructure.
By including automatic downloading, these systems seriously restrict your options as to format and configuration of download sites. I think that is a restriction that is unwise for the situation under discussion.
I most certainly do _not_ want a tool that I invoke to install a library to, on the basis of automated dependencies, update other libraries to a later version (or, worse, downgrade to an earlier version).
On Tue, 22 Apr 2003 13:05:43 -0400
Matthew Donadio
Tanton Gibbs wrote:
Don't forget three others: CPAN, CTAN, and SourceForge
Yes, they are nice, but they dosn't totally solve the building process (though I could be worng about CPAN; I try to ignore perl :) ). The ports tree handles this.
For those unfamiliar with the BSD ports tree, here is how it would be handled in the Haskell world.
You would have a set of directories in the same tree, lets say /usr/haskell/
Libraries would live in this tree, and you would group them into rough categories.
There would also be a directory with Makefile includes.
Each library would include a Makefile to build the library. These Makefiles only define download sites, dependencies, tool requirements, and limitations. They include other Makefiles to do the real work.
There would probably be ghc.mk, nhc.mk, and hugs.mk with compiler specific things, and also hdoc.mk, haddock.mk, etc, for tools. There would be a master Makefile that would detect which compilers are installed, and build library versions for each. This master makefile would also handle tasks like downloading the source, applying patches, etc.
In the BSD world, this works very well. You can install the minimal OS on a machine, cd to /usr/ports/editors/AbiWord2, type make install, and have it download, build, and install _everything_ needed (GNU make, X11, Gnome, etc) to build AbiWord for Gnome2.
Maybe I will create a quick mock up.
Another way to do this is to standardize a library format (analygous to Java JARs), and to extend the build process in hmake (or create a hlibtool), so you could do something like
hmake -ghc -installlib http://some.site/some/path/somelib.tar.gz
To automate everything.
Just some caffine inspired thoughts...
-- Matthew Donadio (m.p.donadio@ieee.org) _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
-- Seth Kurtzberg MIS Corp 480-661-1849 seth@cql.com
G'day all. On Tue, Apr 22, 2003 at 01:05:43PM -0400, Matthew Donadio wrote:
Yes, they are nice, but they dosn't totally solve the building process (though I could be worng about CPAN; I try to ignore perl :) ).
Not true. So long as your C code (in the case of CPAN) is portable, everything else just works for both CPAN and CTAN. The catch is that code in CTAN and CPAN only need work on one implementation (TeX/LaTeX and Perl respectively). It helps that both LaTeX and Perl have ways to specify a "minimum version required".
The ports tree handles this.
The ports tree similarly works because it is only required to work with one OS and compiler. Cheers, Andrew Bromage
Matthew Donadio writes:
I think that in addition to the Report and Tutorial, a Best Practices document needs to be developed. This would outline the preferable naming conventions, indentation, make targets, and also recommend the use of tools like Haddock of HDoc for generating consistent documentation.
Funnily enough, I added a HaskellStyle page to the wiki just yesterday. Please start filling it in with this kind of information.
http://www.haskell.org/hawiki/HaskellStyle
--KW 8-)
--
Keith Wansbrough
Matthew Donadio writes:
I think one way to look at this is to examine similar ideas for other systems that work well.
As a suggestion, I just got done updating cygwin, and I must admit I admire the setup there. You can indicate what packages you are downloading, which you want to keep, which you want to skip on a menu like structure. Once they are downloaded, automatic scripts take care of any building that is necessary. It does just what I think is necessary in updating programs. For those on your system which are already present, it updates automatically, only if necessary (if there has been a version change). Can we just grab this code and update it for our purposes? Dave Barton EDAptive Computing
Simon,
I think this is an important issue and I am willing to help and participate.
The issues for such a system are I believe more complex than is immediately apparent, and some of the early responses indicate that the complexity of the issue may not be well understood.
I would recommend a more incremental approach with more limited initial goals. Specifically, with X meaning to be determined:
1. A Haskell library is distributed in X format
2. After downloading, a Haskell library is built using procedure X. All libraries are built using the same sequence. Preferably the sequence is the same for the three compilers, and the build process is smart enough to determine which compiler is available, and adjust accordingly. There should be a build time option to override the automatic compiler choice.
3. A Haskell library is installed into directory (tree?) X. Again, preferably compiler independent, if practical.
4. A Haskell library includes, at a minimum, documentation X in format Y. This is the least interesting (at least to many developers :) ), but the _most_ important, requirement.
On Tue, 22 Apr 2003 15:00:39 +0100
"Simon Peyton-Jones"
Gentle Haskellers
Haskell has now got a decent, hierarchical module system, supported by GHC, Hugs and NHC, and the library structure is now quite well populated. For example, here are the "base" libraries shipped with GHC (similar libraries come with Hugs and Nhc): http://www.haskell.org/ghc/docs/latest/html/base/index.html
Languages flourish when libraries are plentiful, reliable, and well documented. In an open-source community like the Haskell community, plentiful libraries depend on contributions from the community.
The difficulty is that it is currently not obvious what Joe Programmer should do to contribute a library to the community.
I think this difficulty is a major obstacle to Haskell's progress, perhaps even the largest single obstacle today.
One route to getting a library widely distributed is to get it into the "libraries" source tree used by GHC, Hugs, and NHC. That has the merit that the library will be distributed with each of these compilers. But it's too much of a "cathedral" approach. There is no way to distribute libraries between releases, and it makes the distributions ever larger and more complex for the GHC/Hugs/Nhc teams to build.
To complement this, there ought to be an easy "bazaar" way for Joe Programmer to implement a library, build distributions, put them up on his web site for others to download, and add a link to some central library index so that people can find it. Sadly this is not an entirely simple task, because (a) the library should work on all Haskell implementations, and (b) should be packaged for binary installation on a variety of platforms (Windows, Linux, OpenBSD, Mac etc). Indeed, since GHC, Hugs and Nhc have different packaging mechanisms, there's a kind of (a)*(b) product involved. The bottom line is that today the obstacles are high enough to discourage all but the most motivated.
What is needed is some kind of software infrastructure to automate the common tasks, and to give clear instructions about the non-automatable parts. This infrastructure should not be part of the (complicated) GHC (or Hugs, or Nhc) build systems; indeed, while it will certainly contain implementation-specific parts, its structure should be implementation independent. Simon Marlow emitted a message a while ago about this, but discussion petered out:
http://www.haskell.org/pipermail/glasgow-haskell-users/2002-November/thr ead.html#4322
The lack of this library-building infrastructure is a serious problem. Solving it, or even partially solving it, would be a huge step forward. So this message is really to try to kick-start the process again. Is anyone willing to invest effort in making this happen?
It's probably a task that is too big for one person. I think what we need is
A library-infrastructure Czar, who will a) do the core design and implementation b) coordinate the whole thing to make sure progress happens
A bunch of library-infrastructure Supporters, who are willing a) to contribute thoughtfully to the design b) to implement pieces of it, for example pieces specific to one implementation (e.g. GHC) or one target architecture (e.g. PowerPC)
Perhaps the first thing to do is to assemble list of people who would be willing to contribute as supporters (in the above sense), hoping that a Czar may then emerge by popular acclaim. Would you like to reply, perhaps to the 'libraries@haskell.org' list, so indicate interest.
I really think this is important. If we do not collectively evolve a way to develop libraries in "bazaar" mode, I think Haskell will at best develop more slowly, and at worst wither away.
Simon
PS1: This is all distinct from the question of library *content* --- that is, what the module name, discussion about the signature of the exported types and functions are --- which does seem be handled OK by the 'libraries' list. This message concerns library-building *infrastructure*.
PS2: Another entirely-orthogonal question is where library source code *lives*. One possibility is to use the same CVS repository that GHC/Hugs/Nhc use for libraries, albeit in a separate subtree. Perhaps fptools/user-libs/joe-bloggs/*. Another possibility is SourceForge. Finding backed-up CVS space for source code is not a problem.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
-- Seth Kurtzberg MIS Corp 480-661-1849 seth@cql.com
2003年04月22日(火)の23時00分に Simon Peyton-Jones 曰く:
Perhaps the first thing to do is to assemble list of people who would be willing to contribute as supporters (in the above sense), hoping that a Czar may then emerge by popular acclaim. Would you like to reply, perhaps to the 'libraries@haskell.org' list, so indicate interest.
Yes, I would like to participate. Perhaps I can help with rpm package issues and such.
I really think this is important. If we do not collectively evolve a way to develop libraries in "bazaar" mode, I think Haskell will at best develop more slowly, and at worst wither away.
Agreed. Jens
participants (9)
-
Andrew J Bromage -
David Barton -
Jens Petersen -
Keith Wansbrough -
Matthew Donadio -
Seth Kurtzberg -
Shae Matijs Erisson -
Simon Peyton-Jones -
Tanton Gibbs