On 2007-07-30, Dave Bayer wrote:

My pet example is a PDF library. No language should have its own PDF library, when Postscript is so easy to write, and "Ghostscript" is a cross-platform conversion tool maintained by thousands of our best and brightest.

Except, of course, that there are PDF features that are not easily used by converting from PostScript. And then there's reading and parsing PDF... -- Aaron Denney -><-

On Mon, Jul 30, 2007 at 05:27:21PM +0100, Duncan Coutts wrote:

We have tools to solve the downloading and installing all deps problem. It's called cabal-install. It's sort-of almost ready for wider testing.

duncan - will this have an interactive prompt? i have found "perl -MCPAN -e shell" immensely useful over the years thanks

| On the other hand, it's not entirely true that there's no standard | library, it's just that it's borders are slightly fuzzy. For example, we | do have the library change submission process for modifying the standard | libraries. Up until now that has been taken to mean changes to the base | package. That package is now being split up, so we'll have to think | about what it'll apply to in the future. | | My opinion is that in the past it has been too difficult to get changes | into the base library, that there's been too much stability at the | expense of improving scope and quality. Making it easy to install new | packages and upgrade existing standard libraries should make it easier | to trial more major changes outside of the standard libs before | proposing getting those changes integrated. All true, but not so helpful for Joe User. For Joe, I think it might be helpful to have some easily-discoverable notion of which package quality and stability. - Package X is "blessed"; lots of people have argued over its design, it's stable, widely used, and actively maintained. Changes to this package goes through a quality-control process. - Package Y is a bit specialised, but it's the result of work by a small group, and it's actively maintained. - Package Z is designed, written, and maintained by one person. That person has kindly put it on Hackage so that others may share it, but you probably don't want to rely on it unless you are happy to help maintain it. Then, in effect, the "standard library" is all the X packages. I wonder if it'd help to have some descriptions such as those above (better worded), and use them? Cabal already has a "stability" indication, and that might serve, but we'd want to articulate much more clearly what it meant. Simon

On Tue, 2007-07-31 at 10:15 +0100, Simon Peyton-Jones wrote:

All true, but not so helpful for Joe User. For Joe, I think it might be helpful to have some easily-discoverable notion of which package quality and stability.

- Package X is "blessed"; lots of people have argued over its design, it's stable, widely used, and actively maintained. Changes to this package goes through a quality-control process.

- Package Y is a bit specialised, but it's the result of work by a small group, and it's actively maintained.

- Package Z is designed, written, and maintained by one person. That person has kindly put it on Hackage so that others may share it, but you probably don't want to rely on it unless you are happy to help maintain it.

Then, in effect, the "standard library" is all the X packages.

Yes.

I wonder if it'd help to have some descriptions such as those above (better worded), and use them? Cabal already has a "stability" indication, and that might serve, but we'd want to articulate much more clearly what it meant.

I'm not sure that belongs in the cabal file, afterall, being "blessed" is a central community consensus thing, not a distributed decision taken by each person writing the cabal file for their package. I can't make a blessed package by just saying that it is so. So it's clear at the moment that the base package is blessed, changes to it go through the library submissions process. It's not so clear for the other packages that ghc has distributed and have often been taken to be the standard library. Many of them look more like Y's above (like parsec, regex-*). So yes, I think we should make this clear, and that blessed packages that are covered by the library submission process should be clearly recorded and publicised centrally. Even then though, I think Chris was looking for something slightly wider. For example ghc has distributed quite a range of packages that would probably not be classified as X above, eg OpenGL, GLUT, OpenAL, FGL, HGL, etc. These are not necessarily blessed packages but are known to be of a high quality (ok, except HGL). Chris wanted to know this to distinguish from the many other packages on hackage. What is not clear to me yet is if we should just rely on mechanisms in hackage to distinguish the gems from the failed experiments or something more centralised. Duncan

On Tue, Jul 31, 2007 at 09:16:33AM -0600, Chris Smith wrote:

If there could be built-in quality control in promoting certain packages, that would be great.

it needs to be more fine grained. a new version of a package may indeed rollback some positive attributes (stability for example) that a previous version demonstrated...perhaps intentionally (when an author is choosing to break an api, etc), perhaps not (plain old bugs) we already have "quality claims" of two kinds for hackage packages: implicit (version number, 0.* indicating lack of maturity) and explicit (stability: experimental, stable, etc). allowing two scores to be maintained for "stability" - author score AND audience score, seems like a good way of moderating claims. simply allow people with haskell.org accounts to select a pulldown in the package listing with options for the stability score, with obvious safety features (one vote per account per package version, etc)

On Tue, Jul 31, 2007 at 05:26:31PM +0100, Simon Peyton-Jones wrote:

| I see it as a really big deal that documentation becomes fragmented when | one is using many packages, so that it's harder to find what you want. | In fact, I'd classify that as the single biggest reason that I don't use | many packages now

When you install packages A,B,C, the documentation for A,B,C (and nothing else) ought to be locally available as an integrated whole, much as at the GHC web site. I don't know whether Cabal does, or could do, that, but it's surely what one would expect.

I don't think that would be terribly hard. We would need to modify Haddock with the ability to generate links following a schema like /usr/share/doc/libghc6-$lowercasepackagename/html/$modulepath.html; this would be fairly easy with Haddock-GHC's ability to access package names and ghc-pkg haddock data, assuming Cabal and Haddock-GHC are fixed to work on the same system... Stefan

Duncan Coutts wrote:

What is missing from the local docs is a single integrated index page that lists all the modules and then links off to the various packages's docs like we have on the ghc website.

The problem with generating one of those is what manages it? What package would it belong to etc.

Locally, I've kludged things together to add to the documentation package that GHC builds. That may be the wrong place, but it kinda works anyway. This script gets you much of the way there (with some unfortunate line wrapping at the end that you'd have to fix). Of course, the script does more than just build haddock; and there are several other quirks here to that were needed to get random stuff to work for some packages, and unfortunately there are a number of packages for which it seems that 'runhaskell Setup haddock' just doesn't work at all due to use of features in the source that haddock can't parse. What it doesn't do is fix up the links to contents and index from the other packages so that they point back to the right place. -- begin attached script -- #!/bin/sh ghcver=`ls -d /usr/local/lib/ghc-* | sort` ghcver=`expr match "$ghcver" '.*\(ghc-6.7.[0-9]*\)'` sudo rm /usr/local/lib/${ghcver}/share sudo ln -s /usr/local/share /usr/local/lib/${ghcver}/share sudo cp ../ghc/libraries/libraries-*.txt /usr/local/share/ghc/doc/html for ln in `cat packages.list` do d=${ln:0:1} p=${ln:1} echo =============================================================== echo == BUILDING: $p echo =============================================================== echo $d $p cd $p || exit 1 if [ -d _darcs ] then darcs pull || exit 1 fi if [ -f configure.in -o -f configure.ac ] then autoreconf || exit 1 fi if [ -f Setup.hs -o -f Setup.lhs ] then runhaskell Setup clean || exit 1 runhaskell Setup configure || exit 1 runhaskell Setup build || exit 1 if [ $d = "+" ] then runhaskell Setup haddock --html-location=/usr/local/share/ghc \ || exit 1 fi sudo runhaskell Setup install || exit 1 elif [ -f Makefile -o -f Makefile.in -o -f Makefile.am ] then if [ $d = "+" ] then echo "Don't know how to run haddock" exit 1 fi make distclean || true ./configure || exit 1 make || exit 1 sudo make install || exit 1 else echo "Don't know how to make $p" exit 1 fi cd .. done ls /usr/local/share/*/doc/html/*/haddock.css \ | grep -v '/usr/local/share/ghc' \ | sed 's/\(\/usr\/local\/share\/.*\/doc\/html\/\([^/]*\)\) \/haddock.css/cp -r \1 \/usr\/local\/share\/ghc\/doc\/html\/\2 ; echo \2

\/usr\/local\/share\/ghc\/doc\/html\/\2\/prologue.txt/' \ | sudo /bin/sh ls ../ghc/libraries/*/prologue.txt \ | sed 's/\(\.\.\/ghc\/libraries\/\([^\/]*\)\/prologue.txt\)/cp \1 \/usr\/local\/share\/ghc\/doc\/html\/\2/' \ | sudo /bin/sh cd /usr/local/share/ghc/doc/html sudo ./gen_contents_index

-- Chris Smith

On Tuesday 31 July 2007 19:39, Duncan Coutts wrote:

[...] The docs for those packages would be available for packages installed via cabal (assuming the user did the optional haddock step) and would link to each other.

Well, on a normal Linux distro a user should *never* have to call cabal (or any of its cousins) directly, the distro's package manager should be the used instead. On an e.g. RPM system, the .spec file would use Cabal to e.g. (un-)register a package, because RPM has to know what is installed, which other packages are prerequisites, how to cleanly uninstall, etc. IMHO Cabal should not try to mirror a full-fledged package system, simply because on every (non-Windows ;-) platform there are tons of native tools for this purpose, and Cabal is not "in the driver's seat" when it comes to SW installation.

What is missing from the local docs is a single integrated index page that lists all the modules and then links off to the various packages's docs like we have on the ghc website.

The problem with generating one of those is what manages it? What package would it belong to etc.

Of course we are not the first project to face this kind of problem: Texinfo offers a central contents page as well. To maintain this page, it comes with a tool "install-info", which updates the index page after (de-)installation. On RPM systems, the .spec file calls install-info after (de-)installation of a package with info pages. http://www.gnu.org/software/texinfo/manual/texinfo/html_node/Installing-an-I...

On some systems (windows, gnome) there are dedicated help viewers that can help with this contents/index issue. haddock supports both (mshelp, devhelp). I'm not sure everyone would find that a sufficient solution however.

A "install-haddock" tool would be the solution IMHO. Cheers, S.

Sven Panne wrote:

On Tuesday 31 July 2007 19:39, Duncan Coutts wrote:

...

[...] The docs for those packages would be available for packages installed via cabal (assuming the user did the optional haddock step) and would link to each other.

Well, on a normal Linux distro a user should *never* have to call cabal (or any of its cousins) directly, the distro's package manager should be the used instead.

This is very theoretical. I use debian (stable) and have to install non-deb Haskell libraries all the time. No way distro package maintainers can provide packages for each and every library out there, and even for 'standard' libs (whatever that may mean) sometimes you need a newer or an older version of a certain library (relative to what the distro offers).

...

On some systems (windows, gnome) there are dedicated help viewers that can help with this contents/index issue. haddock supports both (mshelp, devhelp). I'm not sure everyone would find that a sufficient solution however.

A "install-haddock" tool would be the solution IMHO.

Yes. Cheers Ben

On Sat, 2007-09-01 at 18:47 +0200, Sven Panne wrote:

On Tuesday 31 July 2007 19:39, Duncan Coutts wrote:

...

[...] The docs for those packages would be available for packages installed via cabal (assuming the user did the optional haddock step) and would link to each other.

Well, on a normal Linux distro a user should *never* have to call cabal (or any of its cousins) directly, the distro's package manager should be the used instead. On an e.g. RPM system, the .spec file would use Cabal to e.g. (un-)register a package, because RPM has to know what is installed, which other packages are prerequisites, how to cleanly uninstall, etc. IMHO Cabal should not try to mirror a full-fledged package system, simply because on every (non-Windows ;-) platform there are tons of native tools for this purpose, and Cabal is not "in the driver's seat" when it comes to SW installation.

I think it's inevitable that there will always be a mixture of packages that are managed by the system package manager and ones that are too insignificant to be packaged by the distro. So cabal-install should cooperate with the system package manager somehow. Another strategy would be to have tools that the users can use to generate system packages from cabal packages and then install those via the system package manager. We already have such tools for rpm and gentoo ebuilds. Again, these would be for the case of less significant package that the distro does not package itself. For example, gentoo has a tool that can be used to install perl CPAN packages via the system package manager, since there are many 1000's of CPAN packages and only a few hundred of those are included in the main portage collection. There are other cases not covered by system package managers, like unprivileged user installations under $HOME.

...

The problem with generating one of those is what manages it? What package would it belong to etc.

Of course we are not the first project to face this kind of problem: Texinfo offers a central contents page as well. To maintain this page, it comes with

[..]

A "install-haddock" tool would be the solution IMHO.

That re-generates the index page, right. Perhaps haddock itself should be extended with this ability. All it should need to do is read all the .haddock files that ghc-pkg knows about and generate the index page from that. I assume the .haddock files contain enough information to do this, or it could be modified to include enough. Duncan

On Sunday 02 September 2007 03:29, Hugh Perkins wrote:

A really simple way to track the "quality" of a package is to display the number of downloads.

A posteriorae, this works great in other download sites.

We can easily hypothesize about why a download count gives a decent indication of some measure of quality: [...]

... and even more easily hypothesize why this is not always a good indication: High-qualitiy standard libraries which are packaged with GHC/Hugs/... will probably almost never be downloaded separately. Cheers, S.

Sven Panne wrote:

... and even more easily hypothesize why this is not always a good indication: High-qualitiy standard libraries which are packaged with GHC/Hugs/... will probably almost never be downloaded separately.

Solution: change GHC/Hugs so it submits usage counters of which libraries the user "imports" to a webservice, stored in a database ;-) Of course this is subject to the user accepting such a feedback, to protect it's privacy. Google does this all the time!

Hugh Perkins wrote:

A really simple way to track the "quality" of a package is to display the number of downloads.

A posteriorae, this works great in other download sites.

We can easily hypothesize about why a download count gives a decent indication of some measure of quality: - more people downloading it means more people specifically wanted that package - more people downloading it means more people trying it, giving feedback, maybe giving patches - and, of course, it's an objective measure, easy to do

I don't agree. The idea that any arbitrary "objective" measure provides any indication quality seems quite wrong to me, especially this one. The problem is the effect of positive feedback. The popularity of MS Winders or Office Suite are the obvious examples. We all know why these are used on 95% or whatever of the worlds PCs, and it has nothing whatever to do with quality. Or a little closer to home, the popularity of Haskell vs. Clean. Other meaningless measures that have been suggested are the rate of patch submissions of the number of developers involved. I seem to remember someone recently suggesting that libraries that score highly in on this regard should be elevated to "blessed" status. I cynic like me could just as well regard this as an indication of the complete opposite, that the library was being developed by an uncoordinated troop of barely competent code monkeys desperately trying to produce something that works reliably by a process of trial and error :-) Personally the first things I tend to look at are things like the quality of documentation and the presence of of some kind of test suite. Both these are IMO opinion pretty reliable indications that the author(s) have actually devoted some time and effort into deciding what it is that the library aims to achieve and have designed a coherent API (and have made reasonable effort to ensure that it actually works). I tend lose interest pretty fast if even basic Haddock API documentation is either non-existant, or consists of nothing but type signatures, or that plus broken link to some ancient postscript paper. Regards -- Adrian Hey

On 9/3/07, Adrian Hey wrote:

The popularity of MS Winders or Office Suite are the obvious examples. We all know why these are used on 95% or whatever of the worlds PCs, and it has nothing whatever to do with quality.

Oh come on. You've been reading waaayyy too much Slashdot...

On 9/3/07, Adrian Hey wrote:

FYI, I am old enough to actually remember life before MS and I can also remember what's happened to the industry at large and to various the organisations I've worked in and had dealings with over the last 25 years or so.

Fair enough.