Automatic generation of documentation for HackageDB

Hello, HackageDB [1] now contains automatically generated build logs for some packages, most of them also have automatically generated Haddock documentation. This is, for now, the result of my Google Summer of Code project. The progress of the project can be followed from [2], the sources and some statistics can be found at [3]. For now, 104 packages out of 316 can be successfully processed resulting in proper Haddock documentation and a build log. Processing a packages includes configuring, building, testing and installing, besides running Haddock. Also, Hoogle entries are generated, but not used so far. Note that 31 packages of HackageDB caused errors while building or running Haddock, their build logs are also available at [1]. Whenever a package cannot be build or Haddock returns with an error, all packages depending on that one package won't be processed either. The autodoc tool to automatically process packages will be run regularly (daily, every second day) to mirror modifications of HackageDB. Besides some problems and several missing libraries on the test system (which I am about to fix), there are open questions from my side: How should the generated hoogle entries be used? Should build logs of erroneous packages be mailed to the package maintainer? Are there more feature requests? Concerning the last question, there is already a list of ideas at [4], but maybe more ideas or requests are now arising. During the remaining time, I will work on adding a search function to HackageDB which will allow to find packages based on several different keys drawn from a package's description. A detailed plan will be given at [2] later this week. I am open for any requests or ideas on this subject, as well, of course. Thanks, Sascha [1] http://hackage.haskell.org/packages/archive/pkg-list.html [2] http://hackage.haskell.org/trac/summer-of-code/wiki/SoC2007Hackage [3] http://community.haskell.org/~sboehme/index.html [4] http://hackage.haskell.org/trac/hackage/wiki/HackageToDo

Concerning the last question, there is already a list of ideas at [4], but maybe more ideas or requests are now arising.
Great work Sascha! I'd like to be able to: * recategorise packages online, e.g. we have "Compilers/interpreters" "Code generation" "Composition" "Control" "Monads" "Cryptography" "Codec" "Data" "Data structures" "Foreign" "Language" "Scripting" "Interfaces" "User interfaces" - a first step would be to warn, or prevent people creating new categories. * find all packages that depend on a package (the opposite of the current info about all packages a given package depends on) and use that info to rank by popularity -- Don

On Mon, Jul 23, 2007 at 04:03:15PM +1000, Donald Bruce Stewart wrote:
I'd like to be able to:
* recategorise packages online,
There is a plan to have a plain-text file of attribute-value pairs outside of the package, with both a web interface and server-side program to set attributes. The attributes would be shown on the package page and could be used for searching. One of these attributes could be categories, overriding what's given in the .cabal file. (But there are many other uses too.) Independent of the implementation, there's a need to agree a list of category names.

On Mon, Jul 23, 2007 at 07:55:22AM +0200, Sascha Böhme wrote:
Should build logs of erroneous packages be mailed to the package maintainer?
Definitely. What does it mean if there are no build logs at all, not even failed? Why do the docs for functorm 404, when then build log shows success? Stefan

Hi Sascha, On Mon, Jul 23, 2007 at 07:55:22AM +0200, Sascha Böhme wrote:
HackageDB [1] now contains automatically generated build logs for some packages, most of them also have automatically generated Haddock documentation. This is, for now, the result of my Google Summer of Code project.
Great stuff!
Should build logs of erroneous packages be mailed to the package maintainer?
If it would be one e-mail on upload then I think that's reasonable. We don't want to be automatically sending a mail a day or anything, though.
Are there more feature requests?
Can we have contents/index pages covering (the latest version of) all packages, to replace the link to http://haskell.org/ghc/docs/latest/html/libraries/ on the haskell.org frontpage, please? In case it's useful, here's the script GHC uses to generate it: http://darcs.haskell.org/ghc/libraries/gen_contents_index The hackage version probably doesn't want to have all the header and footer text, as there would be far too much of it. Thanks Ian

igloo:
Hi Sascha,
On Mon, Jul 23, 2007 at 07:55:22AM +0200, Sascha Böhme wrote:
HackageDB [1] now contains automatically generated build logs for some packages, most of them also have automatically generated Haddock documentation. This is, for now, the result of my Google Summer of Code project.
Great stuff!
Should build logs of erroneous packages be mailed to the package maintainer?
If it would be one e-mail on upload then I think that's reasonable. We don't want to be automatically sending a mail a day or anything, though.
Are there more feature requests?
Can we have contents/index pages covering (the latest version of) all packages, to replace the link to http://haskell.org/ghc/docs/latest/html/libraries/ on the haskell.org frontpage, please?
In case it's useful, here's the script GHC uses to generate it: http://darcs.haskell.org/ghc/libraries/gen_contents_index
The hackage version probably doesn't want to have all the header and footer text, as there would be far too much of it.
While we're here, is it possible to get the full log of uploaded packages? I'd like to extract this automatically for the HWN, but currently the rss feed just cuts off at 15 items -- we sometimes get more than 15 items a week, and the HWN is not so weekly anyway. -- Don

Could autodoc use haddock.ghc? If it does, more haddock docs will work.
Also, why do some packages not have a build log? For instance,
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/phooey-1.2 .
- Conal
On 7/22/07, Sascha Böhme
Hello,
HackageDB [1] now contains automatically generated build logs for some packages, most of them also have automatically generated Haddock documentation. This is, for now, the result of my Google Summer of Code project.
The progress of the project can be followed from [2], the sources and some statistics can be found at [3]. For now, 104 packages out of 316 can be successfully processed resulting in proper Haddock documentation and a build log. Processing a packages includes configuring, building, testing and installing, besides running Haddock. Also, Hoogle entries are generated, but not used so far. Note that 31 packages of HackageDB caused errors while building or running Haddock, their build logs are also available at [1]. Whenever a package cannot be build or Haddock returns with an error, all packages depending on that one package won't be processed either.
The autodoc tool to automatically process packages will be run regularly (daily, every second day) to mirror modifications of HackageDB.
Besides some problems and several missing libraries on the test system (which I am about to fix), there are open questions from my side:
How should the generated hoogle entries be used?
Should build logs of erroneous packages be mailed to the package maintainer?
Are there more feature requests?
Concerning the last question, there is already a list of ideas at [4], but maybe more ideas or requests are now arising.
During the remaining time, I will work on adding a search function to HackageDB which will allow to find packages based on several different keys drawn from a package's description. A detailed plan will be given at [2] later this week. I am open for any requests or ideas on this subject, as well, of course.
Thanks, Sascha
[1] http://hackage.haskell.org/packages/archive/pkg-list.html [2] http://hackage.haskell.org/trac/summer-of-code/wiki/SoC2007Hackage [3] http://community.haskell.org/~sboehme/index.html [4] http://hackage.haskell.org/trac/hackage/wiki/HackageToDo
_______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://www.haskell.org/mailman/listinfo/cabal-devel

Could autodoc use haddock.ghc? If it does, more haddock docs will work.
I haven't tried yet. I could do so tomorrow.
Also, why do some packages not have a build log? For instance, http://hackage.haskell.org/cgi-bin/hackage-scripts/package/phooey-1.2 .
There has been a problem in an older version of autodoc. Results of recent runs are published today, and probably updated tomorrow.
On 7/22/07, Sascha Böhme
wrote: Hello,
HackageDB [1] now contains automatically generated build logs for some packages, most of them also have automatically generated Haddock documentation. This is, for now, the result of my Google Summer of Code project.
The progress of the project can be followed from [2], the sources and some statistics can be found at [3]. For now, 104 packages out of 316 can be successfully processed resulting in proper Haddock documentation and a build log. Processing a packages includes configuring, building, testing and installing, besides running Haddock. Also, Hoogle entries are generated, but not used so far. Note that 31 packages of HackageDB caused errors while building or running Haddock, their build logs are also available at [1]. Whenever a package cannot be build or Haddock returns with an error, all packages depending on that one package won't be processed either.
The autodoc tool to automatically process packages will be run regularly (daily, every second day) to mirror modifications of HackageDB.
Besides some problems and several missing libraries on the test system (which I am about to fix), there are open questions from my side:
How should the generated hoogle entries be used?
Should build logs of erroneous packages be mailed to the package maintainer?
Are there more feature requests?
Concerning the last question, there is already a list of ideas at [4], but maybe more ideas or requests are now arising.
During the remaining time, I will work on adding a search function to HackageDB which will allow to find packages based on several different keys drawn from a package's description. A detailed plan will be given at [2] later this week. I am open for any requests or ideas on this subject, as well, of course.
Thanks, Sascha
[1] http://hackage.haskell.org/packages/archive/pkg-list.html [2] http://hackage.haskell.org/trac/summer-of-code/wiki/SoC2007Hackage [3] http://community.haskell.org/~sboehme/index.html [4] http://hackage.haskell.org/trac/hackage/wiki/HackageToDo
_______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://www.haskell.org/mailman/listinfo/cabal-devel
-- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer
participants (7)
-
"Sascha Böhme"
-
Conal Elliott
-
dons@cse.unsw.edu.au
-
Ian Lynagh
-
Ross Paterson
-
Sascha Böhme
-
Stefan O'Rear