July 2007 - cabal-devel

Automatic generation of documentation for HackageDB
by Sascha Böhme 26 Aug '07

26 Aug '07

Hello, HackageDB [1] now contains automatically generated build logs for some packages, most of them also have automatically generated Haddock documentation. This is, for now, the result of my Google Summer of Code project. The progress of the project can be followed from [2], the sources and some statistics can be found at [3]. For now, 104 packages out of 316 can be successfully processed resulting in proper Haddock documentation and a build log. Processing a packages includes configuring, building, testing and installing, besides running Haddock. Also, Hoogle entries are generated, but not used so far. Note that 31 packages of HackageDB caused errors while building or running Haddock, their build logs are also available at [1]. Whenever a package cannot be build or Haddock returns with an error, all packages depending on that one package won't be processed either. The autodoc tool to automatically process packages will be run regularly (daily, every second day) to mirror modifications of HackageDB. Besides some problems and several missing libraries on the test system (which I am about to fix), there are open questions from my side: How should the generated hoogle entries be used? Should build logs of erroneous packages be mailed to the package maintainer? Are there more feature requests? Concerning the last question, there is already a list of ideas at [4], but maybe more ideas or requests are now arising. During the remaining time, I will work on adding a search function to HackageDB which will allow to find packages based on several different keys drawn from a package's description. A detailed plan will be given at [2] later this week. I am open for any requests or ideas on this subject, as well, of course. Thanks, Sascha [1] http://hackage.haskell.org/packages/archive/pkg-list.html [2] http://hackage.haskell.org/trac/summer-of-code/wiki/SoC2007Hackage [3] http://community.haskell.org/~sboehme/index.html [4] http://hackage.haskell.org/trac/hackage/wiki/HackageToDo

7 7

Hackage rocks
by Duncan Coutts 06 Aug '07

06 Aug '07

Yes indeed, Hackage rocks. Henning Günther wrote a Hackage -> Gentoo ebuild conversion tool for us and that allowed us to import more than 50 new packages into the Gentoo Haskell overlay. So we now have more than 150 packages available in the overlay, with a subset of 50 in the main Gentoo portage tree. This kind of productivity would have been impossible previously. So with that number of packages available we should be able to give a QA perspective. I've not checked exhaustively but here's my impression of common issues: * haddock docs not building. This is very common. The most common parse problem is un-escaped / chars. This seems to trip people up a lot. I wonder if we could make haddock's parser grok this better? For example most uses of ' don't cause problems, eg "don't". I wonder if we could have a similar rule for / so that things like "I/O" didn't get recognised as beginning an emphasised section. Perhaps the rule should be that / but be at the beginning or end of a token, so a " / " or a "X/Y" would not count, only "/beginning" or "ends/" of words? * packages not building with later versions of dependent packages. Eg changes in what modules are re-exported by some mtl modules between mtl-1.0 and 1.0.1. * inability to automatically find/generate dependencies on external packages like C libraries. Many packages that use FFI need to link to non-standard C libs and it's not always obvious what packages they need. Of course, specifying external libs is not simple since different platforms often call them different things, but perhaps we could have some non-binding hint fields. Or perhaps we should just try and guess based on the names of the libs that need to be linked to (though that doesn't cover deps on other programs rather than C libs). Duncan

5 6

RE: [Haskell-cafe] RE: Definition of the Haskell standard library
by Duncan Coutts 31 Jul '07

31 Jul '07

On Tue, 2007-07-31 at 17:26 +0100, Simon Peyton-Jones wrote: > | I see it as a really big deal that documentation becomes fragmented when > | one is using many packages, so that it's harder to find what you want. > | In fact, I'd classify that as the single biggest reason that I don't use > | many packages now > > When you install packages A,B,C, the documentation for A,B,C (and > nothing else) ought to be locally available as an integrated whole, > much as at the GHC web site. I don't know whether Cabal does, or > could do, that, but it's surely what one would expect. The docs for those packages would be available for packages installed via cabal (assuming the user did the optional haddock step) and would link to each other. What is missing from the local docs is a single integrated index page that lists all the modules and then links off to the various packages's docs like we have on the ghc website. The problem with generating one of those is what manages it? What package would it belong to etc. On some systems (windows, gnome) there are dedicated help viewers that can help with this contents/index issue. haddock supports both (mshelp, devhelp). I'm not sure everyone would find that a sufficient solution however. Duncan

2 1

darcs patch: Treat tabs in .cabal files as 8 spaces (when used as i...
by Thomas Schilling 31 Jul '07

31 Jul '07

Tue Jul 31 18:54:09 CEST 2007 Thomas Schilling <nominolo(a)gmail.com> * Treat tabs in .cabal files as 8 spaces (when used as indentation).

2 1

Before hacking on Haddock support...
by Roberto Zunino 30 Jul '07

30 Jul '07

...would you please review this patch? ;-) Sorry if I am being too pushy, but I'd want to avoid people developing on the "old" Haddock integration code (assuming this patch is worth inclusion, of course). Zun. ===== I am sending a patch[1] for integrating Cabal and HsColour + Haddock. Here is the result of running it on the base package: http://www.di.unipi.it/~zunino/tmp/base/index.html The above was generated by ./Setup configure ./Setup haddock --hscolour=hscolour.css Requirements: HsColour >=1.8, my hscolour.css file[2] (or your own, of course!), darcs Haddock + a link-breaking-bug fix (already sent) + any version bump (required: haddock > 0.8). Enjoy, Zun. [1] http://www.di.unipi.it/~zunino/tmp/cabal-hscolour-20070530 [2] http://www.di.unipi.it/~zunino/tmp/hscolour-emacs.css

2 2

darcs patch: Display a more helpful error message, wh... (and 2 more)
by Thomas Schilling 30 Jul '07

30 Jul '07

Tue Jul 31 00:40:26 CEST 2007 Thomas Schilling <nominolo(a)gmail.com> * Display a more helpful error message, when unknown fields are noticed. Tue Jul 31 00:52:56 CEST 2007 Thomas Schilling <nominolo(a)gmail.com> * Add line numbers to unknown fields error message. Tue Jul 31 00:53:11 CEST 2007 Thomas Schilling <nominolo(a)gmail.com> * Fix order of warnings.

1 0

darcs patch: Display a more helpful error message, when unknown fie...
by Thomas Schilling 30 Jul '07

30 Jul '07

Tue Jul 31 00:40:26 CEST 2007 Thomas Schilling <nominolo(a)gmail.com> * Display a more helpful error message, when unknown fields are noticed.

1 0

darcs patch: Fix self-compile warnings.
by Thomas Schilling 30 Jul '07

30 Jul '07

Tue Jul 31 00:05:37 CEST 2007 Thomas Schilling <nominolo(a)gmail.com> * Fix self-compile warnings.

1 0

Re: make distclean
by Duncan Coutts 30 Jul '07

30 Jul '07

On Mon, 2007-07-30 at 13:36 +0100, Ian Lynagh wrote: > On Mon, Jul 30, 2007 at 01:59:39PM +1000, Manuel M T Chakravarty wrote: > > ...doesn't clean the dist directories. > > I was talking to Thomas about this the other day; I think Cabal clean > should be much simpler than it currently appears to be. It ought to be > able to just do essentially "rm -rf dist", no more and no less. I agree. Let's do that. Anyone think it's not a good idea? Duncan

1 0

Cabal Configurations Status Report
by Thomas Schilling 27 Jul '07

27 Jul '07

Hello cabal-devel, the following is a summary of the changes to Cabal to enable Cabal Configurations. As we are about to integrate Cabal configurations into the main branch in order to get more testing feedback, I think it is a good time to give an overview about my changes. ** Parsing In order to allow the new syntactic categories of sections and if-else blocks, I added those categories to the low-level parsing primitives. Possible section names are "library", "executable", or "flag" of which the latter two require the name of the executable/flag name as a parameter. Conditional blocks may have an optional else-branch. The current implementation is rather restrictive in its permitted style; namely, the opening "{" has to be on the same line as the section name or "if" keyword and the closing "}" has to stand alone on its line unless it is followed by an "else {" branch. At a higher level we then parse this lower-level structure into a 'GenericPackageDescription' structure. This is defined as data GenericPackageDescription = GenericPackageDescription { packageDescription :: PackageDescription, packageFlags :: [Flag], condLibrary :: Maybe (CondTree ConfVar [Dependency] Library), condExecutables :: [(String, CondTree ConfVar [Dependency] Executable)] } The 'packageDescription' field contains a conventional package description with only part of the fields filled in, namely all global settings like version number or package name. However, it does *not* contain a valid "build-depends" field. This and all other library-related parts now go into the library section. The old Cabal format can easily be translated into this format so backwards compatibility is no problem. The 'CondTree' structure is an internal representation of a block of field descriptions and possible conditional blocks. data CondTree v c a = CondNode { condTreeData :: a , condTreeConstraints :: c , condTreeComponents :: [( Condition v , CondTree v c a , Maybe (CondTree v c a))] } A 'CondTree' node contains both data and constraints/dependencies. Conditions are parameterized over the variable type, which in our case will presumably be only one type. data ConfVar = OS String | Arch String | Flag String ** Resolving Conditions/Dependencies 'CondTree's can be resolved to a flat 'a' (the data part) given a function to determine which constraints are satisfied and which aren't. resolveWithFlags :: Monoid a => [(String,[Bool])] -- ^ Domain for each flag name, will be tested in order. -> String -- ^ OS name, as returned by System.Info.os -> String -- ^ arch name, as returned by System.Info.arch -> [CondTree ConfVar [d] a] -- ^ All CondTrees must satisfy the deps -> ([d] -> DepTestRslt [d]) -- ^ Dependency test function. -> (Either [d] -- missing dependencies ( [a] -- result data , [d] -- overall chosen dependencies , [(String, Bool)])) -- chosen flag assignments This function is the heart of the flag resolution process and is implemented rather naively by trying all possible flag assignments and choosing the first that satisfies the dependencies. As long as we don't use too many flags (or set some explicitly) this shouldn't be a problem. Note that the data has to be a Monoid. The problem is, that due to lack of first-class record fields[1] we have to store a partially filled-in 'a' (in our use case a 'Library' or 'Executable') and then rely on the Monoid instance to do the right thing. This strategy also required changing the way defaults are set; before, this was done by setting it in the empty value and just (blindly) overwriting it whenever it was set explicitly in the .cabal file. Now, however, we have to set it after the package has been resolved, and find out which fields haven't been set explicitly. This is a rather hackish ("hacky"?) approach but seems to work fine so far. [1] Implementing approximations would likely require extensions to Haskell98 thus wouldn't be a good idea for Cabal which tries to be as portable as possible. The function that does the actual flag-assignment resolution on packages is finalizePackageDescription :: [(String,Bool)] -- ^ Explicitly specified flag assignments -> Maybe [PackageIdentifier] -- ^ Available dependencies -> String -- ^ OS-name -> String -- ^ Arch-name -> GenericPackageDescription -> Either [Dependency] -- ^ Missing dependencies ( PackageDescription -- ^ Resolved package description , [(String,Bool)]) -- ^ Flag assignments chosen Using this function the 'configure' command determines a resolved package description (or fails) and stores it in the 'LocalBuildInfo' structure. From there it is available to all the other setup commands. Unfortunately, this is not sufficient for all build commands. ** Breaking the Interface to Setup.(l)hs Due to the new staged package resolution process, the 'confHook' arguments had to be changed, and most hooks could be changed to drop the 'PackageDescription' argument. For better backwards compatibility it is still passed in, although it is exactly the same as what is stored in the local build info. There are two notable exceptions to this: the 'sdist' and the 'clean' command. So far, both took only an optional LocalBuildInfo, but required a PackageDescription argument. 'sdist' needs this to determine where all the source and data files reside, 'clean' only needs this because it currently tries to be a bit too smart. The fix for 'sdist' is to provide it with a package argument that represents all possible assignments to the fields, i.e., a package description, where all the "if ... {" and "} else" lines are ignored. This is necessary, so that sdist can make sure that all possible source files are included in the tarball. The problem with this "unconditional flattening" is that it may lead to inconsistent assignments to a field value, e.g., a package that might be buildable under one an not buildable under another condition will be *not* buildable in the resulting package description. However, we don't read those fields in the current implementation, so this is no problem right now (but of course makes things not very maintainable, and custom hook implementations have to be careful.) A possible alternative is to simply disallow certain fields from being specified conditionally. I'm not so sure about this, since for example we'd disallow different source (sub-)trees for different architectures if we'd do so. The best way to fix the 'clean' hook, I think, would be to dumben 'clean' and make it simply be a "rm -rf dist/" equivalent. At the moment it has to clean up the generated files of many preprocessors that were put into the source directory to allow them being included in the 'sdist' generated tarball. I think a better solution is to explicitly copy (or generate) those files on demand (i.e., when running 'sdist'). ** Compatibility In any case we still have a huge problem with backwards compatibility. Whenever there are non-trivial changes to the Cabal API many packages might break, but--even worse--can be fixed to work with only one version of Cabal. I think we need a better way to do this. I will elaborate some ideas in a separate mail. ** Summary I hope this gives a good enough overview of the basic changes to Cabal (without dropping the whole source code at you) that hopefully stimulate some ideas, or questions (or complaints ;) ). HTH, / Thomas

3 2