December 2006 - cabal-devel

ghc requesting package version 1.1.7 but 0.1.7 is installed ?
by Marc Weber 15 Jan '07

15 Jan '07

$ ghc -o setup --make -package Cabal-0.1.7 setup.hs Linking setup ... ghc-6.5.20060917: unknown package: Cabal-1.1.7 I don't understand this. I've substituted Version 1.1.7 by 0.1.7 to get a low version which won't be used unless requested. This is the only Cabal package beeing installed. $ ghc-pkg describe Cabal shows name: Cabal version: 0.1.7 license: BSD3 ... Why is ghc requesting still 1.1.7? set | grep 1.1.7 doesn't list any interesting environment variable except of pwd and OLDPWD The Cabal which has been destributed with my ghc is verison 1.1.5.9.2 thus can't be the problem. Any suggestion is appreciated. Marc

2 1

[David Waern] Re: darcs patch: Add preliminary support for haddock-ghc
by Isaac Jones 31 Dec '06

31 Dec '06

2 1

Hello. some questions and suggestions
by Marc Weber 28 Dec '06

28 Dec '06

Dear Cabal-devel-list reader I've written two mails to Isaac Jones concerning cabal. He kindly asked me to use this list. So here I am. Scroll to the end of this mail to get to the most important text (look for 3) ) 1) ------------------------------------------------ first mail with response Marc Weber <marco-oweber(a)gmx.de> writes: > Hi Isaac Jones. Hi, Marc, thanks for writing. There's a cabal-devel mailing list where you can post questions so that others can take a crack at answering them, and they can get archived on the internet :) > I wanted to learn more about greencard so I started nivestigating how to > use greencard preprocessor of cabal. Then I noticed different things > > When building a lib and one or more executables only one build-depends > setting is allowed for all three, lib and executables, right? > If so it doesn't make much sense as ghc compiles statically so libs will > be included in all executables which aren't necessary. (Correct me if > I'm wrong, I feel still like a noob ;) The build dependencies are on a per-package basis so that we can track dependencies between packages (for instance, to make sure that dependency x is installed before package y). Have you verified that the so libs are compiled in and bloating unnecessary stuff? That's kinda too bad, but I'm not sure offhand what to do about it. As an end-user, you can always break things into separate packages. > Running more than one preprocessor on the same file requires writing > your own PrePrecossor handler and your own extension? Yes, we've never done chained preprocessors. We've been planning it for a while, as noted on this ticket: http://hackage.haskell.org/trac/hackage/ticket/13 If you can implement it, that would be great :) > So I came up with the idea (see post on haskelcafe) to use multiple > extensions, such as > > A.hs.gc.di.chs which should mean: > run c2hs, DrIft and then Greencard preprocessor on file A.hs. > + you can read the file extension and you know what's going on > - long names > - you have to configure your editor to recognize .gc.di.chs as haskell > file > - You can't test for existing files, you have to list all files in all > source directories and remvove those allowed extensions to see wether > a file matches a module name. > > What do you think? > > I've started implementing it and I want to write a test now. I think that searching for the files is going to be the hard part. Might not be too bad, though. > I know I can add my own IO action hook beeing invoked by ./setup test. > The runTests hook function takes a Bool flag without description. dons > on #haskell suggested it might be a verbose flag. This would make sense > as the test hook doesn't take the ConfigFlags argument containing the > verbose Int setting. So is this the verbose flag and if so why Bool and > not Int, too? I can't remember what the Bool in runTest is for, honestly. Might be a mistake. I set up runTest so that you can pass it any flags you want (see the example in test/withHooks/Setup.lhs). > When writing tests for cabal packages, how is this done? > > I think it does make sense to seperate testing compiler/ package setup > flavour from the package because not everyone installing the package > wants to run the tests. > > Would it be nice to be able to add test modules in the cabal file such > like > > test-source-dirs: test > test-is: MyPackageTest.hs > test-build-depends: QuickCheck > test-flavour: ghci > > and let cabal handle compiling/ running the test when invoking > ./setup test > automatically? Maybe we could do that with configurations, as discussed on the mailing list recently: http://hackage.haskell.org/trac/hackage/ticket/27 That might be a more general solution, although not implemented yet. > I think running ghci might be a little bit faster than the compiler. > This would require adding flavour GHCI for those who don't have hugs > installed. > > Can you tell me more about the status of cabal-install? It's mostly done, but quite a few loose ends to tie up, and work on promition and adoption needs to be done. There's a hackathon planned where some folks will work on it: http://www.mail-archive.com/haskell@haskell.org/msg19470.html Maybe you can join in, or join in remotely :) peace, isaac ------------------------------------------------ 2) ------------------------------------------------ response 2: > Thanks Isaac for your wonderful reply You're very welcome. Thanks for your interest in Cabal. We're always looking for good hackers :) >> Hi, Marc, thanks for writing. There's a cabal-devel mailing list >> where you can post questions so that others can take a crack at >> answering them, and they can get archived on the internet :) > > I'd like to join, really. Would you mind pointing me to the mailinglist > you were refering to? I wasn't able to locate it. (neither on > haskell.org -> mailinglists nor on hackage) > Is this the right one? > http://www.haskell.org/mailman/listinfo/libraries http://www.haskell.org/mailman/listinfo/cabal-devel The list of haskell mailing lists is here: http://www.haskell.org/mailman/listinfo/ :) > I've trouble compiling cabal-install because Network/XmlRpc/Internals.hs seems > to be in the path of ghc sources only. Is this meant to be used with ghc > distribution only? That's an older version of cabal-install. Look at this repo, it doesn't require many dependencies: darcs get http://darcs.haskell.org/cabal-with-install/ I don't know if it works w/ anything but ghc at the moment. > I like the idea using indentation (such as the haskell language is > using anyway) Thus either use {;;..} or indentation. Do you know > wether this processing step can be taken from a haskell compiler > parser? Are you referring to the configurations thread? > As soon as I'll be subscribed to the list I'll repost these messages > to be archived and read by others. The archive is here, you don't have to subscribe to read: http://www.haskell.org/pipermail/cabal-devel/ peace, isaac 3) ------------------------------------------------ summary, my steps, suggestions? Refering to http://hackage.haskell.org/trac/hackage/ticket/13 (topic allow preprocessor chaining) I'm working on this. Should be finished soon. Refering to http://hackage.haskell.org/trac/hackage/ticket/27 (enhancement, configuration option, optional dependencies etc) I'd like to work on this too. But I I'm lacking some knowledge here. At the moment cabal is using ReadP, right? Am I also right that ReadP is lacking user friendly error messages compared to parsec? I like the idea of using either { ; ; } or indentation. Thus you have best of both styles, C and python such like the haskell language. So would it be ok to use parsec instead of ReadP? ------- example file name: <name of package> version: <package version> build-depends: lib: <name of lib> hs-source-dirs: version: <lib version> test: test1 { main-is: test.hs; additional-dependencies: QuickCheck } test: test2 { main-is: test2.hs; additional-dependencies: QuickCheck } executable: <name of executable> hs-source-dirs: executable1 main-is: Main.hs additional-dependencies: <dependencies additional to global package dependencies> test: test1 { main-is: test.hs } executables: executable2 { main-is: Main.hs; hs-source-dirs: ex2 } [ ... ] ----------------- ^ Notice that you can attach tests to either libraries or executables. I would run them using ./setup test <executable-name>/<test-name> ./setup test <executable-name>/* ./setup test * -- run all tests Then I would suggest a directory structure like this packagedir/ executable1/ tests/test1/test.hs tests/test2/test.hs executable2/ tests/test1/test.hs tests/test2/test.hs lib/ test/test1/test.hs test/test2/test.hs which would mean setting hs-source-dirs: to the name of executable or library by default Some thoughts about what to support: cross-compiling ? ( I mean ghc can compile via C and gcc can do cross-compiling. I've never done this before) configuration dependend on * os/ compiler flavour eg [configuration: OS=windows, COMPILER=ghc] * selected features eg ( --enable-jpeg-support ) * existance of other libraries ( either gtk2hs or wxhaskell or hscurses ? hsplugins? ( example haskelldb ) ) * custom flags : flag workaround ghc-windows bug = OS=windows compiler=ghc compiler-version < 4.0 does this mean "or" or "and" ? Both should be possible as setup.hs has to be compiled anyway, would it be a good idea to not just print the configuration to a file (.setup-config), but generate a haskell module beeing imported by setup.[l]hs ? This would mean that you can even use haskell-expressions as flags such as hasGreencard : executableInPath "greencard" && greencardVersion > (3,2,0) [configuration: hasGreencard] Thus you can easily use or, and ||, && and so on or `elem` configuration options ... This leads me to the assumption that the only difference between setup.hs and package.cabal is that it's easier to write a cabal file than a module containing the same information, right? There should be the possibility to only use some configuration option depending on others. eg: ./setup --help --use-image-lib1 --with-jpg --with-gif --use-iamge-lib2 -- no suboptions here right? Things are done in this order right now: setup.hs -> tool to * configure * build -- with user hooks * test -- with user hooks * install Would it make sense to allow user hooks when configuring, too? This would result in a two stage tool chain: configure.hs -> tool to * configure -- with user hooks (eg generating Config.hs imported by setup.hs) setup.hs + configuration options -> tool to * build -- with user hooks * test -- with user hooks * install What do you think ?? I have to think about this again. Yet another thing. This is annoying: ./setup configure ./setup build sudo ./setup install It's like doing ebuild `equery which ghc` unpack ebuild `equery which ghc` compile ebuild `equery which ghc` install on gentoo systems Is it the task of cabal or cabal-setup to just use ./setup install which should run configure/ build automatically if necessary? or at least allow ./setup configure build install (in short ./setup cbi) Packags containing libraries and executables: Am I right that there is no difference in a) isntalling lib -> compiling executable with -pacakge lib b) compiling executable with lib modules in path. as long as there lib is a static lib. shared libraries aren't used because of recompilation issues, right? It would be much cleaner to use a) in any way and let cabal install the libraries before compiling those executables automatically... Does it make sense to distribute two libraries usnig one package? Example 1: Cabal contains some functionality to join filenames. This might also be another package to be used in different applications as well? Example 2: MissingH. I think it would be best to split this into different libs (MissingH-logging, MissingH-Prelude, ...) Aeh.. Sorry for this long post and happy replying... Marc Weber

4 5

Patch using multiple preprocessors ...
by Marc Weber 28 Dec '06

28 Dec '06

This patch seems to work. But I haven't done serious testing yet. It passes my own test. The test is in multipleextensiontest and contains the files multiple_extension/ setup.hs ( contains the test preprocessors) src_executable/Multipleextensiontest.1.2.3.a.b module Main where main = do print "start" -- P1 -- P2 -- P3 -- Pa -- Pb -- Pc print "end" multipleextensiontest.cabal The preprocessors 0,1,2,a,b,c (c not used) will each replace -- Px by print "x" and add another print line to get the reversed order of run preprocessors. The diff is made relative to darcs/haskell.org/cabal/cabal-1.1.7 Isaac told me that cabal-install is newer .. You can also get the whole repository with changes here (http://mawercer.de/marcweber/haskell/cabal-1.1.7-my/ ) Greetings Marc diff -rN old-cabal-1.1.7-my/Cabal.cabal new-cabal-1.1.7-my/Cabal.cabal 7c7 < Build-Depends: base --- > Build-Depends: base, mtl diff -rN old-cabal-1.1.7-my/Distribution/PreProcess.hs new-cabal-1.1.7-my/Distribution/PreProcess.hs 64c64 < moduleToFilePath, die, dieWithLocation) --- > moduleToFilePath, die, dieWithLocation, on) 70c70,71 < import System.Directory (removeFile, getModificationTime) --- > import System.Directory (removeFile, getModificationTime, doesFileExist) > import Data.Bits ((.|.)) 73c74,76 < (splitFileExt, joinFileName, joinFileExt) --- > (splitFileName, splitFilePath, splitFileExt, joinFileName, joinFileExt) > import Control.Monad > import Debug.Trace 76c79 < -- external program, but need not be. The arguments are the name of --- > -- /xternal program, but need not be. The arguments are the name of 144,163c147,177 < preprocessModule searchLoc modu verbose builtinSuffixes handlers = do < bsrcFiles <- moduleToFilePath searchLoc modu builtinSuffixes < psrcFiles <- moduleToFilePath searchLoc modu (map fst handlers) < case psrcFiles of < [] -> case bsrcFiles of < [] -> die ("can't find source for " ++ modu ++ " in " ++ show searchLoc) < _ -> return ExitSuccess < (psrcFile:_) -> do < let (srcStem, ext) = splitFileExt psrcFile < pp = fromMaybe (error "Internal error in preProcess module: Just expected") < (lookup ext handlers) < recomp <- case bsrcFiles of < [] -> return True < (bsrcFile:_) -> do < btime <- getModificationTime bsrcFile < ptime <- getModificationTime psrcFile < return (btime < ptime) < if recomp < then pp psrcFile (srcStem `joinFileExt` "hs") verbose < else return ExitSuccess --- > preprocessModule searchLoc modu verbose builtinSuffixes handlers = > let hslhs = zip builtinSuffixes $ repeat (\_ _ _ -> return ExitSuccess) -- .hs,.lhs or .gc using hmake with nhc : do nothing > all_handler = hslhs ++ handlers -- use builtin handler first ( you can't override .hs, lhs anyway as target and source filename would be the same, wouldn't it?) > preprocess f = let (dir, file) = splitFileName f > (fwe, ext) = splitFileExt file > in case ext of > "" -> case splitFileExt fwe of > (_, "") -> return ExitSuccess -- No extension left. do nothing > _ -> die $ "no extension found! don't know how to handle this file " ++ f > ext -> case lookup ext all_handler of > Nothing -> die $ "no handler found for extinsion pant " ++ ext > Just pp -> let dest = dir `joinFileName` fwe > in do de <- doesFileExist dest > recomp <- if de then liftM2 (>) > (getModificationTime f) > (getModificationTime dest) > else return True -- dest file doesn't exist > if recomp then pp f dest verbose >>= > \exitCode -> case exitCode of > ExitSuccess -> return () > _ -> fail $ "preprocessor beeing responsible for extension part " ++ ext ++ "failed" > else return () > -- now do next preprocessing step > preprocess dest > > in moduleToFilePath searchLoc modu (map fst all_handler) >>= \srcFiles -> case srcFiles of > [] -> die $ "!!can't find source for " ++ modu ++ " in " ++ show searchLoc > [f] -> -- run preprocessor if necessary > preprocess f > files -> die $ "muliple fitting source files found for module " > ++ modu ++ " files " ++ foldr1 (\a b->a ++ ", " ++ b) files diff -rN old-cabal-1.1.7-my/Distribution/Simple/Utils.hs new-cabal-1.1.7-my/Distribution/Simple/Utils.hs 44a45 > on, 58a60 > moduleToFilePath2, 89,90c91,93 < import Control.Monad(when, filterM, unless) < import Data.List (nub, unfoldr) --- > import Control.Monad(when, filterM, unless, liftM) > import Data.List (nub, unfoldr, maximumBy, length, isPrefixOf) > import Data.Map (union, toList, fromList) 113a117,125 > f `on` op = \x y -> f x `op` f y > > -- --------------------------------------------------------------------------- List > -- spits the list on element element > splitBy :: (Eq a) => [a] -> a -> [[a]] > splitBy list element = case break (== element) list of > (a, []) -> [a] > (a, b) -> a:splitBy b element > 213a226,249 > -- FIXME/TODO: check wether tail contains only known suffixes. Only checks matching filename yet > moduleToFilePath2 :: [FilePath] -- ^search locations > -> String -- ^Module Name > -> [String] -- ^possible suffixes > -> IO [(FilePath, FilePath)] > moduleToFilePath2 locs mname possibleSuffixes = > let mname_parts = splitBy mname '.' -- "Data.List" -> ["Date","List"] > m_path = init mname_parts > joinPathElements folders = foldr1 joinFileName folders > gDC path = catch (getDirectoryContentsWithoutSpecial path) (\_-> return []) > maxBy [] = Nothing > maxBy l = Just $ maximumBy (\a b -> compare (length a) (length b)) l > -- does file match mname? > filterFile = ( isPrefixOf ((last mname_parts) ++ ['.'] )) > -- filter names beginning with (last mname_parts) ++ ['.'], > -- omitting hs/lhs for compatibility (.gc, .chs, ..) > -- only use longest filename (thus no intermediate file) > filterModule = maxBy . (filter filterFile) > foldr_f (loc, files) l = case filterModule files of > Nothing -> l > Just f -> (loc, f):l > in do files <- mapM (\loc -> gDC $ joinPathElements (loc:m_path) ) locs > return $ foldr foldr_f [] $ zip locs files > 217a254 > moduleToFilePath locs mn ps = (liftM (map (uncurry joinFileName))) $ moduleToFilePath2 locs mn ps 219,238c256,280 < moduleToFilePath pref s possibleSuffixes < = filterM doesFileExist $ < concatMap (searchModuleToPossiblePaths s possibleSuffixes) pref < where searchModuleToPossiblePaths :: String -> [String] -> FilePath -> [FilePath] < searchModuleToPossiblePaths s' suffs searchP < = moduleToPossiblePaths searchP s' suffs < < -- |Like 'moduleToFilePath', but return the location and the rest of < -- the path as separate results. < moduleToFilePath2 < :: [FilePath] -- ^search locations < -> String -- ^Module Name < -> [String] -- ^possible suffixes < -> IO [(FilePath, FilePath)] -- ^locations and relative names < moduleToFilePath2 locs mname possibleSuffixes < = filterM exists $ < [(loc, fname `joinFileExt` ext) | loc <- locs, ext <- possibleSuffixes] < where < fname = dotToSep mname < exists (loc, relname) = doesFileExist (loc `joinFileName` relname) --- > -- moduleToFilePath :: [FilePath] -- ^search locations > -- -> String -- ^Module Name > -- -> [String] -- ^possible suffixes > -- -> IO [FilePath] > -- > -- moduleToFilePath pref s possibleSuffixes > -- = filterM doesFileExist $ > -- concatMap (searchModuleToPossiblePaths s possibleSuffixes) pref > -- where searchModuleToPossiblePaths :: String -> [String] -> FilePath -> [FilePath] > -- searchModuleToPossiblePaths s' suffs searchP > -- = moduleToPossiblePaths searchP s' suffs > > -- -- |Like 'moduleToFilePath', but return the location and the rest of > -- -- the path as separate results. > -- moduleToFilePath2 > -- :: [FilePath] -- ^search locations > -- -> String -- ^Module Name > -- -> [String] -- ^possible suffixes > -- -> IO [(FilePath, FilePath)] -- ^locations and relative names > -- moduleToFilePath2 locs mname possibleSuffixes > -- = filterM exists $ > -- [(loc, fname `joinFileExt` ext) | loc <- locs, ext <- possibleSuffixes] > -- where > -- fname = dotToSep mname > -- exists (loc, relname) = doesFileExist (loc `joinFileName` relname) diff -rN old-cabal-1.1.7-my/Setup.lhs new-cabal-1.1.7-my/Setup.lhs 1,5d0 < #!/usr/bin/runhaskell < > module Main where < > import Distribution.Simple < > main :: IO () < > main = defaultMain diff -rN old-cabal-1.1.7-my/multiple_extension-test/multiple_extension-test.cabal new-cabal-1.1.7-my/multiple_extension-test/multiple_extension-test.cabal 0a1,12 > Name: multipleextensiontest > Version: 0.0 > License: BSD3 > Author: Marc Weber > Category: > Synopsis: test multiple extensions > Build-Depends: haskell98 ,base ,Cabal > > Executable: multipleextensiontest > hs-source-dirs: src_executable > Main-Is: Multipleextensiontest.hs > other-modules: Multipleextensiontest diff -rN old-cabal-1.1.7-my/multiple_extension-test/setup.hs new-cabal-1.1.7-my/multiple_extension-test/setup.hs 0a1,56 > module Main where > import Distribution.Simple > import Distribution.Simple.Configure > import Distribution.Simple.LocalBuildInfo > import Distribution.PreProcess > import System > import System.Process > import System.Exit > import Monad > > import Char ( isSpace ) > -- #ifdef __GLASGOW_HASKELL__ > -- #ifndef __HADDOCK__ > -- import {-# SOURCE #-} GHC.Unicode ( isSpace ) > -- #endif > -- import GHC.List ( replicate ) > -- import GHC.Base > -- #else > -- import Data.Char( isSpace ) > -- #endif > > import Text.ParserCombinators.ReadP > > main= do > args <- getArgs -- check args to launch ghci > when (length args > 0) $ do > when ((args!!0) == "ghci") $ do > lbi <- getPersistBuildConfig > let packageArgs = (concat.concat) [ [" -package ", showPackageId pkg] | pkg <- packageDeps lbi ] > system("ghci " ++ packageArgs) > exitWith ExitSuccess > defaultMainWithHooks $ defaultUserHooks { hookedPreProcessors = hp } > where hp = map (testPreprocessor.(:[])) "123abc" > > testPreprocessor :: String -> PPSuffixHandler > testPreprocessor ext = ( ext, pp) > where pp _ _ src dest verb = do > print $ " preprocessing file " ++ src ++ " and writing to " ++ dest > readFile src >>= \f -> writeFile dest $ unlines . ppFile $ lines f > return $ ExitSuccess > ppFile :: [ String ] -> [ String ] > ppFile =(++ [" print \"preprocessed by "++ ext ++ "\""]) > . (map preprocessLine) > parseLine :: ReadP (String, String) > parseLine = do spaces <- many $ satisfy isSpace > string "--" >> (many $ satisfy isSpace) > char 'P' > ext <- (many1 $ satisfy (not.isSpace)) > rest <- many get > return $ (spaces, ext) > preprocessLine :: String -> String > preprocessLine line = let parseResult = (readP_to_S parseLine) $ line > in case parseResult of > [((spaces, e),_)] -> if ext == e then spaces ++ "print \"" ++ ext ++ "\"" > else line > _ -> line diff -rN old-cabal-1.1.7-my/multiple_extension-test/src_executable/Multipleextensiontest.hs.1.2.3.a.b new-cabal-1.1.7-my/multiple_extension-test/src_executable/Multipleextensiontest.hs.1.2.3.a.b 0a1,11 > module Main where > > main = do > print "start" > -- P1 > -- P2 > -- P3 > -- Pa > -- Pb > -- Pc > print "end"

1 0

package collection
by Ross Paterson 05 Dec '06

05 Dec '06

As I understand it, a package collection, like http://hackage.haskell.org/packages/unstable/ has the structure 00-index.tar.gz <pkg>/ <pkgid>.cabal <pkgid>.tar.gz ... where 00-index.tar.gz tars up all the <pkg>/<pkgid>.cabal files. Such a simple structure makes it easy to add tools. Perhaps it could be made simpler by replacing the index with a file that was just a concatenation of the .cabal files with some separator (say @), and having both gzipped and uncompressed versions. The main missing piece seems to be a means of uploading packages. Any plans there? Perhaps something along the lines of PAUSE (http://www.cpan.org/misc/cpan-faq.html#What_is_PAUSE) except that we only need to upload Cabal tarballs.

3 2

Re: cabal (./Setup.hs) options
by Duncan Coutts 04 Dec '06

04 Dec '06

On Tue, 2006-08-08 at 10:45 +0200, Johan Tibell wrote: > > Hi, > > > > I like Cabal. There are a few things that I'm used to from autotools which > > I can't seem to find: dry-runs, uninstall and a "make distcheck" equivalent. > > <snip> > > > uninstall > > Removes all files and directories created by install > > Ditto. This is really necessary since otherwise I'll have to build a > Debian package anyway before I can install the Cabal package. Getting > software of my system is almost as important as getting it on there in > the first place. Prevents my system from deteriorating over time. I think we didn't include this feature deliberately for some reason. Isaac can you remember? > Would it be hard to implement? Anything I could do to help? In general the way to help is just to submit patches with an accompanying explanation/motivation/rationale. 'darcs send' does the "Right Thing"tm, i.e. it sends them to the cabal-devel list. Oh and everyone is invited to join the cabal-devel mailing list. Subscribe here: http://www.haskell.org/mailman/listinfo/cabal-devel Our main bottleneck on Cabal development is not the lack of people pointing out shortcomings but the number of people sending patches. Send more patches, the last ones tasted great! Duncan (wearing his Cabal patch reviewer hat)

8 12