December 2007 - cabal-devel

[Hackage] #171: Add command to automagically create an initial cabal package
by Hackage 18 Dec '09

18 Dec '09

#171: Add command to automagically create an initial cabal package --------------------------+------------------------------------------------- Reporter: guest | Owner: Type: enhancement | Status: new Priority: normal | Milestone: Component: Cabal | Version: 1.2.2.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.6 Platform: Linux | --------------------------+------------------------------------------------- Don Stewart has a nice tool called mkcabal (http://www.cse.unsw.edu.au/~dons/code/mkcabal/) which creates an initial cabal package. It would be nice to have this built into the 'cabal' program itself so that "cabal create" or "cabal init" or similar would interactively create a cabal package. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/171> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 7

[Hackage] #199: when installing a new version of GHC, there should be a way to upgrade my packages
by Hackage 18 Jul '09

18 Jul '09

#199: when installing a new version of GHC, there should be a way to upgrade my packages ---------------------------------+------------------------------------------ Reporter: guest | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: cabal-install tool | Version: 1.2.3.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.8.2 Platform: Linux | ---------------------------------+------------------------------------------ If I have version 6.6 of GHC installed, with packages foo, bar, and bang, and I install ghc 6.8, I'd like a way to automatically install foo, bar, and bang. This could be part of GHC's installation script possibly, or maybe a feature of cabal install. cabal can check the installed packages for the previous version of GHC and "cabal install" them. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/199> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 10

[Hackage] #185: Temporary executables leave foo.exe.manifest files behind
by Hackage 02 Jun '09

02 Jun '09

#185: Temporary executables leave foo.exe.manifest files behind ----------------------+----------------------------------------------------- Reporter: guest | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: Cabal | Version: HEAD Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.8.1 Platform: Windows | ----------------------+----------------------------------------------------- When Cabal creates and runs a temporary program, as it appears to do with hsc when running over the base libraries doing haddock, it leaves a manifest behind in dist. On Windows, ghc --make with 6.8.1 produces a foo.exe and a foo.exe.manifest. Cabal deletes the former, but not the latter. Very minor, but should be easy to fix too. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/185> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 2

[Hackage] #188: file extensions
by Hackage 16 Mar '09

16 Mar '09

#188: file extensions --------------------------+------------------------------------------------- Reporter: guest | Owner: Type: enhancement | Status: new Priority: normal | Milestone: Component: Cabal | Version: 1.2.2.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.8.1 Platform: Linux | --------------------------+------------------------------------------------- GHC ignores Objective C sources unless -x c is passed to it on the command line before the source files. As far as I could tell there is no way to pass that option to GHC through Cabal so no .m files ever get built (well.. not completely true - setting ghc-options works but that breaks compilation of Haskell code). One way to get this to work would be to just add some extra code in Cabal to automatically pass -x c for .m files. This is probably a bad idea as someone might want or need to compile files using some other extension at some point. Another option is add some way of specifying compile options on a per-file (or group of files) basis: setting ghc-options to -x c just for one set of files. Still very limited but it's an improvement over the first option. Third option I can think of is to add a way of specifying compiler/extension pairs in Cabal so given compilers are automatically used to compile files with corresponding extensions. Depending on how much time I have I might look into adding this functionality myself but would like some feedback on which approach to take. Otherwise I might have to stick with autotools and ignore Cabal until it gets a bit more mature and flexible. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/188> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 6

[Hackage] #189: Handle framework paths (-F) in Cabal
by Hackage 03 Mar '09

03 Mar '09

#189: Handle framework paths (-F) in Cabal --------------------------+------------------------------------------------- Reporter: guest | Owner: Type: enhancement | Status: new Priority: normal | Milestone: Component: Cabal | Version: 1.2.2.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.8.1 Platform: Mac OS | --------------------------+------------------------------------------------- There's been some discussion on the GHC trac ([http://hackage.haskell.org/trac/ghc/ticket/1931 1931], [http://hackage.haskell.org/trac/ghc/ticket/1798 1798], [http://hackage.haskell.org/trac/ghc/ticket/1395 1395]) about adding framework search paths (gcc's `-F` flag, and ghc's `-framework-path` flag). Currently Cabal handles `-framework`, but not `-framework-path`. I propose the following behavior for Cabal: 1. Add a `framework-path:` field which will pass `-framework-path` to `ghc` and `-F` to `gcc`, `hsc2hs`, et. al. 2. Always add `-F$HOME/Library/Frameworks` as an argument to the above programs (regardless of any `framework-path` entries). I believe `#1` to be uncontroversial. My reasoning for `#2` is the following: - `$HOME/Library/Frameworks` is the standard location to put frameworks if you do not have administrative access. - Without it, `readline.cabal` (e.g.) would need {{{ framework-path: /Users/judah/Library/Frameworks }}} which is not portable between machines. Finally, note that these flags only affect build behavior, not runtime loading of libraries (which searches `$HOME/Library/Frameworks` by default). See [http://hackage.haskell.org/trac/ghc/wiki/OSXFrameworks OSXFrameworks] for more info. -Judah -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/189> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 13

[Hackage] #200: Feature request: Simplify the profiling process
by Hackage 25 Jan '09

25 Jan '09

#200: Feature request: Simplify the profiling process ----------------------------+----------------------------------------------- Reporter: guest | Owner: Type: enhancement | Status: new Priority: normal | Milestone: Component: Cabal library | Version: 1.2.3.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.8.2 Platform: | ----------------------------+----------------------------------------------- Add Setup.(l)hs options that: 1) tells ghc to make a profiling library; 2) tells ghc to make a profiling executable 3) pass on the (a) -auto or (b) -auto-all option 4) Let you do (1)-(2)-(3a) or (1)-(2)-(3b) with a single option, instead of three. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/200> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 10

[Hackage] #177: Include-dirs, extra-lib-dirs with spaces doesn't work under Windows
by Hackage 22 Jan '09

22 Jan '09

#177: Include-dirs, extra-lib-dirs with spaces doesn't work under Windows ----------------------+----------------------------------------------------- Reporter: guest | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: Cabal | Version: 1.2.2.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.6 Platform: Windows | ----------------------+----------------------------------------------------- Trying to build the HDBC-PostGreSQL database driver[1] revealed some poor handling of pathames under Windows. The package requires that the paths to postgre header and libraries files be added to the .cabal file. My initial attempt was the obvious: {{{ include-dirs: C:\Program Files\PostgreSQL\8.2\include, C:\Program Files\PostgreSQL\8.2\include\server, . extra-lib-dirs: C:\Program Files\PostgreSQL\8.2\include, C:\Program Files\PostgreSQL\8.2\include\server }}} However, the package would not build. The include files "pg_config.h" and some others were not found. Using single or double quotes did not help. Finally, I moved the installation to "C:\pgsql" and updated the .cabal file: {{{ include-dirs: C:\pgsql\include, C:\pgsql\include\server, . extra-lib-dirs: C:\pgsql\lib }}} And the package built. It seems the spaces in the first path caused the problem. For reference, this it the configure output on my machine: {{{ Configuring HDBC-postgresql-1.0.1.0... configure: Dependency base-any: using base-2.1.1 configure: Dependency mtl-any: using mtl-1.0.1 configure: Dependency HDBC>=1.0.0: using HDBC-1.0.1 configure: Dependency parsec-any: using parsec-2.0 configure: Using install prefix: C:\Program Files configure: Binaries installed in: C:\Program Files\Haskell\bin configure: Libraries installed in: C:\Program Files\Haskell\HDBC- postgresql-1.0.1.0\ghc-6.6.1 configure: Private binaries installed in: C:\Program Files\HDBC- postgresql-1.0.1.0 configure: Data files installed in: C:\Program Files\Common Files\HDBC- postgresql-1.0.1.0 configure: Using compiler: c:\ghc\ghc-6.6.1\bin\ghc.exe configure: Compiler flavor: GHC configure: Compiler version: 6.6.1 configure: Using package tool: c:\ghc\ghc-6.6.1\bin\ghc-pkg.exe configure: Using ar found on system at: c:\ghc\ghc-6.6.1\bin\ar.exe configure: Using haddock found on system at: C:\haddock\haddock-0.8\haddock.exe configure: No pfesetup found configure: Using ranlib found on system at: c:\MinGW\bin\ranlib.exe configure: Using runghc found on system at: c:\ghc\ghc-6.6.1\bin\runghc.exe configure: Using runhugs found on system at: C:\Program Files\WinHugs\runhugs.exe configure: No happy found configure: No alex found configure: Using hsc2hs: c:\ghc\ghc-6.6.1\bin\hsc2hs.exe configure: No c2hs found configure: No cpphs found configure: No greencard found }}} Feel free to email me at jgbailey AT gmail DOT com for more info. [1] Version 1.0.1.0 available at http://hackage.haskell.org/cgi-bin /hackage-scripts/package/HDBC-postgresql-1.0.1.0. HDBC is also required. I used 1.0.1, available at http://hackage.haskell.org/cgi-bin/hackage- scripts/package/HDBC-1.0.1. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/177> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 10

[Hackage] #179: support GHC's main-is extension
by Hackage 20 Jan '09

20 Jan '09

#179: support GHC's main-is extension --------------------------+------------------------------------------------- Reporter: duncan | Owner: Type: enhancement | Status: new Priority: normal | Milestone: Component: Cabal | Version: 1.2.2.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.4.2 Platform: Linux | --------------------------+------------------------------------------------- Query on haskell-cafe: http://haskell.org/pipermail/haskell- cafe/2007-November/034686.html {{{ It seems the meaning of the -main-is switch for GHC and the Main-Is build option for Cabal executables differ. With GHC, I can point to any function "main" in any module, but in Cabal I must point to a filename with precisely the module name "Main". This is tying my hands with regard to organizing a default executable and exposing some of its functionality as a library. Is there a way to get around this restriction? Concretely, I want to point Cabal's Main-Is to Program/Main.hs which starts with module Program.Main where instead of just module Main where }}} So in GHC it has this meaning: http://haskell.org/ghc/docs/latest/html/users_guide/options-phases.html #options-linker {{{ -main-is thing The normal rule in Haskell is that your program must supply a main function in module Main. When testing, it is often convenient to change which function is the "main" one, and the -main-is flag allows you to do so. The thing can be one of: A lower-case identifier foo. GHC assumes that the main function is Main.foo. An module name A. GHC assumes that the main function is A.main. An qualified name A.foo. GHC assumes that the main function is A.foo. }}} In Cabal the {{{main-is:}}} field specifies the '''file name''' of the Main module. GHC's {{{-main-is}}} flag is an extension to Haskell that is not supported by the other Haskell implementations. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/179> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 6

proptotype of make style dep stuff
by Duncan Coutts 24 Aug '08

24 Aug '08

All, At the recent Haskell Hackathon after ICFP, one of the issues that filled a lot of the thinking and drinking time of the Cabal hackers present was the issue of dependency analysis. I'd like to present some initial ideas and code that arose out of discussions, particularly between Lennart Kolmodin, Thomas Schilling and myself. The code I'm presenting was written jointly with them. This message is going to be one of those crazy literate Haskell programs. The code is also available here: darcs get http://haskell.org/~duncan/cabal/dep-experiment/ One thing we debated for some time was whether we needed support for dynamic or just static dependencies. The distinction here is whether one can always construct the entire dependency graph before running any expensive actions like pre-processors or compilers. If we assume that users do not need to specify every module in their package then it's easy to construct examples where it is impossible to construct the entire dependency graph without running pre-processors. So one of the requirements of this design is that we can interleave discovery of the dependency graph with running actions. We would also like to be able to specify and test the implementation. So allowing testing without having to do IO would be an advantage. We also want it to be fairly generic and extensible with respect to the kinds of actions and rules we can express. We want people to be able to extend the system with their own custom pre-processors for example. So let's dive in... > data Graph m = Graph [Rule m] [Target] We'll use an explicit representation of a dependency graph. This representation is not efficient but it's fairly simple. We have a set of rules and a set of targets which we're currently chasing but have not yet got rules to cover. So we would start our make procedure with a graph with no rules and just the targets we wish to build. > data Rule m = Rule { > targets :: [Target], > depends :: [Target], > action :: m [Target] > } A rule has a list of targets and a list of dependencies and an action. So a rule is not a simple edge or node in a dependency graph. It is a bundle of edges with an associated action. The reason we need multiple targets is because many build actions produce multiple output files, think .o and .hi files as the primary example. So a Rule says that there is a dependency or edge between every target and every dependency. This representation is mostly lifted from Neil Mitchel's make code. The difference is in the handling of dynamic dependencies. Here, the action can return a list of targets and this is used in our implementation of dynamic dependencies. I will come back to how that works later. Note also that the Rule and thus the Graph type are parameterised by a type m. This will be the monad that the actions work in. It will allow us to use it with a testing monad that does no IO and enables QuickCheck specifications and also later with a monad that can actually to IO and run programs. > instance Show (Rule m) where > show (Rule ts ds _) = "Rule " ++ show ts ++ " " ++ show ds ++ " (<action>)" We need to be able to show rules, though we cannot show the associated action of course. > type Target = FilePath We've talked about Targets already without saying what they are. Initially we are using the simplification that a target is just a FilePath. We currently do not distinguish different kinds of targets, so called resolved or unresovled targets. We will need a notion of a thing that we depend on, but which we have not yet nailed down to a particular concrete object like a file. It will be necessary later to make have that notion to support search paths and other indirections. We will also later want to have more than a single name space for targets, since not every target corresponds directly to a file. > type DepGenerator m = Target -> m [Rule m] A dependency generator is a function that given a target produces for us one or more rules that 'cover' that target. That is there is one rule that has that target in its targets list. It may also fail if there is no way to find a rule. Again the action is in some monad, so it may go and do actions in the monad like searching the file system. It presumably will also consult some rule schema that say how to make a rule for any file of a particular kind. This is one way we can separate the dependency chasing from the main make algorithm. > make :: Monad m > => DepGenerator m > -> Graph m > -> m () So we come to the make algorithm. As I mentioned before it is parametrised by some Monad m. It takes one of these dependency generation functions and the current Graph. Initially we expect the Graph will have no rules and just the initial targets (which might for example be the .hi/.o files corresponding to the exposed modules in a library). > make gen (Graph [] []) = return () If the graph is completely empty we're done. This will always happen in the end because we remove rules when we've run their actions. > make gen (Graph rules (t:ts)) = do > rules' <- gen t > make gen (Graph (rules' ++ rules) ts) We prefer expanding the graph to running actions. This is because expanding the graph gives us more choice about which actions to run later (possibly in parallel). It also means that when it is possible to generate the whole graph as in the static depgraph approach, that we do just that. So we pick the first unresolved dependency and we ask the dep generator to make some rules that tell us how to build that as a target. We add those rules to our rule set and carry on. There are some requirements on the returned rules here. For one thing they must actually 'cover' the target we were asking about. Secondly they must extend our current graph in a connected way without overlapping any targets. These are all (ill-specified) invariants of the dep graph, that it must be connected and it must not be ambiguous which rule to use to generate a particular target. No doubt there are other invariants that we should specify and check. > make gen (Graph rules []) = > case selectReadyRule rules of > (rule, rules') -> do > dyntargets <- action rule > make gen (Graph (markCompletedDeps (targets rule) dyntargets rules') > (newDeps dyntargets rules')) Ok, now for the interesting and slightly more tricky bit. If we are done for the moment expanding the graph then we turn to reducing the graph by running actions. > selectReadyRule :: [Rule m] -> (Rule m, [Rule m]) > selectReadyRule rules = > case partition (null . depends) rules of > (rule:rules',rules'') -> (rule, rules' ++ rules'') > ([], _) -> error "selectReadyRule: no rules ready to go!?!" We first of all select a rule that is read to be run. A rule is ready if it has no remaining dependencies. This should always be the case if the graph is not empty and has no dangling dependencies and is not cyclic. This property is something our graph invariants should tell us. Note that this is where we have the opportunity for parallelism as we can pick all the rules with no dependencies. For the moment we will just pick one rule. We run the rule's action and get back a list of targets. I mentioned earlier that these targets are to do with the way we've chosen to implement dynamic dependencies. Lets look at that in more detail now. It is perhaps best explained with an example, in pseudo-make syntax: foo.o foo.hi : foo.hs $(include foo.dep) ghc -c foo.hs foo.dep : foo.hs gatherdeps foo.hs > foo.dep What we mean by this is that foo.dep will contain the dependencies of foo.hs that we discover by reading foo.hs and looking at it's imports. They are dynamic in the sense that we can only discover the dependencies once foo.hs exists, since it may be generated by a pre-processor. Then we say that foo.o foo.hi depend on all the dependencies given in foo.dep, which of course requires foo.dep to be up to date. So what happens while we're reducing this graph is that we replace the dependency foo.dep with it's contents. We do that when we run the action that brings foo.dep up to date. So let us express the above rules in our notation: r1 = Rule { targets = ["foo.o", "foo.hi"], depends = ["foo.hs", "foo.dep"], action = ghc "foo.hs" } r2 = Rule { targets = ["foo.dep"], depends = ["foo.hs"], action = return ["bar.hi"] } Obviously the dep action would not be a constant like that but would read the .hs file to find the imports, and cache them in the .dep file. But it would return them all in a list like that. Recalling where we were in our make algorithm... > make gen (Graph rules []) = > case selectReadyRule rules of > (rule, rules') -> do > dyntargets <- action rule > make gen (Graph (markCompletedDeps (targets rule) dyntargets rules') > (newDeps dyntargets rules')) So we have run the rules action which means we can assume the rule's targets are up to date. So we can go and cross off all those targets where they appear as dependencies in other rules. In addition if this action returned any dependencies then we have to go insert those in replacement of the completed target. > markCompletedDeps :: [Target] -> [Target] -> [Rule m] -> [Rule m] > markCompletedDeps targets dyntargets = map updateDepends > where updateDepends rule@Rule { depends = ds } > | length ds == length ds' = rule > | otherwise = rule { depends = dyntargets ++ ds' } > where ds' = ds \\ targets So we do that just by going through all the rules and for each one checking if any of the completed targets occurred in that rule's depends list. If it did we also insert any dynamic targets that the action may have produced. > newDeps :: [Target] -> [Rule m] -> [Target] > newDeps ts rules = [ t | t <- ts, all (\rule -> t `notElem` targets rule) rules ] These dynamic targets may also in fact be completely new targets which were not already in the graph. If this is the case then we have to find out which ones are new and add the to the list of unresolved targets in the Graph before calling make again. In that case we would go back to expanding the graph. So that's it. That's the make algorithm. Now lets try it out... We're going to implement a monad to use make on. It's going to be a simulation of a file system rather than a monad that does real IO and uses the real file system. This will make testing easier. > data State = State { > currentTime :: Timestamp, > filesystem :: FileSystem > } The monad will be a state monad with the state consisting of the current state of the filesystem and the current time. > type FileSystem = Map.Map FilePath File > type Timestamp = Int A filesystem is just a map of paths to files and a timestamp is just an integer. The time will increase monotonically with each interesting action on the filesystem. > data File = File { > timestamp :: Timestamp, > content :: [String] > } > deriving Show A file has a timestamp of when it was created or last modified and it has content which we model as just a list of strings. For different kinds of files we will interpret theses strings differently. > emptyState = State 1 Map.empty The initial state is an empty filesystem at time 1. > instance Show State where > show (State _ tr) = > '\n' : (unlines $ map show $ Map.keys tr) We can show states, just listing the paths in the filesystem. > type Trace = [(Action, State)] It will be useful in specifying the behaviour of make to have specifications that can quantify over all actions in the execution. So we have a Trace which is a history of all actions and the state of the system at the time of that action. > newtype Make a = Make { unMake :: WriterT Trace (Monad.State State) a } > deriving (Functor, Monad, MonadState State, MonadWriter Trace) Our Make monad is then a state monad with the current state and it's also a Writer monad producing the Trace. > runMake :: Make () -> Trace > runMake = > snd . fst . flip runState emptyState . runWriterT . unMake Running an action in this monad gives us the action's Trace. > data Action = Stat FilePath > | ReadFile FilePath > | WriteFile FilePath [String] > deriving Show The actions we can get in a trace are all primitives actions on the filesystem. We distinguish stats from file reads though it is not yet clear if this is necessary. > log :: Action -> Make () > log action = do > state <- get > tell [(action, state)] We can insert actions into the log. > stat :: FilePath -> Make (Maybe Timestamp) > stat path = do > log (Stat path) > State _ filesystem <- get > case Map.lookup path filesystem of > Just file -> return (Just (timestamp file)) > Nothing -> return Nothing Stating a file tests for its existance. If it does exist we get back the file's timestamp but not it's content. > readFile :: FilePath -> Make File > readFile path = do > log (ReadFile path) > State _ filesystem <- get > case Map.lookup path filesystem of > Just file -> return file > Nothing -> fail $ "file does not exist: " ++ path Reading a file gets the File, that is the timestamp and the content. > writeFile :: FilePath -> [String] -> Make () > writeFile path content = do > log (WriteFile path content) > State curtime filesystem <- get > let file = File curtime content > put $ State (curtime + 1) (Map.insert path file filesystem) To write a file you supply the content and the timestamp gets set as the current time. The world timestamp gets incremented at this point. > exists :: FilePath -> Make Bool > exists path = isJust <$> stat path > touch :: FilePath -> Make () > touch path = writeFile path [] A couple convenience functions in terms of the primitives. So now we can write some programs that run in our Make monad and interact with our filesystem. > ghc :: FilePath -> Make () > ghc file = > case splitExtension file of > (baseFile,".hs") -> do > File _ imports <- readFile file > mapM_ readFile (map (<.> "hi") imports) > mapM_ touch $ map (baseFile <.>) ["o", "hi"] > _ -> fail $ "ghc didn't get a .hs file: " ++ file You never realised ghc was so simple eh? So it reads the .hs file. We interpret the content of the .hs file to be a list of module imports. So we map those to paths of .hi files and go and read the .hi files of the imported modules. Finally we write the output .o and .hi files. So we'll want to try it out with our make code. Let's construct an example with a couple modules, a main module foo.hs which imports module Bar from Bar.hs. > test = do > writeFile "foo.hs" ["Bar"] > touch "Bar.hs" > make gen (Graph [] ["foo.hi"]) So there we have it, foo.hs imports Bar and Bar.hs imports nothing. We then want to make foo.hi which will involve building both modules. Of course we need our dependency generator function gen and some rule schema that say how to build .hi files. > gen :: DepGenerator Make > gen target > | ext == ".hi" = return [hiFileRuleSchema file > ,depFileRuleSchema file > ,fileExistsRuleSchema (file <.> "hs")] > > where (file, ext) = splitExtension target For this example we can assume we only have to find rules to cover .hi files. We'll generate three rules, one to compile the .hi and .o file from the .hs file. Though it also depends on the .dep file. So we need another to generate the .dep from the .hs. Finally we need a rule that generates the .hs from nothing. > fileExistsRuleSchema :: FilePath -> Rule Make > fileExistsRuleSchema file = Rule { > targets = [file], > depends = [], > action = readFile file >> return [] > } Our system does not allow dangling dependencies, so even the simple case of a source file needs a rule. Of course it is a trivial rule as it has no depends. The action asserts that the file exists and does not generate any dynamic dependencies. Note that an alternative system to caching deps in using dep files would be for the .hs rule to return the dynamic deps directly. > hiFileRuleSchema :: FilePath -> Rule Make > hiFileRuleSchema file = Rule { > targets = [file <.> "hi", file <.> "o"], > depends = [file <.> "hs", file <.> "dep"], > action = ghc (file <.> "hs") >> return [] > } So the .hi .o rule runs ghc on the .hs file. > depFileRuleSchema :: FilePath -> Rule Make > depFileRuleSchema file = Rule { > targets = [file <.> "dep"], > depends = [file <.> "hs"], > action = do > yep <- exists (file <.> "dep") > if yep > then do File _ deps <- readFile (file <.> "dep") > return deps > else do File _ imports <- readFile (file <.> "hs") > let deps = map (<.> "hi") imports > writeFile (file <.> "dep") deps > return deps > } The .dep rule makes the .dep file from the .hs file. The action is a bit more complicated. Actually I now notice that it's wrong :-). It's caching the dependencies of the .hs file in the .dep file, but it is not re-reading the dependencies if the cached ones are stale. However, in either case it returns the list of dependencies. These get substituted for the *.dep file in the hi rule. So that's the whole system. Here's the result of evaluating > map fst (runMake test) WriteFile "foo.hs" ["Bar"] WriteFile "Bar.hs" [] ReadFile "foo.hs" Stat "foo.dep" ReadFile "foo.hs" WriteFile "foo.dep" ["Bar.hi"] ReadFile "Bar.hs" Stat "Bar.dep" ReadFile "Bar.hs" WriteFile "Bar.dep" [] ReadFile "Bar.hs" WriteFile "Bar.o" [] WriteFile "Bar.hi" [] ReadFile "foo.hs" ReadFile "Bar.hi" WriteFile "foo.o" [] WriteFile "foo.hi" [] The trace shows us discovering the dependencies of foo.hs and going and compiling Bar.hs. We could also inspect the state of the filesystem at the end, or indeed at any intermediate stage. So what's next... We want to write some specifications, both of what make should do and some internal invariants about the graph. For example what specification would detect the bug above about .dep files not being re-generated when they are stale? We should also be able to make performance specifications like saying that (any >> make >> make) extends the trace of (any >> make) with just file stats/reads and no writes. This should correspond to saying that make should be able to find out quickly that there is nothing to be done in a built tree just by checking timestamps and cached deps. We've not actually implemented checking if actions need to run or if they are already up to date with respect to their dependencies. This can be integrated into a smart Rule constructor that wraps the action with checks on the timestamps of the target and depends. It probably does not need to be integrated into the core make algorithm, it can just blindly run actions (though if we want to allow continuous builds we will have to revisit this). Again we ought to be able to construct specifications that check that we do not needlessly run actions but only do them when targets are out of date with respect to their dependencies. We should construct well known tricky examples, like search path shadowing problems and make sure our specifications catch such bugs. We currently have not looked at search paths. This makes things harder and probably requires a distinction between resolved and unresolved Targets. We currently do not cope with extra targets that we discover dynamically, like _stub.c files that pop out of compilation. We will have to track these as we at least have to link _stub.o files and in the case of c2hs producing .c files we may have to do further actions to compile them too. Then there's .hi boot files and what to do when we detect cycles. Lots of fun. So we'd appreciate comments, review and especially help with specifying, building and verifying a more realistic model before we move to an implementation using IO and invoking real programs. darcs get http://haskell.org/~duncan/cabal/dep-experiment/ patches gladly accepted. Of course there are existing systems which solve similar problems and people have had goes already at implementing dep analysis directly in Cabal. We don't want to ignore existing stuff but it's important we have something we can all understand and specify and verify. And extend. It's a tall order. Duncan

6 11

[Hackage] #184: cabal-install should report build results to hackage server
by Hackage 15 Aug '08

15 Aug '08

#184: cabal-install should report build results to hackage server ----------------------------+----------------------------------------------- Reporter: duncan | Owner: Type: enhancement | Status: new Priority: normal | Milestone: Component: cabal-install | Version: 1.2.2.0 Severity: normal | Keywords: Difficulty: normal | Ghcversion: 6.4.2 Platform: Linux | ----------------------------+----------------------------------------------- One way to get a lot more testing data on hackage packages is if cabal- install could report back to the hackage server about build successes and failures. This information should be kept by hackage and used to distill information about which platforms configurations a package builds successfully and which it does not. This should provide useful information to developers to enable them to discover problems more quickly and useful information to users. An important consideration is privacy. Users should always have the option to not report anything and any information they report that is kept should not contain identifying information. This is particularly important when it comes to build logs which may contain paths etc. It should be clear to users what information they are reporting to the can decide for themselves if it meets their privacy needs. Since cabal can be used to build private code it is vital that it reports only on packages that were obtained from the hackage server. It is also vital that the information is sent to the correct hackage server. It's possible to set up private hackage server instances and it'd may be useful to collect build information "in house" too. So what information would be helpful? * build success or failure (qualified by what failed, build, docs, tests) * package name and version * hash of .cabal file just to make sure it's the same one we're all talking about and to detect local modifications. * precise versions of dependent packages that the package was built with. * os and arch strings * compiler flavour and version * versions of important build tools * In the case of a build failure, some part of the build log would be helpful. This is the most problematic part from a privacy point of view. This is quite a bit of raw data and we can expect to have many hundreds of such reports for popular packages. How can we distill useful information from this kind of data? I expect we would want to do some statistical analysis where we look for common traits in the results. The data forms a multi-dimensional space of boolean values. By looking down the rows/columns of this space we should be able to identify trends. For example if it always fails with ghc-6.8.x, or always fails on Windows. Excluding those obvious failures we then may be able to say that it does work on mac osx (except ghc-6.8.1) or that it does work with regex-posix-0.92 (except on windows), etc. This should allow us to give a summary saying yes/no to various properties of the environment. For failure cases developers should be able to get access to the more detailed data including build logs. -- Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/184> Hackage <http://haskell.org/cabal/> Hackage: Cabal and related projects

1 6