ANNOUNCE: Coadjute 0.0.1, generic build tool

Announcing the release of Coadjute, version 0.0.1! Web site: http://iki.fi/matti.niemenmaa/coadjute/ Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Coadjute Coadjute is a generic build tool, intended as an easier to use and more portable replacement for make. It’s not tailored toward any particular language, and is not meant to replace tools which target a specific environment (such as ghc --make or Cabal, taking Haskell as an example). I've been sitting on this for a couple of months now and figured I might as well push it out since it seems to be in relative working order. I've used it on my web site since July and it hasn't resulted in data loss yet. Still, it's a 0.0 version so no guarantees. ;-) A bit of advertising: Portability is striven towards in two ways: * You don’t have to deal with the idiosyncrasies of many make implementations (well, people don’t, but they call their GNU Make files makefiles instead of GNUmakefiles, which causes misunderstandings). * You have Haskell at your disposal, and are encouraged to use that whenever possible instead of system-specific binaries like the POSIX commands we all know and love. Comes with support for: * Parallel task performing. * Advanced out-of-dateness detection: o Choice between timestamps and hashes. o Keeping track of what arguments have been passed. * Haskell! Have a look at the Haddock documentation for a few simple examples and do feel free to comment over at haskell-cafe—or privately, if that's your preference. Any kind of discussion is welcome!

2009/1/18 Matti Niemenmaa
Announcing the release of Coadjute, version 0.0.1!
Hi, trying to build on GHC 6.10.1 I get: Building regex-dfa-0.91... Text/Regex/DFA/Common.hs:6:7: Could not find module `Data.IntMap': it is a member of package containers-0.2.0.0, which is hidden cabal: Error: some packages failed to install: Coadjute-0.0.1 depends on regex-dfa-0.91 which failed to install. regex-dfa-0.91 failed during the building phase. The exception was: exit: ExitFailure 1
Web site: http://iki.fi/matti.niemenmaa/coadjute/ Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Coadjute
Coadjute is a generic build tool, intended as an easier to use and more portable replacement for make. It's not tailored toward any particular language, and is not meant to replace tools which target a specific environment (such as ghc --make or Cabal, taking Haskell as an example).
I've been sitting on this for a couple of months now and figured I might as well push it out since it seems to be in relative working order. I've used it on my web site since July and it hasn't resulted in data loss yet.
Still, it's a 0.0 version so no guarantees. ;-)
A bit of advertising:
Portability is striven towards in two ways: * You don't have to deal with the idiosyncrasies of many make implementations (well, people don't, but they call their GNU Make files makefiles instead of GNUmakefiles, which causes misunderstandings). * You have Haskell at your disposal, and are encouraged to use that whenever possible instead of system-specific binaries like the POSIX commands we all know and love.
Comes with support for: * Parallel task performing. * Advanced out-of-dateness detection: o Choice between timestamps and hashes. o Keeping track of what arguments have been passed. * Haskell!
Have a look at the Haddock documentation for a few simple examples and do feel free to comment over at haskell-cafe—or privately, if that's your preference. Any kind of discussion is welcome!
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Ilmari Vacklin

ilmari.vacklin:
2009/1/18 Matti Niemenmaa
: Announcing the release of Coadjute, version 0.0.1!
Hi,
trying to build on GHC 6.10.1 I get:
Building regex-dfa-0.91...
Text/Regex/DFA/Common.hs:6:7: Could not find module `Data.IntMap': it is a member of package containers-0.2.0.0, which is hidden cabal: Error: some packages failed to install: Coadjute-0.0.1 depends on regex-dfa-0.91 which failed to install. regex-dfa-0.91 failed during the building phase. The exception was: exit: ExitFailure 1
regex-dfa needs this patch, attached. Or use the native package for it on your distro ... :) -- Don

matti.niemenmaa+news:
Announcing the release of Coadjute, version 0.0.1!
Web site: http://iki.fi/matti.niemenmaa/coadjute/ Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Coadjute
Here's an Arch Linux package for it, http://aur.archlinux.org/packages.php?ID=23237

Matti Niemenmaa schrieb:
Announcing the release of Coadjute, version 0.0.1!
Web site: http://iki.fi/matti.niemenmaa/coadjute/ Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Coadjute
Coadjute is a generic build tool, intended as an easier to use and more portable replacement for make. It’s not tailored toward any particular language, and is not meant to replace tools which target a specific environment (such as ghc --make or Cabal, taking Haskell as an example).
I've been sitting on this for a couple of months now and figured I might as well push it out since it seems to be in relative working order. I've used it on my web site since July and it hasn't resulted in data loss yet.
How does it compare to http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hake

Henning Thielemann wrote:
Matti Niemenmaa schrieb:
Announcing the release of Coadjute, version 0.0.1!
Web site: http://iki.fi/matti.niemenmaa/coadjute/ Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Coadjute <snip> How does it compare to http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hake
Short answer: the question that comes to mind is "how does hake compare to make?" Coadjute seems to be more capable, in general, but then I don't know pretty much anything about hake. Somewhat longer answer: Coadjute is better in that: 1. hake's documentation is rather sparse. I have no idea what most functions do, or even what exactly the main program does. 2. hake doesn't seem to do parallel builds, but I'm not sure because of point 1. 3. Coadjute keeps track of command line arguments (see docs for details): for me this is really a killer feature, I don't know of anything else which does this. 4. hake always uses timestamps, Coadjute can use MD5 hashes as well. 5. Coadjute can have arbitrary path specifications, hake's rules seem to be based on file extensions only, thus applying only to the current directory. Peter Miller's "Recursive Make Considered Harmful" comes to mind: http://miller.emu.id.au/pmiller/books/rmch/ hake is better in that: 1. Its syntax is more concise. 2. It looks like it'd be easier to pick up for make users. 3. It seems to have some built-in support for more complex operations than Coadjute, e.g. the 'ruleSS' function. (Coadjute can still do this, just not as concisely. See point 1.) With the disclaimer that all I know about hake comes from its Hackage page and home page, which don't really tell me very much. I can't get any information out of the hake binary either: it doesn't respond to --help or similar. Much longer answer: I figured I'd convert the example there to Coadjute and see what happens. Result, with some inline comments: ---- import Coadjute import System.FilePath (replaceExtension) import System.FilePath.Glob (globDir, compile) import System.Process (rawSystem) main = do -- assuming that hake's rules apply to the current directory only... ([sfx1, sfx2, c, cc, foo],_) <- globDir (map compile ["*.sfx1","*.sfx2","*.c","*.cc","foo*"]) "." coadjute $ do -- Coadjute doesn't offer arbitrary source/target pairing -- currently: it wants you to derive your targets from your -- sources somehow instead of just specifying them directly ruleM' "greeting" (\_ _ -> run "linker" ["-o","greeting","hello.o","good-bye.o"]) (sourceToDatum (\_ -> ( ["good-bye.o"] , ["greeting","greeting.log"] )) ["hello.o"]) rule' ".sfx1 to .o" (buildO "compiler1") (sourceToDatum' (chExt "o") sfx1) rule' ".sfx2 to .o" (buildO "compiler2") (sourceToDatum' (chExt "o") sfx2) -- Things like "make clean" don't really map well to Coadjute -- currently... -- -- Not only can it not be specified nicely, but the rule is always -- applied unless we have it create some kind of dummy file rule' "clean" (\_ _ -> do run "rm" ["-f", "hello.o", "good-bye.o" , "greeting", "greeting.log"] run "touch" ["DUMMY"]) (sourceToDatum' (const "DUMMY") [""]) rule' ".c to .o" (\(s:_) _ -> run "gcc" ["-c", s]) (sourceToDatum' (chExt "o") c) rule' ".cc to .o" (\(s:_) _ -> run "g++" ["-c", s]) (sourceToDatum' (chExt "O") cc) -- No equivalent to hake's ruleSS: deal with C/C++ dichotomy -- yourself rule' "C++ .o to binaries" (buildO "g++") (sourceToDatum' (chExt "") $ map (chExt "o") cc) rule' "C .o to binaries" (buildO "gcc") (sourceToDatum' (chExt "") $ map (chExt "o") c) ruleM' "Not sure what this is" (\[s] (t:_) -> do gen <- readFile s writeFile t $ unlines $ [ "#!/bin/sh", "echo This is script" ] ++ lines gen) (sourceToDatum' (\s -> [replaceExtension s "gen", "Hakefile"]) foo) where chExt = flip replaceExtension run cmd args = rawSystem cmd args >> return () buildO compiler [s] t = run compiler [s,"-o",t] ---- I really don't know what that last rule (the one generated with the 'base' function in hake) is supposed to do. With no documentation I have absolutely no idea: my conversion above is my best guess, but I doubt I got it right. In any case, seems there are some things which hake's interface can do that Coadjute's can't: 1. Arbitrary source/target pairing. This was really a "d'oh" moment for me and is trivial to fix. I think I'll even remove the current sourceToDatum functions and just provide a primitive with which you can map, fold, whatever. 2. Rules which have no targets. Currently these can't work since I haven't yet implemented being able to choose which rules to apply: Coadjute just always runs them all. But this is definitely something on my agenda. Although I've also been thinking about alternative avenues: the main use case for these is probably the 'make clean' type of action, which I, at least, use mainly to bypass the problem that Coadjute's argument tracking already solves: wanting to rebuild with different flags passed to the compiler (or equivalent). And you can of course deal with this by simply doing it in the main function, but outside the Coadjute monad: when ("clean" `elem` userArgs) $ rawSystem "rm" ["-f", ...] Which is one reason why I'm not sure if it's necessary at all. 3. The ruleSS thing: being able to choose a build action based on what rule caused this rule to be built. Looks interesting, but I'm not sure how useful it is in practice, nor how it should be implemented in Coadjute. Another thing that seems clear is that Coadjute is more primitive. hake contains around 40 functions; Coadjute boils down to five, with three convenience functions. And from point 1 above, I'm thinking of removing two, leaving the counts at four and two. Originally I did intend for Coadjute to also offer all kinds of utility functions that could be useful in Adjutant files, but at some point I figured that it's better to provide a minimal, clean, extensible interface and leave the rest to libraries. My Pipe and Glob libraries began their lives as modules in Coadjute.Util. hake seems to provide alternatives by having functions with the same name in different modules: FunSetRaw is for raw commands, FunSetIO for arbitrary IO expressions. Coadjute provides only the latter and discourages the former as a design philosophy—we have Haskell: you should be able to do a lot more than what plain make can without having to call other programs, and you should try hard to avoid doing so. (Platform-agnostic programs with complicated, platform-specific build systems tick me off.) Anyway, hake looks interesting but it's not a replacement for Coadjute; and neither is Coadjute for hake. To be completely honest I'm not sure what use case hake is meant to solve: how does it improve over plain make?

Matti Niemenmaa schrieb:
Anyway, hake looks interesting but it's not a replacement for Coadjute; and neither is Coadjute for hake. To be completely honest I'm not sure what use case hake is meant to solve: how does it improve over plain make?
It seems to be programmable, and thus may better cope with LaTeX, where the simple 'make' philosophy fails. 'latex' must be called repeatedly, interleaved with 'bibtex' and 'mkindex'.

On 2009 Jan 18, at 13:47, Matti Niemenmaa wrote:
3. Coadjute keeps track of command line arguments (see docs for details): for me this is really a killer feature, I don't know of anything else which does this.
It's been done many times before; it never seems to catch on. My personal favorite was Shape (http://user.cs.tu-berlin.de/~shape/) which I used for a few local projects in the late 80s. SCons is perhaps the most popular tool in this class (and itself a Pythonization of the original Perl Cons; maybe it's time for HCons?), followed by Apache's Ant (I don't think that actually caches command lines or binaries though), then Jam and successors. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

Brandon S. Allbery KF8NH wrote:
On 2009 Jan 18, at 13:47, Matti Niemenmaa wrote:
3. Coadjute keeps track of command line arguments (see docs for details): for me this is really a killer feature, I don't know of anything else which does this.
It's been done many times before; it never seems to catch on. My personal favorite was Shape (http://user.cs.tu-berlin.de/~shape/) which I used for a few local projects in the late 80s. SCons is perhaps the most popular tool in this class (and itself a Pythonization of the original Perl Cons; maybe it's time for HCons?), followed by Apache's Ant (I don't think that actually caches command lines or binaries though), then Jam and successors.
Actually, you're right, and I misspoke: I know about SCons and didn't like it (see below for brief rant)—Coadjute is the only truly /generic/ build tool I knew of which does it. The main thing I'll mention that turned me off SCons was that it tries to do too much stuff automatically: it's very much built around the idea that you just give it a filename and it figures out what to do. E.g. "foo.hs" and it detects what files it depends on, what Haskell compilers you've got, picks one (arbitrarily?), and compiles all the sources into a binary. (On my Windows machine this meant fun delays of multiple seconds prior to every build, as it scanned all the directories in PATH for several binaries which it never found: looking for various C compilers while I haven't even specified C dependencies.) I found it quite difficult (verbose and non-obvious) and probably against its philosophies to just say "here is foo.hs, it depends on foo.c, don't do anything else with them and just run the following command if foo.hs needs to be built." To be completely honest I don't like the idea of SCons in itself: it seems to be something in between platform-specific tools like Cabal and generic ones like make or Coadjute. Maybe I'm just too dense, or maybe it's a documentation problem, but it's just not something I'd use for pretty much anything. (C/C++ don't really have tools geared specifically for them (CMake?) so that might be one case where I would.)

Excerpts from Matti Niemenmaa's message of Sun Jan 18 19:47:46 +0100 2009:
Henning Thielemann wrote:
Matti Niemenmaa schrieb:
Announcing the release of Coadjute, version 0.0.1!
Web site: http://iki.fi/matti.niemenmaa/coadjute/ Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Coadjute <snip> How does it compare to http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hake
Short answer: the question that comes to mind is "how does hake compare to make?" Coadjute seems to be more capable, in general, but then I don't know pretty much anything about hake.
Somewhat longer answer:
Coadjute is better in that: 1. hake's documentation is rather sparse. I have no idea what most functions do, or even what exactly the main program does. 2. hake doesn't seem to do parallel builds, but I'm not sure because of point 1. 3. Coadjute keeps track of command line arguments (see docs for details): for me this is really a killer feature, I don't know of anything else which does this.
ocamlbuild does this.
4. hake always uses timestamps, Coadjute can use MD5 hashes as well. 5. Coadjute can have arbitrary path specifications, hake's rules seem to be based on file extensions only, thus applying only to the current directory. Peter Miller's "Recursive Make Considered Harmful" comes to mind: http://miller.emu.id.au/pmiller/books/rmch/
[...] Moreover, it seems that Coadjute and ocamlbuild share a fair number of design choices, maybe having a look at it could be fruitful. Best regards, -- Nicolas Pouillard

Nicolas Pouillard wrote:
Excerpts from Matti Niemenmaa's message of Sun Jan 18 19:47:46 +0100 2009:
3. Coadjute keeps track of command line arguments (see docs for details): for me this is really a killer feature, I don't know of anything else which does this.
ocamlbuild does this.
4. hake always uses timestamps, Coadjute can use MD5 hashes as well. 5. Coadjute can have arbitrary path specifications, hake's rules seem to be based on file extensions only, thus applying only to the current directory. Peter Miller's "Recursive Make Considered Harmful" comes to mind: http://miller.emu.id.au/pmiller/books/rmch/
[...]
Moreover, it seems that Coadjute and ocamlbuild share a fair number of design choices, maybe having a look at it could be fruitful.
Thanks, I hadn't heard of that one. It seems to be somewhat oriented towards OCaml but I'll take a look.

2009/1/18 Matti Niemenmaa
Announcing the release of Coadjute, version 0.0.1! [...] Portability is striven towards in two ways:
Is it intended to work on Windows? (I don't want to spend time downloading and trying to set it up if it was never intended to be Windows-compatible.) Paul.

Paul Moore wrote:
2009/1/18 Matti Niemenmaa
: Announcing the release of Coadjute, version 0.0.1! [...] Portability is striven towards in two ways:
Is it intended to work on Windows? (I don't want to spend time downloading and trying to set it up if it was never intended to be Windows-compatible.)
Yes, all my software is unless I explicitly say it isn't. :-) In fact, I've both developed and used it mostly on Windows.
participants (7)
-
Brandon S. Allbery KF8NH
-
Don Stewart
-
Henning Thielemann
-
Ilmari Vacklin
-
Matti Niemenmaa
-
Nicolas Pouillard
-
Paul Moore