On Wed, 2008-08-27 at 06:13 -0700, John Meacham wrote:
The problem with the way cabal wants to mix with make/autoconf is that it is the wrong way round. make is very good at managing pre-processors, dependency tracking and calling external programs in the right order, in parallel, and as needed. cabal is generally good at building a single library or executable given relatively straightforward haskell source. (I know it _can_ do more, but this is mainly what it is good at).
The way this should work is that make determines what haskell libraries need to be built, and what haskell files need to be generated to allow cabal to run and calls cabal to build just the ones needed. cabal as a build tool that make calls is much more flexible and in tune with each tools capabilities.
I'd say if you're using make for all that, then use it to build the haskell modules too. That gives the advantage of incremental and parallel builds, which Cabal does not do yet (though we've got a GSoC project just coming to an end which does this).
The other issue is with cabal files themselves which are somewhat conflicted in purpose. on one hand, you have declarative stuff about a package. name, version, etc... information you want before you start to build something. but then you have build-depends, which is something that you cannot know until after your configuration manager (whatever it may be, autoconf being a popular one) is run.
Ah, but that's where the autoconf and Cabal models part ways.
What packages you depend on are going to depend on things like what compiler you have installed, your configuration options, which packages are installed, what operating system you are running on, which kernel version you are running, which c libraries you have installed. etc. things that cannot be predicted before the configuration is actually run.
So Cabal takes the view that the relationship between features and dependencies should be declarative. autoconf is essentially a function from a platform environment to maybe a configuration. That's a very flexible approach, the function is opaque and can do whatever feature tests it likes. The downside is that it is not possible to work out what the dependencies are. It might be able to if autoconf explained the result of its decisions, but even then, it's not possible to work out what dependencies are required to get a particular feature enabled. With the Cabal approach these things are explicit. The conditionals in a .cabal file can be read in either direction so it is possible for a package manager to automatically work out what deps would be needed for that optional libcurl feature, or GUI. The other principle is that the packager, the environment is in control over what things the package 'sees'. With autoconf, the script can take into account anything it likes, even if you'd rather it did not. Eg it's important to be able to build a package that does not have that optional dependency, even though the C lib is indeed installed on the build machine, because I may be configuring it for a machine without the C lib. Sure, some good packages allow those automagic decisions to be overridden, but many don't and of course there is no easy way to tell if it's picking up deps it should not. So one of the principles in Cabal configuration is that all decisions about how to configure the package are transparent to the packager and can be overridden. Now currently, Cabal only has a partial implementation of the concept because when it tries to find a configuration that works in the current environment (which it only does if the configuration is not already fully specified by the packager) it only considers dependencies on haskell packages. Obviously there are a range of other dependencies specified in the .cabal file and it should use them all, in particular external C libs. So I accept that we do not yet cover the range of configuration choices that are needed by the more complex packages (cf darcs), but I think that we can and that the approach is basically sound. The fact that we can automatically generate distro packages for hundreds of packages is not insignificant. This is just not possible with the autoconf approach.
Then you have cabal as a packaging system (or perhaps hackage/cabal considered together). Which has its own warts, if it is meant to live in the niche of package managers such as rpm or deb, where are the 'release' version numbers that rpms and debs have for one example? If it is meant to be a tarball like format, where is the distinction between 'distribution' and 'source' tarballs?
Right, it's supposed to be the upstream release format, tarballs. Distro packages obviously have their additional revision numbers.
For instance, jhc from darcs for developers requires perl,ghc,DrIFT,pandoc,autotools, and happy. however the jhc tarball requires _only_ ghc. nothing else. This is because the make dist target is more interesting than just taring up the source. (and posthooks/prehooks don't really help. they are sort of equivalent to saying 'write your own build system'.)
Right. Cabal does that too (or strictly speaking, the Simple build system can do this). For pre-processors that are platform independent (like alex, happy etc) it puts the pre-processed source into the release tarball. It's also possible to make tarballs without the pre-generated files if it's important.
One of the biggest sources of conflict arise from using cabal as a configuration manager. A configuration managers entire purpose is to examine the system and figure out how to adapt your programs build to the system.
Well, that's the autoconf view. It's not the only way of looking at it as I explained above (perhaps not very clearly). I'd say a configuration manager should negotiate between the package and the packager/user/environment to find a configuration that is satisfactory to all (which requires information flow in both directions).
this is completely 100% at odds with the idea of users having to 'upgrade' cabal. Figuring out how to adapt your build to whatever cabal is installed or failing gracefully if you can't is exactly the job of the configuration manager. something like autoconf. This is why _users_ need not install autoconf, just developers. since autoconf generates a portable script is so that users are never told to upgrade their autoconf. if a developer wants to use new features, he gets the new autoconf and reruns 'autoreconf'. The user is never asked to update anything that isn't actually needed for the project itself. This distinction is key fora configuration manager and really conflicts with cabal wanting to also be a build system and package manager. It is also what is needed for forwards and backwards compatibility.
I suppose in principle it'd be possible to ship the build system in every package like autoconf/automake does. Perhaps we should allow that as an option. It's doable since the Setup.hs can import local modules.
All in all, I think these conflicting goals of cabal make it hard to use in projects and have led to very odd design choices. I think external tools should not be the exception but rather the rule. Not that cabal shouldn't come with a full set of said tools. But as long as they are integrated I don't see cabal's design problems being fixed, meerly augmented with various work-arounds.
One issue, with a pick and mix approach is what is the top level interface that users/package managers use? The current choice (which I'm not at all sure is the right one) is a Setup.hs file that imports its build system from a library that's already on the system (or a custom one implemented locally). So a system that uses make underneath still has to present the Setup.hs interface so that package managers can use it in a uniform way. You mention at the top that you think the make/cabal relationship is the wrong way round, but the Cabal/Setup.hs interface has to be the top level one (at least at the moment) so you'd have Setup.hs call make and make call it back again to build various bits like libs etc? Do you think that separating the Simple build system from the declarative part of Cabal would help? It'd make it more obvious that the build system part really is replaceable which currently is not so obvious since they're in the same package. I'm not averse to splitting them if it'd help. They're already completely partitioned internally. Duncan