Hi all, SimonPJ asked me to summarise the problems I've been having with GHC's new build system with the idea that we could perhaps talk a bit about this during today's IRC meeting. The context is the Data Parallel Haskell project and in particular the dph packages which need a number of not quite straightforward things from the build system (this is only going to get worse in the future). A lot of the problems stem from Cabal's lack of extensibility. This means that whenever I need some new Cabal functionality for dph I have to ask Ian or Duncan to implement it. They both are very helpful but this just doesn't scale! IMO, at the moment Cabal is simply not mature enough for building complex projects. But wait, we do have those Setup.hs files which are customisable so why don't I just do what I want with them? Firstly, they aren't customisable enough. And secondly, I have no idea how to do what I want. The development version of Cabal changes on a daily basis and isn't documented. For instance, the black magic that Ian implemented in dph/dph/Setup.hs a month ago (and which I don't really understand) has been broken once already. Fortunately, I was able to fix that myself but if something more complex changes, I'll have to ask Ian again. In effect, I'm not able to properly maintain my own library! I don't think I understand how GHC itself is built any longer, either. What does cabal-bin do? What is runghc.wrapper? What is the difference between make.library.* and build.library.* and why do we need both (and why doesn't one of them work for dph)? It is important to have an idea how the build system works when hacking on a project but how do I find out? And finally, there is the problem of the HEAD being broken quite frequently. I have decided to pull as little as possible and it turns out that others have independently arrived at the same conclusion. This is bad! For instance, I would have spotted the OS X breakage much earlier if I pulled more frequently; now, I have to wade through a couple of weeks' worth of patches to find the culprit (that darcs unpull doesn't work in my repo doesn't help). IMO, the main issue is that we have two moving targets: the GHC build system and the tool it relies on (Cabal). Both change rapidly and there is just no chance, for me at least, to keep up. If I understand correctly, the situation is this: - Ultimately, Cabal will be used for building GHC. - At the moment, Cabal doesn't provide the necessary functionality. The current approach seems to be to gradually implement stuff in Cabal and to immediately use it in the build system. I think this just doesn't work for a highly collaborative project such as GHC. I have two suggestions how to improve things. 1. I think it would be better to implement, document and release all the necessary functionality in Cabal first and only then to modify GHC's build system to use this functionality. In effect, this means that the Cabal version used for building GHC would be frozen at the beginning of a development cycle and not at the end. Critical stuff could still be merged, perhaps, but that should happen very infrequently and be backwards compatible. 2. Ian, it would be great if you could add a couple of lines to build system patches explaining what they do and why. This could ultimately mean less work for you because I will bug you less often :-) Sorry for the long rant, I simply seem to spend too much time wrestling with the build system. Roman