On Fri, 2007-10-26 at 10:20 -0700, Isaac Potoczny-Jones wrote:
Duncan and others,
I have some high-level thoughts:
= Standalone Tool =
Something I've talked about for a long time (and a few times tried to get someone to implement) is a kind of EDSL for Setup files so that people can replace Make more easily by using the Setup files, and this seems like it could be a good start. You could use Parsec as a model for building a little language for dependencies and actions for those dependencies. Make is a mess because it's really hard to build any kind of abstractions using it; the main thing it does for you is the Make algorithm you've shown here, and everything else is really hard. Haskell would be much better at this :)
Aye, I've used make in Gtk2Hs and it's pretty good. The main thing that lets it down is the bad language, well languages. The shell language plus the limited functional language on top. That and make does not handle dynamic dependencies well. So doing make properly would be a good thing since the basic principle is sound. A Cabal configure/build EDSL for the make actions and indeed for most of Cabal would also be a good thing. Some people have given it some consideration. Doing that well should help to reduce complexity and code size in Cabal.
Cabal is getting more and more complex, and I think it would be good to consider how you might separate this from Cabal. Perhaps you could implement a make-a-like library that's pretty standalone, but would be designed for use by Cabal, and indeed, Cabal would probably use it internally, but it could also be used outside of the context of Cabal.
We do have to be careful about dependencies. And I think it might actually be good to integrate this into Cabal directly and change some of the existing imperative code in Cabal to work more in a dependency driven style. I would hope the make code would be reusable if we design it properly.
= Pros & Cons of Cabal Complexity =
I've tended to resist adding complexity like RPMs to Cabal because I tend to think that smaller, standalone tools that can be elegantly composed is better. The good part of composable standalone tools is that you can spread out the development effort among folks who are experts in their smaller tool. You also can keep Cabal from being so complicated that it's hard to bootstrap packages, and so that the changes between versions aren't as painful.
Keeping the core closer to what is needed to bootstrap is a good point. We can perhaps move some tasks to tools built on the Cabal library, like cabal-install. For example there's no need for the sdist feature to be in Cabal directly, we could move it into cabal-install, at least if we make cabal-install the primare command line UI that developers use. That way it can have more dependencies like, zlib and tar to be able to create and unpack .tar.gz file. I've also wondered if the command line UI could be completely separated, though that could make bootstrapping too hard. It could certainly be better separated internally in the Cabal library and that would make us think more clearly about what interface Cabal the library provides, rather than Cabal as a command line tool via Setup.hs.
The downside of keeping Cabal simple is that you'd sorta like it to be the case that Cabal is all you need to build any package. As packages and dependencies among packages get more complex (in part, because Cabal facilitates that, which is good) it's natural to make Cabal more complex as well to accommodate more and more needs of package authors.
My only point, really, is that you think about these trade-offs whenever adding new capabilities to Cabal. Would it be better to have a standalone tool that Cabal depends on?
As I say, I'd go the other way and have a 'cabal' command line tool that depends on the Cabal library and provides more convenience features to developers and users. The cabal-install program is going in that direction.
= Goals of Dependency Chasing =
I've been a bit out of the loop on this, so I apologize if I'm re-covering old ground.
A simpler goal of a module-chasing system is just to replace the need for other-modules. Other-modules is used in compilation (although ghc --make doesn't really need it) and for haddock and for making source tarballs, and probably for other things.
Yes. Note that although ghc --make does not need it, if you miss out any module from other-modules you'll get nasty link errors later. Much later, in a rather confusing way. So that'd be another thing we could fix.
Ideally, the user should be able to specify only "Exposed modules" for libraries and "main modules" for executables, and Cabal should derive the other-modules itself. That's less pain for the user, and you can get around using ghc --make. Well, using ghc --make isn't much of a problem, but other compilers without a --make flag could then be made to work. Maybe this is implemented already?
Dependency chasing is essential to support pre-processors properly, like c2hs, and also for compilers that do not implement their own dependency chasing. It's also important for being able to do parallel builds which will be increasingly important. I also think it'll make it easier for us to support building a collection of related packages in one go.
At first this sounds really easy, but then I wonder, how can Cabal know whether a particular module in an import list is a part of a dependency, or actually a part of the package? Can Cabal query ghc-pkg for that kind of information?
Yes.
Can Cabal assume that if it can find the module in the local directory tree that it is a part of the current package?
Yes. That's ghc's behaviour that local modules shadow package modules.
Can you think of another way to make this work?
We could do with making getting the information out of ghc-pkg more efficient by allowing us to get all the information in one call. Simon suggested something like ghc-pkg dump that just gives us all the info on all packages in one go.
= Overall =
Overall, this sounds really cool, and I encourage you all to keep at it! I trust you to decide if adding this kind of complexity to Cabal is worthwhile, and to build a really nice system. I really do think that Haskell should be able to do better than Make here, and it would be a pleasure to feel like we can replace bits of that toolchain :)
:-)
I'll try to read in more detail and give more feedback.
Great. Duncan