brainstorming ways to stabalize Cabal interface?
Greetings. I'm keen to try to stabalize more of the Cabal interface, but I'm not sure the best way to go about that. I started a wiki page to start to document what is stable and what is not. Eventually this should move into the end-user documentation: http://hackage.haskell.org/trac/hackage/wiki/StableInterfaces Does anyone have clever ideas for how to go about pushing these interfaces toward stability? Do we just need to declare that certain things are unstable for the near-term (and possibly implement the eternal compatibility in theory idea[1]) and continue to experiment with them until they seem done enough? Do we stare at them very hard? Below[2] are my thoughts about what we _know_ to be stable, and the types for the new hooks interface. peace, isaac [1] http://www.haskell.org/tmrwiki/EternalCompatibilityInTheory [2] Here's a summary of the bits of interface that need to be stabalized, or are already stabalized. Stable Commands * ./setup configure o --prefix o --user o --ghc, --hugs o --verbose * ./setup build * ./setup install Unstable Commands * everything else. well, they're really _pretty_ stable. Stable functions * defaultMain * defaultMainWithHooks defaultUserHooks o but regular defaultMainWithHooks isn't stable since userHooks changes. Unstable functions * Various utility functions * UserHooks runTests :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO ExitCode, -- ^Used for @.\/setup test@ readDesc :: IO (Maybe PackageDescription), -- ^Read the description file hookedPreProcessors :: [ PPSuffixHandler ], -- ^Custom preprocessors in addition to and overriding 'knownSuffixHandlers'. hookedPrograms :: [Program], -- ^These programs are detected at configure time. Arguments for them are added to the configure command. -- |Hook to run before configure command preConf :: Args -> ConfigFlags -> IO HookedBuildInfo, -- |Over-ride this hook to get different behavior during configure. confHook :: PackageDescription -> ConfigFlags -> IO LocalBuildInfo, -- |Hook to run after configure command postConf :: Args -> ConfigFlags -> PackageDescription -> LocalBuildInfo -> IO ExitCode, ... * Everything else peace, isaac
On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote:
Greetings. I'm keen to try to stabalize more of the Cabal interface, but I'm not sure the best way to go about that. I started a wiki page to start to document what is stable and what is not. Eventually this should move into the end-user documentation:
http://hackage.haskell.org/trac/hackage/wiki/StableInterfaces
Does anyone have clever ideas for how to go about pushing these interfaces toward stability? Do we just need to declare that certain things are unstable for the near-term (and possibly implement the eternal compatibility in theory idea[1]) and continue to experiment with them until they seem done enough? Do we stare at them very hard?
An idea I've suggested before: Add another (thin) layer. The problem as many people have noted is that Haskell source code in the form of Setup.lhs is quite brittle in the face of changes to the Cabal API and the .cabal file format. Often the first sign of trouble is that Setup.lhs just doesn't compile or if it does it complains that it cannot parse the .cabal file because there is a new field being used that the current Cabal version being used doesn't grok. Neither of these situations provide good error messages or any suggestion to the user of how to proceed. So my suggestion is that we add a thin layer over the current user interface: runhaskell Setup.lhs <command> instead we should use a little *program*, lets call it "hs-pkg" for now: hs-pkg <command> These commands should be the same as for Cabal's Setup.lhs. This program will basically deffer to the existing mechanisms to compile the package but it will be able to do many more checks than the user can. It can read the .cabal file and discover which API version of the Cabal package is required. For the case of ghc it can then check that the required version of Cabal is actually installed and invoke runghc specifying the exact version required. It can do something about the issue of multiple compilers/interpreters and the runhaskell program (that John Mecham has pointed out before). Existing systems already do similar things and are more reliable as a consequence. For Gentoo distro-packages of Cabal packages it doesn't just run runhaskell and hope that the right packages are currently exposed. It invokes ghc with the right -package Cabal-1.1.x flag. This allows it to deal with packages that require different versions of the Cabal library. As a side note, this also allows the Setup.lhs file to be left out entirely when it just does the default stuff, since the hs-pkg program can deal it. Duncan
Duncan Coutts wrote:
On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote:
Greetings. I'm keen to try to stabalize more of the Cabal interface, but I'm not sure the best way to go about that. I started a wiki page to start to document what is stable and what is not. Eventually this should move into the end-user documentation:
http://hackage.haskell.org/trac/hackage/wiki/StableInterfaces
Does anyone have clever ideas for how to go about pushing these interfaces toward stability? Do we just need to declare that certain things are unstable for the near-term (and possibly implement the eternal compatibility in theory idea[1]) and continue to experiment with them until they seem done enough? Do we stare at them very hard?
An idea I've suggested before:
Add another (thin) layer.
[ good idea deleted ] So there are basically three approaches we can take to mitigating the backwards compatibility problem with Cabal. (1) Just declare what we claim to be stable, and try to get people to update their packages as necessary (2) EternalCompatibilityInTheory for Cabal only (3) Provide multiple versions of the Cabal package, with a wrapper program to invoke Setup.lhs as Duncan suggests I think we should do at least 2 or 3, because 1 is likely to cause too much pain for end-users. Many packages will need to use "unstable" parts of the interface just to work properly, so it's unfair to make them second-class citizens. As far as possible, we should ensure that packages keep working for as long as we can support them. In the past I suggested 2, but I think I'd be equally (or perhaps more) happy with 3. To compare 2 and 3: - it seems a bit strange to use ETC just for Cabal, but I don't support doing it for everything (it's too heavyweight a solution) - 3 requires us to provide multiple versions of the Cabal package, rather than one version that supports multiple interfaces. This is perhaps less work for the Cabal developers (keeping the old interfaces working is easier, we just compile the old code), but it means more bloat. - 3 lets us do away with Setup.hs when it is just boilerplate - 3 means an upheaval in the end-user's view of Cabal. They have to run "cabal build" (or whatever) instead of "runhaskell Setup.hs build" - the wrapper program can transparently compile Setup.hs instead of interpreting it each time, which is a lot faster I think I'm coming down on the side of Duncan's suggestion, despite the fact that it's a pretty big interface change. We could migrate slowly by making 'runhaskell Setup.hs' emit a warning for the time being. Cheers, Simon
On Thu, 2006-01-19 at 10:50 +0000, Simon Marlow wrote:
In the past I suggested 2, but I think I'd be equally (or perhaps more) happy with 3. To compare 2 and 3:
- it seems a bit strange to use ETC just for Cabal, but I don't support doing it for everything (it's too heavyweight a solution)
- 3 requires us to provide multiple versions of the Cabal package, rather than one version that supports multiple interfaces. This is perhaps less work for the Cabal developers (keeping the old interfaces working is easier, we just compile the old code), but it means more bloat.
Though people only need the versions that are needed by the packages they install. These can be dragged in on demand by distro package managers. For the manual case the "cabal" wrapper program tells the user what they need to install.
- 3 lets us do away with Setup.hs when it is just boilerplate
- 3 means an upheaval in the end-user's view of Cabal. They have to run "cabal build" (or whatever) instead of "runhaskell Setup.hs build"
- the wrapper program can transparently compile Setup.hs instead of interpreting it each time, which is a lot faster
Yeah, the Gentoo Cabal system does this.
I think I'm coming down on the side of Duncan's suggestion, despite the fact that it's a pretty big interface change. We could migrate slowly by making 'runhaskell Setup.hs' emit a warning for the time being.
We've had a few interface changes and inconsistencies before and generally in practise: * At one time people were supposed to use ./Setup.lhs and the Setup.lhs file was supposed to invoke runhaskell via the #! method. * Some packages supply Setup.hs rather than Setup.lhs. * Some packages supply no Setup.(l)hs at all. * Some instructions suggest to use runghc Setup.lhs rather than runhaskell Setup.lhs on the basis that one is more reliable. By having package developers and users using the same "cabal" script should help reduce these and other inconsistencies. For example some packages rely on optional packages for their Setup.lhs script itself! For example a Setup.lhs file that uses modules from the FilePath package. This is something that happens to work on the developers machine but will not work on the users machine if the user does not have the package. This is true even if the developer lists the FilePath package in their .cabal file, since we can't run the setup program to know that we need the FilePath package! A "cabal" wrapper program could stop the developer from making that kind of mistake by compiling the Setup.lhs using --ignore-all-packages (or whatever the ghc flag is) so that it does not depend on any packages other that base and haskell98. One other change that would be good to support this is to specify in a new field in the .cabal file the version of the Cabal API that is required. It can sort-of be done at the moment by using the build-depends, but this will cause the resulting library to depend on Cabal. If we take this route, I think it'd be good to address John Meacham's query about who provides cabal and the wrapper script/prog. Is it the compilers or something separate? For example, John asked what happens if runhaskell points at jhc (assuming jhc can even provide such a thing) will that work if the user is trying to install a package for ghc? Perhaps this is something that the script can sort out. Perhaps it should use the implementation being targeted to compile/interpret the Setup.(l)hs file. Duncan
As to Duncan's first mail, I am becomming more inclined to agree on an
official wrapper script (which is why I started the cabal-install
effort).
Duncan Coutts
We've had a few interface changes and inconsistencies before and generally in practise: * At one time people were supposed to use ./Setup.lhs and the Setup.lhs file was supposed to invoke runhaskell via the #! method. * Some packages supply Setup.hs rather than Setup.lhs. * Some packages supply no Setup.(l)hs at all. * Some instructions suggest to use runghc Setup.lhs rather than runhaskell Setup.lhs on the basis that one is more reliable.
I can undertand how these have been a problem, and it would be good if we could enforce consistency or write tools to make consistency less important. BTW, these don't represent any interface changes, just people failing to conform to the standards. The story has always been "Provide a Setup.hs or Setup.lhs that the end user compilers or interprets somehow." John M. Says:
yes, yes, a thousand times yes. cabal needs to be a program instead of or in addition to a library. probably in addition to since the library certainly has a lot of useful functionality.
Some of the reasons that I still have resisted makeing cabal into a program are that I've been hoping that cabal-get would make that less necessary, but cabal-get is not yet ready. I've also been hoping that runhaskell would become ubiquitous, but it hasn't. But if I agree to this, you have to get it going for JHC ;) peace, isaac
On Thu, Jan 19, 2006 at 01:59:01PM -0800, Isaac Jones wrote:
But if I agree to this, you have to get it going for JHC ;)
certainly, although, rather than make jhc another special case in cabal, I'd rather work on making a general compiler framework for it so that jhc can just drop a file describing its interface in /usr/share/lib/cabal/compilers/jhc.cabal-compiler or something and cabal will automatically be able to use it. Ideally, one would not have to upgrade their cabal just because they install (or write) a new haskell compiler. I think all compilers conform to one of hmake-like: ghc --make, jhc, nhc + hmake interpreter-like: hugs gcc-like: ghc, nhc98 so a compiler declaration file would not have to be much more complicated than a string telling it how to invoke the compiler and a mapping of various extensions/cabal options -> compiler flags. I'd also like to do something like this for preprocessors, which would be a much simpler project so will probably do first. incidentally, could we get cabal to ignore any field starting with 'x-' as a user defined extension? I'd like to use locally things like x-publish-site: /home/john/public_html/... x-darcs-repo: http://repetae.net/repos/jhc or experimental things like x-jhc-namespace: 0x220 without cabal getting huffy about unknown fields. obviously any popular and generally useful ones would eventually be standardized and the x- can be discarded. this is a fairly standard convention among file formats and is used in mime types too. John -- John Meacham - ⑆repetae.net⑆john⑈
yes, yes, a thousand times yes. cabal needs to be a program instead of or in addition to a library. probably in addition to since the library certainly has a lot of useful functionality. John -- John Meacham - ⑆repetae.net⑆john⑈
On Thu, Jan 19, 2006 at 09:35:32AM +0000, Duncan Coutts wrote:
I'm keen to try to stabilize more of the Cabal interface, [...] The problem as many people have noted is that Haskell source code in the
On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote: form of Setup.lhs is quite brittle in the face of changes to the Cabal API and the .cabal file format.
Indeed. I think the best hope for stability is to make setup scripts optional, and to seek to reduce the situations where they're needed. The vast majority of scripts are main = defaultMain or main = defaultMainWithHooks defaultUserHooks (Incidentally hackage and packages repository contain very few exceptions to this; it seems Gentoo has a much larger collection.) If we were to rely on an external tool like Duncan's hs-pkg, we'd need a field saying which of these (or future ones) to use. We'd also need version numbers for the file format, which might be expected to change less often than Cabal package versions. Reducing the need for setup scripts involves identifying the missing bits people are working around and implementing general facilities. Eliminating the boilerplate scripts will help, if only by flagging the packages of interest. If we're happy with the idea of a wrapper script running another Haskell program, we could also have a different optional program for each function, instead of always calling setup -- a bit like hooks at the program level. This would make it easier to see what special treatment a package needs, and for the wrapper to do the simple thing in other cases. For example, a package might need a hook for testing, so that testing might be brittle, but everything else could be handled by the wrapper. I imagine that major changes to the interface weren't what Isaac had in mind when he asked for stabilization ideas, but I think downplaying Setup.lhs is the best bet for longer term stability.
Ross Paterson
On Thu, Jan 19, 2006 at 09:35:32AM +0000, Duncan Coutts wrote:
I'm keen to try to stabilize more of the Cabal interface, [...] The problem as many people have noted is that Haskell source code in the
On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote: form of Setup.lhs is quite brittle in the face of changes to the Cabal API and the .cabal file format.
Indeed. I think the best hope for stability is to make setup scripts optional, and to seek to reduce the situations where they're needed. The vast majority of scripts are
main = defaultMain or main = defaultMainWithHooks defaultUserHooks
Of course, the setup scripts are only a problem in the situations where they're needed, not in these two situations.
(Incidentally hackage and packages repository contain very few exceptions to this; it seems Gentoo has a much larger collection.) If we were to rely on an external tool like Duncan's hs-pkg, we'd need a field saying which of these (or future ones) to use. We'd also need version numbers for the file format, which might be expected to change less often than Cabal package versions.
Reducing the need for setup scripts involves identifying the missing bits people are working around and implementing general facilities. Eliminating the boilerplate scripts will help, if only by flagging the packages of interest.
The missing bits of cabal? I disagree. I don't think we should eliminate the setup scripts. There will always be special things that folks need in their build system, and they will want a general facility for them, such as the setup script or the multiple scripts you mentioned. I don't think that continuing to make cabal more complex is the answer. There are definitely some things it should do that it doesn't do now, but if we abandon the setup scripts, then we will have to always put everything into cabal itself. We should instead try to stabalize the cabal interface and put the complexity into the layered tools. Stable hooks should solve the problem of the Setup scripts not compiling, and having a cabal-version field and a more flexible parser should solve the problem of new or unknown fields. We've done hardly any work in these directions. I think we should at least try these approaches. peace, isaac
participants (5)
-
Duncan Coutts -
Isaac Jones -
John Meacham -
Ross Paterson -
Simon Marlow