RE: cabal hooks interface (was: Cabal package description sytax)

On 09 December 2004 23:56, ross@soi.city.ac.uk wrote:
On Wed, Dec 08, 2004 at 10:42:26PM -0800, Isaac Jones wrote:
What about this interface for user-supplied command hooks. [...]
For each of the pre-functions, a Maybe PackageDescription is returned. If Nothing, then it gets the description from the file or the default description you passed in. If (Just p), then it uses the description returned by the pre-functions.
I'm not sure what's going on here. It seems only preConf needs to return anything, and that shouldn't be the whole PackageDescription (since you want part of that static), just the build information.
I don't particularly like the idea of: {{system "runhaskell" ["postinst.hs"]}} because it seems like a less elegant IO ExitCode. runhaskell doesn't exist, and it relies on more moving parts.
Sure it exists -- I read about it in the spec.
Ross's solution lets you put the postinst code in the setup script itself, or optionally in an external script if you prefer. This seems more flexible to me. Also, there doesn't seem much point in doing "runhaskell postinst.hs", you might as well just run "./postinst.hs" or "./postinst". Cheers, Simon

"Simon Marlow"
On 09 December 2004 23:56, ross@soi.city.ac.uk wrote:
On Wed, Dec 08, 2004 at 10:42:26PM -0800, Isaac Jones wrote:
What about this interface for user-supplied command hooks. [...]
For each of the pre-functions, a Maybe PackageDescription is returned. If Nothing, then it gets the description from the file or the default description you passed in. If (Just p), then it uses the description returned by the pre-functions.
I'm not sure what's going on here. It seems only preConf needs to return anything, and that shouldn't be the whole PackageDescription (since you want part of that static), just the build information.
Each command needs access to the PackageDescription. The reason the hooks are like this is that we need to pass information between the commands via the file system. If the PreConf function were to return anything, it would be passed to configure, and all configure could do with it is write it to the configure file, which the future commands know how to read in, but that doesn't have the build information like the module list. So what the preConf step needs to do, is perhaps call autoconf on mySetup.description.in, and preBuild reads mySetup.description, likewise for preInstall. This is how we pass information from command to command. Isaac:
I don't particularly like the idea of: {{system "runhaskell" ["postinst.hs"]}} because it seems like a less elegant IO ExitCode. runhaskell doesn't exist, and it relies on more moving parts. Ross: Sure it exists -- I read about it in the spec.
Ross's solution lets you put the postinst code in the setup script itself, or optionally in an external script if you prefer. This seems more flexible to me.
The commands need a way to pass information between one-another. If we use 'system' to run external commands, then the only way to do that is to dump a file that cabal itself knows how to read back in, and cabal has to know where it is. If we rather use the preHooks, Angela can do whatever she wants to get the information between phases, so long as she returns a PackageDescription. If we use 'system' calls, then Cabal must specify the names of the scripts and the format and location of the files they output (or preHooks with an interface similar to mine to read them back in). If Cabal use the preHooks, then all we need to specify is the type of the functions. The preHooks let the user do the obvious thing, which is to read and write to a separate description file of their choosing, or to do something as strange as they like to get the package description. In any case, you can just say defaultUserFunctions{postInst=system "./postinst.hs">> readPackageDescription "thePlaceIKnowItWillEndUp"}. peace, isaac

On Fri, Dec 10, 2004 at 08:40:35AM -0800, Isaac Jones wrote:
The commands need a way to pass information between one-another. If we use 'system' to run external commands, then the only way to do that is to dump a file that cabal itself knows how to read back in, and cabal has to know where it is. If we rather use the preHooks, Angela can do whatever she wants to get the information between phases, so long as she returns a PackageDescription.
If we use 'system' calls, then Cabal must specify the names of the scripts and the format and location of the files they output (or preHooks with an interface similar to mine to read them back in). If Cabal use the preHooks, then all we need to specify is the type of the functions.
The preHooks let the user do the obvious thing, which is to read and write to a separate description file of their choosing, or to do something as strange as they like to get the package description.
In any case, you can just say defaultUserFunctions{postInst=system "./postinst.hs">> readPackageDescription "thePlaceIKnowItWillEndUp"}.
Ah, that's why I was confused: I had thought that at least Cabal would manage the persistence for them so the hooks would be independent. I think this is where I'm supposed to mutter "sometimes it takes a tough man to make a tender chicken". You're very concerned about keeping the interface as simple as possible, and providing maximum flexibility to the library author. The flipside of this is that each author of a library whose build is system-dependent must face the complexity you've excluded. They will end up redoing much the same thing as the previous person only slightly differently, inventing their own file format, writing their own printer and parser. If they write a preConfig hook, they'll need all the others too. Adding utilities for this to the library will help, but that will complicate the interface too, and they still have to do the rest. Maintaining that will be painful. If Cabal were to specify just a little more, the overwhelming majority of library authors would need to write no more than System.description and a preConfig hook to get system-dependent build information. The few who need more flexibility can write it themselves, but most people won't want to do that, and are likely to do a poorer job of it than a general tool would.

ross@soi.city.ac.uk writes: (snip)
You're very concerned about keeping the interface as simple as possible, and providing maximum flexibility to the library author. The flipside of this is that each author of a library whose build is system-dependent must face the complexity you've excluded.
I think that resisting feature-creep is a good policy with the threat of "reinventing make" looming. I want cabal to be maintainable in the long-run, for one thing.
They will end up redoing much the same thing as the previous person only slightly differently, inventing their own file format, writing their own printer and parser. If they write a preConfig hook, they'll need all the others too. Adding utilities for this to the library will help, but that will complicate the interface too, and they still have to do the rest. Maintaining that will be painful.
If Cabal were to specify just a little more, the overwhelming majority of library authors would need to write no more than System.description and a preConfig hook to get system-dependent build information.
I am trying to be accommodating, actually. I prefer the function-hooks because I think they're more flexible and build on Haskell's strengths. I certainly want runhaskell to appear and be reliable, but so far, cabal does not depend on it, and I'd prefer to keep it that way. runhaskell is a pretty simple thing, but there are a number of issues that need to be addressed, and it has several moving parts (compiler registration and un-registration, default compiler, distribution, etc.). I don't think cabal should fall down if one of those parts breaks.
The few who need more flexibility can write it themselves, but most people won't want to do that, and are likely to do a poorer job of it than a general tool would.
We could provide default hooks that read and write to a pre-determined file (except for preConf, which the user will want to provide). The information that the later commands need is exactly what's in PackageDescription, which is why that's what we're returning in the hooks. I'm not sure I see an advantage in adding another file and another parser for the build information. If you want to generate a file during configure, you can generate a Setup.description-formated file. You'd have: -- |all return nothing, provided for convenience emptyUserFunctions :: UserFunctions ... -- |the default name for generated description file generatedDescriptionFile = "Setup.description.generated" -- |read and write to Setup.description.generated or something defaultUserFunctions :: UserFunctions defaultUserFunctions = UserFunctions {preConf = return Nothing, -- override by making generatedDescriptionFile preBuild = do exists <- doesFileExist generatedDescriptionFile when exists (readPackageDescription generatedDescriptionFile) ... etc. } We have a new data structure which is very simple, if repetitive, a few new variables, and a new defaultMain function. There will be some repetition between the required fields of Setup.description and generatedDescriptionFile, and I don't like that, but I sorta like the idea of having a new parser and datafile format (for build info) even less. ------------------------------------------------------------ Now Angela just says something like main = defaultMainWithHooks defaultUserFunctions{preConf=angelasPreConf} I think this gives you everything you want. peace, isaac

As an experiment to see what would be useful, I've switched Hugs's handling of the more peripheral fptools/libraries packages (QuickCheck, mtl, fgl, HaXml, parsec, HUnit, X11, HGL, OpenGL, GLUT and OpenAL) over to a variant of Cabal's simple build infrastructure. The result is in the CVS repository as hugs98/libraries/tools/HugsSetup.hs. It's a Cabal setup script that does configure, build and install, but doesn't yet handle executables, installed package descriptions or source distributions. Nor does it handle any version stuff, and it may be a long time before Hugs can handle that. Still, it does the basic job, and it works with all those packages, some of which require system-dependent parameters, preprocessing, and compilation of FFI stubs. I hope that some of it (or equivalent) can be incorporated into Cabal, though I expect you won't want all of it. Some extra fields were needed: buildPackage :: Bool, ccOptions :: [String], ldOptions :: [String], frameworks :: [String] The first one records whether the package configured successfully on this system, and is therefore buildable. It was most convenient to add these as a subrecord of LocalBuildInfo. With the parsing machinery in Cabal, it was easy to knock up a parser for a file Setup.buildinfo containing these fields. This file can be generated from a template file Setup.buildinfo.in by ./configure (which might be generated by autoconf or supplied by the author). This information would be almost enough to do a similar job for GHC, including generation of the installed package descriptions, I think. For Hugs, it is also very convenient (though not absolutely essential) to have an option --builddir=DIR to setup configure, to specify where the build output goes. That goes in LocalBuildInfo too. Some points I draw from the experiment: 1) The separation between the static Setup.description and the possibly generated Setup.buildinfo(.in) makes a clear distinction between the invariant interface and system-dependent implementation, except that I'd move more fields from the former to the latter. 2) If you decide not to do the interface at the file level, it would still make the interface clearer to use two different records for the different types of information. 3) Even if you don't do that, it would be helpful for the library to take care of the persistence of the extra information returned by the configure hook, just as you do now with LocalBuildInfo.

ross@soi.city.ac.uk writes:
As an experiment to see what would be useful, I've switched Hugs's handling of the more peripheral fptools/libraries packages (QuickCheck, mtl, fgl, HaXml, parsec, HUnit, X11, HGL, OpenGL, GLUT and OpenAL) over to a variant of Cabal's simple build infrastructure.
I'll check this out ASAP. A few days ago, I checked into Darcs the hooks infrastructure I've talked about. I think it'll do what you need... I haven't checked it into CVS yet because I haven't had a chance to put together any examples. If you can, would you look at the Darcs repo, or if you like, I can just check it into CVS. peace, isaac

ross@soi.city.ac.uk writes:
As an experiment to see what would be useful, I've switched Hugs's handling of the more peripheral fptools/libraries packages (QuickCheck, mtl, fgl, HaXml, parsec, HUnit, X11, HGL, OpenGL, GLUT and OpenAL) over to a variant of Cabal's simple build infrastructure. The result is in the CVS repository as hugs98/libraries/tools/HugsSetup.hs. It's a Cabal setup script that does configure, build and install, but doesn't yet handle executables, installed package descriptions or source distributions. Nor does it handle any version stuff, and it may be a long time before Hugs can handle that. Still, it does the basic job, and it works with all those packages, some of which require system-dependent parameters, preprocessing, and compilation of FFI stubs. I hope that some of it (or equivalent) can be incorporated into Cabal, though I expect you won't want all of it.
This is indeed very cool. Much of this should probably just be integrated into the Distribution.Simple stuff for Hugs. You should feel welcome to do that if you like. If not, this will give me a good idea of what you believe needs to get done. I went ahead and committed the hooks stuff to CVS so you can try it out. I would guess that if we integrate the stuff that cabal should really have for Hugs, and use the hooks (which will greatly reduce boilerplate), HugsSetup.hs will be much smaller. I think the only sticking point is whether we have one file or two, the description verses the build info. By way of explanation for the hooks: I added the UserHooks type: data UserHooks = UserHooks { runTests :: Bool -> IO ExitCode, -- ^Used for './setup test' readDesc :: IO (Maybe PackageDescription), -- ^Read the description file preConf :: IO (Maybe PackageDescription), postConf :: IO ExitCode, preBuild :: IO (Maybe PackageDescription), postBuild :: IO ExitCode, preClean :: IO (Maybe PackageDescription), postClean :: IO ExitCode, preCopy :: IO (Maybe PackageDescription), postCopy :: IO ExitCode, preInst :: IO (Maybe PackageDescription), postInst :: IO ExitCode, -- ^guaranteed to be run on target preSDist :: IO (Maybe PackageDescription), postSDist :: IO ExitCode, preReg :: IO (Maybe PackageDescription), postReg :: IO ExitCode, preUnreg :: IO (Maybe PackageDescription), postUnreg :: IO ExitCode } and a file, hookedPackageDesc which the default pre-hooks read from, is the same syntax as the package description. There's a set of default user-hooks: defaultUserHooks :: UserHooks where preConf does nothing, and pre-everything-else reads hookedPackageDesc. Then a new main function: defaultMainWithHooks :: UserHooks -> IO () So you can say: myConf = do whenM (doesFileExist "configure") $ system "./configure" ... main = defaultMainWithHooks defaultUserHooks{preConf=myConf} We could also try to come up with a good default behavior for preConf. It looks like you use the params to configure, so maybe the hook functions need to take the flags for each param as input or something. Hmmmm. This is all very experimental and likely to change. For instance, I had the idea that the post-hooks should exit if they got a bad exit code, but that'll make it impossible to handle exceptions in the user's 'main' function, and maybe the pre-hooks should have input params... peace, isaac

On Thu, Dec 23, 2004 at 01:56:35AM -0500, Isaac Jones wrote:
This is indeed very cool. Much of this should probably just be integrated into the Distribution.Simple stuff for Hugs.
It will require some changes: the extra fields I mentioned, plus some complications in the preprocessing.
I think the only sticking point is whether we have one file or two, the description verses the build info.
If a package requires system-dependent parameters, there will surely be two hunks of information from the author's viewpoint: the file Setup.description (which you want in the source bundle for use by the other tools) and something else for the build information. No one will want to repeat all the information in Setup.description in that other thing, so it will need different handling. The question is whether that should be reflected by the interface. Also, since you're writing code to read (and presumably to save) the output of preConf, if you moved this persistency handling inside defaultMainWorker you could simplify the interface: the other hooks wouldn't need to return anything.
It looks like you use the params to configure, so maybe the hook functions need to take the flags for each param as input or something. Hmmmm.
Well, the preConf would, but maybe that's different from the others.
participants (4)
-
Isaac Jones
-
Ross Paterson
-
ross@soi.city.ac.uk
-
Simon Marlow