simplifying user hooks (non-backward-compatible)

Andres Loeh proposed simplifying the hooks interface now that they've grown to include not only pre-and-post hooks, but also er, "during" hooks too. That is, all of the actions are hooks now (preConf, confHooks, postConf). http://hackage.haskell.org/cgi-bin/trac/trac.cgi/ticket/23 So we could now get rid of the pre and post hooks, as long as the user could call to the default hooks thusly: myHooks = defaultUserHooks { confHook = myConf } myConf pd cf = do myPreConf lbi <- confHook defaultUserHooks myPostConf pd cf lbi (for suitable myPreConf and myPostConf functions) This would greatly simplify the Distribution.Simple.UserHooks structure bringing it from 34 fields to 14 fields, or so. Downsides: Making pre and post hooks would now be slightly harder, and it would break existing hooks-using code. I like it. What are your feelings? peace, isaac

[snip]
This would greatly simplify the Distribution.Simple.UserHooks structure bringing it from 34 fields to 14 fields, or so.
Downsides: Making pre and post hooks would now be slightly harder, and it would break existing hooks-using code.
I just want to mention that these kind of changes represent represent a VERY big problem. I discussed that issue at some length in an email I intended to send to this list very recently, but it seems to have gotten lost. So, to recap: Suppose Alice writes a project A with a Cabal build system that relies on user hooks? Now the interface changes. Bob writes a Cabal build system using the new user hooks for project B. Alice doesn't have time or inclination to update her build system (it still works right? Besides, Alice wants to work on project A; she doesn't want to rewrite her build system every time the cabal folks have a new good idea). Now Jow (Debian package maintainer/gentoo haskell user/whatever) wants to build both projects A and B. What version of Cabal should Joe have? How does he know how to enable the correct versions for which packages, and what versions of which packages? Ugh! What a nightmare. The interesting thing is that cabal seems to solve the versioning problem for most every library except itself! The solution: I suggest that the cabal team very seriously consider using the Eternal Compatibility in Theory method to manage interface change. http://www.haskell.org/tmrwiki/EternalCompatibilityInTheory Rob Dockins

Robert Dockins
[snip]
This would greatly simplify the Distribution.Simple.UserHooks structure bringing it from 34 fields to 14 fields, or so.
Downsides: Making pre and post hooks would now be slightly harder, and it would break existing hooks-using code.
I just want to mention that these kind of changes represent represent a VERY big problem. I discussed that issue at some length in an email I intended to send to this list very recently, but it seems to have gotten lost.
Are you sure they represent a very big problem in practice and not just in theory? The hooks have been a "very experimental" part of Cabal, as the user manual has always said: You can customize the simple build infrastructure using hooks. [...] See Distribution.Simple for the details, but note that this interface is experimental, and likely to change in future releases. I only know of one or two packages that use the hooks. Do you know of backward compatibility problems that cabal has caused? BTW, I just added a "cabal-version" field to cabal so that if a package requires a particular version of cabal, it can say so. Further, my grand plan is that once stuff gets into hackage, we can try making modifications and seeing if it breaks any packages, and if so, offer patches to those package authors (since it's probably eaiser for cabal hackers to write such patches than most package authors). (snip)
I suggest that the cabal team very seriously consider using the Eternal Compatibility in Theory method to manage interface change.
I'll look at this. peace, isaac

On Saturday 10 December 2005 07:14 pm, you wrote:
Robert Dockins
writes: [snip]
This would greatly simplify the Distribution.Simple.UserHooks structure bringing it from 34 fields to 14 fields, or so.
Downsides: Making pre and post hooks would now be slightly harder, and it would break existing hooks-using code.
I just want to mention that these kind of changes represent represent a VERY big problem. I discussed that issue at some length in an email I intended to send to this list very recently, but it seems to have gotten lost.
Are you sure they represent a very big problem in practice and not just in theory?
[snip] Well, obviously it isn't a practical problem yet. If it was we'd have people screaming about it on the lists. The thing that got me thinking about it was Happy. Its recently been put in a darcs, and Simon Marlow mentioned to me that they are looking at moving to Cabal for the build system. The Happy build system now does two things that aren't your standard build actions: 1) It creates module which contains the version string before compilation 2) After compilation it runs CPP on a number of template files with various different "-D" options to genreate the parser templates. I was messing around with doing these using user hooks and I ended up needing to tie pretty deeply into cabal even for the simple task 1). I have attached my initial attempts so you can see what's happening. The user hooks change you are talking about would require a complete rework of my Setup.hs file. Its only ~130 lines, but then Happy is a pretty small project.
I only know of one or two packages that use the hooks. Do you know of backward compatibility problems that cabal has caused?
No, this is all speculation currently.
BTW, I just added a "cabal-version" field to cabal so that if a package requires a particular version of cabal, it can say so.
Well, cabal the library already has to be invoked before the package file is even read! If the Setup.hs doesn't typecheck because the cabal interfaces have changed, you won't even get that far. To make a "cabal-version" field work you'd need some kind of bootstrapping step (which would itself need a very stable interface).
Further, my grand plan is that once stuff gets into hackage, we can try making modifications and seeing if it breaks any packages, and if so, offer patches to those package authors (since it's probably eaiser for cabal hackers to write such patches than most package authors).
That depends on the complexity of the build system. In my experience, just trying to figure out what a build system is doing can be pretty difficult. I can't imagine you want to undertake the maintaince of other people's build systems. OTOH, if Cabal is only aiming at pretty simple projects, this may never be an issue.
(snip)
I suggest that the cabal team very seriously consider using the Eternal Compatibility in Theory method to manage interface change.
I'll look at this.
peace,
isaac

Robert Dockins
On Saturday 10 December 2005 07:14 pm, you wrote:
Robert Dockins
writes: (snip) The Happy build system now does two things that aren't your standard build actions: 1) It creates module which contains the version string before compilation 2) After compilation it runs CPP on a number of template files with various different "-D" options to genreate the parser templates.
I was messing around with doing these using user hooks and I ended up needing to tie pretty deeply into cabal even for the simple task 1).
What I would like to do is to stabalize an interface one of these days. The problem is that, IMO, we need experience with what users want & need in order to stabalize the interface. When writing layered tools like cabal-get, Lemmih had to modify cabal's interface, and when writing cabal-install, I had to modify the interface. But I'll definitely look at the Eternal Compatibility thing!
I have attached my initial attempts so you can see what's happening.
The user hooks change you are talking about would require a complete rework of my Setup.hs file. Its only ~130 lines, but then Happy is a pretty small project.
Not at all. See below.
I only know of one or two packages that use the hooks. Do you know of backward compatibility problems that cabal has caused?
No, this is all speculation currently.
BTW, I just added a "cabal-version" field to cabal so that if a package requires a particular version of cabal, it can say so.
Well, cabal the library already has to be invoked before the package file is even read! If the Setup.hs doesn't typecheck because the cabal interfaces have changed, you won't even get that far. (snip)
True enough, It's mainly useful for parsing the .cabal file. Setup can't use this information for compiling, but humans can use it to understand compile failures, and cabal-get can use it.
Further, my grand plan is that once stuff gets into hackage, we can try making modifications and seeing if it breaks any packages, and if so, offer patches to those package authors (since it's probably eaiser for cabal hackers to write such patches than most package authors).
That depends on the complexity of the build system. In my experience, just trying to figure out what a build system is doing can be pretty difficult. I can't imagine you want to undertake the maintaince of other people's build systems.
I'm certainly not talking about maintaining them, but usually if you modify an interface, the kinds of changes you make are mechanical and repetative.
OTOH, if Cabal is only aiming at pretty simple projects, this may never be an issue.
The plan with cabal was to handle simple projects at first, and add more complex projects as we go along. That's basically happening, and my hope is that we can fix the "simple" interface sooner, and the more complex interface later.
import Data.Version import Control.Exception import System.Cmd import System.IO import System.Exit import System.Directory import Distribution.Setup import Distribution.Simple import Distribution.Simple.Utils import Distribution.Simple.LocalBuildInfo import Distribution.PackageDescription
It would be nice if you used qualified imports so I could see exactly what it is that you need from cabal. Also you'll get better error messages if things disappear.
main = do descFile <- defaultPackageDesc desc <- readPackageDescription descFile let ver = (pkgVersion (package desc)) defaultMainWithHooks (mkHooks desc ver)
mkHooks :: PackageDescription -> Version -> UserHooks mkHooks desc ver = defaultUserHooks { readDesc = return (Just desc) , postConf = doPostConf ver , postBuild = createTemplates , postClean = cleanDerivedFiles }
With the proposed change, all you have to do is something basically like this, not reworking the whole build system:
defaultUserHooks { readDesc = return (Just desc) , confHook = (confHook defaultUserHooks) >> doPostConf ver , buildHook = (buildHook defaultUserHooks) >> createTemplates , cleanHook = (cleanHook) >> cleanDerivedFiles }
(snip)
createVersionModule :: Version -> IO () createVersionModule version = do h <- openFile "src/Version.hs" WriteMode hPutStr h "module Version where\n" hPutStr h ("version = \""++(showVersion version)++"\"") hClose h
That's interesting. I was recently thinking of generating something like CabalInfo.hs which has information like the PackageDescription and the LocalBuildInfo. You would turn on this feature w/ the .cabal file. I'm not sure if any of the information is useful except for the version number, though. We could also add -DPACKAGE_VERSION or something to the cpp line. That would be pretty easy. (snip)
perlRE = "s/^#\\s+(\\d+)\\s+(\\\"[^\\\"]*\\\")/{-# LINE \\1 \\2 #-}/g;s/\\$$(Id:.*)\\$$/\\1/g"
rawSystem "perl" ["-pi.hspp","-e",perlRE,file]
aaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhh!!!!!!!!! /me runs away ;) But seriously folks, you should check out the new (experimental!) "Program" interface in the CVS / darcs HEAD. The nice thing about it is you can add perl as a Program and configure will find it, and users can say: ./setup configure --with-perl=/my/special/perl. peace, isaac

What I would like to do is to stabalize an interface one of these days. The problem is that, IMO, we need experience with what users want & need in order to stabalize the interface. When writing layered tools like cabal-get, Lemmih had to modify cabal's interface, and when writing cabal-install, I had to modify the interface.
But I'll definitely look at the Eternal Compatibility thing!
That's good to hear. I think it is a good fit for this situation.
import Data.Version import Control.Exception import System.Cmd import System.IO import System.Exit import System.Directory import Distribution.Setup import Distribution.Simple import Distribution.Simple.Utils import Distribution.Simple.LocalBuildInfo import Distribution.PackageDescription
It would be nice if you used qualified imports so I could see exactly what it is that you need from cabal. Also you'll get better error messages if things disappear.
I'm not sure what you mean here. Do you mean with explicit import lists? Qualified means something very specific in this context and I don't see how it would help.
perlRE = "s/^#\\s+(\\d+)\\s+(\\\"[^\\\"]*\\\")/{-# LINE \\1 \\2 #-}/g;s/\\$$(Id:.*)\\$$/\\1/g"
rawSystem "perl" ["-pi.hspp","-e",perlRE,file]
aaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhh!!!!!!!!!
/me runs away ;)
Just so you don't think I'm responsable for that perl RE mess, its taken directly from the Makefile ;) Yeah, expecting perl on the path is a little ugly but *shrug* its a pretty safe assumption (also made in the Makefile BTW).
But seriously folks, you should check out the new (experimental!) "Program" interface in the CVS / darcs HEAD. The nice thing about it is you can add perl as a Program and configure will find it, and users can say: ./setup configure --with-perl=/my/special/perl.
That sounds nice; I'll have to take a look. Rob Dockins

Following up on this thread... I modified the hooks interface quite a bit, again. There's good news and bad news about this. The good news is that it's cleaned up and should be easier to maintain and to avoid future modifications. The bad news is that this change itself will break stuff, of course. If you have any trouble building your Setup scripts, please let me know. I really think that it was best to bite the bullet right now in one big go instead of down the road with lots of little changes. I have a lot more confidence in the hooks interface, and I don't actually expect that it'll change as often. I made the types more consistent, and made sure there are accessor functions on each of the Flags types so that if the flags types change in the future, it shouldn't break lots of code. Another piece of good / bad news is that I decided not to get rid of the pre & post hooks. They are nice for convenience and it wouldn't be nearly so easy to write hooks without them. That's bad because the interface to hooks is still pretty big, which means that there's more likelihood that it'll change in the future. Another weakness in the Hooks interface is that with command hooks (like sDistHook) it's tempting to add parameters to them; basically the stuff that we compute between the preSDist and sDist hook. I removed such params and have their values computed elsewhere. Cabal hackers, please avoid adding parameters to these command hooks if at all possible in order to keep the interface steady. If you need to compute a value to pass to these functions, compute it in the function and / or make it available as a function that someone crafting hooks can use as well, or consider whether it belongs in one of the parameters already being passed to the hooks, PackageDescription, LocalBuildInfo, UserHooks, Flags. throwing myself on your mercy, isaac
participants (2)
-
Isaac Jones
-
Robert Dockins