Following long discussions on #haskell, here's some tricky cases we thought of with configurations. 1. build-depends: base configuration: debug build-depends: HUnit>=1.0 configuration: package(HUnit>=1.1) ghc-options: -DBlah The issue with this is what the "package(HUnit)" test means. Under one interpretation it just tests if that package is available (ie already installed), not if it's actually going to be used in this package. Under the other interpretation this can't be expressed since HUnit is not a top level dependency. Alternatively we could evaluate the configurations in order, in which case we've added HUnit>=1.0 to build depends before checking package(HUnit>=1.1). But making it order-dependent is ugly. 2. build-depends: base configuration: package(hdbc) build-depends: hdbc configuration: package(hsql) build-depends: hsql This is not ideal since it suggests that we will just pick up optional dependencies on packages if they are available with no control from the user as to whether they want those dependencies. This also means that the order in which you install packages can affect which features are enabled which makes the use of a dep resolver like cabal-install less intuitive as it becomes hard to predict how the packages will end up getting built. Incidentally this conflicts with distro package mangers like Gentoo's which specifically bans automatically picking up optional deps. It requires that all optional deps be specified before hand and be configurable by the user. This would also be difficult for binary package managers like debian as the debian package would just have to statically decide if hdbc or hsql or both will be used. Again, one option is to ban this on account of neither dep being mentioned in the top level build-depends. A better way to express this dependency might be: build-depends: base configuration: hdbc build-depends: hdbc configuration: hsql build-depends: hsql This means that we have flags "hdbc" and "hsql" and the user must specify at configure time --enable-hdbc or --disable-hdbc This way the optional dependency is not automatic and can be changed by the user. It's perfectly ok for there to be a default setting for the flag so long as the user can override it. This is much like Gentoo's USE flag system. 3. build-depends: base configuration: package(base<=1.0) build-depends: fps>=0.8 This one is ok. Is there a better way to do what we want without getting the bad cases? I am of the opinion that it makes little sense for conditional package tests to be of what packages are available in the environment rather than packages that have actually been chosen for this package. Remember that we can have multiple versions available so here's a case where 1. falls over. Here's case 1. again: 1. build-depends: base configuration: debug build-depends: HUnit>=1.0 configuration: package(HUnit>=1.1) ghc-options: -DBlah Suppose we have HUnit 1 and 2 installed. Suppose our package depends on another package which needs HUnit-1. With the debug flag enabled we depend on HUnit>=1.0. To pick a consistent HUnit version we must use 1.0 since we can't have two versions of the same package (even ghc's recent support for this doesn't help if HUnit gets used in the interface since the types would not match). So we pick HUnit-1.0. Now the configuration test "package(HUnit>=1.1)" is still true because there is version 2 available, even though we're not using it! This is not an intuitive semantics. If we say that package() tests must refer to packages we're going to actually build against on then this is fine and we get what the package author probably meant. So this suggests we should limit package() tests to packages that we depend on or might depend on. But for 1. we still have the issue of the order of evaluation of configuration stanzas being significant because they can add packages to build-depends which other stanzas may test. In gentoo such a dependency might be expressed as: DEPEND="debug? ( HUnit>=1.0 )" but then the action dependent on the version of HUnit that we picked would be done later. So there's a clear ordering. So one solution is to not allow build-depends in conditional stanzas but to specify conditional depends up front. build-depends: base, debug? ( HUnit>=1.0 ) configuration: debug ghc-options: -DDEBUG configuration: package(HUnit>=1.1) ghc-options: -DBlah or: build-depends: base > 1.0 || ( base <= 1.0 && fps >= 0.8 ), os(windows)? ( Win32 ), os(linux)? ( inotify ) configuration: package(fps) ghc-options: -DOLD_FPS configuration: package(base < 1.1) hs-src-dirs: compat So I guess the suggestion is that all deps are given up front and resolved. Then depending on what deps were chosen we can have other effects on the build like specifying extra src dirs or flags or whatever, but not specifying any more build-depends. So the order of checking configurations is independent again. Ok I've gone on long enough. Duncan
Hello This is a formulation after more discussion on #haskell. This is for build-depends with the configurable things. Not sure whether this is the best way. - Einar Karttunen bexp = package-with-version | '(' bexp ')' | bexp '|' bexp | bexp ',' bexp | 'Flag(' string ')' | 'T' Semantics: package-with-version * satisfied iff a suitable version of P installed * depends on p '(' e ')' * satisfied iff e satisfied * dependencies from e a '|' b * satisfied iff a or b is satisfied * if a is satisfied then a else b a ',' b * satisfied iff a and b are satisfied * dependencies are the combination of a and b * union of packages and intersection of versions 'Flag(' string ')' * satisfied iff string in environment * no dependencies 'T' * always satisfied * no dependencies Code follows: data BExp = BP String Version | BOr BExp BExp | BAnd BExp BExp | BFlag String | BTrue eval (BP n v) = return [(n,v)] eval (BOr a b) = eval a `mplus` eval b eval (BAnd a b) = do x <- eval a y <- eval b return [(name, intersectVersions $ map snd xs) | xs@((name,_):_) <- groupByFst (x++y) ] liftM2 (comb) (eval a) (eval b) eval (BFlag s) = getFlag s >> return [] eval (BTrue) = return [] groupByFst = groupBy eqFst where eqFst (a,_) (b,_) = a == b - Einar Karttunen
I was also talking to the Gentoo protage developers about this. They had a couple interesting points. They say that automatically picking up optional dependencies is bad, even if they are tracked. That there must be a way to build without some dep even if it's installed and available. eg to build on one machine and move to another. So even if we make picking up optional deps the default, there must be a manual override mechanism. Then for the Gentoo package we would always use that override and have the optional dep controlled by a USE flag. You can simulate automatic optional dependencies with trickery like: build-depends: foo || base (Since base is always available, this means that foo would be used if it's installed and base otherwise. Since || is left biased.) They said that any Gentoo developer caught doing the equivalent of this would be shot. So while || or equivalents are useful for inconsequential variations in dependencies (like the issue of a module moving from one package to another - think base & fps), only social pressure can stop them from being abused. So, with this in mind, can we come up with a syntax & semantics that is close to our current configurations proposals but is not evil? Duncan
On Thu, 2006-08-10 at 22:13 +0100, Duncan Coutts wrote:
So, with this in mind, can we come up with a syntax & semantics that is close to our current configurations proposals but is not evil?
One thing that might help is if we split the 'package' condional test into two distinct things. I think we're using it for different purposes which correspond roughly to Gentoo's "dep || dep" xor dep syntax and it's "flag? dep" syntax. In one case it's an inconsequential decision to pick one dep, we use the other when the user might legitimately want to control if the dependency is used. So perhaps we could have 'package'(dependecy) and 'use'(dependency). I'll try and explain what both mean. 'package' conditionals would be for selecting things depending on which version of a package we depend on ends up getting used. 'use' conditionals would be for selecting optional dependencies from the environment. These would get corresponding --enable-foo --disable-foo configure flags, though would default depending on the presence of the package. So, examples: -- if the base package is <2.0, depend on fps: build-depends: base configuration: package(base < 2.0) build-depends: fps>=1.0 ghc-options: -DUSE_OLD_FPS -- optional gui build-depends: base configuration: use(gtk) && use(cairo) build-depends: gtk, cairo ghc-options: -DUSE_GUI I'm not sure that's ideal when we need to depend on multiple optional packages. Maybe something like this'd be nicer flag: debug default: False flag: gui default: use(gtk >= 0.9.10) && use(cabal >= 0.9.10) configuration: flag(gui) build-depends: gtk, cairo ghc-options: -DUSE_GUI configuration: flag(debug) ghc-options: -DDEBUG -O0 configuration: !flag(debug) ghc-options: -O2 -funbox-strict-fields So we have a flag with a reasonable default value but all flags can be manually overridden. I'm still concerned that the order of configurations is significant: build-depends: base flag: debug default: False configuration: flag(debug) build-depends: HUnit>=1.0 configuration: package(HUnit>=1.1) ghc-options: -DBlah Since we do not know which packages we're depending on 'til we've dealt with the first configuration. So looking at them in a different order would give different results. Perhaps we just live with it and say that they get done in order. There's still the possibility that we put all build-depends upfront: flag: debug default: False build-depends: base, flag(debug)? ( HUnit>=1.0 ) configuration: package(HUnit>=1.1) ghc-options: -DBlah So I think I'm suggesting that we explicitly specify flags and give them default values which may be based on the environment. Then deps are specified in terms of those flags, and os/arch. Then configurations can be conditional on flags, os/arch, and versions of packages that we're actually going to build with. Duncan
And here is the newest after discussion with Duncan Coutts on #haskell. build-depends := bexp bexp = package-name-with-version | '(' bexp ')' | cexp '?' bexp | bexp ',' bexp | 'T' configuration := cexp cexp = '(' cexp ')' | cexp '|' cexp | cexp '&' cexp | 'T' | '!' cexp | 'Flag(' string ')' So dependencies are only on Flags. A new declaration like: default := 'default:' flag-name '=' (bexp | Os(...) | ...) configure takes the following flags: --no-default-flags Defaults are ignored and all flags default to False. --enable-foo Enable flag foo --disable-foo Disable flag foo --auto-foo Use default for foo Now we don't have any of those evil conditional package dependencies. fps would work like: default: old_fps = base < 2 build-depends: base, Flag(old_fps) ? fps configuration: Flag(old_fps) ghc-options: -DUSE_OLD_FPS - Einar Karttunen
This kills a few typos and makes things more consistent with more #haskell discussion. build-depends := bexp bexp := package-with-version | cexp '?' '(' bexp ')' | bexp ',' bexp configuration := 'configuration:' cexp cexp := '(' cexp ')' | cexp '||' cexp | cexp '&&' cexp | '!' cexp | 'True' | 'Flag(' string ')' flag := 'flag:' name '=' dexp dexp := cexp extended with { package-with-version, Os(...), Compiler(...) } Some observations: * build-depends don't on packages * order does not matter * most of the problems seem to be solved * see the end how this works for practical problems * False = !True. Can be supported in the parser, but not needed for the semantics. And now for the semantics of the whole thing: ceval :: CExp -> M Bool ceval (COr a b) = liftM2 (||) (ceval a) (ceval b) ceval (CAnd a b) = liftM2 (&&) (ceval a) (ceval b) ceval (CNot x) = liftM not (ceval x) ceval (CTrue) = return True ceval (CFlag s) = isDefinedFlag s beval :: BExp -> M (Map Name Version) beval (BPackage n v) = return $ singleton n v beval (BCond c b) = do e <- ceval c if e then beval b else return empty beval (BSeq a b) = do x <- beval a y <- beval b return $ unionWith intersectVersion x y The flag declarations simply introduce intelligent default *guesses* for the flag values. They don't affect the resolving the conditionals at all (except for flag values). ./setup configure support the following additional options: --no-default-flags don't guess - initialize all flags to False --enable-foo set flag foo to true --disable-foo set flag foo to false --auto-foo use guess to determine flag foo The Art of Guessing Package managers etc can just pass --no-default-flags and specify things manually. This is enough for Gentoo etc. Guesses allow the optional package dependencies, but they are more than guesses and can be easily overridden by the user. Solutions to various problems: 1) fps build-depends: base, Flag(old_fps) ? (base < 2, fps >= 0.8) flag: old_fps = base < 2 configuration: Flag(old_fps) ghc-options: -DOLD_FPS If we are using guessing then 6.4 will use external fps and 6.6 the one in base - correctly. 2) HUnit build-depends: Flag(debug) ? (Flag(Hunit11) ? (HUnit-1.1), !Flag(Huni11) ? (HUnit >= 1, Hunit < 1.1 || > 1.1)) flag: debug = !True flag: HUnit11 = HUnit-1.1 && !(Hunit > 1.1) configuration: Flag(HUnit11) ghc-options: -DHUNIT11 This seems quite complex. But it works - the meaning is: * if debug flag is set then: * if HUnit11 is set depend on on HUnit-1.1 and -DHUNIT11 * if HUnit11 is not set depend on version of HUnit that is >1 and (<1.1 or >1.1) that is not 1.1 * thus the -D is correct * the guess is just a guess and does not affect the semantics - Einar Karttunen
Einar Karttunen wrote:
2) HUnit
build-depends: Flag(debug) ? (Flag(Hunit11) ? (HUnit-1.1), !Flag(Huni11) ? (HUnit >= 1, Hunit < 1.1 || > 1.1))
flag: debug = !True flag: HUnit11 = HUnit-1.1 && !(Hunit > 1.1)
configuration: Flag(HUnit11) ghc-options: -DHUNIT11
This seems quite complex. But it works - the meaning is: * if debug flag is set then: * if HUnit11 is set depend on on HUnit-1.1 and -DHUNIT11 * if HUnit11 is not set depend on version of HUnit that is >1 and (<1.1 or >1.1) that is not 1.1 * thus the -D is correct * the guess is just a guess and does not affect the semantics
What can I say? yeuch! I think we've left simplicity behind here. One goal (that I keep having to remind myself of) is that a .cabal file should be modifyable by an IDE such as Visual Haskell. I can't see Visual Haskell being able to successfully modify that complicated build-depends expression. This is one reason I decided to go the simpler route of individual configurations with conditionals, and why a conditional could be evaluated independently of all the others knowing only facts about the environment. FWIW, using my original syntax and semantics, here's how to do what you did above: configuration: debug && !Hunit11 build-depends: HUnit >= 1 configuration: debug && HUnit11 build-depends: HUnit-1.1 configuration: debug && (HUnit11 || package(HUnit>=1, 1.1)) ghc-options: -DHUNIT11 The whole thing is easier to understand IMO, and it's simple to implement too: no ordering, conditionals can be evalutated independently. I'm back to the two-argument package() conditional, because it's actually useful. package(Hunit>=1, 1.1) is not the same as package(HUnit-1.1) or even package(HUnit>=1 && 1.1). It says "the dependency HUnit>=1 resolves to version 1.1", which is exactly what you wanted to know. This is how you write conditionals that test the versions of packages you actually depend on, which is the criticism raised by Duncan at the beginning of this thread. I believe my original syntax addressed this concern. Now there are bad things that you can do with this syntax (auto-optional dependencies), and there are downright ridiculous things you can do with this syntax (test versions of packages you don't depend on), but I believe we could either make Cabal warn about those or disallow them altogether. I don't mind which. Cheers, Simon
On Fri, 2006-08-11 at 12:15 +0100, Simon Marlow wrote:
Einar Karttunen wrote:
2) HUnit
build-depends: Flag(debug) ? (Flag(Hunit11) ? (HUnit-1.1), !Flag(Huni11) ? (HUnit >= 1, Hunit < 1.1 || > 1.1))
flag: debug = !True flag: HUnit11 = HUnit-1.1 && !(Hunit > 1.1)
configuration: Flag(HUnit11) ghc-options: -DHUNIT11
This seems quite complex. But it works - the meaning is: * if debug flag is set then: * if HUnit11 is set depend on on HUnit-1.1 and -DHUNIT11 * if HUnit11 is not set depend on version of HUnit that is >1 and (<1.1 or >1.1) that is not 1.1 * thus the -D is correct * the guess is just a guess and does not affect the semantics
What can I say? yeuch! I think we've left simplicity behind here.
Well the point is to separate the decision to pick up an optional dependency from actually doing it. I'm not going to defend the '?' syntax too much, but I do like the separation of flags with default values that depend on the environment from configuration tests which depend on those flags.
One goal (that I keep having to remind myself of) is that a .cabal file should be modifyable by an IDE such as Visual Haskell. I can't see Visual Haskell being able to successfully modify that complicated build-depends expression.
Is that really any different from the conditionals in the configuration tests?
This is one reason I decided to go the simpler route of individual configurations with conditionals, and why a conditional could be evaluated independently of all the others knowing only facts about the environment.
FWIW, using my original syntax and semantics, here's how to do what you did above:
configuration: debug && !Hunit11 build-depends: HUnit >= 1
configuration: debug && HUnit11 build-depends: HUnit-1.1
configuration: debug && (HUnit11 || package(HUnit>=1, 1.1)) ghc-options: -DHUNIT11
The whole thing is easier to understand IMO, and it's simple to implement too: no ordering, conditionals can be evalutated independently.
Well they're only independent if you take the view that package() conditionals test what's available in the environment rather than what we're going to actually use. If it's what we're going to use then one configuration can affects the condition of another. Having it just pick up things from the environment is also evil. Any package manager (debian, gentoo) need to be able to control/override that. That's why we were proposing a split between flags with their default values and conditions based on those flags. That way it's clear what can be overridden.
I'm back to the two-argument package() conditional, because it's actually useful. package(Hunit>=1, 1.1) is not the same as package(HUnit-1.1) or even package(HUnit>=1 && 1.1). It says "the dependency HUnit>=1 resolves to version 1.1", which is exactly what you wanted to know. This is how you write conditionals that test the versions of packages you actually depend on, which is the criticism raised by Duncan at the beginning of this thread. I believe my original syntax addressed this concern.
Ok if we have package(Hunit>=1, 1.1) then do we not then have an ordering issue? configuration: debug build-depends: HUnit-1.1 configuration: package(HUnit>=1, 1.1) ghc-options: -DHUNIT11 If I do the first one then the second then we know that we're going to depend on HUnit-1.1, if we do it the other way around we don't know that yet. That was one reason for bringing the conditionals up front with the '?' syntax that you object to. We can move it back to the above style if it's easier for users/IDEs but then there must be an ordering.
Now there are bad things that you can do with this syntax (auto-optional dependencies), and there are downright ridiculous things you can do with this syntax (test versions of packages you don't depend on), but I believe we could either make Cabal warn about those or disallow them altogether. I don't mind which.
I really worry that this will make many packages unpackagable because their semantics will not be translatable into sane distro packages. As I said, if I made a gentoo package that did auto-optional dependencies then the QA team would shoot me. I think it's quite easy to make package tests be only internal, that is what deps have we resolved for the package, and not mix that up with what happens to be available in the environment. Then as a separate thing have some facility to make the default decisions about which configurations to use to depend on the environment, but in a clearly separated way so that users or packagers can override the defaults. So let me suggest something with flags but without '?' syntax: flag: debug default: False configuration: flag(debug) build-depends: HUnit-1.1 configuration: using(HUnit==1.1) ghc-options: -DHUNIT11 flag: gui default: os(windows) || (available(gtk>=0.9.10) && available(cairo>=0.9.10)) configuration: flag(gui) && os(windows) build-depends: Win32>=1.1 ghc-options: -DUSE_WIN32_GUI configuration: flag(gui) && !os(windows) build-depends: gtk>=0.9.10, cairo>=0.9.10, glib>=0.9.10 ghc-options: -DUSE_GTK_GUI So in a flag we can make the defaults depend on the environment. In a configuration test we can only depend on internal things like flags and which version of a package that we depend on we decided to use. (I'm deliberately avoiding using 'package' here because it's meaning is overloaded with our various proposals). I'd also say that os/arch tests could be used in flag defaults and also in configuration tests since they're fixed aspects of the environment rather than variable like package availability. Then there is the problem that the order of evaluating configurations is significant. My previous proposal was to move them up front and use '?' but I see that's not great either. We could just say they're done in order. I do not think it's ok that they be made independent by just saying that they only depend on the environment rather than what we're actually using, that stores up too many problems imho. Duncan
Duncan Coutts wrote:
On Fri, 2006-08-11 at 12:15 +0100, Simon Marlow wrote:
One goal (that I keep having to remind myself of) is that a .cabal file should be modifyable by an IDE such as Visual Haskell. I can't see Visual Haskell being able to successfully modify that complicated build-depends expression.
Is that really any different from the conditionals in the configuration tests?
Visual Haskell would ignore complicated configuration sections. It can handle configuration sections that just test a single flag, because this fits in with the Visual Studio notion of "configurations". It also needs to be able to grok and modify the build-depends field. The idea would be that if you want to do anything more complicated, then you add a configuration section to the .cabal file by hand.
The whole thing is easier to understand IMO, and it's simple to implement too: no ordering, conditionals can be evalutated independently.
Well they're only independent if you take the view that package() conditionals test what's available in the environment rather than what we're going to actually use.
Yes, that's the intended semantics (sorry for not saying that explicitly). It's what makes simpler, but possibly too general.
I'm back to the two-argument package() conditional, because it's actually useful. package(Hunit>=1, 1.1) is not the same as package(HUnit-1.1) or even package(HUnit>=1 && 1.1). It says "the dependency HUnit>=1 resolves to version 1.1", which is exactly what you wanted to know. This is how you write conditionals that test the versions of packages you actually depend on, which is the criticism raised by Duncan at the beginning of this thread. I believe my original syntax addressed this concern.
Ok if we have package(Hunit>=1, 1.1) then do we not then have an ordering issue?
No, it's evaluated based on the environment only. That's why the dependency is repeated from the build-depends field.
Now there are bad things that you can do with this syntax (auto-optional dependencies), and there are downright ridiculous things you can do with this syntax (test versions of packages you don't depend on), but I believe we could either make Cabal warn about those or disallow them altogether. I don't mind which.
I really worry that this will make many packages unpackagable because their semantics will not be translatable into sane distro packages. As I said, if I made a gentoo package that did auto-optional dependencies then the QA team would shoot me.
I imagine you could figure out for a given package description whether it had any auto-optional dependencies in it (it's not trivial, but not too hard I think).
I think it's quite easy to make package tests be only internal, that is what deps have we resolved for the package, and not mix that up with what happens to be available in the environment. Then as a separate thing have some facility to make the default decisions about which configurations to use to depend on the environment, but in a clearly separated way so that users or packagers can override the defaults.
Ok - so is it a goal that you want to allow auto-optional dependencies but allow the packaging environment to turn them off? If so, then I agree you need something like the scheme you suggest, I just want to be clear about why we need that (especially if auto-optional deps are considered evil).
So let me suggest something with flags but without '?' syntax:
flag: debug default: False
configuration: flag(debug) build-depends: HUnit-1.1
configuration: using(HUnit==1.1) ghc-options: -DHUNIT11
flag: gui default: os(windows) || (available(gtk>=0.9.10) && available(cairo>=0.9.10))
configuration: flag(gui) && os(windows) build-depends: Win32>=1.1 ghc-options: -DUSE_WIN32_GUI
configuration: flag(gui) && !os(windows) build-depends: gtk>=0.9.10, cairo>=0.9.10, glib>=0.9.10 ghc-options: -DUSE_GTK_GUI
with this you can still write silly tings - an available() condition for a package you don't care about, for example. To summarise there are a couple of issues. I'll use the abbreviations AOD for auto-optional dependencies and SC for silly conditionals (predicates on packages that aren't a dependency). Ordering requirement on configurations: Option O1. No ordering, requires 2-argument package() predicate. Allows AOD and SC, but we could (possibly) detect these. Option O2. Ordering matters, we have using(P) predicates, doesn't allow AOD or SC. Default settings for flags: Option D1. All flags default to off. Option D2. Defaults can be set based on the availability of packages. Allows AOD and SC, but both can be disabled with a command-line switch. If we can do without AOD, then we just choose between O1 and O2 - not a straightforward choice, but I went the O1 route because it's easier to implement and more declarative, on the other hand adding detection of AODs would mean more complexity. If we want AOD, then probably O2/D2 (the dcoutts proposal) is the sensible choice. On the other hand, it means adding yet more stuff to Cabal... Cheers, Simon
On Fri, 2006-08-11 at 15:40 +0100, Simon Marlow wrote:
Duncan Coutts wrote:
On Fri, 2006-08-11 at 12:15 +0100, Simon Marlow wrote:
One goal (that I keep having to remind myself of) is that a .cabal file should be modifyable by an IDE such as Visual Haskell. I can't see Visual Haskell being able to successfully modify that complicated build-depends expression.
Is that really any different from the conditionals in the configuration tests?
Visual Haskell would ignore complicated configuration sections. It can handle configuration sections that just test a single flag, because this fits in with the Visual Studio notion of "configurations". It also needs to be able to grok and modify the build-depends field. The idea would be that if you want to do anything more complicated, then you add a configuration section to the .cabal file by hand.
That's fair enough. Not having '?' deps is good for that I agree. I only introduced that as a way of making configuration sections independent, but there are probably nicer ways of doing that.
The whole thing is easier to understand IMO, and it's simple to implement too: no ordering, conditionals can be evalutated independently.
Well they're only independent if you take the view that package() conditionals test what's available in the environment rather than what we're going to actually use.
Yes, that's the intended semantics (sorry for not saying that explicitly). It's what makes simpler, but possibly too general.
Right. That's the semantics that I objected to and tried to explain why in another recent email.
I'm back to the two-argument package() conditional, because it's actually useful. package(Hunit>=1, 1.1) is not the same as package(HUnit-1.1) or even package(HUnit>=1 && 1.1). It says "the dependency HUnit>=1 resolves to version 1.1", which is exactly what you wanted to know. This is how you write conditionals that test the versions of packages you actually depend on, which is the criticism raised by Duncan at the beginning of this thread. I believe my original syntax addressed this concern.
Ok if we have package(Hunit>=1, 1.1) then do we not then have an ordering issue?
No, it's evaluated based on the environment only. That's why the dependency is repeated from the build-depends field.
I think I get that. "the dependency HUnit>=1 resolves to version >1.1". s/resolves to/would resolve to That means that supposing I were to build-depend on HUnit>=1 then I would get a version satisfying >1.1. So if that evaluates to true it's not to say that I do depend on HUnit>=1 or that we are going to use
1.1. Indeed we may actually end up depending on version 0.9 and this test will still be true.
I think it's quite easy to make package tests be only internal, that is what deps have we resolved for the package, and not mix that up with what happens to be available in the environment. Then as a separate thing have some facility to make the default decisions about which configurations to use to depend on the environment, but in a clearly separated way so that users or packagers can override the defaults.
Ok - so is it a goal that you want to allow auto-optional dependencies but allow the packaging environment to turn them off? If so, then I agree you need something like the scheme you suggest, I just want to be clear about why we need that (especially if auto-optional deps are considered evil).
I don't mind too much if we have auto-optional dependencies but if we do then we must have a way of controlling them manually. It's probably nice to have. I expect most devs and users would like it. Most of the time for end users it does the right thing.
So let me suggest something with flags but without '?' syntax:
flag: debug default: False
configuration: flag(debug) build-depends: HUnit-1.1
configuration: using(HUnit==1.1) ghc-options: -DHUNIT11
flag: gui default: os(windows) || (available(gtk>=0.9.10) && available(cairo>=0.9.10))
configuration: flag(gui) && os(windows) build-depends: Win32>=1.1 ghc-options: -DUSE_WIN32_GUI
configuration: flag(gui) && !os(windows) build-depends: gtk>=0.9.10, cairo>=0.9.10, glib>=0.9.10 ghc-options: -DUSE_GTK_GUI
with this you can still write silly tings - an available() condition for a package you don't care about, for example.
Yes, that's true. I don't think it has any particularly bad consequences though and developers are not likely to do it (unlike auto-optional deps which devs will use if they're available).
To summarise there are a couple of issues. I'll use the abbreviations AOD for auto-optional dependencies and SC for silly conditionals (predicates on packages that aren't a dependency).
Ordering requirement on configurations:
Option O1. No ordering, requires 2-argument package() predicate. Allows AOD and SC, but we could (possibly) detect these.
Option O2. Ordering matters, we have using(P) predicates, doesn't allow AOD or SC.
Default settings for flags:
Option D1. All flags default to off.
Option D2. Defaults can be set based on the availability of packages. Allows AOD and SC, but both can be disabled with a command-line switch.
If we can do without AOD, then we just choose between O1 and O2 - not a straightforward choice, but I went the O1 route because it's easier to implement and more declarative, on the other hand adding detection of AODs would mean more complexity.
Yes, I think that summarises it quite well.
If we want AOD, then probably O2/D2 (the dcoutts proposal) is the sensible choice. On the other hand, it means adding yet more stuff to Cabal...
Aye, ho hum. I've not spotted any nice simple schemes that cope with the proposed use cases. Duncan
On Fri, 2006-08-11 at 12:15 +0100, Simon Marlow wrote:
The whole thing is easier to understand IMO, and it's simple to implement too: no ordering, conditionals can be evalutated independently.
I think that having the semantics of 'package()' in a configuration test be to just check for the availability of a package in the environment must be wrong. It's usually not what you want, and when it is what you want you shouldn't be doing it! :-) configuration: debug build-depends: HUnit >= 1 configuration: package(HUnit>1) ghc-options: -DHUNIT11 So if we use the debug flag then we depend on HUnit-1.0 or later. In the second configuration stanza we're merely asking if a later version than HUnit-1.0 is available to be used, not if we've already decided to use it. This is probably not what we intended. If 1.0 and 1.1 are available then we get -DHUNIT11 even if we're actually using 1.0. So yes we can make the conditional test more complex to get just what we did mean. People are going to make mistakes doing that because you'll only notice it's wrong in the situation that you've got 1.0 and 1.1 installed. So do we ever want to be able to make decisions based just on what's available? No. I don't think you do. Well, one may do but one shouldn't. Every time we're picking up an optional dependency it should be controllable by the user or package manager. So in the above case the first stanza is fine because the 'debug' flag can be set at configure time. So the point is, it's fine to adjust the build configuration depending on what packages we ended up building against, but automatically changing the build configuration based on the environment is not ok. For convenience we can get pretty close to that by specifying externally controllable flags that control the build configuration and additionally providing a mechanism to pick sensible defaults for those flags based on what's available. Most developers like the idea of automatically enabling features when the dependencies are available. Packaging people will tell you that this is evil and that it must be possible for it to be an explicit decision, even if we normally pick a sensible default to enable most features. So I fear that if the syntax allows it, developers will do it. Hence the syntax & semantics must be such that all decisions based on what's in the environment be overridable. Interestingly for the fps/base case we do not need a user flag, it really is an automatic decision. Once we've fixed the version of base we're using then there is no choice about whether we use the external fps package or not. build-depends: base configuration: package(base==1.0) build-depends: fps>=0.8 Duncan
On Fri, 2006-08-11 at 15:08 +0100, Duncan Coutts wrote:
On Fri, 2006-08-11 at 12:15 +0100, Simon Marlow wrote:
The whole thing is easier to understand IMO, and it's simple to implement too: no ordering, conditionals can be evalutated independently.
I think that having the semantics of 'package()' in a configuration test be to just check for the availability of a package in the environment must be wrong. It's usually not what you want, and when it is what you want you shouldn't be doing it! :-)
Ok so let me try and propose something concrete: build-depends := [package-with-version] flag := 'flag:' name 'default:' fexp -- flag default value expression fexp := '(' cexp ')' | cexp '||' cexp | cexp '&&' cexp | '!' cexp | 'True' | 'False' | 'os' '(' string ')' | 'arch' '(' string ')' | 'compiler' '(' string ')' -- this is the important one: | 'available' '(' package-with-version ')' configuration := 'configuration:' cexp -- configuration predicate expression cexp := '(' cexp ')' | cexp '||' cexp | cexp '&&' cexp | '!' cexp | 'os' '(' string ')' | 'arch' '(' string ')' | 'compiler' '(' string ')' -- these are the important ones: | 'flag' '(' string ')' | 'using' '(' package-with-version ')' So, semantics... Default values of flags can be evaluated independently and up front. The only non trivial test is the 'available' one. This checks if a package exists in the environment with that name and with a version that satisfies any given version range. "available (A>1) && available (A<1)" means (exists A. A>1) && (exists A. A<1), so they do not need to be the same A. Where as "available (A >1 <1)" would refer to the same A and therefore not be satisfiable. Configurations are evaluated in order, top to bottom (can we do better? require them to be acyclic?). Each configuration predicate can be evaluated based only on knowing the values of the flags (which may come from their default values or be overridden by the user) and the set of packages we are committed to. The 'using' test is the only non-trivial one. This checks if we are committed to depending on a version of the named package that satisfies the version range. We discover this by starting with the top level build depends. We do the normal checks to decide which version of each package we are going to depend upon. Once we know that then we can evaluate the 'using' tests. Of course a configuration may add extra build-depends and this can effect the value of a 'using' test in later configurations. How does this interact with the possibility of using multiple versions of a dependent package? In that case 'using' refers to any of the versions. So if we end up using Foo-1.0 and Foo-1.1 then both configurations are active: configuration: using(Foo>1.0) configuration: using(Foo<=1.0) Is there any way to make the order of configurations not matter? suppose I've got a configuration: configuration: using(Foo) build-depends: Bar configuration: using(Bar) build-depends: Foo We could ban it, or do it in order, or iterate 'til fixpoint. I think iterating 'til fixpoint would be terminating. Duncan
participants (3)
-
Duncan Coutts -
Einar Karttunen -
Simon Marlow