Extending the dependency syntax

newer
RE: Extending the dependency syntax

Simon Marlow

28 Jul 2005 28 Jul '05

5:13 a.m.

So here's a strawman proposal to address some of the use cases raised recently[1] that Cabal doesn't currently support well. This is just a first attempt; I expect we'll need to make changes or start over if we discover cases that it doesn't handle. [1] http://www.haskell.org//pipermail/libraries/2005-July/004152.html http://www.haskell.org//pipermail/libraries/2005-July/004133.html Firstly, we define certain pseudo-packages on which dependencies can be expressed. The pseudo packages would consist of the compilers and pre-processors that Cabal understands, and any other tools required by the build process. eg. build-depends: ghc>=6.4 happy>=1.5 base>=1.0 Now, I suggest we expand the language of dependencies to include disjunction and optional stanzas. The syntax I have in mind is this: deps ::= adep* dep-expr ::= deps '||' dep-expr adep ::= pkg version-range | '(' dep-expr ')' | '[' stanza ']' The idea is that Cabal evaluates the dependencies trying disjunctions left-to-right, where optional stanzas [foo] are assumed to be satisfied. When a set of satisfying dependencies has been found, Cabal processes all the optional stanzas it contains. We now need to be able to accumulate values from fields specified multiple times, rather than require each field to be specified at most once. Note that a list of dependencies can be empty, so an optional dependency is specified as (|| dep). Here's an example where a library can be built by multiple compilers, where each compiler requires different modules to be compiled: ---------------- name: mylib version: 1.0 exposed-modules: My.Lib build-depends: base>=1.0 (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs]) [ghc62] other-modules: Compat.Ghc62 [hugs] other-modules: Compat.Hugs ----------------- This covers Duncan's requirements, and Brian Smith's 1(a),1(b),1(d),2(a). 1(c) is not covered yet:

...

(c) Cabal depends on HUnit if DEBUG is set, but it doesn't depend on HUnit if DEBUG is not set. (The dependency isn't controlled by the existence of a package, but by a configuration option.)

This is related to the concept of "configurations", which we need in Visual Studio too. The idea is that a package can be built in one of several different ways, where each way specifies different compiler options or other settings. A common use is to have "debug" and "release" configurations, where in the former we #define DEBUG, and in the latter we turn on -O. How might we include configurations in Cabal? A configuration selection behaves rather like a dependency that is satisfied by the environment - perhaps as a command line option to Setup.lhs. We could extend the dependency syntax to include a new kind of dependency: adep ::= ... | config Perhaps the concrete syntax for config is a string (eg. "DEBUG"). We can express the HUnit example now: ---------------- build-depends: base>=1.0 ("DEBUG" HUnit>=1.0 [debug] || [release]) [debug] other-modules: Tests ghc-options: -DDEBUG -Onot [release] ghc-options: -O ---------------- Comments? Cheers, Simon

Show replies by date

Brian Smith

28 Jul 28 Jul

4:13 p.m.

[ I didn't see this message when I sent my previous two, sorry.] On 7/28/05, Simon Marlow wrote:

...

So here's a strawman proposal to address some of the use cases raised recently[1] that Cabal doesn't currently support well. This is just a first attempt; I expect we'll need to make changes or start over if we discover cases that it doesn't handle.

[1] http://www.haskell.org//pipermail/libraries/2005-July/004152.html http://www.haskell.org//pipermail/libraries/2005-July/004133.html

Firstly, we define certain pseudo-packages on which dependencies can be expressed. The pseudo packages would consist of the compilers and pre-processors that Cabal understands, and any other tools required by the build process. eg.

build-depends: ghc>=6.4 happy>=1.5 base>=1.0

This is basically the same thing as what I was trying to do with my proposed "Supported-Configuration." I agree that it makes sense to include them together in build-depends. I seperated them in the hope of backwards-compatibility, but if that is not a requirement then your idea is definitely better.

...

Now, I suggest we expand the language of dependencies to include disjunction and optional stanzas.

<snip>

...

The idea is that Cabal evaluates the dependencies trying disjunctions left-to-right, where optional stanzas [foo] are assumed to be satisfied. When a set of satisfying dependencies has been found, Cabal processes all the optional stanzas it contains. We now need to be able to accumulate values from fields specified multiple times, rather than require each field to be specified at most once.

<snip>

...

Here's an example where a library can be built by multiple compilers, where each compiler requires different modules to be compiled:

---------------- name: mylib version: 1.0 exposed-modules: My.Lib build-depends: base>=1.0 (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs])

[ghc62] other-modules: Compat.Ghc62

[hugs] other-modules: Compat.Hugs -----------------

This covers Duncan's requirements, and Brian Smith's 1(a),1(b),1(d),2(a). 1(c) is not covered yet:

...
(c) Cabal depends on HUnit if DEBUG is set, but it doesn't depend on HUnit if DEBUG is not set. (The dependency isn't controlled by the existence of a package, but by a configuration option.)

Perhaps the concrete syntax for config is a string (eg. "DEBUG"). We can express the HUnit example now: ---------------- build-depends: base>=1.0 ("DEBUG" HUnit>=1.0 [debug] || [release])

[debug] other-modules: Tests ghc-options: -DDEBUG -Onot

[release] ghc-options: -O ----------------

To me, this is saying "use the debug configuration if "DEBUG" is set and "HUnit" is available, otherwise use the release config." I think that instead, we should have only "use the debug configuration if DEBUG is set, otherwise use RELEASE." We should give an error message to the user when they request the DEBUG version without having HUnit installed, instead of silently using the RELEASE config. We could just say that you can name configurations on the command line, and not have any defaulting mechanism (i.e. you must always request either RELEASE or DEBUG). I think that the [stanza name] syntax is not clear enough. How about "Configuration: [name]"? I am also unsure how to represent my use case 2(a) in your proposal: Build an executable if wxHaskell is available, otherwise skip it. The idea is that the whole build should be a failure if the command-line version's dependencies are not satisfied, but that the GUI can be skipped with just a warning if its dependencies are not satisfied. Would you write "[Executable: gui]" instead of "Executable: gui"? In the example below (modified from yours), I added an "Optional" modifier to the "Executable" stanza syntax to indicate this. ---------------- name: mylib version: 1.0 exposed-modules: My.Lib build-depends: base>=1.0 (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs [hugs]) Optional Executable: gui Build-Depends: wx Main-Is: GUIMain Executable: commandline Main-Is: Main Configuration: DEBUG other-modules: Tests ghc-options: -DDEBUG -Onot build-depends: HUnit>=1.0 Configuration: RELEASE ghc-options: -O Configuration: ghc62 other-modules: Compat.Ghc62 Configuration: hugs other-modules: Compat.Hugs ---------------- - Brian

Isaac Jones

5:23 p.m.

"Simon Marlow" writes:

...

So here's a strawman proposal to address some of the use cases raised recently[1] that Cabal doesn't currently support well. This is just a first attempt; I expect we'll need to make changes or start over if we discover cases that it doesn't handle.

(snip proposal) Lots of this is stuff I've been wanting to ad at some point. I think that rather than overloading the build-depends field, we should add a new field for tool dependencies; otherwise, we could get name conflicts between the two and such. I'm not positive that the simple build infrastructure would be able to take all of the information in the tool-dependendencies into account; for instance, discovering the version of happy and ghc might be a bit tricky and error prone. (snip)

...

---------------- name: mylib version: 1.0 exposed-modules: My.Lib build-depends: base>=1.0 (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs])

Debian has something like this, where you can specify optional dependencies. I've always planned to implement something like this for Cabal. At the same time, I think we need to be careful not to re-invent make and re-invent dpkg / rpm via feature-creep. I understand Duncan's desire to have absolutely everything automated for gentoo without the gentoo packager having to intervene, but that's just what the OS packagers have been dealing with for every language. I want Haskell to be easier than other languages (historically, it's been harder) but if we make the package syntax more and more complex, I think we run the risk of expending a lot of energy for a small amount of benefit by creating a whole new language for packages. OTOH, maybe Haskell can become the generic packaging language, and people can start to use it to package everything... if anyone can create that, the Haskell community can ;) I'd really rather see the expansion of Cabal features going in a different direction: the Distribution library. Wouldn't it be great to have an embedded domain-specific language for packaging / building inside Haskell? Then the setup script could be like the sane man's makefile. As Hudak (I think) points out in a paper on embedded domain-specific languages, many languages grow from DSLs (like the .cabal file) into general-purpose languages. The reason I like the setup script is so that we don't have to keep expanding the .cabal file; we have all the power of Haskell at our fingertips in the Setup file. For Debian packages in most languages, we have a hunk of executable stuff for building and installing, and a relatively small amount of metadata to deal with dependencies and what-have-you. I think we should try to stick to that model wherever possible, of course, improving upon Debian where we can. I'm not saying that Simon's ideas can or should be implemented in the Setup script; but that's just something to think about when we're talking about expanding the .cabal file syntax.

...

[ghc62] other-modules: Compat.Ghc62

[hugs] other-modules: Compat.Hugs

Here we're starting to get complicated, though, and I'm a bit scared of this complication. Presumably, then you have something in the source code to optionally #include the correct module? This is perhaps one of the few places where I'd rather see Haskell itself altered to address this need than to patch it with Cabal. Anyway, these are all good ideas, this is just my first reaction and I'll keep thinking, and we should keep discussing this. More later. peace, isaac

Duncan Coutts

5:47 p.m.

On Thu, 2005-07-28 at 14:23 -0700, Isaac Jones wrote:

...

I'd really rather see the expansion of Cabal features going in a different direction: the Distribution library. Wouldn't it be great to have an embedded domain-specific language for packaging / building inside Haskell? Then the setup script could be like the sane man's makefile. As Hudak (I think) points out in a paper on embedded domain-specific languages, many languages grow from DSLs (like the .cabal file) into general-purpose languages. The reason I like the setup script is so that we don't have to keep expanding the .cabal file; we have all the power of Haskell at our fingertips in the Setup file.

There is a tension here between the needs of the package authors and the needs of the packaging systems. And it affects the language that packages are written in. A declarative system that provides lots of information is what the packaging systems want. So they know all dependencies and can have flexibility over building and deploying packages (to do things like build in a protected sandbox, or to build a binary package on one machine and install it on another). Package authors might want more of a complete programming language that allows them to to whatever is needed to build install and register their software. The downside of the latter approach is that it is nigh impossible to make a native package that follows the policies of the native packaging system (like not running arbitrary scripts/programs on the target or build machine with root privileges). My own personal opinion is that cabal/haskell packaging can be made the most sucessful if we take advantage of the native packaging systems and so make it possible to automate as much as possible the process of wrapping a cabal package into a native package. I think we will reach the greatest audience by this approach. Duncan

Ian Lynagh

6:33 p.m.

On Thu, Jul 28, 2005 at 02:23:55PM -0700, Isaac Jones wrote:

...

"Simon Marlow" writes:

...
So here's a strawman proposal to address some of the use cases raised recently[1] that Cabal doesn't currently support well. This is just a first attempt; I expect we'll need to make changes or start over if we discover cases that it doesn't handle.

(snip proposal)

Lots of this is stuff I've been wanting to ad at some point. I think that rather than overloading the build-depends field, we should add a new field for tool dependencies; otherwise, we could get name conflicts between the two and such.

Agreed; currently we know that everything in build-depends is a cabal package we need to depend on, which makes things simpler.

...

...
build-depends: base>=1.0 (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs])

For libraries we(Debian)'d need to be guaranteed that the left-most of the satisfactory options had been chosen. Otherwise, having built a library which "build-depends: foo || bar", I don't think we could work out whether we should depend on libghc6-foo-dev or libghc6-bar-dev (or equivalent for another impl). Thanks Ian

Duncan Coutts

5:36 p.m.

On Thu, 2005-07-28 at 10:13 +0100, Simon Marlow wrote:

...

Now, I suggest we expand the language of dependencies to include disjunction and optional stanzas. The syntax I have in mind is this:

deps ::= adep* dep-expr ::= deps '||' dep-expr adep ::= pkg version-range | '(' dep-expr ')' | '[' stanza ']'

The idea is that Cabal evaluates the dependencies trying disjunctions left-to-right, where optional stanzas [foo] are assumed to be satisfied. When a set of satisfying dependencies has been found, Cabal processes all the optional stanzas it contains.

While the sensible normal behaviour might be to just pick up as many dependencies as the environment supports, packaging systems like to know about all dependencies and control them. For example it is normal to build a package on one system and deploy it on another. If there are extra dependencies that are picked up (because they happen to be in the build environment) it may cause a failure on the target system because the dependency is not present. There is an article that explains this issue in more detail: http://www.onlamp.com/pub/a/onlamp/2005/03/31/packaging.html?page=2 See particularly the section on automatic decisions: http://www.onlamp.com/pub/a/onlamp/2005/03/31/packaging.html?page=2#automati... As an example consider Gtk2Hs which has a number of optional dependencies (libglade, GConf, Mozilla). If you do a normal build then it will automatically build all the optional bits for which the dependencies are satisfied. This is what casual users want. However for packagers we provide a --enable-packager-mode flag which turns off all automatic dependency decisions and so the packager must explicitly specify --enable-gconf --enable-libglade etc. In the Gentoo Gtk2Hs package we reflect these options so that a user may control optional dependencies, eg: USE="gnome -mozilla" emerge gtk2hs translates into enabling the dependencies on gnome and disabling the ones on mozilla. Now if a cabal package automatically picks up dependencies then we have a problem that we may build a package and have it pick up an unrecorded dependency on a package that is in the environment. So we would need to take a worst case approach and specify that all optional dependencies are in fact required. [ Why is this so: consider a package that has an optional dependency on a gui library. We build it and it picks up a dependency on the gui lib. Now if this dependency is not recorded then the package manager can make incorrect decisions (like allowing the gui lib to be uninstalled since it believes there are nothing using it). So the obvious solution is to make the gui lib a dependency of the package in question. But now we have defeated the purpose of of the convenient automatic dependency selection by making everything required. ] So the point is, the package manager must have complete visibility over the dependencies that the package picks up. So I would suggest that while the default should be to pick up dependences automatically that it be possible for a packaging tool to discover what the optional dependencies are and to force the decision rather than letting it be automatic (eg to allow not building the gui even though the gui lib is available on the build machine). example: ---------------- name: myprog version: 1.0 executable: mycliprog build-depends: base>=1.0 (|| gtk>=0.9.8 [gui]) (|| hscgi>=1.0 [cgi]) [gui] executable: myguiprog [cgi] executable: mycgiprog ----------------- Then when we configure we want to say something like: ./setup configure --without-features=gui --with-features=cgi So in summary, if optional things like this are to be introduced we have to be careful to design it in such a way that it is usable by packaging systems. Duncan

Ian Lynagh

6:41 p.m.

On Thu, Jul 28, 2005 at 10:36:00PM +0100, Duncan Coutts wrote:

...

name: myprog version: 1.0 executable: mycliprog build-depends: base>=1.0 (|| gtk>=0.9.8 [gui]) (|| hscgi>=1.0 [cgi])

Unless I'm confused I think what you mean should be written build-depends: base>=1.0 (gtk>=0.9.8 [gui]) (hscgi>=1.0 [cgi]) I'd say that, with what you wrote, not having hscgi installed would satisfy the requirements for the cgi option. (If I said I prefered this syntax: build-depends: base (>= 1.0), gtk [gui] | ctk [cli], hscgi (>=1.0) [cgi] would you all complain I was trying to make it a copy of Debian's syntax? :-) ) Thanks Ian

7236

Age (days ago)

7236

Last active (days ago)

List overview

Download

6 comments

5 participants

participants (5)

Brian Smith
Duncan Coutts
Ian Lynagh
Isaac Jones
Simon Marlow