On 2008.06.13 22:22:06 +0100, Duncan Coutts scribbled 2.1K characters:

On Fri, 2008-06-13 at 11:19 -0400, Darrin Thompson wrote:

...

Cabal-install is looking good. It now, for the record, has only two deps outside of Cabal-1.4.

I installed cabal-install-0.5 on ubuntu with the haskell.org linux binary for ghc 6.8.2.

I then tried cabal-install yi

Got this output: ... many successful installs ... Registering vty-3.0.1... Reading package info from "dist/installed-pkg-config" ... done. Saving old package config file... done. Writing new package config file... done. Downloading yi-0.3... [1 of 1] Compiling Main ( Setup.hs, dist/setup/Main.o )

Setup.hs:9:0: Warning: Deprecated use of `showPackageId' (imported from Distribution.Simple, but defined in Distribution.Package): use the Text class instead Linking dist/setup/setup ... Warning: defaultUserHooks in Setup script is deprecated. Configuring yi-0.3... Warning: Instead of 'ghc-options: -DDYNAMIC -DFRONTEND_VTY' use 'cpp-options: -DDYNAMIC -DFRONTEND_VTY' setup: alex version >=2.0.1 && <3 is required but it could not be found. cabal: Error: some packages failed to install: yi-0.3 failed during the configure step. The exception was: exit: ExitFailure 1

Is this a cabal problem of package problem? I would have expected it to fail immediately instead of discover this problem so far into process.

So it's failing when configuring yi because alex is not installed.

So the question is when should this get checked? I think it's not unreasonable to have the check where it is now though perhaps we could do better by bringing it forward.

The point is, cabal-install only checks that haskell packages are available before beginning to install stuff. It leaves all the other checks (build tools, C libs, custom checks in Setup.hs script etc) to be done at the configure phase of each package.

In theory it's not impossible to imagine doing the configure phase of every package before doing the build phase of any package but it's not clear that it gains us that much and it's a bit more complex to do it that way.

Duncan

I think this may be Cabal's fault anyway. The yi.cabal includes the line: build-tools: alex >= 2.0.1 && < 3 in the 'executable yi' section, right after the build-depends, so Yi is being straightforward and upfront about its needs. Now, Cabal is obviously checking that the build-depends is satisfied first, but why isn't it checking that alex is available when it has the information it needs to check, presumably anything in build-tools is *required*, and the field name suggests that it would be checked? -- gwern watchers 5.0i Fax UXO NORAD Consulting meta Gatt of data

http://hackage.haskell.org/trac/hackage/ticket/227

One problem is that not all build-tools correspond to haskell packages. Some do some don't. We have a hard coded list of them at the moment (which can be extended in Setup.hs files) so we could extend that with what haskell package if any the tools correspond to. Any better suggestions to make it a tad more generic?

That list seems to be in Distribution/Simple/Program.hs, in case anyone else is looking for it. Encoded information includes: dependency name, program name (if different from dependency), option to get version info and code to extract it (with one apparently very special case being hsc2hs). Btw, most of the version extraction code looks like a regular expression match - wouldn't that make the specification easier (and turn the comments into part of the spec)?

If anyone has a godd suggestion I'm happy to hear it. Otherwise we can just add a Maybe Dependency to the Program type to indicate that some build tools have a corresponding haskell package.

I'm not sure what you're suggesting there (currently even those tools that can be built with Cabal do not register with Cabal), but here are my suggestions: 1. Haskell tools should register with Cabal, whether built with it (such as Alex, Happy, ..) or not (such as GHC, ..). That registration should include any build-relevant information (versions/variants, ..). 2. When checking a build-tools dependency, Cabal checks (a) whether the tool is registered with Cabal (b) whether the tool is registered with the system installation manager (c) whether the tool can be found by other means (configure, built-in rules, ..) In other words, make tools look like packages (lifetime dependency management, not just build support), and make system packages look like Cabal packages (Cabal as the interface to native managers), using special treatment only if absolutely necessary. I thought those suggestions were clear from my earlier messages?-) Claus http://www.haskell.org/pipermail/haskell-cafe/2008-May/043368.html http://www.haskell.org/pipermail/haskell-cafe/2008-June/043977.html http://www.haskell.org/pipermail/haskell-cafe/2008-June/043910.html

The main point of the Program abstraction is about configuring and running programs. As it happens some programs are provided by some haskell packages (but not all, eg ld, ar, etc).

...

option to get version info and code to extract it (with one apparently very special case being hsc2hs).

And ld.exe on windows (we find it in ghc's gcc-lib bin dir).

I didn't notice this special case in Program.hs - are my sources just out of date, or is this special handling encoded elsewhere?

...

Btw, most of the version extraction code looks like a regular expression match - wouldn't that make the specification easier (and turn the comments into part of the spec)?

True, in most cases finding the name of the program involves running it with some --version flag and matching some part of the output. However that's not always the case. Some programs do silly things like produce the version output on stderr instead of stdout. We figured the most general thing was just a function

FilePath -> IO (Maybe Version)

which is what we've got. I'm not sure what the advantage would be to make it more declarative by making it into data rather than a extraction function.

Usually, the most general approach for this kind of problem is good as a default/backup, but not so good for maintenance. The more cases can be handled with dedicated declarative specs, the smaller the risk of accidental/hidden breakage (eg, moving the comments into regex patterns), the more compact and understandable the specs (a line per tool, with option and regex, all in one place would be easier to comprehend than the current free-coding style). More importantly, concise declarative specs are easier to update and extend (could be done by users rather than Cabal maintainers, and preferably outside of Cabal sources).

Also, the Cabal lib cannot depend on any regular expression library because they are not part of the bootstrapping library set.

Sigh. I've heard that one before, and the ghc-pkg bulk queries are not as flexible as they could be because I had to use less expressive functions instead of regexes. Since Haskell-only regex packages exist, perhaps one should just be added to the bootlib set? After all, we bother with regexes because they are so frequently useful.

...

1. Haskell tools should register with Cabal, whether built with it (such as Alex, Happy, ..) or not (such as GHC, ..). That registration should include any build-relevant information (versions/variants, ..).

2. When checking a build-tools dependency, Cabal checks (a) whether the tool is registered with Cabal

I'm not sure this helps. We want to know what to install when it's missing. We can already tell if a program (not package) is available by searching for it.

But that means either configure or a subset of special configure rules baked into Cabal's sources, combined with fragile Setup.hs extensions to that subset. IMHO, the less of configure/hardcoded rules/Setup.hs, the better (simpler, less breakage with Cabal updates, etc.). How about this: instead of baking rules for those tools into the Cabal sources, why not have a single "known-tools" package? On (re-)installing that package, its configure/Setup is run once, to register availability and build-related information for all those tools with Cabal (currently, that would mean a package per tool; later, the known-tools package itself could expose multiple tools, but Cabal would still need to be able to check which of the "exposed-tools" have been found). That way, the known-tools could be updated independently, instead of requring Cabal source hacking and releases, and instead of spreading special configure rules for common tools over all packages, they'd be located in a single package (easier to maintain and improve, and improvements are shared).

...

(b) whether the tool is registered with the system installation manager

This is hard.

Why? Because there are more installation managers than OSs, or because so many program installs bypass them? Querying an installation manager shouldn't be any more difficult than querying ghc-pkg, say. If a program is registered with, eg. Windows Add/Remove Programs, I can use "reg query <key>" to find its registration info, and if I don't know the key, I can use reg query HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall /s to list all registered programs, or reg export HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall <file> to get all registration info dumped to a file, and look for DisplayNames of the tools I'm interested in:

reg query HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall /s | find "DisplayName" | find "Opera" DisplayName REG_SZ Opera 9.50

reg query HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall /s | find "DisplayName" | find "Hugs" DisplayName REG_SZ WinHugs

reg query HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall /s | find "DisplayName" | find "Haskell" DisplayName REG_SZ Glasgow Haskell Compiler, version 6.4.1

Of course, the tools I'm interested in very often bypass the Windows installer (eg, I prefer tarballs for GHC, and GHC comes with its own internally used set of tools, or Cygwin handles its own tools itself, and so on) and so won't appear in that output.. Claus

...

(c) whether the tool can be found by other means (configure, built-in rules, ..)

In other words, make tools look like packages (lifetime dependency management, not just build support), and make system packages look like Cabal packages (Cabal as the interface to native managers), using special treatment only if absolutely necessary. I thought those suggestions were clear from my earlier messages?-)

Sure, haskell executable tools built by haskell packages are really packages.

So the problem currently is that build-tools refers to programs not packages. One can list any known program in the build-tools field and the list of programs is extensible in the Setup.hs script.

As they are used so far by current packages on hackage, they always refer to haskell tools built by haskell packages (almost only used to refer to alex, happy and c2hs) so perhaps we should just quietly redefine the meaning on build-tools to be another kind of build-depends. That is it specifies a haskell package.

If we have need to specify non-haskell build tools, perhaps we should do that separately? eg "some-other-kind-of-build-tools: perl >= 5.8"

Duncan

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

In the specific case of parsing --version strings, I'm not sure regexes are any easier. Using words and then selecting the Nth word seems to do pretty well.

for comparison, see the examples below:-) There's no reason this can't be massaged further, but it already allows for IO if necessary (I like that it is compact and self-documenting -the comments have become part of the patterns, but if you don't want to match against the precise message format, you can use more of [^[:space:]]* - pity that \S* isn't posix..). Claus import Text.Regex import System.Process(runInteractiveProcess) import System.IO(hGetContents) test = mapM_ ((print =<<) . getVersion) programs getVersion (prg,flag,io,extract) = do (i,o,e,p) <- runInteractiveProcess prg [flag] Nothing Nothing version <- hGetContents ([o,e]!!(io-1)) >>= extract return (prg,version) defaultMatch :: String -> String -> IO String defaultMatch pattern = return . maybe "" head . (matchRegex (mkRegex pattern)) programs = [ghc,ghcPkg,happy,alex,haddock] ghc = ("ghc","--numeric-version",1, defaultMatch "([^[:space:]]*)") ghcPkg = ("ghc-pkg","--version",1, defaultMatch "GHC package manager version ([^[:space:]]*)") happy = ("happy","--version",1, defaultMatch "Happy Version ([^[:space:]]*)") alex = ("alex","--version",1, defaultMatch "Alex version ([.0-9]*)") haddock = ("haddock","--version",1, defaultMatch "Haddock version ([.0-9]*)")