August 2005 - Libraries

RE: Extending the dependency syntax
by Simon Marlow 12 Aug '05

12 Aug '05

On 10 August 2005 06:56, Brian Smith wrote: > On 8/9/05, Simon Marlow <simonmar(a)microsoft.com> wrote: >> ------------------ >> build-depends: >> ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) >> | hugs? [hugs], debug? >> (HUnit-1.0, [debug]) >> [release], >> gnome? ( libglade >= 2, >> gtksourceview >= 0.6, >> gconf >= 2, [gnome] ), >> mozilla? ( mozilla >= 1.4, [mozilla] ), >> doc? ( haddock >= 0.6 ) >> >> [gnome] >> extra-libraries: gnome >> extra-ghc-options: -DENABLE_GNOME >> ... >> >> [debug] >> extra-ghc-options: -O0 -DDEBUG >> >> [release] >> extra-ghc-options: -O2 > > As described, it seems like I would need to pass "--enable-hugs" or > "--enable-ghc" explicitly, or else this program's build would fail. > But, I don't think that is your intention; instead you are intending > for Cabal to automatically enable these flags as appropriate, right? > Perhaps there would be other automatic flags too, like "windows" and > "unix"? Actually I made a mistake in the above example, I meant to write build-depends: ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) (hugs? [hugs]) but anyway, this syntax has caught several people out, me included, so we should pick something better (again). It seems that most people would be happier if x? (a) | b means b if x is false. I think we can fix it, though. How about this syntax: deps ::= choice | choice ',' deps choice ::= alt1 '|' ... '|' altN (n >= 1) | adep alt ::= adep '?' adep adep ::= pkg version-range | '(' deps ')' | --enable-flag | true The point is that now a disjunction is always of the form (x1?(a1) | ... | xn?(an)), and the meaning is that we take the leftmost branch for which x is true. It's a case statement. Also, to avoid some of the overloading you pointed out, I'm writing flags as --enable-flag. The example in this syntax: build-depends: --enable-ghc? ( ghc>=6.4 ? [ghc64] | ghc>=5.04 ? [ghc-old]) | --enable-hugs? [hugs] | true ? true, -- other compilers are allowed --enable-debug? (HUnit-1.0, [debug]) | true? [release] --enable-gnome? ( libglade >= 2, gtksourceview >= 0.6, gconf >= 2, [gnome] ), --enable-mozilla? ( mozilla >= 1.4, [mozilla] ), --enable-doc? ( haddock >= 0.6 ) I'm not wildly excited about "true ? true", but we could add special syntax for that if it turns out to be used often. > Can configurations contain their own "Build-depends", like this?: > > [windows] > Build-depends: Win32 > extra-libraries: gnome The problem I have with this is that it means you can only decide dependencies based on the settings of flags, rather than the other way around. We use both in the above example - [ghc64] is decided based on the availability of ghc versions, whereas [mozilla] is chosen based on a configuration flag. You had to hack it in your example: > I think it would be clearer to specify the configuration seperately, > like this: > > configuration: > ghc >= 6.4 ? [ghc64] | ghc >= 5.04 ? [ghc-old]) | hugs? > [hugs], debug enabled ? [debug] | [release], > gnome enabled ? [gnome], > mozilla enabled ? [mozilla], > doc enabled ? [doc] > -- the "global" Build-depends applies to everything > Build-depends: base >= 1.0, haskell98 >= 1.0 So configuration contains dependencies like ghc >= 6.4. I think it's clearer to put all the dependencies in one place. > Finally, are you intending to allow multiple executable stanzas inside > configurations?: I've been avoiding this issue for now. I believe we need to completely redesign the way multiple executables are handled, and provide a more general framework for a package that contains multiple libraries & executables (essentially a wrapper for multiple packages with interdependencies). I know Isaac is thinking about this too. Let's deal with one issue at a time. Cheers, Simon

3 2

Cabal on OS X; ghc segfault?
by Gregory Wright 11 Aug '05

11 Aug '05

Hi, I'm trying to get caballized package deployment working on Mac OS X. However, trying to build a package using runhaskell results in: crossroads-able> runhaskell Setup.hs configure Warning: No license-file field. Configuring harp-0.2... configure: searching for ghc in path. configure: found ghc at /opt/local/bin/ghc runhaskell: waitForProcess: interrupted (Interrupted system call) This was trying to build the harp module in the haskell-src-exts package (a prerequisite for hs-plugins). I ran a ktrace on this and it appears as if the underlying GHC is segfaulting: crossroads-able> cat k.txt 414 ghc-6.4 RET write 1 414 ghc-6.4 CALL getpid 414 ghc-6.4 RET getpid 414/0x19e 414 ghc-6.4 CALL stat(0x1b94528,0x1b944c0) 414 ghc-6.4 NAMI "tmp414" 414 ghc-6.4 RET stat -1 errno 2 No such file or directory 414 ghc-6.4 CALL sigprocmask(0x1,0xbfffd4d0,0xbfffd4d4) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL sigaction(0x2,0xbfffd458,0) 414 ghc-6.4 RET sigaction 0 414 ghc-6.4 CALL sigprocmask(0x3,0xbfffd4d4,0) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL sigprocmask(0x1,0xbfffd4d0,0xbfffd4d4) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL sigaction(0x3,0xbfffd458,0) 414 ghc-6.4 RET sigaction 0 414 ghc-6.4 CALL sigprocmask(0x3,0xbfffd4d4,0) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL fcntl(0,0x3,0) 414 ghc-6.4 RET fcntl 6 414 ghc-6.4 CALL fcntl(0,0x4,0x6) 414 ghc-6.4 RET fcntl 0 414 ghc-6.4 CALL ioctl(0,0x4004667a ,0xbfffd4b8) 414 ghc-6.4 RET ioctl 0 414 ghc-6.4 PSIG SIGSEGV SIG_DFL 413 runhaskell RET wait4 414/0x19e 413 runhaskell CALL write(0x2,0xbfffcf40,0xc) 413 runhaskell GIO fd 2 wrote 12 bytes "runhaskell: " 413 runhaskell RET write 12/0xc 413 runhaskell CALL write(0x2,0xbfffcfa0,0x36) 413 runhaskell GIO fd 2 wrote 54 bytes "waitForProcess: interrupted (Interrupted system call) " This is a bit surprising. Any suggestions on how to track down the problem? Best Wishes, Greg

5 5

RE: package description fields
by Simon Marlow 10 Aug '05

10 Aug '05

On 09 August 2005 11:23, Ross Paterson wrote: > On Tue, Aug 09, 2005 at 09:02:50AM +0100, Simon Marlow wrote: >> On 08 August 2005 16:57, Jonathan Cast wrote: >> [Simon wrote:] >>>> >>>> --bindir=<dir> >>>> --libdir=<dir> >>>> --datadir=<dir> >>>> >>>> where the values of these would be directories relative to $prefix >>>> (or absolute? relative to $prefix seems nicer). >>> >>> Relative paths here break compatibility with Autoconf; when I >>> install a program, I don't care where its configure script comes >>> from, I just want to be able to invoke it the way I >>> expect to. Gratuitous incompatibilities between configure >>> scripts is very inconvenient. >> >> I do agree, and I half-expected someone to point out the >> inconsistency. OTOH, this isn't a "configure script", it's the >> Setup.lhs program. You already have to know that it isn't an >> autoconf configure script, because it has different syntax. > > Another incompatibility is that configure has options --datadir etc > with a different meaning to yours. I think yours should be renamed; > we might want the autoconf-style ones too. Really? I intended them to work in the same way. What differences are you referring to? If you think this is a generalisation too far, I guess I could live with having fixed values for all these, but they should be relative to $prefix, and that still leaves us with Krasimir's complaint about the default $bindir/$libdir on Windows. >> One reason I suggested that relative paths would be better is >> because it allows you to build a binary/library that is >> prefix-independent, at least on Windows where a binary knows the >> absolute $bindir and hence can derive $prefix because it knows >> $bindir relative to $prefix. > > What would the code in the packaged program look like? It seems you'd > want some library support for this. I suppose you'd have something like this: getPrefix :: FilePath -- bindir relative to prefix -> FilePath -- returns prefix but it only works on Windows. GHC, Happy, Alex and Haddock all have something like this internally (but bindir is either fixed or assumed to be equal to $prefix right now). From prefix you can get libdir and datadir, if these are also specified relative to prefix. > And what if the builder modified bindir? If the builder can modify bindir, the program gets the value from paths.h (or whatever mechanism we choose). Cheers, Simon

1 0

RE: Extending the dependency syntax
by Simon Marlow 10 Aug '05

10 Aug '05

On 09 August 2005 16:48, Isaac Jones wrote: > I would prefer something like: > > configuration: gnome > extra-libraries: gnome > extra-ghc-options: -DENABLE_GNOME > > to keep the stanza syntax consistent. That's fine by me. > Do you have any ideas for how to interact with environment variables? > I wasn't clear if your previous proposal included this or not... for > instance, you might want to say: > > DEBUG=true ./setup build > > (can you do that on Windows?) I'd like to be able to do this too (I'd intended it originally), but dropped it to avoid confusing the issue too much. > Do you actually like this better, or are you toning it down because of > my previous stressing over the complexity? I like it better, at least with the revisions in my most recent message. Cheers, Simon

1 0

RE: another wash compile problem
by Simon Marlow 10 Aug '05

10 Aug '05

On 10 August 2005 09:04, Frederik Eaton wrote: >>> Compiling a module which imports the 14 smaller modules takes much >>> less time than compiling the monolithic module - it's almost 5 times >>> faster (see below). >> >> We've found the cause of this, and committed a fix. The full >> HTMLMonda98.hs now compiles in 27 seconds for me, without >> optimisation (with 6.4 it is several minutes). Let's hope the same >> issue is affecting other compilations too :-) > > Wow! That's great. > > Have you looked at how compile time varies with module size in the > fixed version? Not in any detail; but I expect it to be worse than linear but not as bad as quadratic. There are several reasons it might be worse than linear: GC is linear in the residency, for one. And the compiler makes a lot of use of FiniteMap, which will add a (log n) factor to a traversal of the tree. We try to fix instances of quadratic behaviour as we find them. Cheers, Simon

1 0

RE: another wash compile problem
by Simon Marlow 10 Aug '05

10 Aug '05

On 06 July 2005 23:38, Frederik Eaton wrote: >> When I compile WASH it takes forever for ghc to finish compiling >> HTMLMonad98, and on my powerpc system with 151MB free RAM, gcc >> crashes during the compile. Should some of the larger WASH modules >> maybe be split into multiple files? I get the sense that ghc >> compilation time/memory is super-linear in the size of the module >> being compiled. > > I've now verified my earlier "ghc compilation complexity is > super-linear" belief. I split up HTMLMonad98 into 10 files, and then > split the 6th and largest, which consists of the following lines - > > instance AdmitChildHEAD HTML > instance AdmitChildFRAMESET HTML > ...(2692 lines) > instance AdmitChildLABEL ACRONYM > instance AdmitChildBUTTON ACRONYM > > into 14 files: > > $ wc -l HTMLMonad98No6.hs > 2696 HTMLMonad98No6.hs > $ wc -l HTMLMonad98No6?.hs > 200 HTMLMonad98No6a.hs > 204 HTMLMonad98No6b.hs > 204 HTMLMonad98No6c.hs > ... > 2748 total > > Compiling a module which imports the 14 smaller modules takes much > less time than compiling the monolithic module - it's almost 5 times > faster (see below). We've found the cause of this, and committed a fix. The full HTMLMonda98.hs now compiles in 27 seconds for me, without optimisation (with 6.4 it is several minutes). Let's hope the same issue is affecting other compilations too :-) Cheers, Simon

2 1

Extending the dependency syntax
by Brian Smith 10 Aug '05

10 Aug '05

On 8/9/05, Simon Marlow <simonmar(a)microsoft.com> wrote: > ------------------ > build-depends: > ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) > | hugs? [hugs], > debug? > (HUnit-1.0, [debug]) > [release], > gnome? ( libglade >= 2, > gtksourceview >= 0.6, > gconf >= 2, [gnome] ), > mozilla? ( mozilla >= 1.4, [mozilla] ), > doc? ( haddock >= 0.6 ) > > [gnome] > extra-libraries: gnome > extra-ghc-options: -DENABLE_GNOME > ... > > [debug] > extra-ghc-options: -O0 -DDEBUG > > [release] > extra-ghc-options: -O2 As described, it seems like I would need to pass "--enable-hugs" or "--enable-ghc" explicitly, or else this program's build would fail. But, I don't think that is your intention; instead you are intending for Cabal to automatically enable these flags as appropriate, right? Perhaps there would be other automatic flags too, like "windows" and "unix"? I think there is too much overloading being used: 1. "ghc" refers to both a flag and a package 2. "debug," "release," and "gnome," are flags and configurations. If I pass "--enable-gnome" as a flag then the "gnome" configuration gets used, if I pass "--enable-release" then the "release" configuration gets used, etc.. But, if I pass "--enable-ghc64" as a flag then the ghc64 configuration will not be used, right? Will the "--enable-ghc64" just be silently ignored? It would be nice to be able to catch typos like "--enable-debgu" What is the difference between the following?: ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) | hugs? [hugs], vs. ( ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old] | hugs, [hugs] | ) vs. ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old] | hugs, [hugs] My understanding is that the first two are equivalent but the third one additional means "fail if building with anything other than GHC >= 5.04 or hugs," assuming that Cabal automatically enables the flags "ghc" or "hugs" appropriately. Am I correct? "a, b" means "require a and b." But, "a, [b]" means "require dependency a and use configuration b" or, equivalently "if dependency a is satisfied, then automatically use configuration b." I think it would be better to avoid overloading the comma operator with these two meanings. Can configurations contain their own "Build-depends", like this?: [windows] Build-depends: Win32 extra-libraries: gnome It seems counterintuitive to allow "extra-libraries" but to disallow "build-depends." Yet, we may want to prohibit the build-depends flag from including other configurations syntax, to prevent cycles and non-intuitive behavior: Build-depends: windows? [windows] [windows] Build-Depends: Win32, wxHaskell [gui] | extra-libraries: gdi32 [gui] Build-Depends: wxHaskell, (Win32 [windows] | ) I think it would be clearer to specify the configuration seperately, like this: configuration: ghc >= 6.4 ? [ghc64] | ghc >= 5.04 ? [ghc-old]) | hugs? [hugs], debug enabled ? [debug] | [release], gnome enabled ? [gnome], mozilla enabled ? [mozilla], doc enabled ? [doc] -- the "global" Build-depends applies to everything Build-depends: base >= 1.0, haskell98 >= 1.0 [gnome] Build-Depends: libglade >= 2, gtksourceview >= 0.6, gconf >= 2, extra-libraries: gnome extra-ghc-options: -DENABLE_GNOME [debug] Build-depends: HUnit-1.0 extra-ghc-options: -O0 -DDEBUG [release] extra-ghc-options: -O2 [mozilla] Build-depends: mozilla >= 1.4 [doc] Build-depends: haddock >= 0.6 Notice here that the "Build-Depends:" syntax would remain the same as in Cabal 1.0, as [stanza] and the "?" operator would only be allowed in "configuration:". Also notice that this requires that dependencies be allowed as the left operand of the "?" operator, but they would no longer be allowed as the right operand; only stanza names are allowed. Finally, are you intending to allow multiple executable stanzas inside configurations?: Build-depends: wxHaskell, [gui] | [gui] Executable: admin-gui Main-Is: AdminGui.hs Executable: another-gui Main-Is: AnotherGui.hs Regards, Brian

1 0

RE: Extending the dependency syntax
by Simon Marlow 09 Aug '05

09 Aug '05

I'd like to wrap up this little design issue which stalled. I'll propose something a little simpler, which should hopefully be closer to what people are used to from other package systems, and which should translate more easily. To recap, the goal is to support: - build options that result in an executable or library being built in a different way, or with a particular feature. The build option may imply additional dependencies. - a choice between dependencies, where the particular choice made dictates certain configuration changes in the build info. - a choice between configurations, for example building a library with or without debugging support. An example first showing all the main ideas: ------------------ build-depends: ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) | hugs? [hugs], debug? (HUnit-1.0, [debug]) [release], gnome? ( libglade >= 2, gtksourceview >= 0.6, gconf >= 2, [gnome] ), mozilla? ( mozilla >= 1.4, [mozilla] ), doc? ( haddock >= 0.6 ) [gnome] extra-libraries: gnome extra-ghc-options: -DENABLE_GNOME ... [debug] extra-ghc-options: -O0 -DDEBUG [release] extra-ghc-options: -O2 ------------------- The semantics of build-depends: - ',' means "and", just as it does now - '|' means "or" - parentheses group dependencies - [name] means "read the optional stanza [name]" - 'flag? dep' means "if the --enable-flag option is given, then dep, otherwise nothing". - 'flag? dep1 dep2' menas "if the --enable-flag option is given then dep1, else dep2". Additionally, we could have an --auto-enable flag, which has the effect of replacing every 'flag? dep' by 'dep|', and 'flag? dep1 dep2' by 'dep1|dep2'. This has the effect of turning on every option for which the dependencies are satisfied, which is normally what you want when building a package for local use. I'm not attached to any particular syntax, so feel free to suggest alternatives. This is somewhat simpler than before, and I think supports everything we need. Comments? I'm happy to implement if there's a concensus. Cheers, Simon

2 1

RE: Foreign language parsers
by Simon Marlow 09 Aug '05

09 Aug '05

On 05 August 2005 12:37, Dominic Fox wrote: > I'm trying to collate a list of currently extant Haskell projects that > include parsers for foreign languages such as PHP and C. > > To this end, I've created a wiki page at > http://haskell.org/hawiki/ParsingForeignLanguagesInHaskell. > > Has anyone looked at positioning foreign language parsing modules in > the standard libs, or specifying a common framework or contract for > such modules? This is what the Language.* branch of the hierarchy was intended for, incedentally. Language.Haskell is currently populated, but I'm not aware of any other libraries under Language. Cheers, Simon

3 4

RE: package description fields
by Simon Marlow 09 Aug '05

09 Aug '05

On 08 August 2005 16:57, Jonathan Cast wrote: >> On 04 August 2005 08:24, Krasimir Angelov wrote: >> >>> I would like to propose more Unix like directory layout for Windows >>> i.e.: >>> >>> executables: %ProgramFiles%\<pkg-name>\bin >>> libraries: %ProgramFiles%\<pkg-name>\lib >>> data files: %ProgramFiles%\<pkg-name>\data >>> >>> The advantage is that if you have a lot of libraries and data files >>> then it is easier to find out the executable files. Happy, Alex and >>> Haddock already use layout like in the Ross's proposal but I found >>> it very inconvenient because usually I am building them together >>> with ghc. The problem is that when I do make install in the fptools >>> directory then ghc is installed in ${prefix}/bin but the above tools >>> are installed in ${prefix}. Usually then ${prefix} directory isn't >>> included in the PATH and this makes them inaccessible. I can move >>> them to ${prefix}/bin but then I have to move all other template >>> files too. It would be nice if we had more consistent directory >>> layout. >> >> Your complaint here seems to be that the default layout of Happy, >> Alex and Haddock is different from that of GHC on Windows, which >> makes it harder to combine all these tools in a single package. >> >> As far as Cabal is concerned, I think the defaults are fine. >> However, I do think we should allow the default >> $bindir/$libdir/$datadir to be overridden (I know this has been >> discussed before). On Windows, an application often finds its files >> relative to the location of the executable, so the values of >> $bindir/$libdir/$datadir need to be passed to the program at build >> time, perhaps via CPP symbols. So, I suggest we have: >> >> --bindir=<dir> >> --libdir=<dir> >> --datadir=<dir> >> >> where the values of these would be directories relative to $prefix >> (or absolute? relative to $prefix seems nicer). > > Relative paths here break compatibility with Autoconf; when I > install a program, I don't care where its configure script comes > from, I just want to be able to invoke it the way I > expect to. Gratuitous incompatibilities between configure scripts > is very inconvenient. I do agree, and I half-expected someone to point out the inconsistency. OTOH, this isn't a "configure script", it's the Setup.lhs program. You already have to know that it isn't an autoconf configure script, because it has different syntax. One reason I suggested that relative paths would be better is because it allows you to build a binary/library that is prefix-independent, at least on Windows where a binary knows the absolute $bindir and hence can derive $prefix because it knows $bindir relative to $prefix. There's no reason we can't have both, since relative/absolute paths are syntactically distinct. We just say you can give a relative path, and it is assumed to be relative to $prefix. Isaac, shall I knock up a patch? Cheers, Simon

2 1