- Libraries - Haskell.org

RE: another wash compile problem
by Simon Marlow 10 Aug '05

10 Aug '05

On 10 August 2005 09:04, Frederik Eaton wrote: >>> Compiling a module which imports the 14 smaller modules takes much >>> less time than compiling the monolithic module - it's almost 5 times >>> faster (see below). >> >> We've found the cause of this, and committed a fix. The full >> HTMLMonda98.hs now compiles in 27 seconds for me, without >> optimisation (with 6.4 it is several minutes). Let's hope the same >> issue is affecting other compilations too :-) > > Wow! That's great. > > Have you looked at how compile time varies with module size in the > fixed version? Not in any detail; but I expect it to be worse than linear but not as bad as quadratic. There are several reasons it might be worse than linear: GC is linear in the residency, for one. And the compiler makes a lot of use of FiniteMap, which will add a (log n) factor to a traversal of the tree. We try to fix instances of quadratic behaviour as we find them. Cheers, Simon

1 0

RE: another wash compile problem
by Simon Marlow 10 Aug '05

10 Aug '05

On 06 July 2005 23:38, Frederik Eaton wrote: >> When I compile WASH it takes forever for ghc to finish compiling >> HTMLMonad98, and on my powerpc system with 151MB free RAM, gcc >> crashes during the compile. Should some of the larger WASH modules >> maybe be split into multiple files? I get the sense that ghc >> compilation time/memory is super-linear in the size of the module >> being compiled. > > I've now verified my earlier "ghc compilation complexity is > super-linear" belief. I split up HTMLMonad98 into 10 files, and then > split the 6th and largest, which consists of the following lines - > > instance AdmitChildHEAD HTML > instance AdmitChildFRAMESET HTML > ...(2692 lines) > instance AdmitChildLABEL ACRONYM > instance AdmitChildBUTTON ACRONYM > > into 14 files: > > $ wc -l HTMLMonad98No6.hs > 2696 HTMLMonad98No6.hs > $ wc -l HTMLMonad98No6?.hs > 200 HTMLMonad98No6a.hs > 204 HTMLMonad98No6b.hs > 204 HTMLMonad98No6c.hs > ... > 2748 total > > Compiling a module which imports the 14 smaller modules takes much > less time than compiling the monolithic module - it's almost 5 times > faster (see below). We've found the cause of this, and committed a fix. The full HTMLMonda98.hs now compiles in 27 seconds for me, without optimisation (with 6.4 it is several minutes). Let's hope the same issue is affecting other compilations too :-) Cheers, Simon

2 1

Extending the dependency syntax
by Brian Smith 10 Aug '05

10 Aug '05

On 8/9/05, Simon Marlow <simonmar(a)microsoft.com> wrote: > ------------------ > build-depends: > ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) > | hugs? [hugs], > debug? > (HUnit-1.0, [debug]) > [release], > gnome? ( libglade >= 2, > gtksourceview >= 0.6, > gconf >= 2, [gnome] ), > mozilla? ( mozilla >= 1.4, [mozilla] ), > doc? ( haddock >= 0.6 ) > > [gnome] > extra-libraries: gnome > extra-ghc-options: -DENABLE_GNOME > ... > > [debug] > extra-ghc-options: -O0 -DDEBUG > > [release] > extra-ghc-options: -O2 As described, it seems like I would need to pass "--enable-hugs" or "--enable-ghc" explicitly, or else this program's build would fail. But, I don't think that is your intention; instead you are intending for Cabal to automatically enable these flags as appropriate, right? Perhaps there would be other automatic flags too, like "windows" and "unix"? I think there is too much overloading being used: 1. "ghc" refers to both a flag and a package 2. "debug," "release," and "gnome," are flags and configurations. If I pass "--enable-gnome" as a flag then the "gnome" configuration gets used, if I pass "--enable-release" then the "release" configuration gets used, etc.. But, if I pass "--enable-ghc64" as a flag then the ghc64 configuration will not be used, right? Will the "--enable-ghc64" just be silently ignored? It would be nice to be able to catch typos like "--enable-debgu" What is the difference between the following?: ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) | hugs? [hugs], vs. ( ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old] | hugs, [hugs] | ) vs. ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old] | hugs, [hugs] My understanding is that the first two are equivalent but the third one additional means "fail if building with anything other than GHC >= 5.04 or hugs," assuming that Cabal automatically enables the flags "ghc" or "hugs" appropriately. Am I correct? "a, b" means "require a and b." But, "a, [b]" means "require dependency a and use configuration b" or, equivalently "if dependency a is satisfied, then automatically use configuration b." I think it would be better to avoid overloading the comma operator with these two meanings. Can configurations contain their own "Build-depends", like this?: [windows] Build-depends: Win32 extra-libraries: gnome It seems counterintuitive to allow "extra-libraries" but to disallow "build-depends." Yet, we may want to prohibit the build-depends flag from including other configurations syntax, to prevent cycles and non-intuitive behavior: Build-depends: windows? [windows] [windows] Build-Depends: Win32, wxHaskell [gui] | extra-libraries: gdi32 [gui] Build-Depends: wxHaskell, (Win32 [windows] | ) I think it would be clearer to specify the configuration seperately, like this: configuration: ghc >= 6.4 ? [ghc64] | ghc >= 5.04 ? [ghc-old]) | hugs? [hugs], debug enabled ? [debug] | [release], gnome enabled ? [gnome], mozilla enabled ? [mozilla], doc enabled ? [doc] -- the "global" Build-depends applies to everything Build-depends: base >= 1.0, haskell98 >= 1.0 [gnome] Build-Depends: libglade >= 2, gtksourceview >= 0.6, gconf >= 2, extra-libraries: gnome extra-ghc-options: -DENABLE_GNOME [debug] Build-depends: HUnit-1.0 extra-ghc-options: -O0 -DDEBUG [release] extra-ghc-options: -O2 [mozilla] Build-depends: mozilla >= 1.4 [doc] Build-depends: haddock >= 0.6 Notice here that the "Build-Depends:" syntax would remain the same as in Cabal 1.0, as [stanza] and the "?" operator would only be allowed in "configuration:". Also notice that this requires that dependencies be allowed as the left operand of the "?" operator, but they would no longer be allowed as the right operand; only stanza names are allowed. Finally, are you intending to allow multiple executable stanzas inside configurations?: Build-depends: wxHaskell, [gui] | [gui] Executable: admin-gui Main-Is: AdminGui.hs Executable: another-gui Main-Is: AnotherGui.hs Regards, Brian

1 0

RE: Extending the dependency syntax
by Simon Marlow 09 Aug '05

09 Aug '05

I'd like to wrap up this little design issue which stalled. I'll propose something a little simpler, which should hopefully be closer to what people are used to from other package systems, and which should translate more easily. To recap, the goal is to support: - build options that result in an executable or library being built in a different way, or with a particular feature. The build option may imply additional dependencies. - a choice between dependencies, where the particular choice made dictates certain configuration changes in the build info. - a choice between configurations, for example building a library with or without debugging support. An example first showing all the main ideas: ------------------ build-depends: ghc? (ghc >= 6.4, [ghc64] | ghc >= 5.04, [ghc-old]) | hugs? [hugs], debug? (HUnit-1.0, [debug]) [release], gnome? ( libglade >= 2, gtksourceview >= 0.6, gconf >= 2, [gnome] ), mozilla? ( mozilla >= 1.4, [mozilla] ), doc? ( haddock >= 0.6 ) [gnome] extra-libraries: gnome extra-ghc-options: -DENABLE_GNOME ... [debug] extra-ghc-options: -O0 -DDEBUG [release] extra-ghc-options: -O2 ------------------- The semantics of build-depends: - ',' means "and", just as it does now - '|' means "or" - parentheses group dependencies - [name] means "read the optional stanza [name]" - 'flag? dep' means "if the --enable-flag option is given, then dep, otherwise nothing". - 'flag? dep1 dep2' menas "if the --enable-flag option is given then dep1, else dep2". Additionally, we could have an --auto-enable flag, which has the effect of replacing every 'flag? dep' by 'dep|', and 'flag? dep1 dep2' by 'dep1|dep2'. This has the effect of turning on every option for which the dependencies are satisfied, which is normally what you want when building a package for local use. I'm not attached to any particular syntax, so feel free to suggest alternatives. This is somewhat simpler than before, and I think supports everything we need. Comments? I'm happy to implement if there's a concensus. Cheers, Simon

2 1

RE: Foreign language parsers
by Simon Marlow 09 Aug '05

09 Aug '05

On 05 August 2005 12:37, Dominic Fox wrote: > I'm trying to collate a list of currently extant Haskell projects that > include parsers for foreign languages such as PHP and C. > > To this end, I've created a wiki page at > http://haskell.org/hawiki/ParsingForeignLanguagesInHaskell. > > Has anyone looked at positioning foreign language parsing modules in > the standard libs, or specifying a common framework or contract for > such modules? This is what the Language.* branch of the hierarchy was intended for, incedentally. Language.Haskell is currently populated, but I'm not aware of any other libraries under Language. Cheers, Simon

3 4

RE: package description fields
by Simon Marlow 09 Aug '05

09 Aug '05

On 08 August 2005 16:57, Jonathan Cast wrote: >> On 04 August 2005 08:24, Krasimir Angelov wrote: >> >>> I would like to propose more Unix like directory layout for Windows >>> i.e.: >>> >>> executables: %ProgramFiles%\<pkg-name>\bin >>> libraries: %ProgramFiles%\<pkg-name>\lib >>> data files: %ProgramFiles%\<pkg-name>\data >>> >>> The advantage is that if you have a lot of libraries and data files >>> then it is easier to find out the executable files. Happy, Alex and >>> Haddock already use layout like in the Ross's proposal but I found >>> it very inconvenient because usually I am building them together >>> with ghc. The problem is that when I do make install in the fptools >>> directory then ghc is installed in ${prefix}/bin but the above tools >>> are installed in ${prefix}. Usually then ${prefix} directory isn't >>> included in the PATH and this makes them inaccessible. I can move >>> them to ${prefix}/bin but then I have to move all other template >>> files too. It would be nice if we had more consistent directory >>> layout. >> >> Your complaint here seems to be that the default layout of Happy, >> Alex and Haddock is different from that of GHC on Windows, which >> makes it harder to combine all these tools in a single package. >> >> As far as Cabal is concerned, I think the defaults are fine. >> However, I do think we should allow the default >> $bindir/$libdir/$datadir to be overridden (I know this has been >> discussed before). On Windows, an application often finds its files >> relative to the location of the executable, so the values of >> $bindir/$libdir/$datadir need to be passed to the program at build >> time, perhaps via CPP symbols. So, I suggest we have: >> >> --bindir=<dir> >> --libdir=<dir> >> --datadir=<dir> >> >> where the values of these would be directories relative to $prefix >> (or absolute? relative to $prefix seems nicer). > > Relative paths here break compatibility with Autoconf; when I > install a program, I don't care where its configure script comes > from, I just want to be able to invoke it the way I > expect to. Gratuitous incompatibilities between configure scripts > is very inconvenient. I do agree, and I half-expected someone to point out the inconsistency. OTOH, this isn't a "configure script", it's the Setup.lhs program. You already have to know that it isn't an autoconf configure script, because it has different syntax. One reason I suggested that relative paths would be better is because it allows you to build a binary/library that is prefix-independent, at least on Windows where a binary knows the absolute $bindir and hence can derive $prefix because it knows $bindir relative to $prefix. There's no reason we can't have both, since relative/absolute paths are syntactically distinct. We just say you can give a relative path, and it is assumed to be relative to $prefix. Isaac, shall I knock up a patch? Cheers, Simon

2 1

Getting Cabal 1.1.1 working on Windows 98
by Toby Allsopp 09 Aug '05

09 Aug '05

The version of Cabal that comes with GHC (6.4 and 6.4.1) doesn't run on Windows 98 because of a missing SHGetFolderPathA function, so I thought I'd try to compile version 1.1.1 myself. I see that some work has been done in CVS for this problem, but I thought I'd post the changes I had to make to get 1.1.1 to work incase this is helpful to other people or to getting it fixed propoerly. I'm using MinGW 4.1, MSYS 1.0.10 and MSYS Developer Tool Kit 1.0.1. I've attached a patch against version 1.1.1. First, I tried compiling setup without any changes: $ make HCBASE=/c/ghc/ghc-6.4.1/bin/ setup ... Compiling Main ( Setup.lhs, dist/tmp/Main.o ) Linking ... dist/tmp/Distribution/Simple/Configure.o(.text+0xe41):fake: undefined reference to `SHGetFolderPath@20' make: *** [setup] Error 1 Ok, this seems to be because we need to pick either the A or W version. I'm not sure how to get this to happen automatically, but I guess we want the A version: $ diff ../cabal-1.1.1.orig/Distribution/Simple/Configure.hs Distribution/Simple /Configure.hs 256c256 < foreign import stdcall unsafe "SHGetFolderPath" --- > foreign import stdcall unsafe "SHGetFolderPathA" Now, I try again: $ make HCBASE=/c/ghc/ghc-6.4.1/bin/ setup ... Skipping Main ( Setup.lhs, dist/tmp/Main.o ) Linking ... Yay, it compiled! Now I try to run it: $ make HCBASE=/c/ghc/ghc-6.4.1/bin/ config "The SETUP.EXE file is linked to missing export SHELL32.DLL:SHGetFolderPathA" (The message is GUI dialog box.) Ah, this is because SHGetFolderPathA is not in SHELL32.DLL on Windows 98, but in SHFOLDER.DLL. It is also in SHFOLDER.DLL on later versions, so this change should be safe: $ diff ../cabal-1.1.1.orig/GNUmakefile GNUmakefile 4c4 < GHCFLAGS= --make -Wall -fno-warn-unused-matches -cpp --- > GHCFLAGS= --make -Wall -fno-warn-unused-matches -cpp -lshfolder Try again: $ make HCBASE=/c/ghc/ghc-6.4.1/bin/ config ... Skipping Main ( Setup.lhs, dist/tmp/Main.o ) Linking ... ./setup configure --ghc --prefix=/usr/local Configuring Cabal-1.1.1... configure: Using install prefix: C:/MSYS/1.0/local configure: Using compiler: c:\GHC\GHC-6.4.1\BIN\ghc.exe configure: Compiler flavor: GHC configure: Compiler version: 6.4.1 configure: Using package tool: c:\GHC\GHC-6.4.1\BIN\ghc-pkg.exe configure: No haddock found configure: No happy found configure: No alex found configure: Using hsc2hs: c:\GHC\GHC-6.4.1\BIN\hsc2hs.exe configure: No c2hs found configure: No cpphs found configure: No greencard found configure: Dependency base-any: using base-1.0 configure: Dependency util-any: using util-1.0 Yay! Now to install it: $ make HCBASE=/c/ghc/ghc-6.4.1/bin/ install ... Installing: C:/MSYS/1.0/local\Haskell\Cabal-1.1.1\ghc-6.4.1 & C:/MSYS/1.0/local\Cabal-1.1.1\bin Cabal-1.1.1... Registering Cabal-1.1.1... Ok, it all seems fine. Now to try to use it: $ cd tests/HSQL $ runghc Setup.lhs configure ghc.exe: unable to load package `Cabal-1.1.1' <interactive>: C:/MSYS/1.0/local\Haskell\Cabal-1.1.1\ghc-6.4.1/HSCabal-1.1.1.o: unknown symbol `_SHGetFolderPathA' Bummer. We need to link with SHFOLDER.DLL. So, we change Cabal's cabal file (similarly to the comment that was in CVS recently): $ cd ../.. $ diff -u ../cabal-1.1.1.orig/Cabal.cabal Cabal.cabal --- ../cabal-1.1.1.orig/Cabal.cabal Sat Aug 6 11:50:00 2005 +++ Cabal.cabal Sat Aug 6 12:17:46 2005 @@ -48,6 +48,7 @@ Distribution.Compat.Exception, Distribution.Compat.RawSystem Extensions: CPP +Extra-Libraries: shfolder executable: cabal-setup main-is: DefaultSetup.lhs Then, install again and try the test again: $ make HCBASE=/c/ghc/ghc-6.4.1/bin/ install ... Registering Cabal-1.1.1... $ cd tests/HSQL/ $ runghc Setup.lhs configure -=-= Cabal executing: ./configure =-=- Bad command or file name Configure Succeeded. Triple yay! Using this version I have successfully installed the real HSQL. How, it seems wrong to have to specify the A or W version explicitly because there is a #define for it that depends on whther UNICODE is defined. I don't know why that's not being picked up (it's in shldobj.h and I see that that's been specified in CVS, but adding it makes no difference for me.) The patch follows:

4 6

RE: datadir (was: RE: new package description fields, sdist, andpreprocessors)
by Simon Marlow 09 Aug '05

09 Aug '05

On 09 August 2005 09:56, Bernard Pope wrote: > On Tue, 2005-08-09 at 09:36 +0100, Simon Marlow wrote: > >> The problem, of course, is how to get the value of $datadir to the >> program if it is configurable. I suggested a CPP symbol, but that is >> far from ideal (we don't do it this way in fptools projects any more >> for various reasons). Another option is to have the Setup script >> generate a file containing the declarations: >> >> -- paths.h >> prefix="/usr/local" >> bindir="bin" -- or absolute >> libdir="lib/foo-1.0/ghc-6.4" >> datadir="share/foo-1.0" >> >> which you can then #include into Haskell source. Another >> alternative, if you want to avoid CPP, is to have Setup generate a >> complete Haskell module with these declarations, but then you have >> to somehow choose an appropriate module name that doesn't clash. >> >> paths.h gets my vote. Anyone have any better ideas? > > What if, say, you have some C code in your package that also wants to > know the values of these paths? There's no single solution that works for all languages. The good news is that prefix/bindir/libdir/datadir would be available in the LocalBuildInfo, so you would be able to implement any scheme you like using the postConf hook. And maybe we could refer to these values in the .cabal file, so you could write extra-cc-options: -Dprefix=$prefix (probably some quoting required somewhere, but you get the idea). Cheers, Simon

1 0

datadir (was: RE: new package description fields, sdist, and preprocessors)
by Simon Marlow 09 Aug '05

09 Aug '05

On 09 August 2005 02:59, Isaac Jones wrote: > Ross Paterson <ross(a)soi.city.ac.uk> writes: >> On Wed, Aug 03, 2005 at 07:27:01PM -0500, Brian Smith wrote: >>> On 8/3/05, Ross Paterson <ross(a)soi.city.ac.uk> wrote: >>>> 3) New field data-files, a list of files to be copied to a place >>>> where an executable can find them (e.g. template-hsc.h for >>>> hsc2hs): Hugs: the directory containing the Main module >>>> GHC/Windows: the directory containing the executable >>>> GHC/Unix: /usr/local/share/<exename> >>>> plus a new function in System.Directory to return the name of >>>> this directory. That would address Dimitry's requirements in >>> >>> How about allowing directories too, which would be copied >>> recursively? >> >> No objection to that. > > This sounds good, except that I'm not sure that System.Directory is > the right place for this. It could also go in Distribution > somewhere, but either way may be fine. > > Also, if we do this, we should probably specify the manner in which > such a directory should be layed out so: > > 1) it doesn't get too cluttered > 2) different packages don't stomp on each-other's files and > 3) different versions of different packages can use the same > filenames. > > One stab at it (where dataFileDir :: FilePath) would be that your > datafiles should be in: > > System.Directory.dataFileDir > `joinFilePath` (packageName ++ "-" packageVersion) > > and this is where cabal will put it. This overlaps with my --datadir proposal. I think that datadir should be configurable at Setup-time, with sensible defaults as per Ross's mail above. The default should at least be specified relative to $prefix (eg. if $prefix is /usr, I don't want $datadir to be /usr/local/share/<pkg>). Also, if the data files are machine-dependent then they should really go in $libdir rather than $datadir. The problem, of course, is how to get the value of $datadir to the program if it is configurable. I suggested a CPP symbol, but that is far from ideal (we don't do it this way in fptools projects any more for various reasons). Another option is to have the Setup script generate a file containing the declarations: -- paths.h prefix="/usr/local" bindir="bin" -- or absolute libdir="lib/foo-1.0/ghc-6.4" datadir="share/foo-1.0" which you can then #include into Haskell source. Another alternative, if you want to avoid CPP, is to have Setup generate a complete Haskell module with these declarations, but then you have to somehow choose an appropriate module name that doesn't clash. paths.h gets my vote. Anyone have any better ideas? Cheers, Simon

3 3

package description fields
by Ross Paterson 09 Aug '05

09 Aug '05

Proposals: 1) New field extra-tmp-files, a list of extra files to be removed by setup clean, beyond those that can be deduced. 2) Rename other-files as extra-source-files for consistency and clarity. 3) New field data-files, a list of files to be copied to a place where an executable can find them (e.g. template-hsc.h for hsc2hs): Hugs: the directory containing the Main module GHC/Windows: the directory containing the executable GHC/Unix: /usr/local/share/<exename> plus a new function in System.Directory to return the name of this directory. That would address Dimitry's requirements in http://www.haskell.org/pipermail/haskell-cafe/2005-July/010886.html Dimitry has a more elaborate proposal: http://www.haskell.org/pipermail/libraries/2005-July/004146.html

4 5