July 2005 - Libraries

external core
by John Meacham 22 Mar '06

22 Mar '06

Now, the external core feature of ghc is great, but it is not utilized very much. The main reason I think is that there is a large barrier to entry in writing something that uses it since you need to parse/generate it. It would be nice if there were a standard library that came with ghc (or cabalized and portable even better) that could parse and pretty print external core. ideally, one could write a simple pass-through core processor that interacts with ghc in a single line and someone working on a new optimization need only worry about that. Also, it would make it very easy for jhc to generate and parse ghc core files (always an ulterior motive). I wanted to use jhcs front end to generate ghc core so I could get a true measure of the difference in just their back-end performance without taking into account ghc's superior high-level optimizations. In any case. I think all the code is out there in ghc and it just needs to be separated and cabalized. I know there was talk of creating a ghc internals library, but it would be nice if this code were portable and independent. John -- John Meacham - ⑆repetae.net⑆john⑈

3 4

Trie implementation
by Keith Wansbrough 10 Mar '06

10 Mar '06

Hi... I was writing some code yesterday to walk over a directory tree, and needed a Trie. Not seeing one around, I wrote a basic implementation of the functions I needed (attached below). Has anyone else done this? Should I polish it up and offer it for inclusion in Data? --KW 8-)

7 8

Cabal on OS X; ghc segfault?
by Gregory Wright 10 Aug '05

10 Aug '05

Hi, I'm trying to get caballized package deployment working on Mac OS X. However, trying to build a package using runhaskell results in: crossroads-able> runhaskell Setup.hs configure Warning: No license-file field. Configuring harp-0.2... configure: searching for ghc in path. configure: found ghc at /opt/local/bin/ghc runhaskell: waitForProcess: interrupted (Interrupted system call) This was trying to build the harp module in the haskell-src-exts package (a prerequisite for hs-plugins). I ran a ktrace on this and it appears as if the underlying GHC is segfaulting: crossroads-able> cat k.txt 414 ghc-6.4 RET write 1 414 ghc-6.4 CALL getpid 414 ghc-6.4 RET getpid 414/0x19e 414 ghc-6.4 CALL stat(0x1b94528,0x1b944c0) 414 ghc-6.4 NAMI "tmp414" 414 ghc-6.4 RET stat -1 errno 2 No such file or directory 414 ghc-6.4 CALL sigprocmask(0x1,0xbfffd4d0,0xbfffd4d4) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL sigaction(0x2,0xbfffd458,0) 414 ghc-6.4 RET sigaction 0 414 ghc-6.4 CALL sigprocmask(0x3,0xbfffd4d4,0) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL sigprocmask(0x1,0xbfffd4d0,0xbfffd4d4) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL sigaction(0x3,0xbfffd458,0) 414 ghc-6.4 RET sigaction 0 414 ghc-6.4 CALL sigprocmask(0x3,0xbfffd4d4,0) 414 ghc-6.4 RET sigprocmask 0 414 ghc-6.4 CALL fcntl(0,0x3,0) 414 ghc-6.4 RET fcntl 6 414 ghc-6.4 CALL fcntl(0,0x4,0x6) 414 ghc-6.4 RET fcntl 0 414 ghc-6.4 CALL ioctl(0,0x4004667a ,0xbfffd4b8) 414 ghc-6.4 RET ioctl 0 414 ghc-6.4 PSIG SIGSEGV SIG_DFL 413 runhaskell RET wait4 414/0x19e 413 runhaskell CALL write(0x2,0xbfffcf40,0xc) 413 runhaskell GIO fd 2 wrote 12 bytes "runhaskell: " 413 runhaskell RET write 12/0xc 413 runhaskell CALL write(0x2,0xbfffcfa0,0x36) 413 runhaskell GIO fd 2 wrote 54 bytes "waitForProcess: interrupted (Interrupted system call) " This is a bit surprising. Any suggestions on how to track down the problem? Best Wishes, Greg

5 5

on when to build ghci libs in cabal
by Isaac Jones 08 Aug '05

08 Aug '05

Henning Günther and Duncan Coutts / the Gentoo team have sent me a patch to move creation of ghci libs to the build phase rather than the register phase. It makes more sense there and should be somewhat better for debian and gentoo. Though Duncan points out that maybe some ppl want to build the ghci libs during register because it could save room in a binary distribution (like debian and rpm). Any opinions here? Also, I noticed the "FIXME" that mentions the -x flag is only supported on some platforms; what effect will this have on platforms that don't support it? Will they be broken during build? What about platforms where we know the ghci libs don't work, like MacOS; will this patch cause building to break? The patch adds a couple extra configure options, --enable-library-for-ghci / --disable-library-for-ghci. I'm sorta thinking of just having one option "--disable-interpreter-libs" or something to make this seem more generic, though at the moment, it still only makes sense for a combo compiler / interpreter build (that is, --ghc). I'll be happy to add this to the manual once we settle on a flag. Do we ever really need to disable it anyway? peace, isaac

4 4

cabal release?
by Isaac Jones 08 Aug '05

08 Aug '05

I would like to make a Cabal release Real Soon Now, version 1.0.2. There are always more changes that would be good to get in, but we have to release sometime. This is just a minor release, and it's not being released with a compiler like the 1.0 release, so it doesn't need to be "perfect". We haven't exactly been releasing early and often, but in that spirit, let's get the many changes since 1.0 out into the wild. Here's a summary from the changelog: * Handles recursive modules for GHC 6.2 and GHC 6.4. * Added "setup test" command (Used with UserHook) * implemented handling of _stub.{c,h,o} files * Added support for profiling * Changed install prefix of libraries (pref/pkgname-version to prefix/pkgname-version/compname-version) * Added pattern guards as a language extension * Moved some functionality to Language.Haskell.Extension * Register / unregister .bat files for windows * Exposed more of the API * Added support for the hide-all-packages flag in GHC > 6.4 * Several bug fixes Oh, and though it's not a big change from Cabal's perspective, there are some changes that are necessary to get Hackage out the door. Does anyone know of any: - critical bugs that we shouldn't release without fixing - interface changes that are so horrible that we shouldn't release without fixing them I'm a bit concerned about the different release cycles that Cabal and the compilers will be following, but I think that if we are careful with version numbers we'll be OK :) peace, isaac

4 10

MySQL / HSQL Won't Build
by Dominic Steinitz 01 Aug '05

01 Aug '05

I've just downloaded and tried to build HSQL for MySQL but I'm getting the errors below. mysql.h does exist. Any help would be appreciated. Thanks, Dominic. [dom@tility MySQL]$ ls /usr/include/mysql/mysql.h /usr/include/mysql/mysql.h [dom@tility MySQL]$ ghc Setup.lhs -o Setup -package Cabal-1.0.1 [dom@tility MySQL]$ ./Setup configure Warning: No license-file field. Configuring hsql-mysql-1.6... configure: Using install prefix: /usr/local configure: Using compiler: /usr//bin/ghc configure: Compiler flavor: GHC configure: Compiler version: 6.4 configure: Using package tool: /usr//bin/ghc-pkg configure: Using haddock: /usr//bin/haddock configure: No happy found configure: No alex found configure: Using hsc2hs: /usr//bin/hsc2hs configure: Using c2hs: /usr//bin/c2hs configure: No cpphs found configure: Dependency base-any: using base-1.0 configure: Dependency hsql-any: using hsql-1.6 configure: Using mysql_config: /usr//bin/mysql_config configure: configuring mysqlclient library [dom@tility MySQL]$ more MySQL.buildinfo buildable: True cc-options: ld-options: frameworks: c-sources: extensions: extra-libraries: mysqlclient z crypt nsl m extra-lib-dirs: '/usr//lib/mysql' includes: include-dirs: '/usr//include/mysql' hs-source-dirs: . other-modules: ghc-options: hugs-options: nhc-options: [dom@tility MySQL]$ ./Setup build Preprocessing library hsql-mysql-1.6... In file included from MySQL.hsc:14: /usr/lib/ghc-6.4/include/config.h:4:2: warning: #warning config.h is deprecated; please use ghcconfig.h instead In file included from MySQL.hsc:33: Database/HSQL/HsMySQL.h:8:19: mysql.h: No such file or directory MySQL.hsc: In function `main': MySQL.hsc:90: error: `CLIENT_MULTI_STATEMENTS' undeclared (first use in this function) MySQL.hsc:90: error: (Each undeclared identifier is reported only once MySQL.hsc:90: error: for each function it appears in.) MySQL.hsc:147: error: `MYSQL_FIELD' undeclared (first use in this function) MySQL.hsc:147: error: syntax error before ')' token MySQL.hsc:148: error: syntax error before ')' token MySQL.hsc:149: error: syntax error before ')' token MySQL.hsc:150: error: syntax error before ')' token MySQL.hsc:151: error: syntax error before ')' token MySQL.hsc:154: error: `NOT_NULL_FLAG' undeclared (first use in this function) MySQL.hsc:157: error: `FIELD_TYPE_STRING' undeclared (first use in this function) MySQL.hsc:158: error: `FIELD_TYPE_VAR_STRING' undeclared (first use in this function) MySQL.hsc:159: error: `FIELD_TYPE_DECIMAL' undeclared (first use in this function) MySQL.hsc:160: error: `FIELD_TYPE_SHORT' undeclared (first use in this function) MySQL.hsc:161: error: `FIELD_TYPE_INT24' undeclared (first use in this function) MySQL.hsc:162: error: `FIELD_TYPE_LONG' undeclared (first use in this function) MySQL.hsc:163: error: `FIELD_TYPE_FLOAT' undeclared (first use in this function) MySQL.hsc:164: error: `FIELD_TYPE_DOUBLE' undeclared (first use in this function) MySQL.hsc:165: error: `FIELD_TYPE_TINY' undeclared (first use in this function) MySQL.hsc:166: error: `FIELD_TYPE_LONGLONG' undeclared (first use in this function) MySQL.hsc:167: error: `FIELD_TYPE_DATE' undeclared (first use in this function) MySQL.hsc:168: error: `FIELD_TYPE_TIME' undeclared (first use in this function) MySQL.hsc:169: error: `FIELD_TYPE_TIMESTAMP' undeclared (first use in this function) MySQL.hsc:170: error: `FIELD_TYPE_DATETIME' undeclared (first use in this function) MySQL.hsc:171: error: `FIELD_TYPE_YEAR' undeclared (first use in this function) MySQL.hsc:172: error: `FIELD_TYPE_BLOB' undeclared (first use in this function) MySQL.hsc:173: error: `FIELD_TYPE_SET' undeclared (first use in this function) MySQL.hsc:174: error: `FIELD_TYPE_ENUM' undeclared (first use in this function) Setup: got error code while preprocessing: Database.HSQL.MySQL [dom@tility MySQL]$

3 2

fptools/libraries package versioning
by Brian Smith 31 Jul '05

31 Jul '05

When we send it patches to fptools/libraries that change the API, should we include a patch to increment the version number too? For example, I have a patch that adds bindings for WaitForMultipleObjects and WaitForSingleObject to the Win32 package. Should I increase the Win32 package's version number in the patch I send? Also, now Hugs is building again on Windows (at least the command-line part) and perhaps there will be a release of (Win)Hugs soon. It would be convenient to be able to say that the next release of Hugs comes with, e.g. Win32-1.1 and HaXml-1.13.1. Thanks, Brian

2 3

Raw filenames vs locales
by Ian Lynagh 30 Jul '05

30 Jul '05

Hi all, I'd like to propose some changes to the IO library to fix some problems (IMO) with hugs' recent closer adherence to the Haskell 98 report. I believe this is orthogonal to the proposed new IO library stuff that's been discussed before. The rest of this mail is in 3 parts. First I describe the problem, then a proposed solution, and finally some comments on backwards compatibility. =========== The problem =========== With it's closer adherence to the Haskell 98 report, it is no longer possible with hugs to manipulate files using the standard IO functions if the filenames are not representable in your locale. To demonstrate the problem consider this: -------------------------------------------------- touch `printf "1\xA4"` echo ' import System.Directory (getDirectoryContents) import Data.Char (ord) main = do xs <- getDirectoryContents "." print (map (map ord) xs) ' > foo.hs for locale in en_GB en_GB.ISO-8859-15 en_GB.UTF-8 do echo "===============================" echo "Doing $locale" LC_ALL=$locale ../runhugs foo.hs done -------------------------------------------------- Here we create a file whose filename if 1\xA4. \xA4 is a "currency sign" in ISO-8859-1, "euro sign" in ISO-8859-15 and not a valid character in UTF-8. We then print the results of getDirectoryContents, converting the Chars to Ints so we can see what's going on. The result is this: =============================== Doing en_GB [[46],[46,46],[49,164],[102,111,111,46,104,115]] =============================== Doing en_GB.ISO-8859-15 [[46],[46,46],[49,8364],[102,111,111,46,104,115]] =============================== Doing en_GB.UTF-8 [[46],[46,46],[49,65533],[102,111,111,46,104,115]] The third file is the interesting one. We have: ISO-8859-1: 164 = U+A4 = "currency sign" ISO-8859-15: 8364 = U+20AC = "euro sign" UTF-8: 65533 = U+FFFD = "replacement character" "replacement character" is "used to replace an incoming character whose value is unknown or unrepresentable in Unicode". ================= Proposed solution ================= My suggestion is essentially that we change all functions using the FilePath type to instead use FilePath a => a. [ By jumping through hoops I think this could be done H98-compatibly, but for simplicity I'll ignore that for now. I'm not sure if it's a problem for any impl anyway? ] I imagine the class would look something like class FilePath a where to_filename :: a -> IO FileName from_filename :: FileName -> IO a from_free_filename :: FileName -> IO a from_free_filename f = do x <- from_filename f free f return x with_filename :: FilePath a => a -> (FileName -> IO b) -> IO b with_filename x f = do x' <- to_filename x res <- f x' free_filename x' return res We would then have System.IO.Impl.getDirContents :: FileName -> [FileName] System.IO.getDirContents :: FilePath a => a -> [a] -- Could be more general System.IO.getDirContents x = do ys <- with_filename x Impl.getDirContents mapM from_free_filename ys On Unix systems FileName would be a Ptr Word8. My knowledge of Windows isn't great, but I think there it would be an array of 16-bit values? We would have instances of FilePath for String and [Word8] to solve the immediate problem. String would be the current behaviour, but [Word8] would be converted to a FileName unchanged. On Windows it would probably be necessary to throw an exception if a [Word8] is passed which is not valid utf8. It would also be nice to have a FileName instance, to avoid unnecessary conversions. A Ptr Word8 instance would also be handy for things like darcs' FastPackedString module to be able to use efficiently (without taking a round trip via a lazy list). ======================= Backwards compatibility ======================= I haven't done any research into it, but I hope that a lot of the time this will not be an issue as the impl will be able to infer the type String is being used, either by a string literal, the fact it is putStrLn'd, there is a type signature saying it is a String, etc. The Haskell 98 modules like IO could re-export the functions with their types restricted to what they are now. This would give us complete backwards compatibility to Haskell 98. It is certainly possible for there to be ambiguities in programs that use the hierarchial libraries, however. Possible solutions are: * Tell people to add type sigs to fix it. * Define the new stuff in System.IO.Impl in a package iobase. The oldio package would then contain System.IO which re-exports all the functions with the old types, and the io package would do the same with the new types. Unfortunately i don't think this would work if you have some libraries compiled against the io package you don't want to use. I think this might be an argument that the package system is not being flexible enough. That's all I've got. Comments welcomed! Thanks Ian

4 4

finding the dependecies of cabal packages
by Duncan Coutts 29 Jul '05

29 Jul '05

Hi all, Over in Gentoo packaging land we're trying to build tools to automate the process of packaging cabal packages. ("package" is far to much of an overlaoded word!) So we have a tool "cabal2ebuild" which will produce an ebuild from a cabal file. An ebuild is a text file that contains all the information that the Gentoo packaging system needs to be able to install the program/library in question. In particular the ebuild must list all the dependencies of the package. Required Haskell libraries are listed in the "build-depends:" field in the cabal file, so that one is easy. However it is not clear how to find out the build tools that the cabal package requires. For example packages may require alex or happy or c2hs or other build time tools. Is there any better way to find if these tools are needed than to look through the source tree for any files with the appropriate extensions ".y", ".x", ".chs" etc? Duncan

5 15

RE: Extending the dependency syntax
by Simon Marlow 29 Jul '05

29 Jul '05

On 28 July 2005 22:24, Isaac Jones wrote: > "Simon Marlow" <simonmar(a)microsoft.com> writes: > >> So here's a strawman proposal to address some of the use cases raised >> recently[1] that Cabal doesn't currently support well. This is just >> a first attempt; I expect we'll need to make changes or start over >> if we discover cases that it doesn't handle. > > (snip proposal) > > Lots of this is stuff I've been wanting to ad at some point. I think > that rather than overloading the build-depends field, we should add a > new field for tool dependencies; otherwise, we could get name > conflicts between the two and such. ok - I put them in the build-depends field because I wanted to do the cool stuff with optional stanzas and use it with compiler and tool versions too. There's no reason you couldn't do this in a separate field. > I'm not positive that the simple build infrastructure would be able to > take all of the information in the tool-dependendencies into account; > for instance, discovering the version of happy and ghc might be a bit > tricky and error prone. We already discover the version of GHC. And it can't be any more tricky than using autoconf :-) > (snip) >> ---------------- >> name: mylib >> version: 1.0 >> exposed-modules: My.Lib >> build-depends: >> base>=1.0 >> (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs]) > > Debian has something like this, where you can specify optional > dependencies. I've always planned to implement something like this > for Cabal. > > At the same time, I think we need to be careful not to re-invent make > and re-invent dpkg / rpm via feature-creep. I understand Duncan's > desire to have absolutely everything automated for gentoo without the > gentoo packager having to intervene, but that's just what the OS > packagers have been dealing with for every language. I want Haskell > to be easier than other languages (historically, it's been harder) but > if we make the package syntax more and more complex, I think we run > the risk of expending a lot of energy for a small amount of benefit by > creating a whole new language for packages. I don't think it's a question of re-invention - we really do need a more expressive language for dependencies in Cabal. Otherwise some packages will have dependencies that can't be written precisely in the .cabal file, and we'll end up with ugly hacks to work around it. Or we have to allow the build-depends field to be generated by the Setup script, perhaps via hooks. But that means Visual Studio won't be able to work with a package with a dynamic build-depends field. I think a small extension to build-depends is warranted - and I think the suggestion I put forward has pretty good bang per buck. > OTOH, maybe Haskell can become the generic packaging language, ... [ daydreaming deleted :-) ] > For Debian packages in most languages, we have a hunk of executable > stuff for building and installing, and a relatively small amount of > metadata to deal with dependencies and what-have-you. I think we > should try to stick to that model wherever possible, of course, > improving upon Debian where we can. That's fine by me, as long as *enough* of the metadata is concrete so that our tools can manipulate it as necessary. There's always going to be a tension here, but I think we can use the dev environment example as a dipstick to tell us whether we've got enough concreteness. >> [ghc62] >> other-modules: Compat.Ghc62 >> >> [hugs] >> other-modules: Compat.Hugs > > Here we're starting to get complicated, though, and I'm a bit scared > of this complication. Presumably, then you have something in the > source code to optionally #include the correct module? Ok, this wasn't a great example. Yes, you'd also have to optionally import the right module using conditional compilation of some kind. Try this instead: [ghc62] hs-source-dirs: ghc62-compat [hugs] hs-source-dirs: hugs-compat Cheers, Simon

2 1