- Libraries - Haskell.org

fptools/libraries package versioning
by Brian Smith 31 Jul '05

31 Jul '05

When we send it patches to fptools/libraries that change the API, should we include a patch to increment the version number too? For example, I have a patch that adds bindings for WaitForMultipleObjects and WaitForSingleObject to the Win32 package. Should I increase the Win32 package's version number in the patch I send? Also, now Hugs is building again on Windows (at least the command-line part) and perhaps there will be a release of (Win)Hugs soon. It would be convenient to be able to say that the next release of Hugs comes with, e.g. Win32-1.1 and HaXml-1.13.1. Thanks, Brian

2 3

Raw filenames vs locales
by Ian Lynagh 30 Jul '05

30 Jul '05

Hi all, I'd like to propose some changes to the IO library to fix some problems (IMO) with hugs' recent closer adherence to the Haskell 98 report. I believe this is orthogonal to the proposed new IO library stuff that's been discussed before. The rest of this mail is in 3 parts. First I describe the problem, then a proposed solution, and finally some comments on backwards compatibility. =========== The problem =========== With it's closer adherence to the Haskell 98 report, it is no longer possible with hugs to manipulate files using the standard IO functions if the filenames are not representable in your locale. To demonstrate the problem consider this: -------------------------------------------------- touch `printf "1\xA4"` echo ' import System.Directory (getDirectoryContents) import Data.Char (ord) main = do xs <- getDirectoryContents "." print (map (map ord) xs) ' > foo.hs for locale in en_GB en_GB.ISO-8859-15 en_GB.UTF-8 do echo "===============================" echo "Doing $locale" LC_ALL=$locale ../runhugs foo.hs done -------------------------------------------------- Here we create a file whose filename if 1\xA4. \xA4 is a "currency sign" in ISO-8859-1, "euro sign" in ISO-8859-15 and not a valid character in UTF-8. We then print the results of getDirectoryContents, converting the Chars to Ints so we can see what's going on. The result is this: =============================== Doing en_GB [[46],[46,46],[49,164],[102,111,111,46,104,115]] =============================== Doing en_GB.ISO-8859-15 [[46],[46,46],[49,8364],[102,111,111,46,104,115]] =============================== Doing en_GB.UTF-8 [[46],[46,46],[49,65533],[102,111,111,46,104,115]] The third file is the interesting one. We have: ISO-8859-1: 164 = U+A4 = "currency sign" ISO-8859-15: 8364 = U+20AC = "euro sign" UTF-8: 65533 = U+FFFD = "replacement character" "replacement character" is "used to replace an incoming character whose value is unknown or unrepresentable in Unicode". ================= Proposed solution ================= My suggestion is essentially that we change all functions using the FilePath type to instead use FilePath a => a. [ By jumping through hoops I think this could be done H98-compatibly, but for simplicity I'll ignore that for now. I'm not sure if it's a problem for any impl anyway? ] I imagine the class would look something like class FilePath a where to_filename :: a -> IO FileName from_filename :: FileName -> IO a from_free_filename :: FileName -> IO a from_free_filename f = do x <- from_filename f free f return x with_filename :: FilePath a => a -> (FileName -> IO b) -> IO b with_filename x f = do x' <- to_filename x res <- f x' free_filename x' return res We would then have System.IO.Impl.getDirContents :: FileName -> [FileName] System.IO.getDirContents :: FilePath a => a -> [a] -- Could be more general System.IO.getDirContents x = do ys <- with_filename x Impl.getDirContents mapM from_free_filename ys On Unix systems FileName would be a Ptr Word8. My knowledge of Windows isn't great, but I think there it would be an array of 16-bit values? We would have instances of FilePath for String and [Word8] to solve the immediate problem. String would be the current behaviour, but [Word8] would be converted to a FileName unchanged. On Windows it would probably be necessary to throw an exception if a [Word8] is passed which is not valid utf8. It would also be nice to have a FileName instance, to avoid unnecessary conversions. A Ptr Word8 instance would also be handy for things like darcs' FastPackedString module to be able to use efficiently (without taking a round trip via a lazy list). ======================= Backwards compatibility ======================= I haven't done any research into it, but I hope that a lot of the time this will not be an issue as the impl will be able to infer the type String is being used, either by a string literal, the fact it is putStrLn'd, there is a type signature saying it is a String, etc. The Haskell 98 modules like IO could re-export the functions with their types restricted to what they are now. This would give us complete backwards compatibility to Haskell 98. It is certainly possible for there to be ambiguities in programs that use the hierarchial libraries, however. Possible solutions are: * Tell people to add type sigs to fix it. * Define the new stuff in System.IO.Impl in a package iobase. The oldio package would then contain System.IO which re-exports all the functions with the old types, and the io package would do the same with the new types. Unfortunately i don't think this would work if you have some libraries compiled against the io package you don't want to use. I think this might be an argument that the package system is not being flexible enough. That's all I've got. Comments welcomed! Thanks Ian

4 4

finding the dependecies of cabal packages
by Duncan Coutts 29 Jul '05

29 Jul '05

Hi all, Over in Gentoo packaging land we're trying to build tools to automate the process of packaging cabal packages. ("package" is far to much of an overlaoded word!) So we have a tool "cabal2ebuild" which will produce an ebuild from a cabal file. An ebuild is a text file that contains all the information that the Gentoo packaging system needs to be able to install the program/library in question. In particular the ebuild must list all the dependencies of the package. Required Haskell libraries are listed in the "build-depends:" field in the cabal file, so that one is easy. However it is not clear how to find out the build tools that the cabal package requires. For example packages may require alex or happy or c2hs or other build time tools. Is there any better way to find if these tools are needed than to look through the source tree for any files with the appropriate extensions ".y", ".x", ".chs" etc? Duncan

5 15

RE: Extending the dependency syntax
by Simon Marlow 29 Jul '05

29 Jul '05

On 28 July 2005 22:24, Isaac Jones wrote: > "Simon Marlow" <simonmar(a)microsoft.com> writes: > >> So here's a strawman proposal to address some of the use cases raised >> recently[1] that Cabal doesn't currently support well. This is just >> a first attempt; I expect we'll need to make changes or start over >> if we discover cases that it doesn't handle. > > (snip proposal) > > Lots of this is stuff I've been wanting to ad at some point. I think > that rather than overloading the build-depends field, we should add a > new field for tool dependencies; otherwise, we could get name > conflicts between the two and such. ok - I put them in the build-depends field because I wanted to do the cool stuff with optional stanzas and use it with compiler and tool versions too. There's no reason you couldn't do this in a separate field. > I'm not positive that the simple build infrastructure would be able to > take all of the information in the tool-dependendencies into account; > for instance, discovering the version of happy and ghc might be a bit > tricky and error prone. We already discover the version of GHC. And it can't be any more tricky than using autoconf :-) > (snip) >> ---------------- >> name: mylib >> version: 1.0 >> exposed-modules: My.Lib >> build-depends: >> base>=1.0 >> (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs]) > > Debian has something like this, where you can specify optional > dependencies. I've always planned to implement something like this > for Cabal. > > At the same time, I think we need to be careful not to re-invent make > and re-invent dpkg / rpm via feature-creep. I understand Duncan's > desire to have absolutely everything automated for gentoo without the > gentoo packager having to intervene, but that's just what the OS > packagers have been dealing with for every language. I want Haskell > to be easier than other languages (historically, it's been harder) but > if we make the package syntax more and more complex, I think we run > the risk of expending a lot of energy for a small amount of benefit by > creating a whole new language for packages. I don't think it's a question of re-invention - we really do need a more expressive language for dependencies in Cabal. Otherwise some packages will have dependencies that can't be written precisely in the .cabal file, and we'll end up with ugly hacks to work around it. Or we have to allow the build-depends field to be generated by the Setup script, perhaps via hooks. But that means Visual Studio won't be able to work with a package with a dynamic build-depends field. I think a small extension to build-depends is warranted - and I think the suggestion I put forward has pretty good bang per buck. > OTOH, maybe Haskell can become the generic packaging language, ... [ daydreaming deleted :-) ] > For Debian packages in most languages, we have a hunk of executable > stuff for building and installing, and a relatively small amount of > metadata to deal with dependencies and what-have-you. I think we > should try to stick to that model wherever possible, of course, > improving upon Debian where we can. That's fine by me, as long as *enough* of the metadata is concrete so that our tools can manipulate it as necessary. There's always going to be a tension here, but I think we can use the dev environment example as a dipstick to tell us whether we've got enough concreteness. >> [ghc62] >> other-modules: Compat.Ghc62 >> >> [hugs] >> other-modules: Compat.Hugs > > Here we're starting to get complicated, though, and I'm a bit scared > of this complication. Presumably, then you have something in the > source code to optionally #include the correct module? Ok, this wasn't a great example. Yes, you'd also have to optionally import the right module using conditional compilation of some kind. Try this instead: [ghc62] hs-source-dirs: ghc62-compat [hugs] hs-source-dirs: hugs-compat Cheers, Simon

2 1

RE: Extending the dependency syntax
by Simon Marlow 29 Jul '05

29 Jul '05

On 28 July 2005 22:48, Duncan Coutts wrote: > My own personal opinion is that cabal/haskell packaging can be made > the most sucessful if we take advantage of the native packaging > systems and so make it possible to automate as much as possible the > process of wrapping a cabal package into a native package. I think we > will reach the greatest audience by this approach. I'd just like to add a big +1 to this comment - that was always my feeling when we were originally thinking about Cabal, but you're right that making the link between Cabal and native package tools hasn't received enough attention yet. Cheers, Simon

2 1

Time Library Date Normalisation and Arithmetic
by Ashley Yakeley 29 Jul '05

29 Jul '05

I'm thinking of something along these lines: Normalisation ------------- class (Eq a,Ord a) => Normalisable a where isNormal :: a -> Bool normaliseTruncate :: a -> a normaliseRollover :: a -> a The normalise functions would work like this: normaliseTruncate 2005/14/32 -> 2005/12/32 -> 2005/12/31 2005/-2/-4 -> 2005/01/-4 -> 2005/01/01 normaliseRollover 2005/14/32 -> 2006/02/32 -> 2006/03/04 2005/-2/-4 -> 2004/10/-4 -> 2004/09/26 (doing both steps in each call) Calendar Arithmetic ------------------- This is one way of providing arithmetic for such things as Gregorian dates: data TimeUnit d = TimeUnit addTimeUnitTruncate :: Integer -> d -> d addTimeUnitRollover :: Integer -> d -> d diffTimeUnitFloor :: d -> d -> Integer days :: (DayEncoding d) => TimeUnit d gregorianMonths :: (DayEncoding d) => TimeUnit d gregorianYears :: (DayEncoding d) => TimeUnit d So for instance, to add three months to a date d, you do this: d' = addTimeUnitTruncate gregorianMonths 3 d This might also be used for minutes and so forth. And you might also be able to do this: multipleTimeUnit :: Integer -> TimeUnit d -> TimeUnit d weeks = multipleTimeUnit 7 days There's an alternative way using classes, for days only: class (Integral u) => TimeUnit u where addTimeUnitTruncate :: (DayEncoding d) => u -> d -> d addTimeUnitRollover :: (DayEncoding d) => u -> d -> d diffTimeUnitFloor :: (DayEncoding d) => d -> d -> u newtype Days = Days Integer instance TimeUnit Days newtype GregorianMonths = GregorianMonths Integer instance TimeUnit GregorianMonths newtype GregorianYears = GregorianYears Integer instance TimeUnit GregorianYears d' = addTimeUnitTruncate (GregorianMonths 3) d My preference is for the former. -- Ashley Yakeley, Seattle WA

4 5

RE: Extending the dependency syntax
by Simon Marlow 28 Jul '05

28 Jul '05

On 28 July 2005 22:36, Duncan Coutts wrote: > While the sensible normal behaviour might be to just pick up as many > dependencies as the environment supports, packaging systems like to > know about all dependencies and control them. For example it is > normal to build a package on one system and deploy it on another. If > there are extra dependencies that are picked up (because they happen > to be in the build environment) it may cause a failure on the target > system because the dependency is not present. A very good point. > As an example consider Gtk2Hs which has a number of optional > dependencies (libglade, GConf, Mozilla). If you do a normal build then > it will automatically build all the optional bits for which the > dependencies are satisfied. This is what casual users want. > > However for packagers we provide a --enable-packager-mode flag which > turns off all automatic dependency decisions and so the packager must > explicitly specify --enable-gconf --enable-libglade etc. Thank you, you've provided an excellent example to demonstrate the flexibility of the scheme I proposed :-) I need one more operator, which I'll write ':', and which means the same as a Prolog cut(!). Don't worry if you're not familiar with Prolog, all will become clear. So here's how you express the Gtk dependencies: build-depends: ("packager-mode": ("enable-gconf": gconf>=1.0 [gconf] ||) ("enable-libglade": libglade>=1.0 [libglade] ||) ) || ( (gconf>=1.0 [gconf] ||) (libglade>=1.0 [libglade] ||) ) In words, this says - if "packager-mode" is defined, then - if "enable-gconf" is defined then require gconf>=1.0 and process the [gconf] stanza - if "enable-libglade" is defined then require libglade>=1.0 and process the [libglade] stanza - otherwise, - if gconf>=1.0 is available, process the [gconf] stanza - if libglade>=1.0 is available, process the [libglade] stanza I have to admit, I think this is beautiful. But I'm prepared to believe that others might prefer to go for something a bit more concrete. I imagine that the packager-mode concept is something that we'd want every package to conform to, so we don't want to require every packager to write the appropriate build-depends magic. OTOH, I still like the simplicity of the design. We could add built-in packager-mode support like this: (a) --packager-mode causes all dependencies inside a disjunction to fail by default, and (b) --enable-foo forces an optional dependency on foo to be required. So then the packager just writes build-depends: (gconf>=1.0 [gconf] ||) (libglade>=1.0 [libglade] ||) But this is starting to get a bit subtle. Cheers, Simon

2 1

RE: Extending the dependency syntax
by Simon Marlow 28 Jul '05

28 Jul '05

On 28 July 2005 23:33, Ian Lynagh wrote: > For libraries we(Debian)'d need to be guaranteed that the left-most of > the satisfactory options had been chosen. Absolutely - that was part of the (intended) specification. Cheers, Simon

1 0

Extending the dependency syntax
by Simon Marlow 28 Jul '05

28 Jul '05

So here's a strawman proposal to address some of the use cases raised recently[1] that Cabal doesn't currently support well. This is just a first attempt; I expect we'll need to make changes or start over if we discover cases that it doesn't handle. [1] http://www.haskell.org//pipermail/libraries/2005-July/004152.html http://www.haskell.org//pipermail/libraries/2005-July/004133.html Firstly, we define certain pseudo-packages on which dependencies can be expressed. The pseudo packages would consist of the compilers and pre-processors that Cabal understands, and any other tools required by the build process. eg. build-depends: ghc>=6.4 happy>=1.5 base>=1.0 Now, I suggest we expand the language of dependencies to include disjunction and optional stanzas. The syntax I have in mind is this: deps ::= adep* dep-expr ::= deps '||' dep-expr adep ::= pkg version-range | '(' dep-expr ')' | '[' stanza ']' The idea is that Cabal evaluates the dependencies trying disjunctions left-to-right, where optional stanzas [foo] are assumed to be satisfied. When a set of satisfying dependencies has been found, Cabal processes all the optional stanzas it contains. We now need to be able to accumulate values from fields specified multiple times, rather than require each field to be specified at most once. Note that a list of dependencies can be empty, so an optional dependency is specified as (|| dep). Here's an example where a library can be built by multiple compilers, where each compiler requires different modules to be compiled: ---------------- name: mylib version: 1.0 exposed-modules: My.Lib build-depends: base>=1.0 (ghc>=6.4 || ghc>=6.2 [ghc62] || hugs-any [hugs]) [ghc62] other-modules: Compat.Ghc62 [hugs] other-modules: Compat.Hugs ----------------- This covers Duncan's requirements, and Brian Smith's 1(a),1(b),1(d),2(a). 1(c) is not covered yet: > (c) Cabal depends on HUnit if DEBUG is set, but it doesn't depend on > HUnit if DEBUG is not set. (The dependency isn't controlled by the > existence of a package, but by a configuration option.) This is related to the concept of "configurations", which we need in Visual Studio too. The idea is that a package can be built in one of several different ways, where each way specifies different compiler options or other settings. A common use is to have "debug" and "release" configurations, where in the former we #define DEBUG, and in the latter we turn on -O. How might we include configurations in Cabal? A configuration selection behaves rather like a dependency that is satisfied by the environment - perhaps as a command line option to Setup.lhs. We could extend the dependency syntax to include a new kind of dependency: adep ::= ... | config Perhaps the concrete syntax for config is a string (eg. "DEBUG"). We can express the HUnit example now: ---------------- build-depends: base>=1.0 ("DEBUG" HUnit>=1.0 [debug] || [release]) [debug] other-modules: Tests ghc-options: -DDEBUG -Onot [release] ghc-options: -O ---------------- Comments? Cheers, Simon

5 6

Error manipulation functions
by Ian Lynagh 28 Jul '05

28 Jul '05

Hi all, I couldn't see a way to write any of these functions without using GHC.IOBase. Did I miss one? If not, can they (or something that would allow them to be written) please be added to System.IO.Error? import GHC.IOBase ( IOException(ioe_location), IOErrorType(UnsupportedOperation, HardwareFault) ) add_to_error_loc :: Exception -> String -> Exception add_to_error_loc (IOException ioe) s = IOException $ ioe { ioe_location = s ++ ": " ++ ioe_location ioe } add_to_error_loc e _ = e isUnsupportedOperationError :: IOError -> Bool isUnsupportedOperationError = isUnsupportedOperationErrorType . ioeGetErrorType isUnsupportedOperationErrorType :: IOErrorType -> Bool isUnsupportedOperationErrorType UnsupportedOperation = True isUnsupportedOperationErrorType _ = False Thanks Ian

1 0