January 2003 - Libraries

Re: Debian library packaging? or, how to get a single deb to work for hugs/ghc/nhc?
by Isaac Jones 30 Jan '03

30 Jan '03

Shae Matijs Erisson Wrote: > In short, I'd like to see some libraries packaged for Debian. I'm > especially interested in seeing the various Gui libs arrive. My > question though, is how to package a lib in such a way that it works > for all three of the packaged Haskell compilers? There are related discussions on various lists. On Haskell-cafe, we talked about packaging Debian stuff: http://haskell.org/pipermail/haskell-cafe/2002-December/003708.html Also on Haskell-cafe, there was a small discussion on the state of libraries and the lack of a central repository for libraries: http://haskell.org/pipermail/haskell-cafe/2002-October/003516.html On Glasgow-haskell-users, there was a discussion about how to create a system for authors of 3rd party libraries independently of a particular compiler: http://haskell.org/pipermail/glasgow-haskell-users/2002-November/004322.html >From Simon Marlow's comments: >> Suppose we had an infrastructure which someone could plug their >> library source code into, and immediately get >> >> - a source tarball which will configure/build/install on a >> Unix/cygwin system with GHC installed, and provide a GHC package >> when installed. >> >> - skeleton binary packages for RPM, Windows installer, *BSD ports, >> Gentoo ebuild, etc. The binary packages will require some >> programmer intervention (such as setting the dependencies >> correctly), but much of the work can be done automatically. Though I haven't used it extensively yet, hmake seems like a good system to start with since it has nice flags like "-nhc" and "-ghc", it can be configured to use either, it has sane defaults, etc. FPTools does have a nice makefile system that 3rd party developers could adopt, though at the moment it sort of relies on those libraries being inside a ghc source directory, and I don't know how well it would work w/ nhc or Hugs. Several people have commented that the problems with Haskell libraries and compilers are similar to the problems with elisp. Maybe a solution would be to augment hmake with an idea of registering packages and {re}compiling them when new / different compilers are installed. I'm starting to like this idea a lot better than trying to make the packages depend on the particular compilers and versions. I notice that the Haskell packages sometimes get out of date, and it would be hard to get all the maintainers coordinated enough to release packages at the same time a new compiler comes out. More thoughts? peace, isaac

6 6

Debian library packaging? or, how to get a single deb to work for hugs/ghc/nhc?
by Shae Matijs Erisson 29 Jan '03

29 Jan '03

In short, I'd like to see some libraries packaged for Debian. I'm especially interested in seeing the various Gui libs arrive. My question though, is how to package a lib in such a way that it works for all three of the packaged Haskell compilers? Emacs and Python both compile source into bytecode. GHC needs access to both .hi and .o files Hugs needs access to source only, I think. Some libraries are picky about requiring -fglasgow-exts, are there any libraries which are picky about requiring the lack of -fglasgow-exts? What other factors are there? What solutions are there? Thanks, PS. after a long discussion on the #haskell irc channel about where to create a new mailing list, we decided to take Simon Marlow's advice and use an existing, but quiet mailing list. If this bothers someone, I'd be happy to move this discussion elsewhere. -- Shae Matijs Erisson - 2 days older than RFC0226 #haskell on irc.freenode.net - We Put the Funk in Funktion

1 0

HaskellDB
by Dominic Steinitz 27 Jan '03

27 Jan '03

Having used HToolkit successfully to extract information from a MySQL database, I am painfully aware of the limitations of sending embedded SQL strings and finding out at runtime that something didn't quite work (in an often subtle and difficult to track down way). I'd love to be able to put HaskellDB on top of HToolkit. There are two approaches: i) add extensible records to ghc or ii) modify HaskellDB so it uses Haskell records not extensible records (I'd have modify HaskellDB anyway to use HToolkit rather than talk to ODBC directly). The latter sounds as though it might be more feasible that the former. Having read the paper and briefly looked at the code, I assume the extensibility is required if you have two tables each with the same column name and therefore you want to use the same label in two different records. Obviously you can't do this with Haskell records as the labels become top level functions. However, if I rename labels then I won't have this problem. For example, if I have two tables, passwd and notify each containing a column called email then I could have labels notifyEmail and passwdEmail and remember that they really referred to email. Does anyone have any views on this? Dominic Steinitz

2 1

RE: Standard module header trivia
by Simon Marlow 25 Jan '03

25 Jan '03

> > I'm personally not very fond of the literate style, so > that's why none > > of the existing hierarchical libraries are literate (I actually > > converted several of them from .lhs into .hs). One reason is that > > Haddock doesn't understand literate scripts - or at least, > it expects > > all the documentation to be in comments in the Haskell source, which > > leaves little reason to use literate comments. > > As I recall, we once had a discussion here about program > documentation in > which we (well, only some of us apparently :) concluded that > literate comments > were good for documenting a module's implementation, but > something else was > needed for documenting the client interface. And that led you > eventually to develop Haddock... > > So I guess you are saying that in fact Haddock is suitable > for documenting the > implementation as well as the interface. Is that right? OK, I > didn't know > that; I will have to take a closer look at it. Not really - Haddock is intended as an interface documenting tool only. But there might be scope for extending it to support inplementation docs too, or extending Haddock to pick out annotations from literate comments somehow. I imagine there would be some overlap between the implementation and the interface docs, so that using completely separate documentation formats might be cumbersome. Cheers, Simon

1 0

RE: Standard module header trivia
by Simon Marlow 24 Jan '03

24 Jan '03

> I have a very nitpicking question about the "standard module > header" described > here: > > http://www.haskell.org/~simonmar/libraries/conventions.html > > I am not sure if the header is intended to be automatically > processed, so: > > If the source file is a literate Haskell file, is the header > supposed to be in > the Haskell code portion of the file? Or, if the literate > part is LaTeX, say, > does one put it in a LaTeX comment so that it can appear at > the head of the file? > > Are literate files even allowed for "standard library > modules"? (I don't see > any in the Hugs libraries distribution, for example.) I'm personally not very fond of the literate style, so that's why none of the existing hierarchical libraries are literate (I actually converted several of them from .lhs into .hs). One reason is that Haddock doesn't understand literate scripts - or at least, it expects all the documentation to be in comments in the Haskell source, which leaves little reason to use literate comments. To answer your question, the header is automatically processed by Haddock to generate parts of the documentation. Actually the document you point to is slightly out of date, because the header should begin with '-- |' (see the actual sources for examples). On the other hand, we're certainly not going to ban .lhs files from fptools/libraries. Feel free to cook up some "guidelines for using literate style in the libraries" and I'll add it to the document. Cheers, Simon

2 1

Standard module header trivia
by Frank Atanassow 23 Jan '03

23 Jan '03

I have a very nitpicking question about the "standard module header" described here: http://www.haskell.org/~simonmar/libraries/conventions.html I am not sure if the header is intended to be automatically processed, so: If the source file is a literate Haskell file, is the header supposed to be in the Haskell code portion of the file? Or, if the literate part is LaTeX, say, does one put it in a LaTeX comment so that it can appear at the head of the file? Are literate files even allowed for "standard library modules"? (I don't see any in the Hugs libraries distribution, for example.) Regards, -- Frank

1 0

RE: Endian mode and hPutBuf & hGetBuf
by Simon Marlow 13 Jan '03

13 Jan '03

> Yes, this seems to be the real problem. To actually *do* > anything sensible > castIOUArray has to know the size of the types. Doesn't the Storable > class give us this BTW? The Storable class is almost the right thing, but not quite. For example, a UArray of Bool uses a bitmap whereas Storable.sizeOf (x :: Bool) is probably 4. > > This might be overkill. Alternatively, castIOUArray could > take the new > > bounds as an argument, or we could have a way to replace > the bounds of > > an IOUArray/STUArray (unsafeReplaceBoundsIOUArray?). Comments? > > Well I guess a simple hack would be to supply the size of the new type > in bytes as an additional argument. (I assume the current size can be > figured out from the current bounds and the No. of bytes allocated > for the array.) > > But really, I think the function should be removed altogether because > even if we sort this problem out it's still not portable (the effect > of re-casting an array of Word32's as an array of Word8's will depend > on the native endian mode). I would say once it's an array of Word32's > it's always an array of Word32's. It is non-portable and documented as such - however it does have its uses (we use it in the compiler in a few places). I considered giving it an 'unsafe' prefix, and perhaps we should since you can currently cause a crash by casting to an array of larger type and then writing off the end. Here are the options I see: 1. Change castIOUArray to take the new bounds as an argument. unsafeCastIOUArray :: (Ix i) => IOUArray i e -> (i,i) -> IOUArray i e (rename to unsafeCastIOUArray, because it is certainly possible to cause a crash by giving inappropriate bounds). 2. Change castIOUArray to take the size of the new type as an argument (doesn't really work for Bools unless the "size" is in bits, and might be tricky in general because the size of array internally is always rounded up to a word boundary). 3. Define a separate class which allows us to get the appropriate scaling factor. 4. Add appropriate operations to the MArray class, and overload castIOUArray on MArray. My opinion: (3) and (4) seem like overkill for an API which is still non-portable (you need to know about representations in order to be able to do anything sensible with castIOUArray, so you also know how to rescale the bounds too), and I don't like (2) very much (see reasons above). That leaves (1). > The only reason I wanted to re-cast in the first place is to write it > to a file and the only function available demanded Word8's. That's > why I think separate put/get functions for Word16's, Word32's, > and Word64's are necessary. These functions should also require the > endian mode to be specified explicitly (either by name or with an > additional argument). These are coming as part of the Binary library. > An alternative might be to keep something like castIOUArray and > demand that endian mode was specified explicitly there (and > if that wasn't the native mode castIOArray would have to do > the necessary juggling.) You don't *have* to use castIOUArray here - you can always create a new array explicitly for passing to hPutArray. That would be completely portable, too. > Also, as a minor niggle, I think the names should reflect the type > of array in question. Something like hPutIOUArray instead of just > hPutArray. Ok, I don't have any strong opinions here. You can't "put" any other types of arrays, though. > My thoughts anyway. Perhaps this isn't the right way to go all. > To be honest, I'm pretty confused about the bewildering variety > of array's that seem to be available in Haskell/ghc and what you > can and can't do with the various options on offer :-) The trouble is that our arrays are parameterised over two features: monadic vs. non-monadic (equivalently: mutable vs. non-mutable), and boxed vs. unboxed. It wasn't clear to me how to put these into the module hierarchy - which feature should be higher up? There are a bewildering variety of possible layouts for the Data.Array hierarchy, lots of which make sense. I'd welcome feedback here - but I'm not sure we'll reach an agreement easily. Cheers, Simon

1 0

RE: Endian mode and hPutBuf & hGetBuf
by Simon Marlow 10 Jan '03

10 Jan '03

> On Wednesday 08 January 2003 2:32 pm, Simon Marlow wrote: > > Oops, I lied. hPutArray and hGetArray work on arrays of > Word8 only, so > > in order to get non-portable results you need to use > castIOUArray, which > > is already documented as having undefined behaviour. > > I guess you must mean it's behaviour has not been defined yet. > (Or is it's behaviour undefined by definition? :-) > > I tried to use it recently to save an IOUArray of Word32's to a file > by castIOUArray'ing it to an IOUArray of Word8's and using hPutArray > (AFAICS the only possible way to do this using the current library > functions). > > It didn't work. For an array of N Word32's using N as the argument > to hPutArray only gave me the first N/4 Word32's in the file. > Using 4N gave me an error message about buffers. So I ended up > not using an IOUArray at all :-( That's a symptom of the bug that castIOUArray doesn't change the bounds of he array to reflect the new type, so hPutArray thinks you're trying to access the array outside its bounds. The trouble is, to do this properly we'd need to change the type of castIOUArray, so that it can get hold of the information it needs to rescale the bounds. Currently we have castIOUArray :: IOUArray ix a -> IO (IOUArray ix b) perhaps we'd need something like castIOUArray :: (C a, C b) => IOUArray ix a -> IO (IOUArray ix b) where C is some new class that lets you get at the scaling factors for the types a and b. This might be overkill. Alternatively, castIOUArray could take the new bounds as an argument, or we could have a way to replace the bounds of an IOUArray/STUArray (unsafeReplaceBoundsIOUArray?). Comments? Cheers, Simon

2 1

RE: Endian mode and hPutBuf & hGetBuf
by Simon Marlow 09 Jan '03

09 Jan '03

> Hope I'm not being to stupid here, but I think there's a portability > problem with these functions for anything other than byte oriented > data because the endian mode in memory is system dependent whereas > for files it will depend on the format specified for the file. Personally, I don't see a problem here. hPutBuf is defined to write a sequence of bytes to a file, and that's exactly what it does: the interpretation of the sequence of bytes is an orthogonal issue. We should certainly also provide ways to write binary data into a file in a portable way... but IMO that should be part of the Binary library which is currently being worked on. > I would like to suggest additional functions.. > hPutBuf<N><E> > hGetBuf<N><E> > where N=16,32,64 and E=L(ittle),B(ig) > > The endian mode refers to the file format and necessary conversion > to/from the systems "native" mode is done automatically. > > It would be nice to have similar functions for reading and writing > individual Words too. > > There would be a similar problem with hPutArray and hGetArray > if they were used for reading writing arrays of words other > than Word8 (using castIOUArray perhaps?) hPutArray and hGetArray do indeed generate non-portable files, and they should be documented as such. Cheers, SImon

2 1

RE: Endian mode and hPutBuf & hGetBuf
by Simon Marlow 09 Jan '03

09 Jan '03

> > There would be a similar problem with hPutArray and hGetArray > > if they were used for reading writing arrays of words other > > than Word8 (using castIOUArray perhaps?) > > hPutArray and hGetArray do indeed generate non-portable > files, and they should be documented as such. Oops, I lied. hPutArray and hGetArray work on arrays of Word8 only, so in order to get non-portable results you need to use castIOUArray, which is already documented as having undefined behaviour. Cheers, Simon

2 1