September 2003 - Libraries

Network.recvFrom strangeness
by Peter Simons 23 Sep '03

23 Sep '03

The declaration of Network.recvFrom looks strange: recvFrom :: HostName -> PortID -> IO String I thought that recvFrom should _return_ the address of the peer, not require it as an argument? At least that's what the corresponding 4.4BSD routine does: | ssize_t recvfrom(int s, void *buf, size_t len, int flags, | struct sockaddr *from, socklen_t *fromlen); | | DESCRIPTION | The recvfrom and recvmsg calls are used to receive messages | from a socket, and may be used to receive data on a socket | whether or not it is connection-oriented. | | If from is not NULL, and the socket is not connection-oriented, | the source address of the message is filled in. The argument | fromlen is a value-result parameter, initialized to the size of | the buffer associated with from, and modified on return to | indicate the actual size of the address stored there. By the way: I am not sure whether HostName is the appropriate type here! IMHO the routine should rather return the _address_ of the peer, not the hostname. (For example, what does it return when the resolver fails?) Peter

1 0

library packages and Debian
by Ross Paterson 21 Sep '03

21 Sep '03

[rather Debian-specific, I'm afraid] On the Wiki, two solutions to the problem of N compilers and M libraries are proposed: 1. NxM binary packages (x number of architectures) 2. M packages, compiled with the available compilers on installation, and recompiled on upgrade. and Ganesh writes: > I think solution 2 is horrible; building Haskell code can take a > long time, and it opens users up to experiencing failures at package > install time. Compiling code is something that properly belongs at > package build time. Solution 1 admittedly has the drawbacks mentioned, > but as long as timely updates happen (e.g. when a new release of GHC > appears) it's superior. Agreed. 2 also requires the user to have a build environment (with more packages), but the failure at upgrade time is the worst problem. Suppose I have several compilers and libraries installed, and one day I upgrade the system, get a new version of GHC, and for whatever reason one of the library packages I have installed doesn't work with that version. My system is broken. Would the following work? - one source package per library package, generating a binary package for each compiler (and each architecture). Debian maintainer for the package gets it working with each compiler, and records dependencies on specific version numbers of each compiler in the control file. (Maybe there are compiler- and architecture-independent packages too.) - when a Debian package of a compiler is upgraded, the Debian maintainer for that compiler also performs (preferably automatically) a non-maintainer upload for each library package, in which the only change is to the version number of that compiler that the library package depends on. Would not the autobuilder then do the rest? Then if the compiler upgrade causes some library package to fail, the user will see a conflict: they either remove the library package or postpone the compiler upgrade. This only checks build failures, but library authors and maintainers could include tests in the build process. Might be a bit hard on the autobuilders, of course.

4 4

RE: RealWorld
by Simon Peyton-Jones 19 Sep '03

19 Sep '03

| > I could be a bit out of date but I think Hugs uses different implementations | > because IO has concurrency and exceptions and ST doesn't. Apart from effort | > required, I don't know if there is any cost in switching. | | I think IORef and STRef have both been MUTVAR as far back as I can see. | The IO and (strict) ST monads used to be very different, and no longer | are (so they both use the same state primitives now), but I don't think | this affects the IORef = STRef IOState question. I'm a bit confused here. Is the proposal that the library design exposes to the programmer the following type synonyms: type IO a = ST IORegion a type IORef a = STRef IORegion a I think it is sound to do so. The advantage would be that newIORef and newSTRef would be the same (newRef, perhaps); ditto readRef, writeRef etc. I think this would be fine for GHC. It might not be fine for Hugs because they implement IO and ST differently I think. No idea about NHC. Or, alternatively, is the question something to do with the internals of the library, of interest to implementers but not to users of the library? Or is it just to do with the type of stToIO, which should be stToIO :: ST IORegion a -> IO a Simon

3 5

RE: RealWorld
by Simon Marlow 18 Sep '03

18 Sep '03

> I'm a bit confused here. > > Is the proposal that the library design exposes to the programmer the > following type synonyms: > > type IO a = ST IORegion a > type IORef a = STRef IORegion a > > I think it is sound to do so. The advantage would be that > newIORef and > newSTRef would be the same (newRef, perhaps); ditto readRef, writeRef > etc. And possibly type IOArray ix e = STArray IORegion ix e type IOUArray ix e = STUArray IORegion ix e and lots of duplication in the overloaded array support goes away. I'd be very careful that we're not burning any bridges before doing this, though. Cheers, Simon

1 0

RE: Packages
by Simon Peyton-Jones 15 Sep '03

15 Sep '03

| > Meta-data that describes the package | ... | > - Dependencies, expressed as a set of triples | > (package identifier, version range, grafting location) | | This raises an issue. | | Someone releases package foo. I release a package bar that uses foo, | where foo is grafted at A.B.Foo. Someone else wants to use bar, but they | have foo grafted at C.D.Foo, because they also have a completely | unrelated otherfoo grafted at A.B.Foo. Does their use of bar force foo | to be grafted at the same place I grafted it at? No, it doesn't, and that should be made clearer in the proposal. Suppose package P imports packages Q, R. Then the grafting locations for Q,R specified in P's meta-data are only needed to make sense of the *source code* of P. to compile P from source you must graft Q, R at the specified places. But, having compiled it (or having downloaded a compiled version) it's not necessary to even know of where P expects Q and R to show up. For an interpreter like Hugs, matters are a bit more complicated, I guess. When loading a module M from P, Hugs would have to remember where Q and R show up in the module hierarchy. Yet that seems an inevitable consequence of not fixing a single global location for every module at the moment of its birth. Simon

1 0

Packages
by Simon Marlow 15 Sep '03

15 Sep '03

Following discussion on and off this list, we've re-written the proposal for changes to the packages/libraries story. I've included the proposal as plain text below so that it can be quoted easily, but if you would prefer to read it in HTML there's a version here: http://www.haskell.org/~simonmar/packages.html The main differences relative to the previous proposal are: - Packages, with unique package names, are given more emphasis. - Grafting, in particular grafting a library in multiple places, is given less emphasis in this new proposal. Multiple grafting isn't essential: the two motivating examples we had previously (versioning and identifying APIs by GUID) are both served by having unique package names. Motivation ---------- This proposal describes an implementation-independent mechanism, called "packages", that allows a library or other group of Haskell modules to be wrapped up as a single unit. Here is why we need this mechanism: - We want to lower the barrier to shipping a new Haskell library. At present a library author must, for each module M in his library, find a place for M in the single global module hierarchy. Either she make the library inconvenient to use (by using deeply-nested module names) or else risks clashing with "popular" sitse in the tree. - An author is likely to produce multiple versions of a library. If these live in different parts of the global module name space, one has to change every importing module to switch to the new version. If they re-use the same names as the previous version, it's hard to know which version is required, and impossible to build a program that simultaneously needs two different versions. For example, perhaps your program uses version N of an API, but you import a library which depends on version N-1 of the same API). - We want to have some support for abbreviating module names in source code (to avoid very long module names), and being able to move a sub-hierarchy of source modules around in the global hierarchy, without modifying the source code directly. - We want to be able to uniquely identify a library API, for the purposes of expressing source code dependencies, and for the purposes of being able to automatically install dependencies. This would make it possible to automate the business of installing the necessary support packages for a given package. Packages -------- In this proposal, a "package" is the unit of distribution. A package defines a sub-tree of modules; eg. GTK, GTK.Window, GTK.Button, ... However, crucially, the package does not define absolute module names, but instead can be grafted into the module hierarchy at different sites, without recompilation (see "grafting" below). Every package has a "package identifier". A package identifier is a string, eg. "gtkhs". It is the intention that package identifiers are globally unique, but we don't intend to enforce this in any rigorous way. There will probably be a web page which maintains a list of package identifiers, and where one can register a new one. Things will go badly wrong if you try to use two packages with the same identifier. A "package name" is defined as a pair of a package identifier and a version number. For example, "gtkhs-0.4". A package name uniquely identifies an API: that is a set of modules, and the interfaces to those modules. The package web interface might well link to the documentation for each package API, as well as the place where the package can be obtained. Note that because we have a way to uniquely identify an API, GUIDs are not required. A package takes two forms. A "source package" consists of Meta-data that describes the package - The package identifier - Package major and minor version - A default grafting location for this package - Dependencies, expressed as a set of triples (package identifier, version range, grafting location) - Etc (e.g. documentation, installation materials) Payload - Haskell modules (source, object, interface, analysis results) - Associated C header files or other support code A "binary package" is the same, except that - The Haskell modules and other source materials are in compiled, object code, form. - Information about which compiler was used, and which version of that compiler - The dependencies are expressed as a set of package names only. The existence of packages offers new opportunities for encapsulation. For example, the meta-data for a package could expose some, but not all, of the modules in the package, giving the package author the chance to securely hide internal modules. Grafting -------- The modules in a package form a sub-hierarchy. This sub-hierarchy can be mapped into the global module hierarchy at any point when the package is used; this operation is called "grafting". For example, if we have modules Gtk Gtk.Window Gtk.Button in the package "gtkhs-0.4", and this package is grafted onto "Graphics.UI", then these modules would be available to a user of the "gtkhs-0.4" package as Graphics.UI.Gtk Graphics.UI.Gtk.Windwow Graphics.UI.Gtk.Button Note that this provides a simple way to abbreviate module names in source code, as well as providing a way to easily move an entire sub-hierarchy of modules around in the global hieararchy without changing every source file. Installing a package -------------------- Installing a package is the action a client takes to make a new package known to a particular Haskell implementation. A package comes with a default grafting location. Installing the package makes it available at that grafting location, without the need for any command-line flags. GHC calls such packages "auto packages", and we will follow that terminology here. At most one version of any given package can be an auto package, and (by convention) it is always the latest installed version. That is, when installing a package, that package only becomes an auto package (available without flags) if its version is later than any other installed version of that package. It should also be possible to install a package at a site different from its default grafting location. Existing package managers such as RPM don't have a way to specify a grafting locations anyhow, but the Haskell library infrastructure (currently in development) would no doubt have a way to change the grafting location if used directly. Specifying Grafting Locations at compile time --------------------------------------------- Each Haskell implementation should provide a means for specifying packages and grafting locations when compiling Haskell source code. One possibility for GHC is to extend the command-line syntax for -package, eg.: ghc -package gtkhs-0.5:Graphics.UI In that case, the command-line choice for a particular package should override (replace) the install-time choice for that package. For example, if gtkhs-1.7 is installed so that it is available by default, then the command above would *remove* gtkhs-1.7 from the module name space, and instead graft in gtkhs-0.5. Why? Because both specify the same package name "gtkhs". In short, any one compilation should see at most one version of each package. Overlapping Packages -------------------- If two packages are grafted in such a way that they both define the same absolute module in the module hierarchy, then it is an error to import that module. (This is akin to the error that is reported if two import statements in a Haskell program bind the same name.) For example, if "gtkhs-0.4" defines a module "GTK.Misc", and "graph-1.8" defines the module "Misc", and one says ghc -package gtkhs-0.4:Graphics.UI -package graph-1.8:Graphics.UI.GTK then the import declaration import Graphics.UI.GTK.Misc would be an error, because it is defined by both packages. Shipping a new library ---------------------- Joe H. Programmer just wrote a small library and wants to share it with the world. What does he have to do? Under our proposed scheme, it would go something like this: - Make up a package name, and register it using the web interface at haskell.org, to avoid anyone else using the same name. - Decide what the default grafting location for the library should be. There will be some hierarchy layout guidelines on haskell.org for library writers to follow - these won't be set in stone, though. The worst that can happen is that your package will overlap with another common one, and will end up getting turned off by default when installed. - Package up your library using the Haskell library infrastructure, and share it. Implementation -------------- Here's what we have to do for GHC: 1. An entity in a Haskell program was previously uniquely identified by its (module name, identifier) pair, where the module name is the module in which the entity is defined. This now becomes a triple: (package name, relative module name, identifier). 2. Extend the package spec syntax to include grafting locations, and lists of overlapping packages. 3. Extend the -package flag syntax to allow specifying a new grafting location. 4. Change the searching semantics to take into account grafting locations. 5. Implement the "version overriding" semantics, and error checking to do with visibility of overlapping packages. (1) is quite a fundamental changes, but (2-5) are all quite straightforward. I think a similar strategy would work for Hugs & NHC, although Hugs at least will need to also acquire support for packages.

2 1

Question regarding template Haskell
by Yu Di 11 Sep '03

11 Sep '03

Hi, I have a question regarding template Haskell: from the documentation, I have the impression that it is impossible to reify the type of the variable passed to the template function, is this true? Or, in other words, can I write a function like: getTupleCount :: a -> ExpQ getTupleCount x= do t <- (reifyType x) --work on t ... which returns an expression that is an integer for any tuple input, but reduces to error for any input that is not a tuple? Thanks! Di, Yu 9.11 __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com

1 0

RE: RealWorld
by Simon Marlow 11 Sep '03

11 Sep '03

> More suggestions: > > 1) Pursuing the view that this thing identifies a special ST-style > state included in the IO monad, we could view the IO state stuff as > a special case of the ST stuff, changing > > newtype IORef a = IORef (STRef IOState a) > newtype IOArray i e = IOArray (STArray IOState i e) > newtype IOUArray i e = IOUArray (STUArray RealWorld i e) > > (currently exported opaquely) to > > type IORef = STRef IOState > type IOArray = STArray IOState > type IOUArray = STUArray IOState > > so that there would only be one type of reference. This constrains the implementations of IORef and STRef to be the same (obviously). I can't immediately see any benefit from using different implementations for these two types, so maybe it's not an issue. > 2) For additional genericity, define a class > > class MonadST s m | m -> s where > liftST :: Control.Monad.ST.ST s a -> m a > [snip] > > then all the operations lift to all such monads, e.g. > > readSTRef :: MonadST s m => STRef s a -> m a > readSTRef r = liftST (Data.STRef.readSTRef r) Same argument as for MonadIO: I don't like the idea of generalising all the ST functions, and just generalising some of them would be inconsistent, so don't generalise any of them. Providing MonadST and liftST is fine, though. Cheers, Simon

3 3

RE: Toward a Unified Packaging System
by Simon Peyton-Jones 11 Sep '03

11 Sep '03

| Isaac has a rather nice writeup in the Wiki: | | http://haskell.org/hawiki/LibraryInfrastructure It's a great start, but (for example) the key section (2) has "DistUtils" as a heading, which means nothing to me, and talks of python-style distutils. I'm utterly ignorant of Python. Learning from existing systems is definitely the Right Thing, and the discussion among experts can use as much terminology as is useful, but I'm still hoping ultimately for a presentation that does not assume you are already au fait with other systems. S

2 1

RE: Toward a Unified Packaging System
by Simon Marlow 11 Sep '03

11 Sep '03

> As you know, I'm dead keen to see this happen. Thank you for leading > this effort. > > What would really help me is a summary, in one place (on the web) of > what you propose, without assuming much prior knowledge. Isaac has a rather nice writeup in the Wiki: http://haskell.org/hawiki/LibraryInfrastructure Cheers, Simon

1 0