September 2003 - Libraries

Subprocesses (was: [ANNOUNCE] shell-haskell 1.0.0)
by Simon Marlow 08 Jul '04

08 Jul '04

Libraries people, There are currently two ways to create and communicate with external processes: - System.Cmd.system (very basic, doesn't give you access to the process's output but you can use shell primitives to redirect its input/output). - Posix.runProcess (can specify Handles for the process's input/output, but it doesn't create pipes so you can't actually communicate with the subprocess using these Handles.). - POpen.popen (can specify an input String, and gives you lazy output Strings. No interactive communication is possible.) Ok, *three* ways. System.Cmd.system, Posix.runProcess, POpen.popen, and shell-haskell. Er. *Four* ways. David Sankel's shell-haskell creates pipes for stdin/stdout/stderr, and lets you communicate interactively with the sub-process via Handles. It could do with a way to specify the environment and working directory of the sub-process (ala POpen and runProcess), but apart from that it seems to be the most flexible of the four. It could be simplified slightly by using one Handle rather than two for stdin/stdout. We should think about what functionality we want to provide in the core libraries. My preference would be to go for a full version with three pipes (like shell-haskell) and perhaps one or two more specialised versions covering common cases (I don't know what these should be, though). What do people think? Cheers, Simon -----Original Message----- From: haskell-admin(a)haskell.org [mailto:haskell-admin@haskell.org] On Behalf Of David Sankel Sent: 06 May 2003 03:06 To: haskell(a)haskell.org Subject: [ANNOUNCE] shell-haskell 1.0.0 =================== shell-haskell 1.0.0 =================== shell-haskell provides utilities to communicate with other process's via Haskell code. This library can be used for something as simple as getting the output of another program or as complex as interacting with an interpreter. see http://www.electronconsulting.com/shell-haskell for details. David J. Sankel _______________________________________________ Haskell mailing list Haskell(a)haskell.org http://www.haskell.org/mailman/listinfo/haskell

5 10

RE: Library Infrastructure Proposal & Home Page
by Simon Marlow 05 Oct '03

05 Oct '03

I think the proposal is great. Here's some random comments: It might be worth saying earlier on that the library infrastructure is expected to be a layer underneath the platform's native package support (if such support exists). For example, I've never used Python's Distutils, but I have a bunch of Python packages on my system that were installed using the native package system on top of Distutils (I presume). FreeBSD's ports is both a source and binary package system. A FreeBSD system functions perfectly well using only binary packages. To a certain extent Gentoo is the same, although they place far less emphasis on binary packages. FreeBSD (in source or binary mode), and Gentoo, have no support for recompiling packages when something they depend on has changed. This is the biggest failing of these systems IMHO. The 3rd paragraph of 2.1 therefore isn't correct - there's still a big problem for source-based distributions. However, this is not *our* problem, it's theirs - when a dependency is updated, everything that depends on it should be recompiled. ./Setup.lhs bdist: wouldn't it be better to create a source RPM than a binary RPM? Once you have a source RPM the RPM tools can be used to build a binary RPM. Similarly for FreeBSD and Gentoo - I'd expect Setup.lhs to produce a port skeleton/ebuild for me, not a binary package. The PackageConfig file: I'd leave out 'provides' and 'isDefault' unless we have a convincing example which needs them. Possibly add: 'haddock_html_root :: String' and 'haddock_interface :: String' for documentation. I'm thinking about removing 'extra_ghc_opts' - it's highly dodgy when combined with 'auto'. I agree with Ross's comments about data vs. code. Let's abstract as much as possible as data. Cheers, Simon

3 2

RE: FW: Library Infrastructure Proposal & Home Page
by Simon Marlow 30 Sep '03

30 Sep '03

Malcolm Wallace writes: > "Simon Peyton-Jones" <simonpj(a)microsoft.com> writes: > > > I'd like to be confident that the package-description proposals that > > Simon and I have put out isn't incompatible in some way with NHC. > > I have been thinking about your proposal for packages, and although > there are many aspects of it that I like, and it isn't incompatible > with nhc98 as such, my gut feeling is that one particular aspect > of it is actually a very large semantic change/addition, and would > probably involve a significant amount of work to implement. I mean > this change: > > An entity in a Haskell program was previously uniquely identified > by its (module name, identifier) pair, where the module name is the > module in which the entity is defined. This now becomes a triple: > (package name, relative module name, identifier) > > Consider how seemingly simple the current Haskell module system is. > Yet dealing with the full implications of qualified imports, > renamings, > and hidings turned out to be pretty non-trivial when we implementors > eventually started exercising all the corner cases. I know for a > fact that Hugs and nhc98 still contain bugs in this area, and the > subject even merited a Haskell Workshop paper in 2002! FWIW, I agree. This has been troubling me for some time, and I think this is the reason that the packages proposal hasn't been greeted with widespread enthusiasm, despite Simon's & my attempts to simplify it. > My feeling is that the latter might apply. For instance, where in > your proposal you have chosen the new concept of "package name", > I think it would almost be possible to substitute "top-level name in > the library hierarchy". That is, instead of uniquely identifying a > Haskell entity as the triple (gtkhs-0.5, Gtk.Window, widget), I think > one could get away with the pair (GtkHS_0_5.Gtk.Window, widget) where > the top-level hierarchy name is derived in some standard (if ugly) > way from your package name. This also occurred to me. But there's a reason for having a two-tiered system: The package namespace is for distribution, dependencies, and versioning, whereas the module hierarchy is for organising modules by functionality. We long ago decided that the module hierarchy should be used for organising by functionality only. It doesn't seem right to start using it for the meta-properties of libraries too. (I guess I'm prepared to be persuaded otherwise here - there's the argument that having both namespaces can be confusing and is potentially redundant). Certainly the dependencies of a library are an essential part of the library itself, so arguably they should be present in the source (including version information), however they are also required by the packaging/building/distribution infrastructre, so keeping them separate is likely to be more convenient. Having said all this, the subject of grafting is orthogonal to whether we have a separate namespace for rank-2 modules. I consider the facilities that packages give us (versioning, unambiguous dependencies etc.) rather more important than grafting. Grafting essentially gives you a way for both source code and binary code to be independent of its absolute location in the module hierarchy. It would probably be enough to have a mechanism whereby only source code is independent of its absolute location. Ideas for relative module names have been floated before - perhaps someone would like to make a concrete proposal? Cheers, Simon

1 0

FW: FW: Library Infrastructure Proposal & Home Page
by Simon Peyton-Jones 29 Sep '03

29 Sep '03

Comments from Malcolm Wallace on the packages proposal -----Original Message----- From: Malcolm Wallace [mailto:Malcolm.Wallace@cs.york.ac.uk] Sent: 29 September 2003 04:23 To: Simon Peyton-Jones Cc: Colin.Runciman(a)cs.york.ac.uk Subject: Re: FW: Library Infrastructure Proposal & Home Page "Simon Peyton-Jones" <simonpj(a)microsoft.com> writes: > I'd like to be confident that the package-description proposals that > Simon and I have put out isn't incompatible in some way with NHC. I have been thinking about your proposal for packages, and although there are many aspects of it that I like, and it isn't incompatible with nhc98 as such, my gut feeling is that one particular aspect of it is actually a very large semantic change/addition, and would probably involve a significant amount of work to implement. I mean this change: An entity in a Haskell program was previously uniquely identified by its (module name, identifier) pair, where the module name is the module in which the entity is defined. This now becomes a triple: (package name, relative module name, identifier) Consider how seemingly simple the current Haskell module system is. Yet dealing with the full implications of qualified imports, renamings, and hidings turned out to be pretty non-trivial when we implementors eventually started exercising all the corner cases. I know for a fact that Hugs and nhc98 still contain bugs in this area, and the subject even merited a Haskell Workshop paper in 2002! The newly proposed package system seems to be a mechanism that does a similar job to the existing module system, but at one level up. Instead of managing the /value/ namespace, It brings whole /modules/ into scope, and allows qualification, renamings, and hidings at the module layer. In other words, it is a rank-2 module system. I'm not confident that it will be any easier to implement, nor analytically tractable, than the current first-order module system. Should we be looking for some generalisation towards rank-n module aggregation? Or conversely, can we show that rank-2 modules (packages) can always be re-expressed (albeit unpleasantly) by simpler rank-1 modules? My feeling is that the latter might apply. For instance, where in your proposal you have chosen the new concept of "package name", I think it would almost be possible to substitute "top-level name in the library hierarchy". That is, instead of uniquely identifying a Haskell entity as the triple (gtkhs-0.5, Gtk.Window, widget), I think one could get away with the pair (GtkHS_0_5.Gtk.Window, widget) where the top-level hierarchy name is derived in some standard (if ugly) way from your package name. With this simple modification, it becomes possible to implement package "grafting" using solely the current module system. e.g. a wrapper module like module Graphics.UI.Gtk.Window (module GtkHS_0_5.Gtk.Window) where import GtkHS_0_5.Gtk.Window could be automatically generated for each grafting point. The only clever part of the package system would be to manage these auto-coded stubs, a far easier task than implementing rank-2 modules. (For instance a clever compiler could choose to generate "virtual" grafting module stubs on-demand, with no real existence in the filesystem, whilst a simpler compiler might just store them as real files.) > Closely related are Isaacs proposals re packaging and distribution > (below). I'm not sure there's been any NHC input into that discussion > so far, and again I'd like to be sure that we don't accidentally make > things hard for NHC. Issac has been in private contact a number of times w.r.t. his proposal. I don't see any particular nhc98-related problems with his distribution and management mechanisms. Of course it is implementation work that I may not have sufficient spare time to keep up with, but I don't foresee any in-principle difficulties ahead there. Regards, Malcolm

1 0

Seeking Partnership
by David Kalon 26 Sep '03

26 Sep '03

David Kalon. reply to : Davidkalon24(a)netscape.net Attention: Sir/Madam REQUEST FOR ASSISTANCE IN A FINANCIAL TRANSACTION I am interested in your partnership in business dealing. This business proposal I wish to intimate you with is of mutual benefit and it?s success is entirely based on mutual trust, cooperation and a high level of confidentiality as regard this transaction. I am representing the board of the contract award and monitoring committe of the Zambian Ministry of Mining and Resources. I am seeking your assistance to enable me transfer the sum of US$35,500,000.00 (Thirty-five Million, Five Hundred Thousand United States Dollars) into your private/company account. The fund came about as a result of a contract awarded and executed for and on behalf of my Ministry. The contract was supposed to be awarded to two foreign contractors to the tune of US$180,000,000.00 (One hundred and Eighty Million United States Dollars). But in the course of negotiation, the contract was awarded to a Bulgarian contractor at the cost of US$144,500,000.00 (One hundred and Forty-four Million, Five Hundred Thousand United States Dollars) to my benefit unknown to the contractor. This contract has been satisfactorily executed and inspected as the Bulgarian firm is presently securing payment from my Ministry, where our Board is in-charge of all foreign contract payment approval. As a civil servant still in active government service, I am forbidden by law to operate an account outside the shores of Zambia. Hence this message to you seeking your assistance so as to enable me present your private/company account details as a beneficiary of contractual claims alongside that of the Bulgarian contractor, to enable me transfer the difference of US$35,500,000.00 (Thirty-five Million, Five Hundred Thousand United States Dollars) into your provided account. On actualisation, the fund will be disbursed as stated below. 1. 20% of the fund will be for you as beneficiary 2. 80% of the fund will be for Us. All logistics are in place and all modalities worked out for a smooth actualisation of the transaction within the next few working days of commencement. For further details as to the workability of this transaction,please reach me as soon as possible for further clarification. Thank you and God bless as I await your urgent response. Yours Sincerely, David Kalon.

1 0

RE: Library Infrastructure Proposal & Home Page
by Simon Marlow 26 Sep '03

26 Sep '03

> This is true for the FreeBSD base system, but it is > recommended nowadays > to use sysutils/portupgrade for managing ports. With > portupgrade you can > simply do `portupgrade -rf changed_pkg` to recompile all packages > depending on the changed one. The only reason why portupgrade is not > part of the base system is that it is written in ruby and we are not > willing to include ruby in the base. Heh, I've been using portupgrade for a while but managed to miss that. I usually use it in '-Rr' mode. <rant> Nevetheless, this behaviour ought to be the default. I strongly believe in this principle: Using the standard package tools on a system, in their default configuration, it should be impossible to end up with an inconsistent set of installed packages. Sadly, on FreeBSD and Gentoo this is all too easy. FreeBSD even lets you install several different versions of a package on top of each other - so that when you delete one of them, it deletes some the files belonging to the other version too (only some, because it does the md5 checksum thing). This drives me nuts. At least Gentoo is better in this regard. And the portupgrade scripts try to work around the problems with the FreeBSD ports system by looking for inconsistencies in the database and applying heuristics to try to figure out how they should be fixed. This is crazy - it should be impossible to get the inconsistencies in the first place. </rant> Cheers, Simon

2 1

RE: RealWorld
by Simon Peyton-Jones 26 Sep '03

26 Sep '03

| The reason for having an MRef class was that there were two reference | types. If there's only one, but parameterized by region, you might as | well parameterize on the region instead, describing a class of monads | that contain a heap region: That sounds plausible. I certainly don't object to the change. Simon

1 0

Library Infrastructure Proposal & Home Page
by Isaac Jones 25 Sep '03

25 Sep '03

Followups to libraries(a)haskell.org please. Greetings, After many rounds of email and discussion, some wiki work, and discussion at the HIM, I've finally put together a proposal for the Library Infrastructure Project. I've inserted the introduction below. The draft proposal itself is available in a variety of formats. I've also started a web page for the project under the haskell.org site: http://www.haskell.org/libraryInfrastructure/ Please let me know if there's anything that needs to be added, if anything is confusing, etc. Comments and questions are welcome by email or on the libraries(a)haskell.org mailing list. peace, isaac ps. I expect to be moving a lot of stuff off of the wiki page since it is duplicated here. ------------------------------------------------------------ Library Infrastructure Project Proposal The Library Infrastructure Project is an effort to provide a framework for developers to more effectively contribute their software to the Haskell community. The Haskell Implementations[1] come included with a good set of standard libraries, but this set is constantly growing and is maintained centrally. This model does not scale up well, and as Haskell grows in acceptance, the quality and quantity of available libraries is becoming a major issue. There are a wide variety of Haskell Implementations (both compilers and interpreters), each of which target a variety of hardware architectures and operating systems. There are also a number of different groups and individuals providing libraries, and these libraries depend upon each other and they depend upon external systems such as Java or GTK. It can be very difficult for an end user to manage these dependencies and build all the necessary software at the correct version numbers on their platform: there is currently no generic build system to abstract away differences between Haskell Implementations and operating systems [2]. The Library Infrastructure Project seeks to provide some relief to this situation by building tools to assist developers, end users, and operating system distributers. This is a draft proposal. If you have comments, please email Isaac Jones. The latest version of this document should be available in a variety of formats from the Library Infrastructure Project home page. _________________________________________________________ Table of Contents 1. High-Level Overview 2. Issues Facing Developers 2.1. Issues Facing Packagers 2.2. Why We Should Solve This 3. A Solution for Haskell in Haskell 3.1. The Module Design 3.2. But Why Should We Use Haskell? 3.3. Setup.lhs Command-Line Interface 3.4. An Example Setup.lhs Script 4. Distribution Module 4.1. Distribution.Build 4.2. Distribution.Install 4.3. Distribution.Package 4.3.1. PackageConfig Data Structure 4.4. haskell-config Command-line interface 4.5. haskell-pkg? 4.6. Distribution.Config 5. Use Cases 6. Means of Distribution and Layered Tools 7. Development Plan A. Related Systems A.1. Debian A.2. Python Distutils A.3. CPAN and Boost A.4. FreeBSD's Ports System A.5. The XEmacs Packaging System A.6. Make-Based Systems A.7. hmake 1. High-Level Overview On a high level, we say that we want the system to help the user to install packages (whether they be libraries or applications) and their dependencies. To accomplish this, we propose a system similar to Python's Distutils (Section A.2) where each Haskell tool is distributed with a script (Section 3.4) which has a standard command-line interface. This script will provide a familiar user interface (Section 3.3) for compiling, installing, and removing packages and their dependencies. For instance, to install the HUnit package, the user might download the source code from the web site, change into the HUnit directory, and type ./Setup.lhs install default, which would build and install the HUnit package for the default compiler. Similarly, he might type ./Setup.lhs install nhc to install the package for NHC. Other tasks might be necessary to make the package known to the system, such as registering it with the Haskell Implementation of interest (Section 4.3). Such tasks would also be performed by this Setup program. FULL TEXT AT: http://www.haskell.org/libraryInfrastructure/proposal/index.html

4 5

RE: RealWorld
by Simon Peyton-Jones 25 Sep '03

25 Sep '03

| -----Original Message----- | From: Ross Paterson [mailto:ross@soi.city.ac.uk] | Sent: 18 September 2003 10:20 | To: Simon Peyton-Jones | Cc: Alastair Reid; Simon Marlow; libraries(a)haskell.org | Subject: Re: RealWorld | | On Thu, Sep 18, 2003 at 08:56:40AM +0100, Simon Peyton-Jones wrote: | > Is the proposal that the library design exposes to the programmer the | > following type synonyms: | > | > type IO a = ST IORegion a | > type IORef a = STRef IORegion a | | The proposal is to expose | | type IORef = STRef IORegion | type IOArray = STArray IORegion | type IOUArray = STUArray IORegion | | (modulo the name of IORegion) but to keep IO opaque. I think that while | IO includes state, it is not itself a state monad (because the "state" | there can be changed by other agents at the same time). Aha. I hadn't realised that you want to expose IORef as an STRef, but not the IO monad itself. I certainly don't object to that, but I'm not sure of all the implications. For example, at the moment one could have class MRef m r | m -> r, r -> m where new :: a -> m (r a) read :: r a -> m a write :: r a -> a -> m () instance MRef IO IORef where ... instance MRef (ST s) (STRef s) where ... I don't think that'll be legal if IORef = STRef IORegion, because the "r->m" dependency becomes ambiguous. So we'd have to lose the "r->m" dependency. Maybe that's ok. (I don't think MRef currently exists in the libraries.) But it's an example of a consequence of the choice you suggest. Simon | | > I think it is sound to do so. The advantage would be that newIORef and | > newSTRef would be the same (newRef, perhaps); ditto readRef, writeRef | > etc. | | If IO is opaque, then we have | | newIORef = stToIO . newSTRef | | etc. It's not quite as generic as your version, but it is a conceptual | simplification, and it does allow generic classes like | | class MonadST s m | m -> s where | liftST :: Control.Monad.ST.ST s a -> m a | | with instances for IO, both ST's and various constructed monads, so we | could have the likes of | | readSTRef' :: MonadST s m => STRef s a -> m a | readSTRef' r = liftST (Data.STRef.readSTRef r) | | My problem with exposing the GHC implementation of IO is that I don't | think it's the Truth, and Haskell has a record of punishing untruths | sooner or later. | | > I think this would be fine for GHC. It might not be fine for Hugs | > because they implement IO and ST differently I think. No idea about | > NHC. | | NHC doesn't have ST (rank-2 types). In Hugs, they are now | | newtype IO a = IO ((a -> IOResult) -> IOResult) | newtype ST s a = ST (forall r. (a -> r) -> r) | | primitive newIORef "newRef" :: a -> IO (IORef a) | primitive newSTRef "newRef" :: a -> ST s (STRef s a) | | But I'm not proposing that this be exposed. | | > Or, alternatively, is the question something to do with the internals of | > the library, of interest to implementers but not to users of the | > library? Or is it just to do with the type of stToIO, which should be | > | > stToIO :: ST IORegion a -> IO a | | It's more a matter of making IORegion work a bit harder since we have to | put it in the interface anyway.

2 1

RE: Network.recvFrom strangeness
by Simon Marlow 23 Sep '03

23 Sep '03

> The declaration of Network.recvFrom looks strange: > > recvFrom :: HostName -> PortID -> IO String > > I thought that recvFrom should _return_ the address of the peer, not > require it as an argument? At least that's what the corresponding > 4.4BSD routine does: Network.Socket.recvFrom is what you want, probably. Network.recvFrom is of dubious utility. Cheers, Simon

1 0