- Libraries - Haskell.org

RE: ghc_extra_opts in package
by Simon Marlow 28 Jun '01

28 Jun '01

> > Another issue is that it turns out to be hard to enable certain > > extensions without enabling others, because it requires you to pass > > around more state in the compiler, and in particular in the parser > > (the number of different languages being parsed increases), which is > > why we lump all the extensions together in GHC. However, I agree > > the situation isn't ideal. > > So here's a concrete suggestion: > > 1) We standardise the definition of various extensions (mptc, > unboxed types, > etc.) so that if two compilers both claim to support > feature X, then > they accept the same code. (We're pretty close to this already.) Definitely - this has been a goal for a while. I think there was a plan to have a section on www.haskell.org listing all the extensions with their specifications. > 2) We extend the package definition with a list of features > used in the > package. > > Package { > name = "net", > extensions = [ "FFI", "ExceptionHandling", ... ], > ... > } Let's be clear: these extensions are those required by the *consumer* of the package, right? The implementation of the package will likely require a different (probably super-) set of extensions. For example: "FFI" isn't typically required by the consumer of the "net" package, but it is certainly used in the implementation. Also, this doesn't mesh well with the portability notion in the new libraries proposal. The idea there is that certain libraries would be optional depending on whether the compiler implemented certain extensions - with your scheme this would have to be done at the package level. I wanted to lump all the core libraries into a single package on GHC, but this would mean that package "core" for GHC would require a different set of extensions than the same package for NHC. Cheers, Simon

2 1

FW: cvs commit: fptools/libraries/core Foreign.hs Main.hi-boot Makefile Prelude.hs fptools/libraries/core/Control Concurrent.hs Exception.hs Monad.hs Parallel.hs fptools/libraries/core/Control/Concurrent CVar.hs Chan.hs MVar.hs ...
by Simon Marlow 28 Jun '01

28 Jun '01

For those not on the GHC CVS mailing list, I've just committed a first cut of the new libraries. You can checkout the tree from GHC's CVS repository by saying 'cvs checkout libraries' (at least after the anonymous mirror has happened, within 24 hours). Cheers, Simon -----Original Message----- From: Simon Marlow [mailto:simonmar@glass.cse.ogi.edu] Sent: Thursday, June 28, 2001 3:15 PM To: cvs-fptools(a)haskell.org Subject: cvs commit: fptools/libraries/core Foreign.hs Main.hi-boot Makefile Prelude.hs fptools/libraries/core/Control Concurrent.hs Exception.hs Monad.hs Parallel.hs fptools/libraries/core/Control/Concurrent CVar.hs Chan.hs MVar.hs ... simonmar 2001/06/28 07:15:04 PDT Added files: libraries/core Foreign.hs Main.hi-boot Makefile Prelude.hs libraries/core/Control Concurrent.hs Exception.hs Monad.hs Parallel.hs libraries/core/Control/Concurrent CVar.hs Chan.hs MVar.hs QSem.hs QSemN.hs SampleVar.hs libraries/core/Control/Monad Cont.hs Error.hs Fix.hs Identity.hs List.hs Monoid.hs RWS.hs Reader.hs ST.hs State.hs Trans.hs Writer.hs libraries/core/Control/Monad/ST Lazy.hs Strict.hs libraries/core/Control/Parallel Strategies.hs libraries/core/Data Array.hs Bits.hs Bool.hs Char.hs Complex.hs Dynamic.hs Either.hs IORef.hs Int.hs Ix.hs List.hs Maybe.hs PackedString.hs Ratio.hs STRef.hs Word.hs libraries/core/Data/Array Base.hs IArray.hs IO.hs MArray.hs ST.hs Unboxed.hs libraries/core/Debug Trace.hs libraries/core/Foreign C.hs ForeignPtr.hs Ptr.hs StablePtr.hs Storable.hs libraries/core/Foreign/C Error.hs String.hs Types.hs TypesISO.hs libraries/core/Foreign/Marshal Alloc.hs Array.hs Error.hs Utils.hs libraries/core/GHC Arr.lhs Base.lhs ByteArr.lhs Conc.lhs Dynamic.lhs Enum.lhs Err.hi-boot Err.lhs Exception.lhs Float.lhs Handle.hsc Handle.hsc.orig IO.hsc IOBase.lhs Int.lhs List.lhs Main.lhs Maybe.lhs Num.hi-boot Num.lhs Pack.lhs Posix.hsc Prim.hi-boot Ptr.lhs Read.lhs Real.lhs ST.lhs STRef.lhs Show.lhs Stable.lhs Storable.lhs TopHandler.lhs Tup.lhs Weak.lhs Word.lhs libraries/core/System CPUTime.hsc Cmd.hsc Environment.hs Exit.hs IO.hs Info.hs Locale.hs Random.hs Time.hsc libraries/core/System/IO Directory.hsc Unsafe.hs libraries/core/System/Mem StableName.hs Weak.hs libraries/core/Text Read.hs Show.hs libraries/core/Text/Show Functions.hs libraries/core/cbits Makefile errno.c inputReady.c lockFile.c system.c writeError.c libraries/core/doc libraries.sgml libraries/core/include CTypes.h Dynamic.h HsCore.h PackedString.h ghc_errno.h lockFile.h Log: First cut of the Haskell Core Libraries ======================================= NOTE: it's not meant to be a working snapshot. The code is just here to look at and so the NHC/Hugs guys can start playing around with it. There is no build system. For GHC, the libraries tree is intended to be grafted onto an existing fptools/ tree, and the Makefile in libraries/core is a quick hack for that setup. This won't work at the moment without the other changes needed in fptools/ghc, which I haven't committed because they'll cause breakage. However, with the changes required these sources build a working Prelude and libraries. The layout mostly follows the one we agreed on, with one or two minor changes; in particular the Data/Array layout probably isn't final (there are several choices here). The document is in libraries/core/doc as promised. The cbits stuff is just a copy of ghc/lib/std/cbits and has GHC-specific stuff in it. We should really separate the compiler-specific C support from any compiler-independent C support there might be. Don't pay too much attention to the portability or stability status indicated in the header of each source file at the moment - I haven't gone through to make sure they're all consistent and make sense. I'm using non-literate source outside of GHC/. Hope that's ok with everyone. We need to discuss how the build system is going to work... _______________________________________________ Cvs-fptools mailing list Cvs-fptools(a)haskell.org http://www.haskell.org/mailman/listinfo/cvs-fptools

2 1

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 28 Jun '01

28 Jun '01

> 24 Jun 2001 14:50:16 -0600, Alastair David Reid > <reid(a)cs.utah.edu> pisze: > > > There are an awful lot of them: one for each country in the world. > > I don't think there's a country called "Prelude" but there probably > > is one with the domain name "IO". > > There is none. I don't believe in any clash if we just mix the domain > hierarchy with the functionality hierarchy. The point is we don't have any control over the DNS namespace, so even if it doesn't clash with our hierarchy now, it might in the future. Cheers, Simon

1 0

Re: ghc_extra_opts in package
by Alastair David Reid 27 Jun '01

27 Jun '01

[Discussion moves to libraries list] Simon Marlow: > Arguably there ought to be some other indication in the source code > of what non-standard features a module requires - the closest we > come to this at the moment is that GHC allows you to say > {-# OPTIONS -fglasgow-exts #-} > to enable GHC extensions in the source. If we went in this direction, we'd need to make it compiler independent and finer grained. I also feel uncomfortable putting it in the source file but I can't quite put my finger on why. > Another issue is that it turns out to be hard to enable certain > extensions without enabling others, because it requires you to pass > around more state in the compiler, and in particular in the parser > (the number of different languages being parsed increases), which is > why we lump all the extensions together in GHC. However, I agree > the situation isn't ideal. So here's a concrete suggestion: 1) We standardise the definition of various extensions (mptc, unboxed types, etc.) so that if two compilers both claim to support feature X, then they accept the same code. (We're pretty close to this already.) Note that standardising the definition of mptc (for example) isn't the same as requiring all Haskell compilers to support it. Here's a list of proposed standards (suggestions for extra features welcome): MPTC FunctionalDependencies PatternGuards UnboxedTypes TREX PreemptiveConcurrency CooperativeConcurrency ImplicitParameters ExceptionHandling FFI 2) We extend the package definition with a list of features used in the package. Package { name = "net", extensions = [ "FFI", "ExceptionHandling", ... ], ... } 3) Package managers translate these standard feature names into a set of compiler-specific flags to pass to the compiler. It would be considered ok for a compiler to enable other extensions at the same time or to have some extensions permanently enabled. (It would also be good if at least one of the compilers supporting each feature generated a warning if you used a feature not mentioned in the package spec. This check might occur while checking static semantics, typechecking, etc. (where it is easy to be selective) instead of being enforced in the parser. -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/

3 2

Re: Glasgow mafia meeting at PLDI
by Ashley Yakeley 27 Jun '01

27 Jun '01

At 2001-06-27 12:39, Marcin 'Qrczak' Kowalczyk wrote: >24 Jun 2001 14:50:16 -0600, Alastair David Reid <reid(a)cs.utah.edu> pisze: > >> There are an awful lot of them: one for each country in the world. >> I don't think there's a country called "Prelude" but there probably >> is one with the domain name "IO". > >There is none. 'IO' is the data code for British Indian Ocean Territory, I believe. <http://www.odci.gov/cia/publications/factbook/geos/io.html>. In any case the domain is registered, see <http://www.io.io/>. -- Ashley Yakeley, Seattle WA

1 0

Re: Glasgow mafia meeting at PLDI
by Marcin 'Qrczak' Kowalczyk 27 Jun '01

27 Jun '01

24 Jun 2001 14:50:16 -0600, Alastair David Reid <reid(a)cs.utah.edu> pisze: > There are an awful lot of them: one for each country in the world. > I don't think there's a country called "Prelude" but there probably > is one with the domain name "IO". There is none. I don't believe in any clash if we just mix the domain hierarchy with the functionality hierarchy. -- __("< Marcin Kowalczyk * qrczak(a)knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK

1 0

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 27 Jun '01

27 Jun '01

> Simon Marlow writes: > : > | > 2) Whilst looking through the hierarchy, we decided that > using "Org" > | > was a bit unfortunate since it could lead to > silly-looking names > | > like Org.Org.Haskell.... and, worse, downright confusing names > | > like Org.Com.Galois. > | > > | > "Domain" was the best alternative we thought of - but it felt > | > a bit lame. > | > | Indeed, better suggestions are welcome. > > Contrib? > > That roughly resembles the way I've seen the /usr/contrib directory > used in some unixes, as a place for contributions which have not been > deemed canonical. Contrib sounds good to me. Simon PS. Malcolm, don't you dare suggest "Contributed" or "Contributions" :-)

1 0

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 26 Jun '01

26 Jun '01

First of all, many thanks to Alastair for typing up the notes from this gathering. I'll add that the latest version of the document is up, at: http://www.haskell.org/~simonmar/libraries/libraries.html and explains in more detail some of the points that Alastair mentions below. There are still lots of blank bits. Please send me any corrections or suggested text for inclusion. This document will be in the repo as soon as I get around to it. (my apologies to the Mathematics guys, I haven't updated the document w.r.t. your latest discussions yet). Quick status update on the core libraries: I've moved over all of GHC's std and lang packages to the new framework, and have the whole thing building and producing a useable Prelude. Object splitting doesn't work yet. I'll commit what I have later this week so folks can take a look (there are some other changes required to GHC to make it all work, I haven't decided how to commit these without breaking the compiler yet.). > 2) Whilst looking through the hierarchy, we decided that using "Org" > was a bit unfortunate since it could lead to silly-looking names > like Org.Org.Haskell.... and, worse, downright confusing names > like Org.Com.Galois. > > "Domain" was the best alternative we thought of - but it felt > a bit lame. Indeed, better suggestions are welcome. > 3) I lamented the fact that GHC's Exception type requires Dynamic > which, in turn, requires that GHC's Prelude contain Dynamic. > If Hugs were to follow suit, Hugs' Prelude would also have to > contain Dynamic - I don't especially like the way the Prelude > gradually sucks all functionality into itself. I got little > sympathy and I think Moore's law was mentioned. > (Next time I'm hacking on the exceptions, I'll probably > add Dynamic in there. Hugs hackers should feel free to add > it first.) Let's face it, at the moment you also need I/O "in the Prelude" so I'm not convinced by the argument that the Prelude is getting bloated. In the new library scheme, the Prelude simply re-exports lots of other stuff from the hierarchy, so in fact you need a large chunk of it anyway. I don't think Hugs' startup times will be an issue; if they are then I think the right thing to do is to slim down the Prelude itself. > 5) We talked a bit about versions. Two attitudes: > > 1) It is just silly to want to have two different versions of a > given library in the same program. We don't have the manpower > to cope with the complexity. Java doesn't try to tackle this. > > 2) I'll bet that's what the glibc folk thought too. > Do we have the manpower to keep the whole libraries (and stuff > not in the main repository) in lockstep? > > I recalled someone saying that when Java added their name > hierarchy, they had the opportunity to deal properly with > versioning but couldn't remember what had been suggested as the > Right Thing to do. I said I'd look around for anything reasonably > palatable that we could do. (First place to look is on this list > - any suggestions?) I think we agreed that it was necessary to place a version on the set of modules as a whole, so you can match library versions with compiler versions, and identify stable points and points when particular functionality was introduced or changed. I had intended to tie this in with the notion of stability - eg. minor version updates can't change the API of a stable library, see the document for the details. > 7) I abdicated from the position of Hugs Library Czar and Andy Gill > assumed the position. > > My resignation didn't go into effect until 7am today so I could > legitimately claim that I was the HLC on my CV :-) The document now mentions the core team and its purpose / responsibilities. > 8) Portability > > All the core libraries will be required to work on all compilers > (defined to mean GHC, NHC and Hugs). Actually, all the core modules marked as *portable* will be required to work on all compilers. All the other modules are optional, but are expected to work with a given compiler/platform combination if (a) the compiler supports any required extensions, (b) the module's dependencies are all satisfied, and (c) the module is compatible with the target platform. > Less portable libraries should be labelled (where?) in the module header, see the document. > with a list of > features that they require (or, looking at it another way, a list > of features that prevent it from being portable). Examples > include, MPTC, unboxed ints, concurrency, preemptive > concurrency, ... > > There seems to be some kind of connection here between: > > 1) What you put in a package description to cause both GHC and Hugs > to accept code which uses MPTC. > > 2) What #ifdef you put in the source code to handle code that needs > MPTC on compilers that lack it. > > 3) The list of non-portable features a library requires. > > 9) Responsibilities of people committing code > > We pondered requiring an extra layer of review before anyone is > allowed to commit to the libs designated as portable and stable. > (This is required in the OpenBSD kernel.) We saw the attraction > but I'm not sure that anyone wanted to actually do it. > > 10) Committing broken code to the CVS repository > > (Broken == not as portable as the lib claims to be, accidentally > reducing portability, breaking a library that depends on it. Doing > something to the code for someone else's compiler that they would > find intolerable if it happened to the version used with their > favourite compiler. A good approximation is anything that will > break the regression suite.) > > Is it ok to commit broken code? > If so, how long can it remain broken before someone has to fix it? > Who's responsibility is it to fix the code. Our lives will be a lot easier if we set up an infrastructure that lets you build all the libs and run tests for all the compilers with minimal hassle. With this in place, we can legimately claim that anyone who commits broken code deserves a red hot poker up the jacksie. Julian Seward's new testsuite stuff is coming along nicely, and should provide us with a good starting point for setting up the tests (see fptools/testsuite). > 11) Maintainer > > Every library will have a single maintainer who acts as a single > point of contact for the library. > > [But some of the discussion seemed to suggest that the maintainer > could be a group of people so maybe SimonM meant that there's a > single email address to write to???] Yup, that's about the size of it. The document makes it a bit clearer. > At this point we heard the mountains calling to us and felt obliged to > answer the call. I'm glad to report that no plants were harmed in the > process and that we gave the summits the respect they deserve by > strenuously avoiding any summit-like points on the trail. and I have the photos to prove it - hopefully available later today... Cheers, Simon

2 1

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 26 Jun '01

26 Jun '01

> > Marcin 'Qrczak' Kowalczyk <qrczak(a)knm.org.pl> writes: > > > Wouldn't putting domain names in the root of the hierarchy be OK? > > > The set of top-level domain names is very stable and there are no > > > clashes. > > > > There are an awful lot of them: one for each country in the > world. I > > don't think there's a country called "Prelude" but there probably is > > one with the domain name "IO". > > Maybe I'm missing something, what's wrong with putting the > standard stuff > under Org.Haskell? For example: > > Org.Haskell.Prelude > Org.Haskell.IO No problem, just that the names tend to get a little long. It makes sense to keep things as flat as possible unless there is a real reason not to do so. Cheers, Simon

2 1

Glasgow mafia meeting at PLDI
by Alastair David Reid 26 Jun '01

26 Jun '01

We got 4 Glaswegians (Andy Gill, Andy Moran, Simon Marlow and me) together in the same room at PLDI for a meeting about the new library scheme. The following is an attempt to summarise what was said. (I'm omitting a bunch of recapping what SimonM has already mailed to this list and I'm surely misrepresenting what was said - it's hard to keep it all straight after your third latte.) 1) SimonM explained the language change needed to support the module hierarchy. 2) Whilst looking through the hierarchy, we decided that using "Org" was a bit unfortunate since it could lead to silly-looking names like Org.Org.Haskell.... and, worse, downright confusing names like Org.Com.Galois. "Domain" was the best alternative we thought of - but it felt a bit lame. 3) I lamented the fact that GHC's Exception type requires Dynamic which, in turn, requires that GHC's Prelude contain Dynamic. If Hugs were to follow suit, Hugs' Prelude would also have to contain Dynamic - I don't especially like the way the Prelude gradually sucks all functionality into itself. I got little sympathy and I think Moore's law was mentioned. (Next time I'm hacking on the exceptions, I'll probably add Dynamic in there. Hugs hackers should feel free to add it first.) 4) SimonM showed us what Prelude.hs looks like in the new scheme: module Prelude( <huge list goes here> ) where #if defined GHC import GHC.Primops import GHC.WeirdWackyStuff import GHC.IO #elsif defined Hugs import Hugs.PreludeList import Hugs.<whatever> #elsif ... #endif Looks cool. Won't work on Hugs because Hugs insists that the Prelude be the first module loaded but under this scheme, the Prelude is only loaded after loading the Hugs.* stuff. Fortunately, I fixed this deficiency in STG-Hugs so all we have to do is copy the diffs over. IIRC, the changes to support this were in a separate commit so it should be easy. In my copious free time, I'll try to get this copied over. 5) We talked a bit about versions. Two attitudes: 1) It is just silly to want to have two different versions of a given library in the same program. We don't have the manpower to cope with the complexity. Java doesn't try to tackle this. 2) I'll bet that's what the glibc folk thought too. Do we have the manpower to keep the whole libraries (and stuff not in the main repository) in lockstep? I recalled someone saying that when Java added their name hierarchy, they had the opportunity to deal properly with versioning but couldn't remember what had been suggested as the Right Thing to do. I said I'd look around for anything reasonably palatable that we could do. (First place to look is on this list - any suggestions?) 6) GHC's deprecated pragma is cool and would probably be easy to add to Hugs. Maybe Johan could be persuaded? 7) I abdicated from the position of Hugs Library Czar and Andy Gill assumed the position. My resignation didn't go into effect until 7am today so I could legitimately claim that I was the HLC on my CV :-) 8) Portability All the core libraries will be required to work on all compilers (defined to mean GHC, NHC and Hugs). Less portable libraries should be labelled (where?) with a list of features that they require (or, looking at it another way, a list of features that prevent it from being portable). Examples include, MPTC, unboxed ints, concurrency, preemptive concurrency, ... There seems to be some kind of connection here between: 1) What you put in a package description to cause both GHC and Hugs to accept code which uses MPTC. 2) What #ifdef you put in the source code to handle code that needs MPTC on compilers that lack it. 3) The list of non-portable features a library requires. 9) Responsibilities of people committing code We pondered requiring an extra layer of review before anyone is allowed to commit to the libs designated as portable and stable. (This is required in the OpenBSD kernel.) We saw the attraction but I'm not sure that anyone wanted to actually do it. 10) Committing broken code to the CVS repository (Broken == not as portable as the lib claims to be, accidentally reducing portability, breaking a library that depends on it. Doing something to the code for someone else's compiler that they would find intolerable if it happened to the version used with their favourite compiler. A good approximation is anything that will break the regression suite.) Is it ok to commit broken code? If so, how long can it remain broken before someone has to fix it? Who's responsibility is it to fix the code. I think the mood was "it's not ok" and "fixing should take priority over everything else" and is the responsibility of the person who committed it. We talked somewhat aimlessly about (the usual idea of) setting up a machine whose only job is to keep trying to compile all the code and run all the tests. Andy Gill suggested two useful ideas: 1) Define clusters of tests - some of which are so cheap to try that you can reasonably be expected to try them before doing a commit. (At least with Hugs and NHC which have more reasonable build times than GHC and all its little ways.) They would then get increasingly expensive until you reach the ones that are so expensive that you have to do round-robin scheduling between clusters "it's Tuesday so I'd better try bootstrapping GHC with itself on the ZX81 port". 2) Every time the test machine successfully runs the test suite, it should tag all the files with the tag "last_stable_version". Those who find themselves able to resist the bleeding edge would tend to use this. [As far as I can tell, this approach will make it safe to do a checkout anytime you like. Unfortunately, it still won't be possible to commit into the head until the head is made stable again - so it won't eliminate the need to be careful not to break the head. - ADR] 11) Maintainer Every library will have a single maintainer who acts as a single point of contact for the library. [But some of the discussion seemed to suggest that the maintainer could be a group of people so maybe SimonM meant that there's a single email address to write to???] 12) Licenses I think we decided that the essence of the BSD/LGPL debate was: o BSD fans think that "contamination" is bad o LGPL fans think that "contamination" is good This is basically a religious argument on which no agreement is likely to be reached by zealots on either side. As far as I [ADR] can tell, the debate about just how the LGPL applies to Haskell is a red-herring. Even if we understood it and agreed it was reasonably specified and appropriate in a Haskell context, BSD fans would still not want LGPL. (That's certainly my own feeling.) Or, to put it another way, we (had already?) agreed to disagree and the repository will be partitioned into 2 separate repositories and packages will contain either pure BSD code or pure LGPL code. [The Galois people (Andy and Andy) actually work at one of the few places that really does have significant intellectual property written in Haskell. I won't attempt to summarise their position on the license issue or their argument but would encourage them to do so. Incidentally, it seems that Andy Gill hasn't been getting library email for the last few weeks - which explains why they've been so quiet.] At this point we heard the mountains calling to us and felt obliged to answer the call. I'm glad to report that no plants were harmed in the process and that we gave the summits the respect they deserve by strenuously avoiding any summit-like points on the trail. -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/

2 3