June 2001 - Libraries

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 27 Jun '01

27 Jun '01

> Simon Marlow writes: > : > | > 2) Whilst looking through the hierarchy, we decided that > using "Org" > | > was a bit unfortunate since it could lead to > silly-looking names > | > like Org.Org.Haskell.... and, worse, downright confusing names > | > like Org.Com.Galois. > | > > | > "Domain" was the best alternative we thought of - but it felt > | > a bit lame. > | > | Indeed, better suggestions are welcome. > > Contrib? > > That roughly resembles the way I've seen the /usr/contrib directory > used in some unixes, as a place for contributions which have not been > deemed canonical. Contrib sounds good to me. Simon PS. Malcolm, don't you dare suggest "Contributed" or "Contributions" :-)

1 0

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 26 Jun '01

26 Jun '01

First of all, many thanks to Alastair for typing up the notes from this gathering. I'll add that the latest version of the document is up, at: http://www.haskell.org/~simonmar/libraries/libraries.html and explains in more detail some of the points that Alastair mentions below. There are still lots of blank bits. Please send me any corrections or suggested text for inclusion. This document will be in the repo as soon as I get around to it. (my apologies to the Mathematics guys, I haven't updated the document w.r.t. your latest discussions yet). Quick status update on the core libraries: I've moved over all of GHC's std and lang packages to the new framework, and have the whole thing building and producing a useable Prelude. Object splitting doesn't work yet. I'll commit what I have later this week so folks can take a look (there are some other changes required to GHC to make it all work, I haven't decided how to commit these without breaking the compiler yet.). > 2) Whilst looking through the hierarchy, we decided that using "Org" > was a bit unfortunate since it could lead to silly-looking names > like Org.Org.Haskell.... and, worse, downright confusing names > like Org.Com.Galois. > > "Domain" was the best alternative we thought of - but it felt > a bit lame. Indeed, better suggestions are welcome. > 3) I lamented the fact that GHC's Exception type requires Dynamic > which, in turn, requires that GHC's Prelude contain Dynamic. > If Hugs were to follow suit, Hugs' Prelude would also have to > contain Dynamic - I don't especially like the way the Prelude > gradually sucks all functionality into itself. I got little > sympathy and I think Moore's law was mentioned. > (Next time I'm hacking on the exceptions, I'll probably > add Dynamic in there. Hugs hackers should feel free to add > it first.) Let's face it, at the moment you also need I/O "in the Prelude" so I'm not convinced by the argument that the Prelude is getting bloated. In the new library scheme, the Prelude simply re-exports lots of other stuff from the hierarchy, so in fact you need a large chunk of it anyway. I don't think Hugs' startup times will be an issue; if they are then I think the right thing to do is to slim down the Prelude itself. > 5) We talked a bit about versions. Two attitudes: > > 1) It is just silly to want to have two different versions of a > given library in the same program. We don't have the manpower > to cope with the complexity. Java doesn't try to tackle this. > > 2) I'll bet that's what the glibc folk thought too. > Do we have the manpower to keep the whole libraries (and stuff > not in the main repository) in lockstep? > > I recalled someone saying that when Java added their name > hierarchy, they had the opportunity to deal properly with > versioning but couldn't remember what had been suggested as the > Right Thing to do. I said I'd look around for anything reasonably > palatable that we could do. (First place to look is on this list > - any suggestions?) I think we agreed that it was necessary to place a version on the set of modules as a whole, so you can match library versions with compiler versions, and identify stable points and points when particular functionality was introduced or changed. I had intended to tie this in with the notion of stability - eg. minor version updates can't change the API of a stable library, see the document for the details. > 7) I abdicated from the position of Hugs Library Czar and Andy Gill > assumed the position. > > My resignation didn't go into effect until 7am today so I could > legitimately claim that I was the HLC on my CV :-) The document now mentions the core team and its purpose / responsibilities. > 8) Portability > > All the core libraries will be required to work on all compilers > (defined to mean GHC, NHC and Hugs). Actually, all the core modules marked as *portable* will be required to work on all compilers. All the other modules are optional, but are expected to work with a given compiler/platform combination if (a) the compiler supports any required extensions, (b) the module's dependencies are all satisfied, and (c) the module is compatible with the target platform. > Less portable libraries should be labelled (where?) in the module header, see the document. > with a list of > features that they require (or, looking at it another way, a list > of features that prevent it from being portable). Examples > include, MPTC, unboxed ints, concurrency, preemptive > concurrency, ... > > There seems to be some kind of connection here between: > > 1) What you put in a package description to cause both GHC and Hugs > to accept code which uses MPTC. > > 2) What #ifdef you put in the source code to handle code that needs > MPTC on compilers that lack it. > > 3) The list of non-portable features a library requires. > > 9) Responsibilities of people committing code > > We pondered requiring an extra layer of review before anyone is > allowed to commit to the libs designated as portable and stable. > (This is required in the OpenBSD kernel.) We saw the attraction > but I'm not sure that anyone wanted to actually do it. > > 10) Committing broken code to the CVS repository > > (Broken == not as portable as the lib claims to be, accidentally > reducing portability, breaking a library that depends on it. Doing > something to the code for someone else's compiler that they would > find intolerable if it happened to the version used with their > favourite compiler. A good approximation is anything that will > break the regression suite.) > > Is it ok to commit broken code? > If so, how long can it remain broken before someone has to fix it? > Who's responsibility is it to fix the code. Our lives will be a lot easier if we set up an infrastructure that lets you build all the libs and run tests for all the compilers with minimal hassle. With this in place, we can legimately claim that anyone who commits broken code deserves a red hot poker up the jacksie. Julian Seward's new testsuite stuff is coming along nicely, and should provide us with a good starting point for setting up the tests (see fptools/testsuite). > 11) Maintainer > > Every library will have a single maintainer who acts as a single > point of contact for the library. > > [But some of the discussion seemed to suggest that the maintainer > could be a group of people so maybe SimonM meant that there's a > single email address to write to???] Yup, that's about the size of it. The document makes it a bit clearer. > At this point we heard the mountains calling to us and felt obliged to > answer the call. I'm glad to report that no plants were harmed in the > process and that we gave the summits the respect they deserve by > strenuously avoiding any summit-like points on the trail. and I have the photos to prove it - hopefully available later today... Cheers, Simon

2 1

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 26 Jun '01

26 Jun '01

> > Marcin 'Qrczak' Kowalczyk <qrczak(a)knm.org.pl> writes: > > > Wouldn't putting domain names in the root of the hierarchy be OK? > > > The set of top-level domain names is very stable and there are no > > > clashes. > > > > There are an awful lot of them: one for each country in the > world. I > > don't think there's a country called "Prelude" but there probably is > > one with the domain name "IO". > > Maybe I'm missing something, what's wrong with putting the > standard stuff > under Org.Haskell? For example: > > Org.Haskell.Prelude > Org.Haskell.IO No problem, just that the names tend to get a little long. It makes sense to keep things as flat as possible unless there is a real reason not to do so. Cheers, Simon

2 1

Glasgow mafia meeting at PLDI
by Alastair David Reid 26 Jun '01

26 Jun '01

We got 4 Glaswegians (Andy Gill, Andy Moran, Simon Marlow and me) together in the same room at PLDI for a meeting about the new library scheme. The following is an attempt to summarise what was said. (I'm omitting a bunch of recapping what SimonM has already mailed to this list and I'm surely misrepresenting what was said - it's hard to keep it all straight after your third latte.) 1) SimonM explained the language change needed to support the module hierarchy. 2) Whilst looking through the hierarchy, we decided that using "Org" was a bit unfortunate since it could lead to silly-looking names like Org.Org.Haskell.... and, worse, downright confusing names like Org.Com.Galois. "Domain" was the best alternative we thought of - but it felt a bit lame. 3) I lamented the fact that GHC's Exception type requires Dynamic which, in turn, requires that GHC's Prelude contain Dynamic. If Hugs were to follow suit, Hugs' Prelude would also have to contain Dynamic - I don't especially like the way the Prelude gradually sucks all functionality into itself. I got little sympathy and I think Moore's law was mentioned. (Next time I'm hacking on the exceptions, I'll probably add Dynamic in there. Hugs hackers should feel free to add it first.) 4) SimonM showed us what Prelude.hs looks like in the new scheme: module Prelude( <huge list goes here> ) where #if defined GHC import GHC.Primops import GHC.WeirdWackyStuff import GHC.IO #elsif defined Hugs import Hugs.PreludeList import Hugs.<whatever> #elsif ... #endif Looks cool. Won't work on Hugs because Hugs insists that the Prelude be the first module loaded but under this scheme, the Prelude is only loaded after loading the Hugs.* stuff. Fortunately, I fixed this deficiency in STG-Hugs so all we have to do is copy the diffs over. IIRC, the changes to support this were in a separate commit so it should be easy. In my copious free time, I'll try to get this copied over. 5) We talked a bit about versions. Two attitudes: 1) It is just silly to want to have two different versions of a given library in the same program. We don't have the manpower to cope with the complexity. Java doesn't try to tackle this. 2) I'll bet that's what the glibc folk thought too. Do we have the manpower to keep the whole libraries (and stuff not in the main repository) in lockstep? I recalled someone saying that when Java added their name hierarchy, they had the opportunity to deal properly with versioning but couldn't remember what had been suggested as the Right Thing to do. I said I'd look around for anything reasonably palatable that we could do. (First place to look is on this list - any suggestions?) 6) GHC's deprecated pragma is cool and would probably be easy to add to Hugs. Maybe Johan could be persuaded? 7) I abdicated from the position of Hugs Library Czar and Andy Gill assumed the position. My resignation didn't go into effect until 7am today so I could legitimately claim that I was the HLC on my CV :-) 8) Portability All the core libraries will be required to work on all compilers (defined to mean GHC, NHC and Hugs). Less portable libraries should be labelled (where?) with a list of features that they require (or, looking at it another way, a list of features that prevent it from being portable). Examples include, MPTC, unboxed ints, concurrency, preemptive concurrency, ... There seems to be some kind of connection here between: 1) What you put in a package description to cause both GHC and Hugs to accept code which uses MPTC. 2) What #ifdef you put in the source code to handle code that needs MPTC on compilers that lack it. 3) The list of non-portable features a library requires. 9) Responsibilities of people committing code We pondered requiring an extra layer of review before anyone is allowed to commit to the libs designated as portable and stable. (This is required in the OpenBSD kernel.) We saw the attraction but I'm not sure that anyone wanted to actually do it. 10) Committing broken code to the CVS repository (Broken == not as portable as the lib claims to be, accidentally reducing portability, breaking a library that depends on it. Doing something to the code for someone else's compiler that they would find intolerable if it happened to the version used with their favourite compiler. A good approximation is anything that will break the regression suite.) Is it ok to commit broken code? If so, how long can it remain broken before someone has to fix it? Who's responsibility is it to fix the code. I think the mood was "it's not ok" and "fixing should take priority over everything else" and is the responsibility of the person who committed it. We talked somewhat aimlessly about (the usual idea of) setting up a machine whose only job is to keep trying to compile all the code and run all the tests. Andy Gill suggested two useful ideas: 1) Define clusters of tests - some of which are so cheap to try that you can reasonably be expected to try them before doing a commit. (At least with Hugs and NHC which have more reasonable build times than GHC and all its little ways.) They would then get increasingly expensive until you reach the ones that are so expensive that you have to do round-robin scheduling between clusters "it's Tuesday so I'd better try bootstrapping GHC with itself on the ZX81 port". 2) Every time the test machine successfully runs the test suite, it should tag all the files with the tag "last_stable_version". Those who find themselves able to resist the bleeding edge would tend to use this. [As far as I can tell, this approach will make it safe to do a checkout anytime you like. Unfortunately, it still won't be possible to commit into the head until the head is made stable again - so it won't eliminate the need to be careful not to break the head. - ADR] 11) Maintainer Every library will have a single maintainer who acts as a single point of contact for the library. [But some of the discussion seemed to suggest that the maintainer could be a group of people so maybe SimonM meant that there's a single email address to write to???] 12) Licenses I think we decided that the essence of the BSD/LGPL debate was: o BSD fans think that "contamination" is bad o LGPL fans think that "contamination" is good This is basically a religious argument on which no agreement is likely to be reached by zealots on either side. As far as I [ADR] can tell, the debate about just how the LGPL applies to Haskell is a red-herring. Even if we understood it and agreed it was reasonably specified and appropriate in a Haskell context, BSD fans would still not want LGPL. (That's certainly my own feeling.) Or, to put it another way, we (had already?) agreed to disagree and the repository will be partitioned into 2 separate repositories and packages will contain either pure BSD code or pure LGPL code. [The Galois people (Andy and Andy) actually work at one of the few places that really does have significant intellectual property written in Haskell. I won't attempt to summarise their position on the license issue or their argument but would encourage them to do so. Incidentally, it seems that Andy Gill hasn't been getting library email for the last few weeks - which explains why they've been so quiet.] At this point we heard the mountains calling to us and felt obliged to answer the call. I'm glad to report that no plants were harmed in the process and that we gave the summits the respect they deserve by strenuously avoiding any summit-like points on the trail. -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/

2 3

RE: Glasgow mafia meeting at PLDI
by Simon Marlow 26 Jun '01

26 Jun '01

> (2) One point we discussed was missed: > > In Haskell you sometimes publish *two* interfaces to your libraries; > one for users, and one for super-users than might want to > extend things. > Typically the super-users can see passed some of the user library > abstractions. > > If you have a public library (say Html), and its called > Text.Html (say), > then you'll use a naming convention for the super-user library. > The best we could come up with was *.Internals > > So: > import Text.Html -- The library > import Text.Html.Internals -- The super library (for building > other libs) > > (Comments, anyone?) Thanks, I've added this to the document. Cheers, Simon

1 0

Re: Glasgow mafia meeting at PLDI
by Marcin 'Qrczak' Kowalczyk 25 Jun '01

25 Jun '01

22 Jun 2001 19:56:15 -0600, Alastair David Reid <reid(a)cs.utah.edu> pisze: > 2) Whilst looking through the hierarchy, we decided that using "Org" > was a bit unfortunate since it could lead to silly-looking names > like Org.Org.Haskell.... and, worse, downright confusing names > like Org.Com.Galois. Wouldn't putting domain names in the root of the hierarchy be OK? The set of top-level domain names is very stable and there are no clashes. -- __("< Marcin Kowalczyk * qrczak(a)knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK

4 4

Re: Glasgow mafia meeting at PLDI
by Ashley Yakeley 24 Jun '01

24 Jun '01

At 2001-06-24 13:55, Alastair David Reid wrote: >We briefly discussed this but it was felt that it was useful to >distinguish between official company libraries and private libraries >of those who happen to work at companies. This was felt most strongly >by those who work for companies than those working at universities. Of course, this assumes that companies will not try to exert control over 'User.Com.SomeCorp.*'... -- Ashley Yakeley, Seattle WA

1 0

RE: Glasgow mafia meeting at PLDI
by Andy Gill 22 Jun '01

22 Jun '01

Alastair Reid said: We got 4 Glaswegians (Andy Gill, Andy Moran, Simon Marlow and me) together in the same room at PLDI for a meeting about the new library scheme. [...] 7) I abdicated from the position of Hugs Library Czar and Andy Gill assumed the position. My resignation didn't go into effect until 7am today so I could legitimately claim that I was the HLC on my CV :-) [...] [The Galois people (Andy and Andy) actually work at one of the few places that really does have significant intellectual property written in Haskell. I won't attempt to summarise their position on the license issue or their argument but would encourage them to do so. Incidentally, it seems that Andy Gill hasn't been getting library email for the last few weeks - which explains why they've been so quiet.] Andy Gill replies: (1) I've re-registered for this mailing list, sorry about not being around! [We suspect that the galois mailer went down overnight at one point, which is peak europe talking time, and a sudden flood of messages triggered an automatic name deletion, when too many messages bounce!] (2) One point we discussed was missed: In Haskell you sometimes publish *two* interfaces to your libraries; one for users, and one for super-users than might want to extend things. Typically the super-users can see passed some of the user library abstractions. If you have a public library (say Html), and its called Text.Html (say), then you'll use a naming convention for the super-user library. The best we could come up with was *.Internals So: import Text.Html -- The library import Text.Html.Internals -- The super library (for building other libs) (Comments, anyone?) (3) I'll mail out a response to the GPL/LGPL/BSD mailings as soon as I've digested the last few (100) messages! In short, though, we (Galois) would strongly prefer the library to be BSD only. Andy Gill Galois Connections

1 0

RE: The next step
by Simon Marlow 03 Jun '01

03 Jun '01

> > Maybe the right thing to do is to make the choice of license a > > fundamental part of the hierarchy? That is, we could have: > > > > Core -- all under BSD license > > Core.Control > > Core.Numeric > > Core.User > > Core.Org > > ... > > > > LGPL -- all under LGPL license > > LGPL.Control > > LGPL.Numeric > > LGPL.User > > LGPL.Org > > ... > > > > GPL -- all under GPL license > > GPL.Control > > GPL.Numeric > > GPL.User > > GPL.Org > > ... > > > > And then we can impose restrictions like "Core can only import other > > Core libraries", "LGPL can import Core and LGPL but not GPL", etc. > > Doesn't look very attractive to me. Me neither. The license is meta-information about the source and should be entirely separate, so the author can change the license at will without breaking programs. Besides, it breaks our convention about organising the hierarchy by functionality. A better way IMO would be to separate licenses at the package level, so that within a package all the code has the same license. This gives a good separation between code with different licenses, while still allowing code which uses libraries from multiple sources to be written with minimal hassle. The restrictions on dependencies between code with different licenses are probably important though. Cheers, Simon

2 1

RE: The next step
by Simon Marlow 03 Jun '01

03 Jun '01

> > I agree with what you're saying, but with cross-module optimisation > > turned on in GHC, virtually any non-trivial change to the > implementation > > of a module causes a change in the .hi file. So *unless* > you turn off > > cross-module optimisation between the application and the > library, you > > have vitually no hope of ever successfully re-linking the > application > > with a modified version of the library. That would be in > violation of > > the license, no? > > If I am not mistaken (anything else wouldn't make much > sense), what the license says is that only if the definition > files (= .hi) are not changed, can you reasonably expect > that re-linking works. This means, in Haskell, you can make > very little change to the library. That's bad for the user > and good for the vendor of the proprietary code if you like. This is indeed what the license says if taken literally(*). But it clearly goes against the stated intention of the license, which is to allow re-linking of the application to a modified library. If this interpretation is correct, then with GHC you can conform to the license while still not providing adequate means by which someone can re-link the program to a modified library. This smells like a loophole, and one I expect the FSF would like to clarify. (*) if we take "definitions files" to equate to interface files in Haskell, which is still an interpretation because they are quite different beasts. > I am quite sure that's that the interpretation, because > otherwise it would mean that I can change a #define in a > header of a lib and expect to relink successfully, which > clearly can't be. > > This is something, I would also be happy to ask the FSF > about if it helps. please do, and feel free to forward this discussion to them. Cheers, Simon

1 0