March 2001 - Libraries

Re: Process issues (was Re: Lang)
by Ashley Yakeley 13 Mar '01

13 Mar '01

At 2001-03-13 09:04, Dylan Thurston wrote: >On Tue, Mar 13, 2001 at 03:18:32PM +0000, Malcolm Wallace wrote: >> Ok, I acknowledge that. I think we seem to be gradually coming to >> a consensus on Marcin's point - that the functionality of a library >> should be paramount in the naming scheme, regardless of implementation, >> standardness, portability, etc. This is good. > >With the exception of the Prelude, right? Is a Prelude even necessary? Might it not be better to import what you want explicitly? -- Ashley Yakeley, Seattle WA

1 0

RE: Summary of progress
by Simon Marlow 13 Mar '01

13 Mar '01

> Johannes Waldmann wrote: > > > > > Such Foo.Bar module is not defined explicitly, but results from > > > having a subdirectory in import directories. Contents of > all Foo.Bar > > > directories from all import directory roots are merged to > form such > > > pseudo-module which contains nothing but modules. > > > > Yes, very good. That's what I wanted. Do the implementors aggree? > > Personally, I don't like the idea at all. :-) I don't see the need > to bring whole namespaces into scope. And even if I did, I wouldn't > overload the standard import syntax to mean namespace scoping > as well - > an entirely different syntax would be better, e.g. > > namespace Foo.Bar > namespace A.B > import Baz -- gets Foo.Bar.Baz or > A.B.Baz or both. I'm not sure about the idea either, mainly because I don't know how to reconcile it with the other possible semantics for importing a non-leaf node of the hierarchy. See the notes with my layout proposal for more details. Cheers, Simon

1 0

RE: Lang. (was Re: Alternative hierarchy proposal)
by Simon Marlow 13 Mar '01

13 Mar '01

> I tend to agree with this. The functionality should be primary in > any naming scheme. The third category (magic required) can also > (sometimes) be important to an end-user - they need to know whether > a library can be used portably across compilers, but that is all. > I guess we might therefore want to consider moving Foreign from > Haskell.Extensions.Foreign > to > Haskell.Foreign > whilst keeping (for instance) > Haskell.Extensions.Concurrent > where it is, because it is only available in ghc. Or come to that, > maybe the latter really belongs in > GHC.Concurrent Actually Hugs has a portable implementation of Concurrent, which nhc could use too. You only need runtime system support if you plan to implement preemptive scheduling, which isn't a requirement for the Concurrent library, although it may be a requirement for certain applications. Cheers, Simon

1 0

RE: Summary of progress
by Simon Marlow 13 Mar '01

13 Mar '01

> On Sun, Mar 11, 2001 at 01:28:38PM +0000, Malcolm Wallace wrote: > > > > Proposal 2b: adopt a "Std." namespace for libraries that > > are common to all implementations. > > > > * There is little agreement here. Simon M, Manuel, and > > others have expressed their doubts that it is workable. > > No-one has defended the idea except me. But then Simon > > posted a hierarchy layout proposal in which "Haskell." > > seemed to take the role of "Std.". So I'm a bit > > confused. I'd like to see some more discussion about > > this. > > I think that a common prefix for the standard libraries will get to be > a pain far more than one for non standard ones, and non standard ones > are going to have a huge prefix for uniqueness anyway. The chances are > most modules will either be in . or be standard anyway. My vote is for > a user.* hierarchy with mangled e-mail addresses as I have previously > described. That'd be fine with me. Actually, to address Malcolm's (understandable) confusion, the differences between "Haskell" and "Std" are mainly that entry into "Haskell" is much easier. For a new library, it could be brought in immediately but marked "non-standard" until such time as the community has discussed and agreed on an interface. In the meantime, compilers would be free to distribute the non-standard version for testing. I've no objection to dropping the "Haskell" prefix and adding a prefix for the non-"Haskell" parts of the tree. An alternative might be to adopt an extension such that import Haskell. would add D/Haskell/ to the search path for each D in the current search path, and perhaps the "import Haskell." could be implicit, like "import Prelude". > > Proposal 3: develop a social process for adding new libraries > > to the "standard" set. > > > > * Well, this list is the starting point, so there's not > > much more to be said on that. > > * The set of criteria by which we as a community might > > judge whether a library is recognised as "standard" > > have not really received any comment. > > In the absence of some sort of committee or community voting, it > essentially comes down to what the hugs, GHC and NHC maintainers agree > on in practise. As things stand I suspect it will be decided between > then with the more controversial ones argued out on a mailing list. Voting is something we want to avoid, I think. Too often you end up with a result you don't like :-). Open discussion, followed by an informed decision by a few trusted individuals would get my vote. Cheers, Simon

4 6

RE: Lang. (was Re: Alternative hierarchy proposal)
by Simon Marlow 12 Mar '01

12 Mar '01

Malcolm Wallace writes: > One abbreviation I have always disliked intensely though is Lang. > [snip] > > The other thing that bothers me about Lang is that I don't see the > connection between many of the things proposed to go into it, and > the Haskell language itself. The FFI is the best candidate, because > it clearly extends the source language. Likewise, Generics and > Dynamic. But why should libraries like Array, Memo, and Monad > be in Lang? They don't extend the language. And of course some > things /not/ currently in Lang seem to have everything to do with > the language - like the Haskell source parser, abstract syntax, > source pretty-printer etc. I disagree. Lang (or Language if you like, or HaskellLanguageFeatures) is a *semantic* category. It contains libraries which provide access to language features. It's not a perfect categorisation (eg. you could argue that Monads aren't really a language feature, or that Chars are), but in most cases it's obvious what belongs in Lang, and that's the main thing. On the other hand, Haskell in your proposal isn't a semantic category. The similarity between the contents seems to be in name only: it's like saying that "Biography" and "Biology" books belong on the same shelf because they both begin with "Bio". > These were the reasons I proposed a hierarchy like > > Haskell > Plus > Foreign > Generics > Dynamic > Language > AbstractSyntax > Parse > PrettyPrint > > Here in my scheme, Haskell.Plus contains extensions to the language, > and Haskell.Language contains utilities to manipulate the source > language itself. Simon proposed Haskell.Source for the latter, which > would also be a fine name, provided there were no Haskell.Lang > category to confuse you. By all means change the name "Lang" to something more mnemonic and unambiguous, but I like the current meaning (and I have to admit it was chosen for its similarity to Java and the existing hslibs layout). > My final difficulty with the Lang category is that, if it contains > extensions to the language, people will be misled into thinking that > the extensions are truly a part of the language standard. After all, > they are categorised in Lang! In my opinion, a clearer name > is needed, > to emphasise the extensional nature of some of the libraries. I don't agree with separating extensions (as you probably noticed from my proposal :-). If we separate language extensions into a separate hierarchy, code will be broken when/if these extensions become part of the base language. Ok so that's probably a long way off, but I don't feel comfortable with a design which we're accepting will inevitably change later. I *do* however believe that extensions should be standardised, so that if a compiler implements a particular extension, it should match other implementations of that extension. You're probably concerned that code which purports to be Haskell 98 will accidentally make use of non-standard libraries - well for a start, the extended module namespace is itself an extension, and secondly it will be easy to restrict the libraries that are available according to compiler flags: at the moment with GHC, if you don't say -fglasgow-exts or -package <anything>, then you get Haskell 98. Full stop. The trouble is, Haskell 98 isn't very useful on its own... Cheers, Simon

3 2

RE: Summary of progress
by Simon Marlow 12 Mar '01

12 Mar '01

> On Sun, Mar 11, 2001 at 01:28:38PM +0000, Malcolm Wallace wrote: > > > > Proposal 2b: adopt a "Std." namespace for libraries that > > are common to all implementations. > > > > * There is little agreement here. Simon M, Manuel, and > > others have expressed their doubts that it is workable. > > No-one has defended the idea except me. But then Simon > > posted a hierarchy layout proposal in which "Haskell." > > seemed to take the role of "Std.". So I'm a bit > > confused. I'd like to see some more discussion about > > this. Oh, here's another thought: if we standardise the locations of as many libraries as possible, then it becomes reasonable for a compiler to do this: $ nhc foo.hs nhc: module `Foo' imports `Lang.Concurrent', which is a Haskell language extension not supported by this compiler. $ ghc bar.hs ghc: module `Bar' imports `Graphics.UI.Gtk' which is not installed. It may be obtained from "http://www.foo.bar.com". Cheers, Simon

2 1

Alternative hierarchy proposal.
by Simon Marlow 12 Mar '01

12 Mar '01

Here's my attempt at a library hierarchy. No ASCII art - I've just used indentation to indicate nesting. I've merged my initial sketch with Malcolm's proposal, so some of the subtrees are identical, but I've changed names here and there (eg. Interface --> Console, Encoding --> Codec or Digest). The leaves are all real modules, some of which already exist in some form in hslibs. The non-leaf nodes may also be imported as Haskell modules: more about this later. Note: I haven't addressed the issue of what parts of the tree should be "standard" or not - I've just populated the tree. I've placed most of the tree under "Haskell.". This is roughly equivalent to Malcolm's "Std.", except that my requirements for entry into Haskell are much slacker :-) I'm assuming there is a separate mechanism for deciding which libraries are standard, and a mechansim by which a library specification can be evolved into a standard. I've included the Haskell 98 standard libraries, placed in their correct places in the tree. No doubt the interfaces to these libraries, and even their existence in the new scheme, is now up for debate. The prelude is Haskell.Prelude, and is probably just a re-export of various other parts of the tree. In practice the implementation will be system-specific. I've noticed that the tree gets fairly deep in places ("Haskell.Lang.Foreign.Marshal.Array" ??) so a modification to the language extension to allow shortening of names might be in order, something like Java's "import java.lang.*". System specific libraries live in GHC.*, NHC.*, Hugs.* etc. One problem with this scheme which I haven't quite resolved, is what happens when you import a non-leaf node. I've identified four possible meanings, each of which is useful in certain cases: (a) bring into scope everything below that node (might be nice for eg. Foreign, Foreign.Ptr etc.) (b) bring into scope some things below the node (eg. Foreign exports everything except C.*) (b) get a default module of some description (eg. Pretty vs. Pretty.HughesPJ) (c) import a unique module (eg. Array vs. Array.IArray) Note that no special compiler support is required for importing a non-leaf node, and all of the above schemes can co-exist. Ok, here we go. I've marked optional libraries with (opt), the rest are assumed to have portable implementations, or be implementable in a portable way, for any Haskell compiler with FFI support. Cheers, Simon ------------------------------------------------------------------------ -- Haskell Prelude -- Haskell98 Prelude -- mostly just re-exports other parts of the tree. Lang -- "language support" Foreign Ptr -- should be in Data??? StablePtr -- should be in System.GC??? ForeignPtr -- should be in System.GC??? Storable Marshal Alloc Array Errors Utils C Types Errors Strings Array -- Haskell 98 Array library IArray -- (opt) GHC's overloaded arr libs MArray -- (opt) IOArray -- mutable arrays in the IO/ST monads STArray Monad -- Haskell 98 Monad library ST LazyST Either -- monad libraries State etc. Exception -- (opt) Generics -- (opt) Memo -- (opt) Unique ShowFunctions -- sounds more impressive than it is Dynamic System IO -- H98 + IOExts - IOArray - IORef Directory Select GC Weak -- (opt) StableName -- (opt) Console GetOpt Readline Time -- H98 + extensions Locale CPUTime -- split H98 "System" (too generic) into: Exit Environment (Args, Prog, Env ...) Numeric DSP FFT FIR Noise Oscillator Gaussian Source -- hslibs/hssource AbsSyn Lexer Parser Pretty Concurrent -- as hslibs/concurrent CVar -- (some of these could also go in "Data"). Chan MVar Merge QSem QSemN SampleVar Semaphore Parallel -- as hslibs/concurrent/Parallel Strategies Net -- won't need to be optional (will use FFI only) Socket -- redesign (merge w/ SocketPrim) BSD -- remove?? URI CGI -- one in hslibs is ok? Text Regex -- previously RegexString PackedString -- previously Regex (remove?) Pretty -- default (HughesPJ?) HughesPJ Wadler ... HTML -- HTML combinator lib XML Combinators Parse Pretty Types Parse -- no default Parsec Hutton_Meijer ... Posix -- redesigned, use FFI only Database SQL ODBC Debug Observe Quickcheck Trace Graphics UI Drawing Format -- perhaps should be under Data.Encoding Data Bits Char -- H98 Complex -- H98 Either -- H98 Int Maybe -- H98 List -- H98 PackedString Ratio -- H98 Word IORef STRef Binary -- Haskell binary I/O Digest MD5 ... -- others (CRC ?) Codec Bzip2 Gzip MPEG -- or perhaps Audio/Graphics.Format.MPEG? Structures Trees AVL RedBlack BTree Queue Bankers FIFO Collection Graphs FiniteMap Set Edison -- (opt, uses multi-param type classes) GHC Primitives UnboxedTypes ... -------------------------------------

5 11

Summary of progress
by Malcolm Wallace 11 Mar '01

11 Mar '01

I think this would be a useful point to stand back and summarise where we seem to have reached in discussing my original proposals (posted on 26th February). Shout if you think I've got something wrong here, missed something important out, or if I mark something as 'decided' that you really want to discuss some more. Proposal 1: introduce nested namespaces. * Everyone seems to agree that this is a good idea. * No-one has objected to using the normal dot (.) as the namespace separator character, nor has anyone suggested a different character. * The idea of an implicit import-renaming clause import A.B.C.D [ as D ] received no objections, although some people would like there to be additional implicit renamings import A.B.C.D [ as C.D ] import A.B.C.D [ as B.C.D ] * The original proposal allowed only absolute naming of modules. Others have proposed an additional mechanism for relative naming of modules within the current (or parent) namespace. I don't see agreement here yet. Proposal 2a: adopt a standardised namespace layout * Again, people seem to agree broadly that this is an essential thing. Discussions are still in progress about the detail of the layout, and I expect these to continue. Proposal 2b: adopt a "Std." namespace for libraries that are common to all implementations. * There is little agreement here. Simon M, Manuel, and others have expressed their doubts that it is workable. No-one has defended the idea except me. But then Simon posted a hierarchy layout proposal in which "Haskell." seemed to take the role of "Std.". So I'm a bit confused. I'd like to see some more discussion about this. Proposal 3: develop a social process for adding new libraries to the "standard" set. * Well, this list is the starting point, so there's not much more to be said on that. * The set of criteria by which we as a community might judge whether a library is recognised as "standard" have not really received any comment. Other issues: * We have mentioned briefly that "standard" libraries should be available in all Haskell systems. But Manuel pointed out that many libraries will be developed independently of particular compilers - and so they should be distributed separately, to ensure they remain up-to-date with external software (e.g. C libraries). I think we need to think carefully about how to manage this process. A single repository of compiler-independent and build-system-independent library code would be a good start. But it won't be sufficient. Regards, Malcolm

3 2

RE: Alternative hierarchy proposal.
by Simon Marlow 09 Mar '01

09 Mar '01

> * Win32 > Dunno what would go in here. Possibly OS.Win32 or something would > be better regardless to kep the top level branches to a minimum? Win32 is a complete Haskell binding to the Win32 API, including, but not limited to, the graphics and user interface API. It doesn't really belong under Graphics, but perhaps if we had a subtree for OS APIs, it could go next to POSIX. > In SM: > * System > THis is equivalent to Haskell.Plus.Unsafe I think. Personally I > think this is a good idea as it is not implementable in Haskell. > I'm not sure the Plus distinction is necessary though. Actually I intended the System hierarchy to contain libraries that interact with the outside world and the Haskell runtime system. All IO-related stuff (except for graphics) belongs in there. I wouldn't object strongly to moving "unsafe" functions into a hierarchy of their own, but IMO unsafePerformIO belongs with the IO monad. > * Source > This appears to be parsers etc for Haskell code, and I don't see > why it should be here rather than Text.Haskell next to HTML, XML > etc Text.Haskell sounds good. But this also contains a definition of an abstract syntax datatype for Haskell, and it may in the future contain reference implementations of a static analyser and type checker for Haskell, so perhaps keeping it in a hierarchy of its own would be better. > * Concurrent > I don't know anything about this but it looks fair enough > * Parallel > As the comment is "as hslibs/concurrent/Parallel" should it not go > under Concurrent? Hmm. The parallel API is separate from the Concurrent API, and there exist systems that implement one or the other, as well as both. I think they're separate. Parallel is small, though, and perhaps doesn't warrant its own top-level category. Lang.Parallel and Lang.Concurrent would be fine. > * Prelude > MW has Haskell.Language.CoreSyntax which I think makes more sense I don't understand - Prelude wasn't intended to be the same as Haskell.Language.CoreSyntax, it's just the Haskell98 Prelude that we have now. Perhaps Lang.Prelude would be a better place for it, though. > * GHC > Probably better under Lang.CompilerSpecific.GHC? Maybe. We're going to be writing the Lang.CompilerSpecific.GHC prefix a lot while implementing libraries, though. Remember most of this hierarchy is under Haskell, but I put the compiler specific parts outside of the Haskell hierarchy. It's not very clear from the diagram, sorry. Haskell.{Lang,System,Text etc.} GHC.{UnboxedTypes etc.} > There's also the Std vs Non-std top branch debate. I agree with the > non-std myself. > > The SM trees in general seem a bit better thought out to me too, e.g. > I don't think Bits or MD5 belong under Data.Encoding. > > To summarise I think something like: > > Algebra: As MW > Concurrent: As SM plus Parallel > Data: As SM > Database: As SM and MW > Debug: As SM and MW > Graphics: As MW (more detailed than SM) > Lang: As SM, plus CompilerSpecific.*, Prelude, { System.* > from SM plus POSIX plus BSD } Do you mean System under Lang, or as a top-level category (I think top-level is better, BTW). > Net: As SM (plus a POSIX module if POSIX talks about this) > Numeric: As SM and MW > OSSpecific: As explained > Text: As SM and MW > User: your(a)email.address gets address.email.your.* off of this As a general comment on the subject of hierarchy layouts, it's clear to me that we can never have a "perfect" hierarchy, since some modules will always belong in more than one place and others won't belong anywhere in particular. I think the best we can do is try to come up with something that is as consistent as possible, not too deeply nested, will extend gracefully as we add new libraries, and won't require reorganising as the language changes. Cheers, Simon

3 2

RE: Module versions
by Simon Peyton-Jones 09 Mar '01

09 Mar '01

| In either case, the actual layout in filespace can be handled | with symbolic links, e.g. | | Graphics/UI/Gtk.hs -- symbolic link to Gtk_1_2.hs | Graphics/UI/Gtk_1_2.hs -- the real module | | At the moment, nhc98 will flag a warning (for instance) that | | module Gtk is called Gtk_1_2 in its interface file | | but this is harmless. I don't know how GHC currently copes | with this kind of module/filename conflict? GHC also complains if the filename and module name differ. Except for Main. Simon

1 0