- Libraries - Haskell.org

new Library Infrastructure spec.
by Isaac Jones 24 Jun '04

24 Jun '04

(followups to libraries(a)haskell.org) Hail Haskellers! A commonly-identified obstacle to the spread of Haskell libraries is that there is no standard way a) for authors to package a library or other tool for distribution to others b) for customers to install such a package Platforms vary. Haskell compilers vary. Packages have dependencies. Etc. Isaac Jones has coordinated an effort to address this problem. We've had a lot of discussion between him and (at least some of) the folk involved in the GHC, Hugs, and nhc implementations. This discussion has led to a new concrete proposed specification[1]. Here it is: http://www.haskell.org/libraryInfrastructure/proposal See also the project home page: http://www.haskell.org/libraryInfrastructure/ We would welcome your feedback, either as a potential library or tool author, or as a potential consumer, or both. The specification isn't complete in every detail, but it seems better to post it before investing in details that may be rendered void by subsequent discussion. We hope this will be an important spec, and will be with us for a long time. So now's the time to look at it. Send feedback to libraries(a)haskell.org. (You can subscribe to this list at: http://www.haskell.org/mailman/listinfo/libraries .) Sincerely, Isaac Jones, Simon Marlow, Ross Paterson, Malcolm Wallace, Simon Peyton Jones [1] This specification differs from the previous proposal both in technical details, and in that it is the combined efforts of the undersigned. Those familiar with the previous proposal will not find large high-level differences, but some details have been made more concrete and some details have changed.

6 11

RE: System.Process
by Simon Marlow 24 Jun '04

24 Jun '04

On 20 June 2004 10:57, David Roundy wrote: >> While doing the Windows port I realised that parts of the interface >> didn't work out too nicely, so I have left out createPipe and >> commandToProcess from the interface. If you think it is imperative >> to have either of these, please yell. > > I don't think either is imperative, but noticed that the documentation > refers to createPipe. Oops, that was left over from a previous version. Now fixed. >> Comments please... perhaps there ought to be some simpler versions of >> the general functions in this library, to cover the more common >> cases? > > While I'm not a fan of the Maybe method of providing optional > arguments, it's hard to see any small set of simpler versions that > one could devise. Having five runProcessX functions doesn't simplify > things. It's probably simplest to let the users define simpler > versions... unless you perhaps wanted to define a runProcess analog > of "system", which differs only in that it accepts a FilePath and > [String] instead of a shell command. Yes, I had in mind providing 'system' and 'rawSystem' replacements too. >> If there are no serious problems, this will be imported into CVS for >> GHC >> 6.4, and possibly future Hugs/nhc98 releases, if there aren't any >> portability problems. > > Sounds great to me, although I should warn you that I haven't actually > tried any of the code... > > Just to make sure I understand things... could runProcess be > implemented in terms of runInteractive as below? Not that this would > be wise--I imagine it goes the other way around--but I want to be > sure that I understand how I'd want to modify things, for example if > I wanted to interactively write to a process, but have its output > logged straight to a file. > > runProcess c as menv (Just inh) (Just outh) (Just errh) = > do (iinh, iouth, ierrh, ph) <- runInteractiveProcess c as menv > instr <- hGetContents inh > outstr <- hGetContents iouth > errstr <- hGetContents ierrh > forkIO $ do hPutStr outh outstr > hClose outh > forkIO $ do hPutStr errh errstr > hClose errh > forkIO $ do hPutStr iinh instr > hClose iinh > return ph Hmm, looks like that might work, yes. It's not quite the same, though: in your version, the inh Handle ends up semi-closed, whereas in the current runProcess implementation it is closed immediately. In your implementation, the outh and errh handles are closed when all the output has been written to them; in my implementaiton they are closed immediately. (I just noticed that my Windows implementation differs from the Unix implementation in this respect). Closing these handles immediately seems more deterministic, but I'm not sure which is the most "useful" behaviour. Cheers, Simon

2 1

HaXml, namespace processing
by Graham Klyne 22 Jun '04

22 Jun '04

Since my last HaXml missive [1], I've managed to get some XML namespace handling code working with HaXml. The namespace processing itself is implemented as a transformation/filter that is applied to XML content after general entity substitution has been performed. The XML content model has been extended to use QNames for element and attribute names (this change was made some time ago), and also to carry XML-Infoset-like information with each element (including a list of in-scope namespaces). When namespace processing is performed, the xmlns attributes are scanned and removed from the XML content. Element and attribute names containing ':' are reprocessed to be in a namespace+local name form. Also, is a default namnespace is in scope, element names not containing a ':' are similarly processed (but not attribute names, because the meaning of unprefixed attributes is determined by the containing XML element). I've created a small suite of test cases for namespace processing, which work by extracting the list of element and attribute QNames from a processed document, and comparing the result with a given list. Also since my last message [1], I've created a test suite for the XML validator, based on the W3C XML conformance data. HaXml passed all but two of these tests without further modification. I've created another snapshot [2] of my work. This represents all of the major functional changes I plan to make at this time, though there is a good deal of code tidying, and a few small features I'd like to add, which I may get round to at some time. My changes are also rather devoid of external documentation. The test harness [3] is one place to look. The namespace handling functions are in module Namespace.hs [4]. #g -- [1] http://www.haskell.org//pipermail/libraries/2004-June/002266.html [2] http://www.ninebynine.org/Software/HaskellUtils/20040622-Haxml-1.12.zip [3] http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/test/TestXml.hs [4] http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/src/Text/XML/HaX… ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

LIP/cabal
by Ian Lynagh 21 Jun '04

21 Jun '04

The example in 1.2 is still a bit mysterious to me. Initially Angela has the following files: ./Data/Set.hs ./Data/Bag.hs ./Angela/Internals.hs then she runs ./Setup.lhs configure --ghc I assume she now has the above files plus ./.Setup-info or somesuch on Unixish OSes, and a hidden file on Windows? ./Setup.lhs build She now has the previous 4 files plus the following? ./Data/Set.o ./Data/Bag.o ./Angela/Internals.o How did that happen? Did it just compile all .hs/.lhs files it found inside .? Did it work out what order to compile them in a la ghc --make? ./Setup.lhs src-dist Presumably angela-coll-1.tar.gz now contains the 3 .hs files? Again, did it just pick out any and all .hs/.lhs files inside .? Is Data/Set.hs or angela-coll-1/Data/Set.hs in the tarball? Also, you haven't actually told cabal Angela.Internals is internal, as mentioned in the text, right? ----- 2.2 says "one or more packages (of which the Prelude is one)". Did you really mean this, i.e. that there is a package Prelude containing the module Prelude? ----- In 3.1.2: Suppose I have A-2.1 installed. I install B-1.0 and my whizzy installing whatsit goes off and grabs A-1.9 for me. I now have 1.9 as the exposed A? I'm sure that's not what I wanted, especially as one of my programs won't work without that bugfix in A-2.0. Three alternatives spring to mind: If A is installed it is exposed if it is the newest version of A only. If A is installed it is exposed if it is the newest version of A *and* the second newest version of A was exposed before. Packages have a manual and automatic mode. In automatic mode we behave like the first option. In manual mode the exposed version is not changed. (this is what Debian's update-alternatives does). There is also the issue of unregistering packages. If an exposed package is unregistered then you could either fall back to the newest installed version or leave everything unexposed. In the third choice above, falling back to the newest and setting it to automatic if it isn't already makes most sense I think (and is what update-alternatives does I believe). Checking the update-alternatives logic if you go that route would be sensible - I am not an expert on it, but have a feeling it has a bit more complexity than I've described to cope with various scenarios in teh best way possible. ----- 3.2 says: When all the -package and -hide-package flags on the compiler's command line have been processed, the resulting module namespace must not contain any overlapping modules; the compiler should check this and report any errors. System packages are implicitly shadowed by user packages of the same name so their modules don't count for this check, right? Are modules in my program allowed to overlap with those of packages? What about modules in user packages overlapping with (non-shadowed) modules in system packages? Also, this isn't going to guarantee no overlaps, e.g. with the As and Bs from 3.1.2, right? A-1.9 could have a module X, A-2.1 not have a module X and package C also have a module X, with A-2.1, B, C exposed. Oh, suppose B-1.0 from earlier is a system package that depends on A-1.9. There is an A-1.9 as one of my user packages, and also as a global package. Presumably it, and thus the rest of my program, uses the global A-1.9? ----- 4.1: In the discussion about quotes and skipping fields, with this sort of format I'd expect a line starting with a non-comment/whitespace character to start a new field. Also, if cabal can't understand my deps line I'd expect it to tell me it doesn't know what to do, but I think I might be missing the point there. ----- 4.2 I think [install-prefix] shouldn't be there but --install-prefix should be used instead. Why can't we have an uninstall that normally just fails? That's exactly the same as "make uninstall", after all. (I don't actually really care about this, just curious. It seems silly not to support uninstall for those people who take the trouble to write one in their Makefiles). ----- 4.3 With my Debian hat on I think, in the package building script for A, I want to do: ./Setup.lhs clean # # the Debian tools will make the diff for the Debian source package # here # ./Setup.lhs configure --with-compiler=/usr/bin/ghc6 \ --prefix=/usr/lib/ghc6/modules/A ./Setup.lhs build ./Setup.lhs install --user \ --install-prefix=`pwd`/debian/libhaskell-ghc6-A/usr/lib/ghc6/modules/A mv installed-pkg-descr debian/libhaskell-ghc6-A/usr/lib/ghc6/modules/A # # ./Setup.lhs clean ./Setup.lhs configure --with-compiler=/usr/bin/nhc98 \ --prefix=/usr/lib/nhc98/modules/A ./Setup.lhs build ./Setup.lhs install --user \ --install-prefix=`pwd`/debian/libhaskell-nhc98-A/usr/lib/nhc98/modules/A mv installed-pkg-descr debian/libhaskell-nhc98-A/usr/lib/ghc6/modules/A # # Repeat for hugs, ghc5, anything else, as desired but you haven't given me a "./Setup.lhs clean". Is that just an oversight? Is there anything I've got wrong? In the postinst/prerm I'll also want to run ./Setup.lhs (un)register, pointing it at the file I copied for the first case. However, I don't think I have a Setup.lhs any more. I think I want "./Setup.lhs install" to take an additional argument --setup-prefix=`pwd`/debian/libhaskell-ghc6-A/usr/lib/ghc6/module-setup/A as I imagine this could require multiple files and different behaviour for different compilers? I'd then expect /usr/lib/ghc6/module-setup/A/Setup.lhs --register --global \ < /usr/lib/ghc6/modules/A/installed-pkg-descr to register the library (after the deb has been installed). In fact, scratch that. Get rid of the mv lines above and /usr/lib/ghc6/module-setup/A/Setup.lhs --register --global should just do it all for me. ----- Sorry if any of the above has been discussed before and I missed it. Thanks for your good work Ian

1 0

System.Process
by Simon Marlow 20 Jun '04

20 Jun '04

I've now implemented the System.Process library on Windows too. It needs Ross Paterson's new version of the Win32 library, which can be found in CVS under libraries/win32. While doing the Windows port I realised that parts of the interface didn't work out too nicely, so I have left out createPipe and commandToProcess from the interface. If you think it is imperative to have either of these, please yell. Haddock docs here: http://www.haskell.org/~simonmar/process Code here: http://www.haskell.org/~simonmar/process/process.tar.gz (sorry no Makefile, but there's only a .c file and a .hsc file to compile). Comments please... perhaps there ought to be some simpler versions of the general functions in this library, to cover the more common cases? If there are no serious problems, this will be imported into CVS for GHC 6.4, and possibly future Hugs/nhc98 releases, if there aren't any portability problems. Cheers, Simon

4 3

RE: sequences
by Okasaki, C. DR EECS 18 Jun '04

18 Jun '04

I wrote: > For what it's worth, neither real-time nor bootstrapped queues really > make sense in Haskell. The advantage of real-time queues is that, > in a (mostly) strict language, you get O(1) *worst-case* bounds instead > of amortized bounds, but in Haskell, because everything's lazy, you're > stuck with amortized bounds anyway. George Russel wrote: > I don't really see the point here. If I am writing an > application where things need to respond to external > interactions in guaranteed O(1) time, then I am going to need > real-time queues whether I am writing in Haskell or FORTRAN. > The only difference laziness makes is that I may have to put > in strictness annotations to avoid the queue elements > containing unbounded amounts of computation. If you want guaranteed O(1) response time, you're going to have to work really hard to get it. You can't just drop in real-time queues for some other kind of queue and expect them to take care of everything behind the scenes. Real-time queues can take care of what they need to do internally once they get control, but it would be up to the user to make sure (via strictness annotations or seq's or the like) that they get control. Here's the catch. Suppose you do something like f (enqueue x q) Once the enqueue operation gets control, it can do whatever it needs to do with x and q. But the enqueue operation does not get control right away. Instead a thunk gets built, and there's nothing enqueue can do to prevent it. You can easily write code that will generate a long chain of such thunks. Then when you get around to calling something like dequeue q it ends up taking O(N) time because it has to evaluate that chain of enqueues before it can finally start the dequeue. (And in fact you can make it take longer than O(N) time if the chain includes a mix of enqueues and dequeues.) -- Chris

1 0

Re: sequences
by George Russell 18 Jun '04

18 Jun '04

Christopher Okasaki (snipped): > For what it's worth, neither real-time nor bootstrapped queues really > make sense in Haskell. The advantage of real-time queues is that, > in a (mostly) strict language, you get O(1) *worst-case* bounds instead > of amortized bounds, but in Haskell, because everything's lazy, you're > stuck with amortized bounds anyway. I don't really see the point here. If I am writing an application where things need to respond to external interactions in guaranteed O(1) time, then I am going to need real-time queues whether I am writing in Haskell or FORTRAN. The only difference laziness makes is that I may have to put in strictness annotations to avoid the queue elements containing unbounded amounts of computation.

1 0

RE: System.Process
by Simon Marlow 18 Jun '04

18 Jun '04

On 18 June 2004 14:57, Glynn Clements wrote: > Simon Marlow wrote: > >> I've now implemented the System.Process library on Windows too. It >> needs Ross Paterson's new version of the Win32 library, which can be >> found in CVS under libraries/win32. > >> Code here: >> >> http://www.haskell.org/~simonmar/process/process.tar.gz >> >> (sorry no Makefile, but there's only a .c file and a .hsc file to >> compile). >> >> Comments please... perhaps there ought to be some simpler versions of >> the general functions in this library, to cover the more common >> cases? > > Should there be a current directory argument? This is implemented in > Posix.runProcess (in 6.0.1), and seems like it should be implementable > on Windows. Yes, that could be added. Cheers, Simon

1 0

RE: System.Process
by Simon Marlow 18 Jun '04

18 Jun '04

On 18 June 2004 13:47, Krasimir Angelov wrote: > Where are you planning to import the new module in > base package? In such case why you use win32 package? Putting it in the base package would be painful, because of the Win32 and System.Posix dependencies. Hence it will probably go in a separate package. The Win32 package was useful for error handling and some other things. Some of this implementation should probably move into the Win32 library itself. Cheers, Simon

1 0

HaXml revisions (continued)
by Graham Klyne 17 Jun '04

17 Jun '04

Further to my previous message [1], I've created a new snapshot [2] of my developments to the HaXml package. Browseable source code is at [3]. It relies on a version of Network functions that are at [4]. The main feature of this release is the addition of a "filter" to perform general entity substitution in the parsed XML, and some fixes to the parameter entity substitution. [1] http://www.haskell.org//pipermail/libraries/2004-June/002243.html [2] http://www.ninebynine.org/Software/HaskellUtils/20040617-Haxml-1.12.zip [3] http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/ [4] http://www.ninebynine.org/Software/HaskellUtils/20040609-Network.zip The additions have necessitated some further reorganization of the handling of entity definitions, in order that some of the more subtle examples noted in the XML specification work as documented. The code still needs some tidying up, but it works on all the test cases I've assembled to date. Next steps are: - create a test suite for XML validation, based on the W3C conformance test suite, and make sure the validation functions (still) work. - tidy up the code, in particular with a view to pruning out deadwood. - create a filter to perform namespace processing (that which got me started on all this in the first place). ... There's one change I've made which I'm not entirely happy with: in order to be able to collect diagnostic information from XML filter (CFilter) processing, I've added an option CErr to the XML content model. A cleaner solution would, I think, be to extend the return type of a CFilter value, but this would be a significant change to the package interface in an area that I assume is particularly used by applications. ... I've created separate code paths for internal (non-IO) and external (IO using) entity processing, but currently external entities are read using unsafePerformIO. I've thought a little about trying to have the "external" interfaces return an IO value, and avoid using unsafePerformIO, but I can't currently see how to do that without sacrificing a very high degree of code sharing that is currently achieved. As I write this, I think I've just realized how to do this. If all the relevant shared code runs in some unspecified monad (i.e. is polymorphic in a monadic return type), then the non-IO code can use an identity monad and simply pick out the resulting value as a pure value, but code which depends on IO will be forced to return an IO value (or use unsafe...). Does this sound plausible? ... #g -- [1] http://www.haskell.org//pipermail/libraries/2004-June/002243.html [2] http://www.ninebynine.org/Software/HaskellUtils/20040617-Haxml-1.12.zip [3] http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/ [4] http://www.ninebynine.org/Software/HaskellUtils/20040609-Network.zip ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0