June 2004 - Libraries

RE: new Library Infrastructure spec.
by Simon Marlow 15 Jun '04

15 Jun '04

On 14 June 2004 18:51, Isaac Jones wrote: > "Iavor S. Diatchki" <diatchki(a)cse.ogi.edu> writes: > > (snip) >> i am a linux user, and i want to download and play with a haskell >> (binary) library (say wxHaskell). >> however i am at university, where i don't have root permisions, so i >> cannot use rpm (as it has fixed installation path). so i just want >> to put the library in my home directory and use it, >> but i don't want to have to recompile it, as this is slow and may >> require tools i don't have or i don't >> know how to use. furthermore, if i don't like the library i should >> be able to still delete it, and somehow inform my haskell >> implementation that the library is not there anymore. > (snip) > > This is what Simon mentioned in (2): > >> Simon Marlow wrote: > > (snip) > >>> 2. Generic binary distribution, generated by './Setup.lhs bdist' > > And we do hope to support it in the near future (probably at the 1.0 > release), but it's not a priority for the 0.1 release. > > BTW, this is somewhat tricky, since you clearly have to get someone > who has your platform to build it for you, just as they have to build > an RPM, so it's not clear that there will be a lot of "generic binary > distribution" packages out there. Of course, this is a problem that's > inherant to binary distributions, not just Cabal. > > GHC can actually be installed like this, btw. Actually there's a problem with the use case that Iavor describes. We explicitly do not support changing prefix between build and install, so the guy building the binary distribution for Iavor would have to configure with $ ./Setup.lhs configure --prefix=/home/diatchki We do not support building binary distributions that are install-location-neutral. This was a conscious decision - making install-location-neutral binary distributions is hard in general, and not always possible. GHC goes to some lengths to support it. Most other packaging systems do *not* support this (eg. neither RPM nor BSD ports do). If we decide that this is a use case that we really want to support (and I doubt it, personally), then we could allow a package to optionally allow changing the prefix at install time. Most simple Haskell libraries can allow this without difficulty. Perhaps it could be another flag in the Setup.description. The problem is that since we don't want to *require* that a package supports this, you can't guarantee that a random binary distribution you downloaded will necessarily install in your home directory. Cheers, Simon

2 1

Re: sequences
by Okasaki, C. DR EECS 15 Jun '04

15 Jun '04

Ross Patterson wrote: >[Simon Marlow wrote:] >> Why would anyone prefer Catenable over JoinList? > >Catenable has guaranteed bounds for head, even in a persistent setting, >and JoinList doesn't. But Catenable and CatDeque seem to be uniformly >inferior to FingerTree. Similarly real-time and bootstrapped queues >are dominated by Banker's queues. For what it's worth, neither real-time nor bootstrapped queues really make sense in Haskell. The advantage of real-time queues is that, in a (mostly) strict language, you get O(1) *worst-case* bounds instead of amortized bounds, but in Haskell, because everything's lazy, you're stuck with amortized bounds anyway. The advantage of bootstrapped queues is that, in a (mostly) strict language, it only needs a tiny bit of laziness (O(log n) thunks compared to O(n) thunks for banker's queues). But again, this advantage goes away in Haskell because everything's lazy. For FingerTrees, I agree that it would probably make a better general-purpose implementation than Catenable or CatDequeue. As you noted, it's easy enough to create microbenchmarks to make the O(1) append of Catenable or CatDequeue win out, but in practice, FingerTrees would probably be better. I would guess you could probably tweak maybe a few percent improvement out of FingerTrees by using a different balancing scheme than 2-3 (maybe red-black or even one of the AVL variants), but the small gain may not be worth the effort. -- Chris

1 0

RE: new Library Infrastructure spec.
by Simon Marlow 14 Jun '04

14 Jun '04

On 09 June 2004 20:29, S. Alexander Jacobson wrote: > The proposal should simply define the file format > that Angela and Peter use to deliver Haskell > packages to Wally. A Haskell package is > some combiation of: > > * Haskell source > * binary libs for the target platform, > * documentation, > * data, > * and meta-data I don't think you're suggesting anything that isn't already covered by the proposal. Others have already responded to that effect, but I'll reiterate: there are several kinds of binary package which the end user can install. These include: 1. Platform-native packages: RPM, Debian, Windows MSI, etc. 2. Generic binary distribution, generated by './Setup.lhs bdist' I believe what you're asking for is (2). These binary distributions can be installed by anyone with a Haskell compiler on their system, by unpacking the distribution and invoking './Setup install'. However, I don't consider (2) to be a priority. Funcionality-wise, binary distributions are less desirable than platform-native packages, because they don't deal with installing dependencies and they don't let you uninstall the package later. When installing software, it is good practice to co-operate with the system's native packaging system. (2) is useful on systems that don't have a native packaging system, e.g. Solaris. However, on those systems, Joe User can usually just become Bob and install packages from source (it's not hard; just do './Setup.lhs configure; ./Setup.lhs build; ./Setup.lhs install'). Cheers, Simon

5 5

Re: new Library Infrastructure spec.
by Malcolm Wallace 14 Jun '04

14 Jun '04

"S. Alexander Jacobson" <haskell(a)alexjacobson.com> writes: > There are e.g. Win32 installers HDirect and > HaskellScript. Who produced them and who is > intended to use them? Bob the Builder produced them. Joe User uses them. But you are right, there is indeed an intermediate role, the person who downloads and installs the binary package. This does not necessarily involve any building, but still requires a minor degree of knowledge of the packaging scheme. However, in Cabal, there are many different possible binary packaging schemes. Various people will use RPM, .deb, .msi, apt-get, BSD ports, and so on. Essentially, we assume that the person who downloads and installs one of these formats is competent with their chosen mechanism. They just use whatever is standard for their platform. So it seems fair to characterise that role as Joe User, since we don't really expect any familiarity with Cabal. Indeed, such a person does not (knowingly) use Cabal at all! However Cabal does address the person who /creates/ an RPM, .msi, etc, who needs have a documented protocol for building, installing, and registering Haskell code. Roland, Willie, etc, embed that knowledge into the binary package, so the final user, Joe, does not need to think about it. > I run GHC on my Windows box and can't > easily/reliably get C code to compile and produce > DLLs (both because my C is *very* rusty and > because I don't have an up-to-date version of all > the cygwin mingwin blah blah packages). Yet, I do > occasionally download and install Haskell > libraries? Am I Bob, Peter, Sam, Joe, or perhaps > someone new? You are almost certainly Joe, if your ideal world is to download a .msi InstallShield binary package and press the button. Regards, Malcolm

3 4

HaXml revisions
by Graham Klyne 09 Jun '04

09 Jun '04

Over the past few weeks I've been working on the HaXml library with a view to: creating a regression test suite improving handling of diagnostics separating stand-alone functionality from functionality requiring I/O laying groundwork for XML namespace processing improving conformance to the XML specification support for UTF-8 and UTF-18 input, as required by the XML spec access to external entities via HTTP This phase of work is pretty much done (though the code could still be much tidied up in many places, particularly the removal of code in the parser and lexer that is now effectively unused). The work has involved some small but extensive changes to the HaXml source code, including fairly radical restructuring in a couple of areas (notably the introduction of a new layer between the lexer and parser to handle parameter entity subsitution). The package now depends on Network.HTTP and my revised version of Network.URI (which in turn uses the Parsec parser combinators) [Two separate parser libraries required. Oh well.] The regression tests use HUnit. I'd like to think the improvements are sufficient to warrant their eventual folding back into the hierarchical library components, despite some small interface changes (mainly to type declarations rather than function interfaces). Also, I've not done any of the re-packaging work for incorporation into (say) the GHC distribution -- that's still rather a black art to me. I've created a snapshot of my work at: http://www.ninebynine.org/Software/HaskellUtils/20040609-Haxml-1.12.zip http://www.ninebynine.org/Software/HaskellUtils/20040609-Network.zip in the hope that folks can check it out and see if I've introduced any unacceptable incompatibilities with the existing HaXml package. The former ZIP file includes a copy of the W3C XML test suite, parts of which I'm using for my regression tests. A more detailed list of changes made can be found in the file: http://www.ninebynine.org/Software/HaskellUtils/HaXml-1.12/TODO.txt (and also in CVS logs that I've included at the end of each source file I've changed.) I should also note that all my development/testing has been done under Windows using Hugs, so there may possible be some wrinkles to iron out on other platforms (but I've generally not messed with likely platform-dependent code). I plan to push ahead with the various aspects of new functionality that I require for my applications, but I'll be happy to help where I can to fold these changes into the mainstream version of HaXml. #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Network.HTTP module, using simpleHTTP
by Graham Klyne 09 Jun '04

09 Jun '04

Bjorn, I'm trying to add an HTTP entity retrieval capability to HaXml using simpleHTTP as the basis of a new function, readHTTP [1], that works very similarly to prelude.readFile (except that its argument is a Network.URI value). Function simpleHTTP still leaves a fair amount of result analysis to be done by the calling program. I'm thinking that it might be convenient to provide a simple function, say: hasResponseData :: Response -> Bool that can be used to drive a simple binary decision along the lines of: return $ if hasResponseData response then (Right $ rspBody response) else (Left $ show response) where hasResponseData rsp = case rspCode rsp of (2,_,_) -> True otherwise -> False I'm not sure if I've got the condition right here, but it seems about right. Do you think this would be a reasonable addition to the HTTP module, to make it very easy for a program to issue a simple HTTP GET to retrieve a resource? ... Another thought I wanted to raise with you concerns the URI authority parser that is currently part of the HTTP module. My revised version of URI already does most of what this micro-parser does (apart from not separating the username and password in userinfo). When the revised URI specification (successor to RFC2396) looks stable, I'm planning to update the Network.URI module in the hierarchical libraries. It occurs to me that the added functionality here could mean that module HTTP might be simplified. A copy of the current state of my Network.URI module can be found here: http://www.ninebynine.org/Software/HaskellUtils/Network/URI.hs There is also a test program and a test suite, described here: http://www.ninebynine.org/Software/HaskellUtils/Network/URITestDescriptions… An earlier, stand-alone, test suite is here: http://www.ninebynine.org/Software/HaskellUtils/Network/URITest.hs #g -- [1] here's a simple readHTTP function I've cooked up... does it look workable?: [[ readHTTP :: URI -> IO String readHTTP uri = withSocketsDo $ do { res <- simpleHTTP (defaultGETRequest uri) ; case res of Left err -> return ("\nError! failed to read "++show uri++": "++show err++"\n") Right rsp -> return $ if hasResponseData rsp then rspBody rsp else show rsp } hasResponseData :: Response -> Bool hasResponseData rsp = case rspCode rsp of (2,_,_) -> True otherwise -> False defaultGETRequest uri = Request { rqURI=uri , rqBody="" , rqHeaders=[ Header HdrContentLength "0" , Header HdrUserAgent "haskell-haxml/0.1" ] , rqMethod=GET } ]] ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

2 2

HaXml refinements
by Graham Klyne 08 Jun '04

08 Jun '04

Malcolm, FYI. I've completed and tested the restructuring of parameter entity handling I described previously. Some additional conformance test cases now pass, though a couple that did pass now fail. (These are non-well-formed test cases that the parser doesn't recognize as such. I've made a decision not to try and catch every non-well-formed case that still makes sense as an XML document.) I've also got relative file access working properly for external entity access (i.e. relative to the referring file rather than the program's "current directory") I've also generalized the external entity access to support HTTP, but that's not yet tested. I think these represent the extent of changes I intend to make that affect the existing core HaXml code. The remaining functions (GE substitution, namespace processing, etc) I hope to implement as filters rather than changes to the core code. Do you want to think about preparing a new release? I'll bundle up the code when I've tested the HTTP access. #g -- PS: I just remembered, I also need to check out that external DTD subset handling works properly with the revised external entity handling. ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

Re: new Library Infrastructure spec.
by S. Alexander Jacobson 08 Jun '04

08 Jun '04

Isaac, > I don't think the "runhugs" bit is a requirement. You can execute the > setup script however you like. Any reason we can't simplify Joe's interface to: ghc --install http://url/of/some.pkg --or-- hugs --install /path/to/some.pkg Where some.pkg is some combination of tarball and setup specification. The install system of the compiler/interpreter would retrieve some.pkg from the specified location, and use the setup spec to compile it and put it in the appropriate locations such that Joe can use it from his Haskell program or perhaps invoke it as an executable in whatever manner is appropriate to his local OS (mouse-click or command-line). The open question is what this setup specification should look like. There is an obvious tradeoff between the interests of Joe and Angela on the one hand and Marcus on the other. Marcus creates packages that require some combination of GCC and Perl and perhaps some random Perl libs from CPAN to build. There is really no way to make installation of systems he creates simple and generic. My feeling is that we shouldn't even bother to try. Marcus and Sam both know enough to communicate effectively to make it happen and neither of them is going to be happy trusting some generic install script to get it right on their totally customized systems! Angela OTOH knows little about platform dependencies and other stuff and simply wants to make a couple of Haskell libraries available with minimum hassle. Angela would almost certainly prefer a system in which the compiler can guarantee her that her package will install correctly on all conforming Haskell compilers to a system where she has the full freedom to script arbitarily complex installation and make arbitrarily obscure mistakes on any system. I don't know whether the setup spec should be a static data structure or Arie's setup monad, but I'm pretty sure that Angela would prefer either one to generic IO. -Alex- PS I am dropping the issue of malware here to focus on making things as easy as possible for Joe and Angela. PPS I realize that you really want to suport Marcus as well. Perhaps it might make sense to separate hc-pkg from the rest of the document? ___________________________________________________________________ S. Alexander Jacobson Check out my new blog!!! 1-917-770-6565 voice http://alexjacobson.com

4 7

RE: new Library Infrastructure spec.
by Simon Peyton-Jones 07 Jun '04

07 Jun '04

| > I like the simplicity but would also like the spec | > to make it easy for me to guarantee that that I | > don't end up running/installing malware. | > | > I think Haskell's typesystem and purity should | > make it relatively easy to make sure that: I don't think so, alas. The IO monad lets you do *anything*, and of course Setup.lhs runs in the IO monad, else it would not be able to move files or run a compiler. So I'm not optimistic. Perhaps a package whose Setup.lhs did nothing but import Distribution.Simple (which you perhaps trust) would be more trustworthy than a big pile of goop. But remember that you are installing a library that you will later (presumably) run, and that might be bad too. I'm not optimistic here. Simon

5 11

FunctorM
by John Meacham 04 Jun '04

04 Jun '04

Pretty simple, and quite useful. perhaps it could be added as Control.FunctorM ? just as Functor generalises 'map', FunctorM generalises 'mapM'. there are probably some instances I missed. note that unlike Functor, an instance for IO cannot be written for FunctorM. module FunctorM where import Array class FunctorM f where fmapM :: Monad m => (a -> m b) -> f a -> m (f b) instance FunctorM [] where fmapM f xs = mapM f xs instance FunctorM Maybe where fmapM _ Nothing = return Nothing fmapM f (Just x) = f x >>= return . Just instance Ix i => FunctorM (Array i) where fmapM f a = sequence [ f e >>= return . (,) i | (i,e) <- assocs a] >>= return . array b where b = bounds a -- John Meacham - ⑆repetae.net⑆john⑈

3 3