April 2004 - Libraries

Re: #ifdef considered harmful (was: DData)
by Jan-Willem Maessen - Sun Labs East 20 Apr '04

20 Apr '04

Jan-Willem Maessen - Sun Labs East wrote: Simon Marlow wrote: >> Hmm, wouldn't it be nice to have a "FastInt" library which re-bound >> the appropriate types and functions? Haskell libraries are >> chock-a-block with instances of "#ifdef __GHC__" which define things >> in terms of Int# instead of Int. We could simplify all those >> libraries if we did it just once, in one place. >> >> The problem, of course, is that the naive non-GHC user could write a >> library which used "FastInt" polymorphically, and write code that >> wasn't portable to GHC. I don't think there's an easy, magic >> solution there. Perhaps such a library would be named >> FastIntWhichIPromiseToTestWithGHC. > > > > Most of the time it's unnecessary to use an explicit Int#. If your Int > is in a data structure, you can use > {-# UNPACK #-} !Int > > which is portable, and compiles to an unboxed Int in GHC >= 6.2. The real problem, of course, isn't the data structures at all---GHC does an excellent job with these (though the "UNPACK" pragma is a revelation). The actual problem is function arguments. GHC can't do worker-wrapper unless a function is strict. Often we end up with something like this: data Tree a = Branch !Int a (Tree a) (Tree a) lookfor :: Int -> Tree a -> Maybe a lookfor Leaf key = Nothing lookfor (Branch k v l r) key | key==k = Just v | key < k = lookfor l key | otherwise = lookfor r key Here we don't use "key" in the first clause, so GHC isn't allowed to unbox. Most of the uses of Int# that I've seen in library code have been workarounds to this problem. In this case, being able to write lookfor :: FastInt -> Tree a -> Maybe a might give us some hope of getting the compiler to generate the unboxed code we might want, yet still afford us the opportunity to use boxed types if that's not possible. We could do this today without changing the language or the compilers. The library would have to be carefully coded... -Jan-Willem Maessen

3 3

RE: aha! file descriptor leak in 'readFile'
by Simon Marlow 20 Apr '04

20 Apr '04

On 19 April 2004 08:32, John Meacham wrote: > I have finally tracked down a test case which results in the file > descriptor leak which has plauged one of my programs since its birth. > > this code: > sequence (replicate 100 $ handle (\_ -> return "") ( readFile ".")) > > will leak 100 file descriptors, apparently, after a > Fail: .: openFile: inappropriate type (is a directory) > exception from readFile, the file descriptor will not be closed, and > instead be left open til the end o' time. > > it took me a long time to find because I thought the bug was in > the listing of my directories, not the readFile's of its contents > (including "." and ".."). Yes, that's a bug. Now fixed. Cheers, Simon

1 0

aha! file descriptor leak in 'readFile'
by John Meacham 19 Apr '04

19 Apr '04

I have finally tracked down a test case which results in the file descriptor leak which has plauged one of my programs since its birth. this code: sequence (replicate 100 $ handle (\_ -> return "") ( readFile ".")) will leak 100 file descriptors, apparently, after a Fail: .: openFile: inappropriate type (is a directory) exception from readFile, the file descriptor will not be closed, and instead be left open til the end o' time. it took me a long time to find because I thought the bug was in the listing of my directories, not the readFile's of its contents (including "." and ".."). John -- John Meacham - ⑆repetae.net⑆john⑈

1 0

RE: #ifdef considered harmful (was: DData)
by Simon Marlow 16 Apr '04

16 Apr '04

On 16 April 2004 12:26, Alastair Reid wrote: > What I'd like to see is a general facility for defining strict types. > That is, types whose values will never be thunks so let binding > causes evaluation (as in ML) and use as a subexpression (which is not > under a lambda) causes evaluation (as in ML). The GHC syntax seems > too restrictive for this. Maybe something like this: > > strict_data Int# = Int# Int -- declare a strict Int type > > The typechecking issues would be the same as for GHC's unboxed types: > there are two kinds of typevar * and # and never the twain shall > meet. (I think GHC plays some neat tricks to ease the pain of this > restriction?) Yes, GHC has a separate kind for unlifted types. Unlifted types have kind #, so you can't pass something of unlifted type to a polymorphic function or store it in a polymorphic data structure, because ordinary type variables have kind *. I think there's a small superkind relationship, so kind *# is a superkind of both * and #, which is useful for things like the return type of 'error'. Do you want your strict types to be truly unlifted? GHC could usefully take advantage of that knowledge. > A semantically correct (but maybe a bit slow) implementation would be > easy enough: > > 1) let-bind all strict non-atomic subexpressions. > 2) float strict variable bindings out to nearest enclosing lambda > (float out any lazy variable bindings if required) > 3) Use 'seq' to force all the strict var thunks. > > For example: > > insert :: a -> (Int#,[a]) -> (Int#,[a]) > > insert x (cnt,xs) = (cnt+1,x:xs) > ===> > insert x (cnt,xs) = let cnt'=cnt+1 in seq cnt' $ > (cnt',x:xs) You have some kind of implicit coercion between Int and Int#. How does the type system work here? In GHC you can't put Int# in a pair, of course. Cheers, Simon

1 0

DData
by JP Bernardy 16 Apr '04

16 Apr '04

Hello everyone, After a month or so spent on stabilizing DData, I think I've reached a point where it is hopefully acceptable for everyone. In any case, I don't feel like I can make it much better (within the self-imposed restrictions). Therefore, I hereby submit it to the people in charge of the hierarchical libraries, for integration. Of course, if any specific detail deserves work, I remain available. The code/doc is at the usual url: http://users.skynet.be/jyp/DData/doc/ http://users.skynet.be/jyp/DData/ddata.tar.gz Thank you all for your feedback, Cheers, JP. __________________________________ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/

18 40

RE: #ifdef considered harmful (was: DData)
by Simon Marlow 16 Apr '04

16 Apr '04

> "Simon Marlow" <simonmar(a)microsoft.com> writes: > > > Most of the time it's unnecessary to use an explicit Int#. > If your Int > > is in a data structure, you can use > > > > {-# UNPACK #-} !Int > > > > which is portable, and compiles to an unboxed Int in GHC >= 6.2. > > Is this different from > !Int > and > -funbox-strict > ? > > (I.e., should I change from this to UNPACK pragmas?) They both have the same effect, but I prefer to put these hints in the source code which is why I prefer to use {-# UNPACK #-} rather than the command-line flag. Also, sometimes it is necessary to have fine-grained control over which fields are unpacked. Cheers, Simon

1 0

RE: #ifdef considered harmful (was: DData)
by Simon Marlow 16 Apr '04

16 Apr '04

> Hmm, wouldn't it be nice to have a "FastInt" library which re-bound > the appropriate types and functions? Haskell libraries are > chock-a-block with instances of "#ifdef __GHC__" which define things > in terms of Int# instead of Int. We could simplify all those > libraries if we did it just once, in one place. > > The problem, of course, is that the naive non-GHC user could write a > library which used "FastInt" polymorphically, and write code that > wasn't portable to GHC. I don't think there's an easy, magic > solution > there. Perhaps such a library would be named > FastIntWhichIPromiseToTestWithGHC. Most of the time it's unnecessary to use an explicit Int#. If your Int is in a data structure, you can use {-# UNPACK #-} !Int which is portable, and compiles to an unboxed Int in GHC >= 6.2. However, depending on how you use the Int, GHC might need to insert reboxing code; e.g. if you store the Int in a list. The only reason to use an explicit Int# would be if you really wanted to be sure that no boxing was ever happening. The above trick is used in several places in the libraries currently: GHC's IO libraries and Data.HashTable are two good examples. Cheers, Simon

2 1

Re: [Haskell] ANNOUNCE: Haskell XML Toolbox Version 4.00
by Graham Klyne 15 Apr '04

15 Apr '04

Further to my previous message: http://www.haskell.org//pipermail/libraries/2004-April/002039.html I realize that my previous testing didn't check that HTTP retrieval actually works (as opposed to compiles). I find that the HTTP module shipped with HXml Toolbox does NOT work with Hugs under MS-Windows. I don't know why, but I do find that HXml Toolbox works fine with a version of HTTP that Bjorn Bringert has been working on, in my case by editing file 'hparser/ProtocolHandlerHttpNative.hs': [[ -- [GK] use version of HTTP prepared for hierarchical libraries, -- that works on Windows. import Network.HTTP -- http modules import Network.Browser ]] I had to make a couple of modifications to the HTTP module relative to Bjorn's: [[ -- ** Changes by Graham Klyne: -- - export httpVersion -- - use new URI module (similar to old, but uses revised URI datatype) ]] I also found a bug in the compatibility interface function "authority" in my URI module. The working HTTP and URI code can be found here: http://www.ninebynine.org/Software/HaskellUtils/Network/ I also noticed, in passing, that HXml Toolbox does require a version of Hugs that supports Char values greater than 255. I have versions of the 20040109 Hugs distribution recompiled for Windows with this support (and enlarged code space), which can be found at: http://www.ninebynine.org/Software/FatHugs.html #g ------------ Graham Klyne For email: http://www.ninebynine.org/#Contact

1 0

RE: sequences
by Simon Marlow 15 Apr '04

15 Apr '04

> Sven Panne writes: > > Grand Unified Collection Framework whenever it arrives. > Robert Will writes: > Since we already have a concrete proposal with prototype > implementation, that "whenever" may not be so far away. There are (at least) two concrete proposals with prototype implementations, and a large number of half-baked proposals for unified collection frameworks. None of them are obviously the right thing. It takes a large investment in order to fully evaluate any of the solutions, and I don't believe any of the proposals have received nearly enough scrutiny. That's not a problem as such: it just means that we won't arrive at a situation where there is agreement on what should be *the* Haskell collection framework. But what we should do is move in a direction where it is easier for people to install libraries on their machines and play around with them. Perhaps then there will be no need to determine a "standard" - a widely-used implementation may emerge on its own. At any rate, we're a long way off. Cheers, Simon

1 0

Proposal for a Standard of Abstract Collections (with Reference Implementation)
by Robert Will 15 Apr '04

15 Apr '04

hi all, Curiously I have finished the promised draft standard for the Abstract Collections just by a time, where many are busily working on DData. It's there: http://www.stud.tu-ilmenau.de/~robertw/dessy/fun/ I'm making this discussion hot now, since my Dessy library implements the proposed collection standard and features all of the concrete structures that are also in DData, and some more, especially Deques and Democratic Sequences (explanation: http://www.stud.tu-ilmenau.de/~robertw/dessy/fun/democratic/ ). But I also want to have your attention for another aspect of the problem: the introduction of abstract data structures makes the programming language much more useful and suggests some generalisations of the manifest syntax. Thus my proposal does not only contain - a formal specification for a collection hierarchy (based on Abstract Algebraic Data Structures (TM)), but also - generalisations of all the standard functions from Prelude and Data.List, and - a formal specification of Haskell extensions, that make abstract structures as easy to use as lists -- but with much more flexibility. Incidentally the current discussion around DData already shows signs of what I predict in the standard's rationale: when concrete data structures are extended to become more general, the number of functions increases so fast, that a general standard isn't more complicated any more, but simpler actually. For example, you discussed whether Sets and Maps should be converted from and to Sequences or the built-in "lists". Well, the best answer is to convert from and two an type classes of which "lists" (if one uses them at all) are an instance. Exactly this is done by the function "convert" proposed in the standard: > convert :: (Collection coll a, Collection coll' a) => > coll a -> coll' a > convert = fold empty single (<+>) (The function can be specialised in the same way as 'fromIntegral' on the numeric types, so we don't loose efficiency.) Abstraction is more: the proposed standard helps to use a lot of different data structure implementations together, using one interface that has been designed with the user first in mind, not the implementation. But this abstraction also allows an entirely new programming style, that is especially useful when TEACHING programming. Data Structures are deeply intertwined with programming as such, that the proposed standard gives Haskell something that NO other language does have today. (Perhaps Eiffel comes closest, but EiffelBase is not standardised.) (Remind me to write "A Gentle Introduction into Haskell with Abstract Collections".) It has been mentioned that an advantage is of functional programming is the ability to easily _compose_ programs. But for this composition data structures are neccessary, without a standard for data structures functional languages loose one of their biggest advantages. In specifications Sets are the most important data structure, but in Haskell today, the so-called "lists" are the linguaga franca and that reminds me too much of the stacks in Forth (remember "lists" are actually functional stacks) or dynamic arrays in imperative languages: it's universal, but unflexible. (I'll have to write a paper on "Liberating programming from the McCarthy-style".) Since "a standard of Abstract Collections" has already been mentioned by some, I expect that there are high expectations regarding my proposal, and perhaps its many disadvantages (all the big ones mentioned in the documentation) will discourage the reader. On the other hand, there are so many advantages that one can hardly neglect that something big is going on. So the least thing that someone interested in the future of (functional) programming should is to download and try out the Collections, or to read a little bit in the design documents. Things to do: - Build examples for the use of the new Collections. (From now on anyone can port his code to get rid of "lists".) - Develop the Haskell Documentation Standard and apply it to Collections. (Specifications are exported and used for Tests.) - Using those specifications make a complete test framework. (Complete meaning every function is tested with the complete specification, so no Bugs can persist.) Given all that, it is trivial to add more implementations of the Abstract Collections. (Just added two concrete structures Yesterday, although Dessy has already much more functionality than DData, but only half its code size. (Or one third if you don't count the type classes and specifications.) I also think that Dessy has more readable code, but I don't know how important that is for anyone.) Have a good time, Bob

9 21