November 2002 - Libraries

RE: getting a Binary module into the standard libs
by Simon Marlow 18 Dec '02

18 Dec '02

> (I'll try to > do some perf measurements on BinIO vs. BinMem later today, that should > give us a rough idea). I did a rough test using GHC's current Binary library hacked to be standalone. The test was writing 100000 Ints, followed by reading back 100000 Ints. Results: using BinMem took about 0.25 secs, BinIO took about 1.1 secs. That is, BinMem is current 4-5 times faster than BinIO. I should stress that this isn't a realistic benchmark. I conjecture the difference is mostly due to the overhead of going through the IO library each time a character is written. When I wrote the Binary library I was pretty careful to make sure that putWord8/getWord8 on a BinMem was just straight-line code (that's what the FastMutInt stuff is for). The upshot is that if you want really good performance for BinIO, there will have to be a cache in the BinHandle. Cheers, Simon

3 5

RE: getting a Binary module into the standard libs
by Simon Marlow 03 Dec '02

03 Dec '02

> > BinMem and BinIO differ quite a bit here: for BinMem you can write > > straight into the array, whereas for BinIO we need a cache > - a single > > byte at the least, but ideally more. BinMem is the most > important case > > to optimise (for us in GHC anyhow), since BinIO is already > significantly > > slower due to the overhead of the Handle interface. > > Currently the BinIO implementation doesn't do any caching, though, > right? I actually rarely use the BinMem implementation, so I would be > more interested in improving the speed of BinIO. Presumably, > however, the > Handle actually does buffering, which should take care of a > large part of this, right? The problem is the overhead of the Handle interface. hPutChar is quite expensive, because it has to lock/unlock the Handle. Using a cache in the BinIO implementation would help a lot, but it also means that you have to keep exclusive access to the Handle while doing binary operations. There is already a similar problem, because the library caches the file pointer outside the Handle, so someone doing an hSeek on the same Handle will cause the cached version to be out of sync with the real one. Should multiple threads be allowed to access the same BinHandle simultaneously? Probably not in write mode, but it might be useful when reading. Maybe we could provide dupBin :: BinHandle -> IO BinHandle this is easy enough to implement, and the separate BinHandles could be used from different threads. > > There should really be a closeBin function too; it's quite simple to > > add. > > On files this would flush the current byte then close the file handle, > right? What would it do on BinMems (other than flush the byte)? closeBin can throw away a BinMem. If it needs to be written to a file, then we provide writeBinMem. Cheers, Simon

3 3

RE: IOError vs. Exception vs. IOException
by Simon Marlow 27 Nov '02

27 Nov '02

> Just a thought, but what are the ramifications of taking the other > route, relaxing the haskell 98 spec to allow implementations to expand > the set of valid exceptions? a function can be used to > determine whether > it is something not covered by the haskell 98 spec (even if > we dont know > anything else about it..) and we can at least rethrow it or > 'show' it in > logs or whatever. lots of useful little functions can be written which > are haskell 98 but should be general enough to handle random > exceptions, > it would be silly to have to keep 2 versions around... <snip> It's difficult to "relax the Haskell 98 spec" in this way, since most compilers can be used in a strict Haskell 98 mode. Up to now this hasn't required using a different set of libraries, but if -fglasgow-exts changed the meaning of IO.catch then it would. Instead, I'd suggest deprecating System.IO.catch and perhaps the rest of the IOError support in System.IO. It is all available from System.IO.Error and Control.Exception anyhow. Cheers, Simon

5 4

Re: re-opening a closed stdin?
by Dean Herington 21 Nov '02

21 Nov '02

Bernard James POPE wrote: > [Dean wrote:] > Wouldn't you want the debugger to use stdin and stdout for its > interaction with the > > user, and run the object program with stdin and stdout handles of its own > > creation? (In that case your question seems moot.) I've always been very confused > > by debuggers that try to share stdin/stdout with the debugged program. > > Dean > > Hi Dean, > > You make a good point. > > One thing to note is that in my debugger the user's program is run to > completion before debugging begins. This means that the debugger does not need > to use stdin/stdout until after the user's program is finished with them. > > In a debugger that intersperses its own I/O with that of the user's program, > it would make a lot of sense to do as you say above. > > I (perhaps naively) thought that it would be simpler for my debugger to > reuse the stdin/stdout handles. > > Were you thinking of using a system call to dup, or something else. > > I am not a terminal expert, as you can probably guess, so > I'd appreciate any comments that you feel are appropriate. > > Thanks, > Bernie. Glynn Clements mentioned some good reasons why you shouldn't try to "reuse" stdin and stdout. For example, stdin or stdout (or both) could be connected to files, pipes, etc. If your debugger truly runs the user program to completion, there shouldn't be any problem: The operating system "reuses" "terminal"-connected stdin/stdout when a subprocess completes. So I suspect you need the user program process to stick around for post-"completion" debugging. In this case I still think the clean approach is to have the debugger create fresh stdin/stdout handles for execution of the user program. When the user program does want to do "terminal" input and/or output, on Unix I would first consider pseudo-terminals (man pty). Dean

1 0

RE: re-opening a closed stdin?
by Simon Marlow 21 Nov '02

21 Nov '02

> Simon Marlow writes: > > I've been thinking about duplicating/replacing Handles for a while. > > Here's a possible interface: > > > > -- |Returns a duplicate of the original handle, with its > own buffer > > -- and file pointer. The original handle's buffer is flushed, > > including > > -- discarding any input data, before the handle is duplicated. > > hDuplicate :: Handle -> IO Handle > > -- |Makes the second handle a duplicate of the first handle. The > > -- second handle will be closed first, if it is not already. > > hDuplicateTo :: Handle -> Handle -> IO () > > I'm not too sure of the issues here. Some examples that use > them would be helpful. Well, I can redirect stdout like this: > hPutStrLn stdout "foo" foo > h <- openFile "/tmp/foo" WriteMode > :!cat /tmp/foo > hPutStrLn h "bar" > :!cat /tmp/foo > hFlush h > :!cat /tmp/foo bar > hDuplicateTo h stdout > putStrLn "wibble" > :!cat /tmp/foo bar wibble > (note that it was GHCi that set stdout into NoBuffering mode after the hDuplicateTo). > The only suggestion I'd make is that the names be something > with handle in them: > > huDupHandle, hDupHandleTo Hmm, I chose the names to be consistent with the rest of the IO library, where anything beginning with 'h' is assumed to act on a Handle. > > The remaining questions are: > > - Should you be allowed to duplicate a Handle which refers > > to a file opened in WriteMode? Haskell 98 forbids having > > two Handles pointing to the same file opened for writing, > > but IMHO it's quite a reasonable thing to do. If we don't allow > > this, then there needs to be another version of hDuplicateTo > > which invalidates the original Handle. > > Why does Haskell 98 make this restriction (I don't think that > the library report says why)? One reason, I think, is lazy I/O(*). It's to stop you accidentally writing to a file which is being read from lazilly and getting unexpected results. Cheers, Simon (*) kill it! die! die!

1 0

RE: re-opening a closed stdin?
by Simon Marlow 21 Nov '02

21 Nov '02

> There's probably a really obvious answer to this, but I can't find it. > > Is there any way in GHC to reopen stdin if it has been closed? There's no way to do this at present, except in GHCi where you can revert CAFs to their unevaluated state. > You may wonder why I'd want this. Well I'm writing a debugger > for Haskell 98 (*) and my debugger wants to do some > interaction on the terminal > _after_ the user's program has run. If the user's program > puts stdin into > a closed or semi-closed state then that causes trouble for my > debugger. > > What I'd like to do is close stdin after the end of the > user's program, > flush any input waiting in the buffer, then reopen it fresh > for reading. > > If this can't be easily done perhaps there is another > solution you can think of. I've been thinking about duplicating/replacing Handles for a while. Here's a possible interface: -- |Returns a duplicate of the original handle, with its own buffer -- and file pointer. The original handle's buffer is flushed, including -- discarding any input data, before the handle is duplicated. hDuplicate :: Handle -> IO Handle -- |Makes the second handle a duplicate of the first handle. The -- second handle will be closed first, if it is not already. hDuplicateTo :: Handle -> Handle -> IO () These are fairly straightforward to implement in GHC (I've got a working prototype), and given these primitives you can implement the other interfaces that have been talked about in the past (withStdin/withStdout/withStderr, for example). The remaining questions are: - Should you be allowed to duplicate a Handle which refers to a file opened in WriteMode? Haskell 98 forbids having two Handles pointing to the same file opened for writing, but IMHO it's quite a reasonable thing to do. If we don't allow this, then there needs to be another version of hDuplicateTo which invalidates the original Handle. - Should these be exported by System.IO? Are there any fundamental problems which would make these hard or impossible to implement in Hugs or NHC? Cheers, Simon

2 1

RE: mapException
by Simon Peyton-Jones 18 Nov '02

18 Nov '02

There's no good reason that mapException isn't there. Would someone like to add it to Control.Exception, and document it? Something like mapException f v = unsafePerformIO (catch (evaluate v) (\x -> throwIO (f x))) The unsafePerformIO is, of course, safe in this case. Simon | -----Original Message----- | From: Lauri Alanko [mailto:la@iki.fi] | Sent: 16 November 2002 09:18 | To: libraries(a)haskell.org | Subject: mapException | | On Mon, Nov 04, 2002 at 01:00:39PM -0000, Simon Marlow wrote: | > and the awkward squad paper will have to be updated to comply (although | > we just noticed it is already wrong about the behaviour of | > Prelude.catch). | | This reminds me: in the paper "A semantics for imprecise exceptions" a | primitive "mapException" is introduced: | | mapException :: (Exception -> Exception) -> a -> a | | However, I don't think GHC's exception library has ever had this. Is | this just a lapse, or is there some fundamental problem with a non-IO | mapException? The paper says that "its usefulness and desirability might | be debatable", but to me this seems very useful. Eg. in Java this is a | common idiom: | | try { | aux.library.Thing.doSomething(); | } catch (aux.library.ThingException e) { | throw new MyException(e.getProblem()); | } | | That is, one module uses another module internally to do a task, but of | course wishes to hide any implementation details from the caller. | Therefore we don't want to show the auxiliary library's exception | interface to the caller, but convert exceptions to match the publicly | defined exception interface. | | It'd be very inconvenient if doing this in Haskell would require making | a formerly pure function use the IO monad. | | | Lauri Alanko | la(a)iki.fi | | _______________________________________________ | Libraries mailing list | Libraries(a)haskell.org | http://www.haskell.org/mailman/listinfo/libraries

1 0

RE: IOError vs. Exception vs. IOException
by Simon Marlow 16 Nov '02

16 Nov '02

> [redirecting to libraries] > Two (mostly) independent proposals: > > 1) Move bracket and bracket_ from System.IO (or GHC.Exception) to > haskell98/IO.hs. These two should now never be used anyway (except > in all-H98 programs), and this would save users of the new > libraries > from having to hide them. > > 2) Define > > type IOError = IOException -- was Exception > > (or vice versa), leave the type of Prelude.ioError as IOError -> IO a, > but add to Control.Exception > > throwIO :: Exception -> IO a > > If both are done, the only overlap seen by users of the new libraries > is Prelude.catch vs Control.Exception.catch and System.IO.try vs > Control.Exception.try. In each case there's a type distinction > reflecting the semantic distinction. Our feeling over here is that this is an unforced change, so in isolation it probably wouldn't be worthwhile. It'll break some code, and the awkward squad paper will have to be updated to comply (although we just noticed it is already wrong about the behaviour of Prelude.catch). However, if there's concensus that folk would prefer the alternate definition of IOError, then we're happy to go along with it. Cheers, Simon

5 5

RE: Instances in libraries (e.g., Control.Monad)
by Simon Marlow 15 Nov '02

15 Nov '02

I said, accidentally: > IMO, given the current design of Haskell 98, instances should be > considered to be global. There is only room for one instance for each > type/class pair in any program (including the libraries). > Otherwise the > library documentation has to publish for each module, a list > of all the > instances defined by it and all modules below it in the > dependency tree. > I think this is just silly. Of course, the library documentation still has to publish the list of modules from which an instance can be imported. I meant to delete the above paragraph before sending the msg. Please disregard. Cheers, Simon

1 0

RE: Instances in libraries (e.g., Control.Monad)
by Simon Marlow 15 Nov '02

15 Nov '02

> As noted elsewhere, we've been running into problems with Haskell's > "fine" handling of instances while trying to combine some largish > Haskell projects. As long as the language isn't changed, this > problem needs to be accounted for in library designs. > > Concrete examples: Control.Monad.Error, Control.Monad.Reader A meta point: as far as I'm concerned, the design of any library that is marked "experimental" (see the top of the page for each module in the Haddock docs) is up for grabs. Come up with a proposal, and if there's a concensus it goes in. (all the Control.Monad.* libraries are marked experimental, BTW). > These modules define lots of instances for (Either a) and ((->) a). > Why? Obviously, they want to use them, but that's no excuse. > > Both Either and (->) are very popular type constructors, and > currently, every project that wants to use these instances defines > them somewhere. Result: as soon as you put two of these projects > together, you get a clash! > > In current Haskell, defining instances of public classes for public > types/constructors has side-effects running through the whole of any > program using such a library. Quite apart from the facts that the > module names typically gives no indication of these side-effects, > and that the modules could have been implemented without them (using > standard types/constructors was just more convenient), there's no > way to get two such modules to work together in a larger project > without changing them! IMO, given the current design of Haskell 98, instances should be considered to be global. There is only room for one instance for each type/class pair in any program (including the libraries). Otherwise the library documentation has to publish for each module, a list of all the instances defined by it and all modules below it in the dependency tree. I think this is just silly. If the library defines a particular instance, then you can't redefine it. (interesting bits snipped) > ShowFunctions.hs would have been an example of this (where did > that go, btw?). Text.Show.Functions, of course ;-) (more interesting bits snipped) > PS. perhaps this suggestion got lost in the other thread, but > could Haddock please show the source of any instances in the > generated documentation? Showing the source of each instance isn't enough. You need to list each module that exports it. That includes every module which depends, directly or indirectly, on the module that defines it. I *really* don't want to do this - every module in the Haddock docs is going to have a huge list of irrelevant instances attached to it. I could put them in a separate page, but it's still a pain. While correct, this is going to be quite unwieldy. But there's no getting away from the fact that you still need to know which module to import in order to gain access to a particular instance. Cheers, Simon

1 0