November 2002 - Glasgow-haskell-users

re-opening a closed stdin?
by Bernard James POPE 21 Nov '02

21 Nov '02

Hi all, There's probably a really obvious answer to this, but I can't find it. Is there any way in GHC to reopen stdin if it has been closed? You may wonder why I'd want this. Well I'm writing a debugger for Haskell 98 (*) and my debugger wants to do some interaction on the terminal _after_ the user's program has run. If the user's program puts stdin into a closed or semi-closed state then that causes trouble for my debugger. What I'd like to do is close stdin after the end of the user's program, flush any input waiting in the buffer, then reopen it fresh for reading. If this can't be easily done perhaps there is another solution you can think of. (*) www.cs.mu.oz.au/~bjpop/buddha/ Cheers, Bernie.

4 4

Re: re-opening a closed stdin?
by Dean Herington 21 Nov '02

21 Nov '02

Bernard James POPE wrote: > [Dean wrote:] > Wouldn't you want the debugger to use stdin and stdout for its > interaction with the > > user, and run the object program with stdin and stdout handles of its own > > creation? (In that case your question seems moot.) I've always been very confused > > by debuggers that try to share stdin/stdout with the debugged program. > > Dean > > Hi Dean, > > You make a good point. > > One thing to note is that in my debugger the user's program is run to > completion before debugging begins. This means that the debugger does not need > to use stdin/stdout until after the user's program is finished with them. > > In a debugger that intersperses its own I/O with that of the user's program, > it would make a lot of sense to do as you say above. > > I (perhaps naively) thought that it would be simpler for my debugger to > reuse the stdin/stdout handles. > > Were you thinking of using a system call to dup, or something else. > > I am not a terminal expert, as you can probably guess, so > I'd appreciate any comments that you feel are appropriate. > > Thanks, > Bernie. Glynn Clements mentioned some good reasons why you shouldn't try to "reuse" stdin and stdout. For example, stdin or stdout (or both) could be connected to files, pipes, etc. If your debugger truly runs the user program to completion, there shouldn't be any problem: The operating system "reuses" "terminal"-connected stdin/stdout when a subprocess completes. So I suspect you need the user program process to stick around for post-"completion" debugging. In this case I still think the clean approach is to have the debugger create fresh stdin/stdout handles for execution of the user program. When the user program does want to do "terminal" input and/or output, on Unix I would first consider pseudo-terminals (man pty). Dean

1 0

RE: re-opening a closed stdin?
by Simon Marlow 21 Nov '02

21 Nov '02

> > You can call 'System.Posix.IO.dup stdin' and save this value. > > However, I think you then need to explicitely read from this fd as > > it is not possible to reset what GHC thinks stdin is currently to > > this new fd (I'll dig into this and maybe we'll get a > > setStdin :: Fd -> IO () > > from this, IIRC somebody else was asking for this, too). > > How does this interact with Simon's proposal for hDuplicate? If you actually want to change what stdin/stdout/stderr refer to, or re-open one of these file descriptors after it has been closed, then you need hDuplicateTo. I've enclosed a version of these functions which works with GHC 5.04.1. Cheers, Simon

1 0

RE: re-opening a closed stdin?
by Simon Marlow 21 Nov '02

21 Nov '02

> Simon Marlow writes: > > I've been thinking about duplicating/replacing Handles for a while. > > Here's a possible interface: > > > > -- |Returns a duplicate of the original handle, with its > own buffer > > -- and file pointer. The original handle's buffer is flushed, > > including > > -- discarding any input data, before the handle is duplicated. > > hDuplicate :: Handle -> IO Handle > > -- |Makes the second handle a duplicate of the first handle. The > > -- second handle will be closed first, if it is not already. > > hDuplicateTo :: Handle -> Handle -> IO () > > I'm not too sure of the issues here. Some examples that use > them would be helpful. Well, I can redirect stdout like this: > hPutStrLn stdout "foo" foo > h <- openFile "/tmp/foo" WriteMode > :!cat /tmp/foo > hPutStrLn h "bar" > :!cat /tmp/foo > hFlush h > :!cat /tmp/foo bar > hDuplicateTo h stdout > putStrLn "wibble" > :!cat /tmp/foo bar wibble > (note that it was GHCi that set stdout into NoBuffering mode after the hDuplicateTo). > The only suggestion I'd make is that the names be something > with handle in them: > > huDupHandle, hDupHandleTo Hmm, I chose the names to be consistent with the rest of the IO library, where anything beginning with 'h' is assumed to act on a Handle. > > The remaining questions are: > > - Should you be allowed to duplicate a Handle which refers > > to a file opened in WriteMode? Haskell 98 forbids having > > two Handles pointing to the same file opened for writing, > > but IMHO it's quite a reasonable thing to do. If we don't allow > > this, then there needs to be another version of hDuplicateTo > > which invalidates the original Handle. > > Why does Haskell 98 make this restriction (I don't think that > the library report says why)? One reason, I think, is lazy I/O(*). It's to stop you accidentally writing to a file which is being read from lazilly and getting unexpected results. Cheers, Simon (*) kill it! die! die!

1 0

RE: re-opening a closed stdin?
by Simon Marlow 21 Nov '02

21 Nov '02

> There's probably a really obvious answer to this, but I can't find it. > > Is there any way in GHC to reopen stdin if it has been closed? There's no way to do this at present, except in GHCi where you can revert CAFs to their unevaluated state. > You may wonder why I'd want this. Well I'm writing a debugger > for Haskell 98 (*) and my debugger wants to do some > interaction on the terminal > _after_ the user's program has run. If the user's program > puts stdin into > a closed or semi-closed state then that causes trouble for my > debugger. > > What I'd like to do is close stdin after the end of the > user's program, > flush any input waiting in the buffer, then reopen it fresh > for reading. > > If this can't be easily done perhaps there is another > solution you can think of. I've been thinking about duplicating/replacing Handles for a while. Here's a possible interface: -- |Returns a duplicate of the original handle, with its own buffer -- and file pointer. The original handle's buffer is flushed, including -- discarding any input data, before the handle is duplicated. hDuplicate :: Handle -> IO Handle -- |Makes the second handle a duplicate of the first handle. The -- second handle will be closed first, if it is not already. hDuplicateTo :: Handle -> Handle -> IO () These are fairly straightforward to implement in GHC (I've got a working prototype), and given these primitives you can implement the other interfaces that have been talked about in the past (withStdin/withStdout/withStderr, for example). The remaining questions are: - Should you be allowed to duplicate a Handle which refers to a file opened in WriteMode? Haskell 98 forbids having two Handles pointing to the same file opened for writing, but IMHO it's quite a reasonable thing to do. If we don't allow this, then there needs to be another version of hDuplicateTo which invalidates the original Handle. - Should these be exported by System.IO? Are there any fundamental problems which would make these hard or impossible to implement in Hugs or NHC? Cheers, Simon

2 1

RE: -Werror Request
by Simon Peyton-Jones 20 Nov '02

20 Nov '02

Done! SImon | -----Original Message----- | From: Ashley Yakeley [mailto:ashley@semantic.org] | Sent: 14 November 2002 02:25 | To: GHC List | Subject: -Werror Request | | If it's not too much work, I'd like to request a "-Werror" option for GHC | that would turn warnings into errors. Sometimes warnings one would like | to catch get lost in a long make process. | | Thanks... | | -- | Ashley Yakeley, Seattle WA | | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

2 1

RE: adding new primitives to ghci...
by Simon Marlow 20 Nov '02

20 Nov '02

> I am trying to add a new primitive to ghc (5.04.1) ... > I followed > the instructions in primops.txt.pp: > > -- - or, for an out-of-line primop: > -- ghc/includes/PrimOps.h (just add the declaration) > -- ghc/rts/PrimOps.hc (define it here) > -- ghc/rts/Linker.c (declare the symbol for GHCi) > -- > > My new primitive works fine when I compile the programs that > use it but when I try to load ghci I get: > > lxdsg[137](2.05)> ./ghc-inplace --interactive > ___ ___ _ > / _ \ /\ /\/ __(_) > / /_\// /_/ / / | | GHC Interactive, version 5.04.1, > for Haskell > 98. > / /_\\/ __ / /___| | http://www.haskell.org/ghc/ > \____/\/ /_/\____/|_| Type :? for help. > > Loading package base ... linking ... > /data/dubois/ghc-5.04.1/libraries/base/HSbase.o: unknown symbol > `packcczh_fast' > ghc-5.04.1: panic! (the `impossible' happened, GHC version 5.04.1): > can't load package `base' > > Please report it as a compiler bug to > glasgow-haskell-bugs(a)haskell.org, > or http://sourceforge.net/projects/ghc/. You need to bootstrap GHCi with the compiler that contains the new primitive. Cheers, Simon

1 0

adding new primitives to ghci...
by Andre Rauber Du Bois 20 Nov '02

20 Nov '02

Hello! I am trying to add a new primitive to ghc (5.04.1) ... I followed the instructions in primops.txt.pp: -- - or, for an out-of-line primop: -- ghc/includes/PrimOps.h (just add the declaration) -- ghc/rts/PrimOps.hc (define it here) -- ghc/rts/Linker.c (declare the symbol for GHCi) -- My new primitive works fine when I compile the programs that use it but when I try to load ghci I get: lxdsg[137](2.05)> ./ghc-inplace --interactive ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | | GHC Interactive, version 5.04.1, for Haskell 98. / /_\\/ __ / /___| | http://www.haskell.org/ghc/ \____/\/ /_/\____/|_| Type :? for help. Loading package base ... linking ... /data/dubois/ghc-5.04.1/libraries/base/HSbase.o: unknown symbol `packcczh_fast' ghc-5.04.1: panic! (the `impossible' happened, GHC version 5.04.1): can't load package `base' Please report it as a compiler bug to glasgow-haskell-bugs(a)haskell.org, or http://sourceforge.net/projects/ghc/. So, is there any other file that I have to change to have my new primitive working with ghci? Cheers, Andre. -- --------------------------------- André Rauber Du Bois dubois at macs.hw.ac.uk http://www.macs.hw.ac.uk/~dubois/

1 0

Re: Native Threads in the RTS
by Wolfgang Thaller 19 Nov '02

19 Nov '02

I've now written up a slightly more formal proposal for native threads. (OK, it's only a tiny bit more formal...) I doubt I have explained everything clearly, please tell me which points are unclear. And of course please tell me what you like/don't like about it. I have some rough ideas on how to implement the proposal. I would be ready to invest some time, but I don't have enough free time to make any promises here. The discussion has to be finished first, anyway. Cheers, Wolfgang ******************* Native Threads Proposal, version 1 Some "foreign" libraries (for example OpenGL) rely on a mechanism called thread-local storage. The meaning of an OpenGL call therefore usually depends on which OS thread it is called from. Therefore, some kind of direct mapping from Haskell threads to OS threads is necessary in order to use the affected foreign libraries. Executing every haskell thread in its own OS thread is not feasible for performance reasons. However, perfomance of native OS threads is not too bad as long as there aren't too many, so I propose that some threads get their own OS threads, and some don't: Every Haskell Thread can be either a "green" thread or a "native" thread. For each "native" thread, there is exactly one OS thread created by the RTS. For a green thread, it is unspecified which OS thread it is executed in. The main program and all haskell threads forked using forkIO are green threads. Threads forked using forkNativeThread :: IO () -> IO () are native threads. Execution of a green thread might move from one OS thread to another at any time. A "green" thread is never executed in an OS thread that is reserved for a "native" thread. A "native" haskell thread and all foreign imported functions that it calls are executed in its associated OS thread. A foreign exported callback that is called from C code executing in that OS thread is executed in the native haskell thread. A foreign exported callback that is called from C code executing in an OS thread that is not associated with a "native" haskell thread is executed in a new green haskell thread. Only one OS thread can execute Haskell code at any given time. If a "native" haskell thread enters a foreign imported function that is marked as "safe" or "threadsafe", all other Haskell threads keep running. If the imported function is marked as "unsafe", no other threads are executed until the call finishes. If a "green" haskell thread enters a foreign imported function marked as "threadsafe", a new OS thread is spawned that keeps executing other green haskell threads while the foreign function executes. Native haskell threads continue to run in their own OS threads. If a "green" haskell thread enters a foreign imported function marked as "safe", all other green threads are blocked. Native haskell threads continue to run in their own OS threads. If the imported function is marked as "unsafe", no other threads are executed until the call finishes. Finalizers are always run in green threads. Issues deliberately not addressed in this proposal: Some people may want to run several Haskell threads in a dedicated OS thread (this is what has been called "thread groups" before). Some people may want to run finalizers in specific OS threads (are finalizers predictable enough for this to be useful?). Everyone would want SMP if it came for free (but SMP seems to be too hard to do at the moment...) Other things I'm not sure about: What should we do get if a foreign function spawns a new OS thread and executes a haskell callback in that OS thread? Should a new native haskell thread that executes in the OS thread be created? Should the new OS thread be blocked and the callback executed in a green thread? What does the current threaded RTS do? (I assume the non-threaded RTS will just crash?)

1 0

RE: Strings are slow
by Simon Marlow 19 Nov '02

19 Nov '02

> module Main where > > import Text.Regex > import System.IO > import System.Environment > import Control.Monad > import Data.Maybe > > main = do [re] <- getArgs > let rx = mkRegex re > let loop = do line <- getLine > when (isJust (matchRegex rx line)) > (putStrLn line) > eof <- isEOF > unless eof loop > loop > > > It turned out that this is remarkably slow. The first problem was with > inlining. If this is compiled with ghc-5.04 -O -ddump-simpl, I get: > > case GHC.IOBase.unsafePerformIO > @ (Data.Maybe.Maybe > (GHC.Base.String, > GHC.Base.String, > GHC.Base.String, > [GHC.Base.String])) > (Text.Regex.Posix.regexec > (Text.Regex.mkRegex re) a731) > of wild4 { This is indeed an optimiser bug, but it's the result of a design decision: GHC is a bit laid back about inlining things inside the state lambda in the IO monad, because it often enables important optimisations. However, we're experimenting with modifying this "optimisation" so that it will be less likely to kill performance in the way it did in your example. In the meantime, you can add rx as an argument to loop, that will be enough to fool GHC into not inlining rx. > Ie. the regex is compiled anew every time a string is matched. A bug? > > Anyway, without optimization the code produced is reasonable, > but still > horrendously slow. Testing with a simple word as a pattern > from a 7.3MB, > 800kline file, the running time was 37.5 seconds. For > comparison, a similar > program in mzscheme (interpreted!) took 7.3 seconds while the system > grep, of course, took 0.4 seconds. > > I did some profiling by creating new top-level bindings for matchRegex > and getLine (is there a better way?): > > > total time = 53.34 secs (2667 ticks @ 20 ms) > total alloc = 1,172,482,496 bytes (excludes > profiling overheads) > > COST CENTRE MODULE %time %alloc > > match Main 69.7 56.8 > getl Main 23.9 40.2 > main Main 6.3 3.0 > > > So it seems like all the time is spent just converting ByteArrays to > char lists to C arrays. This makes me wonder how sensible it really is > to represent strings by char lists. Yes, it's nice and > uniform and lazy, but... String processing in Haskell is very slow, due to the list-of-characters representation. A more complete PackedString library with better integration with other libraries (like Text.Regex) would help a lot for these kind of examples. Cheers, Simon

1 0