November 2002 - Glasgow-haskell-users

RE: Native Threads in the RTS
by Simon Marlow 27 Nov '02

27 Nov '02

Alastair Reid wrote: > Design > ~~~~~~ > > Haskell threads may be associated at thread creation time with either > zero or one native threads. There are only two ways to create Haskell > threads so there are two cases to consider: Umm, Alastair, I think you've got things a bit mixed up here. Did you mean two ways to create a native thread? There are currently three ways to create a Haskell thread (forkIO, foreign export, finalizers) and Wolfgang has proposed a fourth (forkNativeThread). Or perhaps you're proposing we do away with forkIO and only have forkNativeThread? That was discounted a while back because we want to continue to have light-weight threads independent of OS threads. > 1) forkNativeThread :: IO () -> IO () > > The fresh Haskell thread is associated with a fresh native thread. > > (ToDo: do we really need to use a fresh native thread or would a > pool of threads be ok? The issue could be avoided by separating > creation of the native thread from the 'associate' operation.) > > 2) Calls to a threadsafe foreign export allocate a fresh Haskell > thread which is then associated with the Haskell thread. I don't know what you mean by a "threadsafe foreign export". Did you mean "threadsafe foreign import" perhaps? And I'm not sure how a fresh Haskell thread is associated with a Haskell thread ;-) > Calls to threadsafe foreign imports by threads which have an > associated native thread are performed by that native thread. > > Calls to any other foreign imports (i.e., 'safe' or 'unsafe' calls) > may be made in other threads or, if it exists, in the associated > native thread at the implementation's discretion. > > [ToDo: can Haskell threads with no associated thread make foreign > calls using a thread associated with some other thread? er... overloading the word "thread" was maybe not such a good idea. I think you're asking whether a green thread can grab a native thread to make a foreign call. The current answer is no... but can you think of a reason we might want this feature? Cheers, Simon

4 4

Template Haskell
by Mike Thomas 27 Nov '02

27 Nov '02

Hi there. Could somebody please let me know where I've gone wrong in the program below (yesterday's CVS HEAD stage 3 compiler on Windows)? ------------- TH - printf.hs --- module Main where import Language.Haskell.THSyntax data Format = D | S | L String main = putStrLn ( $(pr "Hello") ) parse :: String -> [Format] parse s = [ L s ] gen :: [Format] -> Expr gen [D] = [| \n -> show n |] gen [S] = [| \s -> s |] gen [L s] = string s pr :: String -> Expr pr s = gen (parse s) ------------- Command Line ----- /c/cvs/i386-unknown-mingw32/stage3/ghc/compiler/ghc-inplace -fglasgow-exts - package haskell-src printf.hs -o printf.exe ------------- GHC output--------- printf.hs:7: Stage error: `pr' is bound at stage 1 but used at stage 0 In the first argument of `putStrLn', namely `($[splice](pr "Hello"))' In a right-hand side of function `main': putStrLn ($[splice](pr "Hello")) In the definition of `main': main = putStrLn ($[splice](pr "Hello")) ----------------------------------- Thanks Mike Thomas.

1 0

Re: Native Threads in the RTS
by Wolfgang Thaller 26 Nov '02

26 Nov '02

Nice design, Alastair. I've stolen lots of ideas and some text for the complete rewrite of the proposal. The concept of "associating" haskell threads to native threads proved to be a good way of explaining my original idea in a different way --- and then I found out that forkNativeThread needn't be a primitive, but can be implemented on top of the FFI. After that I found out what the bound/free exports buisiness was all about and why we might need it. As for the questions regarding my previous proposal, I think they are answered in the new one. If they aren't, let me know. [It's no use explaining what I meant to say if I already want to say something different ;-) ] Cheers, Wolfgang =============================== Threads Proposal, version 3 Goals ~~~~~ Since foreign libraries sometimes exploit thread local state, it is necessary to provide some control over which thread is used to execute foreign code. In particular, it is important that it should be possible for Haskell code to arrange that a sequence of calls to a given library are performed by the same native thread and that if an external library calls into Haskell, then any outgoing calls from Haskell are performed by the same native thread. This specification is intended to be implementable both by multithreaded Haskell implementations and by single-threaded implementations and so it does not comment on which particular OS thread is used to execute Haskell code. Design ~~~~~~ Haskell threads may be associated at thread creation time with either zero or one native threads. Each Native thread is associated with zero or one Haskell threads (no native thread may be associated with two Haskell threads at a time). An associated pair of a Haskell thread and a native thread can only execute either foreign code or haskell code at any one time. The thread that main runs in, threads created using forkIO and threads created for running finalizers or signal handlers are not associated with a native thread [Actually, we might make this an implementation detail: These Haskell threads are not guaranteed to be associated with a native thread. If it makes sense for some implementation, all threads might be associated with native threads]. There are now two kinds of foreign exported [and foreign import wrapped] functions: bound and free [I'm not happy with these names, for the same reasons as given by Seth before]. When a "bound" foreign exported function is invoked [by foreign code], the implementation checks whether a Haskell thread is associated with the current OS thread. If there is one, this Haskell thread is used to execute the callback. If there is none, a new Haskell thread is created and associated with the native thread. This is the only situation where a Haskell thread is associated with a native thread. The new associated Haskell thread is then used to execute the callback. When the callback finishes, the Haskell thread is terminated, the association is dissolved, but the OS thread continues to run. When a "free" foreign exported function is invoked, the implementation may freely choose what Haskell thread the function is executed in. It is not specified whether this thread is associated with a particular OS thread or not. When a foreign imported function is invoked [by Haskell code], the foreign code is executed in the native thread associated with the current Haskell thread, if an association exists. If the current Haskell thread is not associated to a native thread, the implementation may freely decide which thread to run the foreign function in. This must be done for all foreign imports, including "unsafe". An unsafe call must not call back to Haskell or otherwise cause a garbage collection. Other Haskell threads may be blocked while the unsafe call executes. A safe call allows callbacks to Haskell. Other Haskell threads may be blocked while the unsafe call executes. A threadsafe call additionally guarantees other Haskell threads not to block. forkNativeThread :: IO () -> IO ThreadID may be implemented using the FFI and an OS-specific thread creation routine. Issues ~~~~~~ Finalizers and signal handlers cannot be associated with a particular native thread. If they have to trigger an action in a particular native thread, a message has to be sent manually (via MVars and friends) to the Haskell thread associated with the native thread in question. This might be tedious, but if we want to avoid the nastiness of multiple Haskell threads associated with one OS thread (lots of unpredictable blocking), it looks like our only choice.

1 0

foldl' ?
by Mark Carroll 26 Nov '02

26 Nov '02

Where do I find foldl' in GHC? It's mentioned on http://www.haskell.org/ghc/docs/latest/html/base/Data.List.html but importing List and using "-package data" don't seem to make it appear. I'm using GHC 5.02.2. I must be making some simple mistake. -- Mark

3 5

RE: Native Threads in the RTS
by Simon Marlow 26 Nov '02

26 Nov '02

> ***************** > Native Threads Proposal, version 2 > > Some "foreign" libraries (for example OpenGL) rely on a mechanism > called thread-local storage. The meaning of an OpenGL call therefore > usually depends on which OS thread it is called from. > Therefore, some > kind of direct mapping from Haskell threads to OS threads is > necessary > in order to use the affected foreign libraries. > Executing every haskell thread in its own OS thread is not > feasible for > performance reasons. However, perfomance of native OS threads is not > too bad as long as there aren't too many, so I propose that some > threads get their own OS threads, and some don't: > > Every Haskell Thread can be either a "green" thread or a "native" > thread. > For each "native" thread, there is exactly one OS thread > created by the > RTS. Is there an invariant that there is only one Haskell thread for each native thread? > For a green thread, it is unspecified which OS thread it is > executed in. > The main program and all haskell threads forked using forkIO > are green > threads. Threads forked using forkNativeThread :: IO () -> IO () are > native threads. (Note: The type of the current haskell thread does > _not_ matter when forking new threads) > > Execution of a green thread might move from one OS thread to > another at > any time. A "green" thread is never executed in an OS thread that is > reserved for a "native" thread. > A "native" haskell thread and all foreign imported functions that it > calls are executed in its associated OS thread. A foreign exported > callback that is called from C code executing in that OS thread is > executed in the native haskell thread. > A foreign exported callback that is called from C code > executing in an > OS thread that is not associated with a "native" haskell thread is > executed in a new green haskell thread. > > Only one OS thread can execute Haskell code at any given time. > > If a "native" haskell thread enters a foreign imported > function that is > marked as "safe" or "threadsafe", all other Haskell threads keep > running. If the imported function is marked as "unsafe", no other > threads are executed until the call finishes. I'd change this paragraph as follows: If a "native" haskell thread enters a foreign imported function that is marked as "threadsafe", all other Haskell threads keep running. If the imported function is marked as "safe" or "unsafe", no other threads are executed until the call finishes. in other words, foreign imports for native threads behave just like they do for green threads. As Simon P.J. pointed out, we don't do any OS-thread synchronisation on a "safe" foreign import. > Other things I'm not sure about: > What should we do get if a foreign function spawns a new OS > thread and > executes a haskell callback in that OS thread? Should a new native > haskell thread that executes in the OS thread be created? Should the > new OS thread be blocked and the callback executed in a green > thread? > What does the current threaded RTS do? (I assume the > non-threaded RTS will just crash?) The Haskell callback will execute in a green thread, according to the proposal above, and I think that's the right thing to do. It's possible to get into a situation in which several Haskell threads in the system are associated with the same OS thread: - Haskell native thread calls a foreign import, which - invokes a Haskell callback, which - calls a foreign import, which - invokes another Haskell callback at this point the two Haskell threads created by the callbacks are both attached to the same native thread. One of them is blocked waiting for its foreign import to return, however. I'm not sure if this is important. Cheers, Simon

1 0

RE: Native Threads in the RTS
by Simon Peyton-Jones 26 Nov '02

26 Nov '02

| I've now written up a slightly more formal proposal for native threads. | (OK, it's only a tiny bit more formal...) | I doubt I have explained everything clearly, please tell me which | points are unclear. And of course please tell me what you like/don't | like about it. Great, thanks. I hope you'll keep it up to date so that by the time the discussion converges it can serve as a specification and rationale. We can put it in CVS too... Simon will think of where! Ultimately it'd be worth integrating with http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/rts-libs/multi-thread. html Simon | A "native" haskell thread and all foreign imported functions that it | calls are executed in its associated OS thread. This part is ok | A foreign exported | callback that is called from C code executing in that OS thread is | executed in the native haskell thread. This is the bit I don't understand. Is the only scenario you have in mind here native Haskell thread calls C which calls Haskell and you want all that in the same native thread? What about this? native Haskell thread calls C which installs a pointer to a foreign-exported Haskell function in some C data structure Later... some other Haskell thread calls C which waits for an event which calls the callback So the callback was installed by a native thread, but won't be executed by it. Is that ok? Anyway I think it would be worth explaining what is guaranteed a bit more clearly. | If a "green" haskell thread enters a foreign imported function marked | as "safe", all other green threads are blocked. Native haskell threads | continue to run in their own OS threads. No, I don't think so. The reason that 'safe' is cheaper than 'threadsafe' is that the current worker OS thread does not need to release the Big Lock it holds on the Haskell heap, thereby allowing other green threads to run. Instead, it holds the lock, executes the call, and returns. At least I think this is the idea, but it's all jolly slippery. | Other things I'm not sure about: Presumably if a native thread spawns a thread using forkIO, it gets just a green thread? If it used forkNativeThread it gets a distinct native thread. Better say this.

6 8

RE: Template metaprogramming for Haskell article.
by Simon Peyton-Jones 26 Nov '02

26 Nov '02

At the moment you have to build from source to get it, or you can grab a development snapshot binary, from http://haskell.cs.yale.edu/ghc/download.html We're swithering about when to do a full release. Template Haskell is still moving, and making a release takes time. We'll be strongly affected by GHC users on this: if you really want TH and the snapshot route or build-from-source don't work for you, let us know. Simon | -----Original Message----- | From: Immanuel Litzroth [mailto:immanuell@enfocus.be] | Sent: 25 November 2002 12:57 | To: glasgow-haskell-users(a)haskell.org | Subject: Template metaprogramming for Haskell article. | | | I am reading the article template meta-programming for haskell and am | wondering if | 1) this is already implemented in ghc. | 2) if not when this will be released | Thanks in advance | Immanuel | | ******************************************************************** | ******* | His endeavours to improve these conditions by innovations in the | current methods of teaching are fully described in his writings; these | made him unpopular with his brethren in the order and led to his | removals to the monastery of Pomposa near Ferrara, Italy. Here the | same lot seems to have befallen him. | About Guido of Arezzo | | Immanuel Litzroth | Software Development Engineer | Enfocus Software | Kleindokkaai 3-5 | B-9000 Gent | Belgium | Voice: +32 9 269 23 90 | Fax : +32 9 269 16 91 | Email: Immanuell(a)enfocus.be | web : www.enfocus.be | ******************************************************************** | ******* | | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0

RE: RFC: External library infrastructure
by Simon Marlow 26 Nov '02

26 Nov '02

> I'd like to throw the following into the pot: > > The Hugs and GHC developers work pretty hard to keep the two > compilers compatible. For example, the next Hugs release will ship > with libraries from the same source tree as GHC uses and the same > foreign function interface as GHC and the GHC folk recently gave up > a feature they dearly wanted in the foreign function interface in > the name of portability. > > So as people try to come up with a distribution and build mechanism > that will work for GHC, it would be good to think about how that > same mechanism would work for Hugs too. Absolutely. I didn't mean to sound so GHC-centric. It would be great if the same infrastructure supports multiple compilers/interpreters. > For example, an extension > of GHC's existing package mechanism might work well because it would > be easy for Hugs to extract the pieces it needs and the package > mechanism has little irrelevant stuff. In contrast a variation on > the existing Makefile system would work poorly because the > complexity for the library developer of buying into the > infrastructure far exceeds the complexity of the task (which for > Hugs is overcoming a few minor platform dependencies using autoconf > or the like, building any foreign function interface libraries, and > overcoming a few remaining Hugs-GHC incompatabilities). Could you be more concrete? What extension of the package mechanism did you have in mind? (personally I had in mind a standard autoconf + Makefiles story for the build system, but I'm sure there are better ways). It'll be tricky to balance the requirements of a glorious all-encompassing infrastructure with keeping a low "buy-in" cost (as you put it), but that must be one of the goals or we won't have achieved anything. > We might also want to develop a common testing infrastructure for > libraries which, ideally, would work with all compilers. Right on. I'd like to propose we start from GHC's current testing system. It's a small collection of Python scripts (about 900 lines), and is designed to be compiler-independent, although I haven't used it with anything except GHC yet. Cheers, Simon

4 4

Re: re-opening a closed stdin?
by Albert Lai 25 Nov '02

25 Nov '02

"Simon Marlow" <simonmar(a)microsoft.com> writes: > It is nice, but it introduces too many problems. What happens to any > I/O errors encountered by the lazy I/O? They have to be discarded, > which means you can't effectively use lazy I/O for robust applications > anyway. Damn, I forgot that "file" means file in the real world, not Platonic file! I now agree that it is actually not nice.

1 0

-O doesn't work with gcc-3.2?
by Anatoli Tubman 25 Nov '02

25 Nov '02

ghc fails with this message: Prologue junk?: .globl __stginit_Main __stginit_Main: pushl %ebp movl %esp, %ebp It works without -O. I'm on stock Mandrake 9.0 system. How do I overcome this? tia -- anatoli

3 3