RE: [Haskell-cafe] Re: FFI and callbacks

On 20 July 2005 18:49, John Goerzen wrote:
On 2005-07-20, Simon Marlow
wrote: This paper might help shed some light:
Forgot to reply to this. *Very helpful* link, I had always wondered what the bound thread functions in Control.Concurrent were for :-)
So let me see if I understand this correctly.
Let's say that I have:
* A program that uses multiple lightweight Haskell threads (all started with plain forkIO calls)
* An event-driven C library, not not thread-aware, with a blocking main loop
* GHC 6.4
* My C calls all imported "safe".
Now then, if I understand this correctly, that a call to the C main loop will create a new bound OS thread, so it will not interrupt any other forkIO'd threads in Haskell.
Not necessarily a *bound* OS thread. A bound thread is only created by an in-call to Haskell. An out-call may happen in a separate OS thread if the foreign import is "safe", that is, another OS thread will continue to run the remaining Haskell threads while the call is in progress.
However, if one of my Haskell-based callbacks creates new threads with forkIO, I could be in trouble; if they make any calls into C, a new bound OS thread would be created for them, and this could wind up causing trouble in C. I would probably need some sort of "global MVar" to synchronize access into the C world.
Bingo. This is why you need to make all your calls to the C library from a single thread.
I also have some follow-up questions after looking at the Control.Concurrent API:
1. It seems that there is no function that says "block the current thread until the thread given by ThreadId dies"
You can do something like this with an exception handler and an MVar or TVar: do died <- atomically $ newTVar False forkIO (later (atomically $ writeTVar died True) $ ...) let wait = atomically $ do b <- readTVar died when (not b) retry where later = flip finally granted, it's hard to implement exactly what you were asking for. Another way to do it would be to put a finalizer on the ThreadId, but that would incur a delay until the GC discovered the thread was unreachable. Hmm, perhaps we should have threadIsAlive :: ThreadId -> STM Bool
2. What is the preferred way to implement a simple lock? With an MVar?
TVars are the way to go, although MVars do perform slightly better at the moment (at least if you stick to the simple putMVar/takeMVar operations). Cheers, Simon

Hello Simon, Thursday, July 21, 2005, 1:16:10 AM, you wrote:
However, if one of my Haskell-based callbacks creates new threads with forkIO, I could be in trouble; if they make any calls into C, a new bound OS thread would be created for them, and this could wind up causing trouble in C. I would probably need some sort of "global MVar" to synchronize access into the C world.
SM> Bingo. This is why you need to make all your calls to the C library SM> from a single thread. you can either: 1) made all calls from single thread 2) put all calls in "withMVar lock", where `lock` is a global MVar JG> 1. It seems that there is no function that says "block the current JG> thread until the thread given by ThreadId dies" GHC concurrency stuff is very basic and low-level. there is several libs which adds some more high-level abilities, including MissingH (see Child.hs) and some functions in http://www-i2.informatik.rwth-aachen.de/~stolz/Haskell/CA.hs -- Best regards, Bulat mailto:bulatz@HotPOP.com

On Thu, Jul 21, 2005 at 11:07:15AM +0400, Bulat Ziganshin wrote:
Hello Simon,
Thursday, July 21, 2005, 1:16:10 AM, you wrote:
SM> from a single thread.
you can either: 1) made all calls from single thread 2) put all calls in "withMVar lock", where `lock` is a global MVar
OK, that makes sense.
JG> 1. It seems that there is no function that says "block the current JG> thread until the thread given by ThreadId dies"
GHC concurrency stuff is very basic and low-level. there is several libs which adds some more high-level abilities, including MissingH (see Child.hs) and some functions in http://www-i2.informatik.rwth-aachen.de/~stolz/Haskell/CA.hs
Err, got a bit of egg on my face for that one. Would have hoped I'd have remembered what's in the library I maintain ;-) But actually, looking at it now, it's not a plain ThreadId that it waits for, but rather a Child, which is: data Child a = Child ThreadId (MVar a) Great if you use the Child.hs (which, for the record, is one of Peter Simons' works), but if somebody isn't using it, it may be nice to have another option. I'll look into Simon's recipe and see if I can understand it ;-) FWIW, I think Python has some really great threading primitives. From a brief look at what's in Haskell, I think they are all implementable in terms of the low-level features Haskell has (I'll trust you all on the waiting for a thread thing). I may write something up along those lines, stick it in MissingH and offer it up for fptools if anyone is interested. Also, I think it's annoying that ThreadId is not showable. This could be very useful for debugging. Python's stuff is at http://www.python.org/doc/current/lib/module-threading.html -- John

On 7/21/05, John Goerzen
On Thu, Jul 21, 2005 at 11:07:15AM +0400, Bulat Ziganshin wrote:
Hello Simon,
Thursday, July 21, 2005, 1:16:10 AM, you wrote:
SM> from a single thread.
you can either: 1) made all calls from single thread 2) put all calls in "withMVar lock", where `lock` is a global MVar
OK, that makes sense.
JG> 1. It seems that there is no function that says "block the current JG> thread until the thread given by ThreadId dies"
GHC concurrency stuff is very basic and low-level. there is several libs which adds some more high-level abilities, including MissingH (see Child.hs) and some functions in http://www-i2.informatik.rwth-aachen.de/~stolz/Haskell/CA.hs
Err, got a bit of egg on my face for that one. Would have hoped I'd have remembered what's in the library I maintain ;-)
But actually, looking at it now, it's not a plain ThreadId that it waits for, but rather a Child, which is:
data Child a = Child ThreadId (MVar a)
Great if you use the Child.hs (which, for the record, is one of Peter Simons' works), but if somebody isn't using it, it may be nice to have another option. I'll look into Simon's recipe and see if I can understand it ;-)
FWIW, I think Python has some really great threading primitives. From a brief look at what's in Haskell, I think they are all implementable in terms of the low-level features Haskell has (I'll trust you all on the waiting for a thread thing). I may write something up along those lines, stick it in MissingH and offer it up for fptools if anyone is interested.
Also, I think it's annoying that ThreadId is not showable. This could be very useful for debugging.
Prelude Control.Concurrent> :i ThreadId data ThreadId = ThreadId GHC.Prim.ThreadId# -- Imported from GHC.Conc instance Eq ThreadId -- Imported from Control.Concurrent instance Ord ThreadId -- Imported from Control.Concurrent instance Show ThreadId -- Imported from Control.Concurrent -- Friendly, Lemmih

Hello John, Thursday, July 21, 2005, 4:20:16 PM, you wrote:
you can either: 1) made all calls from single thread 2) put all calls in "withMVar lock", where `lock` is a global MVar
JG> OK, that makes sense. Simon Marlow has corrected me :)
JG> 1. It seems that there is no function that says "block the current JG> thread until the thread given by ThreadId dies"
GHC concurrency stuff is very basic and low-level. there is several libs which adds some more high-level abilities, including MissingH (see Child.hs) and some functions in http://www-i2.informatik.rwth-aachen.de/~stolz/Haskell/CA.hs
JG> Err, got a bit of egg on my face for that one. Would have hoped I'd JG> have remembered what's in the library I maintain ;-) it's cool :) JG> But actually, looking at it now, it's not a plain ThreadId that it waits JG> for, but rather a Child, which is: JG> data Child a = Child ThreadId (MVar a) JG> Great if you use the Child.hs (which, for the record, is one of Peter JG> Simons' works), but if somebody isn't using it, it may be nice to have JG> another option. I'll look into Simon's recipe and see if I can JG> understand it ;-) please see Child.hs sources - it uses the same technique as Simon suggested. currently there is no way to directly determine whether some thread was finished or not, all solutions add some code which is is executed after actual thread code. if you will clean Simon Marlow's code, you will get: finished <- newEmptyMVar forkIO (do_thread >>= putMVar finished) and then using `isEmptyMVar finished` in main thread to test whether the code runned by `do_thread` is done JG> FWIW, I think Python has some really great threading primitives. From a JG> brief look at what's in Haskell, I think they are all implementable in JG> terms of the low-level features Haskell has (I'll trust you all on the JG> waiting for a thread thing). I may write something up along those JG> lines, stick it in MissingH and offer it up for fptools if anyone is JG> interested. i will see, it's interesting. btw, i also make small process-related lib, but it is very narrow-purposed - it helps organizing threads into unix-fashion pipes like, for example, "reader |> modifier |> writer" -- Best regards, Bulat mailto:bulatz@HotPOP.com
participants (4)
-
Bulat Ziganshin
-
John Goerzen
-
Lemmih
-
Simon Marlow