RE: [Haskell-cafe] FFI and callbacks

On 23 July 2005 03:38, Duncan Coutts wrote:
The problem then as John noted is that the main loop of these toolkits block and so the other Haskell threads would not get a chance to schedule. So the challenge is to give the Haskell threads a chance to schedule.
[ good description of the multi-threaded GUI problem deleted ] Thanks for describing the problem in detail, I understand it better now. I think it comes down to this conflict: - you want to take advantage of the fact that GHC has lightweight "green" threads in order to do multithreading within a single OS thread, but - our "bound threads" design does not require the implementation to support lightweight threads, and hence doesn't let the programmer take advantage of them. When we were thinking about bound threads, the idea was to accommodate multiple implementations including the one-to-one and giant lock approaches. We consider(ed) GHC's mixture of lightweight and heavyweight threads to be an optimisation. However, the GUI library example is interesting, because if we could take advantage of lightweight threads it seems we could have multithreaded access to the GUI with better performance and no requirement to make all GUI calls from a single thread. But I'm not convinced. There are advantages to using the single GUI thread approach, and we don't have any measurements for whether the overhead is too much. Suppose you set up a channel containing requests of type (IO a, MVar a), where the MVar takes the result. You can build a combinator withGUI :: IO a -> IO a to perform a GUI operation (and this allows you to try multiple approaches without changing the code). Using withGUI you can send a whole batch of GUI operations in one go, it doesn't have to be one request per GUI call. You could also build asyncGUI :: IO a -> IO () which fires off a GUI request that doesn't require a result, and combines it with existing requests in a single batch if possible. Ordering would be retained, of course. The GUI thread will run multiple requests sequentially, it doesn't have to context switch between each request. And you get more parallelism, because the other threads continue to run while the GUI thread is in the main loop. Cheers, Simon

On 2005-07-25, Simon Marlow
On 23 July 2005 03:38, Duncan Coutts wrote: Thanks for describing the problem in detail, I understand it better now. I think it comes down to this conflict:
- you want to take advantage of the fact that GHC has lightweight "green" threads in order to do multithreading within a single OS thread, but
- our "bound threads" design does not require the implementation to support lightweight threads, and hence doesn't let the programmer take advantage of them.
In my particular case, the C library isn't aware of threading at all. While it may be safe to call it from any arbitrary Haskell thread, it would not be safe to call it from two threads simultaneously. It seems that some sort of global lock -- implemented as a global variable of some sort, I guess -- is necessary. I'm not sure exactly how to accomodate that. It seems the current model doesn't work all that well in this scenario, unless I'm missing something? -- John

On Mon, 2005-07-25 at 13:47 +0000, John Goerzen wrote:
On 2005-07-25, Simon Marlow
wrote: On 23 July 2005 03:38, Duncan Coutts wrote: Thanks for describing the problem in detail, I understand it better now. I think it comes down to this conflict:
- you want to take advantage of the fact that GHC has lightweight "green" threads in order to do multithreading within a single OS thread, but
- our "bound threads" design does not require the implementation to support lightweight threads, and hence doesn't let the programmer take advantage of them.
In my particular case, the C library isn't aware of threading at all. While it may be safe to call it from any arbitrary Haskell thread, it would not be safe to call it from two threads simultaneously.
That's ok. Without the threaded rts, you get just one OS thread being used to make the foreign calls.
It seems that some sort of global lock -- implemented as a global variable of some sort, I guess -- is necessary. I'm not sure exactly how to accomodate that.
It seems the current model doesn't work all that well in this scenario, unless I'm missing something?
So long as you can arrange for your C lib to call back to Haskell reasonably often you can use the same solution as Gtk2Hs uses. I've outlined the technique here (mostly on page 2 for implementation notes): http://haskell.org/gtk2hs/archives/2005/07/24/writing-multi-threaded-guis/ Duncan

On Mon, Jul 25, 2005 at 04:05:15PM +0100, Duncan Coutts wrote:
On Mon, 2005-07-25 at 13:47 +0000, John Goerzen wrote:
- our "bound threads" design does not require the implementation to support lightweight threads, and hence doesn't let the programmer take advantage of them.
In my particular case, the C library isn't aware of threading at all. While it may be safe to call it from any arbitrary Haskell thread, it would not be safe to call it from two threads simultaneously.
That's ok. Without the threaded rts, you get just one OS thread being used to make the foreign calls.
Right, but let's say I do want to use the threaded RTS so I can use multiple OS threads; what should I be doing here? (Thanks also for the blog link; I'll take a look)

On Mon, 2005-07-25 at 10:08 -0500, John Goerzen wrote:
On Mon, Jul 25, 2005 at 04:05:15PM +0100, Duncan Coutts wrote:
On Mon, 2005-07-25 at 13:47 +0000, John Goerzen wrote:
- our "bound threads" design does not require the implementation to support lightweight threads, and hence doesn't let the programmer take advantage of them.
In my particular case, the C library isn't aware of threading at all. While it may be safe to call it from any arbitrary Haskell thread, it would not be safe to call it from two threads simultaneously.
That's ok. Without the threaded rts, you get just one OS thread being used to make the foreign calls.
Right, but let's say I do want to use the threaded RTS so I can use multiple OS threads; what should I be doing here?
That's harder. Simon and I have been talking about that and he has been suggesting a few experiments to try. Duncan

On 7/25/05, Simon Marlow
On 23 July 2005 03:38, Duncan Coutts wrote:
The problem then as John noted is that the main loop of these toolkits block and so the other Haskell threads would not get a chance to schedule. So the challenge is to give the Haskell threads a chance to schedule.
[ good description of the multi-threaded GUI problem deleted ]
Thanks for describing the problem in detail, I understand it better now. I think it comes down to this conflict:
- you want to take advantage of the fact that GHC has lightweight "green" threads in order to do multithreading within a single OS thread, but
- our "bound threads" design does not require the implementation to support lightweight threads, and hence doesn't let the programmer take advantage of them.
Well how about not touching the "bound threads" design at all, but modifying the lightweigh threads design to also include "forkIOHere" (or something) which would produce a lightweight thread which is always run in the "main" OS thread. That way you could just start doing GUI calls in the main thread and then anywhere you please spawn off lightweight threads which can also do GUI calls since they all get scheduled to run in the same OS thread as "main". So instead of changing the "bound threads" design you simiply make the GHC "optimization" of using lightweigh threads offer more control to the programmer. /S -- Sebastian Sylvan +46(0)736-818655 UIN: 44640862
participants (4)
-
Duncan Coutts
-
John Goerzen
-
Sebastian Sylvan
-
Simon Marlow