RE: [HOpenGL] HOpenGL and --enable-threaded-rts
This sounds like a lot of work and a porting nightmare (what do you mean Linux/Win32/HPUX/... doesn't have thread manipulation function X, it's available on FreeBSD/Win32/... What if there are other forms of thread-local state (e.g., errno)? What about setjmp/longjmp?).
Yes, these are all problems. However, there is a nice abstraction of the OS thread API in GHC's RTS, thanks to Sigbjorn. So I'm sure this API could be extended to include some thread-local state operations.
The only viable solution I can see is to provide a way to capture the calling thread (as a first class entity) when you call into Haskell and to explicitly specify whcih thread to use when you call out from Haskell. (Hmmm, sounds like callcc for C :-))
The trouble is, that is *way* too much overhead for a C call. HOpenGL does lots of these, and I strongly suspect that adding a full OS-thread context switch (well two, including the return) for each one would be a killer. Cheers, Simon
I'm slowly losing track of this discussion... My initial suggestion was that it is guaranteed that the same OS thread which created the f.i.w. thunk is used to call back to Haskell for *this* wrapped function. There is no overhead for calls Haskell->C, only for the comparatively rare case of C->Haskell. What's wrong with this? Cheers, S.
Yes, these are all problems. However, there is a nice abstraction of the OS thread API in GHC's RTS, thanks to Sigbjorn. So I'm sure this API could be extended to include some thread-local state operations.
A further piece of what one might call thread local state is 'recursive locks' like those found in Java. With normal locks, if a thread executes this: take(lock); take(lock); ... release(lock); release(lock); then the 2nd call to take will block because a thread already has the lock. With recursive locks, the implementation of take records who has the lock and just increments a counter if the same thread takes the lock again. Likewise, release decrements the counter and only releases the lock when the counter reaches 0. And arbitrary user code and libraries are free to implement all kinds of code that depends on the current thread id. I'll bet you're going to see a lot of cases like this.
The only viable solution I can see is to provide a way to capture the calling thread (as a first class entity) when you call into Haskell and to explicitly specify whcih thread to use when you call out from Haskell. (Hmmm, sounds like callcc for C :-))
The trouble is, that is *way* too much overhead for a C call. HOpenGL does lots of these, and I strongly suspect that adding a full OS-thread context switch (well two, including the return) for each one would be a killer.
So don't do it for every foreign function call - only do it for the ones that request it. Here's the implementation I imagine: For foreign imports and exports that have not requested any special thread behaviour, do exactly what GHC currently does. Overhead == 0. For foreign exports that have requested thread capture, the call goes like this: 1) Get a thread from GHC's thread pool 2) Allocate a first-class C-thread object on the Haskell heap. and fill in the details for this thread. 3) Add the normal foreign export function arguments plus a pointer to the C thread object to the GHC thread and make it runnable. 4) Block this thread so that it is ready to use later. 5) When C function returns, do so in the first-class C-thread object. For foreign imports that have requested explicit thread choice, the call goes like this: 1) Get the C-thread object (it's an argument to the Haskell function so this is easy), perform suitable sanity checks (i.e., not already in use). 2) Marshall argumens to C function into the C-thread. 3) Unblock the C thread, block the Haskell thread waiting for response. 4) When C function returns, context switch back to a GHC thread. Overhead for foreign export is higher. Overhead for foreign import is not much different from existing safe foreign imports. Overhead only occurs if you request this feature. -- Alastair Reid reid@cs.utah.edu http://www.cs.utah.edu/~reid/
I have implemented a hack that makes HOpenGL work with --enable-threaded-rts. It currently contains 3 lines of MacOS X-specific code, and it needs cleaning up. More about that later.
Simon and I don't understand GLUT's requirements at all clearly. Why is the context thread-local?
OpenGL is a state based API. All operations are defined in terms of how they affect global state. All global state used by OpenGL is contained in a so called "context". Every OpenGL command implicitly operates on a "current context". If a program draws to two different windows, it uses two different contexts. In order to allow programs to use OpenGL draw into two different contexts in two different threads, the "current context" pointer has to be made thread-local. Most OpenGL implementations do this by now, although there may be a few outdated ones left. Context creation and management is _not_ part of the OpenGL standard. There is a different API for every platform. GLUT is a cross-platform "utility toolkit" for creating simple OpenGL applications. It is supposed to isolate its user from all this context buisiness by always setting up the correct context before calling back to the program. But even if don't use GLUT, OpenGL requires the current context to be set _in the current OS thread_ in order to operate. We cannot rely on OpenGL implementations to use any known general mechanism for thread-local storage. Making OS threads correspond directly to Haskell threads is probably an extensive change to the RTS that has lots of disadvantages. My proposal is to leave dealing with thread-local state to the library binding (in this case, HOpenGL). This requires just a little support from the RTS. The library binding would have to include C-language routines that get called by the RTS at certain points: * when the RTS starts executing Haskell code in an OS thread (grabCapability) * when the RTS stops executing Haskell code in an OS thread (releaseCapability) * when the RTS is about to spawn a new thread in response to a callback (scheduleThread_). HOpenGL would just need to have about 3 platform-specific lines of code. This would effectively make OpenGL's thread local state global again. For real multithreaded use of OpenGL we would need some more RTS support. We'd just need to agree on a nice little addition to the RTS API . All libraries that currently can't be used because they rely on thread-local state could then be made to work using a few lines of C code. The problems with recursive locks are, of course, not solved by this, but I don't see a "perfect" all-round solution anywhere on the horizon. (...and apart from that, I don't need recursive locks right now :-) ) Cheers, Wolfgang
G'day all. On Tue, Jun 18, 2002 at 05:58:15PM +0100, Alastair Reid wrote:
A further piece of what one might call thread local state is 'recursive locks' like those found in Java.
Recursive locks arguably should be part of the lock abstraction, not "thread local state ". Since all it costs is another counter, it's almost trivial for the lock implementation (usually the OS) to implement it, of only as an option. For comparison: all Win32 built-in locks (Win32 confusingly differentiates "mutexes" and "critical sections" even though they are essentially the same thing) are recursive. Under all pthreads implementations that I'm aware of, THREAD_MUTEX_RECURSIVE is supported. I don't know enough about any other platforms to comment, but my bet is you'll find it pretty much everywhere. I think you need to look no further than the thread id itself for "thread local state". As soon as you want to do anything with the thread id other than compare equality (even Ord-type comparison is not supported under pthreads), you have thread local state. I don't mean to detract from the fine work which the HOpenGL people have achieved, but I think that binding to the C implementation of GLUT was, in retrospect, a mistake. Binding to foreign language-specific frameworks in general is a mistake, IMO. Today it's only threads which you may be able to hack around, but tomorrow, your called-back function will want to throw an exception, or something even hairier will turn up and you'll be back where you started. Sorry for the pessimism, but this is bitter experience talking. Cheers, Andrew Bromage
On Mittwoch, Juni 19, 2002, at 01:43 , Andrew J Bromage wrote:
I don't mean to detract from the fine work which the HOpenGL people have achieved, but I think that binding to the C implementation of GLUT was, in retrospect, a mistake.
The problem here is with OpenGL, not with GLUT. OpenGL requires some thread-local state to be set correctly in order to operate. Surely you wouldn't suggest reimplementing OpenGL (plus all OpenGL video drivers for every platform/OS/graphics card combination out there) in Haskell?
Binding to foreign language-specific frameworks in general is a mistake, IMO.
I agree that it is difficult, and it can cause headaches. But in order to write "real-world" applications in Haskell, there is no way around it. In order to write GUI applications, I need a binding to Win32 (a "C-specific" framework) for Windows, to Carbon (C-specific) or Cocoa (Objective-C or Java) for MacOS X, and to some toolkit for X11 (Gtk, for example). Most of those use callbacks of some sort. It cannot be a "mistake" because it is the _only_ option for achieving a certain goal (writing "native" GUI applications for any platform in Haskell).
Today it's only threads which you may be able to hack around, but tomorrow, your called-back function will want to throw an exception, or something even hairier will turn up and you'll be back where you started.
Exceptions just require a little more marshaling, I think. As for the even hairier things, I don't think "they" are going to add many more crazy things to the C language family, so C-specific frameworks shouldn't hold too many surprises in store for us...
Sorry for the pessimism, but this is bitter experience talking.
Sorry for the optimism, that's just my determination to use OpenGL from Haskell talking :-) Cheers, Wolfgang
G'day all. On Wed, Jun 19, 2002 at 10:29:52AM +0200, Wolfgang Thaller wrote:
The problem here is with OpenGL, not with GLUT. OpenGL requires some thread-local state to be set correctly in order to operate. Surely you wouldn't suggest reimplementing OpenGL (plus all OpenGL video drivers for every platform/OS/graphics card combination out there) in Haskell?
Point taken. Actually, many high-level language GUI interface bindings spawn a dedicated thread to handle all the graphical I/O and everything else is done with inter-thread communication. It would be fiddly and unnecessarily painful, but you could probably do that with no changes to the rts so long as the dedicated thread wasn't a Haskell thread.
I agree that it is difficult, and it can cause headaches. But in order to write "real-world" applications in Haskell, there is no way around it. In order to write GUI applications, I need a binding to Win32 (a "C-specific" framework) for Windows, [...]
This may be a definitional issue, but I wouldn't call Win32 a framework. MFC is a framework. POSIX isn't. BeOS is somewhere on the boundary.
Most of those use callbacks of some sort.
Most of them don't require thread local state for correct operation.
Exceptions just require a little more marshaling, I think. As for the even hairier things, I don't think "they" are going to add many more crazy things to the C language family, so C-specific frameworks shouldn't hold too many surprises in store for us...
I'm constantly surprised by the hairiness you can find even in constructing C++ bindings for C APIs. It's not going to be long, for example, before you find an API which requires a callback to use varargs, at which point we'll be having this conversation yet again. Cheers, Andrew Bromage
participants (5)
-
Alastair Reid -
Andrew J Bromage -
Simon Marlow -
Sven Panne -
Wolfgang Thaller