June 2002 - HOpenGL - Haskell.org

RE: [HOpenGL] HOpenGL and --enable-threaded-rts
by Simon Peyton-Jones 20 Jun '02

20 Jun '02

Simon and I have discussed this multi-threading question a bit. This message tries to summarise the story as we understand it. GHC's approach to threading ~~~~~~~~~~~~~~~~~~~~ Make certain you read the "Supporting multi-threaded inter-operation" section of the GHC commentary http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/rts-libs/multi-thread. html The current GHC model has the basic assumption that OS threads are inter-changeable. One Haskell thread may be executed by one OS thread or by many; you just can't tell. There is a good reason for this: the current OS thread may block in some I/O call (getChar, say), and we don't want that to block all Haskell threads. If you want all Haskell threads executed by a single OS thread, then you don't want --enable-threaded-rts, and your I/O calls may block all Haskell threads. There is no notion of a Haskell-thread-local location right now. We don't know how to make it type-safe. If you want that, you have to define your own environment monad, which isn't too hard. Or use implicit parameters, also not hard. GLUT's approach to threading ~~~~~~~~~~~~~~~~~~~~~ Simon and I don't understand GLUT's requirements at all clearly. Why is the context thread-local? Because different threads can have different contexts? Do you really need that? If not, would it suffice to make all the threads have (a pointer to) the same context)? Much the tidiest thing is to make the context an explicit parameter to GLUT calls. Or, if there is only one context of interest, keep it in a process-global location (not thread-local) and, sigh, heave it into the thread-local location before every GLUT call. In short, we don't understand GLUT well enough to be able to propose a solution. I hope that the above remarks may help the GLUT experts understand GHC well enough to write down a summary of what is going on. Try not to rely on someone having read all the past mail! Simon | -----Original Message----- | From: Sven Panne [mailto:Sven_Panne@BetaResearch.de] | Sent: 18 June 2002 15:06 | To: hopengl(a)haskell.org | Cc: GHC List | Subject: Re: [HOpenGL] HOpenGL and --enable-threaded-rts | | | Simon Marlow wrote: | > [...] a given Haskell thread can even migrate from one OS thread to | > another during its execution. | | Uh, oh! %-( Under which circumstances exactly? I fear this | will give loads of fun with many C libraries out there, so | it's important to know... | | Cheers, | S. | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-| haskell-users |

2 1

Real ZOO web site, welcome! ID<krshqjoCkdvnhoo1ruj>
by Farmgirl10470 20 Jun '02

20 Jun '02

2 1

Re: HOpenGL and --enable-threaded-rts
by Andrew J Bromage 19 Jun '02

19 Jun '02

G'day all. On Wed, Jun 19, 2002 at 10:29:52AM +0200, Wolfgang Thaller wrote: > The problem here is with OpenGL, not with GLUT. OpenGL requires some > thread-local state to be set correctly in order to operate. Surely you > wouldn't suggest reimplementing OpenGL (plus all OpenGL video drivers > for every platform/OS/graphics card combination out there) in Haskell? Point taken. Actually, many high-level language GUI interface bindings spawn a dedicated thread to handle all the graphical I/O and everything else is done with inter-thread communication. It would be fiddly and unnecessarily painful, but you could probably do that with no changes to the rts so long as the dedicated thread wasn't a Haskell thread. > I agree that it is difficult, and it can cause headaches. But in order > to write "real-world" applications in Haskell, there is no way around > it. In order to write GUI applications, I need a binding to Win32 (a > "C-specific" framework) for Windows, [...] This may be a definitional issue, but I wouldn't call Win32 a framework. MFC is a framework. POSIX isn't. BeOS is somewhere on the boundary. > Most of those use callbacks of some sort. Most of them don't require thread local state for correct operation. > Exceptions just require a little more marshaling, I think. As for the > even hairier things, I don't think "they" are going to add many more > crazy things to the C language family, so C-specific frameworks > shouldn't hold too many surprises in store for us... I'm constantly surprised by the hairiness you can find even in constructing C++ bindings for C APIs. It's not going to be long, for example, before you find an API which requires a callback to use varargs, at which point we'll be having this conversation yet again. Cheers, Andrew Bromage

1 0

RE: [HOpenGL] HOpenGL and --enable-threaded-rts
by Simon Marlow 18 Jun '02

18 Jun '02

> This sounds like a lot of work and a porting nightmare (what do you > mean Linux/Win32/HPUX/... doesn't have thread manipulation function X, > it's available on FreeBSD/Win32/... What if there are other forms > of thread-local state (e.g., errno)? What about setjmp/longjmp?). Yes, these are all problems. However, there is a nice abstraction of the OS thread API in GHC's RTS, thanks to Sigbjorn. So I'm sure this API could be extended to include some thread-local state operations. > The only viable solution I can see is to provide a way to capture the > calling thread (as a first class entity) when you call into Haskell > and to explicitly specify whcih thread to use when you call out from > Haskell. (Hmmm, sounds like callcc for C :-)) The trouble is, that is *way* too much overhead for a C call. HOpenGL does lots of these, and I strongly suspect that adding a full OS-thread context switch (well two, including the return) for each one would be a killer. Cheers, Simon

4 3

RE: [HOpenGL] HOpenGL and --enable-threaded-rts
by Simon Marlow 18 Jun '02

18 Jun '02

> Has anyone already thought about how to solve this problem? > I'm thinking about adding hooks to the RTS (in grabCapability, > releaseCapability and scheduleThread_) which would be used > for setting > up the correct thread-local state whenever Haskell execution > "switches" > to a different OS thread. Those hook routines would have to > be written > in C and would be platform-specific most of the time. It's not a nice > solution, but it's the only one I can think of at this time. > Unless someone comes up with a better idea _quickly_, I'll try it out > and then report how ugly it really is... :-) I can't see a nice solution to this problem - it really invalidates one of the key assumptions in the threaded implementation of the RTS, namely that it doesn't matter which OS thread we use to execute Haskell code. I suspect that a "safe" foreign import could be made to switch threads to the right OS thread before entering C land, but that's way too much overhead to impose on every single FFI call from Haskell. Yes, my guess is that this will probably have to be done by manipulating the thread-local state in the RTS - grab the current thread-local state whenever a call is made into Haskell, and whenever we run a Haskell thread we have to set the appropriate thread-local state. Perhaps it should be stored in the TSO and inherited by child threads too. The trouble is that there isn't a single object representing the whole thread-local state. Does OpenGL use pthread_getspecific() and pthread_setspecific() to access its thread-local state? Cheers, Simon

3 2

RE: [HOpenGL] HOpenGL and --enable-threaded-rts
by Simon Marlow 18 Jun '02

18 Jun '02

> I'm slowly losing track of this discussion... so am I :-( > My initial suggestion > was that it is guaranteed that the same OS thread which created the > f.i.w. thunk is used to call back to Haskell for *this* > wrapped function. > There is no overhead for calls Haskell->C, only for the comparatively > rare case of C->Haskell. What's wrong with this? There are two problems with this approach, I think. The situation is like this (correct me if I'm wrong): - Haskell thread H1 running on OS thread O1 registers a callback C. - Haskell thread H1/O1 makes a blocking call into HOpenGL. This call is made also in O1. The RTS allocates another OS worker thread O2, and continues running Haskell threads. - HOpenGL, running in O1, invokes the callback C. The RTS stops O1, creates a new Haskell thread H2 in which to run C, and eventually runs H2 in O2. Problem #1 is the call-in: our current implementation *always* runs the callback in a different OS thread from the calling thread. It was simpler this way, but perhaps this can change. Problem #2 is that we would have to add some extra machinery to guarantee that a given Haskell thread executes in a particular OS thread, and somehow do it in a way that was "fair" (i.e. the Haskell thread with a preference for its OS thread doesn't get starved, and doesn't starve other threads). The RTS currently doesn't make any distinction between OS threads; a given Haskell thread can even migrate from one OS thread to another during its execution. Cheers, Simon

2 1

RE: HOpenGL and --enable-threaded-rts
by Simon Peyton-Jones 17 Jun '02

17 Jun '02

This is nasty. As I understand it, * GLUT assumes that the same OS thread executes some Haskell code, calls C, which calls back to Haskell. * Concurrent Haskell never lets this happen, for very good reason. The thread executing Haskell code is just an anonymous OS thread executing multiple Haskell threads. The good reason is this: if one designated OS thread must execute a particular bunch of Haskell code, what if that OS thread gets blocked doing I/O on behalf of some other Haskell thread? It's hard to say just which Haskell threads "belong" to that designated OS thread. The only thing I can think of is that the Haskell call-back thread somehow says that it can only be executed by the designated OS thread; and when blocks (the Haskell call-back, that is) the OS thread doesn't grab another Haskell thread to execute, but instead steps aside for an anonymous worker thread to do the job. Makes it all more complicated, alas. Is there no way to get the "current context" explicitly, and treat it as some kind of "handle" that accompanies GLUT calls? I counsel caution here. It's easy to hack something up that will puzzle us for generations. Check out Sigbjorn's notes in the GHC commentary. Whatever we end up doing should be documented there. Simon | -----Original Message----- | From: Wolfgang Thaller [mailto:wolfgang.thaller@gmx.net] | Sent: 16 June 2002 20:57 | To: hopengl(a)haskell.org | Cc: GHC List | Subject: HOpenGL and --enable-threaded-rts | | | In short, it doesn't work :-( . | OpenGL (at least on MacOS) keeps track of the "current context" on a | per-thread basis. The GLUT library sets the current context and calls | back to the program. With GHC 5.03 compiled with | --enable-threaded-rts, | the callback gets executed in a different thread. There is no OpenGL | context set up for that thread, so the first OpenGL call | crashes. This might also be a problem for other state-based | interfaces that use | thread-local state. | | Has anyone already thought about how to solve this problem? | I'm thinking about adding hooks to the RTS (in grabCapability, | releaseCapability and scheduleThread_) which would be used | for setting | up the correct thread-local state whenever Haskell execution | "switches" | to a different OS thread. Those hook routines would have to | be written | in C and would be platform-specific most of the time. It's not a nice | solution, but it's the only one I can think of at this time. | Unless someone comes up with a better idea _quickly_, I'll try it out | and then report how ugly it really is... :-) | | Cheers, | | Wolfgang | | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-| haskell-users |

3 2

Re: [HOpenGL] HOpenGL1.02 on Sun's?
by C.Reinke 17 Jun '02

17 Jun '02

> > [...] according to my reading of the OpenGL spec, it seems part of > > the update to 1.2 (it's hidden in some table as GL_TEXTURE_BINDING_xD, > > where x should presumably include 3 now). > > Hmmm, the OpenGL spec is a bit strange here: GL_TEXTURE_BINDING_xD is > not mentioned in the index, but is used in the tables describing the > state queries. But it's quite clear that GL_TEXTURE_BINDING_3D should > be there if texture objects and 3D textures are supported (both OpenGL > 1.2 features). SUN's header claims to implement 1.2, but fails to #define > the above token. I guess it's a bug. It's easy enough not to see this in the spec, but I don't understand that the conformance tests (if any) didn't pick it up and that noone has complained before.. If I get around to it, I might let them know.. > Simple solution: Either comment it out if you don't need it or > have a look at your favourite correct header and substitute the > numeric value, perhaps the query is implemented nevertheless. Forgot to answer this, because I ran into an unrelated problem, but I should at least reassure other Sun HOpenGL users a bit;-). Uncommenting seems to work, but I noticed some other installation issues, related to us not having GLUT in the standard place. I guess they are bugs in HOpenGL's configure script? I append a log of what I had to do below. This doesn't help me because our Sun's are used as servers, accessed mostly from PCs, via Exceed, and that doesn't seem to support GLX by default. I've seen Exceed 3d mentioned on their product page http://www.hummingbird.com/products/nc/exceed/ which looks like the solution to this problem - does anyone here have experience with that?-) Btw, Ronald Legere (who first confirmed the Sun header problem off list) vaguely recalled two other problems, default shell and black windows, which I might be able to shed some light on: - default make on Suns tends not to be gnu make, so you explicitly have to call gmake, unless you've got a portable makefile. > which make /usr/ccs/bin/make - black windows can easily be caused by a sub-optimal color-allocation scheme in some Sun graphics implementations: if one application "steals" all colors in the map (netscape was a favorite for that), new applications may come up empty-handed. I seem to recall that there was a way to make programs behave better, but otherwise moving the mousepointer into the new black window might be sufficient to cause another disconcerting effect - now the new app gets its colors alright but all the surroundings look very strange. > This is just another example why the next release of HOpenGL won't > depend on those #defines, but will use numeric values directly... ? bypassing the specified API in favour of values taken from sample implementations doesn't strike me as a particularly good idea? Cheers, Claus -------------- installation log # GLUT isn't in the standard place here.. # configure recommends to set environment variables in such cases, # but in this case, this is going to cause problems below setenv LDFLAGS "-L/usr/local/packages/glut/lib/glut/" setenv CPPFLAGS "-I/usr/local/packages/glut/include/" # Sun doesn't seem to have read the OpenGL1.2 spec: # GL_TEXTURE_BINDING_3D is missing in GL/gl.h.. # remove GL_TEXTURE_BINDING_3D from lib/GL_EnumQueries3.gc # ghc and green card were not in standard places, no problem ./configure \ --with-hc=/home/cur/cr20/public/haskell/ghc-5.02.2/bin/ghc \ --with-green-card=/home/cur/cr3/software/gc-2.03/src/green-card # Sun's default make is *not* gnu make! gmake depend # unfortunately, HOpenGL's configure hasn't recorded the settings # for LDFLAGS and CPPFLAGS to pass them on to ghc.. # # also, configure has added all kind of strange flags, but *not* # the ones that are needed, namely: -lGLU -lGL # # so we need to modify in config.mk by hand: # 1. add -lGLU -lGL to GL_LIBS # 2. add LDFLAGS to GL_LIBS # 3. add CPPFLAGS to HC_OPTS ---snip GL_LIBS += -lSM -lICE -L/usr/openwin/lib -R/usr/openwin/lib -L/usr/local/packages/glut/lib/glut/ -lglut -lGLU -lGL -lX11 -lXext -lXmu -lXt -lXi -lsocket -lnsl -lm HC_OPTS += -fglasgow-exts -I/usr/openwin/include -I/usr/local/packages/glut/include/ "-\#include <GL/glut.h>" ---snip gmake

3 3

HOpenGL Tutorial news
by Andre W B Furtado 16 Jun '02

16 Jun '02

Hi everybody. I've just concluded a new lesson in my HOpenGL Tutorial: A Simple Sound Example, which explains how to issue simple sound calls in HOpenGL applications (for Win32 platforms only). I've also added some screenshots to the examples section. The tutorial is available in www.cin.ufpe.br/~haskell/hopengl Hope you enjoy, -- Andre

1 0

HOpenGL and --enable-threaded-rts
by Wolfgang Thaller 16 Jun '02

16 Jun '02

In short, it doesn't work :-( . OpenGL (at least on MacOS) keeps track of the "current context" on a per-thread basis. The GLUT library sets the current context and calls back to the program. With GHC 5.03 compiled with --enable-threaded-rts, the callback gets executed in a different thread. There is no OpenGL context set up for that thread, so the first OpenGL call crashes. This might also be a problem for other state-based interfaces that use thread-local state. Has anyone already thought about how to solve this problem? I'm thinking about adding hooks to the RTS (in grabCapability, releaseCapability and scheduleThread_) which would be used for setting up the correct thread-local state whenever Haskell execution "switches" to a different OS thread. Those hook routines would have to be written in C and would be platform-specific most of the time. It's not a nice solution, but it's the only one I can think of at this time. Unless someone comes up with a better idea _quickly_, I'll try it out and then report how ugly it really is... :-) Cheers, Wolfgang

1 0