bringing xmonad into the threaded world

There have been various times when I've wished I could just spawn a thread in xmonad. Just recently, for example, it was pointed out to me that the default mod+q binding starts a brand new process just to keep xmonad responsive while it recompiles the configuration file. This is silly. Here is a patch bundle that makes xmonad ask X to be thread-safe, and rewrites the default mod+q binding to take advantage of the new thread safety. ~d

On Mon, 07 May 2012 02:06:31 -0400 wagnerdm@seas.upenn.edu wrote:
There have been various times when I've wished I could just spawn a thread in xmonad. Just recently, for example, it was pointed out to me that the default mod+q binding starts a brand new process just to keep xmonad responsive while it recompiles the configuration file. This is silly.
Why is it silly? It's not like the overhead of starting a new process is going to be anything but noise compared to recompiling the configuration file.
Here is a patch bundle that makes xmonad ask X to be thread-safe, and rewrites the default mod+q binding to take advantage of the new thread safety.
What's the performance hit from making X thread-safe?
For that matter, if you are using Haskell threads instead of OS
threads, do you need to make X thread safe? [Sorry if this is a silly
question - I'm not really familiar with using Haskell threads.]

On Mon, May 7, 2012 at 5:29 AM, Mike Meyer
There have been various times when I've wished I could just spawn a thread in xmonad. Just recently, for example, it was pointed out to me that the default mod+q binding starts a brand new process just to keep xmonad responsive while it recompiles the configuration file. This is silly.
I'm not convinced that is either a problem or "silly", but it can be changed without changing X11 to threaded because it doesn't actually use X11 in a dangerous way within a thread. Also be aware that a number of contrib modules *already* use threads... green threads, which open their own $DISPLAY connections. This is entirely safe as long as you don't do something that would block in an FFI call or that would expect the xmonad main loop to run while simultaneously expecting a *synchronous* response (as the infamous putSelection did). Why is it silly? It's not like the overhead of starting a new process
is going to be anything but noise compared to recompiling the configuration file.
Indeed. ghc and its invocation of ld are going to be the visibly expensive part.
Here is a patch bundle that makes xmonad ask X to be thread-safe, and rewrites the default mod+q binding to take advantage of the new thread safety.
What's the performance hit from making X thread-safe?
Significant. It's worth remembering that one of the more significant open bugs has to do with spending too much time context switching with the X server when sweeping the mouse across multiple windows... this will get a *lot* worse if Xlib is constantly grabbing and releasing a mutex on every single call (which is how it does thread safety; it's not even remotely smart about it). For that matter, if you are using Haskell threads instead of OS
threads, do you need to make X thread safe? [Sorry if this is a silly question - I'm not really familiar with using Haskell threads.]
Green threads are fine, aside from you can't safely use the main connection to the Display because you can't push back arbitrary numbers of arbitrary events that you don't care about, and if you silently drop them because you don't care about them then the main loop isn't seeing stuff it needs to see. As mentioned above, the usual workaround is to open your own connection. Building xmonad with -threaded gets you OS threads, with all their potential problems. -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

Quoting Brandon Allbery
Also be aware that a number of contrib modules *already* use threads... green threads, which open their own $DISPLAY connections. This is entirely safe as long as you don't do something that would block in an FFI call or that would expect the xmonad main loop to run while simultaneously expecting a *synchronous* response (as the infamous putSelection did).
These two caveats sound like a great argument for applying this patch bundle to me. However, if there's significant concern, I would be willing to create and apply a patch bundle that doesn't call initThreads but still forked a thread for rebuilding and restarting (by opening a new X connection before the restart). ~d

On Mon, May 7, 2012 at 10:31 AM,
Quoting Brandon Allbery
: Also be aware that a number of contrib modules *already* use threads... green threads, which open their own $DISPLAY connections. This is entirely safe as long as you don't do something that would block in an FFI call or that would expect the xmonad main loop to run while simultaneously expecting a *synchronous* response (as the infamous putSelection did).
These two caveats sound like a great argument for applying this patch bundle to me.
Then you don't understand the reason for them, and need to reconsider what you think threading is going to accomplish here. Synchronous is exactly what it sounds like; the code was "threaded" but not in a useful way, and using an OS thread instead of a green thread would not have fixed its problem. (The main thread was in fact blocked on the child thread, waiting for it to produce a result. The rewrite I've been working on makes this asynchronous... and in fact no longer requires any threads at all; the threading it was doing wasn't necessary or even sensible. It was just broken code, possibly written by someone who thought that threads are magic sauce that fixes any situation where you need two chunks of code in the same program to process data at the same time. (The thread magic sauce just turned their logic error into a deadlock.) -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

Quoting Brandon Allbery
On Mon, May 7, 2012 at 10:31 AM,
wrote: These two caveats sound like a great argument for applying this patch bundle to me.
Then you don't understand the reason for them, and need to reconsider what you think threading is going to accomplish here.
You're right. I misunderstood. Probably the proposed patch bundle should be discarded. Back to the drawing boards, I guess. *sigh* ~d

xmonad isn't threaded because the X server doesn't allow concurrent access
-- it will crash.
-- Don
On Mon, May 7, 2012 at 1:27 PM,
Quoting Brandon Allbery
: On Mon, May 7, 2012 at 10:31 AM,
wrote: These two caveats sound like a great argument for applying this patch
bundle to me.
Then you don't understand the reason for them, and need to reconsider what you think threading is going to accomplish here.
You're right. I misunderstood. Probably the proposed patch bundle should be discarded.
Back to the drawing boards, I guess. *sigh*
~d
______________________________**_________________ xmonad mailing list xmonad@haskell.org http://www.haskell.org/**mailman/listinfo/xmonadhttp://www.haskell.org/mailman/listinfo/xmonad

On Mon, May 7, 2012 at 6:18 PM, Don Stewart
xmonad isn't threaded because the X server doesn't allow concurrent access -- it will crash.
Modern libX11 has thread "support" (it wraps every API call in a mutex) if so compiled and you make the appropriate call to tell it to do the wrapping; that's part of what the proposed patch did. It's not very smart and slows things down a lot; most people prefer to pretend it's still not threaded as a result. XCB has slightly better thread support. -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

Quoting Don Stewart
xmonad isn't threaded because the X server doesn't allow concurrent access -- it will crash.
This is only partly accurate, as I've learned over the last few days: one Display per thread is a (the) safe way to do concurrent server access. Even if it *were* only possible to access X from exactly one thread, it's not clear to me that this would definitely mean that xmonad itself couldn't be multithreaded. Only that messages intended for the X server would all have to go through one centrally coordinated thread. I'd love to see the architecture of xmonad shift so that there are separate threads listening for X events and sending X commands (using two different Displays), but it's not clear to me what the right API for user code is. Perhaps the current model -- where it's your responsibility to open a display whenever you want to fork, and there's no central coordination -- is the best we can hope for. It's also a much more significant change than the small one I had wrongly hoped would be sufficient. ~d

Quoting Mike Meyer
On Mon, 07 May 2012 02:06:31 -0400 wagnerdm@seas.upenn.edu wrote:
There have been various times when I've wished I could just spawn a thread in xmonad. Just recently, for example, it was pointed out to me that the default mod+q binding starts a brand new process just to keep xmonad responsive while it recompiles the configuration file. This is silly.
Why is it silly? It's not like the overhead of starting a new process is going to be anything but noise compared to recompiling the configuration file.
Perhaps silly is the wrong word. But: Because it means any time I want to do something with X, but keep xmonad's response time snappy, I have to create a process, make sure that process is in xmonad's PATH with all the confusing debugging associated with that (some people on #xmonad actually didn't have xmonad in their $PATH!), figure out how to do IPC...
Here is a patch bundle that makes xmonad ask X to be thread-safe, and rewrites the default mod+q binding to take advantage of the new thread safety.
What's the performance hit from making X thread-safe?
There's a hit, but considering xmonad's current strategy of completely syncing its internal state with X all the time, this is the wrong place to fix a performance problem.
For that matter, if you are using Haskell threads instead of OS threads, do you need to make X thread safe? [Sorry if this is a silly question - I'm not really familiar with using Haskell threads.]
I'm fairly certain that GHC will not context-switch during an FFI call. So using green threads may be OK. But I'm also not enough of an expert to say, and this way is certainly OK. ~d

On Mon, 07 May 2012 10:28:32 -0400 wagnerdm@seas.upenn.edu wrote:
Quoting Mike Meyer
: Why is it silly? It's not like the overhead of starting a new process is going to be anything but noise compared to recompiling the configuration file. Perhaps silly is the wrong word. But:
Because it means any time I want to do something with X, but keep xmonad's response time snappy, I have to create a process, make sure that process is in xmonad's PATH with all the confusing debugging associated with that (some people on #xmonad actually didn't have xmonad in their $PATH!), figure out how to do IPC...
Ah, it's doing a spawn() instead of a simple fork() to run code in the
existing process. I agree, that's silly.
After a quick glance at the patch, it should mostly work if you use a
real OS fork instead of a forkIO (because you're still doing IPC via
the X server), and also have the nifty protected address space that
comes with using a modern facility instead of threading.

Quoting Mike Meyer
After a quick glance at the patch, it should mostly work if you use a real OS fork instead of a forkIO (because you're still doing IPC via the X server), and also have the nifty protected address space that comes with using a modern facility instead of threading.
My new understanding of the situation (which is very possibly still flawed!) is that the patch I proposed has too much thread safety in one axis and not enough in another. It's too much thread safety in the sense that it calls initThreads, and too little thread safety in that it uses the same connection to the display for both the main event loop and the thread that signals xmonad to restart once the recompilation is finished. The result is that the recompilation happens concurrently with the event loop, but as soon as it comes time to signal xmonad to restart, only one of the event loop thread or the signal thread runs. If it happens to be the event loop thread running, this means the signal to restart blocks until the next X event happens -- which may be far, far in the future if the user sits there doing nothing waiting for xmonad to restart. Dumb. The right thing to do (I think! I'm still no expert) is to open a new connection to the display in the spawned recompile-and-signal thread; then it's unnecessary to call initThreads. It should be easy to make a patch that does this kind of thing instead of spawning a new process, I just haven't done it. ...and I probably won't, either, because it's fixing the symptom, not the problem: currently, the only way for spawned threads to communicate with the main thread is through X! ~d
participants (4)
-
Brandon Allbery
-
Don Stewart
-
Mike Meyer
-
wagnerdm@seas.upenn.edu