Excerpts from Alexander Kjeldaas's message of Fri Mar 08 02:06:07 -0800 2013:
Yes. Actually, it's simpler than that; just put the listener in the front of the queue. You will be sad if the listener takes too long to figure out what to do though!
With this scheduling trick, it seems that it should be possible to guarantee that at most one scheduled quantum + one nursery of memory is used. (The non-allocating process is still a problem if I understand correctly).
Well, that's why you compile with -falways-yields http://hackage.haskell.org/trac/ghc/ticket/367
Another API semantics that comes to mind is limit the resident size in a tree-like fashion so that whoever does forkIO will get the child resident size attached to his own process. This requires a listener to be able to kill off whole process-trees (think cgroups in linux).
This is a little more difficult, because I do not know how to do inherited resource counts efficiently. (The profiler does do them, but only at the *end* of the process, and it involves walking all of the stacks.) It may still be worth implementing.
This would correspond to the ability to "freeze" Haskell threads in mid-execution; no MVar blocking necessary. This is not possible with the current RTS. I don't know how hard it would be to do.
Random idea: Maybe there could be a never-executed capability in the RTS. Schedule the frozen process on that capability. This seems to be doable from the RTS side of things. When some condition arises, schedule it on the frozen capability. Then notify something in Haskell land. I don't know how to do the notification - when I looked at notifying through MVars for the event log it wasn't as easy as pie.
That would be pretty cute implementation strategy; freezing and unfreezing corresponds to moving process on and off the cap. Of course, why not just dispense with the entire capability and just have a list of frozen TSOs? Edward