Questions on the RTS C API regarding threads and tasks

Hi everybody, I’m new on this list, congratulations to everyone for the great work that you’re doing on GHC! I apologize if this is not the right list where to ask the following questions. Please point me to the correct list or IRC channel otherwise. I’m part of a small group of people who is going to work on a project that seems to need low-level handling of haskell threads. I’ve started delving into the ghc runtime API to understand if everything we need is exposed to the world or if we have to modify the runtime itself (which I would avoid if possible). I have a few questions about the functions that the runtime exports to the C world regarding the manipulation of tasks: - Fundamentally, is there a way for a C function called by a foreign call to suspend the haskell thread that called it, to be resumed later when appropriate? I see that the runtime has a concept of “blocking queue”, but I also see that the functions that work on blocking queues are not exported. I can manage a queue of TSOs myself, but then I need a way to put in sleep the task indefinitely up to a wake signal. In other words, can I sleep and awake haskell threads from a C function called by a foreign call? - I see the “suspendThread” and “resumeThread” functions exist in include/rts/Threads.h Are these the functions that I need for the point above? My main concern is that the comments in the source code for suspendThread say that these functions are used in case of a CCall, and put the thread in a specific queue, so they seems to have more specific role than what the name might suggest. - Also, I’m unable to find what a StgRegTable is (which is taken as argument by suspendThread). Thank you in advance for your help, Greetings, Nicola

Excerpts from Nicola Gigante's message of 2015-11-04 11:12:39 -0800:
I’ve started delving into the ghc runtime API to understand if everything we need is exposed to the world or if we have to modify the runtime itself (which I would avoid if possible).
I agree with this goal in principle, but: (1) Depending on what you want to do, it may be quite difficult to do this, and (2) The RTS is actually quite hackable (as long as you are not changing any of the interfaces with generated Haskell code, you could even build your own RTS and have users link against that.) Something to keep in mind.
I have a few questions about the functions that the runtime exports to the C world regarding the manipulation of tasks:
- Fundamentally, is there a way for a C function called by a foreign call to suspend the haskell thread that called it, to be resumed later when appropriate? I see that the runtime has a concept of “blocking queue”, but I also see that the functions that work on blocking queues are not exported. I can manage a queue of TSOs myself, but then I need a way to put in sleep the task indefinitely up to a wake signal. In other words, can I sleep and awake haskell threads from a C function called by a foreign call?
Not easily. If you make a safe foreign call, the capability is given up before the C code executes, so you actually lose ownership of the capability and TSO by the time you're running C code. The only way to write a foreign call to properly suspend the thread that called it is a primop function (e.g. stg_putMVar), and the operation is quite delicate and you will probably want to use some existing (internal) code in the RTS to do it. But if you just need to sleep/awake threads, why not just use an MVar? Edward

On 04/11/2015 19:12, Nicola Gigante wrote:
I’m new on this list, congratulations to everyone for the great work that you’re doing on GHC!
I apologize if this is not the right list where to ask the following questions. Please point me to the correct list or IRC channel otherwise.
I’m part of a small group of people who is going to work on a project that seems to need low-level handling of haskell threads.
I’ve started delving into the ghc runtime API to understand if everything we need is exposed to the world or if we have to modify the runtime itself (which I would avoid if possible).
I have a few questions about the functions that the runtime exports to the C world regarding the manipulation of tasks:
- Fundamentally, is there a way for a C function called by a foreign call to suspend the haskell thread that called it, to be resumed later when appropriate? I see that the runtime has a concept of “blocking queue”, but I also see that the functions that work on blocking queues are not exported. I can manage a queue of TSOs myself, but then I need a way to put in sleep the task indefinitely up to a wake signal. In other words, can I sleep and awake haskell threads from a C function called by a foreign call?
When you call a C function from Haskell, the Haskell thread that called it *is* suspended, until the C call returns. So I'm not sure exactly what it is you want to do - maybe you could describe in more detail? Cheers, Simon
- I see the “suspendThread” and “resumeThread” functions exist in include/rts/Threads.h Are these the functions that I need for the point above? My main concern is that the comments in the source code for suspendThread say that these functions are used in case of a CCall, and put the thread in a specific queue, so they seems to have more specific role than what the name might suggest.
- Also, I’m unable to find what a StgRegTable is (which is taken as argument by suspendThread).
Thank you in advance for your help,
Greetings, Nicola _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Il giorno 12 nov 2015, alle ore 12:24, Simon Marlow
ha scritto: When you call a C function from Haskell, the Haskell thread that called it *is* suspended, until the C call returns. So I'm not sure exactly what it is you want to do - maybe you could describe in more detail?
Yes, the haskell thread is suspended but it will be resumed when the C code returns. If you see the haskell + C code as a single computation, nothing gets suspended, you just switch environment and language. I’m sorry for the vagueness in my question. At the time we had a blind review of a paper pending so I couldn’t give more details because some reviewer could be a member of mailing lists like this. Now I can explain at a little bit higher level what I was talking about. There are a few places in the haskell concurrency libraries where haskell threads gets suspended until something happens. MVars suspend on reads, STM transactions block after a retry to wait for the change of some TVar, and so on… Each case is internally treated explicitly by the runtime. By looking at the implementation I see that there is a concept of “Blocking Queue”, and each TSO can be added to one of these queues. Each TSO also has a why_blocked flag that tells why it is blocked, with all these cases explicitly enumerated (e.g. BlockedOnCCall and BlockedOnSTM). We are implementing some kind of concurrency primitive, which is similar in spirit to the STM but with a number of differences, and we’d need access to a primitive to block the thread exactly as it happens with the readMVar function or the retry action of the STM: simply put a thread to sleep until something happens. We are implementing a concurrency primitive, so everything has to be fast and low-overhead. As Edward pointed out, it sounds like premature optimization to a priori exclude the possibility to implement our primitive on top of MVars. For this reason, that is the way that we are exploring in the last days. Nevertheless, it does seem strange to us that the GHC runtime doesn’t expose some generic way to do something that is needed by so many subsystems. What do you think about this? Are we missing something? Thank you for your help,
Cheers, Simon
Greetings, Nicola
participants (3)
-
Edward Z. Yang
-
Nicola Gigante
-
Simon Marlow