secret of light-weight user thread
Hello, Recently I exchanged information about user threads with Ruby community in Japan. The user threads of Ruby 1.8 are heavy weight and Ruby 1.9 switched to kernel threads. The reason why user threads of Ruby 1.8 are heavy weight is *portability*. Since Ruby community does not want to prepare assembler to change stack pointers for each supported CPU architecture, Ruby 1.8 copies the stack of user threads on context switch. Because I need to explain why the user threads of GHC are light weight, I gave a look at GHC's source code and found the loadThreadState function in compiler/codeGen/StgCmmForeign.hs. In this function, the stack pointer is changed in the Cmm level. So, my interpretation is as follows: Since GHC has Cmm backend, it is easy to provide assembler to change stack pointers for each supported CPU architecture. That's why GHC can provide light weight user threads. Is my understanding correct? Thanks in advance. --Kazu
On 06/09/2011 09:47, Kazu Yamamoto (山本和彦) wrote:
Recently I exchanged information about user threads with Ruby community in Japan.
The user threads of Ruby 1.8 are heavy weight and Ruby 1.9 switched to kernel threads. The reason why user threads of Ruby 1.8 are heavy weight is *portability*. Since Ruby community does not want to prepare assembler to change stack pointers for each supported CPU architecture, Ruby 1.8 copies the stack of user threads on context switch.
Because I need to explain why the user threads of GHC are light weight, I gave a look at GHC's source code and found the loadThreadState function in compiler/codeGen/StgCmmForeign.hs. In this function, the stack pointer is changed in the Cmm level.
So, my interpretation is as follows: Since GHC has Cmm backend, it is easy to provide assembler to change stack pointers for each supported CPU architecture. That's why GHC can provide light weight user threads.
Is my understanding correct?
There are a couple of reasons why GHC's threads are cheaper than OS threads, it's not really to do with the Cmm backend: - We have an accurate GC, which means that the Haskell stack can be movable, whereas the C stack isn't. So we can start with small stacks and enlarge them as necessary. - We only preempt threads at safe points. This means that the context we have to save at a context switch is platform-independent and typically much smaller than the entire CPU context. Safe points are currently on allocation, which is frequent enough in GHC for this to be indistinguishable (most of the time) from true preemption. User-space threads are often dismissed because of the problems with implementing concurrent foreign calls. We have a nice solution for this problem in GHC that I think is probably under-appreciated in the wider language community: http://community.haskell.org/~simonmar/papers/conc-ffi.pdf Cheers, Simon
Simon, Thank you for explanation.
- We have an accurate GC, which means that the Haskell stack can be movable, whereas the C stack isn't. So we can start with small stacks and enlarge them as necessary.
What is the difference between the Haskell stack and the C stack? I guess that the C stack means CPU's stack. Is the Haskell stack a virtual stack for a virtual machine (STG machine or something)? I quickly read several papers but I have no idea.
- We only preempt threads at safe points. This means that the context we have to save at a context switch is platform-independent and typically much smaller than the entire CPU context. Safe points are currently on allocation, which is frequent enough in GHC for this to be indistinguishable (most of the time) from true preemption.
I seems to me that StgRun saves CPU registers. You mean that StgRun depends on CPU a bit but the rest part of context is CPU independent? --Kazu
On 07/09/2011 08:13, Kazu Yamamoto (山本和彦) wrote:
Simon,
Thank you for explanation.
- We have an accurate GC, which means that the Haskell stack can be movable, whereas the C stack isn't. So we can start with small stacks and enlarge them as necessary.
What is the difference between the Haskell stack and the C stack? I guess that the C stack means CPU's stack. Is the Haskell stack a virtual stack for a virtual machine (STG machine or something)?
There's no fundamental difference between the Haskell stack and the C stack, they are both just runtime data structures used by compiled code. We designed the Haskell stack so that pointers within it can be identified by the GC, that's all. When running Haskell code there's a register that points to the top of the Haskell stack, just like when running C code (it's a different register, but in principle there's no reason it has to be different).
I quickly read several papers but I have no idea.
- We only preempt threads at safe points. This means that the context we have to save at a context switch is platform-independent and typically much smaller than the entire CPU context. Safe points are currently on allocation, which is frequent enough in GHC for this to be indistinguishable (most of the time) from true preemption.
I seems to me that StgRun saves CPU registers. You mean that StgRun depends on CPU a bit but the rest part of context is CPU independent?
StgRun is the interface between the C world and the Haskell world, which have different ABIs. In particular, the C ABI requires that function calls do not modify certain registers (the callee-saves registers), whereas in Haskell there are no callee-saves registers. So StgRun saves the callee-saves registers while running Haskell code, that's all. It may have to do other things depending on what specific conventions are used by C or Haskell code on the current platform. This is just something we have to do so that we can call Haskell code from C. It's not related to threads, except that the GHC scheduler is written in C so we have to go through StgRun every time we start or stop a Haskell thread. Cheers, Simon
Hello Simon, Now everything is clear to me. I wrote a blog article on this in Japanese. Thanks. --Kazu
On 07/09/2011 08:13, Kazu Yamamoto (山本和彦) wrote:
Simon,
Thank you for explanation.
- We have an accurate GC, which means that the Haskell stack can be movable, whereas the C stack isn't. So we can start with small stacks and enlarge them as necessary.
What is the difference between the Haskell stack and the C stack? I guess that the C stack means CPU's stack. Is the Haskell stack a virtual stack for a virtual machine (STG machine or something)?
There's no fundamental difference between the Haskell stack and the C stack, they are both just runtime data structures used by compiled code. We designed the Haskell stack so that pointers within it can be identified by the GC, that's all.
When running Haskell code there's a register that points to the top of the Haskell stack, just like when running C code (it's a different register, but in principle there's no reason it has to be different).
I quickly read several papers but I have no idea.
- We only preempt threads at safe points. This means that the context we have to save at a context switch is platform-independent and typically much smaller than the entire CPU context. Safe points are currently on allocation, which is frequent enough in GHC for this to be indistinguishable (most of the time) from true preemption.
I seems to me that StgRun saves CPU registers. You mean that StgRun depends on CPU a bit but the rest part of context is CPU independent?
StgRun is the interface between the C world and the Haskell world, which have different ABIs. In particular, the C ABI requires that function calls do not modify certain registers (the callee-saves registers), whereas in Haskell there are no callee-saves registers. So StgRun saves the callee-saves registers while running Haskell code, that's all. It may have to do other things depending on what specific conventions are used by C or Haskell code on the current platform.
This is just something we have to do so that we can call Haskell code from C. It's not related to threads, except that the GHC scheduler is written in C so we have to go through StgRun every time we start or stop a Haskell thread.
Cheers, Simon
participants (2)
-
Kazu Yamamoto -
Simon Marlow