On Dec 13, 2005, at 9:49 AM, Tomasz Zielonka wrote:
On Mon, Dec 12, 2005 at 04:00:46PM +0000, Joel Reymont wrote:
One particular thing that bugs me is that I cannot really use TChan for thread mailboxes. I don't think I experienced this problem with Erlang but using a TChan with a logger thread quickly overwhelms the logger and fills the TChan and a lot (hundreds? thousands) of other threads are logging to it.
I wonder what Erlang does to solve this problem? Perhaps we should track the number of unprocessed messages in TChans and the bigger it is the more favor consumers over producers.
It sounds to me that introducing thread priorities would be key. Here's a reply from Ulf Wiger (and Erlang expert): Erlang has four process priorities: - 'max' and 'high' are strict priorities (you stay at that level while there are processes ready to run) - normal and low are scheduled "fairly", I believe with 8000 "reductions" (roughly function calls) at normal priority and one low-priority job (if such a job exists) The scheduler works on reduction count. A context switch happens if the current process would block (e.g. if it's in a receive statement and there is no matching message in the queue), or when it's executed 1000 reductions. Since not all operations are equal in cost, there is an internal function called erlang:bump_reductions(N). File operations are usually followed by a call to erlang:bump_reductions(100) (see prim_file:get_drv_response/1) which means that processes writing to disk run out of their time slice in fewer function calls. This is of course to keep them from getting an unfair amount of CPU time. A logging process would probably therefore do well to fetch all messages in the message queue and write them using disk_log:alog_terms/2 (logging multiple messages each time). One could also possibly run the logger process at high priority. This means that normal priority process will have a hard time starving it. If the disk is slow, the logger process will yield while waiting for the disk (which won't block the runtime system as long as you have the thread pool enabled). In general, I think that processes that mainly dispatch messages, and don't generate any work on their own, should usually run on high priority. Otherwise, they tend to just contribute to delays in the system. /Uffe -- http://wagerlabs.com/