Number of outstanding messages in Chan or TChan
Is there a way to check the number of outstanding messages in a Chan or TChan? I suspect my problem is related to not reading messages fast enough and I'd like to troubleshoot this by scanning my channels once every few seconds and dumping statistics. Thanks, Joel -- http://wagerlabs.com/
On Tue, Dec 06, 2005 at 09:54:57AM +0000, Joel Reymont wrote:
Is there a way to check the number of outstanding messages in a Chan or TChan?
I suspect my problem is related to not reading messages fast enough and I'd like to troubleshoot this by scanning my channels once every few seconds and dumping statistics.
I would create a new ADT consisting of a TChan and TVar Int, increased on writeTChan and decreased on readTChan. Indeed, not reading messages fast enough can cause something similar to space leak. Perhaps use a bounded TChan, it should be quite easy to do in STM. Best regards Tomasz -- I am searching for a programmer who is good at least in some of [Haskell, ML, C++, Linux, FreeBSD, math] for work in Warsaw, Poland
Hello Joel, Tuesday, December 06, 2005, 12:54:57 PM, you wrote: JR> Is there a way to check the number of outstanding messages in a Chan JR> or TChan? either 1) use MVars/TMVars instead of Channels. in any case your logging thread must consume data not slower than other channels produce then. in fact, using Chan have meaning only to smooth temporary speed differences between different threads. are you really need this?? 2) i hope that you already replaced passing a Chan to subroutines with passing an actions that read/write this Chan. in this case you can go further and add to this actions incrementing/decrementing MVar counter. but even without tests it's evident that 1000 producer threads will get 1000 times more attention than 1 consumer thread if you don't have any restrictions on producing and consuming data. if you absolutely don't want to use MVars instead of channels, then at least you can modify these actions so that number of Chan elements will be limited (use additional "MVar full" which is filled in case of too large number of elements in channel and emptied by logger thread) -- Best regards, Bulat mailto:bulatz@HotPOP.com
On Dec 6, 2005, at 1:47 PM, Bulat Ziganshin wrote:
either 1) use MVars/TMVars instead of Channels. in any case your logging thread must consume data not slower than other channels produce then. in fact, using Chan have meaning only to smooth temporary speed differences between different threads. are you really need this??
This makes total sense for the logger thread. I will implement this.
2) i hope that you already replaced passing a Chan to subroutines with passing an actions that read/write this Chan. in this case you can go further and add to this actions incrementing/decrementing MVar counter. but even without tests it's evident that 1000 producer threads will get 1000 times more attention than 1 consumer thread if you don't have any restrictions on producing and consuming data.
The problem is that I have hundreds of other network client threads that do not directly read from a socket. They communicate through TChans instead. I believe they block someplace and I'm trying to troubleshoot this. Of course moving the logger to use TMVars instead of TChan might immediately solve my problems. Thanks, Joel -- http://wagerlabs.com/
I must eat crow :-(. Moving the thread and logger mailboxes from TChan to TMVar reduced memory consumption by an order of magnitude. I found my space leak. Moving serialization from [Word8] to unboxed arrays did not hurt either. On Dec 6, 2005, at 1:47 PM, Bulat Ziganshin wrote:
either 1) use MVars/TMVars instead of Channels. in any case your logging thread must consume data not slower than other channels produce then. in fact, using Chan have meaning only to smooth temporary speed differences between different threads. are you really need this??
Hello Joel, Tuesday, December 06, 2005, 6:42:32 PM, you wrote: JR> On Dec 6, 2005, at 1:47 PM, Bulat Ziganshin wrote:
either 1) use MVars/TMVars instead of Channels. in any case your logging thread must consume data not slower than other channels produce then. in fact, using Chan have meaning only to smooth temporary speed differences between different threads. are you really need this??
btw, why not pass to worker thread just the logging action itself? and, about waitForChildren: waitForChildren :: IO () waitForChildren = do logDead <- newEmptyMVar forkIO (logger `finally` putMVar logDead ()) c <- takeMVar children mapM_ takeMVar c ...send Nothing to logger thread takeMVar logDead as simple as possible -- Best regards, Bulat mailto:bulatz@HotPOP.com
participants (3)
-
Bulat Ziganshin -
Joel Reymont -
Tomasz Zielonka