Tomasz Zielonka wrote:
On Thu, Dec 29, 2005 at 01:20:41PM +0000, Joel Reymont wrote:
Why does it take a fraction of a second for 1 thread to unpickle and several seconds per thread for several threads to do it at the same time? I think this is where the mistery lies.
Have you considered any of this:
- too big memory pressure: more memory means more frequent and more expensive GCs, 1000 threads using so much memory means bad cache performance - a deficiency of GHC's thread scheduler - giving too much time one thread steals it from others (Simons, don't get angry at me - I am probably wrong here ;-)
I don't think there's anything really strange going on here. The default context switch interval in GHC is 0.02 seconds, measured in CPU time by default. GHC's scheduler is stricly round-robin, so therefore with 100 threads in the system it can be 2 seconds between a thread being descheduled and scheduled again. I measured the time taken to unpickle those large 50k packets as 0.3 seconds on my amd64 box (program compiled *without* optimisation), so the thread can get descheduled twice during while unpickling a large packet, giving a >4s delay with 100 threads running. The actual context switch interval seems to often be larger than 0.2 seconds; I'm not sure exactly why this is, it might be due to delays in the OS delivering the signal. This does mean that the timeleak program reports alerts for as little as 50 threads, though. Cheers, Simon