Re: [Haskell-cafe] binary IO

From: Joel Reymont To: Tomasz Zielonka CC: Jeremy Shaw ,haskell-cafe@haskell.org Subject: Re: [Haskell-cafe] binary IO Date: Tue, 27 Dec 2005 09:18:54 +0000

Tomasz,

Try http://wagerlabs.com/timeleak.tgz. See the "Killer pickler combinators" thread as well.

My desired goal is to have 4k bots (threads?) running at the same time. At, say, 1k/s per bot I figure something like 4Mb/s round-trip. Each bot cannot spend more than a couple of seconds on pickling/ unpickling. I'm not even close to hitting that goal even reading from a file.

I'm getting delays of 4s+ with just 100-200 bots reading from a file and even less than that in a networked environment. The more bots I run the higher the delays, to the point of frequent delays of 10s+. The kicker is that some packets come in at 4k compressed with Zlib but become something like 50k uncompressed and then expand to a list of 500+ records, etc.

I have C++ concurrent server that performs 2600 reqs/sec on about 500 connections and dual Xeon 2.8Ghz, but no pickling /unpickling, just short text. Has sepparate IO threads that divide descriptor sets (num descs / IO thread) and worker threads as number of CPU's * 2, no locking of shared queue. So with 4k connections I guess that would be maximum 2k requests on *dual* box per second, without pickling / unpickling, just short textual protocol and simple services. I think that you will get hard time even with C to achieve your goal. Greetings, Bane. _________________________________________________________________ Don't just search. Find. Check out the new MSN Search! http://search.msn.com/