On Tue, Dec 27, 2005 at 09:18:54AM +0000, Joel Reymont wrote:
Try http://wagerlabs.com/timeleak.tgz. See the "Killer pickler combinators" thread as well.
Let's see if I understand correctly. There are 17605 messages in trace.dat. On my hardware the average message unpicking time is 0.0002s when you only have a single thread. So, it indeed seems that with 1000 threads it should be possible to process every message in under 4 seconds. Right now I can think of two reasons: - 1000 treads need much data in the help, which increases the cost of GC and with frequent context switches results in poor cache performance - the GHC's process scheduler is not as intelligent as Erlang's
I'm getting delays of 4s+ with just 100-200 bots reading from a file and even less than that in a networked environment. The more bots I run the higher the delays, to the point of frequent delays of 10s+. The kicker is that some packets come in at 4k compressed with Zlib but become something like 50k uncompressed and then expand to a list of 500+ records, etc.
One possible solution is to reduce the number of simultaneously running unpicklings/ picklings (I think someone already proposed it, was that Bulat?). It would reduce the amount of needed memory and should improve cache usage. But then, what will be the delay observed by the server? Anyway, I've made some tests, and I see the problem persists. I'll try to test with my unpickling code.
Each bot is given 5, 15 or 35 seconds to respond by the poker server and this is proving to be too little for my Haskell implementation.
This is per message, right?
My only requirement is that there be a _single_ spec for pickling and unpickling, i.e. no separate methods. The following is not acceptable to me ;-).
What if the spec was the data type itself? When I was dealing with a proprietary Intel-based binary format, I derived my picklers / unpicklers with TH from data type definitions. Of course, there were cases were the derived code would be wrong, and then I had to write the code myself, but it was for about 2-3 record types out of total 30. Best regards Tomasz -- I am searching for a programmer who is good at least in some of [Haskell, ML, C++, Linux, FreeBSD, math] for work in Warsaw, Poland