Parallel performance drops off a cliff
Hi, Using one benchmark I have, which doesn't create any threads, I have: $ benchmark +RTS -Nx x time (Seconds) 1 2 2 2 3 2 4 3 5 3 6 3 7 3 8 aborted after 2 minutes This is using GHC 6.10.2 on Windows XP, 2 processors. Is this a known bug, or should I try and replicate it? (benchmark is fairly big and very dependent on internal things, but I suspect the dramatic performance slowdown is unlikely to be related to these bits). Thanks Neil
2009/4/20 Neil Mitchell
Hi,
Using one benchmark I have, which doesn't create any threads, I have:
$ benchmark +RTS -Nx
x time (Seconds) 1 2 2 2 3 2 4 3 5 3 6 3 7 3 8 aborted after 2 minutes
This is using GHC 6.10.2 on Windows XP, 2 processors. Is this a known bug, or should I try and replicate it? (benchmark is fairly big and very dependent on internal things, but I suspect the dramatic performance slowdown is unlikely to be related to these bits).
Yes, what's happening is this: GHC 6.10.2 contains some slightly bogus heuristics about when to turn on the parallel GC, and it just so happens that 8 processors tips it over the point where the parallel GC is enabled for young-generation collections. In 6.10.2 the parallel GC really didn't help most of the time, but it has undergone a lot of tuning since then, and in the HEAD things are much better (see the results from our ICFP submission). In the meantime you might get somewhere by disabling parallel GC altogether (+RTS -g1), but as the results in our paper show, sometimes the parallel GC is essential for retaining locality in parallel programs. Cheers, Simon
Hi
This is using GHC 6.10.2 on Windows XP, 2 processors. Is this a known bug, or should I try and replicate it? (benchmark is fairly big and very dependent on internal things, but I suspect the dramatic performance slowdown is unlikely to be related to these bits).
Yes, what's happening is this: GHC 6.10.2 contains some slightly bogus heuristics about when to turn on the parallel GC, and it just so happens that 8 processors tips it over the point where the parallel GC is enabled for young-generation collections. In 6.10.2 the parallel GC really didn't help most of the time, but it has undergone a lot of tuning since then, and in the HEAD things are much better (see the results from our ICFP submission).
Fantastic! I'll disable parallel garbage collection (I'm fairly certain it won't have any effect for this application). Thanks Neil
ndmitchell:
Hi
This is using GHC 6.10.2 on Windows XP, 2 processors. Is this a known bug, or should I try and replicate it? (benchmark is fairly big and very dependent on internal things, but I suspect the dramatic performance slowdown is unlikely to be related to these bits).
Yes, what's happening is this: GHC 6.10.2 contains some slightly bogus heuristics about when to turn on the parallel GC, and it just so happens that 8 processors tips it over the point where the parallel GC is enabled for young-generation collections. In 6.10.2 the parallel GC really didn't help most of the time, but it has undergone a lot of tuning since then, and in the HEAD things are much better (see the results from our ICFP submission).
Fantastic! I'll disable parallel garbage collection (I'm fairly certain it won't have any effect for this application).
How many cores are you planning to use? ... -- Don
Yes, what's happening is this: GHC 6.10.2 contains some slightly bogus heuristics about when to turn on the parallel GC, and it just so happens that 8 processors tips it over the point where the parallel GC is enabled for young-generation collections. In 6.10.2 the parallel GC really didn't help most of the time, but it has undergone a lot of tuning since then, and in the HEAD things are much better (see the results from our ICFP submission).
Fantastic! I'll disable parallel garbage collection (I'm fairly certain it won't have any effect for this application).
How many cores are you planning to use? ...
16 on the best machine we've got. This particular application is a bit weird, I suspect it will be spending very little time actually executing Haskell and most of the time waiting for shell commands. As such, the efficiency of Haskell isn't a problem - parallel garbage collection shouldn't make any difference. Thanks Neil
participants (3)
-
Don Stewart -
Neil Mitchell -
Simon Marlow