On 04/08/2014 13:13, Joachim Breitner wrote:
Hi,
Am Montag, den 04.08.2014, 13:08 +0200 schrieb Joachim Breitner:
Am Montag, den 04.08.2014, 12:02 +0100 schrieb Edward Z.Yang:
Yes, on my box, this test is now failing (because the stat is too good):
Expected haddock.base(normal) max_bytes_used: 127954488 +/-10% Lower bound haddock.base(normal) max_bytes_used: 115159039 Upper bound haddock.base(normal) max_bytes_used: 140749937 Actual haddock.base(normal) max_bytes_used: 113167424 Deviation haddock.base(normal) max_bytes_used: -11.6 %
ugh.
What are your compilation settings? Plain "validate"?
Looks like the ghcspeed instance settings still don’t quite match what validate does...
But I don’t see anything in mk/validate-settings.mk which would yield different results than echo 'GhcLibHcOpts += -O -dcore-lint' >> mk/build.mk echo 'GhcStage2HcOpts += -O -dcore-lint' >> mk/build.mk
I’m starting a plain validate run on that machine, to see if it is compilation settings or some other variable.
validate goes through without a problem. So it seems to be dependent on other things.
Are these very flaky measures (max_bytes_used) at all useful? So far, I have only seen friction due to them, and any real problem would likely be caught by either bytes_allocated or nofib measurements (I hope). Maybe we should simply remove them from the test suite, and stop worrying?
From testsuite/tests/perf/compiler/all.T: # Note [residency] # # Residency (peak_megabytes_allocated and max_bytes_used) is sensitive # to when the major GC runs, which makes it inherently inaccurate. # Sometime an innocuous change somewhere can shift things around such # that the samples occur at a different time, and the residency # appears to change (up or down) when the underlying profile hasn't # really changed. # # However, please don't just ignore changes in residency. If you see # a change in one of these figures, please check whether it is real or # not as follows: # # * Run the test with old and new compilers, adding +RTS -h -i0.01 # (you don't need to compile anything for profiling or enable profiling # libraries to get a heap profile). # * view the heap profiles, read off the maximum residency. If it has # really changed, then you know there's an issue. This advice is hard to follow for the Haddock tests, because the test doesn't actually run anything, it just uses the information previously collected. I think we should probably remove the max_bytes_used limits for the Haddock tests. Cheers, Simon