RE: [commit: testsuite] master: Update some perf results for master (4afbacc)

Wait. Are you really increasing "bytes allocated" from 97M to 237M? That's a big change, and in an important number. Do you know why?
Simon
| -----Original Message-----
| From: ghc-commits [mailto:ghc-commits-bounces@haskell.org] On Behalf Of
| git@git.haskell.org
| Sent: 28 November 2013 12:36
| To: ghc-commits@haskell.org
| Subject: [commit: testsuite] master: Update some perf results for
| master (4afbacc)
|
| Repository : ssh://git@git.haskell.org/testsuite
|
| On branch : master
| Link :
| http://ghc.haskell.org/trac/ghc/changeset/4afbacc299d843eb632df1f71eee2
| c91746c85a8/testsuite
|
| >---------------------------------------------------------------
|
| commit 4afbacc299d843eb632df1f71eee2c91746c85a8
| Author: Joachim Breitner

Hi, Am Donnerstag, den 28.11.2013, 12:42 +0000 schrieb Simon Peyton-Jones:
Wait. Are you really increasing "bytes allocated" from 97M to 237M? That's a big change, and in an important number. Do you know why?
no; but this is master, not my branch, and it seems to be a test not run by validate (which calls "make -C testsuite fast"), so I have to assume it could be anything in the recent past. If I get access to a fast machine to build stuff, I can try to bisect it. But that why we need a proper CI setup, which should then probably run "./validate.sh --slow"! The test suite is only half as useful as it could be if one first has to clean it up before one can reliably investigate the effect of one own’s changes. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org

On 28/11/13 13:20, Joachim Breitner wrote:
Hi,
Am Donnerstag, den 28.11.2013, 12:42 +0000 schrieb Simon Peyton-Jones:
Wait. Are you really increasing "bytes allocated" from 97M to 237M? That's a big change, and in an important number. Do you know why?
no; but this is master, not my branch, and it seems to be a test not run by validate (which calls "make -C testsuite fast"), so I have to assume it could be anything in the recent past.
But T6048 *is* run by validate. Cheers, Simon

I now get: update the test so that GHC doesn't regress again) Expected bytes allocated: 237077056 +/-10% Lower bound bytes allocated: 213369350 Upper bound bytes allocated: 260784762 Actual bytes allocated: 101150736 Joachim: if you're building with DEBUG, that will adversely affect the performance tests, so you probably shouldn't run those. Cheers, Simon On 28/11/13 12:42, Simon Peyton-Jones wrote:
Wait. Are you really increasing "bytes allocated" from 97M to 237M? That's a big change, and in an important number. Do you know why?
Simon
| -----Original Message----- | From: ghc-commits [mailto:ghc-commits-bounces@haskell.org] On Behalf Of | git@git.haskell.org | Sent: 28 November 2013 12:36 | To: ghc-commits@haskell.org | Subject: [commit: testsuite] master: Update some perf results for | master (4afbacc) | | Repository : ssh://git@git.haskell.org/testsuite | | On branch : master | Link : | http://ghc.haskell.org/trac/ghc/changeset/4afbacc299d843eb632df1f71eee2 | c91746c85a8/testsuite | | >--------------------------------------------------------------- | | commit 4afbacc299d843eb632df1f71eee2c91746c85a8 | Author: Joachim Breitner
| Date: Thu Nov 28 12:36:15 2013 +0000 | | Update some perf results for master | | | >--------------------------------------------------------------- | | 4afbacc299d843eb632df1f71eee2c91746c85a8 | tests/perf/compiler/all.T | 11 ++++++----- | tests/perf/space_leaks/all.T | 2 +- | 2 files changed, 7 insertions(+), 6 deletions(-) | | diff --git a/tests/perf/compiler/all.T b/tests/perf/compiler/all.T | index a580bf9..759b5d0 100644 | --- a/tests/perf/compiler/all.T | +++ b/tests/perf/compiler/all.T | @@ -344,10 +344,11 @@ test('T5837', | test('T6048', | [ only_ways(['optasm']), | compiler_stats_num_field('bytes allocated', | - [(wordsize(32), 48887164, 10), | - # prev: 38000000 (x86/Linux) | - # 2012-10-08: 48887164 (x86/Linux) | - (wordsize(64), 97247032, 10)]) | - # 18/09/2012 97247032 amd64/Linux | + [(wordsize(32), 48887164, 10), | + # prev: 38000000 (x86/Linux) | + # 2012-10-08: 48887164 (x86/Linux) | + (wordsize(64), 237077056, 10)]) | + # 18/09/2012 97247032 amd64/Linux | + # 2013-11-26: 237077056 amd64/Linux | ], | compile,['']) | diff --git a/tests/perf/space_leaks/all.T | b/tests/perf/space_leaks/all.T | index a1fd641..ac60c8f 100644 | --- a/tests/perf/space_leaks/all.T | +++ b/tests/perf/space_leaks/all.T | @@ -6,7 +6,7 @@ test('space_leak_001', | # 5 (x86/Linux) | [stats_num_field('peak_megabytes_allocated', (4, 1)), | stats_num_field('max_bytes_used', | - [(wordsize(64), 440000, 10), | + [(wordsize(64), 440000, 15), | # 440224 (amd64/Linux) | # 417016 (x86/OS X) | # 415672 (x86/Windows) | | _______________________________________________ | ghc-commits mailing list | ghc-commits@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-commits _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Hi, Am Donnerstag, den 28.11.2013, 13:36 +0000 schrieb Simon Marlow:
update the test so that GHC doesn't regress again) Expected bytes allocated: 237077056 +/-10% Lower bound bytes allocated: 213369350 Upper bound bytes allocated: 260784762 Actual bytes allocated: 101150736
Joachim: if you're building with DEBUG, that will adversely affect the performance tests, so you probably shouldn't run those.
ah, ok, sorry, that’s it. My bad. With these you mean perf/compiler, but probably not all perf tests, right? Could we mark those tests as with when(compiler_debugged(),skip()) then? At the risk of repeating my self, and especially with a ever growing number of contributors to GHC, especially new ones, the test should should give reproducible and correct results. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org

On 28/11/13 13:48, Joachim Breitner wrote:
Hi,
Am Donnerstag, den 28.11.2013, 13:36 +0000 schrieb Simon Marlow:
update the test so that GHC doesn't regress again) Expected bytes allocated: 237077056 +/-10% Lower bound bytes allocated: 213369350 Upper bound bytes allocated: 260784762 Actual bytes allocated: 101150736
Joachim: if you're building with DEBUG, that will adversely affect the performance tests, so you probably shouldn't run those.
ah, ok, sorry, that’s it. My bad.
With these you mean perf/compiler, but probably not all perf tests, right? Could we mark those tests as with when(compiler_debugged(),skip()) then?
Yes, that would be good.
At the risk of repeating my self, and especially with a ever growing number of contributors to GHC, especially new ones, the test should should give reproducible and correct results.
Of course! If you find cases where that's not true, please go ahead and fix them. Cheers, Simon

Hi, Am Donnerstag, den 28.11.2013, 14:11 +0000 schrieb Simon Marlow:
With these you mean perf/compiler, but probably not all perf tests, right? Could we mark those tests as with when(compiler_debugged(),skip()) then?
Yes, that would be good.
Done in the test runner for all tests with compiler_stats_num_field, and mentioned on https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Adding in the documentation for compiler_stats_num_field. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata@debian.org
participants (3)
-
Joachim Breitner
-
Simon Marlow
-
Simon Peyton-Jones