Simon Peyton Jones via ghc-devs
When I run `sh validate -legacy`, towards the end I see stuff like this
Performance Metrics (test environment: local):
Conversions(normal) runtime/bytes allocated 107696.000
DeriveNull(normal) runtime/bytes allocated 112050960.000
InlineArrayAlloc(normal) runtime/bytes allocated 1600041088.000
InlineByteArrayAlloc(normal) runtime/bytes allocated 1440041088.000
InlineCloneArrayAlloc(normal) runtime/bytes allocated 1600041248.000
ManyAlternatives(normal) compile_time/bytes allocated 840985728.000
ManyConstructors(normal) compile_time/bytes allocated 4540766560.000
MethSharing(normal) runtime/peak_megabytes_allocated 2.000
MethSharing(normal) runtime/bytes allocated 480098136.000
MultiLayerModules(normal) compile_time/bytes allocated 5856970504.000
... It is intermingled with other apparently unrelated output.
Hmm, it really should be a distinct block of output. Are you saying that you are seeing lines of unrelated output interspersed in the performance metrics table?
What should I conclude form this? Is it good or bad? By what amount have these figures changed, and relative to what?
Below the performance metrics table you should see a blurb of text like the following: Missing Baseline Metrics these metrics trivially pass because a baseline (expected value) cannot be recovered from previous git commits. This may be due to HEAD having new tests or having expected changes, the presence of expected changes since the last run of the tests, and/or the latest test run being too old. MultiLayerModules ... If the tests exist on the previous commit (And are configured to run with the same ways), then check out that commit and run the tests to generate the missing metrics. Alternatively, a baseline may be recovered from ci results once fetched: git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git refs/notes/perf:refs/notes/ci/perf The suggested command will fetch up-to-date performance metrics from the metrics repository (populated by CI). If you then run the testsuite again you will see output comparing each test's output to the baseline from CI. For instance, Performance Metrics (test environment: local): MultiLayerModules(normal) compile_time/bytes allocated 5710260920.000 (baseline @ HEAD~1) 5726848340.000 [unchanged 0.3%] The "unchanged" here refers to the fact that the +0.3% observed change is within the indicated acceptance window of the test.
How can I run a single perf test?
Perf tests are treated like any other test. A single test can be run under Hadrian with the following: ./hadrian/build.cabal.sh test --build-root=_validatebuild \ --flavour=Validate --only=MultiLayerModules Cheers, - Ben