[GHC] #10557: Use `+RTS -G1` for more stable residency measurements
#10557: Use `+RTS -G1` for more stable residency measurements -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: task | Status: new Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Tests that measure `peak_megabytes_allocated` and `max_bytes_used` can be very sensitive to small changes in, well, just about anything: order of command line flags, what should be irrelevant compiler patches, minor changes in the test file to compile. From `Note [residency]` in `testsuite/tests/perf/compiler/all.T`: {{{ # Residency (peak_megabytes_allocated and max_bytes_used) is sensitive # to when the major GC runs, which makes it inherently inaccurate. # Sometime an innocuous change somewhere can shift things around such # that the samples occur at a different time, and the residency # appears to change (up or down) when the underlying profile hasn't # really changed. }}} I propose setting `+RTS -G1` for these tests. This seems to make them more robust to small changes, as can be seen in the following two graphs, adapted from the test for #9675: https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-peak_megabytes... https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-max_bytes_used... This shows the `peak_megabytes_allocated` and `max_bytes_used` as a function of the number fields when compiling a single record: {{{ data Foo = Foo { field1 :: Int -> Int , field2 :: Int -> Int ... }}} The default `GHC -O` line is wiggling up and down, while the `GHC -O +RTS -G1` line is nice and smooth, which is what we want. I don't know if making this change defeats the purpose of some tests, which is why I'm opening this ticket. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10557 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10557: Use `+RTS -G1` for more stable residency measurements -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: task | Status: new Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): Sounds like a jolly good idea to me. If it defeats the purpose for something, we can change the something -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10557#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10557: Use `+RTS -G1` for more stable residency measurements
-------------------------------------+-------------------------------------
Reporter: thomie | Owner:
Type: task | Status: new
Priority: normal | Milestone: 7.12.1
Component: Test Suite | Version: 7.10.1
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
Type of failure: None/Unknown | Unknown/Multiple
Blocked By: | Test Case:
Related Tickets: | Blocking:
| Differential Revisions:
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#10557: Use `+RTS -G1` for more stable residency measurements -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: task | Status: merge Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => merge -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10557#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10557: Use `+RTS -G1` for more stable residency measurements -------------------------------------+------------------------------------- Reporter: thomie | Owner: Type: task | Status: closed Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.10.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10557#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC