#10474: Suggested options for "Use a bigger heap!" seem wrong -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: bug | Status: new Priority: normal | Milestone: Component: Documentation | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Description changed by ezyang: Old description:

In https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/faster.html we have guidance for how to use a bigger heap, which can speed up programs when they are spending too long in GC:

...

If your program's GC stats (-S RTS option) indicate that it's doing lots of garbage-collection (say, more than 20% of execution time), more memory might help—with the -M<size> or -A<size> RTS options (see Section 4.17.3, “RTS options to control the garbage collector”).

The suggested `-M` seems bad: `-M` won't make GHC use more memory; in fact, you'd expect it to have no/bad effect on programs. Here's one particular example benchmarked on 7.10:

ezyang@sabre:~/Dev/serum$ dist/build/serum/serum 15 gen-binaryE +RTS -s -M128m benchmarking gen-binaryE time 53.34 ms (51.51 ms .. 54.90 ms) 0.997 R² (0.992 R² .. 0.999 R²) mean 52.58 ms (51.82 ms .. 54.08 ms) std dev 1.864 ms (1.158 ms .. 2.769 ms) 3,715,656,736 bytes allocated in the heap 1,863,677,408 bytes copied during GC 2,070,152 bytes maximum residency (846 sample(s)) 102,968 bytes maximum slop 6 MB total memory in use (0 MB lost due to fragmentation)

Tot time (elapsed) Avg pause Max pause Gen 0 6235 colls, 0 par 2.508s 2.693s 0.0004s 0.0033s Gen 1 846 colls, 0 par 1.628s 1.618s 0.0019s 0.0070s

INIT time 0.000s ( 0.000s elapsed) MUT time 2.252s ( 2.702s elapsed) GC time 4.136s ( 4.312s elapsed) EXIT time 0.000s ( 0.000s elapsed) Total time 6.388s ( 7.014s elapsed)

%GC time 64.7% (61.5% elapsed)

Alloc rate 1,649,936,383 bytes per MUT second

Productivity 35.3% of total user, 32.1% of total elapsed

ezyang@sabre:~/Dev/serum$ dist/build/serum/serum 15 gen-binaryE +RTS -s -H128m benchmarking gen-binaryE time 25.02 ms (23.80 ms .. 26.15 ms) 0.988 R² (0.973 R² .. 0.997 R²) mean 28.79 ms (27.61 ms .. 30.55 ms) std dev 3.059 ms (1.871 ms .. 4.412 ms) variance introduced by outliers: 43% (moderately inflated)

6,571,453,840 bytes allocated in the heap 137,492,888 bytes copied during GC 117,280 bytes maximum residency (3 sample(s)) 35,624 bytes maximum slop 132 MB total memory in use (0 MB lost due to fragmentation)

Tot time (elapsed) Avg pause Max pause Gen 0 132 colls, 0 par 0.464s 0.475s 0.0036s 0.0115s Gen 1 3 colls, 0 par 0.004s 0.002s 0.0008s 0.0016s

INIT time 0.000s ( 0.000s elapsed) MUT time 5.556s ( 6.254s elapsed) GC time 0.468s ( 0.477s elapsed) EXIT time 0.000s ( 0.000s elapsed) Total time 6.024s ( 6.732s elapsed)

%GC time 7.8% (7.1% elapsed)

Alloc rate 1,182,767,069 bytes per MUT second

Productivity 92.2% of total user, 82.5% of total elapsed }}}

It seems to me the writer really meant to suggest `-H`.

I'll volunteer to write the patch if someone confirms.

New description: In https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/faster.html we have guidance for how to use a bigger heap, which can speed up programs when they are spending too long in GC:

If your program's GC stats (-S RTS option) indicate that it's doing lots of garbage-collection (say, more than 20% of execution time), more memory might help—with the -M<size> or -A<size> RTS options (see Section 4.17.3, “RTS options to control the garbage collector”).

The suggested `-M` seems bad: `-M` won't make GHC use more memory; in fact, you'd expect it to have no/bad effect on programs. Here's one particular example benchmarked on 7.10: {{{ ezyang@sabre:~/Dev/serum$ dist/build/serum/serum 15 gen-binaryE +RTS -s -M128m benchmarking gen-binaryE time 53.34 ms (51.51 ms .. 54.90 ms) 0.997 R² (0.992 R² .. 0.999 R²) mean 52.58 ms (51.82 ms .. 54.08 ms) std dev 1.864 ms (1.158 ms .. 2.769 ms) 3,715,656,736 bytes allocated in the heap 1,863,677,408 bytes copied during GC 2,070,152 bytes maximum residency (846 sample(s)) 102,968 bytes maximum slop 6 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 6235 colls, 0 par 2.508s 2.693s 0.0004s 0.0033s Gen 1 846 colls, 0 par 1.628s 1.618s 0.0019s 0.0070s INIT time 0.000s ( 0.000s elapsed) MUT time 2.252s ( 2.702s elapsed) GC time 4.136s ( 4.312s elapsed) EXIT time 0.000s ( 0.000s elapsed) Total time 6.388s ( 7.014s elapsed) %GC time 64.7% (61.5% elapsed) Alloc rate 1,649,936,383 bytes per MUT second Productivity 35.3% of total user, 32.1% of total elapsed ezyang@sabre:~/Dev/serum$ dist/build/serum/serum 15 gen-binaryE +RTS -s -H128m benchmarking gen-binaryE time 25.02 ms (23.80 ms .. 26.15 ms) 0.988 R² (0.973 R² .. 0.997 R²) mean 28.79 ms (27.61 ms .. 30.55 ms) std dev 3.059 ms (1.871 ms .. 4.412 ms) variance introduced by outliers: 43% (moderately inflated) 6,571,453,840 bytes allocated in the heap 137,492,888 bytes copied during GC 117,280 bytes maximum residency (3 sample(s)) 35,624 bytes maximum slop 132 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 132 colls, 0 par 0.464s 0.475s 0.0036s 0.0115s Gen 1 3 colls, 0 par 0.004s 0.002s 0.0008s 0.0016s INIT time 0.000s ( 0.000s elapsed) MUT time 5.556s ( 6.254s elapsed) GC time 0.468s ( 0.477s elapsed) EXIT time 0.000s ( 0.000s elapsed) Total time 6.024s ( 6.732s elapsed) %GC time 7.8% (7.1% elapsed) Alloc rate 1,182,767,069 bytes per MUT second Productivity 92.2% of total user, 82.5% of total elapsed }}} It seems to me the writer really meant to suggest `-H`. I'll volunteer to write the patch if someone confirms. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10474#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10474: Suggested options for "Use a bigger heap!" seem wrong -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: bug | Status: new Priority: normal | Milestone: Component: Documentation | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thoughtpolice): Yeah, this is way better. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10474#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10474: Suggested options for "Use a bigger heap!" seem wrong -------------------------------------+------------------------------------- Reporter: ezyang | Owner: ezyang Type: bug | Status: new Priority: normal | Milestone: Component: Documentation | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Documentation | Unknown/Multiple bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: Phab:D937 -------------------------------------+------------------------------------- Comment (by Edward Z. Yang ): In [changeset:"6adfb88a4eb1988d5c674513545280fc66bf1627/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="6adfb88a4eb1988d5c674513545280fc66bf1627" Suggest -H to improve GC productivity, fixes #10474. Signed-off-by: Edward Z. Yang Test Plan: none Reviewers: rwbarton, austin Subscribers: bgamari, thomie Differential Revision: https://phabricator.haskell.org/D937 GHC Trac Issues: #10474 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10474#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler