#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by varosi): I vote for this! It's not hard to be implemented I hope, but affect performance a lot on more hardware. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by carter): do we mean the Nursery or the Gen1 heap after the nursery? I'd imagine we want the nursery to fit in L1 or L2 caches (where applicable) and the Gen1 heap to fit in the rest of the Cache left in Level3 after we account for nurseries? Perhaps {{{ size of nursery = size of L2 cache per cpu core size of gen1 >= max(#capabilities * size of nursery , size of L3 cache in socket - (#capabilities * size of nursery) ) }}} we definitely (at least in many core systems) do *not* want nurseries on the same Socket creating cache thrash with eachother (ie under heavy allocation workloads)? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by varosi): For nursery may be there is no such deal, but see a benchmark made on a small raytracer (https://bitbucket.org/varosi/cgraytrace/overview) that do a lot of allocations on two different machines: https://docs.google.com/spreadsheets/d/1dnhQTrm_EgKab3IJQAC4Rw1IJOfOcryJ56W0... There is a clear difference for different GC sizes. As sizes grow beyond times of needed memory GC is almost not used and then the benefit of larger GC is brought back. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by sjakobi: Old description:

This will improve performance a lot by default.

If anyone needs different size - there is "-A" RTS option. Machines are very different. Currently this size is 512k by default for all different architectures and hardware no matter what. In most cases machines with larger caches have more RAM as well and vice versa. So this will affect positively both small and larger machines. It will be most efficient in most cases to keep short lived objects in caches. Most modern workstation and server machines have L3 cache as well, that is why I'm asking for "largest cache size".

Second idea will be if there are two short lived generations on machines with second and third level caches with sizes that match both.

For NUMA machines with non-unified caches (like [http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143452239825 this strange and non-common ARM]) the common solution could be to set first generation to be with size of the largest cache of smallest core. Which will not be the optimal, but close to.

[https://www.reddit.com/r/haskell/comments/5rf1r5/ghc_first_generation_of_gc_... Related Reddit discussion]

New description: This will improve performance a lot by default. If anyone needs different size - there is "-A" RTS option. Machines are very different. Since 8.2 this size has been 1MB by default for all different architectures and hardware no matter what. In most cases machines with larger caches have more RAM as well and vice versa. So this will affect positively both small and larger machines. It will be most efficient in most cases to keep short lived objects in caches. Most modern workstation and server machines have L3 cache as well, that is why I'm asking for "largest cache size". Second idea will be if there are two short lived generations on machines with second and third level caches with sizes that match both. For NUMA machines with non-unified caches (like [http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143452239825 this strange and non-common ARM]) the common solution could be to set first generation to be with size of the largest cache of smallest core. Which will not be the optimal, but close to. [https://www.reddit.com/r/haskell/comments/5rf1r5/ghc_first_generation_of_gc_... Related Reddit discussion] -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: sjakobi Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by sjakobi): I have [https://github.com/sjakobi/ghc/tree/T13362 a branch] that works for me on Windows 10 and Linux using an i7-4790K CPU. It would be great if y'all could test this on: * macOS / OS X * FreeBSD * ARM and other non-x86 architectures * Intel Haswell and Broadwell CPUs with L4 cache To ensure that the code works as intended, run a "Hello world"-program with `+RTS -s` and check that the report shows `(N+1) MB total memory in use` where `N` MB is the size of your largest cache. PRs to support other operating systems are also very welcome! :) Open design questions as of now: 1. If we only find an L1 cache, should we really go with an allocation area of typically just 32 or 64 kB? IMHO it might be better to ignore any L1 caches and to simply default to the old 1 MB in these cases. 2. What if we find an L4 cache with 64 or 128 **MB**? This would easier to decide if we got some benchmark results, for example in the style of [comment:7 varosi's]. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: sjakobi Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by sjakobi): Replying to [comment:12 varosi]:

Great! Is it possible to share your Windows executable so I could experiment on a few machines from a few cores up to close to hundred?

You can download a binary distribution [https://drive.google.com/file/d/1sNf93dZ9KEZT6yYdfSCM3fAVxn7YyFil/view?usp=s... here]. It's not an optimized build though, so at least building with it should be slower than with official releases. Regarding running on Windows machines with close to a hundred cores, the current implementation will only detect caches within its current processor group of at most 64 logical processors (see "Remarks" [https://msdn.microsoft.com/en- us/library/windows/desktop/ms683194(v=vs.85).aspx here]). As long as there aren't any larger caches outside of the processor group it will still set the allocation area to the correct size. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

Replying to [comment:12 varosi]:

...

Great! Is it possible to share your Windows executable so I could experiment on a few machines from a few cores up to close to hundred?

You can download a binary distribution [https://drive.google.com/file/d/1sNf93dZ9KEZT6yYdfSCM3fAVxn7YyFil/view?usp=s... here]. It's not an optimized build though, so at least building with it should be slower than with official releases.

Regarding running on Windows machines with close to a hundred cores, the current implementation will only detect caches within its current

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: sjakobi Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by varosi): How can I experiment with non-optimized version as I doesn't have reference for comparison? I could try to build some optimized version. Replying to [comment:13 sjakobi]: processor group of at most 64 logical processors (see "Remarks" [https://msdn.microsoft.com/en- us/library/windows/desktop/ms683194(v=vs.85).aspx here]). As long as there aren't any larger caches outside of the processor group it will still set the allocation area to the correct size. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: sjakobi Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by varosi): I cannot build my ray-tracer with the new GHC version, but I'll try to fix problems and get back here. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4679 Wiki Page: | -------------------------------------+------------------------------------- Changes (by sjakobi): * owner: sjakobi => (none) Comment: There are several interesting comments on ​Phab:D4679 pointing out that a more intricate method is required for auto-sizing the allocation area. As I need to focus on my GSoC project (and due to my lack of understanding of the GC) I'm unlikely to be of much further help with this ticket. I'm also unsure whether this is still a good newcomer ticket. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: 8.8.1 Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomers Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4679 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.6.1 => 8.8.1 Comment: We won't be doing this for 8.6. Bumping to 8.8. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13362: GHC first generation of GC to be as large as largest cache size by default -------------------------------------+------------------------------------- Reporter: varosi | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: Component: Runtime System | Version: 8.0.2 Resolution: | Keywords: numa cache gc | newcomer Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4679 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.8.1 => Comment: This will certainly need more measurement before we do this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13362#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler