[GHC] #7847: Maintain per-generation lists of weak pointers
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: new Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Currently the runtime system keeps a list of all live weak pointers, and traverses it once every (major or minor) GC. This is very slow if the program keeps many weak pointers alive. Some comments in the rts source suggest that it should be using one weak pointer list per generation, so these patches implement that. The attached test program tries to imitate the memory behavior of code that uses a particular FRP library. The patches make the program 3x faster on my machine. One problem I can see with the patches is that it creates a race condition between finalizeForeignPtr and addForeignPtrFinalizer. I'm not even sure what the correct behavior is when addForeignPtrFinalizer is called after finalizerForeinPtr. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Changes (by akio): * status: new => patch -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Changes (by liyang): * cc: hackage.haskell.org@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Comment(by akio): I updated and re-validated the patches, following the suggestions given by Edward Yang on ghc-devs. I believe now the code doesn't have the addCFianlizerToWeak# / finalizeWeak# race condition. It also fixes a finalizeWeak# / finalizeWeak# race condition that existed before my changes. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Comment(by akio): I updated the patches again, thanks to more comments by Edward Yang. Now addCFinalizerToWeak# and finalizeWeak# use lockClosure() rather than cas() for synchronization. I believe the first patch is much simpler now. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Comment(by akio): Updated and rebased the patches. A remaining race condition between finalizeForeignPtr and addForeignPtrFinalizer has been fixed. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Comment(by akio): Updated comments in the patches. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Changes (by AndreasVoellmy): * cc: andreas.voellmy@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers -----------------------------------------+---------------------------------- Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Component: Runtime System Version: 7.7 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Comment(by akio): I rebased the patches. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers ---------------------------------+------------------------------------------ Reporter: akio | Owner: Type: feature request | Status: infoneeded Priority: normal | Milestone: Component: Runtime System | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Compile-time performance bug Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by simonmar): * status: patch => infoneeded * difficulty: => Unknown Comment: patch 0001 looks good to me. On patch 0002: * Why do we have `dead_weak_ptr_list_tail` rather than just consing new items on the front of the list? If you haven't already, I recommend testing these patches with `libraries/base/tests/memo001.hs` and `libraries/base/tests/memo002.hs`. Those are pretty good stress tests of the weak pointer implementation. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers ---------------------------------+------------------------------------------ Reporter: akio | Owner: Type: feature request | Status: infoneeded Priority: normal | Milestone: Component: Runtime System | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Compile-time performance bug Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by akio): Thank you for review. Replying to [comment:9 simonmar]:
patch 0001 looks good to me.
On patch 0002:
* Why do we have `dead_weak_ptr_list_tail` rather than just consing new items on the front of the list?
This is a trace of an earlier design that survived by accident. I'll clean it up in a few days.
If you haven't already, I recommend testing these patches with
`libraries/base/tests/memo001.hs` and `libraries/base/tests/memo002.hs`. Those are pretty good stress tests of the weak pointer implementation. Yes, the patches passed these tests (and the rest of validation). -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers ---------------------------------+------------------------------------------ Reporter: akio | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Runtime System | Version: 7.7 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: Compile-time performance bug Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by akio): * status: infoneeded => patch Comment: I cleaned up the code and removed `weak_ptr_list_tail` and `dead_weak_ptr_list_tail`. Rebased and validated. This change caused one of the test output to change, so I also attached a patch to the test suite. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7847: Maintain per-generation lists of weak pointers
---------------------------------+------------------------------------------
Reporter: akio | Owner:
Type: feature request | Status: patch
Priority: normal | Milestone:
Component: Runtime System | Version: 7.7
Keywords: | Os: Unknown/Multiple
Architecture: Unknown/Multiple | Failure: Compile-time performance bug
Difficulty: Unknown | Testcase:
Blockedby: | Blocking:
Related: |
---------------------------------+------------------------------------------
Comment(by aljee@…):
commit fe652a8b56c864167ecf1fac899bb3d99363dfcf
{{{
Author: Takano Akio
#7847: Maintain per-generation lists of weak pointers -------------------------------------------+-------------------------------- Reporter: akio | Owner: Type: feature request | Status: closed Priority: normal | Milestone: Component: Runtime System | Version: 7.7 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by igloo): * status: patch => closed * resolution: => fixed Comment: All applied, thanks -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7847#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (2)
-
GHC -
GHC