[GHC] #8039: RTS linker: unloadObj() does not actually unload the code
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ We've known about this for a long time, but it hasn't been a pressing issue in GHCi. The problem with actually unloading code is that there might be pointers into it from the heap or other RTS data structures, so we need to do a full heap traversal to discover whether it is safe to unload code or not. This is an issue for a project at Facebook that needs to have long running processes that regularly unload and load code using the RTS linker, and right now the process grows over time. This ticket is to track the issue, and I'm also working on a fix. The same problem affects the dynamic linker, although I'm not planning to fix that (yet). -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by ezyang): Excellent! I've run into this problem, and I'm curious what the fix's approach is. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by simonmar): I've attached a patch that sort of works. There's at least one problem still to fix (it doesn't get the static objects correctly when the GC is running in parallel). It seems to work with some simple testing in GHCi, but I still need to make some automated test cases. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by igloo): By "in GHCi", I assumed you mean in 7.6, or HEAD with `DYNAMIC_GHC_PROGRAMS=NO`? -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by ezyang): OK, so essentially, do a heap census. Sounds good to me. (I think I'd like it to work for the dynamic linker to but we want to eject that code, don't we?) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by simonmar): @igloo: yes, I have `DYNAMIC_GHC_PROGRAMS=NO` in my `mk/build.mk`. I don't know how to do this with the system linker yet, because there doesn't seem to be an easy way to get hold of the start and end address of the image in memory - there's a fairly complex API in `rtld-audit`, but even that only seems to give you the start address. Anyway, for the application I need this for, we're using the RTS linker and not shared libraries. So please don't make the RTS linker go away :-) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Comment(by ezyang): OK, so maybe the comments here are out of date? http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#8039: RTS linker: unloadObj() does not actually unload the code ---------------------------------+------------------------------------------ Reporter: simonmar | Owner: simonmar Type: task | Status: new Priority: high | Milestone: 7.8.1 Component: Runtime System | Version: 7.6.3 Keywords: | Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown | Testcase: Blockedby: | Blocking: Related: | ---------------------------------+------------------------------------------ Changes (by heisenbug): * cc: ggreif@… (added) Comment: Great! This is a dearly needed feature for http://hackage.haskell.org/package/dynamic-loader too... -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/8039#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC