[GHC] #9314: Huge space leak of GHC API 7.8.x
#9314: Huge space leak of GHC API 7.8.x ------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 7.8.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: Other Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- GHC API 7.8.x uses much more memory than GHC API 7.6.x. Attached two files demonstrate this: - A.hs -- Simple program using GHC API (copied from Wiki) - B.hs -- A target file, just hello world You can compile A.hs as follows: {{{ % ghc A.hs -package ghc -package ghc-paths }}} A.hs stays in 10 seconds. So, we can investigate its memory usage with the "top" command.The following is the result: {{{ Mac (64bit) Linux (64bit) GHC 7.6.3: 20MB 4MB GHC 7.8.3: 106MB 13MB }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by kazu-yamamoto): From Karel Gardas: On Solaris 11 i386 (32bit binary) {{{ GHC 7.6.3 53MB (size), 44MB (RSS) GHC 7.8.2: 91MB (size), 81MB (RSS) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Changes (by snoyberg): * cc: michael@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by JohnWiegley): I'm seeing a problem too, with an application that builds fine using 7.6.3. When building with -O2 and 7.8.3, GHC exhausts system memory (16G) and ultimately is killed. With 7.8.3 and -O1, or with 7.6.3, it finishes. I can't paste my code here, but would be happy to try any suggestions to help isolate the problem. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Comment (by simonpj): -O2 adds `SpecConstr`, a notorious source of blow-up. Try switching it off with `-fno-spec-constr`. The `SpecConstr` blow-up needs love and attention, and I keep being too distracted. Help most welcome. I don't think it's fundamental. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.4 Component: GHC API | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: Other | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: -------------------------------------+------------------------------------ Changes (by simonmar): * priority: normal => high * milestone: => 7.8.4 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Operating System: Unknown/Multiple Differential Revisions: | Type of failure: Other Architecture: | Test Case: Unknown/Multiple | Blocking: Difficulty: Unknown | Blocked By: | Related Tickets: | -------------------------------------+------------------------------------- Comment (by kazu-yamamoto): In my case, this happens even with -O0. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Operating System: Unknown/Multiple Differential Revisions: | Type of failure: Other Architecture: | Test Case: Unknown/Multiple | Blocking: Difficulty: Unknown | Blocked By: | Related Tickets: | -------------------------------------+------------------------------------- Changes (by chak): * cc: chak@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Operating System: Unknown/Multiple Differential Revisions: | Type of failure: Other Architecture: | Test Case: Unknown/Multiple | Blocking: Difficulty: Unknown | Blocked By: | Related Tickets: | -------------------------------------+------------------------------------- Comment (by kazu-yamamoto): Are there any -fxxx added for GHC 7.8, which enlarges loaded modules? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Operating System: Unknown/Multiple Differential Revisions: | Type of failure: Other Architecture: | Test Case: Unknown/Multiple | Blocking: Difficulty: Unknown | Blocked By: | Related Tickets: | -------------------------------------+------------------------------------- Comment (by simonmar): GHC loads more interface files in 7.8.x, due to the AMP warnings. This does make it use more memory (but a constant amount per compilation). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by rwbarton): OK here's the situation. Kazu's program is statically linked against the GHC API (ghc builds statically-linked executables by default) which means GHC is loading the static library versions of ghc-prim, integer-gmp, base. This was the case in GHC 7.6 also, but that version shipped with .o files (e.g. HSghc-prim-0.3.0.0.o) which the GHC linker can read as a single unit. These are not included with GHC 7.8 on platforms that use dynamic libraries by default (I guess since GHCi would not use them), so GHC 7.8 has to read the .a files instead. As described in a comment in `loadArchive()`, for reasons having to do with alignment, each member object file of an archive is loaded into its own mmap()ed page(s). The base package consists of approximately a zillion tiny object files (split objects), each of which costs 4 kilobytes, so as the comment mentions, this is quite wasteful. Almost all of the memory mapped by the program under 7.8 is attributable to either these mappings or to the executable file itself. We could perhaps come up with a more efficient scheme for loading archive files, but a workaround is to just build the executable with `-dynamic`, so that it will load the GHC API as a shared library and avoid this excessive allocation. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by kazu-yamamoto): I confirmed that -dynamic reduces the memory usage. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Huge space leak of GHC API 7.8.x -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.8.4 Priority: high | Version: 7.8.3 Component: GHC API | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Reid, thank you. I have no opinion about dynamic linking (except that it is generally the work of the devil, and has caused us a totally unreasonable amount of pain), but we should all be very grateful to you for diagnosing what is going on. I would never have thought of that in a million (or zillion) years. Thanks! Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: 7.10.1 Priority: high | Version: 7.8.3 Component: Compiler | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by thomie): * component: GHC API => Compiler Comment: The comment in `rts/Linker.c` that rwbarton referred to in comment:10: {{{ /* We can't mmap from the archive directly, as object files need to be 8-byte aligned but files in .ar archives are 2-byte aligned. When possible we use mmap to get some anonymous memory, as on 64-bit platforms if we use malloc then we can be given memory above 2^32. In the mmap case we're probably wasting lots of space; we could do better. */ }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 7.12.1 Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by dfeuer): * milestone: 7.10.1 => 7.12.1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 7.12.1 Component: Compiler | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by hsyl20): I have made a patch for this one (https://phabricator.haskell.org/D985) With the example given in the ticket I get: {{{ Linux (64bit) 7.10.1 174 MB / 20209 calls to mmap HEAD with patch 95 KB / 332 calls to mmap }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: high | Milestone: 7.12.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: Phab:D985 -------------------------------------+------------------------------------- Changes (by hsyl20): * cc: simonmar (added) * status: new => patch * differential: => Phab:D985 * component: Compiler => Runtime System -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D985 Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): Phab:D985 has been merged into into Phab:D975, which has been merged into HEAD, but I'm not seeing the huge improvements that hsyl20 mentioned in comment:16. ||= GHC =||= RSS =|| || 7.11.20151111 (HEAD) || 123M || || 7.10.2 || 142M || || 7.8.4 || 131M || || 7.6.3 || 41M || I compile `ghc A.hs -package ghc -package ghc-paths`, then run `./A`, look at the `RES` column in `top`, and convert from Kb to Mb. What am I missing? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * status: patch => new * differential: Phab:D985 => -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by hsyl20): @simonmar wrote: "I like not having to memcpy all the memory for an object file into the 1-2GB region, we just mmap it - this is also useful because various tools (like perf) understand the mapping and can give symbol names. For .a files I think it's fine to memcpy the bits though." (https://phabricator.haskell.org/D985#26591) I think that's why D975 doesn't bring the memory improvement while D985 did. Maybe we should add a flag (or use -g) to switch between the mmap mode that tools like perf understand and the memcpy mode that is much cheaper (the latter becoming the default mode). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by hsyl20): I have created a diff to put the fix back: Phab:D1470 It shouldn't be an issue for perf-like tools because the mmap's I replaced with m32_alloc's were not mapping a file. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 8.0.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): bgamari: what is the status here? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 8.2.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.0.1 => 8.2.1 Comment:
bgamari: what is the status here?
hsyl20 and I were discussing it a few weeks ago; I fixed a few issues in Phab:D1470 but there is still a fair amount of debugging to do before it is mergeable. It certainly isn't 8.0 material at this point. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 7.8.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.2.1 => 8.4.1 Comment: Sadly hsyl20's patch stalled and has bitrotted to the point where it will need to be re-written. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:25 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: closed Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 7.8.3 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonmar): * status: new => closed * resolution: => fixed Comment: This was done - I merged the changes into https://phabricator.haskell.org/rGHC04e8366608fee4f5e3358acc855bc6f556c3f508 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9314: Each object file in a static archive file (.a) is loaded into its own mmap()ed page -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: closed Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 7.8.3 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Ahh right; I had forgotten about this. Thanks Simon. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9314#comment:27 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC