[GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

newer
[GHC] #13214: Orphan instances in...

GHC

3 Oct 2013 3 Oct '13

6:33 p.m.

#8405: experiment with using function-sections for linking (for smaller libs and executables) ------------------------------------+------------------------------------- Reporter: carter | Owner: Type: task | Status: new Priority: normal | Milestone: 7.10.1 Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- currently the only support we have for making small (ish) executables using split objects, which 1. is not on by default, for various good reasons 2. can seriously penalize compile times 3. in tandem, something like strip is often used, but on some platforms/targets (such as the iOS cross compiler), strip will actually break things might be worth seriously experimenting with function-sections / gc- sections in the object files we generate. example slide deck on this http://elinux.org/images/2/2d/ELC2010-gc- sections_Denys_Vlasenko.pdf -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

Show replies by date

GHC

4 Oct 4 Oct

3:35 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

3:37 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

6:05 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

18 Oct 18 Oct

12:52 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

6 Sep 6 Sep

5:42 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

#8405: experiment with using function-sections for linking (for smaller libs and executables) -------------------------------------+------------------------------------- Reporter: carter | Owner: Type: task | Status: new Priority: normal | Milestone: 7.12.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by olsner): Another benefit of this would be getting rid of the satanic splitter, build system support for split objects, and any other splitting-related hacks there might be laying around. FWIW, I've been hacking at a patch to do this in the native code generator (https://github.com/olsner/ghc/commits/function-sections). It passes almost all of the test suite now, and generated binaries for a simple "hello world" did get smaller. But that's compared to a build without split objs - for some reason function sections doesn't seem to help nearly as much as split-objs... So that's something I'll want to fix first. I have also only tested this on Linux so far. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

6:18 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

8 Sep 8 Sep

12:18 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

6:38 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

10:59 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

...

[...] that made my "hello world" 10% smaller with sections than with

split-objs (20% diff after stripping). Very cool, thanks for working on this! Link times with `--gc-sections` should be also much faster than with split-objs, especially for something like OpenGL. And it looks like `--gc-sections` [https://sourceware.org/git/gitweb.cgi?p =binutils-gdb.git;h=0f088b2a9417b1d4ed597849ffa671eba25f5051 is now finally supported on Windows] (though the patch still needs testing). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

9 Sep 9 Sep

7:37 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

#8405: experiment with using function-sections for linking (for smaller libs and executables) -------------------------------------+------------------------------------- Reporter: carter | Owner: olsner Type: task | Status: new Priority: normal | Milestone: 7.12.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by olsner): * owner: => olsner Comment: It seems GNU ld actually has scalability issues with very large numbers of sections too, unfortunately. Found out because I was building libHSghc with split sections, and getting 10 minute link times on some test cases with plugins linking statically against ghc... Since split-objs is specifically disabled for that code I simply disabled split-sections as well. A much cleaned up version of the patch now at https://github.com/olsner/ghc/commits/function-sections-2 - this adds a -split-sections flag that controls the splitting when compiling Haskell code. --gc-sections is always added to the linker command line. (Vague rationale being that after building libraries with split sections, we'll want to bring space savings to users without manual steps or extra flags building their programs/libraries.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

10 Sep 10 Sep

5:32 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

...

It seems GNU ld actually has scalability issues with very large numbers of sections too, unfortunately.

Have you tried the gold linker? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

13 Sep 13 Sep

7:15 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

1 Nov 1 Nov

4:46 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

#8405: experiment with using function-sections for linking (for smaller libs and executables) -------------------------------------+------------------------------------- Reporter: carter | Owner: olsner Type: task | Status: new Priority: normal | Milestone: 8.0.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D1242 -------------------------------------+------------------------------------- Comment (by Ben Gamari ): In [changeset:"3021cc06f84109c6e7a529318d690c520748f936/ghc" 3021cc06/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="3021cc06f84109c6e7a529318d690c520748f936" Add rts/Linker support for more than 64k sections Since some ELF fields ran out of range to represent that many sections, they've been extended with magic numbers that indicate that the full value is stored in another field. This will be necessary for GHCi with -split-sections on ELF platforms that don't use GNU ld. Reviewers: austin, bgamari, simonmar, erikd Reviewed By: bgamari, simonmar, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1357 GHC Trac Issues: #8405 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

12 Nov 12 Nov

9:54 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

#8405: experiment with using function-sections for linking (for smaller libs and executables) -------------------------------------+------------------------------------- Reporter: carter | Owner: olsner Type: task | Status: new Priority: normal | Milestone: 8.0.1 Component: Compiler | Version: 7.6.3 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D1242 -------------------------------------+------------------------------------- Comment (by Ben Gamari ): In [changeset:"4a32bf925b8aba7885d9c745769fe84a10979a53/ghc" 4a32bf9/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="4a32bf925b8aba7885d9c745769fe84a10979a53" Implement function-sections for Haskell code, #8405 This adds a flag -split-sections that does similar things to -split-objs, but using sections in single object files instead of relying on the Satanic Splitter and other abominations. This is very similar to the GCC flags -ffunction-sections and -fdata-sections. The --gc-sections linker flag, which allows unused sections to actually be removed, is added to all link commands (if the linker supports it) so that space savings from having base compiled with sections can be realized. Supported both in LLVM and the native code-gen, in theory for all architectures, but really tested on x86 only. In the GHC build, a new SplitSections variable enables -split-sections for relevant parts of the build. Test Plan: validate with both settings of SplitSections Reviewers: dterei, Phyx, austin, simonmar, thomie, bgamari Reviewed By: simonmar, thomie, bgamari Subscribers: hsyl20, erikd, kgardas, thomie Differential Revision: https://phabricator.haskell.org/D1242 GHC Trac Issues: #8405 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

11:11 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

10:16 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

14 Nov 14 Nov

12:49 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

17 Nov 17 Nov

6:04 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

...

Would this make it practical to disable stripping on libraries and executables (so that dwarf metadata could be available by default?)

I didn't know that was a problem, so if this feature fixes anything it's purely a bonus :) I suspect the debug info could get pretty large regardless of this patch, so many users might still not want it. (Though it might be improved compared to `-split-objs`?) Replying to [comment:10 refold]:

...

And it looks like `--gc-sections` [https://sourceware.org/git/gitweb.cgi?p=binutils- gdb.git;h=0f088b2a9417b1d4ed597849ffa671eba25f5051 is now finally supported on Windows] (though the patch still needs testing).

Since I had my system in Windows today, I decided to test this out - but ran into some problems. First problem is that linking executables with `SplitSections=YES` gets you a "too many sections" error. But when making an executable, those sections should be getting merged into just a handful of sections... I think this is `ld`'s fault, so I submitted https://sourceware.org/bugzilla/show_bug.cgi?id=19254 for it. It also seems the `--gc-sections` patch was committed after binutils 2.25 branched off, so we need to upgrade binutils to use this. (Probably to a yet-unreleased version that also has a fix for the sections issue...) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

21 Nov 21 Nov

7:43 a.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

#8405: experiment with using function-sections for linking (for smaller libs and executables) -------------------------------------+------------------------------------- Reporter: carter | Owner: olsner Type: task | Status: closed Priority: normal | Milestone: 8.0.1 Component: Compiler | Version: 7.6.3 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D1242 Wiki Page: | -------------------------------------+------------------------------------- Comment (by Tamar Christina ): In [changeset:"7f77e4e9b301493db0782cf2d129cf62dbcd5af6/ghc" 7f77e4e/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="7f77e4e9b301493db0782cf2d129cf62dbcd5af6" Fix Windows builds after D1242 Summary: Only use the merge_sections.ld linker script if SplitSections is enabled. I thought I had a way to make the linker script actually work on Windows and produce object files instead of image files (by using the "INSERT AFTER" linker script command to get the default script's output format setting), but that fix caused ghci to crash with an illegal instruction on startup. Gave up and made a simpler fix of just disabling this for normal builds, as it's only really relevant with SplitSections enabled anyway. Reviewers: austin, thomie, bgamari Reviewed By: austin, thomie Subscribers: Phyx, thomie Differential Revision: https://phabricator.haskell.org/D1505 GHC Trac Issues: #8405 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC

24 Dec 24 Dec

11:20 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

11 Feb 11 Feb

8:08 p.m.

New subject: [GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

#8405: experiment with using function-sections for linking (for smaller libs and executables) -------------------------------------+------------------------------------- Reporter: carter | Owner: olsner Type: task | Status: closed Priority: normal | Milestone: 8.0.1 Component: Compiler | Version: 7.6.3 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D1242 Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari ): In [changeset:"a50082c115bed1891b2e5aac4a21462935f4f0d6/ghc" a50082c/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="a50082c115bed1891b2e5aac4a21462935f4f0d6" Apply SplitSections to all C compilations Previously this was added only to the RTS's C files (those are the bulk of it though), but there are C bits in ghc-prim, integer-gmp and base too. Followup for #8405, allows the large table of character properties in base to be stripped when not used. Test Plan: validate Reviewers: austin, bgamari, simonmar Reviewed By: bgamari Subscribers: thomie, snowleopard Differential Revision: https://phabricator.haskell.org/D3121 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8405#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

3020

Age (days ago)

4248

Last active (days ago)

List overview

Download

21 comments

1 participants

[GHC] #8405: experiment with using function-sections for linking (for smaller libs and executables)

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

GHC

tags

participants (1)