[GHC] #13624: loadObj() does not respect alignment

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime | Version: 8.0.1 System (Linker) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- This is perhaps known, but I'll write it down here in case somebody else runs into this problem as well. Since `loadObj()` just `mmap()`s the entire object file and decodes it ''in place'', it does not respect the alignment requirements specified in the section headers. This is problematic for instructions which require alignment, e.g. SSE, AVX. The attached `map.ll` program is `map (+1)` over an array of floating point numbers. In particular, the core loop is 8-way SIMD vectorised x 4-way unrolled, for 32-elements per loop iteration. A tail loop handles any remainder one-at-a-time. You can compile it using `llc -filetype=obj -mcpu=native map.ll`. For a CPU with AVX instructions (sandy bridge or later) you should get the following: {{{ $ objdump -d map.o Disassembly of section .text: 0000000000000000 <map>: 0: 49 89 f3 mov %rsi,%r11 3: 49 29 fb sub %rdi,%r11 6: 0f 8e f9 00 00 00 jle 105

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by tmcdonell): * Attachment "test.c" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by tmcdonell): * Attachment "map.ll" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by tmcdonell): * Attachment "build.sh" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: linker Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * priority: normal => high * cc: angerman (added) * keywords: => linker * milestone: => 8.4.1 Comment: Wow, I didn't know that. That is terrible! My understanding is that the linker (at least in the case of ELF) actually maps sections incorrectly anyways: We ought to be using the object file's segments, not sections, for the purposes of mapping. This will likely reduce the number of mappings we need to setup, and allow us to set the correct RWX access control bits (e.g. currently we map code as writable, which is rather terrifying from a security perspective). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: linker Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by angerman): For what it's worth the macho/aarch64 does this already, and elf/armv7, elf/aarch64 linker will do it. They essentially requests memory via mmap, and memcpy the data per section; this makes sections being page aligned, which I believe satisfies most alignment requirements. An additional check wouldn't hurt though. However we can't use segments, as segments can contain multiple sections that are not bound to the same type. A segment can contain data and text section; which in turn need to live in different pages for them to be properly mprotected. This then means we can't mmap segments. However what we could try to reduce memory footprint is to aggregate text and data sections across object files. Of course mmap + memcpy is slower than directly mmapping. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: linker Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): For the record, each segment has an associated set of permission flags. IIRC, two sections of mapping characteristics won't end up in the same segment. However, we nevertheless can't use segments for an unrelated reason: only executables and shared objects are organized into segments. Our runtime linker is only concerned with object files, which have no useful segment organizations AFAIK. Oh well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: linker Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Fixing this will unblock #8949. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: linker Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I've opened #14069 to fix the protection bits issue. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.4.1 Component: Runtime System | Version: 8.0.1 (Linker) | Resolution: | Keywords: linker Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by George): * cc: George (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * keywords: linker => linker, newcomer * milestone: 8.4.1 => 8.6.1 Comment: This is something that we should try to fix (along with #14069) in 8.6. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by George): * failure: None/Unknown => Runtime crash -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by carter): What are the platforms that need this fix? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by carter): And what are the challenges to doing this correctly? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by George): Trying to find out if MacOS on 8.6.1 is one of the platforms where this needs to be fixed but I can't reproduce: {{{ ./build.sh + llc -filetype=obj -mcpu=native map.ll + ghc --make -no-hs-main test.c bash-3.2$ ./a.out array size is 31 calling function... ok 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0 12.0 13.0 14.0 15.0 16.0 17.0 18.0 19.0 20.0 21.0 22.0 23.0 24.0 25.0 26.0 27.0 28.0 29.0 30.0 31.0 bash-3.2$ lldb a.out (lldb) target create "a.out" Traceback (most recent call last): File "<input>", line 1, in <module> File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy.py", line 52, in <module> import weakref File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/weakref.py", line 14, in <module> from _weakref import ( ImportError: cannot import name _remove_dead_weakref Current executable set to 'a.out' (x86_64). (lldb) quit quit }}} I don't understand why lldb is needed to reproduce this in any case, in particular how it changes the arr could you explain. Above it says "The #define N on line 7 will change the size of the array...For 32 elements or larger (i.e. entering the core loop) the program will (almost certainly) segfault. " I trying changing N to 32 but then running build.sh a.out just worked, no segfault. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by tmcdonell): Replying to [comment:13 George]:
Trying to find out if MacOS on 8.6.1 is one of the platforms where this needs to be fixed but I can't reproduce:
The problem still exists. I verified this just now; macOS 10.13.6 {{{ $ ghc --version The Glorious Glasgow Haskell Compilation System, version 8.6.1 $ ghc --print-project-git-commit-id 0d2cdec78471728a0f2c487581d36acda68bb941 }}}
I don't understand why lldb is needed to reproduce this in any case.
In particular how it changes the array size to 32, could you explain? In
Obviously, `lldb` is not required. Just run the executable and watch it segfault. The point of using a debugger (any would suffice) is to demonstrate which instruction is causing the problem, which for this issue is important to understand. the description it says "The #define N on line 7 will change the size of the array...For 32 elements or larger (i.e. entering the core loop) the program will (almost certainly) segfault. " I trying changing N to 32 but then running build.sh a.out just worked, no segfault. If you read the description, you will see that the core loop is a 8-way vectorised and 4-way unrolled; i.e. 32-elements per trip. For any remainder it backs out to a single element per loop. Thus, to activate the code path which exhibits the problem, you require at least 32 elements. The description also says "For a CPU with AVX instructions (sandy bridge or later) you should get the following". Since you are running an original corei7 (nephalem) and don't have AVX instructions, this exact example obviously won't trigger for you. If you checked the objdump output as suggested, you would also have seen this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

Replying to [comment:13 George]:
Trying to find out if MacOS on 8.6.1 is one of the platforms where
#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by George): Sorry, when I first read the bug I realized that I wouldn't be able to reproduce it on my machine but was interested in seeing it fixed as I plan on getting a new machine in the near future. When I saw comment 11 I forgot all that and tried to reproduce. Sorry for wasting your time but thanks for taking the time to answer. Replying to [comment:14 tmcdonell]: this needs to be fixed but I can't reproduce:
The problem still exists. I verified this just now; macOS 10.13.6
{{{ $ ghc --version The Glorious Glasgow Haskell Compilation System, version 8.6.1 $ ghc --print-project-git-commit-id 0d2cdec78471728a0f2c487581d36acda68bb941 }}}
I don't understand why lldb is needed to reproduce this in any case.
Obviously, `lldb` is not required. Just run the executable and watch it
segfault. The point of using a debugger (any would suffice) is to demonstrate which instruction is causing the problem, which for this issue is important to understand.
In particular how it changes the array size to 32, could you explain?
In the description it says "The #define N on line 7 will change the size of the array...For 32 elements or larger (i.e. entering the core loop) the program will (almost certainly) segfault. " I trying changing N to 32 but then running build.sh a.out just worked, no segfault.
If you read the description, you will see that the core loop is a 8-way
vectorised and 4-way unrolled; i.e. 32-elements per trip. For any remainder it backs out to a single element per loop. Thus, to activate the code path which exhibits the problem, you require at least 32 elements.
The description also says "For a CPU with AVX instructions (sandy bridge
or later) you should get the following". Since you are running an original corei7 (nephalem) and don't have AVX instructions, this exact example obviously won't trigger for you. If you checked the objdump output as suggested, you would also have seen this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.10.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by artempyanykh): FWIW, `map.ll` needs to be compiled with `-mcpu=sandybridge`. Otherwise on Haswell the compiler emits `vbroadcastss` instead of `vmovaps` and the issue doesn't manifest itself. ----- I'm currently figuring out Mach-O part. I've adjusted section placement on x86_64 analogous to aarch64, but it's not enough as current x86_64 relocation code works directly with macho_sections. I need to take a closer look at it. I have concerns re: allocating a page per section. It seems feasible do some prep work before `ocGetNames` to calculate the size of future r-x segment and rw- segment and exact placement of sections inside the mmaped areas (although, currently not sure about subsequent changes to mprotecting and unloading code). Oh, and If somebody's already working on this, please let me know :) ---- One question, if I may. Moritz, for iOS you use regular `mmap` instead of `mmapForLinker`. I don't know much about ARM and wonder if allocating in low 2GB is not necessary there? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.10.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by angerman): @artempyanykh, sorry for the late reply, I've been way too busy the last day. On iOS we don't have largeAddressSpace, and I doubt we have any issues with address allocation. I don't remember exactly why I didn't use `mmapForLinker`. But I *do* remember trying to use it and running into some issues where making using `mmapForLinker` work for iOS as well would have unnecessarily complicated and such just went for `mmap`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.10.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by artempyanykh): No worries! Thanks Moritz, makes sense. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.10.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by artempyanykh): I've adjusted section mapping and relocations for x64 Mach-O - the attached program runs fine and ghci, rts, TH tests are green. I've found some other issues along the way, but no show stoppers. I'll assign this ticket to myself and hope to prepare a MR soon. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13624: loadObj() does not respect alignment -------------------------------------+------------------------------------- Reporter: tmcdonell | Owner: artempyanykh Type: bug | Status: new Priority: high | Milestone: 8.10.1 Component: Runtime System | Version: 8.0.1 (Linker) | Keywords: linker, Resolution: | newcomer Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: 8949 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by artempyanykh): * owner: (none) => artempyanykh -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13624#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC