[GHC] #10885: Crashes on FFI calls to Visual Studio-built C library

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Windows Architecture: x86_64 | Type of failure: Runtime crash (amd64) | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- I want to call third-party C library using ffi on Windows. Everything works fine with 32-bit GHC distribution, however 64-bit program crashes as soon the ffi call is being made. Interestingly, the same program works when run with {{{runghc}}}. Just the compiled .exe crashes. Consider the following C++ library code: {{{#!c++ // adder.cpp extern "C" { __declspec(dllexport) int _cdecl add(int a, int b) { return a + b; } } }}} and the following Haskell code using the library: {{{#!hs import Foreign.C foreign import ccall unsafe "add" add :: CInt -> CInt -> IO CInt main = (add 5 2) >>= print }}} It does work on 32-bit architecture and in ghci (on all architectures). Why does it not work as 64-bit executable? Notes: * it works if the library is built with MinGW. However, this is not an acceptable workaround for me, since the library I want to call is external and closed-source * the problem seems to be related to the import library file handling — if I replace the MSVC-generated one with the MinGW-generated .lib import library, it works (even with MSVC-generated .dll) = Repro = I attach archive {{{repro_src.7z}}} with source files (as presented here) and a {{{build.bat}}} script building binaries. It needs to have both GCC/GHC and MSVC compilers in the path. (so it should be called from MSVC's {{{x64 Native Tools Command Prompt}}}) It creates two folder — in one it builds the library with MinGW, in the another one — with MSVC. The Haskell program crashes only with MSVC build of library. However, the Haskell source file works fine with runghc/ghci. Run the {{{build.bat}}} and ender the bin-msvc directory. The {{{Caller.exe}}} program crashes. However, {{{runghc -ladder ..\src\Caller.hs}}} gives the expected result. If you don't have MSVC installed, I provide also a package with all the sources built — {{{repro_bin.7z}}}. I have used Microsoft Visual Studio 2013 and MinGHC 7.10.1, both 64-bit targeting. Same happens with GHC 7.8.4. Please let me know, if you need any further information to investigate this report. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by mwu): * Attachment "repro_src.7z" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by mwu): * Attachment "repro_bin.7z" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Description changed by mwu: Old description:
I want to call third-party C library using ffi on Windows. Everything works fine with 32-bit GHC distribution, however 64-bit program crashes as soon the ffi call is being made. Interestingly, the same program works when run with {{{runghc}}}. Just the compiled .exe crashes.
Consider the following C++ library code: {{{#!c++ // adder.cpp
extern "C" { __declspec(dllexport) int _cdecl add(int a, int b) { return a + b; } } }}}
and the following Haskell code using the library: {{{#!hs import Foreign.C
foreign import ccall unsafe "add" add :: CInt -> CInt -> IO CInt
main = (add 5 2) >>= print }}}
It does work on 32-bit architecture and in ghci (on all architectures). Why does it not work as 64-bit executable?
Notes: * it works if the library is built with MinGW. However, this is not an acceptable workaround for me, since the library I want to call is external and closed-source * the problem seems to be related to the import library file handling — if I replace the MSVC-generated one with the MinGW-generated .lib import library, it works (even with MSVC-generated .dll)
= Repro = I attach archive {{{repro_src.7z}}} with source files (as presented here) and a {{{build.bat}}} script building binaries. It needs to have both GCC/GHC and MSVC compilers in the path. (so it should be called from MSVC's {{{x64 Native Tools Command Prompt}}})
It creates two folder — in one it builds the library with MinGW, in the another one — with MSVC. The Haskell program crashes only with MSVC build of library. However, the Haskell source file works fine with runghc/ghci. Run the {{{build.bat}}} and ender the bin-msvc directory. The {{{Caller.exe}}} program crashes. However, {{{runghc -ladder ..\src\Caller.hs}}} gives the expected result.
If you don't have MSVC installed, I provide also a package with all the sources built — {{{repro_bin.7z}}}. I have used Microsoft Visual Studio 2013 and MinGHC 7.10.1, both 64-bit targeting. Same happens with GHC 7.8.4.
Please let me know, if you need any further information to investigate this report.
New description: I want to call third-party C library using ffi on Windows. Everything works fine with 32-bit GHC distribution, however 64-bit program crashes as soon the ffi call is being made. Interestingly, the same program works when run with {{{runghc}}}. Just the compiled .exe crashes. Consider the following C++ library code: {{{#!c++ // adder.cpp extern "C" { __declspec(dllexport) int _cdecl add(int a, int b) { return a + b; } } }}} and the following Haskell code using the library: {{{#!hs import Foreign.C foreign import ccall unsafe "add" add :: CInt -> CInt -> IO CInt main = (add 5 2) >>= print }}} It does work on 32-bit architecture and in ghci (on all architectures). Why does it not work as 64-bit executable? Notes: * it works if the library is built with MinGW. However, this is not an acceptable workaround for me, since the library I want to call is external and closed-source * the problem seems to be related to the import library file handling — if I replace the MSVC-generated one with the MinGW-generated .lib import library, it works (even with MSVC-generated .dll) = Repro = I attach archive {{{repro_src.7z}}} with source files (as presented here) and a {{{build.bat}}} script building binaries. It needs to have both GCC/GHC and MSVC compilers in the path. (so it should be called from MSVC's {{{x64 Native Tools Command Prompt}}}) It creates two folder — in one it builds the library with MinGW, in the another one — with MSVC. The Haskell program crashes only with MSVC build of library. However, the Haskell source file works fine with runghc/ghci. Run the {{{build.bat}}} and ender the bin-msvc directory. The {{{Caller.exe}}} program crashes. However, {{{runghc -ladder ..\src\Caller.hs}}} gives the expected result. If you don't have MSVC installed, I provide also a package with all the C++ sources built — {{{repro_bin.7z}}}. I have used Microsoft Visual Studio 2013 and MinGHC 7.10.1, both 64-bit targeting. Same happens with GHC 7.8.4. Please let me know, if you need any further information to investigate this report. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by tmcdonell): * cc: tmcdonell (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): I hardly know anything about Windows and its calling conventions (for example I have no idea how `OSMinGW32` is a possible platform when compiling 64-bit code) but a while ago I noticed at https://en.wikipedia.org/wiki/X86_calling_conventions#Microsoft_x64_calling_... something about a shadow space which as far as I can see we don't take into account in `genCCall64'`. Could this be the cause? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by mwu): @rwbarton From what I see {{{OSMinGW32}}} is just an OS name (meaning simply "Windows") and has no relation to the 32/64-bit targeting (i.e. targeted architecture). The shadow space you mention is interesting. I know little about GHC inner workings and calling conventions, so take my speculations with a grain of salt. Still, from what I understand, both MinGW and MSVC on x64 Windows use the same calling convention. If it was the problem, it should fail with both. Moreover, Haskell code is able to correctly call x64 MSVC-built dll. The problem seems to be related only to the MSVC-generated import library (the {{{.lib}}} file). You might want to compare MinGW's and MSVC's {{{adder.lib}}} files in the {{{repro_bin.7z}}} package I uploaded. With one, calling {{{addd}}} method works, with another one it crashes. I'd suspect that difference should be somewhere there. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): Yuras pointed out that I missed this bit of code {{{ -- On Win64, we also have to leave stack space for the arguments -- that we are passing in registers lss_code <- if platformOS platform == OSMinGW32 then leaveStackSpace (length (allArgRegs platform)) else return nilOL }}} which seems to deal correctly with the shadow space. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by awson): This is a bug in binutils. It can't link MS-created 64-bit import libraries properly. The bug was fixed in latest MSys2 binutils distribution: http://repo.msys2.org/mingw/x86_64/mingw-w64-x86_64-binutils-2.25.1-1-any.pk.... Note that the bugfix was not in vanilla 2.25.1. MSys2 maintainers backported it from trunk binutils. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by mwu): Thanks for the answer, good to know that the bug is already fixed in the MSYS2. Would it be possible to ship the corrected version of binutils in next GHC Windows releases? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by awson): It's [https://ghc.haskell.org/trac/ghc/ticket/10726 done] already. At the moment you can safely replace them manually. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by mwu): That's great. I can confirm that replacing binutils binaries with the ones from the MSys2 distribution you linked does solve the issue. :-) Looking forward to see a build with updated MinGW distribution, the dated GCC 4.5/4.6 gave us some other headaches as well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10885: Crashes on FFI calls to Visual Studio-built C library -------------------------------------+------------------------------------- Reporter: mwu | Owner: Type: bug | Status: closed Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: duplicate | Keywords: Operating System: Windows | Architecture: x86_64 | (amd64) Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #10726 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thomie): * status: new => closed * resolution: => duplicate * related: => #10726 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10885#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC