Why doesn't text-icu work on 64 bit Windows?
The text-icu library works fine with a 32 bit build of GHC on Windows. But with a 64 bit build of GHC, any use of a function from this library causes a segfault. See the issue on github for simple instructions for how to reproduce the problem: https://github.com/bos/text-icu/issues/12 As far as we know now, this could be caused by any of the following: 1. Some difference in the way ICU builds their DLLs for 64 bit Windows. 2. Some difference in the FFI bindings that is needed specifically for 64 bit Windows. 3. A problem with GHC's linking on 64 bit Windows. 4. A problem with Cabal, e.g., some option or flag that needs to be passed by default on 64 bit Windows. This is an important (and extremely high quality, thanks to Bryan O'Sullivan) library for the community, and very important for us, so I'm posting about the issue here to raise visibility. Any ideas about possible approaches to investigate or eliminate any of the above possibilities would be highly appreciated. Thanks, Yitz
Didn't look into details, so a couple of general advises: 1. On 64-bit platforms gcc and GHC silently link against Visual C-created import libraries, but resulting exe segfaults. Import libraries should be recreated with native 64-bit dlltool. 2. Perhaps the most important difference between 64-bit Windows and Linux ABIs is that "long" is 4-byte on Windows. Cheers, Kyra On 09-Dec-14 13:52, Yitzchak Gale wrote:
The text-icu library works fine with a 32 bit build of GHC on Windows. But with a 64 bit build of GHC, any use of a function from this library causes a segfault. See the issue on github for simple instructions for how to reproduce the problem:
https://github.com/bos/text-icu/issues/12
As far as we know now, this could be caused by any of the following:
1. Some difference in the way ICU builds their DLLs for 64 bit Windows.
2. Some difference in the FFI bindings that is needed specifically for 64 bit Windows.
3. A problem with GHC's linking on 64 bit Windows.
4. A problem with Cabal, e.g., some option or flag that needs to be passed by default on 64 bit Windows.
This is an important (and extremely high quality, thanks to Bryan O'Sullivan) library for the community, and very important for us, so I'm posting about the issue here to raise visibility. Any ideas about possible approaches to investigate or eliminate any of the above possibilities would be highly appreciated.
Thanks, Yitz _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
I wrote:
1. Some difference in the way ICU builds their DLLs for 64 bit Windows.
kyra wrote:
1. On 64-bit platforms gcc and GHC silently link against Visual C-created import libraries, but resulting exe segfaults. Import libraries should be recreated with native 64-bit dlltool.
Hmm, that would be a pretty complex task, but maybe I'll try it. The question is - if this is the problem, then why do the DLLs generated by Visual Studio work for 32 bits but not 64 bits? I found instructions how to do it here: http://www.willus.com/mingw/colinp/win32/dll/make.html Summary: Create a .def file with this format: EXPORTS Foo Bar where Foo and Bar are the symbols to be exported by the DLL. Then run these commands: gcc -mdll -o junk.tmp -Wl,--base-file,base.tmp bar.o del junk.tmp dlltool --dllname bar.dll --base-file base.tmp --output-exp temp.exp --def bar.def del base.tmp gcc -mdll -o bar.dll bar.o -Wl,temp.exp del temp.exp The first few steps use the .def file to create an .exp file. ICU provides the Visual Studio 2010 .exp files, so I tried just using those. It didn't work. So in order to figure out what symbols to export for each DLL, It looks like I'll need to open the ICU4C projects in Visual Studio 2010. Is this really what you need to go through to compile a basic library from Hackage on 64 bit Windows?
2. Some difference in the FFI bindings that is needed specifically for 64 bit Windows.
2. Perhaps the most important difference between 64-bit Windows and Linux ABIs is that "long" is 4-byte on Windows.
The long C type doesn't seem to be mentioned (at least directly) anywhere in the FFI bindings. Thanks, Yitz
On 09-Dec-14 20:18, Yitzchak Gale wrote:
Hmm, that would be a pretty complex task, but maybe I'll try it. The question is - if this is the problem, then why do the DLLs generated by Visual Studio work for 32 bits but not 64 bits? I don't know why but this is so.
I found instructions how to do it here:
http://www.willus.com/mingw/colinp/win32/dll/make.html
Summary:
Create a .def file with this format:
EXPORTS Foo Bar
where Foo and Bar are the symbols to be exported by the DLL.
Then run these commands:
gcc -mdll -o junk.tmp -Wl,--base-file,base.tmp bar.o del junk.tmp dlltool --dllname bar.dll --base-file base.tmp --output-exp temp.exp --def bar.def del base.tmp gcc -mdll -o bar.dll bar.o -Wl,temp.exp del temp.exp
The first few steps use the .def file to create an .exp file. ICU provides the Visual Studio 2010 .exp files, so I tried just using those. It didn't work. So in order to figure out what symbols to export for each DLL, It looks like I'll need to open the ICU4C projects in Visual Studio 2010.
Is this really what you need to go through to compile a basic library from Hackage on 64 bit Windows?
No, no. If you have existing .def file the single thing to do is 'dlltool -l <import library name> -D <dll file name> -d <.def filename>'. The problem is how to obtain .def file from existing dll. There exists 'pexports' utility generating .def file from given dll, but I'm not sure it exists for 64-bit case. There are 2 possibilities though: 1. Use Microsoft 'dumpbin' utility, which gives a list of exported symbols when run with '/EXPORTS' flag. The utility should be available from VS community edition at least. 2. Try to link to DLLs *directly* -- using binutils' facility to link against DLL directly instead of it's import lib. Cheers, Kyra
I wrote:
I found instructions how to do it here: http://www.willus.com/mingw/colinp/win32/dll/make.html
Kyra wrote:
No, no. If you have existing .def file the single thing to do is 'dlltool -l <import library name> -D <dll file name> -d <.def filename>'.
I tried that - there was no output. Note that in dlltool --help it says that -D is for *input* DLL file.
The problem is how to obtain .def file from existing dll… 1. Use Microsoft 'dumpbin' utility, which gives a list of exported symbols when run with '/EXPORTS' flag. The utility should be available from VS community edition at least.
I have full VS 10, so I have dumpbin. But the exports list it creates has many C++ mangled symbols, like: ??0?$EnumSet@W4UDateFormatBooleanAttribute@@$0A@$03@icu_54@@QEAA@AEBV01@@Z When I try to create a .exp using dlltool with a .def containing junk like that, it fails: dqsfc.s: Assembler messages: dqsfc.s:4102: Error: bad expression dqsfc.s:4102: Fatal error: rva without symbol dlltool: as exited with status 1
2. Try to link to DLLs *directly* -- using binutils' facility to link against DLL directly instead of it's import lib.
That would be much better. If we can get that to work, we can teach cabal how to do it too. Then FFI will work again on Windows 64 bits. The problem is that binutils does not understand DLLs from VS on 64 bits, it seems. Thanks, Yitz
On 10-Dec-14 15:56, Yitzchak Gale wrote:
No, no. If you have existing .def file the single thing to do is 'dlltool -l <import library name> -D <dll file name> -d <.def filename>'. I tried that - there was no output. Note that in dlltool --help it says that -D is for *input* DLL file. I did this multiple times with success :) I think there is something wrong with your .def file, perhaps.
In this use case dlltool needs *no* DLL itself -- only its name. For some reasons dlltool ignores a clause of 'LIBRARY' statement in .def file and uses whatever name you supply to -D option. For example, to generate import library for icuin52.dll you can use the following command-line: dlltool -l libicudt.a -D icuin52.dll -d icuin52.def where icuin52.def is the single input *file*, others arguments are parameters only, you need no have physically present icuin52.dll at all.
The problem is how to obtain .def file from existing dll… 1. Use Microsoft 'dumpbin' utility, which gives a list of exported symbols when run with '/EXPORTS' flag. The utility should be available from VS community edition at least. I have full VS 10, so I have dumpbin. But the exports list it creates has many C++ mangled symbols, like:
??0?$EnumSet@W4UDateFormatBooleanAttribute@@$0A@$03@icu_54@@QEAA@AEBV01@@Z
When I try to create a .exp using dlltool with a .def containing junk like that, it fails:
dqsfc.s: Assembler messages: dqsfc.s:4102: Error: bad expression dqsfc.s:4102: Fatal error: rva without symbol dlltool: as exited with status 1 These are mangled Visual C++ names, you can't use them at all and you don't need them at all, you should simply remove all of them from your .def file.
2. Try to link to DLLs *directly* -- using binutils' facility to link against DLL directly instead of it's import lib. That would be much better. If we can get that to work, we can teach cabal how to do it too. Then FFI will work again on Windows 64 bits. The problem is that binutils does not understand DLLs from VS on 64 bits, it seems. I successfully used several Windows 64 Visual C++- generated DLLs through FFI. Usually there are no problems with them if your generate import libraries as described above.
Cheers, Kyra
Kyra wrote:
No, no. If you have existing .def file the single thing to do is 'dlltool -l <import library name> -D <dll file name> -d <.def filename>'.
I wrote:
I tried that - there was no output. Note that in dlltool --help it says that -D is for *input* DLL file.
I did this multiple times with success :) I think there is something wrong with your .def file, perhaps.
Yes that makes sense. My .def file has only this: EXPORTS symbol1 symbol2 … What does a correct .def file look like?
These are mangled Visual C++ names, you can't use them at all and you don't need them at all, you should simply remove all of them from your .def file.
OK I did that. Now I was able to use dlltool to create new .lib files from the original DLLs. And I was able to compile my test program against the new .lib files. But it still segfaults when I try to run it with the original DLLs.
I successfully used several Windows 64 Visual C++- generated DLLs through FFI. Usually there are no problems with them if your generate import libraries as described above.
That is great! Can you get a successful build of my simple test code: https://github.com/ygale/test-text-icu and give a list of exactly what commands you entered to get it to build? (Check that the compiled executable runs without segfault.) That would be very helpful. Thanks, Yitz
participants (2)
-
kyra -
Yitzchak Gale