Coping with weirdo third party packages
As part of the LLVM stuff, I've run into a few more stumbling blocks.
1. "ghc-options" only seems to be passed to GHC during compilation of a
library, not at link time when building an executable against an
installed copy of the library. As LLVM is written in C++, I need to
link using the C++ compiler instead of the C compiler. I'd like to get
"-pgml g++" into the linking phase somehow.
2. LLVM has some slightly unusual code building conventions: some code
is shipped as .o files, with the expectation that a program will do
something like this at link time:
g++ -o foo foo.o `llvm-config --ldflags --libs engine`
The llvm-config command prints stuff like this:
$ llvm-config --libs engine
/usr/lib/LLVMInterpreter.o /usr/lib/LLVMExecutionEngine.o -lLLVMTarget
-lLLVMCore -lLLVMSupport -lLLVMbzip2 -lLLVMSystem
Notice the absolute paths to .o files in there. I had hoped to be able
to use Cabal to avoid the need for a user to run llvm-config --ldflags
or --libs, but I can't list the .o files in the extra-libraries section,
because Cabal slaps a "-l" in front:
/usr/bin/ld: cannot find -l/usr/lib/LLVMExecutionEngine.o
This could be patched easily enough.
3. It would also be nice to have a leading "-l" stripped off if present,
the better to feed the output of llvm-config directly into a .buildinfo
file without stuffing it through sed first.
Cabaliers, would you like some patches for 2 and 3 above? What should
we do about 1?
On Thu, 2007-12-27 at 23:26 -0800, Bryan O'Sullivan wrote:
As part of the LLVM stuff, I've run into a few more stumbling blocks.
1. "ghc-options" only seems to be passed to GHC during compilation of a library, not at link time when building an executable against an installed copy of the library. As LLVM is written in C++, I need to link using the C++ compiler instead of the C compiler. I'd like to get "-pgml g++" into the linking phase somehow.
Usually you would use "ld-options:" but in this case that will not help you since your flag really is for ghc. I'm not sure what you want to do is currently possible, except perhaps by passing --ghc-option=-pgmlg++ at configure time, but that's not very convenient.
2. LLVM has some slightly unusual code building conventions: some code is shipped as .o files, with the expectation that a program will do something like this at link time:
g++ -o foo foo.o `llvm-config --ldflags --libs engine`
ld-options should help here, and you could use Setup.hs to call llvm-config.
The llvm-config command prints stuff like this:
$ llvm-config --libs engine /usr/lib/LLVMInterpreter.o /usr/lib/LLVMExecutionEngine.o -lLLVMTarget -lLLVMCore -lLLVMSupport -lLLVMbzip2 -lLLVMSystem
Notice the absolute paths to .o files in there. I had hoped to be able to use Cabal to avoid the need for a user to run llvm-config --ldflags or --libs, but I can't list the .o files in the extra-libraries section, because Cabal slaps a "-l" in front:
/usr/bin/ld: cannot find -l/usr/lib/LLVMExecutionEngine.o
This could be patched easily enough.
In that case try just hard-coding the ld-options rather than using Setup.hs to call llvm-config.
3. It would also be nice to have a leading "-l" stripped off if present, the better to feed the output of llvm-config directly into a .buildinfo file without stuffing it through sed first.
extra-libraries really is for libraries, it's fairly easy to convert -l flags and similar into the right stuff. Cabal provides a function to do so (though only available in Cabal-1.3.x): ccLdOptionsBuildInfo :: [String] -> [String] -> BuildInfo ccLdOptionsBuildInfo ccflags ldflags = ... it looks for -I, -l, -L flags and puts them in the appropriate sections. Other flags end up in cc-options or ld-options.
Cabaliers, would you like some patches for 2 and 3 above? What should we do about 1?
See if using ld-options fixes 2 and 3. I'm not sure about 1 either. It's part of the more general problem of developing multi-language stuff using C++. What do other people do when using a mixture of compilers, each of which wants to be used for the final link? Duncan
Duncan Coutts wrote:
Usually you would use "ld-options:" but in this case that will not help you since your flag really is for ghc. I'm not sure what you want to do is currently possible, except perhaps by passing --ghc-option=-pgmlg++ at configure time, but that's not very convenient.
Yeah. Alas!
2. LLVM has some slightly unusual code building conventions: some code is shipped as .o files, with the expectation that a program will do something like this at link time:
g++ -o foo foo.o `llvm-config --ldflags --libs engine`
ld-options should help here, and you could use Setup.hs to call llvm-config.
It has a weird side effect. I'm preprocessing some files using hsc2hs,
and Cabal is using ld-options to link the little C program built as an
intermediate product by hsc2hs. Is this expected? I'd have thought
that program would be completely standalone, since pretty much all it
does is call printf and fputs.
This causes further problems. Even if I pass --ghc-option=-pgmlg++ to
"setup configure", it seems to only be used for compilation, not
linking. I thus have to add -lstdc++ as an ld-option.
The second problem is that preprocessing becomes very slow, since the
LLVM libraries are huge. It takes longer to preprocess two source files
than to build the entire library.
In order to make progress, I suppose I could either hard-code a
dependency on g++ (aka -lstdc++) or try to figure out the extra
libraries that the C++ runtime needs and add those as ld-options.
Neither way seems especially satisfactory :-(
On Fri, 2007-12-28 at 11:40 -0800, Bryan O'Sullivan wrote:
Duncan Coutts wrote:
Usually you would use "ld-options:" but in this case that will not help you since your flag really is for ghc. I'm not sure what you want to do is currently possible, except perhaps by passing --ghc-option=-pgmlg++ at configure time, but that's not very convenient.
Yeah. Alas!
Aye :-(
2. LLVM has some slightly unusual code building conventions: some code is shipped as .o files, with the expectation that a program will do something like this at link time:
g++ -o foo foo.o `llvm-config --ldflags --libs engine`
ld-options should help here, and you could use Setup.hs to call llvm-config.
It has a weird side effect. I'm preprocessing some files using hsc2hs, and Cabal is using ld-options to link the little C program built as an intermediate product by hsc2hs. Is this expected?
Yes.
I'd have thought that program would be completely standalone, since pretty much all it does is call printf and fputs.
It's essential. hsc2hs compiles a C program that references things in the header files you're importing, linking that program to run it requires linking to the appropriate C libs. Though admittedly it's not clear to my why it should reference the C functions and thus have to link to the libs, but whenever I've tried without (eg in the gtk2hs build system) I get linker errors.
This causes further problems. Even if I pass --ghc-option=-pgmlg++ to "setup configure", it seems to only be used for compilation, not linking. I thus have to add -lstdc++ as an ld-option.
It should be used when linking an executable since we call ghc to do that. So I'm a bit confused about what's going on there. Any more details? However that doesn't help much if you're building a library that needs to be linked using the C++ compiler. That's just not going to work at all since you cannot propagate ghc options in packages to be used when compiling/linking programs that use the package.
The second problem is that preprocessing becomes very slow, since the LLVM libraries are huge. It takes longer to preprocess two source files than to build the entire library.
Why is that? Because linking the hsc2hs temporary executables with the LLVM libs takes a long time?
In order to make progress, I suppose I could either hard-code a dependency on g++ (aka -lstdc++) or try to figure out the extra libraries that the C++ runtime needs and add those as ld-options. Neither way seems especially satisfactory :-(
I think the second is much more likely to work in the long run. In my little experiment with gcc/g++ it seems that indeed all you need is -lstdc++ which translates to "extra-libraries: stdc++" in the .cabal file. g++ -c foo.c++ -o foo.o gcc foo.o -o foo -lstdc++ Duncan
Duncan Coutts wrote:
I thus have to add -lstdc++ as an ld-option.
It should be used when linking an executable since we call ghc to do that. So I'm a bit confused about what's going on there. Any more details?
Not really. It's definitely not in the command line at link time. Beyond inspecting the code to see why it's not happening, I don't know what else might be useful to you.
That's just not going to work at all since you cannot propagate ghc options in packages to be used when compiling/linking programs that use the package.
That's a shame - I'd hoped it was exactly what would happen :-)
It takes longer to preprocess two source files than to build the entire library.
Why is that? Because linking the hsc2hs temporary executables with the LLVM libs takes a long time?
Right.
I suppose I could either hard-code a dependency on g++ (aka -lstdc++) or try to figure out the extra libraries that the C++ runtime needs and add those as ld-options.
I think the second is much more likely to work in the long run.
Yeah, it's just brittle. gcc is ubiquitous on Unixy platforms, but LLVM
runs on Windows too, and my Windows compiler fu is weak.
I think that a README will have to suffice for now.
By the way, is there a reason why a .cabal file can specify multiple
executables, but only one library? LLVM wants to be built in a modular
fashion to minimise the number of dependencies, and it would be nice to
be able to put llvm-common, llvm-core, and llvm-engine all in a single
.cabal file, each with different ld-options.
On Sat, 2007-12-29 at 19:16 -0800, Bryan O'Sullivan wrote:
Duncan Coutts wrote:
I thus have to add -lstdc++ as an ld-option.
It should be used when linking an executable since we call ghc to do that. So I'm a bit confused about what's going on there. Any more details?
Not really. It's definitely not in the command line at link time.
This is linking an executable right?
Beyond inspecting the code to see why it's not happening, I don't know what else might be useful to you.
That's just not going to work at all since you cannot propagate ghc options in packages to be used when compiling/linking programs that use the package.
That's a shame - I'd hoped it was exactly what would happen :-)
GHC used to support doing that, putting ghc-options in package files but the feature was removed because it was abused and led to much confusion (eg adding extra pre-processors to all dependent packages).
It takes longer to preprocess two source files than to build the entire library.
Why is that? Because linking the hsc2hs temporary executables with the LLVM libs takes a long time?
Right.
I suppose I could either hard-code a dependency on g++ (aka -lstdc++) or try to figure out the extra libraries that the C++ runtime needs and add those as ld-options.
I think the second is much more likely to work in the long run.
Yeah, it's just brittle. gcc is ubiquitous on Unixy platforms, but LLVM runs on Windows too, and my Windows compiler fu is weak.
It'll use g++/gcc on windows too I presume, so -lstdc++ ought to work there too. Have you tried it?
I think that a README will have to suffice for now.
By the way, is there a reason why a .cabal file can specify multiple executables, but only one library? LLVM wants to be built in a modular fashion to minimise the number of dependencies, and it would be nice to be able to put llvm-common, llvm-core, and llvm-engine all in a single .cabal file, each with different ld-options.
It's because we don't yet support building collections of related packages very well. There have been suggestions that when we do support that better that there should only be one library or executable per .cabal file. Duncan
participants (2)
-
Bryan O'Sullivan -
Duncan Coutts