On 8 December 2011 22:42, Dave Tapley
Hi all, I've been working on the build system for wxhaskell, and ran in to a problem using "runProgram".
As an experiment I've decided to put it on hpaste instead of having the mail thread get out of hand (I'm also cross posting it to the wxhaskell list and have mentioned it in #haskell, and I want to keep all correspondence in one place). Here is a link which jumps to the line where I first describe the problem I'm having:
Ha, well that was a good time for hpaste to go down! Here it is: I've been trying to resurrect the idea of building a shared library for the C++ component of the wxhaskell library. Jeremy (to my knowledge) first put this forward, here: http://wewantarock.wordpress.com/2010/11/01/working-around-the-static-libstd... In addition to the advantages he lists there, I have the following reason: When building wx-core (in its current incarnation) cabal will *always* rebuild all the C++ code, which takes a majority of the build time. This becomes very frustrating when you change one line of Haskell and have to wait seven minutes to rebuild. I complained about it here: http://sourceforge.net/mailarchive/message.php?msg_id=28099997 So, I decided to try and stop this complete C++ rebuild, I partially succeed, but I eventually got stuck for reasons I complained here: http://www.haskell.org/pipermail/cabal-devel/2011-October/007816.html Side note: I did eventually discover why (or, where) the c-sources (cSources) list is used to compile and link, and it is here: http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/src/Distri... You can see: "| filename <- cSources libBi]" under "-- build any C sources", and you can see: "cSharedObjs = map (`replaceExtension` ("dyn_" ++ objExtension)) (cSources libBi)" under "-- link:". Is this assumption correct? At this point I thought about either: 1. Getting the cabal source and starting to write code to give BuildInfo a "cObjs" in addition to "cSources". 2. Picking up Jeremy's shared library code, so wxhaskell would have its own code to build the library, in which I could do sensible re-compilation. Given that there were other advantages to 2, I went with that. Jeremy had written one blog post on building a such a shared library, here: http://wewantarock.wordpress.com/2010/11/03/building-a-shared-library-in-cab... In response to an email on the wxhaskell-devel list he also kindly put up this gist, with the code he'd been working on: https://gist.github.com/1301115 I dutifully took this gist, and have now attempted to integrate in to my wxhaskell-dev branch, which you can find here: http://darcsden.com/DukeDave/wxhaskell-dev (note: I haven't pushed any of the shared library code to darcsden yet, for reasons I'm about to explain) This is where things got interesting, after a few hours of hacking I have built a shared library, but in this very back-handed way: Firstly, in Jeremy's code the to-be-compiled shared library is added to the wxcore BuildInfo through a custom hook. We can see this because:
let all_dlls = parseDLLs ["x-dll-name", "x-dll-extra-libraries"] custom_bi (the modified wxcore.cabal contains the line: "x-dll-name: wxc")
However on a clean build of wxcore we get the following error: setup: Missing dependency on a foreign library: * Missing C library: wxc This problem can usually be solved by installing the system package that provides this library (you may need the "-dev" version). If the library is already installed but in a non-standard location then you can use the flags --extra-include-dirs= and --extra-lib-dirs= to specify where it is. The error is dumped *before* cabal calls myBuildHook, and since this actually builds the library the error makes sense; cabal is looking for the shared library before we've built it. To get around this I modified the offending line, thus:
let all_dlls = parseDLLs ["x-dll-extra-libraries"] custom_bi
With that modification cabal would now get to myBuildHook, but then another curious error arose: We see that the actual linking is called here:
runProgram verbose gcc (opts' ++ objs' ++ lib_dirs' ++ libs')
But on hitting that line the following error is spat out: /usr/bin/ld: cannot open output file dist/build/libwxc.so.0.13.1: No such file or directory I checked all my permissions and couldn't see anything wrong, I could touch the file. Conveniently I noticed that, if the verbosity is set high enough, runProgram will call printRawCommandAndArgs: http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/src/Distri... The output (i.e. the linker invocation) looks like this: /usr/bin/gcc -fno-stack-protector -shared -Wl,-soname,libwxc.so.0 -o dist/build/libwxc.so.0.13.1 dist/build/src/cpp/apppath.o dist/build/src/cpp/dragimage.o dist/build/src/cpp/eljaccelerator.o [snip-rest-of-.o-files] -L/usr/local/lib -lstdc++ -lwx_baseu-2.9 [snip-rest-of-wx-libs] Now if I cd into the wxcore and paste the command *verbatim* then gcc works and generates libwxc.so.0.13.1 as expected. You can see in Jeremy's code the linkShareLib function contains:
cwd <- getCurrentDirectory
I used this to confirm that the we were in ./wxcore and we are, even making the path for -o absolute didn't sovle the issue. I ended up replacing runProgram line with this, less satisfactory line:
system $ (unwords ([show . locationPath . programLocation $ gcc] ++ opts' ++ objs' ++ lib_dirs' ++ libs'))
Which works, but still doesn't explain why runProgram doesn't. Any suggestions? So with that complete I was able to link a shared library to: wxcore/dist/build/libwxc.so.0.13.1 Of course, at this point we've generated the shared library, but wxcore still isn't aware of it, so the build completes successfully, but any attempt to compile against it results (understandably) in hundreds of linker errors. To resolve this I had to add "x-dll-name" (read in from wxcore.cabal as "wxc") back to parseDLLs:
let all_dlls = parseDLLs ["x-dll-name", "x-dll-extra-libraries"] custom_bi
Well, not quite, attempt to build wxcore again and we're back to this error: * Missing C library: wxc Of course, the .so isn't in a 'normal' place, so add its location as an extra lib directory:
{ extraLibDirs = extraLibDirs libbi ++ extraLibDirs wx ++ ["/full/path/to/wxcore/dist/build"] (note: I tried using just "dist/build", but cabal said: "library-dirs: dist/build is a relative path")
This still isn't quite enough, it took one more kludge: /full/path/to/wxcore/dist/build/$ ln -s /full/path/to/wxcore/dist/build/libwxc.so.0.13.1 libwxc.so I'd like to think there's a more elegant solution than this and I'm open to suggestions. I should note that at this point I became aware that we could have an entirely separate cabal project (perhaps "wxc") for this shared library, and then have wxcore depend on it? Running one more time I *thought* I was finally there, until I noticed this line, hidden in cabal's output: /full/path/to/wxcore/dist/build/libwxc.so: file not recognized: File truncated As far as I can tell, this occurs because we now load(?) "libwxc.so.0.13.1" in myConfHook, but then in myBuildHook we want to use it as the destination for the linker. Worse still it seems that 'truncated' actually means 'deleted'. To get around this I had to one again remove "x-dll-name" from the parseDLLs call (so I could get to myBuildHook and create "libwxc.so.0.13.1" again), build wxcore, then add "x-dll-name" back again, but this time also comment out the call to linkSharedLib (and so stop cabal attempting to open "libwxc.so.0.13.1") before building wxcore one last time. Finally everything seemed to work. Now, I should note that I have been using cabal-dev, and so I now have a package-conf I need to reference when building with my new shared library wxhaskell. To test it I built the wxhaskell HelloWorld sample thus: /full/path/to/samples/wxcore$ ghc --make -package-conf ../../cabal-dev/packages-7.0.3.conf HelloWorld.hs It worked, but then when I try to run HelloWorld I get: ./HelloWorld: error while loading shared libraries: libwxc.so.0: cannot open shared object file: No such file or directory Okay, again, libwxc.so.0 isn't in a 'normal' place, so I kludged it with another symlink: /full/path/to/samples/wxcore$ ln -s /full/path/to/wxcore/dist/build/libwxc.so libwxc.so.0 And now, I can report that HelloWorld runs. [imagine a screen shot of the HelloWorld sample application running here] Rejoice. But wait, there's more: One of the promises of the shared library approach is that we'd all be able to use wxhaskell with ghci again, and to my utter disbelief, it's true: /full/path/to/samples/wxcore$ ghci -package-conf ../../cabal-dev/packages-7.0.3.conf HelloWorld.hs Ok, modules loaded: Main. Prelude Main> main [imagine a screen shot of the HelloWorld sample application running here] Double rejoice.
Thanks, Dave