Re: [Haskell-cafe] [Haskell] Linker flags for foreign export.

Hi Jason,
On 8 March 2011 05:28, Jason Dusek
gcc -g -Wall -O2 -fPIC -Wall -o import \ -I/usr/lib/ghc-6.12.1/include/ \ import.c exports.so
In my experience, the easiest way to do this is to use gcc to build object files from C source files, and then specify those object files on the ghc command line in order to get GHC to do the linking step. This will deal with linking in the correct RTS and, if you specify appropriate -package flags, dependent packages as well. If you want a C main function then see user guide section 8.2.1.1 at http://www.haskell.org/ghc/docs/latest/html/users_guide/ffi-ghc.html. Cheers, Max

On Tue, Mar 8, 2011 at 08:23, Max Bolingbroke
On 8 March 2011 05:28, Jason Dusek
wrote: gcc -g -Wall -O2 -fPIC -Wall -o import \ -I/usr/lib/ghc-6.12.1/include/ \ import.c exports.so
In my experience, the easiest way to do this is to use gcc to build object files from C source files, and then specify those object files on the ghc command line in order to get GHC to do the linking step. This will deal with linking in the correct RTS and, if you specify appropriate -package flags, dependent packages as well.
If you want a C main function then see user guide section 8.2.1.1 at http://www.haskell.org/ghc/docs/latest/html/users_guide/ffi-ghc.html.
Following your advice, I was able to get a working main, linking the .o's (no attempt at an SO this time) with GHC. However, what I was hoping to do was build an SO and that could be linked without GHC, for example via Postgres's "LANGUAGE C" functionality (load SOs and run them) or Ruby's DL/Import (same idea for Ruby). Requiring GHC for linking would really frustrate that goal :) Is there a tutorial I should be following? Well-Typed's blog post on this in the early days of shared object support seemed to be doing what I was doing. -- Jason Dusek Linux User #510144 | http://counter.li.org/

Hi Jason,
Following your advice, I was able to get a working main, linking the .o's (no attempt at an SO this time) with GHC.
I haven't tried it, but how about this: 1. Use ghc to link a standard Haskell executable that requires your libraries. Run the link step with -v so you can see the linker flags GHC uses. 2. Use those link flags to link a .so instead. Importantly, this .so will have already been linked against the Haskell RTS, so you will be able to link it into a C program with no further dependencies. Now, there will be at least one additional complication. Before C can call into Haskell functions you will need to arrange for hs_init and hs_add_root to be executed. In order to do this you will probably have to write an additional C file, bootstrap.c. This should contain an initialization procedure that calls hs_init and hs_add_root in an function decorated with the __attribute__((constructor)) GCC extension. Bootstrap.o should then be statically linked in with the .so in order to initialise the Haskell RTS when C dynamically links in that .so.
Is there a tutorial I should be following? Well-Typed's blog post on this in the early days of shared object support seemed to be doing what I was doing.
Last time I checked Well-Typed's blog posts were one of the best sources of information on this rather arcane topic. There is no easy way to make this work at the moment, AFAIK :-( Cheers, Max

I've gleaned a little bit of useful info from looking at what GHC spits out with -v; I found that ordering the libraries in the way they do it makes one of my undefined symbols (`hs_free_stable_ptr') go away. However, my library ends up with a couple undefined __stginit_* symbols which prevent it from loading. I thought this might be a symptom of building Foo.o from Foo.hs with -dynamic; and indeed, building it without the -dynamic flag gives me an SO with many, many __stginit_* functions defined in it; but that SO causes the loader to segfault (both Ruby's DL/Import and my little test program). I'm trying to hew relatively close to Duncan Coutts' blog posting in working through this; so I have different code and a new Makefile: https://github.com/solidsnack/bash/tree/0e93b6aed7971886c12b95646e5baadc40fe... The three different permutations of SOs -- fully dynamic, fully static and hybrid -- each fail to load differently: :; ./loadfoo ./libfoo.dynamic-dynamic.so **** trying to load ./libfoo.dynamic-dynamic.so **** .so load error: /usr/lib/ghc-6.12.1/base-4.2.0.0/libHSbase-4.2.0.0-ghc6.12.1.so: undefined symbol: forkOS_createThread :; ./loadfoo ./libfoo.static-static.so **** trying to load ./libfoo.static-static.so Segmentation fault :; ./loadfoo ./libfoo.dynamic-static.so **** trying to load ./libfoo.dynamic-static.so **** .so load error: ./libfoo.dynamic-static.so: undefined symbol: __stginit_base_Prelude_dyn The little tester program, `loadfoo', is drawn from the blog post's example; Ruby's DL/Import fails the same way in each case. It would be pretty nice to demo an easy way to load and work with Haskell functions from Ye Olde Favorite Language. Seems like SOs "for the masses" will have to wait a little bit, though. -- Jason Dusek Linux User #510144 | http://counter.li.org/

On 10 March 2011 04:04, Jason Dusek
I'm trying to hew relatively close to Duncan Coutts' blog posting in working through this; so I have different code and a new Makefile:
Starting with your code I've managed to make it work (OS X 10.6, GHC 7). The Makefile is: """ loadfoo: loadfoo.c gcc -arch i386 loadfoo.c -o loadfoo libfoo.dynamic-dynamic.so: Foo.hs fooinit.c ghc -fPIC -c fooinit.c ghc --make -dynamic -fPIC -c Foo.hs ghc -shared -dynamic -o libfoo.dynamic-dynamic.so \ Foo.o Foo_stub.o fooinit.o \ -lHSrts_debug-ghc7.0.1.20101215 test: loadfoo libfoo.dynamic-dynamic.so ./loadfoo libfoo.dynamic-dynamic.so """ (I have to build loadfoo in 32-bit mode because GHC generates 32-bit code on OS X). The fact that we supply the RTS to the GHC link line is important because the shared library would not otherwise link against any particular RTS - in the normal course of things, GHC only decides on the RTS when linking the final executable. I'm linking against the debug RTS there but it should work with the normal one too - I just used the debug version to work out why I got a bus error upon load.. .. and the reason was that your fooinit.c was buggy - that probably account for the crashes you were seeing. The problem is that hs_init takes a *pointer to* argc and argv, not argc and argv directly. You supplied 0 for both of these so GHC died when it tried to dereference them. My new fooinit.c (that works) is as follows: """ #include "HsFFI.h" extern void __stginit_Foo(void); static void Foo_init (void) __attribute__ ((constructor)); void Foo_init (void) { char *arg1 = "fake-main"; char *argv[1] = { arg1 }; char **argv_p = argv; char ***pargv_p = &argv_p; int argc = sizeof(argv) / sizeof(char *); hs_init(&argc, pargv_p); hs_add_root(__stginit_Foo); } """ (Apologies for the extreme ugliness, it's been so long since I wrote C in anger the chain of initializers is the only way I could get this to build without a GCC warning about casts between different pointer types). I tested it with this Foo.hs: """ {-# LANGUAGE ForeignFunctionInterface #-} module Foo where import Foreign.C foreign export ccall foo :: CInt -> CInt foo :: CInt -> CInt foo = (+1) """ And a change to loadfoo.c that actually tries to calls foo: """ printf("**** %s\n", "symbol lookup okay"); + printf("%d\n", foo(1336)); } else { """ (You need "int (*foo)(int) = dlsym(dl, "foo")" as well) The output is 1337 as expected. Cheers, Max

I now have it working for static-static on Linux; but not with dynamic anything yet. Thanks for all your help. -- Jason Dusek () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments

I have managed to get both static and dynamic to work on Ubuntu; and I've set aside a repo on Github to collect notes on this stuff as I work out building on various systems. https://github.com/solidsnack/hso I need rpath for the dynamic-dynamic case on Ubuntu: ghc -shared -dynamic -o libfoo.dynamic-dynamic.so \ Foo.o Foo_stub.o fooinit.o \ -L/usr/lib/ghc-6.12.1 \ -optl-Wl,-rpath,/usr/lib/ghc-6.12.1 \ -lHSrts-ghc6.12.1 The linker options in both cases could be derived automagically from ghc-pkg, I think. I've set aside dynamic-static for now (not sure what use it would be). It turns out there is a simpler way to write Foo_init: extern void __stginit_Foo(void); static void Foo_init (void) __attribute__ ((constructor)); void Foo_init (void) { int argc = 1; char *arg = ""; char **argv = &arg; hs_init(&argc, &argv); hs_add_root(__stginit_Foo); } Is there any case in which the empty string would be unsafe? -- Jason Dusek () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments

On 13 March 2011 22:02, Jason Dusek
Is there any case in which the empty string would be unsafe?
AFAIK this stuff is only used to setup the +RTS options and some of the stuff in System.Environment. I think that the contents of the program name will only cause problems if some code that uses getProgName chokes on the empty string. Cheers, Max
participants (2)
-
Jason Dusek
-
Max Bolingbroke