Re: [Haskell-cafe] Use Haskell shared library with foreign exports with dlopen/dlsym

10 Sep 2014

      This is going to be very cunning.

hs_init is not needed in this case because the executable comes from

     ghc --make -dynamic test.hs

The "-dynamic" causes the exemption. This is similar to the case of ghci 
7.8 on Linux.

(I agree that there are differences too. For now, the similarity 
dominates. The executable has already brought in a dynamic RTS, and it 
has already been hs_inited and running. Subsequently, any dynamic 
library that looks for a dynamic RTS will simply find the running one.)

Here is the story of the segfault:

1. "foreign export" causes the following dynamic library constructor to 
be included in the dynamic library:

static void stginit_export_Test_zdfstableZZC0ZZCmainZZCTestZZCtest() 
__attribute__((constructor));
static void stginit_export_Test_zdfstableZZC0ZZCmainZZCTestZZCtest()
{
   foreignExportStablePtr((StgPtr) 
&Test_zdfstableZZC0ZZCmainZZCTestZZCtest_closure);
}

foreignExportStablePtr is in the RTS (and we do have a dynamic RTS 
already running). Test_zdfstableZZC0ZZCmainZZCTestZZCtest_closure is in 
the dynamic library representing the test::Int in module Test.

I found out by -keep-tmp-files and examining the tmp files.

2. dlopen causes a mmap, and the closure's address is in that mmap region.

3. dlopen also causes the constructor to be invoked. I suppose it means 
a StablePtr is created, therefore the closure's address is registered in 
the global table of StablePtrs.

4. dlclose causes a munmap. The closure's address is now invalid.

5. Garbage collection visits everything registered in the global table 
of StablePtrs (among other roots). But one of the registered addresses 
is invalid. Segfault.

I found out by asking gdb for backtrace. So in fact I can specifically 
point at "evacuate()".

At this point, my recommendation:

Don't use "-dynamic" on the executable. Do it static.

This implies you will have two RTSes, the static one used by the 
executable for itself, and the dynamic one used by the dynamic library 
for itself.

This is a good thing because the static GCtor will never visit the 
closure in the dynamic library, hell it will never hear of it.

The dynamic library will find and use foreignExportStablePtr from the 
dynamic RTS only. It doesn't see the static RTS, or any static library 
for that matter.

The dynamic GCtor is the only one who knows of the closure, but this 
GCtor no longer runs after dlclose, hell it no longer exists.

The down side is that you now have to do the full thing to the dynamic 
library:
http://www.vex.net/~trebla/haskell/so.xhtml
section "Making The Library"

P.S. I believe that those things about hs_add_root and __stginit_* are 
outdated.

On 14-09-08 12:47 PM, Carter Schonwald wrote:
...
what OS and other things?
what does ghc --info say?
Also, when calling haskell code as if it were C code, you need to init
the RTS its using, its not going to magically know you're linking to it
from haskell.(though maybe you can arrange things to use the pre inited
hs runtime, but i'm not familiar with how to do so)
http://www.haskell.org/haskellwiki/Calling_Haskell_from_C
On Mon, Sep 8, 2014 at 10:05 AM, Tobias Neumann mailto:mail@tobias-neumann.eu> wrote:
Hello Cafe,
I am trying to use a Haskell shared library with foreign exports from
    Haskell again via dlopen/dlsym. Sadly it segfaults, and the segfaults
    happen on dlclose during garbage collection points (as figured out by
    monochrom in #haskell). So right now I can only open a library once and
    may not dlclose it. Can someone point me to a mistake I made, or is this
    rather a ghc (7.8.3) bug? Please see attached minimal example.
Regards, Tobias
test.hs:
    module Main where
import qualified System.Posix.DynamicLinker as DL
    import Foreign
foreign import ccall "dynamic"
         mkTest :: FunPtr Int -> Int
main = do
         DL.withDL ("./libtest.so") [DL.RTLD_NOW] $ \dl -> do
             dimPtr <- DL.dlsym dl "test"
             let test = mkTest dimPtr
             print test
libtest.hs:
    module Test() where
import Foreign
foreign export ccall test :: Int
test :: Int
    test = 124
build with:
             ghc --make -shared -dynamic -fPIC libtest.hs -o libtest.so
             ghc --make -dynamic test.hs
_______________________________________________
    Haskell-Cafe mailing list
    Haskell-Cafe@haskell.org mailto:Haskell-Cafe@haskell.org
    http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] Use Haskell shared library with foreign exports with dlopen/dlsym

Albert Y. C. Lai