Am Samstag, 25. März 2006 20:00 schrieb Brian Hulley:

I've found a workaround to the problem below: instead of trying to use hs_free_fun_ptr, I instead pass a FunPtr to the Haskell function freeHaskellFunPtr into my DLL, and use this to free everything, finally using it to free itself (!) which I assume should be safe. [...]

It has been quite some time since I've worked on GHC's Adjustor.c and Hugs' FFI, but IIRC it is in general *not* safe to do this. On some platforms code is being generated dynamically for these kind of callbacks, which has already been freed by the time the callback returns. This might appear to work, depending on your processor architecture and dynamic memory management behaviour, but it's better not to rely on this. Perhaps the FFI spec should be clearer here. I had a similar problem in for timer callbacks in my GLUT binding. They are one-shot and have to clean themselves up after the callback executes. The solution was to register the FunPtrs to be cleaned up only and do the real work in another "scavenger" callback, see http://darcs.haskell.org/packages/GLUT/Graphics/UI/GLUT/Callbacks/Registrati... Cheers, S.

Sven Panne wrote:

Am Samstag, 25. März 2006 20:00 schrieb Brian Hulley:

...

I've found a workaround to the problem below: instead of trying to use hs_free_fun_ptr, I instead pass a FunPtr to the Haskell function freeHaskellFunPtr into my DLL, and use this to free everything, finally using it to free itself (!) which I assume should be safe. [...]

It has been quite some time since I've worked on GHC's Adjustor.c and Hugs' FFI, but IIRC it is in general *not* safe to do this. On some platforms code is being generated dynamically for these kind of callbacks, which has already been freed by the time the callback returns. This might appear to work, depending on your processor architecture and dynamic memory management behaviour, but it's better not to rely on this. Perhaps the FFI spec should be clearer here.

I'm pretty sure this *does* work with GHC, at least on some platforms (definitely x86 and x86_64, I'm not sure about the others). We're careful not to require any dynamically-allocated code to run when the callback returns. I agree the FFI spec should be clearer here. Cheers, Simon

Am Montag, 27. März 2006 14:27 schrieb Brian Hulley:

[...] For example, in:

foreign import ccall "wrapper" mkIO :: IO () -> IO (FunPtr (IO ())) foreign import ccall set_callback :: FunPtr (IO ()) -> IO () foreign import ccall run :: IO ()

foo1 :: IO () foo1 = do set_callback foo2 something_else -- does this code still exist?

foo2 :: IO () foo2 = ...

main = do set_callback foo1 run -- causes foo1 to be called at some point

where set_callback wraps its arg in a FunPtr and calls freeHaskellFunPtr if a previous callback function has already been specified.

If foo1 is generated as dynamic code, then freeing the FunPtr to it as a consequence of changing the callback to foo2, becomes problematic unless there is a guarantee that currently executing code is always considered to be "rooted" by the program counter.

So my questions are:

1) Is my understanding of the FFI definition of freeHaskellFunPtr incorrect?

Partially, yes. Your example above leaves out an important point: Calling mkIO. Regardless of the implementation details, this has to accomplish two things: * Making sure that the Haskell function being wrapped is not garbage collected. This is important, because the returned FunPtr might contain the last reference to the wrapped function internally. * Setup some mechanism which does the "impedance matching" between C and Haskell, i.e. put the arguments in the right places, do some stack fiddling, etc. For GHC (and Hugs, IIRC), this is solved by generating a StablePtr to the Haskell function and "baking" it somehow into dynamically generated code (see ghc/rts/Adjustor.c), which does the argument/stack fiddling and then jumps to some C stub, which in turn uses the StablePtr and the rest of the arguments to call the Haskell function. If this sounds tricky and highly platform-dependent: Yes, it is! :-) The main point here is: The Haskell runtime doesn't know the time when it is safe to free the StablePtr and the dynamically generated adjustor code. The FunPtr's C equivalent could be stored anywhere in C land, perhaps in some GUI callback table, some global variables, it could be in use by some other (non-Haskell) thread, etc. etc. Consequently, the programmer must help via freeHaskellFunPtr/hs_free_fun_ptr, but this should not happen while the FunPtr is in use, e.g. while the Haskell callback is currently being executed. The technical reason for this is that after returning from Haskell land, the adjustor code might need to do some cleanup: C -> adjustor -> stub -> Haskell -> stub -> adjustor -> C It could be the case that the adjustor tail-jumps to the stub, but this is not guaranteed to be the case for all platforms.

2) Are some implementations not treating the program counter and call stack as roots for blocks of memory which contain dynamically generated code that is currently executing?

As hopefully explained above, a pointer to the adjustor could be everywhere, so GC is not an option here.

3) Are some implementations treating a FunPtr as the sole owner of the code it is pointing to, thus requiring the workaround of a scavenger function to safely free FunPtr's at a point where it is known that none of the code pointed to by any of the FunPtr's is currently executing?

Ownership is not the problem here, but lifetime: A callback should simply not free its own FunPtr. This requirement is not much different from having to avoid accessing memory which has already been freed. I know that this is all complicated stuff, but if you would really like to understand the mechanisms behind, it would probably be best to pick a platform you know and have a look at dsFExportDynamic in ghc/compiler/deSugar/DsForeign.lhs and ghc/rts/Adjustor.c. Cheers, S.

C -> adjustor -> stub -> Haskell -> stub -> adjustor -> C

It could be the case that the adjustor tail-jumps to the stub, but this is not guaranteed to be the case for all platforms.

Hmmm, I thought it was. Well, the FFI addendum is rather vague on this point; this seems to be all it says about freeHaskellFunPtr. freeHaskellFunPtr :: FunPtr a -> IO () Release the storage associated with the given \code{FunPtr}, which must have been obtained from a wrapper stub. This should be called whenever the return value from a foreign import wrapper function is no longer required; otherwise, the storage it uses will leak. Do I need the FunPtr value in order to return to C land? I didn't think so. The addendum doesn't say so. (It doesn't clearly say otherwise, though). All of the implementations in Adjustor.c *except* for the IA-64 one go to some lengths to ensure that a tail-calll is used: C -> adjustor -> stub -> Haskell -> stub -> (maybe some static code) -> C I think this confusion should be fixed... Cheers, Wolfgang

Even as the author of some parts of Adjustor.c and some parts of Hugs' FFI implementation I have to admit that it isn't clear to me at all if tail-calls are used everywhere. %-)

Hugs uses tail-jumps or static return code on all supported platforms, GHC on all platforms except IA64.

And even if this is currently the case: Can we really guarantee this for all eternity for every strange ABI people will come up with? I'm not so sure about this, therefore I'd recommend against "self-destruction", as handy as it admittedly is. I'd be happy to be convinced of the opposite...

You'd be happy? OK, I'll try. I think it will be possible for all ABIs. Tearing down the stack frame or whatever needs to be done on the way out can always be done by a piece of static code; even if some "dynamic" information (like, for example, the size of the stack frame) is needed, this information can be stored in the stack frame or in a register by the function that sets up the stack frame. Or if you want to look at it differently: 1.) We can generate a thunk that will tail-call to a static piece of code and pass an additional constant argument to that static piece of code, with a custom calling convention. 2.) We can dynamically generate a call to the stub function. If those two axioms are fulfilled for a platform, we can support self- destruction. 2 will definitely be supported (otherwise NO interpreters with FFI would work on that platform), and I can't imagine how a platform can NOT support 1.... Grüße, Wolfgang

Krasimir Angelov wrote:

Hi Brian,

The problem is that hs_free_fun_ptr is defined in a static library (the Haskell RTS) while you have declared it with __declspec(dllimport). In this case the compiler is tring tp optimize the call and it is using __imp__hs_free_fun_ptr instead of hs_free_fun_ptr. You have to remove the dllimport pragma and to link your program with the Haskell RTS library. Unfortunatelly the the static library format is different for VC++ and GCC and I expect that you may have problems if you try to statically link gcc code with VC++ code.

Hi Krasimir, Yes I was getting confused about the difference between static and dynamic libraries. I've mostly been using static libs in my C++ projects, and when you build a static lib it doesn't matter that some functions are not defined in it: no linking is done at this point, so the library can be created without needing to specify where any extern functions are actually defined. However when building a DLL, the linker is invoked and so it needs to know the location of any function not defined in the DLL itself. But since my DLL needs to be used by multiple Haskell applications, and since the RTS is built into the application code as opposed to being supplied by a separate DLL itself, there is no way to link the DLL against RTS API functions, even if the VC/GCC imcompatibilites could be overcome. However there is no need for this anyway because of the simple workaround of just wrapping freeHaskellFunPtr in a FunPtr and passing this into the DLL, effectively doing dynamic linking of this function explicitly rather than relying on the low level OS-specific linking process to accomplish this. Thanks, Brian.