Dynamically loading and unloading (C) object files
Hi guys, We started experimenting with dynamically loading and unloading C object files (using the GHC API). I have a simple example with an object file exporting a single symbol "f"; I have two versions of the object file; in the first one, f is defined int f() { return 1234; } The second one uses the value "4321" instead. I can dynamically load and unload these object files (using "linkObj" and "unlinkObj" from "ObjLink"). I can load the first, dynamically compile some Haskell code that imports this symbol, then upload this object file and load the second, and as long as I make sure that the Haskell code gets recompiled (perhaps a relink step suffices, haven't experimented an awful lot yet) it can then interact with the second object file. All this is good news -- in fact, it is more than we had hoped for, because ghci does not allow dynamically unloading or loading any object files at all. Object files specified on the command line are handled in initDynLinker and the ghci top-level function doesn't even get to see them, and we cannot load object files from inside a ghci session. So that leads me to wonder: are there limitations that we should be aware of? Have I simply been lucky so far? Any points or suggestions for things to try would be appreciated, Edsko
So that leads me to wonder: are there limitations that we should be aware of? Have I simply been lucky so far?
If you are loading Haskell code, you will need to be very careful to make sure you load all of the dependencies as well. There are a number of plugins packages and a paper: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.7627 GHC API changes a lot, so it's easy for these packages to get bitrotted. The upcoming release of GHC will run constructors (link-time initializers), but will not run destructors. There is only partial support for weak symbols. http://ghc.haskell.org/trac/ghc/ticket/3333 Edward
The published version of that paper in the ACM digital library...
http://dl.acm.org/citation.cfm?id=1017478
On Thu, Oct 31, 2013 at 5:10 PM, Edward Z. Yang
So that leads me to wonder: are there limitations that we should be aware of? Have I simply been lucky so far?
If you are loading Haskell code, you will need to be very careful to make sure you load all of the dependencies as well. There are a number of plugins packages and a paper: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.7627
GHC API changes a lot, so it's easy for these packages to get bitrotted.
The upcoming release of GHC will run constructors (link-time initializers), but will not run destructors.
There is only partial support for weak symbols. http://ghc.haskell.org/trac/ghc/ticket/3333
Edward _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
On 31/10/2013 18:06, Edsko de Vries wrote:
Hi guys,
We started experimenting with dynamically loading and unloading C object files (using the GHC API). I have a simple example with an object file exporting a single symbol "f"; I have two versions of the object file; in the first one, f is defined
int f() { return 1234; }
The second one uses the value "4321" instead. I can dynamically load and unload these object files (using "linkObj" and "unlinkObj" from "ObjLink"). I can load the first, dynamically compile some Haskell code that imports this symbol, then upload this object file and load the second, and as long as I make sure that the Haskell code gets recompiled (perhaps a relink step suffices, haven't experimented an awful lot yet) it can then interact with the second object file.
So you're loading C code using linkObj, and then compiling some Haskell code that uses 'foreign import' to reference the C code? That *should* be fine, but it's not a use-case that we have thought about. I certainly wouldn't try to load Haskell code manually while also using the GHC API. The RTS linker isn't thread-safe, so be careful about using the GHC API and calling linkObj etc. concurrently.
All this is good news -- in fact, it is more than we had hoped for, because ghci does not allow dynamically unloading or loading any object files at all. Object files specified on the command line are handled in initDynLinker and the ghci top-level function doesn't even get to see them, and we cannot load object files from inside a ghci session.
Nobody asked for this, so we never implemented it :-) It ought to be entirely reasonable to extend the GHC API to allow for loading and unloading of foreign object files, and to expose that functionality in GHCi. Cheers, Simon
So that leads me to wonder: are there limitations that we should be aware of? Have I simply been lucky so far?
Any points or suggestions for things to try would be appreciated,
Edsko _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Ok, great. We are only loading object files from C code, not Haskell
code -- we use the GHC API for that. And we're not using *anything*
concurrently when it comes to GHC, so that should be okay too. I've
been experimenting more with this and it seems to work okay :)
Will let you know if we come across any problems.
Edsko
On Tue, Nov 5, 2013 at 8:51 AM, Simon Marlow
On 31/10/2013 18:06, Edsko de Vries wrote:
Hi guys,
We started experimenting with dynamically loading and unloading C object files (using the GHC API). I have a simple example with an object file exporting a single symbol "f"; I have two versions of the object file; in the first one, f is defined
int f() { return 1234; }
The second one uses the value "4321" instead. I can dynamically load and unload these object files (using "linkObj" and "unlinkObj" from "ObjLink"). I can load the first, dynamically compile some Haskell code that imports this symbol, then upload this object file and load the second, and as long as I make sure that the Haskell code gets recompiled (perhaps a relink step suffices, haven't experimented an awful lot yet) it can then interact with the second object file.
So you're loading C code using linkObj, and then compiling some Haskell code that uses 'foreign import' to reference the C code? That *should* be fine, but it's not a use-case that we have thought about. I certainly wouldn't try to load Haskell code manually while also using the GHC API.
The RTS linker isn't thread-safe, so be careful about using the GHC API and calling linkObj etc. concurrently.
All this is good news -- in fact, it is more than we had hoped for, because ghci does not allow dynamically unloading or loading any object files at all. Object files specified on the command line are handled in initDynLinker and the ghci top-level function doesn't even get to see them, and we cannot load object files from inside a ghci session.
Nobody asked for this, so we never implemented it :-) It ought to be entirely reasonable to extend the GHC API to allow for loading and unloading of foreign object files, and to expose that functionality in GHCi.
Cheers, Simon
So that leads me to wonder: are there limitations that we should be aware of? Have I simply been lucky so far?
Any points or suggestions for things to try would be appreciated,
Edsko _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
participants (4)
-
Andrew Farmer -
Edsko de Vries -
Edward Z. Yang -
Simon Marlow