
I'm afraid this is going to be another ghc feature request. Let me try to explain why it would be worth it. People want to write programs using Haskell as an extension language. Think of a plugin system or an embeded language. This is currently possible with ghc but requires some cunning hacks and fiddling with the linker and it has some limiations. The difficulty is obviously in accessing the linker. It would be great if the linker was made avaliable directly in some ghc-specific library. (I though this was the case but apparently not.) In particular so that such programs don't need an instalation of ghc lying around to be able to run. Ie the dyamic linker could be statically linked into a program. I'd guess that this stuff is already in ghci so maybe it'd just be a matter of exposing it for other progrms to use. The second problem is that for an extension language, the dynamicaly loaded modules need to be able to call functions in the 'host' program. This means that the linker needs access to the host program's symbol table (ie its own symbol table), or at least the portion of it that provides the plugin API. Simon PJ noted that this might be a nice feature for ghci as well as for enabling plugin systems. It might allow ghci to not have to load a copy of the standard librarys since it could link programs to the copy that is already loaded inside ghci itself. This would save time and memory when loading programs into ghci. He also suggested a couple possible ways this might be used. The simpest might be to add (yet!) another flag while compiling a module to indicate that the symbol table for the current module should be made avaliable to the dyamic linker. One would then compile the host program's API module with this flag so extension modules can link against it. A more general scheme might be to add pragmas to explicity indicate the external symbols to associate with haskell functions. This sounds like more work however and I don't know that the extra generality is necessary. He also suggested I beg Simon Marlow to think about a sensible implementation. :-) The model for using a plugin system would be quite simple and easy to use. One writes a module which imports the API module, the .hi file for which is provided by the host program. The module is then compiled to a .o file. That's it, the host program can then dynamically load the .o file and link it against the actual API code in its own runtime image. The particular application I have in mind is adding Haskell as an extension language to hIDE the GTK Haskell IDE. http://www.dtek.chalmers.se/~d99josve/hide/ I believe there are a number of people who would be interested in making dynamic linking easier and might help out with testing and documentation. So how difficult would this be? Is it worth it? Duncan Coutts

Duncan Coutts
[...] People want to write programs using Haskell as an extension language. Think of a plugin system or an embeded language. This is currently possible with ghc but requires some cunning hacks and fiddling with the linker and it has some limiations. [...]
It seems like what you're asking for is partly covered by the foreign function interface specification http://www.cse.unsw.edu.au/~chak/haskell/ffi/ especially foreign export (section 3.4), dynamic wrappers (section 4.1.3), and hs_init and friends (section 6.1). It'd be good if you could express your request in terms of things not provided by this interface, provided in an awkward or inappropriate way, etc. (bearing in mind that the ffi is often accessed indirectly through preprocessors like GreenCard, c2hs, hsc2hs, KDirect, HDirect, etc. which can eliminate much of the awkwardness). -- Alastair Reid alastair@reid-consulting-uk.ltd.uk Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/

I don't understand something about this discussion which is probably just ignorance on my part, so apologies in advance. Or, perhaps what I'm thinking doesn't apply in all environments. Watching what happens when ghc compiles a module (this doesn't apply to ghci, just to compilation), the executable is actually created by a call to gcc. Gcc is already capable of linking statically, creating a program that can run without the presence of ghc on a machine. Furthermore, when a (normally, that is, dynamically) linked executable (created by ghc) is executed, the dynamic loader is not anything specific to ghc but is just the normal O/S dynamic loader (typically ld.so). So, if the code can be executed by the O/S with ld.so, the symbol table is already present. I ran a quick test on these ideas and was able to run executables produced by GHC on systems that have no part of GHC installed. What am I missing here? TIA, On Tuesday 27 August 2002 07:32, Alastair Reid wrote:
Duncan Coutts
writes: [...] People want to write programs using Haskell as an extension language. Think of a plugin system or an embeded language. This is currently possible with ghc but requires some cunning hacks and fiddling with the linker and it has some limiations. [...]
It seems like what you're asking for is partly covered by the foreign function interface specification
http://www.cse.unsw.edu.au/~chak/haskell/ffi/
especially foreign export (section 3.4), dynamic wrappers (section 4.1.3), and hs_init and friends (section 6.1).
It'd be good if you could express your request in terms of things not provided by this interface, provided in an awkward or inappropriate way, etc. (bearing in mind that the ffi is often accessed indirectly through preprocessors like GreenCard, c2hs, hsc2hs, KDirect, HDirect, etc. which can eliminate much of the awkwardness).
-- ----------------------------------- Seth Kurtzberg M. I. S. Corp. 1-480-661-1849

On Tue, 27 Aug 2002 08:46:12 -0700
Seth Kurtzberg
I don't understand something about this discussion which is probably just ignorance on my part, so apologies in advance. Or, perhaps what I'm thinking doesn't apply in all environments.
Sorry, it is rather confusing. X linking with Y linkng with itself! Eugh. :-)
Watching what happens when ghc compiles a module (this doesn't apply to ghci, just to compilation), the executable is actually created by a call to gcc. Gcc is already capable of linking statically, creating a program that can run without the presence of ghc on a machine.
Yes.
Furthermore, when a (normally, that is, dynamically) linked executable (created by ghc) is executed, the dynamic loader is not anything specific to ghc but is just the normal O/S dynamic loader (typically ld.so).
There's two different kinds of dynamic linking when it come to ghc! ghc produces executibles that are statically linked with the Haskell libraries but dynamically linked with certian C libraries ie libc and gmp. ghci has its own dynamic linker for linking pre-compiled .o modules or packages with each other and with byte code.
So, if the code can be executed by the O/S with ld.so, the symbol table is already present.
Yes / maybe. As the Haskell object code is staticaly linked, there is no need to keep the symbol table around, so if you strip (the final executable) they will disapear. .so files obviously have to keep their symbols so that they can be linked at runtime, I guess they use some special ELF section for that.
What am I missing here?
Does this make any sense? :-) Duncan

On 27 Aug 2002 15:32:53 +0100
Alastair Reid
People want to write programs using Haskell as an extension language. Think of a plugin system or an embeded language. This is currently possible with ghc but requires some cunning hacks and fiddling with the linker and it has some limiations. [...]
It seems like what you're asking for is partly covered by the foreign function interface specification
I'd really like to do it without using the ffi. It would be possible to use a C interface between the main program and the extension modules but this would make it much harder to use and restricts the way data and control is passed between the modules. It also seems unncecessary as all we're trying to do here is link Haskell code with Haskell code! An advantage of using the ffi would be that we could use proper .so files and the system dynamic linker. ghci does all this quite nicely already, I want the same functionality avaliable for other programs to use. There is one crucial difference however, ghci assembles 'other' progams albeit within its own address space. A plugin system needs to link things to itself.
It'd be good if you could express your request in terms of things not provided by this interface, provided in an awkward or inappropriate way, etc. (bearing in mind that the ffi is often accessed indirectly through preprocessors like GreenCard, c2hs, hsc2hs, KDirect, HDirect, etc. which can eliminate much of the awkwardness).
The thing is I'm not really linking to foriegn code, I'm just linking a Haskell program on the fly rather than statically. If this were to be done through the ffi perhaps it would be an extra language-dependant target - Haskell! Actually this might be a good approach for a semi-stable ghc ABI to enable .so 's to be useful. That's much more ambitious than what I need. If I've not been very clear, let me try to explain by example. Every decent editor / IDE has an embeded scripting language of some sort. Emacs has elisp for example. A Haskell IDE should really have Haskell as its embeded / enxension language so that us Haskell hackers don't have to learn elisp to make modifications to our programming environment. One could embed hugs into your editor / IDE and provide it with some extra IO () primitives for accessing / modifying the state of the editor. It seems much nicer to reuse the exsisting framework we have in the ghc runtime system. In fact this is possible at the moment if we were to load the IDE in ghci and then use ghci to load up extension modules. Done like this, the distinction between internal module and external plugin modules dissapears. So under that model we have an external agent, ghci, linking the IDE to its extension modules at runtime. The dependancy on ghci is unsatisfactory however so it would be nice to be able to move the linking ability from the external agent into the 'host' program itself. Perhaps the calls from the host program to the extension modules could/should go via the ffi. (eg each module exports a pluginInit or main function or something) But calls from the plugin into the the host program can be direct since it knows all about the program it has been linked into since it imported the IDE's API module. It's just a matter of delayed linking, the plugins could be statically linked without any problem. Creating a plugin would be no different to writing an internal module exept for the module dependancies and the delayed linking. If this could be done through the ffi in a reasonable way, I'd be interested to hear about alternatives particularly if the ghc implementors say this ghci style dynamic linking stuff is too difficult or silly! Duncan Coutts

On Tue, Aug 27, 2002 at 04:57:15 +0100, Duncan Coutts wrote:
Alastair Reid
wrote: People want to write programs using Haskell as an extension language. Think of a plugin system or an embeded language. This is currently possible with ghc but requires some cunning hacks and fiddling with the linker and it has some limiations. [...]
It seems like what you're asking for is partly covered by the foreign function interface specification
I'd really like to do it without using the ffi. It would be possible to use a C interface between the main program and the extension modules but this would make it much harder to use and restricts the way data and control is passed between the modules.
I'm currently using the FFI, but I'm (ab)using the FFI in such a way that it's possible to pass back any types you wish to the calling program. So a host program can actually load any function in a plugin module, which the FFI doesn't normally allow. The best thing is that you can load a function of type Dynamic, and then coerce that back to whatever type the function was originally -- which means that you even get some type-checking done. (The chiba/runtime_loader/RuntimeLoader.hs file does all this right now; check out the files in tests/ChibaTest0.hs). If people are interested, I can package up a very small demo of all the runtime loading stuff separately from the rest of my project, so you guys can check it out without being confused by the code which actually does useful stuff ;-).
The thing is I'm not really linking to foriegn code, I'm just linking a Haskell program on the fly rather than statically. If this were to be done through the ffi perhaps it would be an extra language-dependant target - Haskell! Actually this might be a good approach for a
Or maybe the "Haskell" language-dependent target should be "GHC"
(which then allows you to load modules compiled by other Haskell
systems, like nhc). Sounds like a good idea, although I have no
idea what's involved in implementing that sort of things :).
--
#ozone/algorithm
participants (4)
-
Alastair Reid
-
Andre Pang
-
Duncan Coutts
-
Seth Kurtzberg