- Glasgow-haskell-users

addFinalizer & ThreadId
by Volker Stolz 28 Aug '02

28 Aug '02

I remember that addFinalizer never did work quite the way people expected it to. A current instantiation of this problem: adding a finalizer to a ThreadId. Could somebody please give me a short explanation why the following never prints "Finalize"? [ghc is 5.04-cvs] \begin{code} module Main where import IO import Concurrent import GHC.Weak main = do mv <- newEmptyMVar child <- forkIO $ do takeMVar mv print "terminated" addFinalizer child fin putMVar mv () yield yield yield threadDelay $ 1000*1000*30 print "done" fin :: IO () fin = do print "Finalize" \end{code} -- http://www-i2.informatik.rwth-aachen.de/stolz/ *** PGP *** S/MIME

1 0

RE: RFC: ghc's dynamic linker
by Simon Marlow 28 Aug '02

28 Aug '02

Ok, let's start with a possible API to the library I think you're asking for: loadLibrary :: FilePath -> IO () lookupEntity :: String -> IO a (there's no type checking at dynamic link time of course, so you get to claim the returned object is whatever type you like). Right, now what would it take to implement this. As Duncan points out, this is almost possible already using the GHCi dynamic linker, which is available to any program compiled with GHC via the FFI. The interface is fairly straightforward, eg: foreign import "initLinker" unsafe initLinker :: IO () foreign import "lookupSymbol" unsafe c_lookupSymbol :: CString -> IO (Ptr a) foreign import "loadObj" unsafe c_loadObj :: CString -> IO Int but the main problem is that the dynamic linker can't link new modules to symbols in the currently running binary. So, in order to link a new Haskell module, you first have to load up a fresh copy of the 'base' and 'haskell98' packages, just like GHCi does. It *almost* works to do this, except that you get strange effects, one of which is that you have two copies of stdout each with their own buffer. Going the final step and allowing linkage to the current binary is possible, it just means the linker has to know how to read the symbol table out of the binary, and you have to avoid running 'strip'. I believe reading the symbol table is quite straightforward, the main problem being that on Unix you don't actually know where the binary lives, so you have to wire it in or search the PATH. Another problem is that you don't normally link a *complete* copy of the base package into your binary, you only link the bits you need. Linking the whole lot would mean every binary would be about 10M; but doing this on the basis of a flag which you turn on when you want to do dynamic linking maybe isn't so bad. There are a couple of other options: - make your program into a collection of dynamically-linked libraries itself. i.e. have a little stub main() which links with the RTS, and loads up 'base' followed by your program when it starts. The startup cost would be high (we don't do lazy linking in Haskell), but you'd only get one copy of the base package and this is possible right now. - make GHC generate objects that play nicely with the standarad dynamic linker on your system. This is entirely non-trivial, I believe. See previous discussions on this list. However, it might get easier in the future; I'm currently working on removing the need to distinguish code from data in GHC's RTS, which will eliminate some of the problems. Summary: extending GHC's dynamic linker to be able to slurp in the symbol table from the currently running binary would be useful, and is a good bite-sized GHC hacker task. I can't guarantee that we'll get around to it in a timely fashion, but contributions are, as always, entirely welcome... Cheers, Simon

5 8

RE: RFC: ghc's dynamic linker
by Simon Marlow 28 Aug '02

28 Aug '02

> I just want to remark that anyone contemplating hacking on object > files should steer well clear of BFD which is the most inconsistently > implemented, poorly (though abundantly) documented obstacle to quality > programming it has ever been my misfortune to use. Particularily > galling is the fact that BFD provides little indication (in > documentation, compile-time errors, etc.) that certain functionality > is completely unimplemented on a particular platform. > > In contrast, libelf is widely available (multiple implementations > exist), well documented, a joy to use and covers everything (well, > everything that GHC runs on) but Windows. Thanks, I didn't know about libelf. That might be a good replacement for our custom linker (at least on ELF platforms) at some point. Cheers, Simon

2 1

RE: Hardware STG?
by Simon Peyton-Jones 28 Aug '02

28 Aug '02

| advantages. The cost of course is always an issue, but my | point is that the | difference in performance may in many cases be insignificant. | | I see a definite place for this hardware, if it can be engineered | successfully. For example, I could build a firewall that is provably | inpenetrable. I could build embedded medical devices that I | can prove will not fail mysteriously. Yes, you are right. One possible goal is performance, but another, as you point out, is reliability and correctness. The issues are quite different and I'm sure Keean will want to be clear which objective he is shooting for. If you could really build a device where you had a proof of correctness that stretched from the original source code right through to hardware execution, that would be a real tour de force. Simon

1 0

RE: Posix.Fd questions
by Simon Marlow 28 Aug '02

28 Aug '02

> 1) Is the Posix stuff for doing IO operations on Fd in the > new hierarchy? Not yet. > If so where? The current plan is to put it in its own System.Posix (or possibly System.Unix) sub-hierarchy. > If not about when will it be there? When someone gets around to doing it :-) > (I'm wondering when I should > schedule the job of changing over all > the imports in our existing source code to the new hierarchy.) > 2) Is there any way of finding out whether an Fd you are > reading from has characters available? You can do a non-blocking read of a single character, or a select (not sure if the Select library still works, though). Cheers, Simon

1 0

RE: RFC: ghc's dynamic linker
by Simon Marlow 28 Aug '02

28 Aug '02

> On Tue, Aug 27, 2002 at 05:19:15 +0100, Simon Marlow wrote: > > > 'haskell98' packages, just like GHCi does. It *almost* works to do > > this, except that you get strange effects, one of which is > that you have > > two copies of stdout each with their own buffer. > > If it's not too much trouble, do you mind explaining why this is > so? It's just to satisfy my curiosity; don't worry if it's too > long-winded or contains really heavy wizardry :). It's quite simple: there will be one stdout Handle in your program, and one in the base package that has been loaded. Each is a separate Handle in its own right, with its own buffer, although they both point to the same underlying file descriptor. This can lead to strange effects because different parts of the program will be using different stdout handles, and the output might not be ordered in the way you expect. For functions, it's ok to have two identical copies, because you can't observe which one is being used. For most normal CAFs, the same is true except that the program might take longer to run than expected. You would be able to observe that there are two copies of the random number stream from the Random library, however. > You've already got the symbol table in memory though, right? Is > it absolutely necessary to re-read the binary? not sure what you mean here. The symbol table isn't normally read when a binary is executed - the same binary works fine after it is stripped, after all. Cheers, Simon

1 0

RE: RFC: ghc's dynamic linker
by Simon Marlow 28 Aug '02

28 Aug '02

> On 28 Aug 2002 01:34:37 +0100 > Alastair Reid <alastair(a)reid-consulting-uk.ltd.uk> wrote: > > > > 2. The BFD way > > I was just about to ask about this. I was going to ask what > ghc uses for > manipulating .o files. I've come accross BFD through gdb but > have never had to use it seriously. We do it The Hard Way, i.e. we have our own complete object linker. It's a lot simpler than BFD of course, because it only has to handle static linking. It currently supports ELF and the Windows PE object format. Cheers, Simon

1 0

RE: RFC: ghc's dynamic linker
by Simon Marlow 28 Aug '02

28 Aug '02

> From: Duncan Coutts [mailto:duncan@coutts.uklinux.net] > Sent: 28 August 2002 00:51 > > Would it be easier/better to exlicitly specify a symbol map > file for the > linker to use to locate the appropriate points in the current binary. > Then perhaps we need a flag to ask ghc to spit out a symbol map along > with the .o. Alternatively there tools to extract the map from a .o, I > don't know - I'm not a bin-utils guru! Possibly, but our linker already has most of the machinery necessary to read the symbol table out of a binary (the binary format is largely the same as an object file, it just has a fixed load address), so I'm guessing that using a symbol map will be extra work. > > Summary: extending GHC's dynamic linker to be able to slurp in the > > symbol table from the currently running binary would be > useful, and is a > > good bite-sized GHC hacker task. I can't guarantee that we'll get > > around to it in a timely fashion, but contributions are, as always, > > entirely welcome... > > Having made the suggestion, I'ts only right that I contribute > my (limited) > skills. I have done some gdb hacking before (not out of choice you > understand!) so I ought to know a bit about .o's ELF sections > and such. Take a look in ghc/rts/Linker.c if you're interested. Cheers, Simon

1 0

Posix.Fd questions
by George Russell 28 Aug '02

28 Aug '02

1) Is the Posix stuff for doing IO operations on Fd in the new hierarchy? If so where? If not about when will it be there? (I'm wondering when I should schedule the job of changing over all the imports in our existing source code to the new hierarchy.) 2) Is there any way of finding out whether an Fd you are reading from has characters available?

1 0

RFC: ghc's dynamic linker
by Duncan Coutts 28 Aug '02

28 Aug '02

I'm afraid this is going to be another ghc feature request. Let me try to explain why it would be worth it. People want to write programs using Haskell as an extension language. Think of a plugin system or an embeded language. This is currently possible with ghc but requires some cunning hacks and fiddling with the linker and it has some limiations. The difficulty is obviously in accessing the linker. It would be great if the linker was made avaliable directly in some ghc-specific library. (I though this was the case but apparently not.) In particular so that such programs don't need an instalation of ghc lying around to be able to run. Ie the dyamic linker could be statically linked into a program. I'd guess that this stuff is already in ghci so maybe it'd just be a matter of exposing it for other progrms to use. The second problem is that for an extension language, the dynamicaly loaded modules need to be able to call functions in the 'host' program. This means that the linker needs access to the host program's symbol table (ie its own symbol table), or at least the portion of it that provides the plugin API. Simon PJ noted that this might be a nice feature for ghci as well as for enabling plugin systems. It might allow ghci to not have to load a copy of the standard librarys since it could link programs to the copy that is already loaded inside ghci itself. This would save time and memory when loading programs into ghci. He also suggested a couple possible ways this might be used. The simpest might be to add (yet!) another flag while compiling a module to indicate that the symbol table for the current module should be made avaliable to the dyamic linker. One would then compile the host program's API module with this flag so extension modules can link against it. A more general scheme might be to add pragmas to explicity indicate the external symbols to associate with haskell functions. This sounds like more work however and I don't know that the extra generality is necessary. He also suggested I beg Simon Marlow to think about a sensible implementation. :-) The model for using a plugin system would be quite simple and easy to use. One writes a module which imports the API module, the .hi file for which is provided by the host program. The module is then compiled to a .o file. That's it, the host program can then dynamically load the .o file and link it against the actual API code in its own runtime image. The particular application I have in mind is adding Haskell as an extension language to hIDE the GTK Haskell IDE. http://www.dtek.chalmers.se/~d99josve/hide/ I believe there are a number of people who would be interested in making dynamic linking easier and might help out with testing and documentation. So how difficult would this be? Is it worth it? Duncan Coutts

4 5