June 2002 - FFI - Haskell.org

deRefStablePtr exception?
by Alastair Reid 12 Jun '02

12 Jun '02

What exception does deRefStablePtr raise when handed an invalid pointer such as 0? -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/

1 0

How do you represent a C pointer type?
by George Russell 07 Jun '02

07 Jun '02

I have a small C interface with a C type DB. DB handles are only administed on the C side; at the Haskell end they are a black box. The Haskell side never sees DB values, just values of type DB * (pointers to DB). I'm wondering how to represent this. One solution would be just to represent the equivalent of DB * as type DBP = Addr (or Ptr () or something) but then of course it might be confused with any other Addr. A rather more ingenious solution would be something like newtype DBPInternal = DBPInternal Int type DBP = Ptr (Ptr DBPInternal) I think this might actually work in that a DBP would be marshallable and couldn't be confused with any other type not involving DBPInternal. Elegant it isn't, since it involves a pretend pointer inside the Ptr just to get the types working. I think the most elegant solution would be if I could use something like newtype DBP = DBP Addr deriving (-- some class --) and have it work automatically, but I don't know that I can. So tell me, what is the correct solution? George Russell

2 1

RE: Problem with FFI?
by Simon Peyton-Jones 06 Jun '02

06 Jun '02

Manuel: Do you agree that something in the FFI spec needs to be fixed? And hence is on your to-do list? (Just wanting to be sure this one doesn't fall off the radar.) Simon | -----Original Message----- | From: Alastair Reid [mailto:reid@cs.utah.edu] | Sent: 02 June 2002 18:03 | To: John Meacham | Cc: ffi(a)haskell.org | Subject: Re: Problem with FFI? | | | | [reply redirected to ffi(a)haskell.org] | | John Meacham <john(a)repetae.net> writes: | > I may be missing something obvious here, but from the | current FFI spec | > it appears that it is impossible to create libraries in | haskell which | > are meant to be called from c code without running into undefined | > behavior. The problem is in the definition of hs_init() and | hs_exit() | > . now, it is acceptable to have hs_init and hs_exit called in the | > initialization and finalization of your library, but the problem | > arrises when you link against more than one library which is | > implemented in haskell, suddenly whichever library is initialized | > secondly segfaults! (or whatever undefined behaviour means.). | > programs could suddenly stop working when a library is | changed from a | > c implementation to a haskell one, which seems to be a bad thing. | | Hmmm, I see what you mean. | | > proposed fix: allow nested calls to hs_init, hs_exit, a counter is | > incremented when hs_init is called more than once, and | decremented on | > hs_exit. only the last call to hs_exit will actually do | whatever needs | > to be done. | | The hs_init function takes arguments - what if each call to | hs_init specifies a different set of arguments? How about: | | hs_set_hs_argv (int argc, char *argv[]); | hs_set_prog_argv (int argc, char *argv[]); | hs_init (); | hs_exit (); | | Where: | | hs_set_hs_argv sets command line parameters for the Haskell | runtime system | | It can be called at most once and that call must be before the | first call to hs_init | | Warning! The flags are not remotely portable between different | Haskell runtimes. | | hs_set_prog_argv sets arguments seen by getProgName and getArgs | | It can be called at most once and that call must be before the | first call to hs_init | | hs_init initializes the Haskell runtime | | It can be called multiple times and must be called before the first | call to any function exported by the Haskell code. | | hs_exit finalizes the Haskell runtime | | hs_init and hs_exit are required to satisfy the usual | bracketing rules: | | 1) At any time, the number of calls that have been made to | hs_exit must be | no more than the number of calls that have been made to hs_init. | | 2) If the number of calls to hs_exit is equal to the number of calls | to hs_init, then no further calls to hs_init may be made. | | | By the way, the name 'hs_exit' is a little confusing - I | expected it to have an ExitCode argument. 'hs_fini' would | better match its actual purpose. | | > note that this cannot be implemented by the programmer | himself since | > there might be several third party libraries also implemented in | > haskell which an app wishes to link against. | | You can implement the modified API (or your modified | semantics) quite readily as a little library which you link | against your program. All you have to do is tweak the names | a little (e.g., s/hs/HS/) to avoid name clashes. | | -- | Alastair Reid reid(a)cs.utah.edu | http://www.cs.utah.edu/~reid/ | | | | | _______________________________________________ | FFI mailing list | FFI(a)haskell.org | http://www.haskell.org/mailman/listinfo/ffi |

1 0

RE: Minor tweaks to ffi addendum
by Simon Marlow 05 Jun '02

05 Jun '02

> > Alternatively, we could have an explicit operation > > > > runFinalizers :: IO () > > > > which causes finalizers to run in a non-pre-emptive or > > non-concurrent system. > > Of course, this solution is no use at all if your use of ffi is to > pull a big mostly pure C library into Haskell. True. > On two separate occasions, I've added interfaces to image processing > libraries where pretty much every function was pure (it either > generated a fresh image or returned an Int, Float or whatever). > Images tend to be big (640x400x24bits is 3/4 Mbyte) so a few dozen > image processing steps might quickly use up a lot of memory - and in > all that time you don't go anywhere near the IO monad. Sure, but relying on ForeignPtrs and the garbage collector is quite dodgy. To support this kind of use we'd have to be much stricter about when finalizers are run: currently there's no guarantee about the promptness of running the finalizer. Not to mention the fact that there's no guarantee at all about whether the garbage collector will detect that the ForeignPtr is unreachable when you think it should be. I think for these kind of applications using ForeignPtrs isn't the right thing - you really want to do the allocation/deallocation explicitly in the IO monad. Cheers, Simon

2 1

RE: Minor tweaks to ffi addendum
by Simon Marlow 05 Jun '02

05 Jun '02

> In fact, I > always considered it something of a mistake to rely on automatic > finalizers to do anything with human visible consequences (though > I seem to remember that Self or Smalltalk relied on this for > closing windows and found it very effective). Ah yes - if you consider that one instance of "human visible consequences" is "not running out of memory", then I agree :-) Cheers, Simon

1 0

RE: Minor tweaks to ffi addendum
by Simon Marlow 04 Jun '02

04 Jun '02

> section 5.5: > > It's going to be really hard to implement ForeignPtr as specified. > The problem is that invocation of the cleanup function is triggered > by the garbage collector. I don't want the garbage collector to be > recursively invoked so I don't want the garbage collector to > directly invoke Haskell function. The Hugs garbage collector is not > supposed to be a mutator of the heap - so putting a simple wrapper > round the garbage colector won't work. And there's no mechanism > outside the GC to look for cleanup functions to execute. > > GHC gets round this by scheduling a (preemptive) thread to execute > the cleanup code. How on earth does NHC get round this? Does > anyone have a suggestion for how it might be implemented in Hugs? > > Proposed change: none at present but I'm deeply sceptical of a > design which takes a simple task (invoke a C cleanup function) and, > for no discernable reason, generalizes it to the point that it > requires a much more complex runtime system. I've attached some mail I dug up on the subject, from before the creation of ffi(a)haskell.org (there's more in the archives of that list). The upshot is: - Haskell finalizers are much more in keeping with the general philosophy of the FFI: do as much in Haskell as possible. It looks a bit strange if you have to do your allocation in Haskell but deallocation in C. - If we forced finalizers to be C functions because calling Haskell from the GC is inconvenient, then you have to add a constraint that the C function invoked from a finalizer can't call any functions foreign-exported from Haskell land. That's an annoying constraint to have to add, because it means that C libraries can't be transparent w.r.t. whether they invoke Haskell code or not (actually the hs_init() problem that someone else brought up recently also has this side-effect). Malcolm can comment on exactly how nhc98 handles the finalizers, but I believe that there's a list of pending finalizers maintained by the RTS which are run on re-entry to Haskell land. The equivalent would be to add a check in Hugs's eval loop, I imagine. > Table 2 (section 6): > > Is there a reason to be so coy about the concrete C types used for > HsChar, HsFloat, HsDouble and HsBool? > > Proposed change: > > C symbol Haskell Symbol Constraint on concrete C type > HsChar Char unsigned char HsChar is an unsigned 32-bit int in GHC. We probably ought to say that the type is unsigned, at least - and the Haskell 98 standard requires that it can at least contain all the Unicode character values (20 bits?). Cheers, Simon

3 2

RE: Minor tweaks to ffi addendum
by Simon Marlow 03 Jun '02

03 Jun '02

> > > It's going to be really hard to implement ForeignPtr as > specified. > > > The problem is that invocation of the cleanup function > is triggered > > > by the garbage collector. > > > > Malcolm can comment on exactly how nhc98 handles the > finalizers, but I > > believe that there's a list of pending finalizers > maintained by the RTS > > which are run on re-entry to Haskell land. > > That's the theory, yes. In practice, I've never been able to get it > to work correctly, so the only finalisers that are guaranteed to run > in nhc98's runtime system are ones written in C rather than Haskell. > The underlying problem is when do you run a Haskell finaliser in a > non-concurrent system? You can't run it when you discover it (in > the middle of a GC). When the GC is called, the mutator is probably > right in the middle of some reduction step. If I run the finalisers > immediately the GC finishes, then I'll probably corrupt the reduction > that was already in progress. So the pending finalisers need to be > held off until a "safe" moment, but I haven't yet been able to find a > way to determine when that is. > > In any case the most likely definition of a Haskell finaliser is just > a call into C-land anyway, so cut out the middle-man is what I say! Well, looking at the big picture I suppose having Haskell finalizers does imply a kind of pre-emptive concurrency, which might not otherwise be present in the system. Just by adding Haskell finalizers, assuming they can be implemented without concurrency, it is possible that IO code can be arbitrarily interleaved. This is a pretty big step to take for a single-threaded system, since it will likely impact other parts of the system (libraries have to be thread-safe, etc.). Hmm. Maybe we should have the pure C finalizer version of addForeignPtrFinalizer, with the version with a Haskell finalizer optional, available iff pre-emptive Concurrent Haskell is supported. We have to specify the constraint that the C finalizer can't directly or indirectly invoke any Haskell code (is this checkable at run time?). Alternatively, we could have an explicit operation runFinalizers :: IO () which causes finalizers to run in a non-pre-emptive or non-concurrent system. Cheers, Simon

2 2

Minor tweaks to ffi addendum
by Alastair Reid 03 Jun '02

03 Jun '02

While implementing the full FFI for Hugs, I'm coming across a bunch of little things that confuse me and some wee typos. Here's a list of what I've found so far. If no-one objects in the next few days, I'll go ahead and make minor changes to the report where required. section 4.1.1: The definition of impents suggests that there must/can be spaces around the keywords 'dynamic' and 'wrapper' and that the static form can have spaces at start and end. I'm going ahead and implementing this interpretation but want to remark that it seems a bit odd. No action proposed. section 4.1.3: It took me a while to understand the explanation of Dynamic import. After a while, I realized that I should read the type as: (FunPtr ft) -> ft instead of FunPtr (ft -> ft) The current text 'FunPtr ft -> ft' does mean the former but it would be easier to read if written '(FunPtr ft) -> ft'. Proposed change: write the type as '(FunPtr ft) -> ft' Incidentally, how am I to interpret the type rules? Which of the following have the right type: foreign import ccall "dynamic" mkFun1 :: FunPtr (Int -> IO ()) -> (Int -> IO ()) type Int2 = Int foreign import ccall "dynamic" mkFun2 :: FunPtr (Int2 -> IO ()) -> (Int -> IO ()) newtype Int3 = Int3 Int foreign import ccall "dynamic" mkFun3 :: FunPtr (Int3 -> IO ()) -> (Int -> IO ()) data Int4 = Int4 Int foreign import ccall "dynamic" mkFun4 :: FunPtr (Int4 -> IO ()) -> (Int -> IO ()) type T5 = FunPtr (Int -> IO ()) -> (Int -> IO ()) foreign import ccall "dynamic" mkFun5 :: T5 Proposed change: none at present section 5.5: It's going to be really hard to implement ForeignPtr as specified. The problem is that invocation of the cleanup function is triggered by the garbage collector. I don't want the garbage collector to be recursively invoked so I don't want the garbage collector to directly invoke Haskell function. The Hugs garbage collector is not supposed to be a mutator of the heap - so putting a simple wrapper round the garbage colector won't work. And there's no mechanism outside the GC to look for cleanup functions to execute. GHC gets round this by scheduling a (preemptive) thread to execute the cleanup code. How on earth does NHC get round this? Does anyone have a suggestion for how it might be implemented in Hugs? Proposed change: none at present but I'm deeply sceptical of a design which takes a simple task (invoke a C cleanup function) and, for no discernable reason, generalizes it to the point that it requires a much more complex runtime system. section 5.6: castStablePtrToPtr and castPtrToStablePtr can break typesafety so one might want to try to make sure that any program that uses them will be forced to have the word 'unsafe' in them. Proposed change: none at present Table 2 (section 6): Is there a reason to be so coy about the concrete C types used for HsChar, HsFloat, HsDouble and HsBool? Proposed change: C symbol Haskell Symbol Constraint on concrete C type HsChar Char unsigned char HsFloat Float float HsDouble Double double HsBool Bool int I see no reason at all not to specify HsBool. For HsChar, I'd like to know if it is signed or unsigned. I can imagine that the vague specification is to allow for wide characters but it's not clear to me that one could write much portable code without knowing that so it seems we should pick wide or narrow characters and stick with it. (Hmmm, wonder how much code it would take to make Hugs work with wide characters?) For HsFloat/Double, I imagine we're being vague because Hugs implements Double with single precision floats and GHC on the Alpha (used to) implement Float with double precision floats. I think Hugs can be fixed (in fact, was almost all done ages ago). That leaves the question of whether we want to force Haskell implementations to follow C conventions. I'm not sure. Table 3 (section 6): Proposed change: Rename column 1 as 'Preprocessor symbol'. In the last 2 lines, change column 1 to read HS_BOOL_FALSE and HS_BOOL_TRUE. (This change merely standardizes the existing GHC fix for this problem.) -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/

2 2