- FFI - Haskell.org

RE: efficiency
by Simon Marlow 04 Dec '00

04 Dec '00

> I don't want to talk about [Char] -> [CChar] specifically, but the > mention of efficiency reminds me of something that has been bothering > me for a little while. > > In the current design of the FFI, it is assumed that most of the > marshalling work is going to be done in Haskell. For example, to > return more than one result from a C function, the results > are packaged > as a C struct, passed as an Addr (or Ptr) to Haskell, which then > dereferences from the Addr to each individual component of the struct. > > Now in GHC, the treatment of the IO monad may make this acceptably > fast, but in other implementations (like nhc98 (of course)), this > is just horribly inefficient, and can slow up FFI calls by an order > of magnitude or more. (Comparing to previous implementations using > either the builtin 'primitive' mechanism or GreenCard.) I agree that > it is better in principle to have the marshalling code in Haskell > because it has much better type-safety than C. But in practice, > I have found other ways to do the marshalling, mainly because of the > efficiency question. > > Do other people have any thoughts about the efficiency of the current > scheme? The one thing that concerns me is the overhead of the malloc/free for allocating the memory into which we marshal structs. This and the overhead of installing an exception handler to free the memory in case of an error. GHC has ByteArrays which are more efficient (no explicit free, no need for an exception handler), but they have the problem that they can move around and therefore can only be passed to unsafe foreign calls. However, I'd really like a way to integrate them into the current FFI scheme. An alternative is to write our own memory allocator - we can't use thread stacks, because they tend to move around too, but we could use a static area of memory allocated in a stack-like fashion. This won't be a huge win over malloc/free though. I'm not too worried about the IO monad overhead, which is allegedly zero in GHC and in fact speeds things up by virtue of being strict. Perhaps the right thing to do in nhc98 is to make the IO monad more efficient? Cheers, Simon

2 1

RE: errno again
by Simon Marlow 02 Dec '00

02 Dec '00

> Yes, I am persuaded that hard-coded datatypes tend to be bad in > this situation. The way hsc2hs deals with constants is quite good, > and it is certainly a better, more flexible, approach. However, I do > have some hesitation in relying on yet another pre-processing tool, > with yet another "little language" outside of Haskell itself. > > Of course hsc2hs does more than just filling in constants, but for > just that one task, I wonder if we should be extending the primitive > FFI mechanism itself. For instance, we already have "foreign label", > so what about perhaps "foreign value": > > foreign value "F_GETFL" fGetFL :: CInt > > This is just off the top of my head, so I'm probably missing some > subtleties. What do people think of this idea? Are there some > real killer situations which would make it impossible? Uh-oh! The return of litlits! But seriously, this approach pollutes the language-independence of the low level FFI, unless you add an extra language descriptor to the declaration. It's also awkward to implement unless you compile via C - you couldn't implement this using a dumb preprocessor, for example. The preprocessor approach seems to be the lesser of several evils. Cheers, Simon

2 1

RE: qforeign-0.62
by Simon Peyton-Jones 02 Dec '00

02 Dec '00

| So, how about the following? Instead of the | | newtype Window = Window ForeignObj | | way of wrapping foreign ADTs that we used so far, should we | use | | newtype WindowTag = WindowTag () | type Window = ForeignObj WindowTag A fine idea. I accept what you say about ADTs, but if it's really so, we woudn't need the objectionable > withForeignObj :: ForeignObj -> (Ptr a -> IO b) -> IO b because all ForeignObj-manipulating things would be foreign imported foreign import f :: Window -> IO () Anyway, it strikes me as good to tag each ForeignObj with an indication of what it points to, just so that one does not mix up ForeignObjs that should not be mixed. But it's just a suggestion. You guys who are deeply into the FFI should decide. It would be nice to get to closure on this FFI stuff -- it's been going on a long time! Simon

3 2

RE: qforeign-0.62
by Simon Peyton-Jones 01 Dec '00

01 Dec '00

Dear foreigners Simon and I noticed this morning that ForeignObj should really be parameterised. The current type of newForeignObj is newForeignObj :: Ptr a -> IO () -> IO ForeignObj This immediately loses the type information on the Ptr! Shouldn't it be newForeignObj :: Ptr a -> IO () -> IO (ForeignObj a) Or maybe we should shorten it to (FPtr a)? ("F" for "finalised".) Similarly withFPtr :: FPtr a -> (Ptr a -> IO b) -> IO b It's bizarre to have withForeignObj :: ForeignObj -> (Ptr a -> IO b) -> IO b What think you? Simon

5 6

RE: errno
by Simon Marlow 30 Nov '00

30 Nov '00

> > As for the errno issues, I think the hsc2hs solution is simple and > > straightforward and doesn't require an > architecture-dependent algebraic > > datatype. BTW Maloclm, if you haven't yet taken a look at > hsc2hs I urge > > you to do so (fptools/ghc/utils/hsc2hs): it should work > unmodified with > > nhc98, and it'll let you write C constants like EINTR directly in > > portable Haskell code. > > I could imagine such a solution being quite nice. Is there any > documentation for hsc2hs? I don't know what its input should look > like. There's some documentation in fptools/ghc/docs/users_guide/utils.sgml (except it's called hs2c there; I didn't get around to updating the name). I can't connect to haskell.org at the moment so I'm not sure if the docs there are up to date enough to have this stuff. Marcin's QForeign library has plenty of examples too. Cheers, Simon

3 4

RE: qforeign-0.62
by Simon Marlow 29 Nov '00

29 Nov '00

> I'm with Manuel. Module FFI/Foreign is already stable, and I object > to adding things to it arbitrarily. By all means make a new module > with these functions, which I can see could be useful. But please > don't just allow "feature creep" over already-agreed specifications. Perhaps those functions which are implemented on top of the Storable primitives should be in another module; I don't really have any preferences. But if we move some of the derived functions out of Storable, they should probably all be moved. > Once these newer features are settled, I look forward to seeing a > portable implementation. Yes, that would be nice. In fact I think most of these features are trivially implemented on top of Storable. As for the errno issues, I think the hsc2hs solution is simple and straightforward and doesn't require an architecture-dependent algebraic datatype. BTW Maloclm, if you haven't yet taken a look at hsc2hs I urge you to do so (fptools/ghc/utils/hsc2hs): it should work unmodified with nhc98, and it'll let you write C constants like EINTR directly in portable Haskell code. Cheers, Simon

2 1

question about debugging haskell/C using ffi
by Axel Krauth 09 Nov '00

09 Nov '00

Hi! at the moment I'm writing a haskell/C programm using the FFI. But I'm getting always a segfault in the haskell part (according to gdb) . With Linux the segfault happens at random (while I'm doing a lot of calls to haskell via a dynamically exported function). With Unix gdb says it occurs in 'StgReturn' called from 'stg_returnToStackTop' after the C part finishes it job. I already put a C frontend instead of the haskell part and searched for access violations and other memory missmanagement in the C part (none). I also disabled (with no effect) all finalizers of the ForeignObj's used. As I'm relatively new to Haskell I don't know how to hunt that one down. Unfortunately the programm is already huge (ca. 4000 LOC Haskell & C) so the buggy part of the programm is not obvious (for me). Could someone give me a hint/technique how to approach the problem? bye, Axel the system I used are Linux : Suse 7.0, ghc4.08.1 gcc 2.95.2 Unix : Solaris 2.6, ghc4.08 gcc 2.95.2 and ghc4.08.1 egcs1.1.2

2 1

RE: References to values and weak references.
by Simon Marlow 09 Nov '00

09 Nov '00

> I have just come up against an application where I would like > the user to be > able (if required) to link in a file containing a set of > default options to the > application. So what I would IDEALLY like is > (1) the ability for FFI to give me access to an external > value as well as an external > function, as in > foreign import "default_arguments" defaultArguments :: Addr We have this. It's called "foreign label", but it's in an obscure part of the documentation - you're not the first person to miss it :) > (2) the ability for FFI to link in values as weak external > references. (In other words > the reference doesn't have to be resolved and you can > find out if it has been.) > So assuming (1) has been implemented, I might want > something additionally like > foreign import weak "default_arguments" > getDefaultArguments :: IO (Maybe Addr) > Now while I would LIKE these features, they aren't there, and > it won't cause me great > distress to work around their absence. But why not implement > them anyway? Do you know how to use existing support for weak references in the linker/assembler? I believe GNU as/ld has them, but I can't find any reference in the docs. Alternatively we'd have to implement this using something like dlsym, which only looks for references in shared libraries. Cheers, Simon

3 2

References to values and weak references.
by George Russell 08 Nov '00

08 Nov '00

I have just come up against an application where I would like the user to be able (if required) to link in a file containing a set of default options to the application. So what I would IDEALLY like is (1) the ability for FFI to give me access to an external value as well as an external function, as in foreign import "default_arguments" defaultArguments :: Addr (2) the ability for FFI to link in values as weak external references. (In other words the reference doesn't have to be resolved and you can find out if it has been.) So assuming (1) has been implemented, I might want something additionally like foreign import weak "default_arguments" getDefaultArguments :: IO (Maybe Addr) Now while I would LIKE these features, they aren't there, and it won't cause me great distress to work around their absence. But why not implement them anyway?

1 0

RE: question about debugging haskell/C using ffi
by Simon Marlow 08 Nov '00

08 Nov '00

> at the moment I'm writing a haskell/C programm using the FFI. But I'm > getting always a segfault in the haskell part (according to gdb) . Unfortunately these really are tricky to track down. We don't have good debugging tools for Haskell code or the RTS at the moment. First thing to check is whether changing the heap size has any effect; if so, it's probably a GC-related problem of some sort(*). You can make a start by compiling up a debugging RTS, which enables lots of assertions and debugging code. Check in config.mk for the two lines you need to add to build.mk. Then, with your program linked against the debugging RTS, run it with +RTS -D128 to enable GC sanity checking. This may or may not cause an assertion failure (unfortunately it has some false positives at the moment). To actually debug the Haskell code, the only way is to run gdb as if it was debugging an assembly program - use si instead of s etc. If you can get a repeatable crash then we'll happily take a look for you - this will be much easier if you can package it up with a Makefile so that we can just "unpack & go". Cheers, Simon (*) you really want the answer to this question to be "no" :-)

1 0