May 2001 - FFI - Haskell.org

FFI Definition
by Manuel M. T. Chakravarty 07 May '01

07 May '01

Folks, As promised, I have started on a definition of the Haskell FFI. I started by formalising those aspects of foreign exports and imports that we have recently discussed here. You can access the current text at http://www.cse.unsw.edu.au/~chak/haskell/ffi.ps.gz In addition to general comments, I would particularly welcome your opinion on the FIXME issues. You may note that I have tilted the specification of the calling convention and the related string determining the external entity very much into the direction of the execution platform. In my opinion that makes the whole description much more consistent. In particular, we are now talking about JVM calls rather than Java calls. This fits much better with ccall, wincall, and dotnet. It also makes it much more natural to talk about calling conventions rather than programming languages implemented in terms of these calling conventions. I will add more material (including also a specification of the FFI library) as we go. Cheers, Manuel

5 12

RE: Revised FFI syntax + typing
by Alastair Reid 07 May '01

07 May '01

> Interesting language, but can anyone remind me what these > #if etc are supposed to me - I can't find them in the > Haskell report. They're not? But I see code like that all over the place. > There is no point in trying to fix a problem that H98 already > introduced. The H98 Committee in their infinite wisdom chose to > have types with system-dependent sizes and I think, we should be as > orthogonal as possible whether we like it or not. But H98 doesn't have Word so I don't see a requirement to add it or to provide ffi support for it. -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/ ps As I understand it, underspecifying the size of Int in Haskell 1.0 was largely to allow compilers to use one or two bits as tags to distinguish boxed values (pointers) from unboxed values (ints, chars, etc.). In particular, this lets Haskell compilers generate Lisp code (since Lisp compilers often use that GC trick and lack the type information which would make it redundant). No current Haskell compilers use this trick (it's not very effective in lazy/polymorphic languages). Hugs comes close but, though it would be easier to provide ints with restricted range, it opts for providing a full 32 bits. (The decision to do this predates my involvement with Hugs.) The other reason for doing it is the hope of going a little bit faster on 64 bit machines. I'm not sure if there actually is a performance benefit or if it is significant. pps I'm assuming that UShort and the like have system-dependent sizes but I'm not objecting to them because it's just not possible to work with C without making the types match up - Int8 and friends don't go quite far enough towards speaking C's language. Pretty much every ffi interface I've ever written starts by figuring out the sizes of each of the major C types and selecting the corresponding explicitly sized Haskell type to represent it.

1 0

Re: FFI Definition
by Alastair Reid 07 May '01

07 May '01

> foreign export dynamic is a shortcut for importing an automatically > generated nontrivial C function, with argument type not conforming to > regular imports, which could not be written by hand. I don't get it. How do you use a function with a type like: (Int -> IO Int) -> Addr to import a C function? The closest I can get is to use foreign label to get an Addr (or FunPtr if preferred) but now I'm stuck. What's up? > So it's actually an *import*! It could be spelled as import with a > magical name - again all parameters are in the type. But it is also used for exporting so the name doesn't seem that inappropriate. Unless you're arguing that, since it brings a name into scope it can be thought of as a kind of import??? (But I don't think you are saying that.) -- Alastair Reid reid(a)cs.utah.edu http://www.cs.utah.edu/~reid/

2 1

RE: Revised FFI syntax + typing
by Simon Marlow 06 May '01

06 May '01

> -- allowed for import/export > callconv : 'ccall' -- default > | 'wincall' > | 'stdcall' -- deprecated, same as wincall > | 'cplusplus' > | 'jvm' > | 'dotnet' A totally minor point, but 'wincall' doesn't feel right. This alternative calling convention has been around since long before windows (it's always been the default calling convention for Pascal, I think). I'd stick with 'stdcall' because that's what everyone else seems to call it. gcc has a 'stdcall' function attribute, BTW. > word_ty : 'Word8' > | 'Word16' > | 'Word32' > | 'Word64' > | 'Word' -- GHC extension Why is Word a GHC extension? Someone remind me? Cheers, SImon

5 5

RE: Revised FFI syntax + typing
by Simon Marlow 06 May '01

06 May '01

> > Why is Word a GHC extension? Someone remind me? > > Hmmm, I thought that the Word type itself is a GHC extension, the > hslibs docs for module Word only talk about the explictly sized > variants. OTOH the implementation exports Word, but for legacy > reasons, IIRC. If this is really the case, we should document this > somehow in the sources. I don't think I have any strong feelings either way. Word has a pleasing symmetry with Int, but on the other hand if it isn't useful then we should omit it. I agree in principle with Alastair's portability argument, but don't think it is worth applying for this particular case since we already have Int. Cheers, Simon

3 2

RE: FFI Definition
by Simon Peyton-Jones 02 May '01

02 May '01

| > language specific stuff inside the "..." string | > language independent stuff outside | | But static/dynamic probably means different things, depending | on the callconv. I think it's arguable that static/dynamic should be inside the ext_ent string. Indeed, one might use static/dynamic for ccall, and virtual/non-virtual/static for Java, etc. "Baking in" static/dynamic for all languages may be inappropriate. Nevertheless, you propose keeping 'mode' outside the ext_ent string. Why? (Apart from backward compat.) Also should 'label' be there? Doesn't make sense for Java, does it? Simon

1 0

RE: FFI Definition
by Simon Peyton-Jones 02 May '01

02 May '01

| * I'm not very happy with the fact that static/dynamic are | only viewed | as modifiers, like unsafe/safe. They almost completely change the | meaning of a foreign declaration, including the typing rules. In my | view, we have *5* different language constructs: `foreign import', | `foreign import dynamic', `foreign export', `foreign | export dynamic', | and `foreign label'. | * How will we differentiate between virtual vs. non-virtual | methods in | the cplusplus calling convention? What about class methods vs. | instance methods? Will these distinctions have influence on the | typing rules? If yes, coding this into the extent would be wrong. | | * The JVM has *4* different instructions for method invocation | (invokeinterface, invokespecial, invokestatic, and | invokevirtual), so | our static/dynamic distinction seems a bit inflexible here. And the | introduction of the "new" prefix looks like a hack, which doesn't | even handle all situations. These are good questions. I think it's inevitable that the form of type of the foreign thing is going to depend on per-language details. For C we have call call indirect label For Java we have 4 different invocations new For C++ we have call virt call non-virt new Etc. Each of these implies a particular form for the type. I think we have to bite the bullet and put this stuff in the language specific string, including our current static/dynamic thing. The compiler has to be able to parse this string in order to generate an appropriate call, so it's no big deal for this parse info to feed into the type checker too. This is moving in exactly the opposite of what Sven suggests, but I don't see any alternative. My criterion is language specific stuff inside the "..." string language independent stuff outside Simon

2 1

Revised FFI syntax + typing
by Sven Panne 02 May '01

02 May '01

OK, after SPJ's mail and a few beers I'm now convinced that typing the foreign construct is not as simple and orthogonal as I initially hoped. After a close look into GHC's internals, H/Direct and Hugs I propose the following in the hope that it's backwards compatible and not too confusing: ------------------------------------------------------------------------ Syntax for the 'foreign' language construct ------------------------------------------------------------------------ topdecl : 'foreign' fkind modifier* var '::' type -- even this could be merged into modifier somehow, but... :-P fkind : 'import' | 'export' | 'label' modifier : mode | callconv | safety | ext_ent -- allowed for import/export mode : 'static' -- default | 'dynamic' -- allowed for import/export callconv : 'ccall' -- default | 'wincall' | 'stdcall' -- deprecated, same as wincall | 'cplusplus' | 'jvm' | 'dotnet' -- allowed for import safety : 'safe' -- default | 'unsafe' -- allowed for static import/export and label ext_ent : string -- default: the string rep. of var | string string -- deprecated, first string ignored ------------------------------------------------------------------------ Types related to the FFI ------------------------------------------------------------------------ Note that data type renamings are completely "transparent", just like type synonyms. prim_type_i : ['IO'] prim_result_i | prim_arg_i '->' prim_type_i prim_arg_i : prim_arg | foreign_ty | unsafe_arr_ty -- GHC extension | unlifted_prim_ty -- GHC extension prim_result_i : prim_result | ['IO'] unlifted_prim_ty -- GHC extension prim_type : ['IO'] prim_result | prim_arg '->' prim_type prim_result : prim_arg | '()' prim_arg : int_ty | word_ty | float_ty | ptr_ty | 'Bool' | 'Char' | 'StablePtr' a int_ty : 'Int' | 'Int8' | 'Int16' | 'Int32' | 'Int64' word_ty : 'Word8' | 'Word16' | 'Word32' | 'Word64' | 'Word' -- GHC extension float_ty : 'Float' | 'Double' ptr_ty : 'Addr' -- deprecated | 'Ptr' prim_result_i | 'FunPtr' prim_type_i -- arguments of this type are kept alive automagically across FFI calls. foreign_ty : 'ForeignObj' -- deprecated | 'ForeignPtr' prim_result -- GHC extension, only in conjunction with 'unsafe' modifier unsafe_arr_ty : 'ByteArray' a | 'MutableByteArray' i s a -- GHC extension unlifted_prim_ty : <any unlifted primary type> ------------------------------------------------------------------------ Typing rules for the ccall/wincall/stdcall calling convention ------------------------------------------------------------------------ Note that safety and ext_ent can be ignored for typing purposes, and that prim_type{,_i}1 and prime_type{,_i}2 must be equal. foreign import static :: prim_type_i foreign import dynamic :: 'FunPtr' prim_type_i1 '->' prim_type_i2 foreign export static :: prim_type foreign export dynamic :: prim_type1 -> 'IO' ('FunPtr' prim_type2) foreign label :: ptr_ty Cheers, Sven

1 0

RE: ForeignObj and ForeignPtr
by Simon Marlow 01 May '01

01 May '01

> In the course of trying to find out GHC's status quo of typing foreign > declarations and scribbling together the forthcoming typing rules, I > stumbled over ForeignObj/ForeignPtr. There was a lively > discussion about > this some time ago IIRC, but I'm unsure about a few things: > > * ForeignObj has been deprecated for a long time, but we > should still > keep it in the FFI for a transition time and pass the associated > Addr when calling out, correct? Yes, at least another major release or two. > * Does it make sense to allow ForeignObj/ForeignPtr as a > prim_result? > I think it doesn't, because the non-Haskell side can't > know if and > when the finaliser has been run. That's right, ForeignPtrs exist inside the Haskell world only. When a ForeignPtr is passed outside, it is passed as a plain address, and an address cannot be converted back to a ForeignPtr for returning to Haskell. > * What did we conclude about the lifetime rules? Should > ForeignObj/ForeignPtr > arguments be handled specially by the FFI or is > {touch,with}Foreign{Obj,Ptr} > required? I'd opt for the latter, but the current FFI > docs seem to imply > a special handling. :-P I'd prefer to keep the "special" handling. Otherwise, we might as well disallow ForeignPtrs as arguments to foreign import, because you'd end up writing withForeignObj f $ \ p -> c_fun p all the time. Much simpler to make this automatic, and in some sense the "special" behaviour is really the expected behaviour (or rather, the alternative behaviour is simply not useful). > On the other hand, I can't see anything > special in the code generated by GHC. Hmmm... That's because it got lost after some changes to the pre-code-generation passes in GHC 5.00. It's on my list of things to fix. Cheers, Simon

1 0

ForeignObj and ForeignPtr
by Sven Panne 01 May '01

01 May '01

In the course of trying to find out GHC's status quo of typing foreign declarations and scribbling together the forthcoming typing rules, I stumbled over ForeignObj/ForeignPtr. There was a lively discussion about this some time ago IIRC, but I'm unsure about a few things: * ForeignObj has been deprecated for a long time, but we should still keep it in the FFI for a transition time and pass the associated Addr when calling out, correct? * Does it make sense to allow ForeignObj/ForeignPtr as a prim_result? I think it doesn't, because the non-Haskell side can't know if and when the finaliser has been run. * What did we conclude about the lifetime rules? Should ForeignObj/ForeignPtr arguments be handled specially by the FFI or is {touch,with}Foreign{Obj,Ptr} required? I'd opt for the latter, but the current FFI docs seem to imply a special handling. :-P On the other hand, I can't see anything special in the code generated by GHC. Hmmm... Cheers, Sven

1 0