February 2001 - FFI - Haskell.org

RE: Summary of current change suggestions
by Simon Peyton-Jones 22 Feb '01

22 Feb '01

I must say that I like Manuel's proposal below. I'm a bit alarmed about the proliferation of features in the FFI, but Manuel's story makes it seem more tractable. It's in the spirit of putting all the language-specific info needed for a particular foreign import into a single string. The FFI spec should have a section specifying the syntax of this string for a selection of languages. Incidentally, we should make the order of 'import/export' , 'dynamic', 'unsafe' keywords arbitrary. So you should be able to write foreign import unsafe dynamic ... or foreign import dynamic unsafe ... I'm not sure exactly which the unordered set should be, maybe the optional components. Simon | I am still against cpp, but maybe we can relegate defaults | to the tools domain in general (and whoever is fond of cpp, | can use cpp). Without defaults, it is still possible to use | all features of the FFI. If a user wants the convenience of | defaults, she has to use a tool (as with all the other | "features" that we considered too heavy for the core FFI). | | Then, we could use the following general form of foreign | imports: | | foreign import [callconv] [funname] ['unsafe'] varid :: prim_type | | where funname is calling-convention dependent and in the | case of ccall is either merely the name of the imported | symbol or an include specification followed by a colon and, | then, the imported symbol. So, we have | | foreign import ccall "gtk_window_new" | windowNew :: CInt -> IO (Ptr Window) | | or | | foreign import ccall "<gtk/gtk.h>:gtk_window_new" | windowNew :: CInt -> IO (Ptr Window) | | If the <.> are omitted in the include specification, it | corresponds to an #include"...".

2 1

RE: Modification to foreign import/export
by Simon Marlow 22 Feb '01

22 Feb '01

> Perhaps it should be integrated with proposed Malcolm's scheme of > structured module hierarchy. It's not obvious how ghc's packages fit > into this idea. I'm not sure either, but we haven't seen the proposal yet :) Simon

1 0

RE: Typing f.e.d.
by Simon Peyton-Jones 22 Feb '01

22 Feb '01

What became of this suggestion of Sven's? Did we adopt it? (While deprecating but not dropping Addr.) Simon | -----Original Message----- | From: Sven Panne [mailto:Sven.Panne@informatik.uni-muenchen.de] | Sent: 11 February 2001 16:01 | To: The Happy Bit Fiddlers | Subject: Typing f.e.d. | | | I have a small change request regarding foreign export dynamic. | Currently the FFI doc says: | | topdecl | : ... | .. | | 'foreign' 'export' [callconv] 'dynamic' varid :: | prim_type -> IO Addr | | GHC additionally allows: | | prim_type -> IO Ptr | | As usual the FFI "looks through" newtypes. But now that we have | FunPtr, the following typing makes much more sense: | | 'foreign' 'export' [callconv] 'dynamic' varid :: prim_type | -> IO (FunPtr prim_type) | | where both prim_types have to be the *same*. We should probably allow | the old Addr-typing as well for some time to facilitate the | transition, | but not the Ptr-typing (bleeding edge people will know what to do :-). | The corresponding changes to GHC look easy, so I'd like to commit this | if there are no objections. | | Furthermore, the FFI docs still talk about Addr only, not Ptr/FunPtr. | Now that Addr is deprecated, this should be changed, too. | | Cheers, | Sven | | _______________________________________________ | FFI mailing list | FFI(a)haskell.org | http://www.haskell.org/mailman/listinfo/ffi |

1 0

RE: Summary of current change suggestions
by Simon Marlow 22 Feb '01

22 Feb '01

I've kept pretty quiet on this issue before, because I didn't have a strong view. Now I'm concerned that some of the suggestions people are making would take the design in the wrong direction. * specifying libraries in the source isn't the right way to go. Library names change independently of APIs, and can be platform-specific. This should be a compiler-specific feature: eg. GHC uses package specifications to identify libraries. Addtionally allowing libraries to be specified in the source would just be a pain for us. * specifying include files in the source is fine; but I'm actually quite happy to leave this to an external tool like hsc2hs to sort out rather than adding extra syntax to the language, but that's just me. If we really must add new syntax, let's make it as general as possible to accomodate future design enhancements. I'd go with foreign decl cinclude "<foo/bar.h>" in other words, foreign decl VARID [ STRING ] Cheers, Simon

3 3

Modification to foreign import/export
by Manuel M. T. Chakravarty 22 Feb '01

22 Feb '01

Among other things, the FFI was one of the topics at the recent Haskell Implementor's Meeting. One specific issue that was discussed following a proposal for better Java support by SPJ and Erik Meijer was a change to foreign import/export declarations. For some languages (eg, Java) a simple function name is not sufficient to identify an external function/method (eg, because of overloading). So, it seems like a good idea to extend the string specifying the external function name to contain more information. This string should include all information that is necessary to identify the external entity uniquely. This has two consequences: (1) The interpretation of the string becomes dependent on the foreign language and (2) the optional specifier of a dll in the current proposal should be folded into this string. Re (1): We obviously need to identify the foreign language. This can be achieved by extending the ccall/stdcall keyword to generally specify the foreign language (eg, we might use "java"). After that change, I think, the keyword for C should be "c" rather than "ccall" (we are not writing "javacall" either). Not sure what a good alternative for stdcall would be...win? pascal? Re (2): In Java, we obviously want to use qualified names (like "java.lang.foo"), but in the case of dynamic libraries and C calls it is less clear what the syntax should look like. "gtk: gtk_new_window" to specify that the dynamic library "gtk" (on Unix that would happen by loading "libgtk.so") has to be loaded to call the function "gtk_new_window"? The latter raises two further issues: (a) When a compiler generating C code translates a module including C bindings, it should be given the name of the relevant header file, too. Generally, the name of the header and the dynamic library may be different. So, we must be able to specify two names. (b) Some of the language specific information (like the header file and the dynamic library) often is the same for all foreign declarations in a module. It is annoying to have to repeat it in every declaration and makes it more difficult to alter conditionally. Is it reasonable to require that only foreign declarations of a *single* foreign language are allowed per Haskell module? Would one single global declaration of the relevant information, then, be sufficient? And do we want to specify the foreign language used in this module in this global declaration, too? Summary of Open Points ~~~~~~~~~~~~~~~~~~~~~~ - Language specifiers: Other names for "ccall" (possibly, "c") and "stdcall". - Header files: Syntax for specifying a header in case of a foreign declaration for C. - Only one foreign language per Haskell module? - Global specification of parameters such as the foreign language, dynamic library to be loaded, and header file to be included? Opinions? Cheers, Manuel

4 11

Motivation for proposed changes to FFI
by malcolm-ffi＠cs.york.ac.uk 16 Feb '01

16 Feb '01

I can appreciate Alastair's reluctance to make changes to the FFI specification at this late stage. I would suggest that the rest of us really want it to stay stable and solid too. But at the recent implementers' meeting in Cambridge, it emerged that the current design has one or two very small flaws. So, because Alastair wasn't at that meeting where we discussed the possibility of some small changes, I thought it might be useful to rehearse some of the motivation here on the list. First, Simon PJ and Erik Meijer have been thinking about whether the current FFI story works adequately with Java as the foreign language rather than C. (Ghc now has a backend that compiles to Java rather than C or native code, so it makes sense to try to use foreign Java code directly.) The good news is that the FFI mostly goes through ok. The only points where they need to extend the mechanism are: foreign import [ccall|stdcall] "linklib" "foreignname" hname :: type * Neither of the calling conventions 'ccall' or 'stdcall' are appropriate for Java, so they suggested adding 'java' to the allowable set. (This raises the prospect of continually having to add new calling convention specifiers every time some compiler implementer gets a brainstorm for a new backend target language.) * The "foreignname" specification cannot be just the name of the function/method in Java. For one thing in Java, the same name can refer to lots of different methods distinguished by number and types of arguments. For another, Java uses namespaces. So the foreignname specifier ends up looking something like foreign import java "int Foo.Bar.g(bool)" g :: Bool -> IO Int But you will notice that this string encapsulates some of the information that was originally supposed to be in the "linklib" part of the specifier - namely "Foo.Bar". * The obvious corollary is that in fact a single string would be sufficient to specify both the "linklib" and "foreignname" specifiers in C as well. Indeed, for an arbitrary foreign language, all you need is to define the format of a single string, and the compiler implements a mini-parser to extract the information it needs (header / link-library / function name / namespace / whatever). Ok, so at this point we discussed what is currently used in the existing C foreign interface. It turns out that the current two-string specifier of "linklib" and "foreignname" is not the whole story. Sometimes (e.g. when compiling via C rather than native code) a compiler needs a prototype for the foreign function. Nhc98 simply generates a plausible prototype based on the Haskell type, but this is wrong in general because the compiler cannot always correctly guess the actual C prototype. For this reason, nhc98 has a non-standard hack foreign import noproto "foreignname" f :: type to fix those situations where it gets it wrong. Ghc on the other hand never tries to generate a prototype, but this means it needs its own non-standard hack, the compilation flag -#include <header.h> to get the correct prototype in almost all cases. Thus it seems that when the FFI is targetted at C, it actually needs three strings, not just two, for instance: foreign import "<header.h>" "linklib" "foreignname" f :: type So now we come back to the idea that a single string is actually sufficient, provided we know how to parse it for the foreign language in question. We played a little bit with deciding how to format the string for C, for instance: foreign import "<header.h>/linklib/foreignname" f :: type but the objection was raised that in a binding to a foreign library, a large part of the string would be identical in all the specifiers. Hence, finally, the proposal was that perhaps a single decl foreign decl [c|stdcall|java|...] "<header.h> / linklib.dll" could abstract all this information to a single place in the source file. (Note, we didn't even talk at the meeting about the possible syntax of such a decl - we have only started to discuss possibilities now on this list.) Does that help to motivate the current discussion a bit more Alastair? Regards, Malcolm

1 0

RE: Typing f.e.d.
by Simon Peyton-Jones 14 Feb '01

14 Feb '01

This looks plausible to me too --- but I agree with Alastair that we should not remove support for Addr, including in f.e.d. Backward compatibility is a pain, but lack of is a fast way to lose customers. Simon | -----Original Message----- | From: Sven Panne [mailto:Sven.Panne@informatik.uni-muenchen.de] | Sent: 11 February 2001 16:01 | To: The Happy Bit Fiddlers | Subject: Typing f.e.d. | | | I have a small change request regarding foreign export dynamic. | Currently the FFI doc says: | | topdecl | : ... | .. | | 'foreign' 'export' [callconv] 'dynamic' varid :: | prim_type -> IO Addr | | GHC additionally allows: | | prim_type -> IO Ptr | | As usual the FFI "looks through" newtypes. But now that we have | FunPtr, the following typing makes much more sense: | | 'foreign' 'export' [callconv] 'dynamic' varid :: prim_type | -> IO (FunPtr prim_type) | | where both prim_types have to be the *same*. We should probably allow | the old Addr-typing as well for some time to facilitate the | transition, | but not the Ptr-typing (bleeding edge people will know what to do :-). | The corresponding changes to GHC look easy, so I'd like to commit this | if there are no objections. | | Furthermore, the FFI docs still talk about Addr only, not Ptr/FunPtr. | Now that Addr is deprecated, this should be changed, too. | | Cheers, | Sven | | _______________________________________________ | FFI mailing list | FFI(a)haskell.org | http://www.haskell.org/mailman/listinfo/ffi |

1 0

RE: Typing f.e.d.
by Simon Marlow 13 Feb '01

13 Feb '01

> What libraries is the evolving design being tried out on (I > know about your > OpenGL library). In GHC we have implementations of Directory, Time and CPUTime using the new FFI and hsc2hs. The new implementations should work on Hugs with only minor tweaks, given suitable implementations of the FFI libraries. I'm hoping at some point to write a portable implementation of IO using the new FFI. > 1) As a user, I only _have_ to use the low level ffi > libraries: Addr, Word, > Int64, ForeignObj, etc. Its nice if the high level > libraries are useful > too but not essential. > > 2) As an ffi implementer/maintainer, I only have to implement > the agreed > ffi interface and all the libraries on top will "just work". > > 3) They're not supposed to be standard so it's ok to > experiment with the > design a bit. The libraries *are* part of the standard. The FFI language extensions are just one part of the propsed FFI standard; the low and high-level marshalling libraries form the other part. Cheers, Simon

1 0

RE: Typing f.e.d.
by Simon Marlow 12 Feb '01

12 Feb '01

> As usual the FFI "looks through" newtypes. But now that we have > FunPtr, the following typing makes much more sense: > > 'foreign' 'export' [callconv] 'dynamic' varid :: prim_type > -> IO (FunPtr prim_type) > > where both prim_types have to be the *same*. We should probably allow > the old Addr-typing as well for some time to facilitate the > transition, > but not the Ptr-typing (bleeding edge people will know what to do :-). > The corresponding changes to GHC look easy, so I'd like to commit this > if there are no objections. Sounds reasonable to me. Simon

1 0

ANN: C->Haskell 0.8.1
by Manuel M. T. Chakravarty 11 Feb '01

11 Feb '01

I am pleased to announce the availability of version 0.8.1 of the interface generator C->Haskell. It works with the current stable release series 4.08.x of GHC as well as the current development series 4.11. For both versions of GHC, it supports the *same* FFI library that GHC natively only supports in the development version 4.11 and which constitutes the result of the work of the FFI Task Force over the last couple of months. The interface specification of the library is online available at http://www.cse.unsw.edu.au/~chak/haskell/c2hs/docu/c2hs-4.html For more information on C->Haskell and for downloading, see http://www.cse.unsw.edu.au/~chak/haskell/c2hs/ The main feature in this release is the new FFI library. It should allow users of the GHC stable series to use the new FFI library interface, which we hope to keep stable from now on. An update of the C->Haskell tool proper is being worked at. Happy Hacking, Manuel

1 0