- FFI - Haskell.org

RE: cpp
by Simon Marlow 23 Feb '01

23 Feb '01

> > hsc2hs is in ghc-4.11. > > Did I miss the last 3 releases? I'm still stuck with > something between 4.08.1 > and 4.08.2 waiting for a debian package and freebsd port to > appear for 4.08.2. > :-) 4.11 is the current devel version, which will be released as 5.00. I submitted the 4.08.2 port to the FreeBSD folks just after the release, but they haven't committed it yet. Cheers, Simon

1 0

Summary of current change suggestions
by malcolm-ffi＠cs.york.ac.uk 23 Feb '01

23 Feb '01

Ok FFI folks, I think we're all agreed now on the desirability of tweaking the FFI to permit (a) arbitrary new foreign languages to be added; (b) the "foreignname" specifier to be (potentially) richer, depending on the particular foreign language in question; (c) abstraction of common header/linklib information to a single new sort of decl. For backwards compatibility, I suppose on (b) we are going to have to continue to allow double-string specifiers of the form foreign import "linklib" "cname" hname :: type for some time, although that form would be deprecated from now on? We know mostly what the richer form of the "foreignname" specifier will be for Java, and in any case we can leave that design to SPJ+EM. However, we have also heard one or two half-hearted suggestions to enrich the "foreignname" string for C as well. For instance, Manuel suggested "Gtk: gtk_new_window". Is anyone convinced that this is a good route for C, or should we settle instead on getting (c) right, leaving the "foreignname" untouched for now? As far as (a) is concerned, we have three decisions to make: (1) Do we use quotes around the language name or accept a simple varid? E.g. "java" vs java. As Marcin points out, the names of the languages C and C++ are not varids, so would need to be enclosed in quotes. On the other hand, see decision point (2) below. (2) The actual names for languages or calling conventions. Manuel suggests a change from naming the calling convention to naming the foreign language. This is somewhat attractive. However, I think many languages have a simple C-callable interface and we don't need to know anything more about the actual language. If this is true, I think we should continue to name the calling convention rather than the language. Hence, the names ccall and stdcall might be retained, java can be added. If anyone actually comes up with a C++ binding, a name like cplusplus (for instance) seems good enough to me. This also has the advantage of backwards compatibility, which incidentally would force decision point (1) above towards no-quotes. Incidentally, the intention would _not_ be that all compilers must support all foreign languages. As at present, only the ccall convention is mandatory. Any other named calling convention should be parsed, but if the compiler doesn't support it, a warning (at least) should be issued. In that situation, it is unclear whether the compiler can actually generate sensible code. (At the moment, I think the FFI standard says if your platform doesn't support stdcall, code should default to ccall? Obviously that doesn't scale well to java for instance!) (3) Where do we allow the language specifier? In my opinion, definitely in foreign import and export decls. This is at minimum for backwards compatibility, but looking forward to the day when someone writes FFI code for more than one language in the same module (eek!), it will be needed there. For the same reason, I would argue that it belongs in the new kind of decl required for (c). Now how do we tackle the biggest change, issue (c): abstraction of common header/linklib information to a single new sort of decl? Disregarding syntax for the moment, Marcin has suggested a concrete design that conflicts in some ways with backwards compatibility: * the primary specification is of the language not the calling convention * the calling convention is defined as only one attribute of a language * the calling convention is mandatorily removed from its current position in all functions His proposal also adds many of the features that we want: * names of include files * names of link libraries * decl appears once and scopes over all foreign functions in that module * it is not language specific: the attribute-based style allows new languages to have attributes we haven't even thought of yet. and some features that might be nice additions: * the global decl can be overridden for a single function * using an attribute style directly in the existing foreign import/export decls is potentially cleaner, as it can tidy up current blots like 'unsafe', and admits the possibility of needing further ad-hoc specifications in the future. (Didn't someone have an example that involved blocking/non-blocking thread-safety?) Since I believe backward compatibility is very important, would anyone like to either modify Marcin's proposal, or make a new concrete proposal that is perhaps slightly more conservative but still achieves our goals? I suppose backwards compatibility might not be entirely possible. If it isn't, then we have a choice between making the changes as small as possible (to make porting old code easier), or making them as radical as possible (to ensure that the design is really clean and extensible). Regards, Malcolm

4 11

RE: Summary of current change suggestions
by Simon Marlow 23 Feb '01

23 Feb '01

> > * specifying libraries in the source isn't the right way to > > go. Library names change independently of APIs, and can be > > platform-specific. > > I like the high level goal this is heading towards and was in complete > agreement with implementing it in rules like this until.... > I realised that > all that Hugs sees is the source (i.e., there's no makefile > to put extra > compilation info into) - so if it ain't in the source, where is it? > > But I like the rule... Well... there's no reason why hugs couldn't use something like GHC's packages. In fact, it could use the scheme largely unchanged; but perhaps the package spec should be extended to include a path to source files (the alternative is to overload the path to the interface files and put the sources in the same place if you want to share a package between Hugs & GHC). The package spec format is designed to be used with "read" (i.e. it's Haskell syntax), so Hugs could just read it using its built-in parser. Cheers, Simon

3 2

RE: Summary of current change suggestions
by Simon Marlow 22 Feb '01

22 Feb '01

> On Thu, 22 Feb 2001, Simon Marlow wrote: > > > The package spec format is designed to be used with "read" > (i.e. it's > > Haskell syntax), so Hugs could just read it using its > built-in parser. > > The format should be extensible, i.e. adding future fields should not > break old code which does not use these fields. IMHO each field which > is a list should be allowed to be skipped, defaulting to the > empty list. yep, good point. Simon

1 0

RE: Summary of current change suggestions
by Simon Peyton-Jones 22 Feb '01

22 Feb '01

I must say that I like Manuel's proposal below. I'm a bit alarmed about the proliferation of features in the FFI, but Manuel's story makes it seem more tractable. It's in the spirit of putting all the language-specific info needed for a particular foreign import into a single string. The FFI spec should have a section specifying the syntax of this string for a selection of languages. Incidentally, we should make the order of 'import/export' , 'dynamic', 'unsafe' keywords arbitrary. So you should be able to write foreign import unsafe dynamic ... or foreign import dynamic unsafe ... I'm not sure exactly which the unordered set should be, maybe the optional components. Simon | I am still against cpp, but maybe we can relegate defaults | to the tools domain in general (and whoever is fond of cpp, | can use cpp). Without defaults, it is still possible to use | all features of the FFI. If a user wants the convenience of | defaults, she has to use a tool (as with all the other | "features" that we considered too heavy for the core FFI). | | Then, we could use the following general form of foreign | imports: | | foreign import [callconv] [funname] ['unsafe'] varid :: prim_type | | where funname is calling-convention dependent and in the | case of ccall is either merely the name of the imported | symbol or an include specification followed by a colon and, | then, the imported symbol. So, we have | | foreign import ccall "gtk_window_new" | windowNew :: CInt -> IO (Ptr Window) | | or | | foreign import ccall "<gtk/gtk.h>:gtk_window_new" | windowNew :: CInt -> IO (Ptr Window) | | If the <.> are omitted in the include specification, it | corresponds to an #include"...".

2 1

RE: Modification to foreign import/export
by Simon Marlow 22 Feb '01

22 Feb '01

> Perhaps it should be integrated with proposed Malcolm's scheme of > structured module hierarchy. It's not obvious how ghc's packages fit > into this idea. I'm not sure either, but we haven't seen the proposal yet :) Simon

1 0

RE: Typing f.e.d.
by Simon Peyton-Jones 22 Feb '01

22 Feb '01

What became of this suggestion of Sven's? Did we adopt it? (While deprecating but not dropping Addr.) Simon | -----Original Message----- | From: Sven Panne [mailto:Sven.Panne@informatik.uni-muenchen.de] | Sent: 11 February 2001 16:01 | To: The Happy Bit Fiddlers | Subject: Typing f.e.d. | | | I have a small change request regarding foreign export dynamic. | Currently the FFI doc says: | | topdecl | : ... | .. | | 'foreign' 'export' [callconv] 'dynamic' varid :: | prim_type -> IO Addr | | GHC additionally allows: | | prim_type -> IO Ptr | | As usual the FFI "looks through" newtypes. But now that we have | FunPtr, the following typing makes much more sense: | | 'foreign' 'export' [callconv] 'dynamic' varid :: prim_type | -> IO (FunPtr prim_type) | | where both prim_types have to be the *same*. We should probably allow | the old Addr-typing as well for some time to facilitate the | transition, | but not the Ptr-typing (bleeding edge people will know what to do :-). | The corresponding changes to GHC look easy, so I'd like to commit this | if there are no objections. | | Furthermore, the FFI docs still talk about Addr only, not Ptr/FunPtr. | Now that Addr is deprecated, this should be changed, too. | | Cheers, | Sven | | _______________________________________________ | FFI mailing list | FFI(a)haskell.org | http://www.haskell.org/mailman/listinfo/ffi |

1 0

RE: Summary of current change suggestions
by Simon Marlow 22 Feb '01

22 Feb '01

I've kept pretty quiet on this issue before, because I didn't have a strong view. Now I'm concerned that some of the suggestions people are making would take the design in the wrong direction. * specifying libraries in the source isn't the right way to go. Library names change independently of APIs, and can be platform-specific. This should be a compiler-specific feature: eg. GHC uses package specifications to identify libraries. Addtionally allowing libraries to be specified in the source would just be a pain for us. * specifying include files in the source is fine; but I'm actually quite happy to leave this to an external tool like hsc2hs to sort out rather than adding extra syntax to the language, but that's just me. If we really must add new syntax, let's make it as general as possible to accomodate future design enhancements. I'd go with foreign decl cinclude "<foo/bar.h>" in other words, foreign decl VARID [ STRING ] Cheers, Simon

3 3

Modification to foreign import/export
by Manuel M. T. Chakravarty 22 Feb '01

22 Feb '01

Among other things, the FFI was one of the topics at the recent Haskell Implementor's Meeting. One specific issue that was discussed following a proposal for better Java support by SPJ and Erik Meijer was a change to foreign import/export declarations. For some languages (eg, Java) a simple function name is not sufficient to identify an external function/method (eg, because of overloading). So, it seems like a good idea to extend the string specifying the external function name to contain more information. This string should include all information that is necessary to identify the external entity uniquely. This has two consequences: (1) The interpretation of the string becomes dependent on the foreign language and (2) the optional specifier of a dll in the current proposal should be folded into this string. Re (1): We obviously need to identify the foreign language. This can be achieved by extending the ccall/stdcall keyword to generally specify the foreign language (eg, we might use "java"). After that change, I think, the keyword for C should be "c" rather than "ccall" (we are not writing "javacall" either). Not sure what a good alternative for stdcall would be...win? pascal? Re (2): In Java, we obviously want to use qualified names (like "java.lang.foo"), but in the case of dynamic libraries and C calls it is less clear what the syntax should look like. "gtk: gtk_new_window" to specify that the dynamic library "gtk" (on Unix that would happen by loading "libgtk.so") has to be loaded to call the function "gtk_new_window"? The latter raises two further issues: (a) When a compiler generating C code translates a module including C bindings, it should be given the name of the relevant header file, too. Generally, the name of the header and the dynamic library may be different. So, we must be able to specify two names. (b) Some of the language specific information (like the header file and the dynamic library) often is the same for all foreign declarations in a module. It is annoying to have to repeat it in every declaration and makes it more difficult to alter conditionally. Is it reasonable to require that only foreign declarations of a *single* foreign language are allowed per Haskell module? Would one single global declaration of the relevant information, then, be sufficient? And do we want to specify the foreign language used in this module in this global declaration, too? Summary of Open Points ~~~~~~~~~~~~~~~~~~~~~~ - Language specifiers: Other names for "ccall" (possibly, "c") and "stdcall". - Header files: Syntax for specifying a header in case of a foreign declaration for C. - Only one foreign language per Haskell module? - Global specification of parameters such as the foreign language, dynamic library to be loaded, and header file to be included? Opinions? Cheers, Manuel

4 11

Motivation for proposed changes to FFI
by malcolm-ffi＠cs.york.ac.uk 16 Feb '01

16 Feb '01

I can appreciate Alastair's reluctance to make changes to the FFI specification at this late stage. I would suggest that the rest of us really want it to stay stable and solid too. But at the recent implementers' meeting in Cambridge, it emerged that the current design has one or two very small flaws. So, because Alastair wasn't at that meeting where we discussed the possibility of some small changes, I thought it might be useful to rehearse some of the motivation here on the list. First, Simon PJ and Erik Meijer have been thinking about whether the current FFI story works adequately with Java as the foreign language rather than C. (Ghc now has a backend that compiles to Java rather than C or native code, so it makes sense to try to use foreign Java code directly.) The good news is that the FFI mostly goes through ok. The only points where they need to extend the mechanism are: foreign import [ccall|stdcall] "linklib" "foreignname" hname :: type * Neither of the calling conventions 'ccall' or 'stdcall' are appropriate for Java, so they suggested adding 'java' to the allowable set. (This raises the prospect of continually having to add new calling convention specifiers every time some compiler implementer gets a brainstorm for a new backend target language.) * The "foreignname" specification cannot be just the name of the function/method in Java. For one thing in Java, the same name can refer to lots of different methods distinguished by number and types of arguments. For another, Java uses namespaces. So the foreignname specifier ends up looking something like foreign import java "int Foo.Bar.g(bool)" g :: Bool -> IO Int But you will notice that this string encapsulates some of the information that was originally supposed to be in the "linklib" part of the specifier - namely "Foo.Bar". * The obvious corollary is that in fact a single string would be sufficient to specify both the "linklib" and "foreignname" specifiers in C as well. Indeed, for an arbitrary foreign language, all you need is to define the format of a single string, and the compiler implements a mini-parser to extract the information it needs (header / link-library / function name / namespace / whatever). Ok, so at this point we discussed what is currently used in the existing C foreign interface. It turns out that the current two-string specifier of "linklib" and "foreignname" is not the whole story. Sometimes (e.g. when compiling via C rather than native code) a compiler needs a prototype for the foreign function. Nhc98 simply generates a plausible prototype based on the Haskell type, but this is wrong in general because the compiler cannot always correctly guess the actual C prototype. For this reason, nhc98 has a non-standard hack foreign import noproto "foreignname" f :: type to fix those situations where it gets it wrong. Ghc on the other hand never tries to generate a prototype, but this means it needs its own non-standard hack, the compilation flag -#include <header.h> to get the correct prototype in almost all cases. Thus it seems that when the FFI is targetted at C, it actually needs three strings, not just two, for instance: foreign import "<header.h>" "linklib" "foreignname" f :: type So now we come back to the idea that a single string is actually sufficient, provided we know how to parse it for the foreign language in question. We played a little bit with deciding how to format the string for C, for instance: foreign import "<header.h>/linklib/foreignname" f :: type but the objection was raised that in a binding to a foreign library, a large part of the string would be identical in all the specifiers. Hence, finally, the proposal was that perhaps a single decl foreign decl [c|stdcall|java|...] "<header.h> / linklib.dll" could abstract all this information to a single place in the source file. (Note, we didn't even talk at the meeting about the possible syntax of such a decl - we have only started to discuss possibilities now on this list.) Does that help to motivate the current discussion a bit more Alastair? Regards, Malcolm

1 0