RE: Modified proposal for default decls
| I am reasonably convinced by the need for an extensible | attribute-style specification such as Marcin has been proposing. I've been talking about the FFI stuff with Simon and Julian. We aren't (yet, anyway) convinced by the need for anything nearly as elaborate as what is now proposed. Most notably, Malcolm's latest proposal introduces a new form of abstraction (named thing), a bundle of attributes like Gtk or Bzip. Another environment for the compiler to manage! Soon people will want to export these things and import them elsewhere! This way lies madness. What is particularly tanatlising is that to a first approximation none of this is necessary. For example, the native code generator is quite happy without any of the new stuff now proposed. I think we can usefully distinguish three kinds of thing: a) What is necessary to uniquely identify the thing you want to call. In C this is just a single identifier; in Java, it's a fully qualified method name, plus the method type. Other languages doubtless lie between b) What is necessary to allow the compiler to typecheck the foreign import/export. In part, we seek #includes so that in the via-C route the C compiler will compare the call with the prototype, and complain if they don't match. But this is an implementation thing. In principle, the Haskell compiler could consult type info and compare. The important thing is that there's enough info to get from the thing-being-called to the type-of-the-thing-being-called. c) What is necessary for particular compilers, compiling in a particular way, to generate the right code. In particular, GHC compiling via C. GHC can't generate prototypes, because they might be #included. GHC can't omit prototypes unless prototypes are guaranteed to be #included. Hence the motivation to specify #include stuff. Notice that a) is essential b) is desirable, but not implemented at all c) is essential, but extremely language and compiler specific Therefore we propose that a) be part of the FFI standard b) be achievable, if desired, via (a) c) be not part of the FFI standard In particular, we propose to stick to a mild variant of the current FFI story foreign import ccall "gtk:foo" foo :: Int -> IO Int * "ccall" is one of a small number of language specifiers, "java" and ".net" "stdcall" being other possibilities. C is strange in having more than one calling convention; we treat "stdcall" as a second "language". * The "gtk:foo" is a string in language-specific format that uniquely specifies the thing being called. It'll be quite different for Java, when it'll contain the type signature of the method and whether it's a "new" constructor etc. * The "gtk:" part is a concession to (b). It specifies a C package from which this procedure comes. There is then some compiler-specific mechanism for mapping the name of a C package to the location of its header files and .o file. * If the Haskell compiler wants to typecheck the import, it is free to consult the header file, meta data or whatever, based on the foreign package name and the name of the thing being called. No "foreign library", no "foreign default"... You get to repeat everything at every foreign import. Notice that cross-module exports of inlinings are now straightforward: the foreign call carries its package name with it. You may think this is too minimalist, but there are big advantage to simple, minimal designs, even if they can't cope with every single case. Do we really need the elaboration of Malcolm's proposal? Simon
Most notably, Malcolm's latest proposal introduces a new form of abstraction (named thing), a bundle of attributes like Gtk or Bzip. Another environment for the compiler to manage! Soon people will want to export these things and import them elsewhere! This way lies madness.
Ok, so it was a bit of a wild idea. I quite like it, but I recognise that it is kind of elaborate and may be a step too far at the moment. I'm not so much bothered about whether we have a complicated 'foreign default' thing or not. In fact, I'm only really personally interested in being able to specify header files correctly - the very thing you propose to exclude!
What is particularly tanatlising is that to a first approximation none of this is necessary. For example, the native code generator is quite happy without any of the new stuff now proposed.
So, because one system is clean and clever and doesn't need all this junk, all the other systems should simply suffer for their ignorant poverty? :-)
c) What is necessary for particular compilers, compiling in a particular way, to generate the right code.
Therefore we propose that (c) be not part of the FFI standard.
In that case, I'd like you to be completely consistent and remove 'unsafe' from the FFI spec. nhc98 and Hugs don't need it, so other compilers shouldn't need it either! Make it a compile-time option to GHC, but please don't allow it in the source code. It is a compiler-specific option intended *only* to help GHC generate the right code, and doesn't belong in the standard spec. Well, I'm not serious, but you have to admit that it _is_ the logical conclusion of your approach.
Notice that c) is essential, but extremely language and compiler specific
I disagree - while it is certainly essential, there is no need for it to be compiler specific. That was the whole point of having this discussion - to establish a standard mechanism.
* The "gtk:" part is a concession to (b). It specifies a C package from which this procedure comes. There is then some compiler-specific mechanism for mapping the name of a C package to the location of its header files and .o file.
How does this idea of mapping from a "package name" like "gtk:" to a bunch of header and object files differ from the suggestion that we be explicit in the source code about that mapping with a 'foreign library' decl? You say it _should_ be compiler-specific: but why? That will really cause grief to users who are trying to write portable code. And besides, do any compilers currently have _any_ mechanism to map "gtk:" onto for instance #include <glib/glib.h> #include <gtk/gtk.h> #include <gtk/gtk_extra.h> ? No? I thought not. So why not let's try to develop a common mechanism that will work with all compilers?
You may think this is too minimalist, but there are big advantage to simple, minimal designs, even if they can't cope with every single case.
My worry is that it cannot cope well with the common case. Don't get me wrong - I like minimalism. But my feeling is that punting the 'difficult' stuff off into non-standard compiler-specific extensions is the opposite of minimal: users who want portability will end up needing to learn three different mechanisms, not one. Regards, Malcolm
On 26-Feb-2001, malcolm-ffi@cs.york.ac.uk <malcolm-ffi@cs.york.ac.uk> wrote:
What is particularly tanatlising is that to a first approximation none of this is necessary. For example, the native code generator is quite happy without any of the new stuff now proposed.
So, because one system is clean and clever and doesn't need all this junk, all the other systems should simply suffer for their ignorant poverty? :-)
The reason that the system in question (ghc's native code compiler) doesn't need all that junk is not that it is clever, rather it is that it only supports an impoverished interface. The Mercury native code generator does use the C header file names, for typechecking C imports and for interfacing with C macros. Likewise the Mercury .NET code generator's C++ interface does the same. Typechecking C imports and interfacing with C macros are both very useful features, and if ghc wasn't so lame ;-), it would support those features, and then it would need the C header file names too. <grinning, ducking, and running for cover> ;-)
c) What is necessary for particular compilers, compiling in a particular way, to generate the right code.
Therefore we propose that (c) be not part of the FFI standard.
In that case, I'd like you to be completely consistent and remove 'unsafe' from the FFI spec. nhc98 and Hugs don't need it, so other compilers shouldn't need it either!
The Mercury compiler has an attribute `will_not_call_mercury', which I believe is analagous to ghc's `unsafe' attribute. So ghc is not entirely alone here. (Furthermore, AFAIK these two instances evolved independently, rather than either being influenced by the other, which makes the case for such an attribute stronger.) -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
On 26-Feb-2001, Simon Peyton-Jones <simonpj@microsoft.com> wrote:
a) What is necessary to uniquely identify the thing you want to call. In C this is just a single identifier; in Java, it's a fully qualified method name, plus the method type. Other languages doubtless lie between
b) What is necessary to allow the compiler to typecheck the foreign import/export. In part, we seek #includes so that in the via-C route the C compiler will compare the call with the prototype, and complain if they don't match. But this is an implementation thing. In principle, the Haskell compiler could consult type info and compare.
That's true, in principle. However, the vast majority of C compilers don't store the necessary type info. On the other hand, all C compilers support header files. Using header files and generating C wrappers for imported functions is the only portable way of doing type checking of foreign import for C, I'm pretty sure.
The important thing is that there's enough info to get from the thing-being-called to the type-of-the-thing-being-called.
c) What is necessary for particular compilers, compiling in a particular way, to generate the right code. In particular, GHC compiling via C. GHC can't generate prototypes, because they might be #included. GHC can't omit prototypes unless prototypes are guaranteed to be #included. Hence the motivation to specify #include stuff.
Many C interfaces contain a significant number of macros. And C is one of the most widely used languages that we want to interface with. So it seems to me that any FFI which aims to provide *good* support for C ought to provide a way of accessing C macros. To handle macros, you need to know which header files are involved, since the same macro can have different definitions in different header files. And, since the definitions of the macros may well contain compiler-specific C extensions, I'm pretty sure that the only portable way of handling those is to generate a C function wrapper that calls to macro, and compiler that with the C compiler on it.
Notice that a) is essential b) is desirable, but not implemented at all
Well, it is implemented in the Mercury compiler; that's not a Haskell implementation, of course, but I don't think the differences between Mercury and Haskell have much significance to the design or implementation of this aspect of the FFI. It's a good idea to avoid standardization on something before it has been implemented, in case it turns out to be unimplementable, or to have significant problems that remained undiscovered until it was implemented. But given that we've been using this approach with C header files in the Mercury compiler for a long time, I think that is unlikely in this case. So for any Haskell implementation that wants to be able to provide convenient interfacing with C macros and to type-check C imports, and wants to be able to do this with arbitrary C compilers, the Haskell implementation will need to know which headers need to be included. You're right that Haskell implementations which only interfaces with a specific C implementation (or a specific set of them) can do things differently. But I'm not sure that is the right thing to focus on.
In particular, we propose to stick to a mild variant of the current FFI story
foreign import ccall "gtk:foo" foo :: Int -> IO Int [...] * The "gtk:foo" is a string in language-specific format that uniquely specifies the thing being called. [...] * The "gtk:" part is a concession to (b). It specifies a C package from which this procedure comes. There is then some compiler-specific mechanism for mapping the name of a C package to the location of its header files and .o file.
I could probably live with that. -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Many C interfaces contain a significant number of macros. And C is one of the most widely used languages that we want to interface with. So it seems to me that any FFI which aims to provide *good* support for C ought to provide a way of accessing C macros.
Since an explicit goal of the original ffi design was that it would not require a C compiler to implement it and since at least one implementation (STG-Hugs) doesn't require a C compiler, I'd like to see a lot of discussion before we add support for macros to the ffi. Alastair ps
However, the vast majority of C compilers don't store the necessary type info.
Since it's non-portable, it has no real bearing on this discussion but, nevertheless, people on this list might like to try compiling a C file with -g and then using 'objdump --debugging' to view the debugging information it contains. Besides variable and function types, it lists lots of typedefs and fields in struct declarations are annotated with the bit size and offset of the field.
On 26-Feb-2001, Alastair Reid <reid@cs.utah.edu> wrote:
Many C interfaces contain a significant number of macros. And C is one of the most widely used languages that we want to interface with. So it seems to me that any FFI which aims to provide *good* support for C ought to provide a way of accessing C macros.
Since an explicit goal of the original ffi design was that it would not require a C compiler to implement it and since at least one implementation (STG-Hugs) doesn't require a C compiler, I'd like to see a lot of discussion before we add support for macros to the ffi.
Given the current state of affairs, I think it would be reasonable to only require implementations to support interfacing with C functions, and to not require implementations to support interfacing directly with C macros, variables, enumeration constants, etc. After all, you can only standardize on the lowest common denominator. But I think it would be very nice to standardize on a syntax for doing that, even if supporting that syntax is optional rather than a requirement. I think that in the long run most implementations will want to support it. I also think that it would be nice to standardize on the syntax used to tell the compiler which include files are needed. -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Many C interfaces contain a significant number of macros. And C is one of the most widely used languages that we want to interface with. So it seems to me that any FFI which aims to provide *good* support for C ought to provide a way of accessing C macros.
[Sorry for replying twice to the same paragraph...] I just remembered that my project (Knit - a C component language) has severe problems with use of macros in header files. We are planning to solve these problems by writing a little tool that generates C function wrappers for macros. The idea is that you'd list the functions you want generated, their C type and the name of the corresponding macro and the tool would generate functions to call them and a header file to #include in place of the original file. In a way, this is like an ffi from C to CPP :-) Obviously this won't handle every possible macro but we hope it will handle enough of the common cases to be useful. The most obvious omission will be macros (like stderr) which look like variables - gcc provides a few tricks which might handle some of these problems. My project is planning to write this tool but would be more than happy if someone else did it first :-) -- Alastair
Mon, 26 Feb 2001 11:17:07 -0700, Alastair Reid <reid@cs.utah.edu> pisze:
I just remembered that my project (Knit - a C component language) has severe problems with use of macros in header files.
With hsc2hs it's easy enough to wrap macros: foreign import "hs_incRef" unsafe incRef :: Object -> IO () #def inline void hs_incRef (PyObject *o) {Py_INCREF (o);} If you are lucky, everything will be inlined across languages and modules. -- __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK
Mon, 26 Feb 2001 07:07:55 -0800, Simon Peyton-Jones <simonpj@microsoft.com> pisze:
Most notably, Malcolm's latest proposal introduces a new form of abstraction (named thing), a bundle of attributes like Gtk or Bzip. Another environment for the compiler to manage! Soon people will want to export these things and import them elsewhere!
Yes, we will! :-) What's wrong with it? Yet another name instead of 'foreign default': foreign module I am specifying some details about a foreign module I want to interface to.
* The "gtk:" part is a concession to (b). It specifies a C package from which this procedure comes. There is then some compiler-specific mechanism for mapping the name of a C package to the location of its header files and .o file.
This gives no advantage over the current scheme. I already can specify header names in a compiler-specific way, and I still can't do it compiler-independently, and the compiler still won't push it automatically to modules which inline functions infected with C calls.
Notice that cross-module exports of inlinings are now straightforward: the foreign call carries its package name with it.
How the package is defined?
You may think this is too minimalist,
I would say: too incomplete.
Do we really need the elaboration of Malcolm's proposal?
Yes, unless a simpler design provides what is needed. -- __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZASTÊPCZA QRCZAK
participants (5)
-
Alastair Reid -
Fergus Henderson -
malcolm-ffi＠cs.york.ac.uk -
qrczak＠knm.org.pl -
Simon Peyton-Jones