Extension to the FFI to allow macro expansions to be avoided
[This has come from the cvs-ghc list. All the info should be in this mail, though]. The problem =========== Some C libraries (curses is the one I have to hand) provide macros in addition to functions in some cases for efficiency reasons. If you use the FFI to import such a function thus: ----- module Q where import Foreign.Ptr (Ptr) import Foreign.C.Types (CInt) foreign import ccall unsafe "static curses.h wstandend" wstandend :: Ptr a -> IO CInt ----- Then GHC fails if not using the NCG (which AFAIK is available on only handful of platforms; the generated code efficiency may also be an issue if you don't have enough foreign imports to warrant splitting them off into their own module): $ ghc -fffi -c Q.hs -O /tmp/ghc1164.hc: In function `Q_zdwccall_entry': /tmp/ghc1164.hc:29: warning: dereferencing `void *' pointer /tmp/ghc1164.hc:29: error: request for member `_attrs' in something not a structure or union $ as it is creating code like this (line 29 is the one containing "wstandend"): { I_ _ccall_result; StgAddr _ccall_arg1=_B0_; CALLER_SAVE_SYSTEM _ccall_result = (wstandend((_ccall_arg1))); CALLER_RESTORE_SYSTEM _B3_=_ccall_result; } which then gets expanded to this by cpp: { I_ _ccall_result; StgAddr _ccall_arg1=_B0_; _ccall_result = (((((_ccall_arg1))->_attrs = (0L)))); _B3_=_ccall_result; } (there was some disagreement over whether such a macro was poorly written for not casting its argument; either way, the fact remains that this sort of thing is out there, and convincing the rest of the world to change to make things a bit easier for us is not going to happen.). Possible solutions ================== 1 = One solution is to write a C wrapper around each potentially affected function. I don't think this is very satisfactory as it significantly increases the amount of effort needed to create a binding (rather than just a single foreign import statement you now also need a line in a C file and to make sure the object file is built and linked in correctly). 2 = Surrounding the function with parentheses so the macro doesn't match. The offending line above would be _ccall_result = ((wstandend)((_ccall_arg1))); 3 = I haven't tested this, but I assume you could just generate #undef wstandend before the function call. When to fix the problem ======================= (I'm assuming solution 1 is rejected from here on - if it's the one chosen then the rest is irrelevant) The above gives ways to fix the problem, but there were worries that it could cause a significant performance penalty to always do this, including in some of the bits of GHC implementation. Two possible alternatives come to mind. A = Where a cid is permitted also permit '(' cid ')' B = Before the optional '&' in a C impent, allow an optional "nomacro". The implementation would then make sure you didn't get a macro using solution 2, 3 or some other means. ===== Both A and B allow the programmer to highlight where they cannot trust macros to behave as needed (by ghc -fvia-C at least), so the performance hit is taken only when we can't be sure it is safe not to. ===== In case it isn't clear, I don't like solution 1 (which is what we have at the minute), but would be happy with any combination of {1,2} and {A,B}. Thanks Ian
Ian Lynagh writes:
Some C libraries (curses is the one I have to hand) provide macros in addition to functions in some cases for efficiency reasons. If you use the FFI to import such a function thus:
I was a bit confused by what Ian meant by this so, in case it helps others, here's an extract from curses.h. extern NCURSES_EXPORT(int) wstandend (WINDOW *); /* generated */ #define wstandend(win) (wattrset(win,A_NORMAL)) There seem to be a substantial number of these: $ grep generated /usr/include/curses.h | wc -l 117 As far as I can see, there is no conditional compilation to let you turn the macros off so you would have to write 117 individual #undefs, 117 stub functions or whatever. Now that I understand the problem, my feeling is that the problem is not with curses but with GHC's compilation method. GHC is using a shortcut by pretending that the ffi is for interfacing to cpp+C whereas the ffi is quite clear that it is for interfacing to C. So, I think the thing to do is fix GHC. For that reason I vote for:
Surrounding the function with parentheses so the macro doesn't match. The offending line above would be _ccall_result = ((wstandend)((_ccall_arg1)));
where the decision to insert parens is decided by a command-line flag. (Ideally, this flag would affect foreign imports in that file but not any obtained from imported modules.) I like this approach because it means that we don't have to strip the parens off when using the other compilation techniques such as GHC's NCG or Hugs generating stub functions for everything. Alternatively add an appropriate set of #undefs for this particular file. The following comes close to doing the job. grep generated /usr/include/curses.h | perl -pe 's/^.*\)\s+(\w+).*/#undef \1/' Stick the result in a header file like this: curses_wrapper.h: #include "curses.h" #ifdef __GHC__ #undef macro1 ... #undef macro117 #endif and use curses_wrapper.h instead of curses.h in the Haskell code. -- Alastair Reid
Alastair Reid wrote:
[...] where the decision to insert parens is decided by a command-line flag. [...]
I'm still not convinced that any change is really necessary, but I'd really prefer a command-line flag over a change in the FFI syntax. The latter will lead to much more portability trouble than the former. Cheers, S.
This is exactly the problem I ran into when using the FFI to interface with curses. I just created a my_curses.h with appropriate #undefs in it. John -- John Meacham - ⑆repetae.net⑆john⑈
On Thu, Apr 08, 2004 at 10:18:24AM +0100, Alastair Reid wrote:
Now that I understand the problem, my feeling is that the problem is not with curses but with GHC's compilation method. GHC is using a shortcut by pretending that the ffi is for interfacing to cpp+C whereas the ffi is quite clear that it is for interfacing to C. So, I think the thing to do is fix GHC.
And Hugs too. The issue isn't extending the FFI but implementing it more accurately and consistently. As you point out, systems compiling via C have been extending the FFI to a function+macro interface, which is incompatible with systems compiling to native code. Having been bitten by the same thing in the opposite direction (macros that work with ffihugs or ghc -fvia-C don't work with ghc -fasm), I'd favour turning off the macro interface, preferably with #undef, at least by default.
And Hugs too. The issue isn't extending the FFI but implementing it more accurately and consistently. As you point out, systems compiling via C have been extending the FFI to a function+macro interface, which is incompatible with systems compiling to native code. Having been bitten by the same thing in the opposite direction (macros that work with ffihugs or ghc -fvia-C don't work with ghc -fasm), I'd favour turning off the macro interface, preferably with #undef, at least by default.
Maybe the code generated by Hugs should be something like: #ifdef MACROS_ARE_COOL #undef malloc #endif ? (I'm not likely to implement this anytime soon but if someone has the energy to do so, go right ahead.) From what I know of NHC, it probably has the same extension and would benefit from the same cleanup. -- Alastair
On 13-Apr-2004, Ross Paterson <ross@soi.city.ac.uk> wrote:
And Hugs too. The issue isn't extending the FFI but implementing it more accurately and consistently. As you point out, systems compiling via C have been extending the FFI to a function+macro interface, which is incompatible with systems compiling to native code. Having been bitten by the same thing in the opposite direction (macros that work with ffihugs or ghc -fvia-C don't work with ghc -fasm), I'd favour turning off the macro interface, preferably with #undef, at least by default.
I'd rather fix GHC so that interfacing with C macros works with ghc -fasm. The equivalent works fine with the Mercury implementation ("mmc --target asm" generates assembler for the Mercury code and C stubs for the FFI glue), so it's clearly possible. -- Fergus Henderson | "I have always known that the pursuit | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Fergus Henderson wrote:
I'd rather fix GHC so that interfacing with C macros works with ghc -fasm. The equivalent works fine with the Mercury implementation ("mmc --target asm" generates assembler for the Mercury code and C stubs for the FFI glue), so it's clearly possible.
How does Mercury handle something like the "broken" macros we discussed recently in its different back-ends? Cheers, S.
On 15-Apr-2004, Sven Panne <Sven.Panne@aedion.de> wrote:
Fergus Henderson wrote:
I'd rather fix GHC so that interfacing with C macros works with ghc -fasm. The equivalent works fine with the Mercury implementation ("mmc --target asm" generates assembler for the Mercury code and C stubs for the FFI glue), so it's clearly possible.
How does Mercury handle something like the "broken" macros we discussed recently in its different back-ends?
The macros, of course, are not broken; it the handling of them by Hugs/GHC and/or the Haskell FFI that is broken :) In Mercury, it would look like this: :- module ffi_example. :- interface. :- import_module io. :- type window. :- pred wstandend(window::in, io::di, io::uo) is det. :- implementation. :- pragma foreign_decl("C", "#include <curses.h>"). :- pragma foreign_type("C", window, "WINDOW *"). :- pragma import(wstandend(in,di,uo), [will_not_call_mercury], "wstandend"). When using the "--target asm" option, which tells the Mercury compiler to compile directly to assembler, the Mercury compiler will generate some C glue code to handle the FFI interfacing pragmas. The glue code calls the wstandend() macro with a variable of type "WINDOW *" as its argument: #include "mercury.h" #include <curses.h> void ffi_example__wstandend_3_p_0(void *ffi_example__Arg1_1) { { WINDOW * Arg1; MR_MAYBE_UNBOX_FOREIGN_TYPE(WINDOW *, ffi_example__Arg1_1, Arg1); wstandend(Arg1); } The macro MR_MAYBE_UNBOX_FOREIGN_TYPE converts from "void *" to "WINDOW *". If sizeof(void *) == sizeof(WINDOW *), as is the case on most architectures, the macro will end up just doing an assignment; but if the C type is one that doesn't fit in a void *, it will get boxed when passed from C to Mercury and unboxed when passed from Mercury to C. This generated C glue code will be compiled with a C compiler. Then for any Mercury code which calls the "wstandend" Mercury procedure, we will generate assembler code which calls the C function in our generated glue code. For example, if you have :- pred foo(window::in, io::di, io::uo) is det. foo(W) --> wstandend(W). then the Mercury compiler will generate the following assembly code: .globl ffi_example__foo_3_p_0 .type ffi_example__foo_3_p_0,@function ffi_example__foo_3_p_0: jmp ffi_example__wstandend_3_p_0 .Lfe5: .size ffi_example__foo_3_p_0,.Lfe5-ffi_example__foo_3_p_0 -- Fergus J. Henderson | "I have always known that the pursuit Galois Connections, Inc. | of excellence is a lethal habit" Phone: +1 503 626 6616 | -- the last words of T. S. Garp.
W liście z czw, 15-04-2004, godz. 18:14 -0700, Fergus Henderson napisał:
When using the "--target asm" option, which tells the Mercury compiler to compile directly to assembler, the Mercury compiler will generate some C glue code to handle the FFI interfacing pragmas.
Which - in case it's a plain function - is an unnecessary extra function call. Perhaps we should differentiate by a calling convention between linking to a function directly and generating a source-level C call? The calling conventions we know would correspond to linking to them directly. They would perform the necessary magic to turn off macros when compiling via C, and they would not need a C compiler when compiling via assembler. There is no need for specifying a C header in this case. Another calling convention would generate a function wrapper with proper argument types. It would need "foreign type" declarations and included headers. The wrapped calling convention in this case is of course irrelevant, so it's a calling convention rather than a separate flag. -- __("< Marcin Kowalczyk \__/ qrczak@knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/
I really like that the Haskell FFI is defined at the ABI level, and that it doesn't need to generate its own stub funtions or whatnot. I especially like that it doesn't require having a C compiler installed. it is a foreign function interface that supports the C calling convention. If you want a C aware version, that seems like what 'hsc2hs' is for (as well as some of the more heavyweight stuff out there). then you can do (#type WINDOW) and (#const KEY_LEFT) and such things in your haskell code and take the preprocessor and C constructs into account. John -- John Meacham - ⑆repetae.net⑆john⑈
On 08-Apr-2004, Alastair Reid <alastair@reid-consulting-uk.ltd.uk> wrote:
Now that I understand the problem, my feeling is that the problem is not with curses but with GHC's compilation method. GHC is using a shortcut by pretending that the ffi is for interfacing to cpp+C whereas the ffi is quite clear that it is for interfacing to C. So, I think the thing to do is fix GHC.
I think this is a mischaracterization of the problem. Talking about "cpp+C" is a misnomer; the C preprocessor is a part of C, and has been so since long before Haskell even existed. The Haskell FFI is designed for interfacing with foreign languages at the ABI level, rather than the API level. While I understand the reasons for that decision, any interface with C which can only interface at the ABI level rather than the C API level is a second-class C interface; the Haskell FFI has sacrificed quality for easy of implementation. If this starts causing problems for users, I think the right solution is to raise the bar for implementations, not to lower it. GHC should be applauded for providing users with an interface at the C API level, and we should encourage other Haskell implementations to do the same, even if this goes above and beyond the requirements standardized in the Haskell FFI report. -- Fergus Henderson | "I have always known that the pursuit | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Fergus Henderson wrote:
[...] the Haskell FFI has sacrificed quality for easy of implementation. If this starts causing problems for users, I think the right solution is to raise the bar for implementations, not to lower it. [...]
I definitely support Fergus' point of view, GHC and Hugs really rushed head over heels towards a wrong direction, IMHO. If I see this correctly, the only problems were that GHC's native code generator doesn't handle macros currently and the handling of "broken" macros in the C back-end and Hugs. I think we can safely forget about the latter (by using #undef or a small wrapper), and we should have a look how to solve the former. Mercury seems to handle this, although I'm currently not sure at what price (in terms of implementation effort and possible performance loss). Cheers, S. P.S. to Malcolm: I did not mention nhc98, because I don't know how its back-end works, not because of an evil master plan against nhc98... :-)
Sven Panne <Sven.Panne@aedion.de> writes:
Fergus Henderson wrote:
[...] the Haskell FFI has sacrificed quality for easy of implementation. If this starts causing problems for users, I think the right solution is to raise the bar for implementations, not to lower it. [...]
P.S. to Malcolm: I did not mention nhc98, because I don't know how its back-end works, not because of an evil master plan against nhc98... :-)
S'okay. nhc98's default backend always compiles via C, and therefore FFI declarations that use macros will work in much the expected way (I think). Regards, Malcolm
On 08-Apr-2004, Ian Lynagh <igloo@earth.li> wrote:
The problem ===========
Some C libraries (curses is the one I have to hand) provide macros in addition to functions in some cases for efficiency reasons.
That is a feature, not a problem. And it is not just for efficiency reasons. Macros, enums, and other entities which exist only at the C API level rather than the ABI level are a fundamental part of the C language which is used in many C APIs. Any C interface which does not provide good support for interfacing with these entitites is IMHO not a good C interface.
If you use the FFI to import such a function thus:
----- module Q where
import Foreign.Ptr (Ptr) import Foreign.C.Types (CInt)
foreign import ccall unsafe "static curses.h wstandend" wstandend :: Ptr a -> IO CInt -----
Then GHC fails if not using the NCG
The documented type of wstandend() is int wstandend(WINDOW *win); not int wstandend(void *win); So if you declare this with the Haskell type "Ptr a -> IO CInt", in general the Haskell implementation doesn't have enough information about the argument type to know how to call the function. On any C implementation which uses name mangling or the equivalent, e.g. the GNU Portable.NET C compiler that compiles to the .NET CLR, the argument types will have a real effect on the ABI. And as you have observed, the arguments types also have a real effect on the C API.
(there was some disagreement over whether such a macro was poorly written for not casting its argument;
I would strongly disagree with anyone who said that the macro should cast its argument; that could hide type errors. This macro is an ordinary, reasonably-written C macro, and a good FFI should support interfacing with it. I the right way to solve this is to support some way for the Haskell user to tell the Haskell implementation what the right type is, for example perhaps something along the lines of data WINDOW foreign import ccall unsafe "static curses.h wstandend" wstandend :: Ptr WINDOW -> IO CInt or perhaps the above plus a new declaration to associate the Haskell name "WINDOW" with the C type name "WINDOW": foreign type "curses.h WINDOW" WINDOW Then ghc could generate C code which uses "WINDOW *" rather than "void *", and there would be no need to use parentheses or #undef to stop the macro being invoked. -- Fergus Henderson | "I have always known that the pursuit | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Fergus Henderson wrote:
[...] I the right way to solve this is to support some way for the Haskell user to tell the Haskell implementation what the right type is, for example perhaps something along the lines of
data WINDOW
foreign import ccall unsafe "static curses.h wstandend" wstandend :: Ptr WINDOW -> IO CInt
or perhaps the above plus a new declaration to associate the Haskell name "WINDOW" with the C type name "WINDOW":
foreign type "curses.h WINDOW" WINDOW
Then ghc could generate C code which uses "WINDOW *" rather than "void *", and there would be no need to use parentheses or #undef to stop the macro being invoked.
"foreign type" looks like something really useful and it came to my mind several times in the past, too. But the pressure of having it was never high enough to spend a non-trivial amount of brain-cycles, we're all lazy programmers... :-] While we're at it: Being able to specify "const" is high on my wish list for some years now. And if we don't go the whole way here, I'd even be happy with a simple way to specify a "const blah*" argument/return value. Cheers, S.
participants (9)
-
Alastair Reid -
Fergus Henderson -
Fergus Henderson -
Ian Lynagh -
John Meacham -
Malcolm Wallace -
Marcin 'Qrczak' Kowalczyk -
Ross Paterson -
Sven Panne