ANN: H98 FFI Addendum 1.0, Release Candidate 10
I have incorporated the feedback to RC 9 into RC 10, which you can get, as always, from http://www.cse.unsw.edu.au/~chak/haskell/ffi/ I also tracked the addition of `FinalizerPtr' in fptools. If there are no further suggestions, I will post RC 10 on the main list for final (hopefully!) review. Cheers, Manuel -=- Changes since RC 9 * 1: Mentioning interaction with foreign threads as an open problem. * 2 & 3: Removed `threadsafe' again, as the proposal for thread support is still evolving and it is not yet clear whether a new safety level is required. * 5.5: Added the type synonym `FinalizerPtr' and rewrite the documentation of finalizers. * 5.6: Clarified the description of `StablePtr' * 5.8: Added `finalizerFree' * 6.2: All the types in CTypes must be newtypes that are exported abstractly.
Minor nits: there's a footnote saying "Finalizers in Haskell cannot be savely [sic] realised without requiring support for pre-emptive concurrency". I'd suggest dropping "pre-emptive": with cooperative concurrency it's perfectly safe to collect the finalizers in GC and fork them next time you're back in the IO monad (though it's not very timely).
Ross Paterson <ross@soi.city.ac.uk> wrote,
Minor nits: there's a footnote saying "Finalizers in Haskell cannot be savely [sic] realised without requiring support for pre-emptive concurrency". I'd suggest dropping "pre-emptive": with cooperative concurrency it's perfectly safe to collect the finalizers in GC and fork them next time you're back in the IO monad (though it's not very timely).
True. Thanks, Manuel
Manuel M T Chakravarty <chak@cse.unsw.edu.au> writes:
-=- Changes since RC 9
* 6.2: All the types in CTypes must be newtypes that are exported abstractly.
This change makes things highly inconvenient in nhc98. A newtype can only be passed across the FFI in nhc98 if we have enough information to determine the original type that is being renamed. If Foreign.C.Types exports them abstractly, then currently we do not have that information, and so these types cannot be used at all! I am not suggesting that the newtype constructors should necessarily be part of the API seen by the user, but the problem comes down to a known deficiency in nhc98's interface file conventions. Regards, Malcolm
This was discussed here before and there seemed to be some support for it, but how about adding the empty data declaration extension to the FFI spec? They are used quite often to represent the abstract targets of pointers as in: data Foo data Bar foreign lookupBar :: Ptr Foo -> IO (Ptr Bar) -- psuedocode the FFI is the perfect place to use the extension and it is trivial to implement. The problem is made even worse by the fact that you can't use the beginning underscore trick with data constructors since an underscore counts as a lowercase letter so there is no way to avoid unused constructor warning messages without resorting to using extensions or resorting to compiler-specific pragmas which are unelegant. so I guess my argument is in a nutshell, everyone implements them. they are a very good fit with the FFI and they make code more readable and supress spurious warnings. John -- --------------------------------------------------------------------------- John Meacham - California Institute of Technology, Alum. - john@foo.net ---------------------------------------------------------------------------
On Saturday 31 May 2003 10:11 pm, John Meacham wrote:
This was discussed here before and there seemed to be some support for it, but how about adding the empty data declaration extension to the FFI spec?
I strongly agree that we should definitely add the ability to declare types whose definition is provided externally. (i.e., provide the feature that empty datatype decls currently provide.) Before adding them, we need to agree on the semantics and syntax (in that order I think). The obvious semantics based on the syntax is: [[ The declaration data T introduces a type T whose only value is bottom. ]] This semantics is obviously flawed though because it would suggest that any two values of type T are equal (and equal to bottom) and that optimizations based on that equality are valid. Using an unpointed type (i.e., the value is not bottom) or saying there are no values bottom or otherwise don't help. The correct semantics has to be something roughly like: [[ The declaration data T declares a type T whose set of values are defined externally to the language. [optional sentence: There are no legal Haskell operations on values of type T.] ]] If we go with a semantics like that, different syntax suggests itself like: external data T or, better, foreign import data T I'm not especially keen to change the syntax (especially since the existing syntax is so trivial to implement) but if we're going to add this to the language, we should make sure the syntax and semantics are tolerably clear and in agreement with each other. -- Alastair Reid
Dnia nie 1. czerwca 2003 23:55, Alastair Reid napisaĆ:
introduces a type T whose only value is bottom. ]]
This semantics is obviously flawed though because it would suggest that any two values of type T are equal (and equal to bottom) and that optimizations based on that equality are valid.
Since no values of type T are ever created or manipulated, what difference does it make? -- __("< Marcin Kowalczyk \__/ qrczak@knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/
Alastair Reid writes:
I strongly agree that we should definitely add the ability to declare types whose definition is provided externally. (i.e., provide the feature that empty datatype decls currently provide.)
Before adding them, we need to agree on the semantics and syntax (in that order I think).
The obvious semantics based on the syntax is:
[[ The declaration
data T
introduces a type T whose only value is bottom. ]]
This semantics is obviously flawed though because it would suggest that any two values of type T are equal (and equal to bottom) and that optimizations based on that equality are valid. Using an unpointed type (i.e., the value is not bottom) or saying there are no values bottom or otherwise don't help.
The correct semantics has to be something roughly like:
[[ The declaration
data T
declares a type T whose set of values are defined externally to the language. [optional sentence: There are no legal Haskell operations on values of type T.] ]]
If we were actually using values of type T in the way you suggest, I would agree. But the way these empty datatypes are being used is as dummy type arguments to Ptr, as in 'Ptr T'. So I don't think there's any need to give special semantics to datatypes declared with empty declarations. (this is essentially what Marcin just said, but I'm being a bit more verbose). Cheers, Simon
John Meacham wrote:
This was discussed here before and there seemed to be some support for it, but how about adding the empty data declaration extension to the FFI spec?
I agree, it would be a suitable addition to the FFI, since it is the place where such types make most sense. Alastair Reid <alastair@reid-consulting-uk.ltd.uk> writes:
The correct semantics has to be something roughly like:
[[ The declaration
data T
declares a type T whose set of values are defined externally to the language. [optional sentence: There are no legal Haskell operations on values of type T.] ]]
I don't really like the optional sentence, since it suggests that a foreign imported function that operates over such values is not legal Haskell. How about something like: The declaration data T declares an abstract datatype T, whose values and operations are defined external to the Haskell language. Values of T follow the semantics of the foreign language, in particular, with respect to mutability and the admission of the undefined value. This raises the question of whether it should be possible to declare foreign functions directly over such types, rather than through pointers, e.g. data T foreign import mkT :: IO T foreign import fooT :: T -> T rather than data T foreign import mkT :: IO (Ptr T) foreign import fooT :: Ptr T -> Ptr T Obviously the former is much less safe than the latter. However, I know that nhc98 internally at least uses the former style in order to implement built-in types like arrays, big integers, etc. I'm not suggesting we should necessarily formalise that hack as part of the FFI standard - I'm just raising the question for consideration (and probable rejection), so that our choice is an informed one.
I'm not especially keen to change the syntax (especially since the existing syntax is so trivial to implement) but if we're going to add this to the language, we should make sure the syntax and semantics are tolerably clear and in agreement with each other.
I'm not keen to change the syntax either. The current style is pretty clear to my mind. But it is right that we should think about it. Regards, Malcolm
On Monday 02 June 2003 11:14 am, Malcolm Wallace wrote:
How about something like:
The declaration data T declares an abstract datatype T, whose values and operations are defined external to the Haskell language. Values of T follow the semantics of the foreign language, in particular, with respect to mutability and the admission of the undefined value.
I don't think we have much choice about whether undefined values are part of the type. If you can create a value of that type: x <- derefPtr (px :: Ptr T) then you can create a thunk of that type: let y = if <expression> then x else y and that thunk can be bottom.
This raises the question of whether it should be possible to declare foreign functions directly over such types, [...] However, I know that nhc98 internally at least uses the former style in order to implement built-in types like arrays, big integers, etc.
Hugs implements its builtin types this way too (and did so long before the ffi came along). It works because Hugs magically knows how toi implement Int, Float, etc. - it knows about any typechecking rules and, most importantly, it knows how to represent and marshall values of builtin types. If all the compiler is given is: data T where T is not a builtin type, the compiler can't possible know how to represent or marshall it. (And since we can already do that using 'newtype T = MkT Int8' I don't think we need to add another mechanism.) Although it is similar, I think data Int with a builtin type is a different thing from what we're doing when we write: data T for some foreign type. -- Alastair Reid
On 02-Jun-2003, Alastair Reid <alastair@reid-consulting-uk.ltd.uk> wrote:
If all the compiler is given is:
data T
where T is not a builtin type, the compiler can't possible know how to represent or marshall it. (And since we can already do that using 'newtype T = MkT Int8' I don't think we need to add another mechanism.)
Using `newtype T = MkT Int8' or equivalent only works for foreign data types whose representation is known. How do you deal with C's "time_t", for example? Or with C "struct" types? The user doesn't want to make non-portable assumptions about what padding the C compiler is going to insert. Like Haskell, Mercury allows data definitions with no body: :- type t. But to deal with cases like the ones mentioned above, Mercury also allows the user to declare how such types are represented in foreign languages, using `pragma foreign_type' declarations: :- pragma foreign_type("C", t, "some_c_typename"). The Mercury compiler will generate appropriate glue code to marshal and unmarshal values of that type. You might want to consider something like this for Haskell. (Either that, or resign yourselves to this aspect of Haskell's FFI being less expressive than Mercury! ;-) In Mercury, you can even give a simple type different foreign_type declarations for different languages: :- type time. :- pragma foreign_type("C", time, "time_t"). :- pragma foreign_type("il", time, "[mscorlib]System.DateTime"). :- pragma foreign_type("java", time, "Java.util.Date"). The Mercury compiler will choose whichever language is most appropriate given the compilation options (e.g. when compiling to C or assembler, it will use the "C" version, when compiling to .NET it will use the "il" version, and when compiling to Java it will use the Java version).
Although it is similar, I think
data Int
with a builtin type is a different thing from what we're doing when we write:
data T
for some foreign type.
Sure. In fact, the "data Int" declaration need not even be physically present; the implementation could just insert it in the Prelude automatically. (That's what the Mercury implementation does for its builtin types.) -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Using `newtype T = MkT Int8' or equivalent only works for foreign data types whose representation is known. How do you deal with C's "time_t", for example? Or with C "struct" types? The user doesn't want to make non-portable assumptions about what padding the C compiler is going to insert.
The usual way is to use some autoconf magic or the hsc2hs preprocessor to figure it out. The key part of fptools/aclocal.m4:FPTOOLS_CHECK_HTYPE is this program: typedef $1 testing; main() { FILE *f=fopen("conftestval", "w"); if (!f) exit(1); if (((testing)((int)((testing)1.4))) == ((testing)1.4)) { fprintf(f, "%s%d\n", ((testing)(-1) < (testing)0) ? "Int" : "Word", sizeof(testing)*8); } else { fprintf(f,"%s\n", (sizeof(testing) > sizeof(double)) ? "LDouble" : (sizeof(testing) == sizeof(double)) ? "Double" : "Float"); } fclose(f); exit(0); }
But to deal with cases like the ones mentioned above, Mercury also allows the user to declare how such types are represented in foreign languages, using `pragma foreign_type'
declarations: :- pragma foreign_type("C", t, "some_c_typename").
The Mercury compiler will generate appropriate glue code to marshal and unmarshal values of that type.
How does it figure out what concrete type to use? Does it use trial compilations a la autoconf?
In fact, the "data Int" declaration need not even be physically present; the implementation could just insert it in the Prelude automatically. (That's what the Mercury implementation does for its builtin types.)
Hugs used to insert it automatically but we found that putting the declarations in the Prelude source code: 1) Made the source closer to the report and made the Prelude a little easier to understand. 2) Reduced the amount of magic required in the compiler. -- Alastair
On 02-Jun-2003, Alastair Reid <alastair@reid-consulting-uk.ltd.uk> wrote:
The Mercury compiler will generate appropriate glue code to marshal and unmarshal values of that type.
How does it figure out what concrete type to use? Does it use trial compilations a la autoconf?
No. We don't try to second-guess the C compiler. We just let the C compiler figure it out. The Mercury compiler generates a small amount of glue code in C, and invokes the C compiler on that code. The glue code consists of a wrapper around every foreign function (although if you wanted to optimize things, you might note that this is only really needed for foreign functions whose arguments or result are foreign types). The glue code interface uses only known types such as WORD, so it can be called in a straight-forward manner from the assembly code that the Mercury compiler's assembly-language back-end generates. The glue code for each function calls the function, boxing or unboxing the arguments and/or results if needed, i.e. if they have a foreign type which doesn't fit in a word. This approach of boxing all foreign types has the major advantage that these foreign types can be passed to polymorphically typed Mercury procedures or inserted in polymorphically typed Mercury data structures. Generating a wrapper for every foreign function also means that it is possible to interface to C macros as well as to C functions. In fact, we also allow interfacing with arbitrary C code fragments, not just macros or functions, so you can use this feature to write inline asm, for example. To illustate how the wrappers work, here's what the glue code for a function "foo :: T -> T", where "T" is a foreign type implemented by the C type "CT", could look like: WORD foo_glue(WORD boxed_x) { WORD boxed_y; CT x, y; /* maybe unbox the function's input argument */ x = *(CT *)(sizeof(CT) > sizeof(WORD) ? boxed_x : &boxed_x); /* call the function or macro foo() */ y = foo(x); /* maybe box the function's result */ if (sizeof(CT) > sizeof(WORD)) { boxed_y = (WORD) GC_malloc(sizeof(CT)); *(CT *)boxed_y = y; } else { *(CT *)&boxed_y = y; } return boxed_y; } Note that if compiling to C, such wrappers can be inlined. -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
participants (8)
-
Alastair Reid -
Fergus Henderson -
John Meacham -
Malcolm Wallace -
Manuel M T Chakravarty -
Marcin 'Qrczak' Kowalczyk -
Ross Paterson -
Simon Marlow