By the way, you lot are aware that under the old ANSI C standard at least, pointer types can have different sizes depending on the type? Thus a pointer to a properly-aligned 64-bit entity needs 3 less bits than one to an 8-bit entity, and implementations are entitled to use this. The only guarantee I can remember is that (void *) is big enough to represent everything. Does this affect things at all?
By the way, you lot are aware that under the old ANSI C standard at least, pointer types can have different sizes depending on the type? Thus a pointer to a properly-aligned 64-bit entity needs 3 less bits than one to an 8-bit entity, and implementations are entitled to use this. The only guarantee I can remember is that (void *) is big enough to represent everything. Does this affect things at all?
C has its faults but at least it was easy to understand what things meant (albeit in a very low-level manner) and the various implementation options. Then along come the standards organizations and everything that was simple gets complicated. I can see exactly why they did this (e.g., I'm currently working with an embedded processor that uses 16-bit addresses to index its 128K 16-bit-aligned instruction space and 12-bit addresses to index its 4K 8-bit aligned data space) but... ugh! I'm not sure whether it needs a change in the ffi implementations unless the function calling convention on some platform were to pass these pointers differently. But, let's imagine a pointer to an aligned 64K object in a 32-bit address space being stored in just 16 bits. I think the only change we'd need in the ffi spec to accomodate this is: Any casting between Ptr types (and FunPtr, StablePtr, etc.) has to apply the appropriate bit-shift (or whatever transformation is required). The simplest way to handle this may be to remove all the cast* functions from the ffi spec so that they have to be done in the C world (which understands these things). If this were simply intolerable (I must confess I don't know why we have them), I guess I'd define a ForeignCastable typeclass providing appropriate operations to cast to and from the equivalent of void*. As long as forall alpha. sizeof(HsPtr) >= sizeof(alpha*) I think the rest of the FFI spec is fine because the Haskell side never tries to do anything with a pointer - it just treats it as a curious looking bit-pattern that the C world would like it to hang onto for a while (or, in less anthropomorphic language, it is an opaque/abstract datatype). -- Alastair Reid reid@cs.utah.edu http://www.cs.utah.edu/~reid/
Alastair Reid wrote: [snip]
Any casting between Ptr types (and FunPtr, StablePtr, etc.) has to apply the appropriate bit-shift (or whatever transformation is required). The simplest way to handle this may be to remove all the cast* functions from the ffi spec so that they have to be done in the C world (which understands these things). If this were simply intolerable (I must confess I don't know why we have them), I guess I'd define a ForeignCastable typeclass providing appropriate operations to cast to and from the equivalent of void*. [snip] Also I think something would need to be done to allow the user to interface to C libraries which need pointers to their own types (as in my first example). You don't know whether DB * is a pointer to char, or int, or whatever. Also the suggested solution
data DB -- note that we have no constructors foreign import ... :: blah -> IO (Ptr DB) does not necessarily guarantee that a Ptr DB is actually able to represent a DB *, whatever it is. So my suggestion would be that (1) a data DB (or any other type with constructors) would be regarded as like C's type void, so that casting to a Ptr DB and back from any other pointer type will preserve the pointer. (2) the implementation be required to perform any necessary casting between pointer types in argument and result types. Then if a DB * is actually a pointer to a 4-byte quantity (say), it will silently cast from Ptr DB. I hope this would not in fact be too hard; where a C compiler is used to read the header file, calling an import'd function with non-matching pointer types one hopes the compiler will insert the relevant casts. Still these are deep waters; if this issue is to be addressed perhaps someone should check the latest copy of the C standard and see if it still exists. (I think Harbisson & Steele's reference to standard C mentions a machine where differing pointer sizes actually occur, but I don't know if any are still around.)
If I understand you correctly, your concern is that a pointer argument/result might be passed differently depending on what it is a pointer to. We already have a special case for FunPtr because they can be different sizes from normal Ptrs (especially on a Harvard architecture). What I think you're suggesting is that we might need to make a special case for other pointer types too. This feels like a very unlikely thing to happen and I don't understand it yet so I propose the following (btw note that I don't speak for the whole FFI group). Put off acting on this until we find a platform like this. I believe that all currently supported platyforms are not like this because if they were we'd have noticed when we ported the compiler to this platform. On finding a platform like this, we can do one of: 1) Define a new calling convention ccall-plus (or whatever) which is just like the ccall convention except that the extloc strings can contain extra information to resolve the problem. 2) Require any affected functions to go through C wrappers which do the coercion. My feeling is that this will be viable because I suspect there would be something very unusual about the pointers which are passed differently (e.g., they will be pointers into I/O space or pointers to 32-bit aligned memory) so they will be rare and easy to find. 3) Find a way to add the missing type information to the Haskell type. I think this is much more than being careful to use Ptr (Ptr DB) instead of Ptr DB because I would be hugely surprised if void* and void** had different representations. Rather, I think any differences would only show between types like these: char* char[1 << 16] * char[1 << 32] * Haskell doesn't have a notion of sized types so we would deal with this either by extending Haskell (yoiks!) or by encoding the information using something like this: data Ptr8 a -- pointer to 8-bit aligned object data Ptr16 a -- pointer to 16-bit aligned object data Ptr32 a -- pointer to 32-bit aligned object data Ptr64 a -- pointer to 64-bit aligned object That is, we'd extend the FunPtr approach in some appropriate way. (Even if Haskell did have a notion of sized types, it wouldn't be enough because I'll bet there would be non-standard extensions which let C programmers override the default alignment of a type.) -- Alastair Reid reid@cs.utah.edu http://www.cs.utah.edu/~reid/
Alastair Reid wrote:
If I understand you correctly, your concern is that a pointer argument/result might be passed differently depending on what it is a pointer to.
We already have a special case for FunPtr because they can be different sizes from normal Ptrs (especially on a Harvard architecture). What I think you're suggesting is that we might need to make a special case for other pointer types too.
This feels like a very unlikely thing to happen and I don't understand it yet so I propose the following (btw note that I don't speak for the whole FFI group).
Put off acting on this until we find a platform like this. I believe that all currently supported platyforms are not like this because if they were we'd have noticed when we ported the compiler to this platform.
[snip] Unfortunately there appears to be no way this can be fixed in general for ANSI C without parsing the header file. For example if DB is an external type unknown to the Haskell compiler, and a function requires us to pass in a **DB, to which it writes a *DB as output, we cannot interface to it without knowing what sort of pointer a *DB is, since we cannot create a Ptr (Ptr DB) which will correspond to a **DB in any useful way. So we will just have to cross our fingers and hope it doesn't happen. It's not just a question of differently-sized words. For MMIX (which already has a gcc port) while pointers all have the same size, the bottom bits are junk (simply ignored) so compilers might conceivably zap these bits during casts and use them to store other information.
[snip] Unfortunately there appears to be no way this can be fixed in general for ANSI C without parsing the header file. For example if DB is an external type unknown to the Haskell compiler, and a function requires us to pass in a **DB, to which it writes a *DB as output, we cannot interface to it without knowing what sort of pointer a *DB is, since we cannot create a Ptr (Ptr DB) which will correspond to a **DB in any useful way. So we will just have to cross our fingers and hope it doesn't happen.
The ffi philosophy is to provide a mechanism out of which more convenient tools can be built. There's nothing to stop an external tool from doing this parsing. Indeed, that's what you have to do when interfacing to something like Xlib. Just how big an int is each of the umpteen integral types in Xlib? Which ones are the same size on all platforms and which are different sizes? Probably the best way to figure it out is to write a small script along the lines of: #include "X.h" main() { printf("sizeof(Display*) == %d\n", sizeof(Display*)); printf("sizeof(Window*) == %d\n", sizeof(Window*)); ... }
It's not just a question of differently-sized words. For MMIX (which already has a gcc port) while pointers all have the same size, the bottom bits are junk (simply ignored) so compilers might conceivably zap these bits during casts and use them to store other information.
Ok, if they're passed the same way and all we have to do is respect the bottom bits, that's easy. Don't use the cast functions provided by the Foreign libraries and all will be cool. (We might want to do an audit of the libraries to determine which other functions use the cast functions.) -- Alastair Reid reid@cs.utah.edu http://www.cs.utah.edu/~reid/
On 11-Jun-2002, Alastair Reid <reid@cs.utah.edu> wrote:
If I understand you correctly, your concern is that a pointer argument/result might be passed differently depending on what it is a pointer to.
We already have a special case for FunPtr because they can be different sizes from normal Ptrs (especially on a Harvard architecture). What I think you're suggesting is that we might need to make a special case for other pointer types too.
This feels like a very unlikely thing to happen and I don't understand it yet so I propose the following (btw note that I don't speak for the whole FFI group).
Put off acting on this until we find a platform like this. I believe that all currently supported platyforms are not like this because if they were we'd have noticed when we ported the compiler to this platform.
On finding a platform like this, we can do one of:
1) Define a new calling convention ccall-plus (or whatever) which is just like the ccall convention except that the extloc strings can contain extra information to resolve the problem.
2) Require any affected functions to go through C wrappers which do the coercion.
My feeling is that this will be viable because I suspect there would be something very unusual about the pointers which are passed differently (e.g., they will be pointers into I/O space or pointers to 32-bit aligned memory) so they will be rare and easy to find.
3) Find a way to add the missing type information to the Haskell type. I think this is much more than being careful to use
Ptr (Ptr DB)
instead of
Ptr DB
because I would be hugely surprised if void* and void** had different representations.
On word-addressed machines, `void *' will most likely have a different representation than `void **'. `void *' will require additional bits to distinguish which byte within the word is being pointed to, but `void **' is guaranteed to point to a word-aligned `void *', so these additional bits are not needed. On word-addressed machines, there are usually two representations for pointers: most pointers are represented using the native machine representation, but pointers of type `char *', `signed char *', `unsigned char *', and `void *' need a different (and probably larger) representation, because they can point to bytes within a word. -- Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
George Russell <ger@tzi.de> wrote,
By the way, you lot are aware that under the old ANSI C standard at least, pointer types can have different sizes depending on the type? Thus a pointer to a properly-aligned 64-bit entity needs 3 less bits than one to an 8-bit entity, and implementations are entitled to use this. The only guarantee I can remember is that (void *) is big enough to represent everything. Does this affect things at all?
I have noticed that, but I don't think we should care about this in the FFI spec. There are two possible scenarios: * An architecture just doesn't use all address bits for all memeory areas, but stores addresses in a uniform word size. No problem as long as we don't scramble these bits. The FFI never requires to scramble such bits, so it is on the implementation to get this right. * An architecture really stores such pointers in differently sized objects and passes them around like this. As we can't really distinguish these pointers reliably in Haskell, all sorts of things, like all the methods in the class Storable break and I don't see any halfway easy way to prevent this by changing the spec. Anybody know an example of such an architecture? Cheers, Manuel
participants (4)
-
Alastair Reid -
Fergus Henderson -
George Russell -
Manuel M. T. Chakravarty