RE: Cheap ForeignPtr allocation
Hi Folks, I'd like to propose two new functions for the ForeignPtr interface:
I'm a bit uneasy about this.
It seems like decisions made on technological grounds (i.e., because they enable an optimization) tend to date rather quickly because the technology tends to change too.
Hmm, I see your point. Nevertheless, I think even without the tricks I'm using in GHC, the case where a ForeignPtr is used in conjunction with malloc()/free() is one which is likely to be optimisable in any system with its own memory management.
The functions may cover a fairly common case of using ForeignPtrs (though I've never used ForeignObjs or MallocPtrs that way) - but most code doesn't use ForeignPtrs so it's a common case of an uncommon thing.
Ok, I'll give some more motivation. I've been converting over some old FFI code, namely GHC's POSIX library. This was written back in the old days before the FFI, and uses lots of GHC extensions. One such extension, which doesn't have an equivalent in the FFI, is to use ByteArrays as garbage-collected chunks of memory which can be passed to foreign functions. The example I'm looking at right now is the SignalSet type in the POSIX library: a SignalSet is a pure value, and there are operations to create new empty/full sets, and create new sets by adding/deleting elements from existing sets. All these operations are pure and work outside the IO monad: emptySignalSet :: SignalSet fullSignalSet :: SignalSet addSignal :: Signal -> SignalSet -> SignalSet deleteSignal :: Signal -> SignalSet -> SignalSet inSignalSet :: Signal -> SignalSet -> Bool However, inside a SignalSet is a real sigset_t which can be passed to the C functions which operate on it. An ideal opportunity for a ForeignPtr, one might think: but using a ForeignPtr here, with free() as its finalizer, adds so much overhead that it would be better to go back to manipulating SignalSets in the IO monad (perhaps this isn't such a good example, it's unlikely that someone is going to be doing a lot of SignalSet mangling, but anyway). Another possibility is to fully marshal the sigset_t into Haskell. The marshalling would be quite expensive, but if there was a lot of manipulation of the SignalSet going on in Haskell land then this might be a good choice. This is a well-known tradeoff; and sometimes it isn't possible to do the marshalling because the C type is abstract. The main point is that having a cheap way to use ForeignPtr with malloc()/free() makes it much more likely that we'll settle on a nice pure interface in this case, because the overhead for doing so is significantly lower.
I'm inclined to say: add it to GHC for now and prod us again once a decent body of examples builds up.
Well, I'll do this if I have to, but I don't really want to make my FFI code non-portable. Cheers, Simon
Nevertheless, I think even without the tricks I'm using in GHC, the case where a ForeignPtr is used in conjunction with malloc()/free() is one which is likely to be optimisable in any system with its own memory management.
I wasn't meaning so much that only GHC could take advantage of it (though I think that is true at present) but that someone might come along next week with a technique which avoids the problem altogether.
[...] using a ForeignPtr here, with free() as its finalizer, adds so much overhead that [...]
Where is the overhead coming from? Is it the cost of a C call or the cost of the standard malloc library? If the latter, I imagine that a custom allocator would have similar performance to using pinned objects. (I'm sort of assuming that pinned objects are more expensive than normal objects.) btw I don't know if it's relevant but there's an important semantic difference between allocating on the GHC heap and allocating on the C heap. The C version can be manipulated by the malloc.h functions. In particular, you can call realloc on it. Do that on the GHC heap version and... well, it won't be pretty. I don't know if this is relevant because using realloc with ForeignPtr'd memory is likely to be a delicate procedure no matter how it got allocated. -- Alastair
participants (2)
-
Alastair Reid -
Simon Marlow