September 2001 - Glasgow-haskell-users

RE: Alternative Design for Finalisation
by Ashley Yakeley 20 Sep '01

20 Sep '01

At 2001-09-20 04:56, Simon Marlow wrote: >ForeignPtrs are defined like this > > data ForeignPtr a = ForeignPtr ForeignObj# > >where ForeignObj# is a *boxed* address, it corresponds to a real heap >object. The heap object is primitive from the point of view of the >compiler - it can't be optimised away. So it works to attach a >finalizer to the ForeignObj# (but not to the ForeignPtr!). OK, you are saying that the ForeignObj# is a representation of a 'Ptr a', but in a way such that the compiler can't optimise away the box. Is that correct? Could ForeignPtr be generalised so that its ForeignObj# could represent any type, rather than just a Ptr? -- Ashley Yakeley, Seattle WA

1 0

Re: Ptr and ForeignPtr Questions
by Ashley Yakeley 20 Sep '01

20 Sep '01

At 2001-09-19 23:45, Manuel M. T. Chakravarty wrote: >> What if the type is polymorphic (e.g. >> declared as 'Storable a => Ptr a' rather than something like 'Ptr Word8')? > >Also possible, as the argument to `Ptr' is just dummy. What? What about 'withObject'? A Haskell 'Ptr a' is a C array-pointer of whatever corresponds to 'a'. I don't think the argument is dummy. Consider this: int foo (char selector,char* arg) { if (selector == 100) return reinterpret_cast<unsigned char*>(arg)[1]; if (selector == 200) return reinterpret_cast<unsigned short*>(arg)[1]; return 0; } foreign import "foo" foo :: Storable a => Word8 -> Ptr a -> IO Int32; a <- withObject ([1,2] :: [Word8]) (foo 100); b <- withObject ([3,4] :: [Word16]) (foo 200); Will this work as expected? I expect 'a' to be 2 and 'b' to be 4... >> 3. What about ForeignPtr? Can instances of 'Storable a => ForeignPtr a' >> be used in FFI? > >They can be passed to C, but you can't get them back. (The >storange manager wouldn't know what finaliser to attach.) OK. Are ForeignPtrs intelligible in the C function as pointers to the named type? >> 4. Does newForeignPtr work safely with null pointers, and will the >> finalizer get called? For instance: >> >> fp <- newForeignPtr nullPtr finalFunc; >> let {isNull = (foreignPtrToPtr fp) == nullPtr}; >> r <- withForeign fp (\p -> foo p); >> >> Will foo be passed nullPtr? Will finalFunc ever get called? Is my use, >> above, of foreignPtrToPtr safe, and will isNull be True? > >Should work. From the storage managers point of view, a >`Ptr' is just an uninterpreted bit-pattern. A glorified >`Int'. So you are saying that the ForeignPtr code is not interested in the "pointer-ness" of the Ptr contents of a ForeignPtr, except when a ForeignPtr is used as an FFI argument? Presumably this also means that one can create two separate ForeignPtrs around the same Ptr, each with their own finaliser set. Presumably they would not be equal (note that Eq (ForeignPtr a)). Is this correct? Also, I assume that a ForeignPtr is eligible for garbage collection whenever it is no longer 'reachable', even if the Ptr it contains is reachable. Is that correct? Is there anything resembling Java's 'soft' and 'weak references'? > Of course, you should better make sure that >`finalFunc' can handle getting a `nullPtr'. Of course... -- Ashley Yakeley, Seattle WA

2 1

RE: Alternative Design for Finalisation
by Simon Marlow 20 Sep '01

20 Sep '01

> Unfortunately it isn't possible to add a finalizer to a (Ptr a). We > already have a generic finalization mechanism: see the Weak module in > package lang. But the only reliable way to use finalizers(*) is to > attach one to an atomic heap object - that way the compiler's > optimiser > can't interfere with the lifetime of the object. I forgot the footnote: (*) sometimes it doesn't matter that the object being finalized might be optimised away or duplicated. For example, we use the notion of finalizers on arbitrary objects for our implementation of memo tables, which rely on pointer-equality (actually StableName equality). If an object gets optimised away or duplicated by the optimiser, then the memo table is just less effective. Cheers, Simon

1 0

RE: Alternative Design for Finalisation
by Simon Marlow 20 Sep '01

20 Sep '01

> If ForeignPtrs work the way I think they do, then I'm > surprised they're > designed as pointers. I believe the 'pointer' functionality > is orthogonal > to the 'finalisable' functionality and should be separated like this: > > -- > data Finalisable a -- abstract handle to finalisable object > instance Eq (Finalisable a); > newFinalisable :: a -> IO () -> IO (Finalisable a); > addFinaliser :: Finalisable a -> IO () -> IO (); > withFinalisable :: Finalisable a -> (a -> IO b) -> IO b; > touchFinalisable :: Finalisable a -> IO (); > finalisableContents :: Finalisable a -> a; > > type ForeignPtr a = Finalisable (Ptr a); > newForeignPtr :: Ptr a -> IO () -> IO (ForeignPtr a); > newForeignPtr = newFinalisable; > addForeignPtrFinalizer :: ForeignPtr a -> IO () -> IO () ; > addForeignPtrFinalizer = addFinaliser; > withForeignPtr :: ForeignPtr a -> (Ptr a -> IO b) -> IO b; > withForeignPtr = withFinalisable; > touchForeignPtr :: ForeignPtr a -> IO (); > touchForeignPtr = touchFinalisable; > foreignPtrToPtr :: ForeignPtr a -> Ptr a; > foreignPtrToPtr = finalisableContents; Unfortunately it isn't possible to add a finalizer to a (Ptr a). We already have a generic finalization mechanism: see the Weak module in package lang. But the only reliable way to use finalizers(*) is to attach one to an atomic heap object - that way the compiler's optimiser can't interfere with the lifetime of the object. The Ptr type is really just a boxed address - it's defined like data Ptr a = Ptr Addr# where Addr# is an unboxed native address (just a 32- or 64- bit word). Putting a finalizer on a Ptr is dangerous, because the compiler's optimiser might remove the box altogether. ForeignPtrs are defined like this data ForeignPtr a = ForeignPtr ForeignObj# where ForeignObj# is a *boxed* address, it corresponds to a real heap object. The heap object is primitive from the point of view of the compiler - it can't be optimised away. So it works to attach a finalizer to the ForeignObj# (but not to the ForeignPtr!). There are several primitive objects to which we can attach finalizers: MVar#, MutVar#, ByteArray#, etc. We have special functions for some of these: eg. MVar.addMVarFinalizer. So a nicer interface might be something like class Finalizable a where addFinalizer :: a -> IO () -> IO () instance Finalizable (ForeignPtr a) where ... instance Finalizable (MVar a) where ... (apologies for the different spelling of finalize - apparently both are correct and I randomly settled on the 'z' version some time ago). So you might ask why we don't just get rid of Ptr and rename ForeignPtr to Ptr. The reason for that is just efficiency, I think. > The only time when ForeignPtrs act like Ptrs is when they are > used as FFI > arguments. But I believe that's purely syntactic sugar for > withForeignPtr, and would be no loss. Indeed, the latest version of the FFI spec dissallows ForeignPtrs as FFI arguments. Hope this all makes some sense. I'll copy the text into the GHC commentary if so. Cheers, Simon

1 0

Alternative Design for Finalisation
by Ashley Yakeley 20 Sep '01

20 Sep '01

If ForeignPtrs work the way I think they do, then I'm surprised they're designed as pointers. I believe the 'pointer' functionality is orthogonal to the 'finalisable' functionality and should be separated like this: -- data Finalisable a -- abstract handle to finalisable object instance Eq (Finalisable a); newFinalisable :: a -> IO () -> IO (Finalisable a); addFinaliser :: Finalisable a -> IO () -> IO (); withFinalisable :: Finalisable a -> (a -> IO b) -> IO b; touchFinalisable :: Finalisable a -> IO (); finalisableContents :: Finalisable a -> a; type ForeignPtr a = Finalisable (Ptr a); newForeignPtr :: Ptr a -> IO () -> IO (ForeignPtr a); newForeignPtr = newFinalisable; addForeignPtrFinalizer :: ForeignPtr a -> IO () -> IO () ; addForeignPtrFinalizer = addFinaliser; withForeignPtr :: ForeignPtr a -> (Ptr a -> IO b) -> IO b; withForeignPtr = withFinalisable; touchForeignPtr :: ForeignPtr a -> IO (); touchForeignPtr = touchFinalisable; foreignPtrToPtr :: ForeignPtr a -> Ptr a; foreignPtrToPtr = finalisableContents; -- I am slightly bothered by the type of finalisableContents/foreignPtrToPtr. Shouldn't it be in the IO monad? Apart from 'finalisers already run' risk, is it safe? But 'castForeignPtr' would not be definable, unless you wanted to do something like this: -- instance Functor Finalisable; castForeignPtr :: ForeignPtr a -> ForeignPtr b; castForeignPtr = fmap castPtr; -- ...which I don't believe is appropriate. The only time when ForeignPtrs act like Ptrs is when they are used as FFI arguments. But I believe that's purely syntactic sugar for withForeignPtr, and would be no loss. -- foreign import "foo" fooFP :: ForeignPtr a -> IO (); foreign import "foo" fooP :: Ptr a -> IO (); fooFP' :: ForeignPtr a -> IO (); fooFP' fp = withForeignPtr fp fooP; -- -- Ashley Yakeley, Seattle WA

1 0

RE: Ptr and ForeignPtr Questions
by Ashley Yakeley 20 Sep '01

20 Sep '01

At 2001-09-20 02:46, Simon Marlow wrote: >No, it's not safe. The reason is that the compiler can track a >ForeignPtr to discover when it dies, in order to run the finalizer, but >it can't track a Ptr. As soon as you drop all references to the >ForeignPtr then the finalizer will run, even if you converted it to a >Ptr and you're still using it. OK, there are different types of safety. The risk here is that the finalisers may have been called by the time you use the Ptr. But if you don't mind that, for instance, if for some reason the finalisers don't render the Ptr invalid, I assume you can still use the Ptr. -- Ashley Yakeley, Seattle WA

1 0

RE: Ptr and ForeignPtr Questions
by Simon Marlow 20 Sep '01

20 Sep '01

> At 2001-09-20 02:14, Simon Marlow wrote: > > >I'll just add that the docs have been updated for 5.02, > > Do you have a URL for that? Not yet, but the release is imminent (really!) so it'll be up on the web site shortly. > >And don't forget that using foreignPtrToPtr is quite dangerous; much > >better to use withForeignPtr instead, otherwise you might find the > >ForeignPtr being finalised earlier than you expect. > > This is because foreignPtrToPtr is not in the IO monad, correct? > > foreignPtrToPtr :: ForeignPtr a -> Ptr a; > > A function like this: > > ioForeignPtrToPtr :: ForeignPtr a -> IO (Ptr a); > ioForeignPtrToPtr fp = withForeignPtr fp return; > > ...would surely be safe? The 5.00 documentation claims that it isn't, > however, but I don't see why. No, it's not safe. The reason is that the compiler can track a ForeignPtr to discover when it dies, in order to run the finalizer, but it can't track a Ptr. As soon as you drop all references to the ForeignPtr then the finalizer will run, even if you converted it to a Ptr and you're still using it. withForeignPtr is safe to use because it ensures that the ForeignPtr is kept live until it returns. The documentation is a little terse on this issue; I'll see if I can improve it. Cheers, Simon

1 0

Re: Ptr and ForeignPtr Questions
by Ashley Yakeley 20 Sep '01

20 Sep '01

At 2001-09-20 02:31, I wrote: > int foo (char selector,char* arg) ... > if (selector == 200) I guess that should be int foo (unsigned char selector,char* arg) -- Ashley Yakeley, Seattle WA

1 0

RE: Ptr and ForeignPtr Questions
by Ashley Yakeley 20 Sep '01

20 Sep '01

At 2001-09-20 02:14, Simon Marlow wrote: >I'll just add that the docs have been updated for 5.02, Do you have a URL for that? >And don't forget that using foreignPtrToPtr is quite dangerous; much >better to use withForeignPtr instead, otherwise you might find the >ForeignPtr being finalised earlier than you expect. This is because foreignPtrToPtr is not in the IO monad, correct? foreignPtrToPtr :: ForeignPtr a -> Ptr a; A function like this: ioForeignPtrToPtr :: ForeignPtr a -> IO (Ptr a); ioForeignPtrToPtr fp = withForeignPtr fp return; ...would surely be safe? The 5.00 documentation claims that it isn't, however, but I don't see why. -- Ashley Yakeley, Seattle WA

1 0

RE: Ptr and ForeignPtr Questions
by Simon Marlow 20 Sep '01

20 Sep '01

> Ashley Yakeley <ashley(a)semantic.org> wrote, > > > The documentation for FFI in the GHC user's guide seems to > be out of date > > with regard to passing Ptrs across. > > > > 1. My reference is > > <http://www.haskell.org/ghc/docs/latest/set/ffi.html> > > (from > <http://www.haskell.org/ghc/docs/latest/set/book-users-guide.html>) > > Is this the latest documentation available? > > > > 2. My understanding is that you can use any instance of > 'Storable a => > > Ptr a' as an FFI argument and return type for both imported > and exported > > functions? Is this correct? > > If I understand you question correctly and you want to pass > a pointer to C land and back, yes, this is possible. I'll just add that the docs have been updated for 5.02, and the FFI section now refers to Ptr and ForeignPtr instead of Addr and ForeignObj. > > 4. Does newForeignPtr work safely with null pointers, and will the > > finalizer get called? For instance: > > > > fp <- newForeignPtr nullPtr finalFunc; > > let {isNull = (foreignPtrToPtr fp) == nullPtr}; > > r <- withForeign fp (\p -> foo p); > > > > Will foo be passed nullPtr? Will finalFunc ever get called? > Is my use, > > above, of foreignPtrToPtr safe, and will isNull be True? > > Should work. From the storage managers point of view, a > `Ptr' is just an uninterpreted bit-pattern. A glorified > `Int'. Of course, you should better make sure that > `finalFunc' can handle getting a `nullPtr'. And don't forget that using foreignPtrToPtr is quite dangerous; much better to use withForeignPtr instead, otherwise you might find the ForeignPtr being finalised earlier than you expect. Cheers, Simon

1 0