November 2002 - FFI - Haskell.org

Finalizers: conclusion?
by Simon Marlow 24 Jan '03

24 Jan '03

Ok, I'm sad to say that the problem we recently uncovered to do with finalizers sharing values with the rest of the program essentially kills off the possibility of doing Haskell finalizers in systems without proper concurrency support. I'm rather embarassed that I didn't notice this before; sorry for wasting everyone's time :-( Let's keep C finalizers for the FFI spec. In GHC I imagine we'll continue to offer Haskell finalizers as an extension, but I haven't decided on an interface yet (suggestions welcome). >From now on I intend to keep track of a strict quota that prevents me from writing more email than code :-) Cheers, Simon

6 8

White space in ccall impents
by Ian Lynagh 22 Jan '03

22 Jan '03

Is it just me or does the FFI spec not explain what whitespace is needed between the components of the ccall impent? 4.1.1 is the obvious place for it to be. In particular, even after reading between the lines, questions like "are multiple spaces between static and fname allowed", "is a space between & and cid allowed (or required)" and "are tabs allowed" seem to remain unanswered. Thanks Ian

3 5

["Simon Marlow" <simonmar@microsoft.com>] RE: cvs commit: fptools/libraries/base/Foreign ForeignPtr.hs
by Alastair Reid 22 Jan '03

22 Jan '03

[Thread moved from the library cvs mailing list] Simon Marlow: > - Fix the Eq instance for ForeignPtr to match the behaviour > specified by the spec. Two ForeignPtrs are equal iff their > underlying Ptrs are equal (previously they were equal iff they were > the same ForeignPtr). Alastair Reid: > Hmmm, the Hugs implementation goes for 'same foreignptr' and, to me, > that seems to make more sense. > > I wonder why the spec changed the semantics from that in the > Hugs-GHC spec? Simon Marlow: > I don't have any strong opinions on the matter, I just changed it to > match the spec because someone complained. > > Do you have an example where the previous behaviour is more useful? > Perhaps it should be brought up on the FFI list. I don't have a really compelling example just a nagging feeling and worry that a design is being changed without (as far as I can recall) any discussion of the matter. Here's some questions that might help understand the difference: 1) Is there any practical difference between the old semantics and the new? There used to be a big difference because ForeignObjects supported an operation to replace the underlying Ptr. Equality on mutable objects is clearly different from equality on the contents of the mutable objects. But, ForeignPtrs with the same Ptr but different finalizers are different and the finalizer list _is_ mutable so there is a useful difference. The difference between two ForeignPtrs which contain the same Ptr is especially clear if we think about what happens if the first one dies before the second or the second is the first to die. 2) Can either semantics be implemented in terms of something we already have? We can implement the new semantics using castForeignPtrToPtr. There's no way to implement the old semantics using other existing operations. Even if there was a way to extract the list of finalizers (and assuming we're happy using pointer equality on the finalizer functions), we still couldn't implement it because ForeignPtrs are, effectively, mutable objects. In short, I can see that switching to the new semantics (ForeignPtr equality == equality of contents) removes something we had before and doesn't add anything new but I don't have any examples of where it is actually useful. (I also have no examples of where the new semantics is useful.) I should also say that I have no strong feelings on the matter - I'd just like to hear someone argue that the change is an improvement or be pointed at an old discussion on the subject. -- Alastair

4 7

What does 'threadsafe' mean?
by Alastair Reid 30 Nov '02

30 Nov '02

[This question came from the ghc-users thread on native threads but it seems to belong on the ffi list so I'm copying it and my response to here.] Wolfgang Thaller writes: > I reread Section 3.3 of the FFI document (RC7), and now I think I > cannot clarify my specification in this respect without first asking > others to clarify the current specs - can someone explain the > distinction between unsafe, safe and threadsafe in the current FFI > to me? I think I know what it does in GHC, but what's the general > definition? My reading of the spec is that it means that execution of other runnable Haskell threads and of the foreign function can be interleaved. That is, neither will be 'starved'. Operationally, this requires that the Haskell runtime and the foreign call operate in separate threads. I think it is a mistake for the ffi spec to say that single-threaded systems may safely ignore this spec. Who would specify 'threadsafe' if they were not worried about starvation? How could an implementation that allows starvation be considered correct? I think Haskell compilers that cannot implement 'threadsafe' as it is implemented in GHC should reject Haskell programs which use it. I think the ffi spec should be amended to say this (the spec currently says that such compilers may treat 'threadsafe' as meaning 'safe'). I suspect that 'threadsafety' is actually orthogonal to the 'safe/unsafe' distinction. I think it would be perfectly meaningful to make calls which must execute in a separate thread (i.e., are 'threadsafe') but do not make callbacks into Haskell (i.e., could otherwise be 'unsafe'). We currently ignore this case because the overhead of a context switch is so much greater than the overhead of a safe call. -- Alastair Reid alastair(a)reid-consulting-uk.ltd.uk Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/

2 1

RE: addForeignPtrFinalizer
by Simon Marlow 26 Nov '02

26 Nov '02

> > Actually in the current implementation the finalizers are not > > attached to the ForeignPtr at all: it's the other way around. > > Ah, I see. > > Fortunately, I don't think my sketched implementation depended on such > an attachment since it added an attachment of its own. The design is > intended to sit on top of the current GHC implementation (i.e., one > with Haskell finalizers and no control over the order they are run). Yes, you're right. We could do it that way. I'm also concerned about adding the extra overhead though, especially if this is something that will rarely be needed. Are we sure this is what we want? At least I can optimise mallocForeignPtr again and avoid registering the finalizer until the first call to addForeignPtrFinalizer. Cheers, Simon

3 4

RE: ForeignDependencies: The Semantics
by Simon Marlow 26 Nov '02

26 Nov '02

> ack! no with my intended semantics, cycles are not a problem anymore > than they are for weak pointers or the touchForeignPtr method. > everything will still be GCed just like normal. > > (note that these are pretty much exactly the Weak Pointer > semantics, if > you like replace 'ForeignDependency' with Weak Pointer as you read and > it should make sense) > > addForeignDependency k v creates a weak pointer from k to v. a weak > pointer ONLY affects GCing and guarantees __nothing about the > ordering of > the finalizers__ for the key and value (I will show why this is not a > problem). Thanks for the summary John. Actually this is the conclusion that Simon & I came to about 2 minutes ago, without reading your message. We need to be very clear about the difference between what you're proposing and a stronger requirement on finalizer ordering. In particular, in your example, if the finalizer for k actually follows the pointer to v, then you really need finalizer ordering and just liveness dependencies won't do. The GHC team's view is that we can implement finalizer dependencies pretty easily (with weak pointers, of course), but doing finalizer ordering is another kettle of fish altogether. We think it's better to steer well clear if at all possible, but the methods I outlined in my previous message still apply (I think ordering finalizers placed on the same object with addForeignPtrFinalizer is probably ok, and I'll implement it in GHC when all this has settled down). For Hugs & NHC, it would probably suffice to keep a list of dependencies in the ForeignPtr. After GC, you have to repeatedly traverse the live ForeignPtrs making each of the dependencies live, until you can make no more ForeignPtrs live, then finalize the dead ones. Cheers, Simon

2 2

Cigarettes online at the lowest possible prices ONLY FOR ffi@haskell.org
by pprakash 26 Nov '02

26 Nov '02

1 0

RE: MVar interface [was: Re: ForeignPtr-related signatures]
by Simon Marlow 19 Nov '02

19 Nov '02

> The current `modifyMVar`, though "exception safe", is not > "atomic" in the > same way that the proposed `atomicModifyMVar` would be. Unless I > misunderstand, during the execution of `modifyMVar`'s second > argument, the > mvar is empty and could be filled by some other thread. With > `atomicModifyMVar`, the contents of the mvar are changed > atomically, based > on the given (pure) function; no other thread can intervene. This is true, but I've never come across an application for which atomicModifyMVar is essential; the two main idioms for using MVars that I know of are - each thread does take followed by put in strict sequence - the MVar is used as a 1-place channel, with some threads doing put and some (usually one) doing take. So the fact that modifyMVar isn't atomic in the presence of another thread doing a concurrent put doesn't usually matter, because there are no such threads. However, I agree that atomicModifyMVar does provide functionality which doesn't currently exist. Cheers, Simon

1 0

RE: ForeignPtr-related signatures
by Simon Marlow 19 Nov '02

19 Nov '02

> Simon Marlow wrote: > > [...] There was this, from Dean Herrington: > > > > http://haskell.cs.yale.edu/pipermail/ffi/2002-October/000940.html > > > > I don't have any strong feelings (I rarely do, where names are > > concerned). > > I would be happy with his proposal. We can leave the IORef > stuff in IOExts > as it is and use the new signatures and functions in > Data.IORef. The only > annoying thing is the signature of Data.IORef.modifyIORef, > but we could > declare the old signature as a bug... :-] Actually, looking at the proposal again, I don't think I like the changes to the MVar interface. atomicModifyMVar seems strange, because modifyMVar is already atomic. We have to be careful about trying to smooth over differences between IORef and MVar where those differences are real and important. I don't mind changing modifyIORef to have a signature which is more similar to modifyMVar, and I think adding atomicModifyIORef_ is a good idea (although it needs a new primitive if we're to take advantage of its efficiency over atomicModifyIORef). Summary: - make modifyIORef's type match modifyMVar - add modifyIORef_ - add atomicModifyIORef and atomicModifyIORef_ Cheers, Simon

2 2

Re: ForeignPtr-related signatures
by Sven Panne 18 Nov '02

18 Nov '02

Malcolm Wallace wrote: > [...] there was considerable sympathy for using new names for the > C-finaliser-only functions, preferably incorporating the word "unsafe" > [...] I'd really prefer that, too. Simon Marlow wrote: > GHC 5.04.2 won't adopt any library changes, as it's only a patchlevel > release (we don't make API changes between patchlevels, only major > releases). That's what I expected. Would it be possible to include the new functions in 5.04.2? OK, that's an API change, too, but simple additions are rather harmless. If we don't include this stuff now, there is no useful common ForeignObj API for GHC/Hugs/NHC in the near future. > [...] Also still unresolved is what to do about atomicModifyIORef and > friends, BTW. I remember atomicModifyIORef, but what are its "friends" exactly? PVars have died, IIRC, and some cunning constraints on the execution order of finalizers were rejected, too. Cheers, S.

2 1