February 2003 - FFI - Haskell.org

Native threads
by Simon Peyton-Jones 07 Feb '03

07 Feb '03

Folks There was a spirited debate about the relationship between native OS threads and Haskell threads. I got very confused. To clear up my brain, Simon and I wrote a little operational semantics that tries to make precise what is going on. It's in CVS as haskell-report/ffi/threads.tex (Style files in haskell-report/styles) I did send this round a couple of weeks ago, and others made a couple of updates, but it seems to have killed the discussion stone dead! The (updated) postscript is at http://research.microsoft.com/~simonpj/tmp/threads.ps So I thought I'd mention it again. Apart from anything else, it'd be nice to get a merger of Wolfgang and Alastair's notes into the early sections of the document. Simon

1 0

deriving instances of Storable
by mike V 05 Feb '03

05 Feb '03

Is there any merit in the idea of allowing instances of Storable to be derived in a similar fashion to Show, providing that constituents are of class Storable. for instance (Int, Int) two Ints would be marshalled into consecutive memory locations in their order of declaration. (Int,String) an Int followed by a String dealing with constructors ========================== data X = X1 Int | X2 deriving Storable data Y = Y1 | Y2 deriving Storable three possible alternatives spring to mind A) Indexing Constructors by position X1 -> 0:Int X2 -> 1 where 0, 1 represent the constructor indexes ":" represents that the data items are physically contiguous "->" reads "marshalled as" This has the disadvantage of being sensitive to the constructor order B) using the constructor name as the index X1 -> X1:Int X2 -> X2 has the disadvantage of requiring a variable amount of memory for the tag. C) some other hash on the constructor name ? I would think that implementation would be fairly straight forward? The benefits would be significant - automatic marshalling of structures. uses: storing to disk, passing over network etc etc.. _________________________________________________________________ MSN 8 helps eliminate e-mail viruses. Get 2 months FREE*. http://join.msn.com/?page=features/virus

2 1

Re: haskell finalisers driven underground
by George Russell 05 Feb '03

05 Feb '03

I think essentially there are only three ways you can have Haskell finalizers. (1) every FFI implementation must implement concurrency. I'm not convinced it must be preemptive, but at the very minimum you need some sort of concurrency, since you have some arbitrary number of finalizers (plus the main thread) all running together, all sharing the same space of potentially half-way evaluated values, and so all needing the ability to go to sleep and wake up whoever is half-way through evaluation. This is a problem for NHC, and possibly also for Hugs, so can't be done. However it is how GHC does it. (2) allow values to be evaluated more than once. This breaks various important compiler optimisations, and so right now is not an option for anyone. (3) delay the finalizers until there are no half-way-evaluated values. For example, for non-concurrent Haskells, immediately before or after a primitive IO action in the main trunk of a program (not inside unsafePerformIO !). This would work fine for many people, like me, but those purists who do very long pure computations without ever troubling the IO monad, impurists who run lengthy calculations inside unsafePerformIO, or statists who do lengthy computations in the state monad, may be surprised by the long delays before their finalizer gets run. At the moment the consensus seems to be not to have Haskell finalizer in the standard, but leave them up to the implementation. I think I'm secretly hoping NHC will implement (1) or (3), forcing Hugs to bow to market forces.

1 0

safe and threadsafe
by Wolfgang Thaller 04 Feb '03

04 Feb '03

Hello Everyone, I have recently spent some time improving GHC's support for "threadsafe" foreign calls. As a side effect of fixing a crashing bug, I made "safe" behave in exactly the same way as "threadsafe". Now I have to document it... :-( I'm not very happy with the current definition of safety levels in the FFI addendum (section 3.3, Import Declarations). So before I write up somethhing for the GHC user manual explaining how GHC interprets it and why it does so, I'll try to convince someone to change or at least clarify the FFI spec a bit: First, some things that are unclear: Are safe calls _guaranteed_ to block all other haskell threads and prevent garbage collection, or is that an implementation detail? Are unsafe calls guaranteed to do so? {In a SMP implementation, this might require additional interthread messaging}. What should happen when C code running in a separate OS thread calls a foreign exported function while the Haskell Runtime is blocked on a safe call? Some more points: In what way can a "safe" call be considered safe in a multithreaded environment? After all it may cause very hard-to-predict blocking. I think that "threadsafe" should be the default, where available. It _may_ be possible to implement "safe" in a way that is slightly more efficient than "threadsafe", however this is relatively hard to do correctly in the context of GHC, and I don't expect a big performance gain. But if that is possible, then "safe" would be just a hint for increasing performance by sacrificing some "safety". It should be renamed to something else. "Ceterum censeo" that "threadsafe" should be the default. Some people have said that "safe" calls are intended for serializing access to foreign libraries that are not multithread-safe. I think that MVars should be used for such purposes. Blocking the entire Haskell Runtime for the duration of a "safe" call is overkill, but it's not always enough: Many old C libraries keep state in global variables from one call to the next. Cheers, Wolfgang -- ceterum censeo invocationes "tutas" esse delendas.

1 0

Re: haskell finalisers driven underground
by Sven Panne 04 Feb '03

04 Feb '03

[ Best read while listening to Queen's "I'm going slightly mad" ] Before this futile discussion starts over again, everybody should really read Boehm's "Destructors, Finalizers, and Synchronization" (as suggest by SPJ IIRC): http://www.hpl.hp.com/techreports/2002/HPL-2002-335.html Even with real concurrency, finalizers are a nightmare to implement correctly. (If you think differently and have some JVM background, please send your resume to the company I'm working for. :-} Cheers, S.

1 0

haskell finalisers driven underground
by m v 04 Feb '03

04 Feb '03

outlawing Haskell finalisers / eliding them from the language is a futile exercise ... it will just drive them underground like (using a previous example ) .... >import Foreign >foreign import ccall "wrapper" mkfoo:: (Ptr a -> IO ()) -> IO (FunPtr (Ptr >a -> IO () )) >main = do > p <- mallocBytes 64 > foo <- mkfoo (\y->print x) > newForeignPtr p foo print x > where > x = sum [1..10000] the result of this (using hugs) yields an error as expected .... Main> main INTERNAL ERROR: freeMallocPtr but the utility of haskell finalisers is such that wrapper code like the above is to be expected, which introduces it's own problems. As far as I can see there are no gaurantees for safety using C finalisers as opposed to haskell finalisers so we may as well have the convenience of haskell finalisers. on the finalisation issue, previouse memo's included ... 2) Block the finalizer until the main thread completes evaluation just as GHC does. Blocking the finalizer would require all Haskell implementations to implement preemptive concurrency. This is thought to be an excessive burden. another memo had ... c finalisers are good enough .... I think both propositions bear further examination. 1) c finalisers are good enough... are they ? perhaps I need to justify why haskell finalisers will be driven underground. suppose we have a c routine returning malloc'd memory. Who is going to write a c routine just to free it ? 2) "preemptive concurrency an excessive burden", is preemptive concurrency the only solution ? If it is in what sense is it an excessive burden ? From an implementation point of view or from an operational point of view. _________________________________________________________________ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail

1 0

RE: newForeignPtr
by Simon Marlow 03 Feb '03

03 Feb '03

> "Simon Marlow" <simonmar(a)microsoft.com> writes: > > > > However, only Hugs implements this part of the spec at the moment. > > > > It will be implemented in GHC before the next major > release. I'm not > > intentionally dragging my heels on this one, just haven't > got around to > > it yet :-) > > And of course the ghc-5.04 branch will keep the existing > (non-standard) > signature, because there are no API changes permitted in > bug-fix releases. That's right - the change will be in 5.06 (or 6.00), not in 5.04.3. Cheers, Simon

1 0

RE: newForeignPtr
by Simon Marlow 03 Feb '03

03 Feb '03

> For a summary of the discussion, see > > http://cvs.haskell.org/cgi-bin/cvsweb.cgi/haskell-report/ffi/finalizers. txt > > However, only Hugs implements this part of the spec at the moment. It will be implemented in GHC before the next major release. I'm not intentionally dragging my heels on this one, just haven't got around to it yet :-) Cheers, Simon

2 1

newForeignPtr
by m v 03 Feb '03

03 Feb '03

I do not understand the rational behind the type of, why the change from the previous definition: newForeignPtr :: Ptr a -> FunPtr (Ptr a -> IO ()) -> IO (ForeignPtr a) suppose I have a function foreign import ccall fooalloc :: IO (Ptr Foo) and foreign import ccall foofree :: Ptr Foo -> IO () now how do I create a foreign pointer without either using: foreign import ccall "wrapper" mkfoofree :: (Ptr Foo->IO())->IO (FunPtr (Ptr Foo->IO())) or declaring foreign import ccall "&foofree" foofree :: FunPtr(Ptr Foo->IO()) after all why should I need to import and then re-export or suppose I want foofree as a function or for that matter suppose Foo were a simple memory alloc which could be freed from haskell, why should we be forced to wrap a haskell function to create a foreign pointer, or am I missing something ? _________________________________________________________________ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus

1 0

newForeignPtr
by m v 03 Feb '03

03 Feb '03

I do not understand the rational of the type of: newForeignPtr :: Ptr a -> FunPtr (Ptr a -> IO ()) -> IO (ForeignPtr a) suppose I have a c function foreign import ccall fooalloc :: Ptr Foo and another foreign import ccall foofree :: Ptr Foo -> IO () now I want to combine fooalloc and foofree to create a Foreign pointer without wrapping foofree again, as in: foreign import "wrapper" mkfoofree :: FunPtr (Ptr Foo -> IO ()) which I think counter productive - why should I import a function only to have to re-export it again ! previouse proposals did not require this - or am I missing something ? why should the finaliser be a foreign function ? _________________________________________________________________ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail

2 1