February 2003 - FFI - Haskell.org

Finalizers, again!
by Manuel M T Chakravarty 06 Mar '03

06 Mar '03

I have just read Hans Boehm's POPL paper on finalizers. His suggestion for the use of finalizers in single-threaded systems is to provide a `runFinalizers' routine, instead of relying on the asynchronous execution that, as established, requires support for concurrency. I am not sure whether we have given this option really serious consideration. Why may this option be interesting? It may allow us to have Haskell finalizers after all. As Boehm explains, ``finalizers generally fall into two categories: Those that simply deallocate operating system resources and those that touch other user data structures.'' It seems that we currently, probably due to our FFI bias, concentrate on the former kind. The main benefit of Haskell finalizers, apart from convenience, is the ability to process Haskell structures. As I understand Boehm, we might adopt the following design: * Finalizers are normal Haskell routines. * When the GC discovers that a finalizer becomes enabled, it is added into a list of enabled finalizers (but not run in a single-threaded system). * When the routine runFinalizers :: IO () is called, it runs the finalizers in the list of enabled finalizers one after the other. * Systems supporting pre-emptive concurrency *may* run finalizers at any time after they become enabled, but they must run them when `runFinalizers' is called (similar to `performGC'). An attractive property of this design is that it provides a smooth transition from single-threaded systems to systems providing pre-emptive concurrency. The requirement to invoke `runFinalizers' explicitly might not be too burdensome in practice as code making heavy use of the FFI and finalizers probably spends a lot of time in the IO monad with the opportunity to insert calls to `runFinalizers'. Opinions? Cheers, Manuel PS: Sorry for bringing finalizers up again, but if there is an opportunity to improve the current design, I think, we should consider it.

2 1

Re: Bound Threads
by Sven Panne 01 Mar '03

01 Mar '03

I just skimmed over Wolfgang's PDF and things look quite reasonable. Just to make sure I have understood everything correctly: To make HOpenGL work in the presence of a threaded RTS, the only places which need a change are the stub factories for GLUT callbacks, where a "bound" attribute is required now. What about the foreign import of glutMainLoop (the main native dispatcher for GLUT, which calls the Haskell callbacks mentioned above when an event occurs and blocks the rest of the time)? It is currently "safe", is this OK? Or should it be "threadsafe"? I must admit that I can't remember the exact semantic distinction between those two attributes anymore... :-} Cheers, S.

2 1

Finalizers: the Legend that Won't Die
by George Russell 28 Feb '03

28 Feb '03

I geve the example > let > x = seq (unsafePerformIO runFinalizers) y > finalizer = [... something involving x ...] > [... attach finalizer to something ..] > [... something becomes free, putting finalizer on the to-be-done list ...] > [.., computation requiring x ...] to which Malcolm replied (snipped) > But isn't this just the same as any other program that contains a > blackhole fault? i.e. the program is logically incorrect. No I don't think it is logically incorrect. There's nothing terribly illogical about the user thinking in the middle of a long pure computation "Let's free up some space by running the finalizers". Nor indeed is there anything illogical about unsafePerformIO'ing some other operation which indirectly involves running finalizers. (You might logically do this at the end of some call to an pure-function-like external function, where you need to construct some complex data structure requiring finalizers, and think it would be a good idea to run the finalizers immediately afterwards.) Of course the example I gave is an extremely artificial one, but I see no reason why similar but much more complicated cases should not arise in "real life". > What is more, the runtime system will probably tell you "Blackhole detected!" > or "<<loop>>" or something, which is helpful. Well I suppose it's more helpful than hitting you over the head with a blunt instrument but it's not *very* helpful is it? You see "<<loop>>". The first reaction would be to assume you have a normal circularity. After a few happy minutes, or hours, discovering that this is not the case, you might work out that finalizers are somehow responsible, at which point you have to detect an arbitrarily complex cycle involving finalizers and partially-evaluated values. To make this more fun, you are relying on various unspecified properties of the garbage-collector and whatever external programs you are calling, meaning that it's very possible the bug will only occur sporadically, depending on the amount of memory the Haskell process is able to malloc or whatever.

1 0

Finalizers: the Legend that Won't Die
by George Russell 28 Feb '03

28 Feb '03

Manuel wrote (snipped) > I have just read Hans Boehm's POPL paper on finalizers. His > suggestion for the use of finalizers in single-threaded > systems is to provide a `runFinalizers' routine, instead of > relying on the asynchronous execution that, as established, > requires support for concurrency. I am not sure whether we > have given this option really serious consideration. I can only see one problem with this. Suppose you have the following sequence of events: let x = seq (unsafePerformIO runFinalizers) y finalizer = [... something involving x ...] [... attach finalizer to something ..] [... something becomes free, putting finalizer on the to-be-done list ...] [.., computation requiring x ...] Then during the computation of x the following things are liable to happen: (1) Evaluation of x begins. x is blackholed. (2) runFinalizers is invoked. (3) "finalizer" is begun. (4) finalizer needs value of x. Ooops, what do we do now? Is it safe to run runFinalizers anywhere except in the main IO trunk? Otherwise you always have the risk of coming a cropper on blackholed values, don't you? George

2 1

Bound Threads
by Wolfgang Thaller 17 Feb '03

17 Feb '03

Hello Everyone, I've added new sections to the document about bound threads (in CVS, haskell-report/ffi/threads.tex, version 1.8). The new sections are "Extension to the FFI Syntax", "Library Additions" and "Issues". Please have a look especially at the "Issues" section, I need some feedback! Cheers, Wolfgang P.S.: I've put a PDF of the file at http://www.uni-graz.at/imawww/haskell/threads.pdf

1 0

haskell.cs.yale.edu
by Jane Brooks 14 Feb '03

14 Feb '03

Hi, I thought you might be interested in getting in-depth knowledge about your web audience and web traffic patterns in a reliable and cost-effective way. Stop Guessing - Start Knowing! - CoolStats measures web site traffic and online behavior of your visitors. - CoolStats will help you understand how to optimize your site to meet the needs of your visitors. - You get access to detailed, real-time statistical analysis of your web pages - 24 hours a day. Click at http://www.coolstats.com/viewdemo/index.html to view Online Demo. - CoolStats is the ultimate real-time tracking solution for small and mid-sized businesses. - 100% accuracy by measuring activity at the client, not via server based log files. - The fee of $19.95 is minimal compared to what it would cost you to run a tracking service yourself! Why CoolStats? - no programming to do - no servers to maintain - no software applications to install Special Offer! Now Only $19.95/month (Usual Price/$29.95). Click at http://p1j2m3a4.pdhost.com/pdsvr/www/r?1000007972.1466.15.VFFMdRcoZLV2Dh to Sign Up now! "We needed to make business sense out of our web visitor behavior - CoolStats delivers first-class graphical reports that help us continuously improve and optimize our website to match the requirements of our target audience." BRYAN KASHILIN, BOSTON Click at http://www.coolstats.com/product/customerref.html to check what other customers say about us! For more information about our website tracking services, please visit our website or contact me directly at the below email. I look forward to hearing from you soon. Best regards, Jane Brooks CoolStats Support Email: jane_brooks(a)coolstats.com http://www.coolstats.com Don't be the last one to know! ----------------------------------------------------------------------- This message has been brought to ffi(a)haskell.org. If you do not wish to receive anymore emails, please follow the opt-out instruction below. We apologize for any inconvenience. http://p1j2m3a4.pdhost.com/pdsvr/www/r?1000007972.1466.3.j5I+FKuMxGplsp

1 0

RE: safe and threadsafe
by Simon Peyton-Jones 11 Feb '03

11 Feb '03

| *) I am not convinced that thread synchronization should be done by the | FFI Me neither! It was news to me that anyone was *relying* on the (implementation-specific) fact that a "safe" call blocks other Haskell threads. I'd always regarded "safe" as just like "threadsafe" except that it was a bit more efficient, but with the penalty that blocking in the foreign call could gum up all Haskell threads. As Wolfgang says, the atomicity of "safe" is extremely dodgy. I suppose that "unsafe" could also mean "guaranteed atomic". After all, it can't call in, nor do GC, etc. And the RTS is in a state where nothing else can execute anyway. Even then, I agree with Wolfgang that it's very smelly. If you need atomicity, use an MVar, no? You may not want two concurrent GTK calls (although I don't see why not), but a GTK call concurrent with some other I/O would be ok. If two threads are drawing in two different GTK windows, couldn't they be concurrent? simon

1 0

RE: safe and threadsafe
by Simon Marlow 11 Feb '03

11 Feb '03

> I don't think it was ever the intention that 'safe' should have a > guaranteed serialisation property. I think the idea was that > 'threadsafe' was the most desirable, with 'safe' and 'unsafe' only > available for use if you wanted more efficiency and had some separate > guarantees that the extra efficiency was not at the expense of > correctness. > > To be completely explicit, I think that increasing the safety level of > any foreign import should never make the program fail. If I recall correctly, the motivation for keeping "safe" was that we wanted to be able to make calls into non-threadsafe C libraries. Which, incedentally, would break the property that Simon mentions above: a non-threadsafe library would *require* foreign imports to be labelled "safe" rather than "threadsafe". However, at the time I don't think we appreciated the implementation diffiulties arising from "safe". Also, Wolfgang has pointed out that you can simulate serialisation in Haskell using MVars. Cheers, Simon

3 2

Repeated hs_init()/hs_exit()
by Simon Marlow 10 Feb '03

10 Feb '03

I'm implementing the latest hs_init()/hs_exit() interface in GHC, and came across an ambiguity or omission in the spec. We're clear that this sequence should be allowed: hs_init(..) hs_init(..) hs_exit() hs_exit() but what about hs_init(..) hs_exit() hs_init(..) hs_exit() That is, should the Haskell system be able to start itself up again after shutting down? It looks like this is desirable from a modularity viewpoint: eg. a C program initialises library A which uses Haskell internally, then de-initialises library A, then initialises library B which also uses Haskell internally. Unfortunately, this is going to require quite a bit of extra work in GHC to get right, and it looks like I'm going to have to examine a lot of code to make sure it is "double-init-safe". Thoughts? Cheers, Simon

4 4

RE: safe and threadsafe
by Simon Peyton-Jones 07 Feb '03

07 Feb '03

| I have recently spent some time improving GHC's support for | "threadsafe" foreign calls. | As a side effect of fixing a crashing bug, I made "safe" behave in | exactly the same way as "threadsafe". That's a positive advantage, provided there isn't a massive efficiency cost. I'm all for nuking 'threadsafe' if we can! | Are safe calls _guaranteed_ to block all other haskell threads and | prevent garbage collection, | or is that an implementation detail? Are unsafe calls guaranteed to do | so? {In a SMP implementation, | this might require additional interthread messaging}. | What should happen when C code running in a separate OS thread calls a | foreign exported | function while the Haskell Runtime is blocked on a safe call? Would all these questions go away if we made safe=threadsafe? | Some people have said that "safe" calls are intended for serializing | access to foreign libraries | that are not multithread-safe. I think that MVars should be used for | such purposes. Blocking the | entire Haskell Runtime for the duration of a "safe" call is overkill, | but it's not always enough: | Many old C libraries keep state in global variables from one call to | the next. I don't think it was ever the intention that 'safe' should have a guaranteed serialisation property. I think the idea was that 'threadsafe' was the most desirable, with 'safe' and 'unsafe' only available for use if you wanted more efficiency and had some separate guarantees that the extra efficiency was not at the expense of correctness. To be completely explicit, I think that increasing the safety level of any foreign import should never make the program fail. Simon

2 1