On 20 November 2004 23:02, Benjamin Franksen wrote:
I am using Foreign.Concurrent.newForeignPtr and touchForeignPtr inside the finalizers to express liveness dependencies as hinted to by the documentation. This doesn't seem to work, though, or at least I can't see what I've done wrong. I attached a test module; compiled with ghc -fglasgow-exts --make TestForeignTouchBug.hs, ghc version 6.2.2, this gives
.../foreigntouchbug > ./a.out <hit enter here> before finalizing A after finalizing A before finalizing B after finalizing B <hit enter here>
I expected the order of the finalizer calls be be the other way around, since the finalizer for the Bs explicitly touches the A value.
The problem is that the runtime is running all outstanding finalizers at the end of execution, without regard for GC dependencies like the ones introduced by touchForeignPtr. I've been planning to remove this automatic running of finalizers for other reasons. However, then you will get absolutely no guarantee that your finalizer will ever run at all (indeed, the property isn't always true right now, but it is usually true). Let me share with you something that I've come to appreciate over the last few years: Finalizers are almost always not the right thing. Finalizers look terribly attractive, but often lead to a huge can of worms - best avoided if at all possible. Cheers, Simon
Some thoughts on this, Whilst I agree that finalizers are best avoided, it must be possible to order the finalizers for running on exit... Perhaps a simple multi-pass algorith would do? (ie: run all finalizers that do not refer to other objects with finalizers - repeat until no objects with finalizers remain. What can be done about loops I am not sure (where A refers to B which refers to A and both have finalizers)... The alternative would seem to be writing your finilizers such that they first call the finalizers on any refered objects which haven't been finalized yet (In the case of loops if we specify running the finalizers on any objects refered to on which the finalizer has not been entered yet - it should work too) Whatever happens I think it must make sure all system resources allocated by a program are freed on exit - otherwise the machine will have a resource leak and will need rebooting eventually. Keean. Simon Marlow wrote:
The problem is that the runtime is running all outstanding finalizers at the end of execution, without regard for GC dependencies like the ones introduced by touchForeignPtr.
I've been planning to remove this automatic running of finalizers for other reasons. However, then you will get absolutely no guarantee that your finalizer will ever run at all (indeed, the property isn't always true right now, but it is usually true).
Let me share with you something that I've come to appreciate over the last few years:
Finalizers are almost always not the right thing.
Finalizers look terribly attractive, but often lead to a huge can of worms - best avoided if at all possible.
Cheers, Simon _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Keean Schupke wrote:
[...] Whatever happens I think it must make sure all system resources allocated by a program are freed on exit - otherwise the machine will have a resource leak and will need rebooting eventually.
That's an OS task IMHO, not really the task of an RTS. Looks like you're working on WinDoze... (sorry, couldn't resist :-) Cheers, S.
Nope there are some unix resources that c exit routines do not free like semaphores. Sven Panne wrote:
Keean Schupke wrote:
[...] Whatever happens I think it must make sure all system resources allocated by a program are freed on exit - otherwise the machine will have a resource leak and will need rebooting eventually.
That's an OS task IMHO, not really the task of an RTS. Looks like you're working on WinDoze... (sorry, couldn't resist :-)
Cheers, S.
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Semaphores (SYSV style) are not freed automatically. Currenly I am using C's at_exit funtion (which is even called on a signal)... Perhaps this is the way to deal with foreign resources... bracket notation and at_exit to clean up on signals? Keean. Sven Panne wrote:
Keean Schupke wrote:
Nope there are some unix resources that c exit routines do not free like semaphores.
Which library/OS calls do you mean exactly? I always thought that files are the only resources surviving process termination.
Cheers, S.
Keean Schupke wrote:
Nope there are some unix resources that c exit routines do not free like semaphores.
"C exit routines" aren't responsible for freeing OS resources; the OS
is.
The fact that the SysV IPC objects aren't freed on exit is
intentional; they are meant to be persistent. For the same reason, the
OS doesn't delete upon termination any files which the process
created.
--
Glynn Clements
Glynn Clements wrote:
"C exit routines" aren't responsible for freeing OS resources; the OS is.
The fact that the SysV IPC objects aren't freed on exit is intentional; they are meant to be persistent. For the same reason, the OS doesn't delete upon termination any files which the process created.
Right, which is why if you want to clean up temporary files, or temporary semaphores the OS doesn't do it for you, and you need to put some routine inplace to do it (using at_exit)... It seems this is the only way to guarantee something gets run when a program exits for whatever reason. Keean.
Keean Schupke wrote:
"C exit routines" aren't responsible for freeing OS resources; the OS is.
The fact that the SysV IPC objects aren't freed on exit is intentional; they are meant to be persistent. For the same reason, the OS doesn't delete upon termination any files which the process created.
Right, which is why if you want to clean up temporary files, or temporary semaphores the OS doesn't do it for you, and you need to put some routine inplace to do it (using at_exit)... It seems this is the only way to guarantee something gets run when a program exits for whatever reason.
There isn't any way to *guarantee* that something is run upon
termination. The program may be terminated due to SIGKILL (e.g. due to
a system-wide lack of virtual memory). If you run out of stack, you
may not be able to call functions to perform clean-up.
Also, if the program crashes, handling the resulting SIGSEGV (etc) is
likely to be unreliable, as the memory containing the resource
references may have been trashed. Calling remove() on a filename which
might have been corrupted is inadvisable.
Also, at_exit() isn't standard. atexit() is ANSI C, but that is only
supposed to be called for normal termination (exit() or return from
main()), not for _exit() or fatal signals.
--
Glynn Clements
Although, this does remind me. A suitable atexit-equivalant in the haskell libraries would be much appreciated. John -- John Meacham - ⑆repetae.net⑆john⑈
How can I put this, it is a "best efforts" approach - it does its best to run the finalizers, even after a segmentation fault... however some of the pointers may be messed up... If the cleanup causes a segmentation fault (sometimes called a double bus fault) then we have to abandon the cleanup. With our semaphores we are actually quite clever, in that the at_exit routine is set containing the semaphore ID as a constant, so it should always be runnable... What I am suggesting is that every effort should be made to clean up, just because there are some circumstances (which should be rare) in which it cannot happen, does not mean you shouldn't try... Keean Glynn Clements wrote:
There isn't any way to *guarantee* that something is run upon termination. The program may be terminated due to SIGKILL (e.g. due to a system-wide lack of virtual memory). If you run out of stack, you may not be able to call functions to perform clean-up.
Also, if the program crashes, handling the resulting SIGSEGV (etc) is likely to be unreliable, as the memory containing the resource references may have been trashed. Calling remove() on a filename which might have been corrupted is inadvisable.
Also, at_exit() isn't standard. atexit() is ANSI C, but that is only supposed to be called for normal termination (exit() or return from main()), not for _exit() or fatal signals.
On Monday 22 November 2004 14:45, Simon Marlow wrote:
On 20 November 2004 23:02, Benjamin Franksen wrote:
I am using Foreign.Concurrent.newForeignPtr and touchForeignPtr inside the finalizers to express liveness dependencies as hinted to by the documentation. This doesn't seem to work, though, or at least I can't see what I've done wrong. I attached a test module; compiled with ghc -fglasgow-exts --make TestForeignTouchBug.hs, ghc version 6.2.2, this gives
.../foreigntouchbug > ./a.out <hit enter here> before finalizing A after finalizing A before finalizing B after finalizing B <hit enter here>
I expected the order of the finalizer calls be be the other way around, since the finalizer for the Bs explicitly touches the A value.
The problem is that the runtime is running all outstanding finalizers at the end of execution, without regard for GC dependencies like the ones introduced by touchForeignPtr.
I understand that there are situations where finalizers cannot be guaranteed to run: First because of an unconditional termination signal (SIGKILL), second because of circular dependencies resulting in a deadlock. I don't understand why it is necessary to "performGC" explicitly, in order to run finalizers at *normal* program termination and without a deadlock. BTW, the sensible thing to do in this case would be to throw an exception whenever a deadlock condition is detected. (That is, if it can be detected.) However, what I don't understand is why touchForeignPtr is not honored in my example program: Note that the output text lines from the finalizers appear *before* the last action in the program (which is a second getChar). The finalizers *are* called by the GC, and still the order is wrong.
I've been planning to remove this automatic running of finalizers for other reasons. However, then you will get absolutely no guarantee that your finalizer will ever run at all (indeed, the property isn't always true right now, but it is usually true).
It is unclear to me what "usually" means, here. Wouldn't it be better to state the conditions under which they are called resp. aren't? Documenting the current state of affairs would be to say - Finalizers are *not* guaranteed to run (instead they "usually" are, whatever that means). - Calling touchForeignPtr does *not* guarantee that the targeted foreignPtr is not finalized before this call. This would make it obvious that finalizers are not a reliable tool (which they aren't, at the moment).
Let me share with you something that I've come to appreciate over the last few years:
Finalizers are almost always not the right thing.
Finalizers look terribly attractive, but often lead to a huge can of worms - best avoided if at all possible.
I am ready to believe this, but I am nevertheless somewhat disturbed. I use finalizers to automatically reclaim resources from a foreign library. If I don't use them, these resources must be reclaimed explicitly. This is bad for two reasons: (1) The programmer has to remember to reclaim resources. (2) Handles for such resources become invalid after reclamation. If finalizers are not the right thing, what else is? Ben -- Top level things with identity are evil. -- Lennart Augustsson
If finalizers are not the right thing, what else is?
I've found that when writing an interface to a C library that requires resource management, it's much better to use the withX (see Control.Exception.bracket) style of function than to use finalizers - programs are much easier to reason about and debug. Abe
Abraham Egnor wrote:
I've found that when writing an interface to a C library that requires resource management, it's much better to use the withX (see Control.Exception.bracket) style of function than to use finalizers - programs are much easier to reason about and debug.
... and have a much more deterministic behaviour regarding resource consumption. This is a very important point which is often forgotten. Cheers, S.
On Monday 22 November 2004 18:55, Sven Panne wrote:
Abraham Egnor wrote:
I've found that when writing an interface to a C library that requires resource management, it's much better to use the withX (see Control.Exception.bracket) style of function than to use finalizers - programs are much easier to reason about and debug.
... and have a much more deterministic behaviour regarding resource consumption. This is a very important point which is often forgotten.
Ok, the 'withX' pattern solves both of the problems I mentioned and in addition is more deterministic (and, using bracket, also exception safe). Maybe I should use it instead of finalizers. However, I wonder if it is general enough to be the only method exported from a library. I doubt it, otherwise there would be no 'openFile' and 'hClose' in System.IO but only a 'withFile'. Do you recommend supplying 'newX' and 'freeX' in addition to 'withX'? Or should I rather use 'newX' plus finalizers for those (rare?) cases where 'withX' is not general enough? Ben -- Top level things with identity are evil. -- Lennart Augustsson
participants (7)
-
Abraham Egnor -
Benjamin Franksen -
Glynn Clements -
John Meacham -
Keean Schupke -
Simon Marlow -
Sven Panne