withFile defeats dead handle closure
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can. All that's required is to capture the weak reference that holds the handle's finalizer. Then instead of closing the handle in the end (or on exception) using hClose, withFile can run the handle's finalizer. Should we do this? Or does it run too much against the idea that handles should be managed as explicitly as possible?
I would be afraid of making this change. File descriptors are a scarce resource, and relying on non-deterministic GC behavior to free them is big -1. Also finalizers are not free in current GC. (Though, you probably cannot won't that many open files that it would matter). - Oleg On 30.12.2020 21.23, David Feuer wrote:
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can. All that's required is to capture the weak reference that holds the handle's finalizer. Then instead of closing the handle in the end (or on exception) using hClose, withFile can run the handle's finalizer. Should we do this? Or does it run too much against the idea that handles should be managed as explicitly as possible?
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Finalizers aren't free, but every handle already has a finalizer, so that's
not an extra cost.
On Wed, Dec 30, 2020, 2:50 PM Oleg Grenrus
I would be afraid of making this change. File descriptors are a scarce resource, and relying on non-deterministic GC behavior to free them is big -1.
Also finalizers are not free in current GC. (Though, you probably cannot won't that many open files that it would matter).
- Oleg
On 30.12.2020 21.23, David Feuer wrote:
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can. All that's required is to capture the weak reference that holds the handle's finalizer. Then instead of closing the handle in the end (or on exception) using hClose, withFile can run the handle's finalizer. Should we do this? Or does it run too much against the idea that handles should be managed as explicitly as possible?
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Although it might be nice to have a weak-reference-based alternative to withFile, I think it would be best to keep the behavior of withFile as predictable as possible (i.e. the current behavior), since closing a file can be semantically significant. For instance, if someone is using /dev/watchdog to make their system automatically reboot when their userspace daemon fails, closing the file handle early would cause the watchdog functionality to be disabled, which is almost certainly not what someone would mean by `withFile "/dev/watchdog" $ \_ -> blah`. On 12/30/20 3:47 PM, Henning Thielemann wrote:
On Wed, 30 Dec 2020, David Feuer wrote:
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can.
Why should we? _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
Although it might be nice to have a weak-reference-based alternative to withFile, I think it would be best to keep the behavior of withFile as predictable as possible (i.e. the current behavior), since closing a file can be semantically significant.
Right, if I open and close and re-open a file, it must be closed before I can re-open it.
I think David's suggestion may have been that withFile would still close the file at the end of the block, but would *also* close it if the handle died. On 12/30/20 4:05 PM, Henning Thielemann wrote:
On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
Although it might be nice to have a weak-reference-based alternative to withFile, I think it would be best to keep the behavior of withFile as predictable as possible (i.e. the current behavior), since closing a file can be semantically significant.
Right, if I open and close and re-open a file, it must be closed before I can re-open it.
While maybe not directly related to this, for many small changes suggested to base the answer is often: go write a library and see how it works out. This is often for fairly stylistic changes to "fringe" components of base. Here we're talking about a full blown operational change to something incredibly common. If anything, I think it would have to subject to the same process - try it out, really use it anger, see what breaks and - hopefully! - what gets better. On Wed, 30 Dec 2020, at 9:18 PM, Ryan Trinkle via Libraries wrote:
I think David's suggestion may have been that withFile would still close the file at the end of the block, but would *also* close it if the handle died.
On 12/30/20 4:05 PM, Henning Thielemann wrote:
On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
Although it might be nice to have a weak-reference-based alternative to withFile, I think it would be best to keep the behavior of withFile as predictable as possible (i.e. the current behavior), since closing a file can be semantically significant.
Right, if I open and close and re-open a file, it must be closed before I can re-open it.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
To be clear, I wasn't formally proposing a change. I was trying to get
a sense of what people thought of the notion before going down too
many rabbit holes. I think Ryan Trinkle's concern is quite strong
enough to kill this idea as a change to `withFile` itself.
On Wed, Dec 30, 2020 at 4:26 PM Oliver Charles
While maybe not directly related to this, for many small changes suggested to base the answer is often: go write a library and see how it works out. This is often for fairly stylistic changes to "fringe" components of base. Here we're talking about a full blown operational change to something incredibly common. If anything, I think it would have to subject to the same process - try it out, really use it anger, see what breaks and - hopefully! - what gets better.
On Wed, 30 Dec 2020, at 9:18 PM, Ryan Trinkle via Libraries wrote:
I think David's suggestion may have been that withFile would still close the file at the end of the block, but would *also* close it if the handle died.
On 12/30/20 4:05 PM, Henning Thielemann wrote:
On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
Although it might be nice to have a weak-reference-based alternative to withFile, I think it would be best to keep the behavior of withFile as predictable as possible (i.e. the current behavior), since closing a file can be semantically significant.
Right, if I open and close and re-open a file, it must be closed before I can re-open it.
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
On Wed, Dec 30, 2020 at 02:23:20PM -0500, David Feuer wrote:
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can. All that's required is to capture the weak reference that holds the handle's finalizer. Then instead of closing the handle in the end (or on exception) using hClose, withFile can run the handle's finalizer. Should we do this? Or does it run too much against the idea that handles should be managed as explicitly as possible?
I'm not sure what you mean by "keeps alive", it passes the handle to the action, and closes it when the action returns, but the action, can (if it so chooses) also close the handle in a more timely manner, because "hClose" is idempotent. So it does not seem like any change is needed here... Am I missing something? The below compiles and runs, with no errors: module Main (main) where import System.IO main :: IO () main = do withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f -- Viktor.
The handle will not be closed early if it becomes unreachable within
the user action (without the user closing it). It sounds like the
consensus is that that's how it should be, so I'll leave that well
enough alone.
On Wed, Dec 30, 2020 at 4:32 PM Viktor Dukhovni
On Wed, Dec 30, 2020 at 02:23:20PM -0500, David Feuer wrote:
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can. All that's required is to capture the weak reference that holds the handle's finalizer. Then instead of closing the handle in the end (or on exception) using hClose, withFile can run the handle's finalizer. Should we do this? Or does it run too much against the idea that handles should be managed as explicitly as possible?
I'm not sure what you mean by "keeps alive", it passes the handle to the action, and closes it when the action returns, but the action, can (if it so chooses) also close the handle in a more timely manner, because "hClose" is idempotent. So it does not seem like any change is needed here... Am I missing something?
The below compiles and runs, with no errors:
module Main (main) where import System.IO
main :: IO () main = do withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f
-- Viktor. _______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
On Wed, Dec 30, 2020 at 04:39:52PM -0500, David Feuer wrote:
The handle will not be closed early if it becomes unreachable within the user action (without the user closing it). It sounds like the consensus is that that's how it should be, so I'll leave that well enough alone.
I see, the converse of what I guessed you were saying. I agree this is not worth changing. It would amount to relying on GC to close the handle perhaps a bit earlier, and my take is that any such reliance is a bug. If the action keeps executing long after the file is no longer needed then the programmer used "withFile" in too broad a context and needs to narrow its scope to just the portion of the code that is actually using the file. -- Viktor.
Am Mi., 30. Dez. 2020 um 22:47 Uhr schrieb Viktor Dukhovni < ietf-dane@dukhovni.org>:
[...] It would amount to relying on GC to close the handle perhaps a bit earlier, and my take is that any such reliance is a bug.[...]
+1 As a general rule of thumb: Automatic memory management is good for, well, memory management, but nothing else. Quite the opposite: Most of the time it's horrible for other tasks. There are tons of points in the design space of automatic memory management with vastly different trade-offs regarding throughput, latency, concurrency etc., so tying other "heavy" resources like file descriptors to automatic memory management is a very, very bad idea. People had cunning ideas about how to (ab-)use finalizers etc. for decades, but basically all of them turned out to be bad. Hans-J. Boehm has written extensively about this, and his papers are still worth reading today.
participants (7)
-
David Feuer -
Henning Thielemann -
Oleg Grenrus -
Oliver Charles -
Ryan Trinkle -
Sven Panne -
Viktor Dukhovni