[GHC] #11713: STM Finalizers
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: Runtime | Version: 8.1 System | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- This is some work I did as part of my master's thesis [1]. It's an extension to GHC's STM implementation. The main idea is to introduce a new primitive function {{{#!hs atomicallyWithIO :: STM a -> (a -> IO b) -> IO b }}} Like the existing atomically operation, atomicallyWithIO performs an STM computation. Additionally, it takes a '''finalizer''', which is an arbitrary I/O action that can depend on the result of the STM computation, and combines it with the transaction in such a way that: 1. The finalizer is only performed if the STM transaction is guaranteed to commit. 2. The STM transaction only commits if the finalizer finishes without raising an exception. A detailed specification, including an operational semantics, as well as a discussion of the implementation, can be found in chapter 2 of my thesis [1]. ---- I actually did this work quite some time ago, and even announced it on the mailing list [2] to some positive feedback, but then never got around to actually submitting a patch. Last week Charles Strahan wrote me on GitHub and got me motivated again, so here I am :) I'm submitting a code review to Phabricator right now and will link it to this ticket. ---- [1] https://github.com/mcschroeder/thesis [2] https://mail.haskell.org/pipermail/haskell-cafe/2015-July/120589.html -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): (2) looks as if it'd impose some pretty heavy constraints on the implementation. To implement it you have to hold locks on all the TVars from the transaction (to satisfy (1)). These would usually be held only very briefly. But the IO action could do anything, including more transactions. Which could take arbitrarily long, and themselves would deadlock if they affect any of those TVars. A multi-CAS implementation for transaction commit would not work any more. I'm unconvinced. I've always wanted some `onRetry` thing that would run a specified IO action if transaction retries, to allow users to check for starvation and take evasive action. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: mc.schroeder Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2011 Wiki Page: | -------------------------------------+------------------------------------- Changes (by mc.schroeder): * owner: => mc.schroeder * differential: => Phab:D2011 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: mc.schroeder Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2011 Wiki Page: | -------------------------------------+------------------------------------- Comment (by mc.schroeder): Actually, instead of simply locking all TVars involved in a finalizing transaction, my implementation "freezes" them, meaning they can still be read by other transactions (preserving read-parallelism). Only when another transaction tries to commit a write to a frozen TVar is its thread blocked. It is woken up again when the finalizer that froze the TVar has finished. Of course, this could take a while (potentially forever), and all transactions waiting on frozen TVars are stalled in the meantime. However, I would say that this is exactly the point of finalizers: they create a serialization point. As for the potential of deadlocks due to nested transactions: a deadlock could only occur if the inner transaction writes a TVar used by the outer transaction. In my opinion, this is a rather artifical scenario, that is unlikely to happen by accident. It will also always be detected by the runtime system, which would then gracefully abort the transaction with a BlockedIndefinitelyOnSTM exception. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: mc.schroeder Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2011 Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Well, it may be unlikely but both the specification and the implementation must take care of these cases. Is all this complexity justified by the use-cases. Maybe, but I am not yet convinced. Eg for your serialisation example, I'd just transactionally commit the result to a buffer of things to be serialised by single serialisation thread. That is, in effect, what you are doing by other means. Let's see what Simon M and others GHC devs/users say. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: mc.schroeder Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2011 Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar): This looks like a big extra burden on the implementation - an extra field in every TVar, two fields in the atomically frame, and lots of complexity. I'd really rather avoid this if we can. e.g. for serialization, I would do it exactly the way that Simon suggests, the transaction should add its serialization instructions to a TQueue or something that is processed by a separate thread. If you want to also have the current thread wait until the serialization has completed, then you can program that too using a TVar. This also allows more concurrency than your finalizer, because it doesn't have to freeze any TVars. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: mc.schroeder Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2011 Wiki Page: | -------------------------------------+------------------------------------- Comment (by fryguybob): I also am not convinced that the cost of the implementation changes are worth the feature. It isn't completely clear to me how to write the IO actions safely. Programmers might not always know what `TVar`s they are depending on and there also might be some other ways to deadlock due to transitive dependencies on `TVar`s (Data invariants, blocked transactions). I will take some time to look through everything very closely and hopefully run some performance comparisons as well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11713: STM Finalizers -------------------------------------+------------------------------------- Reporter: mc.schroeder | Owner: mc.schroeder Type: feature request | Status: new Priority: normal | Milestone: Component: Runtime System | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2011 Wiki Page: | -------------------------------------+------------------------------------- Changes (by fryguybob): * cc: fryguybob (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11713#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC