[GHC] #10841: Run handler on STM retry
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Keywords: stm | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- It would be nice to have a function like: {{{#!hs -- | Perform a series of STM actions atomically. -- -- Whenever the transaction retries, run the handler. If it doesn't return 'Nothing', -- stop the retrying and return the value given by the handler. atomicallyWithRetryHandler :: STM a -> IO (Maybe a) -> IO a }}} This would allow tracking retry statistics, printing debug messages on each retry, etc. without resorting to unsafeIOToSTM which can't safely do things like printing. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by shlevy): Alternate type signature: {{{#!haskell (a -> STM b) -> a -> IO a -> IO b }}} The idea being that a potentially different transaction is run every time (or not at all if the function wants to). But that may be just asking for people to shoot themselves in the foot. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by simonpj): * cc: ryates@… (added) Comment: I agree; something like this is sorely needed, so that users can take control of what to do if the transaction starves. Curiously I was talking to [http://www.cs.rochester.edu/u/ryates/ Ryan Yates] about this very thing at ICFP last week. He's been doing work on GHC's STM implementation, and perhaps this retry-handler thing could form part of it? Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by shlevy): A few more thoughts looking back on this a few days later: a) It might be nice to have some structured information from the runtime explaining why a transaction needed to retry, and b) perhaps the only primitive needed is just a "try once" that can be looped as-desired by the user. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by fryguybob): I don't think it is terribly difficult to implement something like this. The retry handler would be stuffed on the atomically frame and when `retry` reaches that frame it could easily be executed instead of blocking. I do wonder though if such a thing can be used effectively and what additional information would make it more effective. Can anyone shed some light on what particular deficiency in STM or STM applications leads to wanting this? I can imagine some other information that could make `retry` align better with intentions like a version of `retry` to invoke when the condition encountered is rare and expected to be resolved quickly making it harder to starve due to a round trip through the scheduler. Your application is also free to catch `retry` throw an exception instead with any internal details of your transaction you want. You don't get a wake up when a `TVar` changes with this option, but you can try again right away run a different transaction or delay by some other means. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): My use-case is this. Suppose a long-running transaction is repeatedly aborted because short-running transactions get in first. It might "starve". Then what? At the moment there is literally nothing the programmer can do because starvation is un-observable. Adding a retry handler makes it observable. Example: a librarian wants to re-organise the science fiction shelves of the library, but his long-running "reorganise" transaction is always aborted by someone doing a short-running "borrow-book" transaction. There is no automatic solution to this; it's an application-domain problem. For example, if the reorg became too long delayed, the librarian might temporarily close the SF section to borrowers. In the transactional setting, the retry handler might (in a separate, preliminary transaction) take a lock on the section, which borrowers must respect. None of this can happen unless the programmer can observe excessive retries and code a response. For example, she could count retries, and use an alternative strategy if the count got too high. (Actually, it might be convenient if the retry count was passed to the hander, so that the handler didn't need to create the state-and-increment stuff to manage the count.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by shlevy): fryguybob perhaps there's some confusion due to my calling it a "retry" handler? I don't mean merely to catch when the `retry` STM primitive is run, I also mean transaction aborts as simonpj mentioned. Unrelated to that: I realize now that my "try once" primitive won't work because there's no good way for the programmer to figure out when stm state has changed enough in other threads to be worth trying again, so the runtime does have to be in control of the looping. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by fryguybob): Yes, the terminology is confusing. There are things you can do inside the current STM to do this all at the user side (I didn't actually type check this): {{{#!hs atomicWithAbortPolicy :: Int -> IO (Maybe a) -> STM a -> IO a atomicWithAbortPolicy tryCount policy action = loop where loop :: IO a loop = do c <- newIORef 0 -- Thread local! res <- atomically $ do n <- unsafeIOToSTM $ readIORef c -- Thread local! if n > tryCount then return Nothing -- Commit transaction nothing in the read-set, so it will -- always succeed. else do unsafeIOToSTM $ writeIORef (n + 1) -- Thread local! Just <$> action case res of Just a -> return a -- action succeeded Nothing -> policy >>= \case Just a -> return a -- give up and return what policy gave. Nothing -> loop -- keep trying. }}} This is all completely safe as the `IORef` used never escapes. Actions on the `IORef` are always performed at the beginning of the transaction. This will also enforce the policy for `retry`, though you really want to make that decision before blocking. By wrapping `action` in `orElse` you can periodically commit `retry`ing transactions (discarding their effects) and run the policy outside of the transaction, loosing the more efficient wakeup. But if you have some `tryCount` you get the best of both by getting the efficient wakeup some of the time, then giving up and doing something else. I have heard that there was a paper that sort of settles the topic of contention management in TM, I will look it up and see what it says. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): Ha ha. That is truly horrible! Providing a decent API is what this ticket is really about. I seriously doubt that there is any automatic way to "settle" the topic of contention management, at least without programmer help. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by fryguybob): Indeed, I don't think contention management is always automatic, just what practices make it viable. There is a danger with serializing via some lock that all future executions will become serial once the lock is taken once. You certainly don't want users to have to write `atomicWithAbortPolicy`! Also note that what I gave above might not work in a system that supports partial aborts like Le and Fluet's ICFP paper: http://www.cs.rit.edu/~ml9951/icfp15.pdf But their work in a way avoids the problem by keeping more progress which might be enough to get longer transactions through anyway. The pathological case being where your long transaction has a long enough prefix that never has a conflict and is always available to restart from. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.10.2 (other) | Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by fryguybob): The relevant paper is here: https://cs.rochester.edu/u/scott/papers/2009_PPoPP_CM.pdf I'll see what I can glean from it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10841: Run handler on STM retry -------------------------------------+------------------------------------- Reporter: shlevy | Owner: (none) Type: feature request | Status: new Priority: normal | Milestone: Component: libraries/stm | Version: 7.10.2 Resolution: | Keywords: stm Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * component: libraries (other) => libraries/stm -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10841#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC