December 2008 - Reactive

Re: black hole detection and concurrency
by Sterling Clover 07 Jan '09

07 Jan '09

On Dec 27, 2008, at 9:02 AM, Bertram Felgenhauer wrote: > > The key part here is 'myThreadId >>= killThread' which throws an > asynchronous exception to the thread itself, causing the update > frames to be saved on the heap. > > Note that 'myThreadId >>= killThread' is not equivalent to > 'throw ThreadKilled'; it is a synchronous exception and replaces > thunks > pointed to by the update frames by another call to the raise > primitive - > the result being that the exception gets rethrown whenever such a > thunk > is evaluated. This happens with 'finally' and 'bracket': they use > 'throw' for re-throwing the exception. > > See rts/RaiseAsync.c (raiseAsync() in particular) for the gory details > for the first case, and rts/Schedule.c, raiseExceptionHelper() for the > second case. > > In the above code, there is a small window between catching the > ThreadKilled exception and throwing it again though, where other > exceptions may creep in. The only way I see of fixing that is to use > 'block' and 'unblock' directly. > That certainly seems to do the trick for the simple example at least. One way to reason about it better would be, instead of folding everything into the race function, to simply modify ghc's bracket function to give us the behavior we'd prefer (speaking of which, I recall there's something in the works for 6.12 or so to improve rethrowing of asynchronous exceptions?) brackAsync before after thing = block (do a <- before r <- catch (unblock (thing a)) (\_ -> after a >> myThreadId >>= killThread >> brackAsync before after thing ) after a return r ) where threadKilled ThreadKilled = Just () threadKilled _ = Nothing This brackAsync just drops in to the previous code where bracket was and appears to perform correctly. Further, if we place a trace after the killThread, we se it gets executed once when the example is read (i.e. a resumption) but it does not get executed if the (`seq` v) is removed from the example So this gives me some hope that this is actually doing what we'd like. I don't doubt it may have further kinks however. Cheers, Sterl.

7 19

Getting reactive-fieldtrip working
by Andy Gill 28 Dec '08

28 Dec '08

Hi, After a 6-month hiatus, I'm trying to get reactive-fieldtrip working again. I've got reactive-fieldtrip compiled, depending on depends: FieldTrip-0.2.2 InfixApplicative-1.0.1 base-4.0.0.0 reactive-0.10.5 reactive-glut-0.1.6 vector-space-0.5.1 But it just hangs (busy waiting with 100% CPU). Editing the main to main = anim2 $ pure $ rotTxt <$> time Gives a frozen display. Any ideas? What currently is working, or am i building the wrong things? Andy Gill

1 0

Essential Reactive functions?
by Peter Verswyvelen 27 Dec '08

27 Dec '08

I'm building a little Reactive library in C# for fun - in the same spirit as Conal's one - with the intension to convert that to F#, which I'm learning now. Right now I implemented Future, Event and Reactive, and functionality to merge/mappend two events. Although I'm not yet sure it fully works, using some tricks from Reactive, it was actually really easy to implement. I don't need unamb stuff since I just spawn one new thread for each merged event, and I use the WaitForMultipleObjects primitive to check which of the two event occurrences was first. The code is of course a zillion times more ugly than in Haskell, and since I'm not using lightweight threads, it might really be slow. I tried to make it as immutable as possible, so it's not really imperative :) But what is the minimal set of functions one must implement to build a Reactive library, in the sense that all others can be implemented using these? Merry Xmas!

3 4

join/mappend bug and unamb
by Conal Elliott 27 Dec '08

27 Dec '08

I've been plugging away on some persistent & subtle Reactive bugs. Here's a progress report. I changed the type of future times in Reactive and have apparently fixed the old join/mappend bug as I'd hoped. This bug shows up at least in the form of delaying response to inputs. The key for the fix was to replace AddBounds (Improving t) with Improving (AddBounds t), since the latter type allows more partial information to seep through than the former. The old representation thwarted laziness/partiality by requiring an outer discrimination between finite and infinite times. The new code relies on unamb more than the old code, and runs afoul of the problems described in http://conal.net/blog/posts/smarter-termination-for-thread-racing/ and the comments that follow it, as well as one or two email threads. Fortunately, Spencer Janssen's hypothosis (thunks get permanently evaluated to aborted computations, leading to ThreadKilled and NonTermination exceptions) validated by Sterling Clover, explains all of the troublingly strange behavior I've seen in Reactive and may point in the direction of a solution. Luke Palmer has some encouraging early results with a custom scheduler that implements unamb. I don't know how it deals with termination of redundant computation and whether it falls afoul of the problems I'm having with the various race implementations floating around. I also don't know how much of the problem that Luke is solving comes simply from the failure of my race implementation to kill threads recursively. That's it for now. I'd really appreciate help with getting a correct & efficient implementation of thread management for unamb. - Conal

2 1

Fwd: black hole detection and concurrency
by Conal Elliott 27 Dec '08

27 Dec '08

Thanks for the probing, Sterl. I'm afraid I'm stuck here. Without the more effective thread killing (in unamb), Reactive seems to be drowning in uselessly running threads. With "improved" thread killing, I get many of these false black holes, instead of computed values. I don't know whether this problem is solvable on top of ghc's current RTS, as you suggest (clever `par` use), but I don't know. I'm out of my depth here and could sure use help. Thanks, - Conal On Fri, Dec 26, 2008 at 6:09 PM, Sterling Clover <s.clover(a)gmail.com> wrote: > I have a good theory on the latter symptom (the "thread killed" message). > Sticking in some traces, as in my appended code, helped me to see what's > going on. It seems to be exactly what you describe -- the variable v is > permanently bound to the exception it "evaluates" to. Since the right hand > True portion of the unamb evaluates more quickly, the spawned threads are > killed and the left hand (the v) "evaluates" to "thread killed". This > remains the value of its thunk when you access it later. This problem seems > sort of innate to a pure unamb utilizing unsafePerformIO and asynchronous > exceptions. A clever use of `par` might conceivably help, given that if the > par spark fails, the thunk can still be evaluated? Might be a dead end. > > Here's the code: > > go = f "f" (f "" True) where f s v = (unamb (s++"f") (s++"g") v True) `seq` > v > > --unamb :: String -> String -> a -> a -> a > unamb s s' a b = unsafePerformIO (race s s' (evaluate a) (evaluate b)) > > --race :: String -> String -> IO a -> IO a -> IO a > race s s' a b = do > v <- newEmptyMVar > let t x = x >>= putMVar v > withThread s (t a) $ withThread s' (t b) $ takeMVar v > where > withThread s u v = bracket (forkIO u) (killNote s) (const $ putStrLn > ("in: " ++ s) >> v >>= \x -> putStrLn ("out: " ++ show x ++ " "++ s) >> > return x) > > killNote s tid = throwTo tid (ErrorCall s) > > And a GHCi session: > > *Un> go > in: ff > in: fg > in: f > in: g > out: True fg > out: True ff > <interactive>: ff > *** Exception: ff > > Cheers, > Sterl. > > > On Dec 26, 2008, at 1:15 AM, Conal Elliott wrote: > > I'm looking for information about black hole detection with ghc. I'm >> getting "<<loop>>" where I don't think there is an actual black hole. I get >> this message sometimes with the unamb package, which is implemented with >> unsafePerformIO, concurrency, and killThread, as described in >> http://conal.net/blog/posts/functional-concurrency-with-unambiguous-choice/ >> and http://conal.net/blog/posts/smarter-termination-for-thread-racing/ . >> >> Suppose I have a definition 'v = unsafePerformIO ...', and v is used more >> than once. Evaluation (to whnf) of v is begun and the evaluation thread >> gets killed before evaluation is complete. Then the second use begins. >> Will the second evaluation be (incorrectly) flagged as a black hole? >> >> I haven't found a simple, reproducible example of incorrect black-hole >> reporting. My current examples are tied up with the Reactive library. I do >> have another strange symptom, which is "thread killed" message. I wonder if >> it's related to the <<loop>> message. Code below. >> >> Thanks, - Conal >> >> >> import Prelude hiding (catch) >> import System.IO.Unsafe >> import Control.Concurrent >> import Control.Exception >> >> >> -- *** Exception: thread killed >> main :: IO () >> main = print $ f (f True) where f v = (v `unamb` True) `seq` v >> >> -- | Unambiguous choice operator. Equivalent to the ambiguous choice >> -- operator, but with arguments restricted to be equal where not bottom, >> -- so that the choice doesn't matter. See also 'amb'. >> unamb :: a -> a -> a >> unamb a b = unsafePerformIO (evaluate a `race` evaluate b) >> >> -- | Race two actions against each other in separate threads, and pick >> -- whichever finishes first. See also 'amb'. >> race :: IO a -> IO a -> IO a >> race a b = do >> v <- newEmptyMVar >> let t x = x >>= putMVar v >> withThread (t a) $ withThread (t b) $ takeMVar v >> where >> withThread u v = bracket (forkIO u) killThread (const v) >> >> _______________________________________________ >> Glasgow-haskell-users mailing list >> Glasgow-haskell-users(a)haskell.org >> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users >> > >

1 0

problem with unamb -- doesn't kill enough threads
by Conal Elliott 20 Dec '08

20 Dec '08

I realized in the shower this morning that there's a serious flaw in my unamb implementation as described in http://conal.net/blog/posts/functional-concurrency-with-unambiguous-choice. I'm looking for ideas for fixing the flaw. Here's the code for racing computations: race :: IO a -> IO a -> IO a a `race` b = do v <- newEmptyMVar ta <- forkPut a v tb <- forkPut b v x <- takeMVar v killThread ta killThread tb return x forkPut :: IO a -> MVar a -> IO ThreadId forkPut act v = forkIO ((act >>= putMVar v) `catch` uhandler `catch` bhandler) where uhandler (ErrorCall "Prelude.undefined") = return () uhandler err = throw err bhandler BlockedOnDeadMVar = return () The problem is that each of the threads ta and tb may have spawned other threads, directly or indirectly. When I kill them, they don't get a chance to kill their sub-threads. Perhaps I want some form of garbage collection of threads, perhaps akin to Henry Baker's paper "The Incremental Garbage Collection of Processes". As with memory GC, dropping one consumer would sometimes result is cascading de-allocations. That cascade is missing from my implementation. Or maybe there's a simple and dependable manual solution, enhancing the method above. Any ideas? - Conal

6 12

GHC and other package versions?
by Clifford Beshers 20 Dec '08

20 Dec '08

I'm gearing up to play with reactive and fieldtrip over the holidays. Is the latest released GHC sufficient? Are there any do's and don't's in auxiliary package version selection? Any other pitfalls?

1 0

I think I found the bug with joinE/mappend
by Conal Elliott 19 Dec '08

19 Dec '08

I think I finally found the cause of a key laziness bug in reactive. It took me a long time to track down. The way I structured the AddBounds type stops me from getting partial info I need. There's a critical max on times in joinE. It'll be 'max (NoBound t0) ta', and I know that ta is either a NoBound or a MaxBound, but without knowing which, no info can come out, which stops the consumer of that max from knowing that the time is at least NoBound t0. Now I think I have to make some pervasive changes to reactive. I'd like to develop the QC tests that will catch these mistakes much earlier. I recently read the "Chasing Bottoms" paper, which some good techniques. - Conal

1 0

Version conflicts despite hiding
by Patai Gergely 17 Dec '08

17 Dec '08

Hello all, I have a little problem I haven't run into yet. I'm not sure if it's Reactive related, but that's what caused it for me, so I'm writing here. I managed to run Tetris.hs with reactive-0.10.3 and reactive-glut-0.1.4 with success (also observing the problems mentioned by Claus Reinke). Now I ran an update, and ghc -v says the following as expected: hiding package reactive-0.10.3 to avoid conflict with later version reactive-0.10.5 hiding package reactive-glut-0.1.4 to avoid conflict with later version reactive-glut-0.1.6 However, when I try to compile Tetris or load it into ghci, it says that it can't match 'UI' (expected) with 'reactive-glut-0.1.4:FRP.Reactive.GLUT.UI.UI' (inferred) in dropPiece, and I've no idea where it is getting the latter from. There are no leftover .hi files or anything else in reach. Is reactive-fieldtrip a possible culprit? Gergely -- http://www.fastmail.fm - One of many happy users: http://www.fastmail.fm/docs/quotes.html

4 5

Fw: [Haskell-cafe] Re: Threads not running under GHC 6.10?
by Robin Green 17 Dec '08

17 Dec '08

This would appear to present a problem for unamb... Begin forwarded message: Date: Wed, 17 Dec 2008 13:05:48 +0000 From: Simon Marlow <marlowsd(a)gmail.com> To: Gwern Branwen <gwern0(a)gmail.com> Cc: Haskell Cafe <haskell-cafe(a)haskell.org> Subject: [Haskell-cafe] Re: Threads not running under GHC 6.10? Gwern Branwen wrote: > On Mon, Dec 15, 2008 at 9:00 AM, Simon Marlow wrote: >> This particular example illustrates a bug in 6.10.1 that we've since >> fixed: >> >> http://hackage.haskell.org/trac/ghc/ticket/2783 > > OK, that's good... > >> However in general you can still write expressions that don't >> allocate anything (e.g. nfib 1000), and your watchdog thread won't >> get a chance to run unless there's a spare CPU available (+RTS -N2). >> >> Cheers, >> Simon > > But that's bad. What are my options here? Will this > threads-not-running issue be fixed in the next release? Since it > worked fine in 6.8 as far as I could tell, that makes me think that it > must not be anything completely fundamental and unfixable. I'm afraid the underlying problem is one that GHC has always had - that we can't preempt threads that aren't allocating. It's not easily fixable, we would have to inject dummy heap checks into every non-allocating loop, which would seriously hurt performance for those tight loops. In general you can't rely on being able to kill a thread; however, this only applies to compiled code, interpreted code should always be preemptable, even if it isn't allocating. Cheers, Simon _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe(a)haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

1 0