December 2005 - Haskell-Cafe

RE: Invalid object in isRetainer
by Simon Marlow 05 Dec '05

05 Dec '05

On 04 December 2005 17:02, Joel Reymont wrote: > ./randomplay +RTS -hr > ... <- some time passes and then > > randomplay: internal error: Invalid object in isRetainer(): 67 > Please report this as a bug to glasgow-haskell-bugs(a)haskell.org, > or http://www.sourceforge.net/projects/ghc/ > > The combination of this and the failure with -hc -hbdrag,void is > preventing me from optimizing the program and it has a huge space > leak somewhere. Retainer profiling (-hr) doesn't currently work with STM. It is a bug, but I briefly looked into it and it's not a trivial fix so I left it. I'll file a ticket for it. Cheers, Simon

2 1

RE: Wierd activation record found on stack
by Simon Marlow 05 Dec '05

05 Dec '05

On 04 December 2005 16:50, Joel Reymont wrote: > To add a little bit of detail (thanks to dons for the tips), the > signal is being caught in one of the threads launched by the main > thread. Simply calling exitWith (ExitFailure 1) from this thread > causes the error below. I changed my code to send messages to running > threads asking them to quit and this made the error go away. > > On Dec 4, 2005, at 12:27 PM, Joel Reymont wrote: > >> randomplay: internal error: scavenge_stack: weird activation record >> found on stack: 0 Please report this as a bug to >> glasgow-haskell-bugs(a)haskell.org, or >> http://www.sourceforge.net/projects/ghc/ >> >> I installed my own signal handlers with the intent of having my >> profile reports written out. The program runs forever and otherwise >> generates no profile reports when I interrupt it. This was fixed recently, I believe: revision 1.276 date: 2005-11-08 10:44:22 +0000; author: simonmar; state: Exp; lines: +1 -2 raiseAsync: fix bug that can cause a scavenge_stack panic for a thread that has just been killed. The fix will be in 6.4.2, or you can grab a STABLE snapshot from the haskell.org. Cheers, Simon

1 0

RE: [Haskell-cafe] STM, orElse and timed read from a channel
by Simon Peyton-Jones 05 Dec '05

05 Dec '05

| > I think you are really asking for some way to create top-level | > transactional variables? | | Yes. Perhaps it would be easier to do it cleanly than arbitrary | top-level IO bindings? It turns out to be easy to provide newTVarIO :: a -> IO (TVar a) which you can call from inside 'unsafePerformIO'. That means you can allocate top-level TVars without fuss. The whole question of top-level IO remains open, but this will address the immediate question very nicely. I've also added a SourceForge bug to remind us to fail more tidily if you do atomically inside atomically. Simon

2 1

RE: [Haskell-cafe] STM, orElse and timed read from a channel
by Simon Peyton-Jones 05 Dec '05

05 Dec '05

Can you provide a small test case that demonstrates the crash? I think it's pretty unlikely that we'll support nested atomically (via unsafePerformIO) anytime soon. UnsafePerformIO is, well, unsafe. I think you are really asking for some way to create top-level transactional variables? Simon | -----Original Message----- | From: haskell-cafe-bounces(a)haskell.org [mailto:haskell-cafe-bounces@haskell.org] On Behalf Of | Tomasz Zielonka | Sent: 04 December 2005 18:32 | To: Simon Marlow | Cc: Haskell Cafe | Subject: Re: [Haskell-cafe] STM, orElse and timed read from a channel | | On Sat, Dec 03, 2005 at 10:35:54PM +0100, Tomasz Zielonka wrote: | > so I could export a waitUntil function with type | > | > waitUntil :: Time -> STM () | > | > but I tripped on something that was reported before, namely that STM | > transactions can't be nested (as a result of unsafePerformIO or | > unsafeInterleaveIO). Is there a plan to support such scenario? | | OK, it can be worked around by running the atomically block in a new | transaction. But still it would be nice if the program didn't segfault, | but cause an exception for example. | | Updated modules attached. | | Best regards | Tomasz | | -- | I am searching for a programmer who is good at least in some of | [Haskell, ML, C++, Linux, FreeBSD, math] for work in Warsaw, Poland

2 1

STM, Concurrent Haskell and network clients (long, code)
by Joel Reymont 05 Dec '05

05 Dec '05

Folks, After two months with Haskell I think I'm starting to get the proper spirit. Inspired by Simon PJ's recent comments about lots of threads not hurting Haskell I introduced STM into my network client. I would like to run my architecture by you to see if there are any improvements to be made. I would like the code to be as bullet-proof as possible. I'm also jumping through some hoops to make sure none of the 3 threads (main, socket reader and socket writer) block indefinitely. I'm getting a "thread blocked indefinitely" exception now and I think this is because the main thread is not getting notified when the socket threads die. My network client has two threads. #1 reads network packets, converts them to events and posts them to a TChan using a function passed to it by the main thread. #2 blocks on a second TChan and waits for events to be sent out through the socket. The main thread creates a TChan to read events from, passes a closure that writes to the TChan to the network client and uses a network client function to send events to the network client. All in all it works out very nicely as there's no networking, FFI, etc. in the network client logic. This is an improvement from what I had before. The function that starts the poker client has this signature: startPokerClient :: (Event -> IO ()) -- fun to send events to the main thread -> HostInfo -- (host, port) -> AuthInfo -- (user, pass, etc.) -> IO PokerClient where data PokerClient = PC !MaybeThread !MaybeThread !(TChan Command) The main network client thread calls this fun to deliver events to the network client: writePokerClient :: PokerClient -> Command -> IO () writePokerClient (PC _ _ chan) cmd = atomically $ writeTChan chan cmd Main thread can stop the network client using stopPokerClient :: PokerClient -> IO () stopPokerClient (PC tmv2 tmv1 _) = do putStrLn $ "stopPokerClient invoked" maybeKillThread tmv2 maybeKillThread tmv1 The abbreviated startPokerClient looks like this.. {-# NOINLINE lock #-} lock :: MVar () lock = unsafePerformIO $ newMVar () startPokerClient post (host, port) auth = do h <- withMVar lock -- getHostByName is not thread-safe $ \_ -> connectTo host (PortNumber $ fromIntegral port) `catchError` (\e -> fail $ "Failed to connect to server: " ++ show e) -- kick off the SSL handshake ssl <- startSSL doSSLHandshake h ssl ... -- we need one thread to always kill the other (tmv1 :: MaybeThread) <- atomically $ newEmptyTMVar (tmv2 :: MaybeThread) <- atomically $ newEmptyTMVar (tmvssl :: MaybeSSL) <- atomically $ newTMVar (Just ssl) -- thread to read from the socket and post commands tid1 <- forkIO $ writeLoop post h ssl `finally` finalize tmv1 tmv2 h tmvssl post -- thread to write to grab commands generated -- by the main thread and write them to the socket chan <- atomically $ newTChan tid2 <- forkIO $ readLoop h chan `finally` finalize tmv2 tmv1 h tmvssl post -- we have both threads now atomically $! do putTMVar tmv1 (Just tid1) putTMVar tmv2 (Just tid2) return $! PC tmv2 tmv1 chan Interestingly enough, I had to use $! otherwise startPokerClient was sitting there for a while, due to due to lazy evaluation I assume. I'm using the following finalizer for my socket reader/writer threads to make sure that if one thread dies the other one is taken with it and that the main thread is notified. I don't have a Maybe around the socket itself since I don't think closing a socket twice has any bad side effects. I put it in a try, though, for good measure. finalize :: MaybeThread -> MaybeThread -> Handle -> MaybeSSL -> (Event -> IO ()) -> IO () finalize me buddy h ssl post = do -- make sure we are not killed again -- killing a dead thread is apparently -- not free of side effects in GHC 6.4.1 atomically $ putTMVar me Nothing -- kill the other thread maybeKillThread buddy maybeFreeSSL ssl try $ hClose h post $ StopScript -- notify main thread return () The rest of the relevant code is below. Please let me know if it can be simplified or made more robust. maybeFreeSSL :: MaybeSSL -> IO () maybeFreeSSL tmv = do putStrLn $ "maybeFreeSSL invoked" mssl <- atomically $ swapTMVar tmv Nothing case mssl of Nothing -> return () Just (ssl, _, _) -> do sslFree ssl sslFree ssl maybeKillThread :: MaybeThread -> IO () maybeKillThread tmv = do putStrLn $ "maybeKillThread invoked " mtid <- atomically $ swapTMVar tmv Nothing case mtid of Nothing -> return () Just tid -> killThread tid --- Socket reader writeLoop :: (Event -> IO ()) -> Handle -> (SSL, BIO, BIO) -> IO () writeLoop post h ssl = do cmd <- read h ssl post $ Cmd cmd writeLoop post h ssl --- Socket writer readLoop :: Handle -> TChan Command -> IO () readLoop h chan = do cmd <- atomically $ readTChan chan write h cmd readLoop h chan --- SSL, etc. not used now but will be used to decrypt later read :: Handle -> (SSL, BIO, BIO) -> IO Command read h _ = do bytes <- hGet h 4 let (size', _) = unpickle endian32 bytes size = fromIntegral $ size' - 4 if size > 0 then do packet <- hGet h size when (size /= length packet) $ fail "Packet size mismatch" unstuff packet else do fail $ "read: size is 0: bytes: " ++ show bytes unstuff :: [Word8] -> IO Command unstuff packet = do let (kind, packet') = unpickle puCmdType packet (cmd', _) = unpickle (puCommand kind) packet' case cmd' of InvalidCommand -> do fail $ "unstuff: Cannot parse: " ++ show packet SrvCompressedCommands sz data' -> do data'' <- uncompress data' $ fromIntegral sz unstuff $ drop 4 data'' _ -> return $! cmd' write :: Handle -> Command -> IO () write h cmd = do let kind = cmdType cmd bytes' = pickle puCmdType kind bytes = bytes' ++ pickle (puCommand kind) cmd when (length bytes <= 0) $ fail $ "write: Could not pack: " ++ show cmd let size = 4 + length bytes size' = pickle endian32 $ fromIntegral size when (length size' <= 0) $ fail $ "write: Could not pack: " ++ show size hPut h (size' ++ bytes) hFlush h -- http://wagerlabs.com/

2 3

Problem with continuations and typing
by tpledger＠ihug.co.nz 04 Dec '05

04 Dec '05

Jerzy Karczmarczuk wrote: : | zeros fc sc = sc 0 zeros | | fails to compile as well. *I do not ask why, I know*. | | But I would like to continue this exercice along these lines, without too much | exotism (no monads, yet...), for my students. Do you have any simple work-around? | Introduce some algebraic constructors? Perhaps higher-rank polymorphism could do | something (but then I would have to explain it to my folk...) : How about this for a non-exotic algebraic type? > newtype G a b = G{ unG :: b -> (a -> G a b -> b) -> b } > glist g = unG g [] (\b g' -> b : glist g') > zeros = G (\no yes -> yes 0 zeros) > disj g1 g2 = G (\no yes -> unG g1 (unG g2 no yes) > (\b g1' -> yes b (disj g1' g2))) I haven't had much practice with continuations, so don't know whether I've just lost some generality there. But it does support *some* avoidance of higher-rank polymorphism, through the use of good old partial application. For example, the type of the state variable s doesn't leak into the result type of unfold: > unfold f s = G (\no yes -> case f s of > Nothing -> no > Just (s', b) -> yes b (unfold f s')) HTH, Tom

1 0

RE: [Haskell-cafe] STM, orElse and timed read from a channel
by Simon Marlow 04 Dec '05

04 Dec '05

On 29 November 2005 12:08, Tomasz Zielonka wrote: > On Tue, Nov 29, 2005 at 12:00:03PM -0000, Simon Marlow wrote: >> threadDelay is IO-only; there's no way to use threadDelay in an STM >> transaction. For example, if you want to wait for a TVar to go from >> Nothing to Just x with a timeout, you could do this: >> >> readOrTimeout :: TVar (Maybe a) -> Int -> STM (Maybe a) >> readOrTimeout t secs = do timeout <- registerTimeout secs >> let check_timeout = do >> b <- readTVar timeout >> if b then return Nothing else retry >> check_t = do >> m <- readTVar t >> case m of >> Nothing -> retry >> Just x -> return x >> atomically $ check_timeout `orElse` check_t > > Wouldn't it be > readOrTimeout :: TVar (Maybe a) -> Int -> IO (Maybe a) > ^^ > ? Sorry, yes. > Alternatively, it would be nice to have a new STM primitive: > > wailUntil :: ClockTime -> STM () > > so you would wait until some time-point passes, not for a number of > time-units (waiting for a number of time-units wouldn't work because > of retries). I think it could be efficiently implemented, wouldn't it? Interesting. You could use that do wait for idle time, for example: atomically $ do t <- readTVar last_mouse_click waitUntil (t+1000) ... so this transaction only completes when some idle time has passed since the last mouse click. But you could also implement this using registerTimeout, albeit with some more code and an extra thread, and waitUntil requires an implementation in the runtime which is not entirely trivial. Cheers, Simon

3 4

Bug: Invalid object in isRetainer
by Joel Reymont 04 Dec '05

04 Dec '05

./randomplay +RTS -hr ... <- some time passes and then randomplay: internal error: Invalid object in isRetainer(): 67 Please report this as a bug to glasgow-haskell-bugs(a)haskell.org, or http://www.sourceforge.net/projects/ghc/ The combination of this and the failure with -hc -hbdrag,void is preventing me from optimizing the program and it has a huge space leak somewhere. Thanks, Joel -- http://wagerlabs.com/

1 0

Wierd activation record found on stack
by Joel Reymont 04 Dec '05

04 Dec '05

randomplay: internal error: scavenge_stack: weird activation record found on stack: 0 Please report this as a bug to glasgow-haskell-bugs(a)haskell.org, or http://www.sourceforge.net/projects/ghc/ I installed my own signal handlers with the intent of having my profile reports written out. The program runs forever and otherwise generates no profile reports when I interrupt it. I install the handlers like this: installHandler sigPIPE Ignore Nothing flip mapM_ [sigINT, sigHUP, sigABRT, sigTERM] $ \sig -> do installHandler sig handler Nothing where handler = Catch $ exitWith (ExitFailure 1) P.S. Is there a prize of some sorts for the highest number of wierd bug reports per time frame? :D -- http://wagerlabs.com/

1 1

Invalid object in processHeapClosureForDead()
by Joel Reymont 04 Dec '05

04 Dec '05

Has anyone seen this before? ./randomplay +RTS -hc -hbdrag,void -sgcstat randomplay: internal error: Invalid object in processHeapClosureForDead(): 68 Please report this as a bug to glasgow-haskell-bugs(a)haskell.org, or http://www.sourceforge.net/projects/ghc/ -- http://wagerlabs.com/

1 0