Hello all, I have a Logger which produces log entries and a new version of itself newtype Logger evt dom log = Lgr {runLogger :: Timed evt -> dom -> (log, Logger evt dom log)} Loggers are used in a function like this runSim :: (Ord evt, Monoid log) => SimBehaviour evt dom log -> SimState evt dom log -> SimState evt dom log runSim (!lgr, !hdr, xtp) (!log,!dom,!evq) = case step of Nothing -> (log, dom, evq) -- end of simulation Just (newEvq, newDom, newHdr, newLgr, newLog) -> runSim (newLgr,newHdr,xtp) (newLog,newDom,newEvq) where -- check for end conditions or run handler step = do (evt, evts) <- H.view evq -- no more Events -> Nothing if xtp (evt,dom) then Nothing else let (evq', dom', hdr') = runHandler hdr evt dom (log',lgr') = runLogger lgr evt dom' -- <-- -- append new event and new log entries in return (evq'<>evts, dom', hdr', lgr', log'<>log) -- <-- I then wrote a function to combine two Loggers addLgr (lgr1) (lgr2) = Lgr lgr where lgr tev dom = let (log1', lgr1') = runLogger lgr1 tev dom (log2', lgr2') = runLogger lgr2 tev dom (!log') = log2' <> log1' -- x -- -- in (log2', addLgr lgr1' lgr2') in (log', addLgr lgr1' lgr2') When called a million times, this produces a heap profile which climbs steadily (with or without the stricness annotation in line x). When I omit the (<>) as in the commented line, the heap stays flat. My log is really just a list of strings and most of the time the loggers do not produce any output, i.e. they return an empty list. Am I on the right track, that this trouble is probably caused by laziness and that forcing strictness is the way to go? Could it be that this is because ! does not fully evaluate its argument, but just to WHNF? Or is there a more obvious reason, I just fail to see. Where to go from here?
On Sat, Nov 14, 2015 at 11:10:15AM +0100, martin wrote:
newtype Logger evt dom log = Lgr {runLogger :: Timed evt -> dom -> (log, Logger evt dom log)}
runSim :: (Ord evt, Monoid log) => SimBehaviour evt dom log -> SimState evt dom log -> SimState evt dom log runSim (!lgr, !hdr, xtp) (!log,!dom,!evq) = case step of Nothing -> (log, dom, evq) -- end of simulation Just (newEvq, newDom, newHdr, newLgr, newLog) -> runSim (newLgr,newHdr,xtp) (newLog,newDom,newEvq) where -- check for end conditions or run handler step = do (evt, evts) <- H.view evq -- no more Events -> Nothing if xtp (evt,dom) then Nothing else let (evq', dom', hdr') = runHandler hdr evt dom (log',lgr') = runLogger lgr evt dom' -- <-- -- append new event and new log entries in return (evq'<>evts, dom', hdr', lgr', log'<>log) -- <--
addLgr (lgr1) (lgr2) = Lgr lgr where lgr tev dom = let (log1', lgr1') = runLogger lgr1 tev dom (log2', lgr2') = runLogger lgr2 tev dom (!log') = log2' <> log1' -- x -- -- in (log2', addLgr lgr1' lgr2') in (log', addLgr lgr1' lgr2')
[...]
Could it be that this is because ! does not fully evaluate its argument, but just to WHNF? Or is there a more obvious reason, I just fail to see.
Yes, forcing a list is more-or-less useless, most of the time. All you're doing is checking whether the length is zero or non-zero.
Where to go from here?
The first thing to do would be to check whether import Control.DeepSeq ... !log' = deepseq (log2' <> log1') solves the space leak. If it does then you have found the problem. That's not the same thing as finding a solution though! There are plenty of pre-rolled logging solutions available. I think I'd just suggest choosing one of those, rather than implementing your own. Tom
I profiled the program and the profile does not say anything about THUNKs. Instead the main culprit seems to be (352)cor.cnd/cor/ex_lgr.lo... 2837612 This probably points to this function: newtype Condition a = Cnd {checkCnd :: a -> (Bool, Condition a)} -- | Create a 'Condition' from two conditions which holds when one of -- them holds. cor :: Condition a -> Condition a -> Condition a cor c1 c2 = Cnd cnd where cnd a = let (b1', c1') = checkCnd c1 a (b2', c2') = checkCnd c2 a in if b1' || b2' then (True, cor c1' c2') else (False, cor c1 c2) logWhen :: Monoid log => Condition (Timed evt,dom) -> Logger evt dom log -> Logger evt dom log logWhen cnd lgrIn = Lgr lgr' where lgr' tev dom = case checkCnd cnd (tev, dom) of (True, cnd') -> let (log', lgrIn') = runLogger lgrIn tev dom in (log', logWhen cnd' lgrIn') (False,cnd') -> (mempty, logWhen cnd' lgrIn) Am I holding on to some data where I shouldn't? I cannot see it. Am 11/14/2015 um 11:10 AM schrieb martin:
Hello all,
I have a Logger which produces log entries and a new version of itself
newtype Logger evt dom log = Lgr {runLogger :: Timed evt -> dom -> (log, Logger evt dom log)}
Loggers are used in a function like this
runSim :: (Ord evt, Monoid log) => SimBehaviour evt dom log -> SimState evt dom log -> SimState evt dom log runSim (!lgr, !hdr, xtp) (!log,!dom,!evq) = case step of Nothing -> (log, dom, evq) -- end of simulation Just (newEvq, newDom, newHdr, newLgr, newLog) -> runSim (newLgr,newHdr,xtp) (newLog,newDom,newEvq) where -- check for end conditions or run handler step = do (evt, evts) <- H.view evq -- no more Events -> Nothing if xtp (evt,dom) then Nothing else let (evq', dom', hdr') = runHandler hdr evt dom (log',lgr') = runLogger lgr evt dom' -- <-- -- append new event and new log entries in return (evq'<>evts, dom', hdr', lgr', log'<>log) -- <--
I then wrote a function to combine two Loggers
addLgr (lgr1) (lgr2) = Lgr lgr where lgr tev dom = let (log1', lgr1') = runLogger lgr1 tev dom (log2', lgr2') = runLogger lgr2 tev dom (!log') = log2' <> log1' -- x -- -- in (log2', addLgr lgr1' lgr2') in (log', addLgr lgr1' lgr2')
When called a million times, this produces a heap profile which climbs steadily (with or without the stricness annotation in line x). When I omit the (<>) as in the commented line, the heap stays flat. My log is really just a list of strings and most of the time the loggers do not produce any output, i.e. they return an empty list.
Am I on the right track, that this trouble is probably caused by laziness and that forcing strictness is the way to go?
Could it be that this is because ! does not fully evaluate its argument, but just to WHNF? Or is there a more obvious reason, I just fail to see.
Where to go from here?
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
On Sat, Nov 14, 2015 at 01:28:58PM +0100, martin wrote:
newtype Condition a = Cnd {checkCnd :: a -> (Bool, Condition a)}
-- | Create a 'Condition' from two conditions which holds when one of -- them holds. cor :: Condition a -> Condition a -> Condition a cor c1 c2 = Cnd cnd where cnd a = let (b1', c1') = checkCnd c1 a (b2', c2') = checkCnd c2 a in if b1' || b2' then (True, cor c1' c2') else (False, cor c1 c2)
logWhen :: Monoid log => Condition (Timed evt,dom) -> Logger evt dom log -> Logger evt dom log logWhen cnd lgrIn = Lgr lgr' where lgr' tev dom = case checkCnd cnd (tev, dom) of (True, cnd') -> let (log', lgrIn') = runLogger lgrIn tev dom in (log', logWhen cnd' lgrIn') (False,cnd') -> (mempty, logWhen cnd' lgrIn)
Am I holding on to some data where I shouldn't? I cannot see it.
Sure, this (log', lgrIn') = runLogger lgrIn tev dom allocates a thunk for log' that holds on to tev and dom. Even if they are small, holding onto them for thousands or millions of iterations is going to leak a lot of space. The only way to avoid this is to force log' sufficiently far that tev and dom can be released. Tom
Thanks Tom, with lots of trial and error I believe I finally put the exclamation marks in the right spots, but I still don't understand. addLgr (lgr1) (lgr2) = Lgr lgr where lgr tev dom = let (!(log1', lgr1')) = runLogger lgr1 tev dom (!(log2', lgr2')) = runLogger lgr2 tev dom (!log') = log1' <> log2' in (log', addLgr lgr2' lgr1') I had previously put the marks inside the tuples as in (!log1', !lgr1') but that didn't help. Can someone explain the difference between (!log1', !lgr1') and !(log1', lgr1'). I thought the former enforces more strictness than the later, but I must be missing something. Am 11/14/2015 um 01:43 PM schrieb Tom Ellis:
On Sat, Nov 14, 2015 at 01:28:58PM +0100, martin wrote:
newtype Condition a = Cnd {checkCnd :: a -> (Bool, Condition a)}
-- | Create a 'Condition' from two conditions which holds when one of -- them holds. cor :: Condition a -> Condition a -> Condition a cor c1 c2 = Cnd cnd where cnd a = let (b1', c1') = checkCnd c1 a (b2', c2') = checkCnd c2 a in if b1' || b2' then (True, cor c1' c2') else (False, cor c1 c2)
logWhen :: Monoid log => Condition (Timed evt,dom) -> Logger evt dom log -> Logger evt dom log logWhen cnd lgrIn = Lgr lgr' where lgr' tev dom = case checkCnd cnd (tev, dom) of (True, cnd') -> let (log', lgrIn') = runLogger lgrIn tev dom in (log', logWhen cnd' lgrIn') (False,cnd') -> (mempty, logWhen cnd' lgrIn)
Am I holding on to some data where I shouldn't? I cannot see it.
Sure, this
(log', lgrIn') = runLogger lgrIn tev dom
allocates a thunk for log' that holds on to tev and dom. Even if they are small, holding onto them for thousands or millions of iterations is going to leak a lot of space.
The only way to avoid this is to force log' sufficiently far that tev and dom can be released.
Tom _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
On Sat, Nov 14, 2015 at 05:17:41PM +0100, martin wrote:
with lots of trial and error I believe I finally put the exclamation marks in the right spots, but I still don't understand.
It's pretty reasonable not to understand! Understanding exactly what's going with strictness annotations can be very mind-bending.
I had previously put the marks inside the tuples as in (!log1', !lgr1') but that didn't help. Can someone explain the difference between (!log1', !lgr1') and !(log1', lgr1'). I thought the former enforces more strictness than the later, but I must be missing something.
Those two are completely independent. let !(a, b) = ... forces the tuple constructor immedately, but not the contents let (!a, !b) = ... doesn't force the tuple constructor, but *when it is forced* a and b will be forced at the same time. Neither is more or less strict than the other. Tom
participants (2)
-
martin -
Tom Ellis