Hi, I've implemented a Neural Net simulator which needs to repeat a training loop many times. For this I used a while function: while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res] However, when I run the program, the interpreter falls over after a few thousand iterations, running out of space. I think that this is because the Monadic implementation of the while loop actually nests functions of type (a -> M b) and there is a maximum ?stack size. Is there a better way to implement (possibly infinite) loops in Haskell?
Hi, On Thu, 17 Jun 2004 21:19:17 +1200, Vivian McPhail <vivian.mcphail@paradise.net.nz> wrote:
Hi,
I've implemented a Neural Net simulator which needs to repeat a training loop many times.
It seams we are doing the same thing. I will share my one quite soon. For this I used a while function:
while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res] However, when I run the program, the interpreter falls over after a few thousand iterations, running out of space. I think that this is because the Monadic implementation of the while loop actually nests functions of type (a -> M b) and there is a maximum ?stack size.
Well, there is a maximum stack size. I use ghc and I can give the runtime system the stacksize it shoudld use. However I guess you real problem is not the stack itself, it is the laziness. I guess you perform every trail or epoch some weight updates. That means you are doing some kind of network transformation. Because Haskell is lazy the result of your transformations won't get evaluated until you actually force it to. In case your condition (cond) doesn't force the evaluation of all transformations you get large unevaluated stuff in you memory.
Is there a better way to implement (possibly infinite) loops in Haskell?
I' am curious as well! Cheers, Georg -- ---- Georg Martius, Tel: (034297) 89434 ---- ------- http://www.flexman.homeip.net ---------
while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res]
do you need the monad here? what monad is it? the problem could to be that the "return $ res: rs" can happen only after it is certain that "while test body" succeeds. so you won't even see the very first cons cell before the last one is evaluated. could you produce a (lazy) list of results instead? the garbage collector might be able to collect the list cells that are no longer needed possibly, a lazy state monad would help (if the computation of "while test body" cannot fail)
Is there a better way to implement (possibly infinite) loops in Haskell?
generally, don't program your own recursions - use pre-defined combinators instead. (I like to think of this as a "higher analogon" of "don't use goto - use block structures" from imperative programming) if you need monads, have a look at sequence, sequence_, mapM, mapM_ http://www.haskell.org/onlinereport/monad.html if you can do with lists, then use iterate, fold etc. http://www.haskell.org/onlinereport/list.html best regards, -- -- Johannes Waldmann, Tel/Fax: (0341) 3076 6479 / 6480 -- ------ http://www.imn.htwk-leipzig.de/~waldmann/ ---------
On Thursday 17 June 2004 12:39, Johannes Waldmann wrote:
while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res]
do you need the monad here? what monad is it?
the problem could to be that the "return $ res: rs" can happen only after it is certain that "while test body" succeeds. so you won't even see the very first cons cell before the last one is evaluated. See also http://haskell.org/hawiki/TailRecursive
could you produce a (lazy) list of results instead? the garbage collector might be able to collect the list cells that are no longer needed
possibly, a lazy state monad would help (if the computation of "while test body" cannot fail)
Is there a better way to implement (possibly infinite) loops in Haskell?
You could have a look at "Tackling the awkward squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell" (Simon Peyton Jones) http://research.microsoft.com/users/simonpj/papers/marktoberdorf/ where quite a few "control structures" are described. Cheers, Peter
generally, don't program your own recursions - use pre-defined combinators instead. (I like to think of this as a "higher analogon" of "don't use goto - use block structures" from imperative programming)
if you need monads, have a look at sequence, sequence_, mapM, mapM_ http://www.haskell.org/onlinereport/monad.html if you can do with lists, then use iterate, fold etc. http://www.haskell.org/onlinereport/list.html
best regards,
An interesting paper on Monads and Recursion is "Merging Monads and Folds for Functional Programming" Erik Meijer and Johan Jeuring best regards, gustavo Peter Robinson said:
On Thursday 17 June 2004 12:39, Johannes Waldmann wrote:
while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res]
do you need the monad here? what monad is it?
the problem could to be that the "return $ res: rs" can happen only after it is certain that "while test body" succeeds. so you won't even see the very first cons cell before the last one is evaluated. See also http://haskell.org/hawiki/TailRecursive
could you produce a (lazy) list of results instead? the garbage collector might be able to collect the list cells that are no longer needed
possibly, a lazy state monad would help (if the computation of "while test body" cannot fail)
Is there a better way to implement (possibly infinite) loops in Haskell?
You could have a look at "Tackling the awkward squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell" (Simon Peyton Jones) http://research.microsoft.com/users/simonpj/papers/marktoberdorf/ where quite a few "control structures" are described. Cheers, Peter
generally, don't program your own recursions - use pre-defined combinators instead. (I like to think of this as a "higher analogon" of "don't use goto - use block structures" from imperative programming)
if you need monads, have a look at sequence, sequence_, mapM, mapM_ http://www.haskell.org/onlinereport/monad.html if you can do with lists, then use iterate, fold etc. http://www.haskell.org/onlinereport/list.html
best regards,
_______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
Hello, Vivian uses this while function: while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res]
However, when I run the program, the interpreter falls over after a few thousand iterations, running out of space.
Maybe making it tail-recursive helps. Let me think...... untested code ahead: while test body = do res <- funnyWhile test body [] return (reverse res) funnyWhile test body revResult = do (cond, res) <- body if test cond then while test body (res:revResult) else return revResult Greetings, Arjan
Vivian McPhail wrote:
Hi,
I've implemented a Neural Net simulator which needs to repeat a training loop many times. For this I used a while function:
while test body = do (cond,res) <- body if (test cond) then do rs <- while test body return (res:rs) else return [res] However, when I run the program, the interpreter falls over after a few thousand iterations, running out of space. I think that this is because the Monadic implementation of the while loop actually nests functions of type (a -> M b) and there is a maximum ?stack size.
As others have pointed out, the problem is that while is not tail recursive, so the stack frame for return (res:rs) remains on the stack during the recursive call: result, you run out of stack. However, this only happens because the monad you are using is strict -- or rather, the implementation of (>>=) is strict. If you use a monad with a lazy bind operator instead, then the recursive call will not be evaluated initially, just bound to rs, and evaluated when rs is used by the called, AFTER the return (res:rs). Result: no deep stack. The advantage of doing this is that the elements of the list are available lazily as they are constructed, and so can be consumed as they are generated. Your solution (using a strict monad), and the tail recursive solution you've been offered, both build the entire list before returning it to the caller. If you are performing many thousands of recursions, and the list points at large structures, then this may give you a problematic space leak. The ST monad comes in strict and lazy versions. Provided that's the monad you're using, or a monad built on top of it, you can switch to the lazy version just by importing Control.Monad.ST.Lazy instead of Control.Monad.ST. Documentation is here: http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control.Monad.ST.... If you are using the IO monad, then you achieve the same effect by wrapping the recursive call in unsafeInterleaveIO from System.IO.Unsafe. That will delay its evaluation until rs is evaluated, once again AFTER the enclosing call has returned. But that is -- well -- unsafe. John Hughes PS You can read about lazy state here: http://portal.acm.org/citation.cfm?id=178246&coll=portal&dl=ACM&CFID=2276901...
participants (7)
-
Arjan van IJzendoorn -
Georg Martius -
Gustavo Villavicencio -
Johannes Waldmann -
John Hughes -
Peter Robinson -
Vivian McPhail