I used to be a victim of GFW, so I can feel your pain. You may try to subscribe to http://leimy9.blogspot.com/feeds/posts/default in your Google Reader. In case that fails too, I've pasted the blog post below, with no images: I've been using Haskell in a serious way for about 2 years. Been using it in a professional sense about 1.5 years now in that, yes, I am one of the lucky ones that gets to use Haskell at work. The ride has been pretty smooth most of the time, as I've found that the type system especially helps me to rule out certain classes of bugs, easily test rather large chunks of programs as they're pure. The interpreter allows experimentation and iteration of ideas that can then be composed into the final compiled programs. All of this gives me a good deal of confidence that the code I'm writing is correct to some degree up front, something I've come to expect from functional programming languages over the years and greatly appreciate. However I've felt compelled to comment that things aren't always so smooth either. I spent the better part of a weekend and a Monday tracking down a space leak in a program that just was not allowed to leak space. I have a stack of a ReaderT StateT IO that I use to communicate with a device through a passthrough program that speaks CAN to a device for the purposes of creating a serial console connection where there is only a CAN bus. The Haskell program is responsible for the management of the data found at the other end o the serial connection and supports operations to the device through the serial channel via a simple text protocol while "forever" polling data on the serial endpoint. What I had done was the equivalent of pollerLoop :: Poller () pollerLoop = forever pollOnce Where Poller is my monad stack. pollOnce is defined as, pollOnce :: Poller () pollOnce = do checkCommandChannel -- see if there's a pending command to run executePoll Yes, this application is multi-threaded. I have a logger thread, a thread watching standard input of the program for queries and issuing commands to the endpoint. I have a thread per Poller, and the ability to poll devices simultaneously. The poller includes an implementation of my little "expect" syntax which was based on a naive implementation of hGetChar and checking for a desired result or timing out eventually. The really data inefficient version is a real screamer, beating the heck out of my Parsec or even ReadP version, but because of the way I wrote it, with a lot of reversing and prefix checking and substring slicing into temporary areas, it's not useful for large input blocks over a long time frame. Still it's so fast that it's appropriate for certain sections of code. I measured and verified this via space profiling to see the real runtime heap utilization (nice feature btw, I'd be dead without it right now I think). So what's the problem you're probably thinking? Well it turns out that since part of my state in StateT is a Data.Map, and that all my polling and parsing of expect-passing blocks caused updates to a Map, coupled with the language's default laziness caused a bit of a "bomb" of PAP (Partial APplications of functions). I had an ill-timed, project wise, discovery of a space leak. I tried sprinkling $! and seq all over the place, rewriting big chunks of code that used Text.Regex.Posix, to use Parsec and only got incremental improvements. The growth problem still existed, and would eventually exhaust memory. This was a real problem as this was an application that was not supposed to stop when the other conditions of the management system were ok. It could run for months or even years! I went through a whirlwind of emotions, and considered that perhaps I should be working a different job. Perhaps I could be a lion tamer? It turned out what I thought was a lion was really an anteater though... but that's literally a different story. It turns out that by looping not inside the monad, but over the execStateT/runReaderT expression I could pull all the state out, and then re-inject it into another execStateT/runReaderT each poll, which forced the strictness I needed on all state data, made the system respond faster, and best of all, not crash!!!! Diagrams below: This first one is the "before" picture. It shows the data growth by cost center in my code. As you can see things are getting worse almost linearly as I poll. This picture illustrates the result of pulling the state out of the Monad, and re-injecting it, forcing it to be evaluated. As you can see, I've got much more manageable memory utilization. This final one shows the new algorithm running with 2 threads, one spawned a few seconds into the run. You can see the initial burst of the fast but inefficient "expect" algorithm, followed by a much more regular memory utilization pattern. I'd like to thank all the folks on #haskell on FreeNode who gave me suggestions and hints to my vague problems regarding data growth and laziness vs strictness. Haskell's got a great user community and is probably one of the most helpful out there. One can learn a lot just asking questions of the right mentors as well as by reading haskell-cafe and the various blogs that are out there. I'm hoping that this anecdote is useful to someone. On Wed, Nov 11, 2009 at 11:20 AM, Andy Stewart wrote:

David Leimbach writes:

...

As some of you may know, I've been writing commercial Haskell code for a little bit here (about a year and a half) and I've just recently had to write some code that was going to run have to run for a really long time before being restarted, possibly months or years if all other parts of the system cooperate as it's part of a server infrastructure management system.

I recently ran into some serious space leak difficulties that would ultimately cause this program to crash some time after startup (my simulator is also written in Haskell, and runs a LOT faster than the real application ever could, this has enabled me to fast forward a bit the data growth issues and crash in minutes instead of days!)

Anyway, rather than try to paste it all here with images and such I thought I'd stick it up on my blog so others could maybe benefit from the anecdote.  It's difficult to disclose enough useful information as it is commercial code not under an open source license, but there's neat diagrams and stuff there so hopefully the colors are at least amusing :-)

http://leimy9.blogspot.com/2009/11/long-running-haskell-applications.html Can you copy you blog at here? http://leimy9.blogspot.com/2009/11/long-running-haskell-applications.html is filter by GFW (http://en.wikipedia.org/wiki/Golden_Shield_Project) i can't see it.

About crash free program, you can consider multi-process design, keep Simple and Stable core running in RootProcess, and Core module won't crash, and make unstable module running in ChildProcess, if you occur some un-catch exception from ChildProcess, just reboot those sub-module.

I'm research some Haskell/Gtk+ program, sometimes un-catch exception is unavoidable, multi-process is good chose to avoid some exception crash all program.

Cheers,

 -- Andy

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Is there a state monad that is strict on the state but lazy on the computation? Of course, strictness in the state will force a portion of the computation to be run, but there may be significant portions of it which are not run. Would there be a way to write a state monad such that it is entirely lazy, but then to wrap either the computation or the state in an 'eager' strategy datatype which takes care of this in a more flexible manner? Thanks, Matthew 2009/11/11 Bryan O'Sullivan

On Wed, Nov 11, 2009 at 7:43 AM, David Leimbach wrote:

...

I recently ran into some serious space leak difficulties that would ultimately cause this program to crash some time after startup (my simulator is also written in Haskell, and runs a LOT faster than the real application ever could, this has enabled me to fast forward a bit the data growth issues and crash in minutes instead of days!)

It sounds to me like you were storing a Map in a StateT. Since the usual State and StateT monads don't force the evaluation of their payload, I'm not terribly surprised that such a leak should arise.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Wed, Nov 11, 2009 at 10:29 AM, David Menendez wrote:

...

Is there a state monad that is strict on the state but lazy on the computation? Of course, strictness in the state will force a portion of

On Wed, Nov 11, 2009 at 1:09 PM, Matthew Pocock wrote: the

...

computation to be run, but there may be significant portions of it which are not run. Would there be a way to write a state monad such that it is entirely lazy, but then to wrap either the computation or the state in an 'eager' strategy datatype which takes care of this in a more flexible manner?

I think replacing "put s" with "put $! s" should guarantee that the state is evaluated.

If you're using get and put in many place in the code, you could try something along these lines:

newtype SStateT s m a = S { unS :: StateT s m a } deriving (Monad, etc.)

instance (Monad m) => MonadState s (SStateT s m) where get = S get put s = S (put $! s)

That's interesting, and once I have time to come back to this part of the project (I was behind schedule at this point!) I'll try something like that.

-- Dave Menendez http://www.eyrie.org/~zednenem/ _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

2009/11/12 Heinrich Apfelmus

Interestingly, this is different from Control.Monad.State.Strict . The latter never forces the state itself, just the pair constructor of the (result,state) pair.

Yes. This bit me the first time I came across it. I think we need a Control.

On Thu, Nov 12, 2009 at 8:01 AM, Matthew Pocock wrote:

Yes. This bit me the first time I came across it. I think we need a Control.Monad.State.StrictOnState with strict behaviour on the state value. I notice this same underlying issue is coming up in more than one thread on these lists.

This monad would indeed be useful, however it may be confusing as well. For example, if you are inserting elements in a Map using a lazy insert, seq'ing the map will only reduce it to WHNF (of course) and move the insertion down one level in the tree. IOW, I'd guess that OP's problem can't be solved just by seq'ing the map, he would have to use strict operations. There's a strict insertWith in Data.Map, but IIRC it's strict only on its combining operation, not on the insertion itself. Hmmm... maybe Control.Monad.State.RnfOnState? =P Cheers, -- Felipe.

On Wed, Nov 11, 2009 at 9:51 AM, Bryan O'Sullivan wrote:

On Wed, Nov 11, 2009 at 7:43 AM, David Leimbach wrote:

...

I recently ran into some serious space leak difficulties that would ultimately cause this program to crash some time after startup (my simulator is also written in Haskell, and runs a LOT faster than the real application ever could, this has enabled me to fast forward a bit the data growth issues and crash in minutes instead of days!)

It sounds to me like you were storing a Map in a StateT. Since the usual State and StateT monads don't force the evaluation of their payload, I'm not terribly surprised that such a leak should arise.

That's exactly what was happening. The system was being far too lazy (by choices I made but didn't fully understand). By pulling the Map out of the state, and pushing it back into the state, as the definition of my looping, things got a lot better. I didn't see another *easy* way to force the state to be evaluated, except by doing IO on intermediate values. seq will only evaluate strictly if it's just underneath something else that's already been evaluated :-). The runtime doesn't look for "seq"s to force evaluation on. I figured I was better off just creating a dependency in the evaluation, near the outermost portion of the program (the loop) that would cause a strict evaluation, and so far I was right :-) Program behaves very well now, and responds much better too. Dave

On Wed, Nov 11, 2009 at 11:19 AM, Don Stewart wrote:

leimy2k:

...

I figured I was better off just creating a dependency in the evaluation, near the outermost portion of the program (the loop) that would cause a strict evaluation, and so far I was right :-)

Program behaves very well now, and responds much better too.

Do you know if Control.Monad.State.Strict is enough to get the behaviour you need?

I'll give that a go. Most of my trouble figuring out the space leak has been around identifying what was really responsible for the problem. The functions that were listed as eating the space in -hc runs were not ultimately the ones causing the lack of strictness, in that they would have had to have been evaluated at a higher layer in a non-lazy way. So my take away from all of this is, when you have a space leak in haskell, start from the outer most evaluations inward, not the other way around!!! I think that would have saved me a ton of time. Dave

-- Don

Also, a *service* should have a persistence periodical action which should evaluate (most part of) the state thunks in their values to be serialized. This work for the structure similarity of NFData and Show/Binary classes. When there is a not directly serializable part of the state , things can get complicate, but resolving the persistence issue for those parts should resolve also the space leak, I think. paolino 2009/11/11 Paolino

Hello leimy, the only simple solution I have found to avoid a leaking state of a server is doing a periodical rnf of it, this implying the NFData constraint on its datatype. The reader should leak only if you nest forever the "local" function.

paolino

2009/11/11 David Leimbach

...

As some of you may know, I've been writing commercial Haskell code for a little bit here (about a year and a half) and I've just recently had to write some code that was going to run have to run for a really long time before being restarted, possibly months or years if all other parts of the system cooperate as it's part of a server infrastructure management system.

I recently ran into some serious space leak difficulties that would ultimately cause this program to crash some time after startup (my simulator is also written in Haskell, and runs a LOT faster than the real application ever could, this has enabled me to fast forward a bit the data growth issues and crash in minutes instead of days!)

Anyway, rather than try to paste it all here with images and such I thought I'd stick it up on my blog so others could maybe benefit from the anecdote. It's difficult to disclose enough useful information as it is commercial code not under an open source license, but there's neat diagrams and stuff there so hopefully the colors are at least amusing :-)

http://leimy9.blogspot.com/2009/11/long-running-haskell-applications.html

Dave

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe