Profiling makes memory leak go away? Is Haskell a practical language?
Chasing down my memory leak I got into a weird situation where adding a magic manual SCC section and compiling with -prof makes the leak disappear. Now, I'll take any solution I can find - but this black voodoo only works for the short program I created for investigating the leak. It fails on the real program. Probably not enough fingernails in the voodoo doll or something :-( The code is in http://hpaste.org/1314#a1 if anyone at all is willing to take pity on me and explain what I'm doing wrong. This is the second system I tried writing in Haskell, and in both cases I hit a brick wall due to uncontrollable memory consumption. I'm just about ready to give up on Haskell as a practical language for non-trivial tasks. I can achieve the results I want in very short, elegant code... and as such it will have a place in my toolbox for various small tasks... but I lack the deep lore required to make it behave well for "large" inputs. At this point I have to take it on faith that it is even possible to do so in the first place :-(
Oren Ben-Kiki wrote:
The code is in http://hpaste.org/1314#a1 if anyone at all is willing to take pity on me and explain what I'm doing wrong.
There is an important point to note about streaming, namely that it conflicts with knowing whether the input is syntactically correct or not. In other words, evaluating (rResult reply) parses the hole input to the bitter end. For example, take your input data cycle "a\n" = "a\na\na\na\n..." Can we determine whether (rResult reply) is Nothing or Just after reading the first 'a'? No, we can't because the hole parse could fail far to the end resulting in Nothing. You've probably noticed this already and introduced rToken for that reason but I'm not sure.
Chasing down my memory leak I got into a weird situation where adding a magic manual SCC section and compiling with -prof makes the leak disappear.
That sounds fishy. Note that manual SCC annotations + optimization currently is a bit buggy in GHC, in the sense that optimization disturbs correct cost attribution. Does the program eat the output of yes 'a' without memory explosion showed by "top"? If so, it could perhaps be a problem of the optimization phase which maybe expands D.append parsed (case result of ...) to case result of { Nothing -> D.append parsed D.empty Just .. -> D.append parsed (D.singleton...) } The latter won't stream because of the reason noted above. But it's highly unlikely that GHC performs a transformation that changes semantics this radically.
I can achieve the results I want in very short elegant code...
In my opinion, your streaming parser is not as elegant as it could be which makes it hard to read the code. With monad transformers, we almost have Parser a ~= StateT State (WriterT (D.DList Token) Maybe) a ~= State -> (D.DList Token, Maybe (a,State)) Your Parser is slightly different but I think that the monad instances behave exactly the same way. The power functional programming is to assemble big things from small things. Monad transformers supply tons of small monads that you can reuse to build big monads. In a sense, the many pattern matches on Maybe have already been written down in general form by someone else. Also, I'm really missing type signatures, especially for many. When reading the code, I expected many :: Parser a -> Parser [a] but had to conclude many :: Parser a -> Parser () by inspecting its code. Regards, apfelmus
participants (2)
-
apfelmus -
Oren Ben-Kiki