On 2017-12-08 01:52 AM, Sven Panne wrote:

… making something a CAF is not always an optimization: If your evaluated CAF uses e.g. hundreds of MB it might be preferable to not have it as a CAF, but to to recompute it instead.

Good point. However, I think it would be very hard for the compiler to detect which CAFs use a lot of memory and aren’t referred to very often.

If I haven’t missed something recently, CAFs can’t be garbage collected, but I would be happy to be corrected here. :-)

There’s a helpful discussion on the Haskell Wiki here. In particular, it says:

A CAF … may only be accessible from within the code of one or more functions. In order for the garbage collector to be able to reclaim such structures, we associate with each function a list of the CAFs to which it refers. When garbage collecting a reference to the function we collect the CAFs on its list.

In my case, since the function itself is top-level, I assume the CAF can’t be garbage-collected.

Enabling profiling shouldn’t change the meaning of a program.

It doesn’t change the meaning, that would really be a desaster, it just changes the performance characteristics. This is still not nice, but not much different from using different levels of optimization: Doing or not doing e.g. strictness analysis might change the space complexity etc.

Yes, I see you’re right. However, I guess I was using ‘meaning’ loosely to mean ‘is a CAF’ and I assumed a CAF would always be floated. The above-mentioned wiki page says, “A CAF can always be lifted to the top level of the program.” However, I realize “can” is not the same as “should”.

I remember back in the day having to be careful with regexes in Python to make sure they were always precompiled outside of loops and functions, but one of the nice things about Haskell is that one can usually let the compiler take care of this. (Nowadays Python gets around this by caching compiled regexes, but I prefer Haskell’s statically-optimized approach.)

I think totally relying on the compiler for performance is a common misconception, because things are often more tricky than initially thought. In our example: Time vs. space tradeoff, and there is no universally “right” solution.

Good point. Having great optimization isn’t a justification for complete mental laziness on the part of the programmer! However, I did find the behaviour in this case surprising and unintuitive, given ghc’s usual ability to optimize things, and having it change on me when I enabled profiling left me wondering where to go next. The code I presented here is considerably simplified from the original program, and represents a lot of work already expended trying to get to the root of the problem.