Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs. In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well. On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering < matthewtpickering@gmail.com> wrote:

Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő wrote:

...

On Mon, 10 Feb 2025, Matthew Pickering wrote:

...

I wonder if you have got your condition the wrong way around.

The only "safe" time to perform rehydration is AFTER the point it can never be used again.

If you rehydrate it just before it is used then you will repeat work which has already been done. If you do this, you will always have a trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?