Hello, @Ben
I tried 8.4 and indeed the renamer/typechecker takes dramatically less memory (yay!), leaving desugar as the largest allocator (allocating around 3GB) while compiling the minimal example. Furthermore, disabling the pattern match checker (via -fmax-pmcheck-iterations=0) dramatically reduces desugarer allocations (to around 60MByte). It looks like you are right.
I have just tried compiling my code with 8.4.2 and using -fmax-pmcheck-iterations=0, I gave GHC 12GB of ram and it still ran out (through `ulimit -v$((1024*1024*12))`). I saw many warnings about the pattern match checker exceeding the iterations I specified: warning: Pattern match checker exceeded (0) iterations in a case alternative. (Use -fmax-pmcheck-iterations=n to set the maximun number of iterations to n) Which makes me wonder, is the option really stopping the checker or just displaying something when it exceeds the given number?
Unfortunately, it's not clear what can be done to work around this short of disabling the match checker for the entire module, as suggested by mpickering. Ideally we would have some notion of local warning flags (e.g. #602), allowing us to disable this (rather expensive) warning for generated code. Sadly, we aren't there yet. Perhaps you would like to propose such a mechanism?
You guys are much more indicated to propose mechanisms to fix this. If you think this is a sensible mechanism to implement, sure! I'd be happy to propose it. @Simon
Let's make a Trac ticket for this (since there is a nice small reproducer). Or, if it seems a dup, attach the reproducer to some existing ticket.
Should I be the one creating it or adding the reproducer? The closest I've seen is: https://ghc.haskell.org/trac/ghc/ticket/11195
Maybe we should revert the (recent) patch that makes TH code behave like source code, until this pattern-match check business is solved.
Let me know when this happens, I'll be happy to test my code again! Thanks! Cheers, Victor