GHC 6.10.2 consuming lots of memory while compiling - help?

I apologize in advance for the vagueness of my report here - it's one of those situations I'm not sure how to cut it down to size yet. I have a module that uses HaskellDB and Template Haskell together. The module itself depends on 23 other modules, each of which give a type definition for a particular database table or view. I only mention that to emphasize that the module depends on some "big" types (HList records w/ 20+ members) and on compile-time generated code. My problem is this - when GHC compile the module, it consumes 1.2 GB of memory, takes about 10 minutes, and finally produces an object file. The memory usage seems related to template haskell, but I'm not positive. I've attached verbose output from compiling the module in question. The command line I used was: ghc -v --make -c DeliveryManagementQueries.hs -XEmptyDataDecls -XTypeSynonymInstances -XTemplateHa skell Now for my question - please ignore the specifics of haskelldb/template haskell - any suggestions for figuring out what GHC is doing, besides tried-and-true divide and conquer? Justin

jgbailey:
I apologize in advance for the vagueness of my report here - it's one of those situations I'm not sure how to cut it down to size yet.
I have a module that uses HaskellDB and Template Haskell together. The module itself depends on 23 other modules, each of which give a type definition for a particular database table or view. I only mention that to emphasize that the module depends on some "big" types (HList records w/ 20+ members) and on compile-time generated code.
My problem is this - when GHC compile the module, it consumes 1.2 GB of memory, takes about 10 minutes, and finally produces an object file. The memory usage seems related to template haskell, but I'm not positive.
I've attached verbose output from compiling the module in question. The command line I used was:
ghc -v --make -c DeliveryManagementQueries.hs -XEmptyDataDecls -XTypeSynonymInstances -XTemplateHa skell
Now for my question - please ignore the specifics of haskelldb/template haskell - any suggestions for figuring out what GHC is doing, besides tried-and-true divide and conquer?
Oh, and I note you're not using -O or -O2 either? -- Don

Ironically adding -O does reduce the compile time in this case. I
shouldn't have been so quick to reply!
On Fri, Jul 24, 2009 at 2:51 PM, Don Stewart
jgbailey:
On Fri, Jul 24, 2009 at 2:02 PM, Don Stewart
wrote: Oh, and I note you're not using -O or -O2 either?
-- Don
This is a compile time problem, wouldn't -O make it worse?
Almost certainly!

On 24/07/2009 23:01, Justin Bailey wrote:
On Fri, Jul 24, 2009 at 2:51 PM, Don Stewart
wrote: jgbailey:
On Fri, Jul 24, 2009 at 2:02 PM, Don Stewart
wrote: Oh, and I note you're not using -O or -O2 either?
-- Don
This is a compile time problem, wouldn't -O make it worse?
Almost certainly!
Ironically adding -O does reduce the compile time in this case. I shouldn't have been so quick to reply!
-O may reduce compilation time by simplifying the code. It's rare in practice, but more likely to happen when you're dealing with a lot of automatically-generated code. To answer the original question, there are various debugging options that will give you an idea of what GHC is up to: -ddump-rn-trace, -ddump-tc-trace -ddump-splices. And +RTS -hT -RTS will give you a heap profile of GHC, which might provide more clues (though matching the profile with what GHC was doing at the time is mostly guesswork). If you think there's something we ought to investigate here, please make a self-contained test case and attach it to a ticket. Cheers, Simon

Justin Presumably DeliveryManagementQueries uses TH to generate lots of glop? *** Desugar: Result size = 616,969 That's a big program! What kind of glop is it? Maybe it's the same kind of thing as one of these? http://hackage.haskell.org/trac/ghc/query?status=new&status=assigned&status=reopened&type=compile-time+performance+bug&order=priority Regardless, if you can make a reproducible test case that'd help us. Probably you can do this simply by generating a file with lots of repeated glop of the same kind as your TH is spitting out. Simon | -----Original Message----- | From: glasgow-haskell-users-bounces@haskell.org [mailto:glasgow-haskell-users- | bounces@haskell.org] On Behalf Of Justin Bailey | Sent: 24 July 2009 20:03 | To: glasgow-haskell-users@haskell.org | Subject: GHC 6.10.2 consuming lots of memory while compiling - help? | | I apologize in advance for the vagueness of my report here - it's one | of those situations I'm not sure how to cut it down to size yet. | | I have a module that uses HaskellDB and Template Haskell together. The | module itself depends on 23 other modules, each of which give a type | definition for a particular database table or view. I only mention | that to emphasize that the module depends on some "big" types (HList | records w/ 20+ members) and on compile-time generated code. | | My problem is this - when GHC compile the module, it consumes 1.2 GB | of memory, takes about 10 minutes, and finally produces an object | file. The memory usage seems related to template haskell, but I'm not | positive. | | I've attached verbose output from compiling the module in question. | The command line I used was: | | ghc -v --make -c DeliveryManagementQueries.hs -XEmptyDataDecls | -XTypeSynonymInstances -XTemplateHa | skell | | Now for my question - please ignore the specifics of | haskelldb/template haskell - any suggestions for figuring out what GHC | is doing, besides tried-and-true divide and conquer? | | Justin
participants (4)
-
Don Stewart
-
Justin Bailey
-
Simon Marlow
-
Simon Peyton-Jones