On 18/04/14 20:42, Austin Seipp wrote:

Yes, you're right regarding the AMP patches. Concerning this, I looked the patch back up to refresh my memory, and there reason we do this was because we need to check (and warn) if there are locally defined names in the current module which would clash with names from the Applicative-Monad proposal, among others - notably, naming functions like 'join' or whatnot in the current scope clashes. This means we wire in the 'Name's for join, <*>, etc from Control.Applicative and Control.Monad so we can check against them later, which implies loading the interfaces I believe.

Perhaps we don't even need a wired Name for this particular step, which would sidestep that issue.

But thinking about it - even fixing that, is this even avoidable, ultimately? The AMP warnings in 7.8 are actually temporary, and will be gone from HEAD soon. But once we do that and AMP is actually *implemented* in base, won't it essentially imply the loading of these same interfaces by default, and thus about the same amount of memory use?

I don't think it will. Except for orphan modules, interfaces are normally only loaded when we actually use something defined in that module. Perhaps this isn't a huge deal since it's a small fixed overhead, but it bugs me nonetheless. Cheers, Simon

Or is there a planned module refactoring/shuffling too, since we're already breaking some user code this cycle? If there is, maybe this won't be problematic in the end or we should revisit it when the numbers are solid. I honestly don't know what all the expected intra-module changes might be (I've CC'd Edward in case he'd like to chime in about that.)

On Fri, Apr 18, 2014 at 10:39 AM, Simon Marlow wrote:

...

I noticed that T1969 is failing again, and decided to do a little investigation. The maximum residency when compiling T1969 with HEAD compared with 7.6.3 looks like this:

7.6.3: 10,473,920 HEAD: 13,783,536

This is with +RTS -h -i0.01 to avoid sampling errors as much as possible. The figures are pretty accurate, and the heap profiles confirm it: we're using a lot more heap now.

-ddump-if-trace shows that HEAD is reading more interface files:

Renamer stats: 10 interfaces read 6 type/class/variable imported, out of 1361 read 0 instance decls imported, out of 105 read 0 rule decls imported, out of 53 read

vs. with 7.6.3:

Renamer stats: 8 interfaces read 2 type/class/variable imported, out of 828 read 0 instance decls imported, out of 40 read 0 rule decls imported, out of 45 read

The extra interface files are Control.Applicative and Control.Monad. So the question is, why are we now reading these on *every* compilation? (T1969 imports only the Prelude). Presumably this is something to do with the AMP warnings, but can't we lazily load these interfaces when they're needed?

Cheers, Simon _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

That would fix it in HEAD, but not 7.8, of course. Cheers, Simon On 18/04/14 23:33, Simon Peyton Jones wrote:

Now we have forked the tree - we can make Applicative a superclass of Monad - and remove all the AMP warning stuff That should resolve this particular issue, shouldn't it?

Austin, would you like to go ahead and do that?

Simon

| -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Simon | Marlow | Sent: 18 April 2014 21:32 | To: Austin Seipp | Cc: ghc-devs@haskell.org | Subject: Re: Heap usage of GHC increased in 7.8 vs. 7.6 | | On 18/04/14 20:42, Austin Seipp wrote: | > Yes, you're right regarding the AMP patches. Concerning this, I looked | > the patch back up to refresh my memory, and there reason we do this | > was because we need to check (and warn) if there are locally defined | > names in the current module which would clash with names from the | > Applicative-Monad proposal, among others - notably, naming functions | > like 'join' or whatnot in the current scope clashes. This means we | > wire in the 'Name's for join, <*>, etc from Control.Applicative and | > Control.Monad so we can check against them later, which implies | > loading the interfaces I believe. | > | > Perhaps we don't even need a wired Name for this particular step, | > which would sidestep that issue. | > | > But thinking about it - even fixing that, is this even avoidable, | > ultimately? The AMP warnings in 7.8 are actually temporary, and will | > be gone from HEAD soon. But once we do that and AMP is actually | > *implemented* in base, won't it essentially imply the loading of these | > same interfaces by default, and thus about the same amount of memory | > use? | | I don't think it will. Except for orphan modules, interfaces are | normally only loaded when we actually use something defined in that | module. | | Perhaps this isn't a huge deal since it's a small fixed overhead, but it | bugs me nonetheless. | | Cheers, | Simon | | | > Or is there a planned module refactoring/shuffling too, since we're | > already breaking some user code this cycle? If there is, maybe this | > won't be problematic in the end or we should revisit it when the | > numbers are solid. I honestly don't know what all the expected | > intra-module changes might be (I've CC'd Edward in case he'd like to | > chime in about that.) | > | > | > On Fri, Apr 18, 2014 at 10:39 AM, Simon Marlow | wrote: | >> I noticed that T1969 is failing again, and decided to do a little | >> investigation. The maximum residency when compiling T1969 with HEAD | >> compared with 7.6.3 looks like this: | >> | >> 7.6.3: 10,473,920 | >> HEAD: 13,783,536 | >> | >> This is with +RTS -h -i0.01 to avoid sampling errors as much as | possible. | >> The figures are pretty accurate, and the heap profiles confirm it: | we're | >> using a lot more heap now. | >> | >> -ddump-if-trace shows that HEAD is reading more interface files: | >> | >> Renamer stats: 10 interfaces read | >> 6 type/class/variable imported, out of 1361 read | >> 0 instance decls imported, out of 105 read | >> 0 rule decls imported, out of 53 read | >> | >> vs. with 7.6.3: | >> | >> Renamer stats: 8 interfaces read | >> 2 type/class/variable imported, out of 828 read | >> 0 instance decls imported, out of 40 read | >> 0 rule decls imported, out of 45 read | >> | >> The extra interface files are Control.Applicative and Control.Monad. | So the | >> question is, why are we now reading these on *every* compilation? | (T1969 | >> imports only the Prelude). Presumably this is something to do with | the AMP | >> warnings, but can't we lazily load these interfaces when they're | needed? | >> | >> Cheers, | >> Simon | >> _______________________________________________ | >> ghc-devs mailing list | >> ghc-devs@haskell.org | >> http://www.haskell.org/mailman/listinfo/ghc-devs | >> | > | > | > | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs