Re: slow load and typecheck

Are you writing interface files (-fwrite-interface)? It makes no sense for HscInterpreted to be faster than HscNothing. Cheers, Matt On Tue, Oct 8, 2019 at 3:30 PM Sam Halliday wrote:

A quick follow-up to this, Rahul Muttinieni gave me some advice to try out

HscInterpreted / LinkInMemory

instead of

HscNothing / NoLink

and now I am no longer seeing home modules being compiled, and everything is a lot faster. Woohoo!

But I have no idea why this speeds things up... my code isn't using TemplateHaskell so HscNothing should really mean "don't do any codegen". Something is causing the HscNothing to be ignored. I'd still really like to get to the bottom of this so if anybody knows how the batch compiler is able to avoid recompiling home modules then please let me know... I would like to continue using HscNothing instead of HscInterpreted.

Sam Halliday writes:

...

Hello all,

I am writing an interactive tool using the ghc api. It is able to load and typecheck a source file in a user's package.

I obtain the flags that cabal uses to compile the user's package via the hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool, then I `setTargets' all the home modules (with targetAllowObjCode=True).

I use HscNothing and NoLink because I only want access to the trees, I don't want to produce any output files.

For the file that I wish to inspect, I `removeTarget' the module and `addTarget` it again but this time providing the full path to the file and don't allow object code.

Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume that the file under inspection only has a module definiton and no imports or top levels.

Functionally, my code is working great and I am able to do what I want with the typechecked tree.

However, load is very slow (~10 seconds user time) on large projects. Here is a cpu time trace of my program (milliseconds):

main 1 parse flags 93 load 20436 typecheck 20437

I can enable a bit more ghc timing info via -Rghc-timings and I see

!!! Chasing dependencies: finished in 157.20 milliseconds, allocated 528.112 megabytes

This seems fine, anything sub-second is ok.

But then I see a bunch of home modules in CodeGen that I was not expecting:

!!! CodeGen [My.Module.Dependency]: finished in 3335.62 milliseconds, allocated 270.615 megabytes

So it looks like the targetAllowObjCode is being ignored... is there any way to force it? Actually I'd prefer to fail fast than to ever compile or codegen a dependency module.

I know that it should be possible to load the module a lot faster because if I make a small change in the file under inspection and ask cabal to recompile the module it is super fast (less than a second).

Could somebody who understands how incremental/partial compiles work please help me out?

PS: If this textual description is confusing, I could put together a minimal reproduction and example project but it will take me some time to do that.

-- Best regards, Sam

-- Best regards, Sam _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs