
I like the idea! And it should be possible to build this without modifying GHC at all, on top of the GHC API. As you say, you'll need a server process, which accepts command lines, executes them, and sends back the results. A local socket should be fine (and will work on both Unix and Windows).
I took a whack at this, but I'm having to backtrack a bit now because I don't fully understand the GHC API, so I thought I should explain my understanding to make sure I'm on the right track. It appears the cached information I want to preserve between compiles is in HscEnv. At first I thought I could just do what --make does, but what it does is call 'GHC.load', which maintains the HscEnv (which mostly means loading already compiled modules into the HomePackageTable, since the other cache entries are apparently loaded on demand by DriverPipeline.compileFile). But actually it does a lot of things, such as detecting that a module doesn't need recompilation and directly loading the interface in that case. So I thought it would be quickest to just use it: add a new target to the set of targets and call load again. However, there are problems with that. The first is it doesn't pay attention to DynFlags.outputFile, which makes sense because it's expecting to compile multiple files. The bigger problem is that it apparently wants to reload the whole set each time, so it winds up being slower rather than faster. I guess 'load' is really set up to figure out dependencies on its own and compile a set of modules, so I'm talking at the wrong level. So I think I need to rewrite the HPT-maintaining parts of GHC.load and write my own compileFile that *does* maintain the HPT. And also figure out what other parts of the HscEnv should be updated, if any. Sound about right? Along the way I ran into the problem that it's impossible to re-parse GHC flags to compare them to previous runs, because static flags only export a parsing function that mutates global variables and can only be called once. So I parse out the dynamic flags, strip out the *.hs args, and assume the rest are static flags. I noticed comments about converting them all to dynamic, I guess that might make a nice housekeeping project some day.