On Feb 7, 2007, at 12:29 AM, Simon Peyton-Jones wrote:

| I am still working on it. Some external events have slowed me down a | little (research, classes, appendicitis), and it has involved more | changes to the innards of GHC than anticipated, but it is still | moving along. If you can wait a little while, it should be possible | to use GHC.

I'm delighted to hear it; thank you. Maybe there are others who'd like to join in?

Keep me posted about those "changes to the innards" you mention!

I think I made that sound scarier than it really is. :-) We talked earlier about replacing the Kind in HsTyVarBndr with a new HsKind which is to HsType as Kind is to Type. Making that change has caused ripples throughout the system, which have taken me a while to track down and fix. All of the changes are superficial (often as simple as changing type signatures), but they touch more modules than I had expected. And that means I've had to figure out the workings of a number of modules that are essentially unrelated to External Core. That, along with many distractions, has made it take me a while. I'll get the wiki page updated in the near future with the current status. Aaron

On Feb 7, 2007, at 12:29 AM, Simon Peyton-Jones wrote:

| I am still working on it. Some external events have slowed me down a | little (research, classes, appendicitis), and it has involved more | changes to the innards of GHC than anticipated, but it is still | moving along. If you can wait a little while, it should be possible | to use GHC.

I'm delighted to hear it; thank you. Maybe there are others who'd like to join in?

Keep me posted about those "changes to the innards" you mention!

Now that I've had a chance to revisit this code for the first time in a while, I think I'm stuck. The difficulty comes from the change I mentioned in my previous reply: the KindedTyVar constructor in the HsTyVarBndr type used to associate a variable with a Kind. This was a quick shortcut in the past, because Haskell source doesn't contain complex kinds with variables that must be renamed and so forth. With External Core and FC's coercions, this assumption no longer holds. So, with the hope of sharing as much code as possible, I've made KindedTyVar associate each variable with an HsType (which encodes kinds similarly to how Type does in later stages). Unfortunately, some code seems to depend on kinds being in the same form throughout much of the compilation pipeline. In particular, the code for type checking type and class declarations (typecheck/ TcTyClsDecls.lhs) maintains a kind environment which I believe is constrained to contain TcTyThings (though I'm not sure about this). At the very least, this environment seems to be intended to map variables to "processed" types or kinds (Type rather than HsType, etc.). Anyway, some of the functions in TcTyClsDecls.lhs, such as kcTyClDecl, take (variable, kind) pairs out of the environment and stick them back into HsSyn things, such as the tcdTyVars field of a ClassDecl. I suspect it's possible to rewrite the type checking code to not do this, but I don't currently feel like I know enough about the intricacies of type checking full Haskell to do this. It might involve some serious changes. The only real alternative I see is to write a function that can convert fully-processed Kinds back into an HsType encoding. This seems like a nasty hack, though. Does anyone with more experience with this area of the code have any suggestions? Thanks, Aaron

Hi, On more careful re-reading of the following email, which you sent at the beginning of December, I think perhaps you meant to say IfaceSyn where you said HsSyn? That's what got me going down the path of implementing external core with HsSyn (and a special type for external core expressions distinct from HsExpr). It seemed like a great idea at the time, but it's definitely getting hairy now that I delve into it. I think I also remember someone saying (though I can't find the email anymore) that it would be best to move away from using IfaceSyn on input, because its typechecker expects well-typed input and doesn't deal with errors nicely. Is there any intrinsic reason why it can't deal with errors better? Perhaps the best strategy is to use IfaceSyn, as before, and later, once it's working, improve the error handling in its typechecker. Anyway, using IfaceSyn in both directions (your alternative (a)) is sounding better to me at this point. I'll get to work on it. Thanks for all of the hand-holding. It's been quite helpful. Aaron

The present situation is very strange - On output we generate a program of type ExternalCore.Module - On input we generate a program of type HsSyn.Module The two data types are different in almost every respect. This seems to be asking for trouble.

The reason we use HsSyn on input is so that we can share typechecking code. That is a good reason in my view.

My suggestion (without a great deal of thought) would be

* Dump the ExternalCore datatype entirely, or at least move it to a reference implementation in a completely separate Darcs project. (The reference impl could read the module, perform a no-op transformation, and print it out again.) The data type makes sense as a way to define what ExtCore is, but it does not make sense as part of GHC's implementation thereof, I think.

* Use HsSyn for the output side, so that (parse (print p)) is the identity. That means that MkExternalCore would generate HsSyn; and PprExternalCore would print the appropriate subset of HsSyn in ExtCore syntax.

* It also means we'd need to add a new constructor to HsBinds.HsBind for ExtCore bindings of form x=e; plus of course a supporting data type HsCoreExpr for the RHS. We'd need new typechecking code for HsCoreExpr; but the code in TcIface is really the Wrong Thing for this -- it does not expect to find errors.

* Be careful to specify the sub-language of HsSyn that is ExtCore

Simon

On Feb 9, 2007, at 1:02 AM, Simon Peyton-Jones wrote:

Aaron

At the moment, on the way *out* of GHC, it goes like this: Core --> converted to ExternalCore --> printed to file

On the way *in* we see file --> parsed to a mixture of HsSyn and IfaceSyn --> typechecked to Core

This is all a bit of a (historically-derived) nonsense, as I think we discussed. And it gives rise to the problems you are seeing, because the HsSyn part has this HsTyVarBndr etc.

The same intermediate data type should be used in both directions. I forget, but I think we discussed these alternatives a) using IfaceSyn in both directions b) or defining a new data type

The advantage of (b) is that you can use a purpose-designed data type. But the disadvantage is that you need to write a type checker.

I'd be inclined to try (a). After all, that's the way that much of the input side works already. And IfaceSyn has stuff for expressing data type declarations etc. The advantage is that all the typechecking part is already done.

Does this ring any bells?