Hi all, Shayan, have you written anything describing how things are going? Ben, thank you for reaching out. I am not sure about the history and the context of the discussions in this thread so far, but here is a brief description of what we intend to do and how far we have come so far. The idea is simple: allow grand larceny, with a clear conscience! We are trying to allow metaprograms to directly access and reuse the "native" machinery inside the compiler. There is no need to have a separate representation of syntax, and the associated sets of tools, as in Template Haskell, or even as in third-party libraries like Haskell-Src-Exts (HSE) and others in Haskell-Suite. GHC has a "native" representation of terms, with many tools already built on top of that, including the compiler passes like the renamer, or the typechecker. We would certainly like to unify these representations and tools as much as possible, and allow metaprograms to directly access these internal machineries. The eventual goal is indeed more than sole reuse of the AST in GHC (HsSyn); metaprograms should also be able to reuse the "infrastructure" like the different environments and monads used for name resolution or typechecking. If we are to treat `deriving`, and many other similar constructs in the language simply as metaprograms, access to types is *sometimes* crucial: they describe a type-directed elaboration process. I am a strong advocate of such simplifications for both the front-end (i.e., how users perceive constructs in the language), and the back-end (i.e., how the compiler implements them). However, there is a huge, often underestimated, gap between having these constructs as built-in, and having them as yet another metaprogram. The gap is both in theory, e.g., the equational and algebraic properties of the constructs, and in practice, e.g., handling the error messages and maintain the general macro/metaprogramming system allowing for such definitions. Many researchers, including me, have worked on these problems; there is still work to be done. We can discuss some of these related works, if you are interested. For now, as the first step, we are focusing on how to reuse GHC AST (HsSyn) for metaprogramming. The immediate problem with reusing GHC AST is that it comes with a large set of extra fields and constructors carrying the information only necessary for the passes inside GHC. Users (metaprogrammers?) do not want to, and do not need to, deal with these extra fields and constructors. Moreover, the AST should be, to some degree, easy to use, and we have no exact metric for the ease of use. Our solution is twofold, addressing the two mentioned problems: (a) after a rather deep analysis, we have come up with a simple, yet powerful, encoding of extensible ASTs that allows us to extend a base AST with new fields and constructors; (b) we are updating GHC AST to match HSE AST, hence unifying the two (based on its popularity, we can say HSE is easy enough to use). In theory, due to (a), code for GHC passes do not need to change. In practice, due to (b), we may need to update the code in GHC passes in some cases. It is needless to say, that this is all work in progress. Following Simon's suggestion, I am planning to give a talk on this in Haskell Implementors' Workshop in Japan. But since then, for further information, you may be brave enough to read our sketchy notes / pieces of code: - An example using the extensible encoding (a bit outdated variant though): https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HsSyn/Examp... - Discussion on why this extensible encoding is suitable: https://github.com/shayan-najd/NativeMetaprogramming/wiki/Extensions-&-Annot... - To see progress on unifying HSE AST and GHC AST compare the updated GHC AST at https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HsSyn/Synta... and the updated HSE AST at https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HSE/SyntaxE... They should be line-by-line comparable unless the comments says so otherwise. There is much to be done without a doubt, specially to synchronise with and learn form communities working on similar topics Yours, Shayan On Fri, Aug 5, 2016 at 11:06 AM, Ben Gamari <ben@smart-cactus.org> wrote:

Ryan Scott <ryan.gl.scott@gmail.com> writes:

...

Sorry for not including the full context on that link. It's part of a Summer of Haskell 2016 project called Native Metaprogramming in Haskell [1] (a.k.a. Introspective Template Haskell [2]), aiming to fix Trac #11081 [3].

On this note, it would be great to hear a bit about the state of this project. Shayan, have you written anything describing how things are going? It would be great if you could update the Wiki page [2] describing a bit about the approach you have taken and the current state of your implementation.

Cheers,

- Ben