Pre-Proposal: Introspective Template Haskell

Richard Eisenberg

11 Nov 2015 11 Nov '15

11:26 a.m.

Hi devs, There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets. I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied): 1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.) 2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices. 3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out. 4. Lifting would remain, doing the obvious thing. 5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows. So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday. And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.) What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem! Richard

Show replies by date

Eric Seidel

11 Nov 11 Nov

12:46 p.m.

I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly as bad as breaking most clients of TH. But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :) All in all, I like this idea a lot! Eric On Wed, Nov 11, 2015, at 08:26, Richard Eisenberg wrote:

...

Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Richard Eisenberg

12:50 p.m.

On Nov 11, 2015, at 12:46 PM, Eric Seidel wrote:

...

I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly as bad as breaking most clients of TH.

This is a very good point. We would want to bless some API that would remain stable. Then, clients that go around that API get what they deserve. A starting point for the stable API would be today's template-haskell (less some unsafe features, like exposing NameG).

...

But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :)

I actually think this *is* a likely scenario. In the last revision of GHC, lots of AST changes have taken place. We would want to insulate TH users from this. Richard

...

All in all, I like this idea a lot!

Eric

On Wed, Nov 11, 2015, at 08:26, Richard Eisenberg wrote:

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Eric Seidel

1:06 p.m.

On Wed, Nov 11, 2015, at 09:50, Richard Eisenberg wrote:

...

On Nov 11, 2015, at 12:46 PM, Eric Seidel wrote:

...
I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly as bad as breaking most clients of TH.

This is a very good point. We would want to bless some API that would remain stable. Then, clients that go around that API get what they deserve. A starting point for the stable API would be today's template-haskell (less some unsafe features, like exposing NameG).

...
But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :)

I actually think this *is* a likely scenario. In the last revision of GHC, lots of AST changes have taken place. We would want to insulate TH users from this.

You're talking about the ApiAnnotations stuff right? Or was there another big change? I had assumed that the ApiAnnotations changes fell into the "extension" category rather than "breaking", because presumably the changes amounted to adding a bunch of extra record fields (I didn't follow this patch so could be way off base...) Then again, even adding a record field is a breaking change as it changes the arity of the datacon... So I agree, we really ought to have a blessed API that we guarantee across versions. We could maintain it via pattern/type synonyms. (This would be a Good Thing even if we ignore your proposal!) Eric

Alan & Kim Zimmerman

1:10 p.m.

...

From an ApiAnnotations point of view, they will change to match changes in the AST, they cannot be a brake on improvements. We do intend to nudge the AST toward being more ApiAnnotations friendly over time.

If the AST is shared, then a scenario of generating TH and being able to pretty-print it in compilable form becomes possible. Alan On Wed, Nov 11, 2015 at 8:06 PM, Eric Seidel wrote:

...

On Wed, Nov 11, 2015, at 09:50, Richard Eisenberg wrote:

...
On Nov 11, 2015, at 12:46 PM, Eric Seidel wrote:

...
I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly

as

...
...
bad as breaking most clients of TH.

This is a very good point. We would want to bless some API that would remain stable. Then, clients that go around that API get what they deserve. A starting point for the stable API would be today's template-haskell (less some unsafe features, like exposing NameG).

...
But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :)

I actually think this *is* a likely scenario. In the last revision of GHC, lots of AST changes have taken place. We would want to insulate TH users from this.

You're talking about the ApiAnnotations stuff right? Or was there another big change?

I had assumed that the ApiAnnotations changes fell into the "extension" category rather than "breaking", because presumably the changes amounted to adding a bunch of extra record fields (I didn't follow this patch so could be way off base...)

Then again, even adding a record field is a breaking change as it changes the arity of the datacon... So I agree, we really ought to have a blessed API that we guarantee across versions. We could maintain it via pattern/type synonyms. (This would be a Good Thing even if we ignore your proposal!)

Eric _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Edward Kmett

2:25 p.m.

On Wed, Nov 11, 2015 at 12:50 PM, Richard Eisenberg wrote:

...

This is a very good point. We would want to bless some API that would remain stable. Then, clients that go around that API get what they deserve. A starting point for the stable API would be today's template-haskell (less some unsafe features, like exposing NameG).

As a data point, in a couple of packages I wind up forced into using mkNameG_v and mkNameG_tc in order to avoid incurring a dependency on a stage2 compiler today. Removing them would force me to drop support for stage1-only platforms offered by some linux distributions. If you're going to drop support for it, please consider offering me some horrible back door to get at the functionality that I can't currently replace by other means. -Edward

Richard Eisenberg

3:03 p.m.

On Nov 11, 2015, at 2:25 PM, Edward Kmett wrote:

...

As a data point, in a couple of packages I wind up forced into using mkNameG_v and mkNameG_tc in order to avoid incurring a dependency on a stage2 compiler today. Removing them would force me to drop support for stage1-only platforms offered by some linux distributions.

If you're going to drop support for it, please consider offering me some horrible back door to get at the functionality that I can't currently replace by other means.

I've had to use these functions, too, mostly because TH didn't export the functionality I needed. But this wouldn't be problematic in the new scenario: just depend on the ghc package instead of template-haskell. Then you can do whatever you like. :)

...

-Edward

Edward Kmett

3:06 p.m.

That would be a sufficient "horrible backdoor" for me. :) -Edward

...

On Nov 11, 2015, at 3:03 PM, Richard Eisenberg wrote:

...
On Nov 11, 2015, at 2:25 PM, Edward Kmett wrote:

As a data point, in a couple of packages I wind up forced into using mkNameG_v and mkNameG_tc in order to avoid incurring a dependency on a stage2 compiler today. Removing them would force me to drop support for stage1-only platforms offered by some linux distributions.

If you're going to drop support for it, please consider offering me some horrible back door to get at the functionality that I can't currently replace by other means.

I've had to use these functions, too, mostly because TH didn't export the functionality I needed. But this wouldn't be problematic in the new scenario: just depend on the ghc package instead of template-haskell. Then you can do whatever you like. :)

...
-Edward

Edward Kmett

2:21 p.m.

In practice I find that almost every piece of template-haskell code I've written gets broken by something every other release of GHC, so it hasn't exactly been a shining beacon of backwards compatibility thus far. Invariably it is always missing _something_ that I need, and anything that ties it to a more canonical form like this would be a very good thing. I'd strongly support this move. A sample just from my current working directory: haskell> grep -r MIN_VERSION_template_haskell */src bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,10,0) bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,7,0) bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,10,0) bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0) bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0) bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0) bifunctors/src/Data/Bifunctor/TH.hs:#ifndef MIN_VERSION_template_haskell bifunctors/src/Data/Bifunctor/TH.hs:#if __GLASGOW_HASKELL__ < 710 && MIN_VERSION_template_haskell(2,8,0) bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0) bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0) bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0) bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0) bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0) bifunctors/src/Data/Bifunctor/TH.hs:# if __GLASGOW_HASKELL__ >= 710 || !(MIN_VERSION_template_haskell(2,8,0)) bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0) free/src/Control/Monad/Free/TH.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Control/Lens/Internal/FieldTH.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Control/Lens/Internal/TH.hs:#ifndef MIN_VERSION_template_haskell lens/src/Control/Lens/Internal/TH.hs:#define MIN_VERSION_template_haskell(x,y,z) (defined(__GLASGOW_HASKELL__) && __GLASGOW_HASKELL__ >= 706) lens/src/Control/Lens/Internal/TH.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Control/Lens/Plated.hs:#if !(MIN_VERSION_template_haskell(2,8,0)) lens/src/Control/Lens/TH.hs:#ifndef MIN_VERSION_template_haskell lens/src/Control/Lens/TH.hs:#define MIN_VERSION_template_haskell(x,y,z) (defined(__GLASGOW_HASKELL__) && __GLASGOW_HASKELL__ >= 706) lens/src/Control/Lens/TH.hs:#if !(MIN_VERSION_template_haskell(2,7,0)) lens/src/Control/Lens/TH.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Control/Lens/TH.hs:#if !(MIN_VERSION_template_haskell(2,7,0)) lens/src/Language/Haskell/TH/Lens.hs:#ifndef MIN_VERSION_template_haskell lens/src/Language/Haskell/TH/Lens.hs:#define MIN_VERSION_template_haskell(x,y,z) 1 lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0) lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0) lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0) linear/src/Linear/V.hs:#ifdef MIN_VERSION_template_haskell linear/src/Linear/V.hs:#if !(MIN_VERSION_reflection(1,3,0)) && defined(MIN_VERSION_template_haskell) linear/src/Linear/V.hs:#if !(MIN_VERSION_reflection(1,3,0)) && defined(MIN_VERSION_template_haskell) nats/src/Numeric/Natural.hs:#if defined(MIN_VERSION_hashable) || defined(MIN_VERSION_template_haskell) nats/src/Numeric/Natural.hs:#ifdef MIN_VERSION_template_haskell nats/src/Numeric/Natural.hs:#ifdef MIN_VERSION_template_haskell tables/src/Data/Table.hs:#if MIN_VERSION_template_haskell(2,10,0) tables/src/Data/Table.hs:#if MIN_VERSION_template_haskell(2,9,0) tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0) tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0) tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0) tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0) On Wed, Nov 11, 2015 at 12:46 PM, Eric Seidel wrote:

...

I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly as bad as breaking most clients of TH.

But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :)

All in all, I like this idea a lot!

Eric

On Wed, Nov 11, 2015, at 08:26, Richard Eisenberg wrote:

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Thomas Bereknyei

2:53 p.m.

If I understand this correctly it would also subsume haskell-src-exts. If so I would definitely support this. On Nov 11, 2015 2:21 PM, "Edward Kmett" wrote:

...

In practice I find that almost every piece of template-haskell code I've written gets broken by something every other release of GHC, so it hasn't exactly been a shining beacon of backwards compatibility thus far.

Invariably it is always missing _something_ that I need, and anything that ties it to a more canonical form like this would be a very good thing.

I'd strongly support this move.

A sample just from my current working directory:

haskell> grep -r MIN_VERSION_template_haskell */src

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,10,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,10,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH.hs:#ifndef MIN_VERSION_template_haskell

bifunctors/src/Data/Bifunctor/TH.hs:#if __GLASGOW_HASKELL__ < 710 && MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:# if __GLASGOW_HASKELL__ >= 710 || !(MIN_VERSION_template_haskell(2,8,0))

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

free/src/Control/Monad/Free/TH.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Control/Lens/Internal/FieldTH.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Control/Lens/Internal/TH.hs:#ifndef MIN_VERSION_template_haskell

lens/src/Control/Lens/Internal/TH.hs:#define MIN_VERSION_template_haskell(x,y,z) (defined(__GLASGOW_HASKELL__) && __GLASGOW_HASKELL__ >= 706)

lens/src/Control/Lens/Internal/TH.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Control/Lens/Plated.hs:#if !(MIN_VERSION_template_haskell(2,8,0))

lens/src/Control/Lens/TH.hs:#ifndef MIN_VERSION_template_haskell

lens/src/Control/Lens/TH.hs:#define MIN_VERSION_template_haskell(x,y,z) (defined(__GLASGOW_HASKELL__) && __GLASGOW_HASKELL__ >= 706)

lens/src/Control/Lens/TH.hs:#if !(MIN_VERSION_template_haskell(2,7,0))

lens/src/Control/Lens/TH.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Control/Lens/TH.hs:#if !(MIN_VERSION_template_haskell(2,7,0))

lens/src/Language/Haskell/TH/Lens.hs:#ifndef MIN_VERSION_template_haskell

lens/src/Language/Haskell/TH/Lens.hs:#define MIN_VERSION_template_haskell(x,y,z) 1

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

linear/src/Linear/V.hs:#ifdef MIN_VERSION_template_haskell

linear/src/Linear/V.hs:#if !(MIN_VERSION_reflection(1,3,0)) && defined(MIN_VERSION_template_haskell)

linear/src/Linear/V.hs:#if !(MIN_VERSION_reflection(1,3,0)) && defined(MIN_VERSION_template_haskell)

nats/src/Numeric/Natural.hs:#if defined(MIN_VERSION_hashable) || defined(MIN_VERSION_template_haskell)

nats/src/Numeric/Natural.hs:#ifdef MIN_VERSION_template_haskell

nats/src/Numeric/Natural.hs:#ifdef MIN_VERSION_template_haskell

tables/src/Data/Table.hs:#if MIN_VERSION_template_haskell(2,10,0)

tables/src/Data/Table.hs:#if MIN_VERSION_template_haskell(2,9,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

On Wed, Nov 11, 2015 at 12:46 PM, Eric Seidel wrote:

...
I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly as bad as breaking most clients of TH.

But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :)

All in all, I like this idea a lot!

Eric

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if

On Wed, Nov 11, 2015, at 08:26, Richard Eisenberg wrote: that's

...
the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Alan & Kim Zimmerman

2:55 p.m.

Well, as of 7.10.2 the GHC AST no longer has bottoms in it, so it can already be used quite easily On Wed, Nov 11, 2015 at 9:53 PM, Thomas Bereknyei wrote:

...

If I understand this correctly it would also subsume haskell-src-exts. If so I would definitely support this. On Nov 11, 2015 2:21 PM, "Edward Kmett" wrote:

...
In practice I find that almost every piece of template-haskell code I've written gets broken by something every other release of GHC, so it hasn't exactly been a shining beacon of backwards compatibility thus far.

Invariably it is always missing _something_ that I need, and anything that ties it to a more canonical form like this would be a very good thing.

I'd strongly support this move.

A sample just from my current working directory:

haskell> grep -r MIN_VERSION_template_haskell */src

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,10,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,10,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH/Internal.hs:#if MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH.hs:#ifndef MIN_VERSION_template_haskell

bifunctors/src/Data/Bifunctor/TH.hs:#if __GLASGOW_HASKELL__ < 710 && MIN_VERSION_template_haskell(2,8,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

bifunctors/src/Data/Bifunctor/TH.hs:# if __GLASGOW_HASKELL__ >= 710 || !(MIN_VERSION_template_haskell(2,8,0))

bifunctors/src/Data/Bifunctor/TH.hs:#if MIN_VERSION_template_haskell(2,7,0)

free/src/Control/Monad/Free/TH.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Control/Lens/Internal/FieldTH.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Control/Lens/Internal/TH.hs:#ifndef MIN_VERSION_template_haskell

lens/src/Control/Lens/Internal/TH.hs:#define MIN_VERSION_template_haskell(x,y,z) (defined(__GLASGOW_HASKELL__) && __GLASGOW_HASKELL__ >= 706)

lens/src/Control/Lens/Internal/TH.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Control/Lens/Plated.hs:#if !(MIN_VERSION_template_haskell(2,8,0))

lens/src/Control/Lens/TH.hs:#ifndef MIN_VERSION_template_haskell

lens/src/Control/Lens/TH.hs:#define MIN_VERSION_template_haskell(x,y,z) (defined(__GLASGOW_HASKELL__) && __GLASGOW_HASKELL__ >= 706)

lens/src/Control/Lens/TH.hs:#if !(MIN_VERSION_template_haskell(2,7,0))

lens/src/Control/Lens/TH.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Control/Lens/TH.hs:#if !(MIN_VERSION_template_haskell(2,7,0))

lens/src/Language/Haskell/TH/Lens.hs:#ifndef MIN_VERSION_template_haskell

lens/src/Language/Haskell/TH/Lens.hs:#define MIN_VERSION_template_haskell(x,y,z) 1

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,8,0)

lens/src/Language/Haskell/TH/Lens.hs:#if !MIN_VERSION_template_haskell(2,10,0)

lens/src/Language/Haskell/TH/Lens.hs:#if MIN_VERSION_template_haskell(2,9,0)

linear/src/Linear/V.hs:#ifdef MIN_VERSION_template_haskell

linear/src/Linear/V.hs:#if !(MIN_VERSION_reflection(1,3,0)) && defined(MIN_VERSION_template_haskell)

linear/src/Linear/V.hs:#if !(MIN_VERSION_reflection(1,3,0)) && defined(MIN_VERSION_template_haskell)

nats/src/Numeric/Natural.hs:#if defined(MIN_VERSION_hashable) || defined(MIN_VERSION_template_haskell)

nats/src/Numeric/Natural.hs:#ifdef MIN_VERSION_template_haskell

nats/src/Numeric/Natural.hs:#ifdef MIN_VERSION_template_haskell

tables/src/Data/Table.hs:#if MIN_VERSION_template_haskell(2,10,0)

tables/src/Data/Table.hs:#if MIN_VERSION_template_haskell(2,9,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

tagged/src/Data/Proxy/TH.hs:#if MIN_VERSION_template_haskell(2,8,0)

On Wed, Nov 11, 2015 at 12:46 PM, Eric Seidel wrote:

...
I think backwards-compatibility is still a potential issue, not because the pattern/type synonym layer seems implausible, but because I suspect people will begin to sidestep the compatibility layer and just use the GHC API (I certainly would). GHC is not shy about breaking backwards-compatibility between major releases, so it seems possible that this could extend to breaking TH. Missing features is not nearly as bad as breaking most clients of TH.

But perhaps this isn't a very likely scenario. TH mostly exports datatypes for haskell syntax, smart constructors, and a few functions for looking up metadata. I doubt these pieces of GHC change very often, and when they do it's probably an extension rather than a breaking change. Someone with more historical knowledge of GHC could comment :)

All in all, I like this idea a lot!

Eric

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if

On Wed, Nov 11, 2015, at 08:26, Richard Eisenberg wrote: that's

...
the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Richard Eisenberg

3:05 p.m.

I've gotten some encouraging feedback here. But I'm worried about the technicalities. I *think* this would all work out, but I'm a little worried about weird recursive dependencies and such. For example, is there something fundamentally important about all the names in THNames being in a package that is not `ghc`? Any thoughts there? Richard On Nov 11, 2015, at 11:26 AM, Richard Eisenberg wrote:

...

Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Geoffrey Mainland

12 Nov 12 Nov

12:12 p.m.

Hi Richard, Please correct me if I misunderstand, but in summary, you propose to change Template Haskell so that GHC's internal AST is produced directly, instead of the current route where there is an intermediate TH AST? Geoff On 11/11/2015 11:26 AM, Richard Eisenberg wrote:

...

Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Richard Eisenberg

12:14 p.m.

Yes, that's right. But with a compatibility shim, no longer tied into GHC, that could provide stability and/or a simpler interface. This compatibility shim would likely be called template-haskell. (I retract the idea of deprecating the package. But we could democratize its maintenance rather easily after this change.) Richard On Nov 12, 2015, at 12:12 PM, Geoffrey Mainland wrote:

...

Hi Richard,

Please correct me if I misunderstand, but in summary, you propose to change Template Haskell so that GHC's internal AST is produced directly, instead of the current route where there is an intermediate TH AST?

Geoff

On 11/11/2015 11:26 AM, Richard Eisenberg wrote:

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Geoffrey Mainland

13 Nov 13 Nov

4:18 p.m.

Sounds like a good pan to me. Note that we would still need a small GHC-version-specific shim, somewhere, to define the Q monad, which does occasionally change. We could push it into the GHC API. Having three Haskell ASTs has always been a pain---it would be nice to finally fix this situation. Using your proposal, we could implement a fully compatible template-haskell library on top of the GHC AST and the "internal" Q monad. We could also implement an analogous package that was based on haskell-src-exts instead; in fact, it would be nice if the latter became the "standard" Template Haskell implementation, as it would also help us avoid hacks like haskell-src-meta. Geoff On 11/12/15 9:14 AM, Richard Eisenberg wrote:

...

Yes, that's right. But with a compatibility shim, no longer tied into GHC, that could provide stability and/or a simpler interface. This compatibility shim would likely be called template-haskell. (I retract the idea of deprecating the package. But we could democratize its maintenance rather easily after this change.)

Richard

On Nov 12, 2015, at 12:12 PM, Geoffrey Mainland wrote:

...
Hi Richard,

Please correct me if I misunderstand, but in summary, you propose to change Template Haskell so that GHC's internal AST is produced directly, instead of the current route where there is an intermediate TH AST?

Geoff

On 11/11/2015 11:26 AM, Richard Eisenberg wrote:

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard

Richard Eisenberg

5:28 p.m.

This proposal now has a home at wiki:TemplateHaskell/Introspective and #11081. I think exploring how this might help haskell-src-exts would be very fruitful. Richard On Nov 13, 2015, at 4:18 PM, Geoffrey Mainland wrote:

...

Sounds like a good pan to me. Note that we would still need a small GHC-version-specific shim, somewhere, to define the Q monad, which does occasionally change. We could push it into the GHC API.

Having three Haskell ASTs has always been a pain---it would be nice to finally fix this situation.

Using your proposal, we could implement a fully compatible template-haskell library on top of the GHC AST and the "internal" Q monad. We could also implement an analogous package that was based on haskell-src-exts instead; in fact, it would be nice if the latter became the "standard" Template Haskell implementation, as it would also help us avoid hacks like haskell-src-meta.

Geoff

On 11/12/15 9:14 AM, Richard Eisenberg wrote:

...
Yes, that's right. But with a compatibility shim, no longer tied into GHC, that could provide stability and/or a simpler interface. This compatibility shim would likely be called template-haskell. (I retract the idea of deprecating the package. But we could democratize its maintenance rather easily after this change.)

Richard

On Nov 12, 2015, at 12:12 PM, Geoffrey Mainland wrote:

...
Hi Richard,

Please correct me if I misunderstand, but in summary, you propose to change Template Haskell so that GHC's internal AST is produced directly, instead of the current route where there is an intermediate TH AST?

Geoff

On 11/11/2015 11:26 AM, Richard Eisenberg wrote:

...
Hi devs,

There's a bunch of open tickets around Template Haskell. A great many of them are attempts to make TH more like what's already in GHC. Of course, when someone improves GHC, then TH also has to be updated. But this doesn't always happen, leading to several of these tickets.

I believe I have a solution to the problem: just eliminate Template Haskell and provide direct access to GHC's internal structures. The idea (still very sketchy; hence pre-proposal) is like this (numbered for easy reference, but no order is implied):

1. TH quotes would remain. DsMeta would desugar quotes into Core code that produces HsExprs. For example, [| 1 |] would have type (Q (LHsExpr Name)). (Or perhaps (Q (LHsExpr RdrName)) if that works out better for clients.)

2. TH splices would remain, working much as they do now. The expression inside, say, an expression splice would have type (Q exp) where we can satisfy the constraint (SpliceExpr exp). There would be instances for (SpliceExpr (LHsExpr Name)) and (SpliceExpr (LHsExpr RdrName)) as well as the non-located variants. Generalizing the type of expressions here allows users not to worry about un-renaming when roundtripping between quotes and splices.

3. Reification would remain, using an Info structure much like we have now. Would we expose the real GHC TyCons as the result of reification? Or is it better to give the users HsDecls? This would need to be fleshed out.

4. Lifting would remain, doing the obvious thing.

5. The template-haskell package could even remain, as a backward-compatibility shim. It would declare gobs of pattern synonyms and type synonyms to wrap the new underlying interface. This re-imagined template-haskell package would not need to be a boot library, and could be upgraded separately from GHC. We could even maintain multiple versions of the library so that TH clients wouldn't have to change their code when GHC upgrades. Perhaps someday we could think about deprecating, if that's the way the wind blows.

So, the end result is a completely re-engineered TH, but I believe we could keep full backward compatibility. (I have not considered Typed TH in any depth yet. But my hope is that it's not too different from the above.) And, we're left with a significantly lower maintenance burden, especially if we eliminate template-haskell someday.

And, tantalizingly, the flexibility in splices might allow us to splice in *Core* code someday. Perhaps we could also reify Core code. Then clients could write their own custom, domain-aware optimizations. Like RULES on steroids. But that's all for the next phase. (Giving due credit, this last bit is inspired by work David Christiansen is doing in Idris.)

What's wrong with this idea? I feel like *something* has to be more complicated than I've made it seem!

Richard

3477

Age (days ago)

3479

Last active (days ago)

List overview

Download

15 comments

6 participants

participants (6)

Alan & Kim Zimmerman
Edward Kmett
Eric Seidel
Geoffrey Mainland
Richard Eisenberg
Thomas Bereknyei