#9821: DeriveAnyClass support for higher-kinded classes + some more comments -------------------------------------+------------------------------------- Reporter: dreixel | Owner: dreixel Type: bug | Status: infoneeded Priority: normal | Milestone: 8.0.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #5462, #9968 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * status: new => infoneeded Comment: What is the status here? Note that #9968 was fixed recently. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9821#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9821: DeriveAnyClass support for higher-kinded classes + some more comments -------------------------------------+------------------------------------- Reporter: dreixel | Owner: dreixel Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.9 Resolution: | Keywords: Generics Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #5462, #9968, | Differential Rev(s): #12144 | Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * related: #5462, #9968 => #5462, #9968, #12144 Comment: To the best of my knowledge, here's a summary of the current state of each of those comments: 1. https://phabricator.haskell.org/D476?id=1443#inline-3652 - The logic of this check is explained more thoroughly in [http://git.haskell.org/ghc.git/blob/498ed2664219f7e8f1077f46ad2061aba2f57de4... Note (Determining whether newtype-deriving is appropriate)], which will be further clarified with Phab:D2280. 2. https://phabricator.haskell.org/D476?id=1443#inline-3648 and https://phabricator.haskell.org/D476?id=1443#inline-3646 ask to clarify that `DeriveAnyClass` does indeed only work on argument types of kind `*` and `* -> *` at the moment. #9968 fixed these. So the only thing that remains is to implement support for deriving types of other kinds with `DeriveAnyClass`. I believe that #12144 is very relevant here, since I don't see a way to properly fix that ticket without overhauling the way `DeriveAnyClass` infers its instance contexts in `deriving` clauses. Simon's comment [https://mail.haskell.org/pipermail /ghc-devs/2016-June/012276.html here] provides one suggestion on how to do this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9821#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9821: DeriveAnyClass support for higher-kinded classes + some more comments -------------------------------------+------------------------------------- Reporter: dreixel | Owner: dreixel Type: bug | Status: infoneeded Priority: normal | Milestone: Component: Compiler | Version: 7.9 Resolution: | Keywords: Generics Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: 12918 | Blocking: Related Tickets: #5462, #9968, | Differential Rev(s): Phab:D2961 #12144 | Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * differential: => Phab:D2961 Comment: A WIP attempt at fixing this is at Phab:D2961. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9821#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9821: DeriveAnyClass support for higher-kinded classes + some more comments -------------------------------------+------------------------------------- Reporter: dreixel | Owner: dreixel Type: bug | Status: patch Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Generics Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #5462, #9968, | Differential Rev(s): Phab:D2961 #12144 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): This is respoinding to [https://phabricator.haskell.org/D2961#inline-25611], concerning the way to use implication constraints to simplify the `DeriveAnyClass` implementation. The original idea is in [https://phabricator.haskell.org/D2961#inline-25543], where I said: I think it would help to explain more carefully what is happening here. How come we don't need to look at the data constructors any more? Example: {{{ class Foo a where bar :: Ix b => a -> b -> String default bar :: (Show a, Ix b) => a -> b -> String bar x y = show x baz :: Eq a => a -> a -> Bool default baz :: (Ord a, Show a) => a -> a -> Bool baz x y = compare x y == EQ }}} Given {{{ deriving instance Foo (Maybe d) }}} we behave as if you had written {{{ instance ??? => Foo (Maybe d) where {} }}} (that is, using the default methods from the class decl), and that in turn means {{{ instance ??? => Foo (Maybe d) where bar = $gdm_bar baz = $gdm_baz }}} where {{{ $gdm_bar :: forall a. Foo a => forall b. (Show a, Ix b) => a -> b -> String }}} is the generic default method defined by the class declaratation. Our task is to figure out what "???" should be. Answer: enough constraints to satisfy the Wanteds arising from the calls of $gdm_bar and $gdm_baz. So we end up with two sets of constraints to simplify: {{{ bar: (Givens: [Ix b], Wanteds: [Show (Maybe d), Ix b]) baz: (Givens: [Eq (Maybe d)], Wanteds: [Ord (Maybe d), Show (Maybe d)]) }}} Important: note that * the Givens come from the ordinary method type, while * the Wanteds come from the generic method. These are just implication constraints. We can combine them into a single constraint: {{{ (forall b. Ix b => Show (Maybe d), Ix b) /\ (forall . Eq (Maybe d) => Ord (Maybe d), Show (Maybe d)) }}} Notice that the type variables from the head of the instance decl (just `d` in this case) are global to this constraint, but any local quantification of the generic default method (e.g. `b` in the case of `bar`) are locally scoped to each implication, as they obviously should be. Now we solve this constraint, getting the residual constraint (RC) {{{ (forall b. Ix b => Show d) /\ (forall . Eq (Maybe b) => Ord d, Show d) }}} Now we need to hoist those constraints out of the implications to become our candidates for the "???". That is done by `approximateWC`, which will return {{{ (Show d, Ord d, Show d) }}} Now we can use `mkMinimalBySCs` to remove superclasses and duplicates, giving {{{ (Show d, Ord d) }}} And that's what we need for the "???". -------------- But we aren't done yet! Suppose we had written {{{ bar :: a -> b -> String default bar :: (Show a, C a b) => a -> b -> String }}} I've replaced `Ix b` by `C b` and I've removed it from the vanilla sig for `Bar`. Now the implication constraint will be {{{ forall b. () => Show (Maybe d), C (Maybe d) b }}} Suppose we have `instance Read p => C (Maybe p) q`. Then after simplification, we get the residual constraint (RC): {{{ forall b. () => (Show d, Read d) }}} and all is well. But suppose we had no such instance, or we had an instance like `instance C p q => C (Maybe p) q`. Then we'd finish up with the residual consraint (RC): {{{ forall b. () => (Show d, C d b) }}} and now `approximateWC` will produce only `Show d` from this, becuase `(C d b)` is captured by the `forall b`. What do to? Easy! Once we have decided "???" should be CX, say, just emit this implication (to be solved later): {{{ forall d. CX => RC }}} where RC is the residual constraint we computed earlier. Bingo! From CX we'll be able to solve all the constraints that `approximateWC` extracted, and we'll be left with the error that we want to report for that `(C d b)` constraint. ------------------ This may seem complicated, but it's '''exactly what happens in `TcSimplify.simplifyInfer`'''. It does a bit more besides (figuring out what type variables to quantify over), but it makes a good guide. We might ultimately wat to share code, but for now I think it's easier to do one step at a time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9821#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9821: DeriveAnyClass support for higher-kinded classes + some more comments -------------------------------------+------------------------------------- Reporter: dreixel | Owner: dreixel Type: bug | Status: patch Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Generics Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #5462, #9968, | Differential Rev(s): Phab:D2961 #12144 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj):

but it talks about inferring most general types, which I'm not sure is relevant to this story or not

Yes it's relevant. In inferring "???" we are indeed trying to infer the most general type for the instances. We seek the context with fewest constraints (i.e. most general). The equalities thing doesn't matter much, but for consistency with `simplifyInfer` set it to `True`.

What are CX and RC here?

Also, should I interpret this as meaning that if there are any implication constraints leftover after solving which contain something of

I defined them earlier in the text you quoted! RC = residual context, CX is what we decide ??? should be. the form forall x. C => Foo x (i.e., if there are constraints which contain type variables other than the class's type variables), that should be an error? Yes, certainly. Notice that `approximateWC` doesn't remove the constraints it floats out; it just floats them out. They are still there in RC; but we'll solve them easily from CX.

Do we already take care of this with ​these lines in TcDerivInfer:

I'm not sure.. too much is in flux. But you don't need to solve it because you don't need to know the answer here: just emit it and it'll get solved later.

Are you saying that I should literally use simplifyInfer to solve these implications

No: sharing code with `simplifyInfer` in due course would be a good plan, but it does a lot more so just use it as a source of inspiration, no more. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9821#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9821: DeriveAnyClass support for higher-kinded classes + some more comments -------------------------------------+------------------------------------- Reporter: dreixel | Owner: dreixel Type: bug | Status: patch Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Generics Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #5462, #9968, | Differential Rev(s): Phab:D2961 #12144 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ryan Scott ): In [changeset:"639e702b6129f501c539b158b982ed8489e3d09c/ghc" 639e702/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="639e702b6129f501c539b158b982ed8489e3d09c" Refactor DeriveAnyClass's instance context inference Summary: Currently, `DeriveAnyClass` has two glaring flaws: * It only works on classes whose argument is of kind `*` or `* -> *` (#9821). * The way it infers constraints makes no sense. It basically co-opts the algorithms used to infer contexts for `Eq` (for `*`-kinded arguments) or `Functor` (for `(* -> *)`-kinded arguments). This tends to produce overly constrained instances, which in extreme cases can lead to legitimate things failing to typecheck (#12594). Or even worse, it can trigger GHC panics (#12144 and #12423). This completely reworks the way `DeriveAnyClass` infers constraints to fix these two issues. It now uses the type signatures of the derived class's methods to infer constraints (and to simplify them). A high-level description of how this works is included in the GHC users' guide, and more technical notes on what is going on can be found as comments (and a Note) in `TcDerivInfer`. Fixes #9821, #12144, #12423, #12594. Test Plan: ./validate Reviewers: dfeuer, goldfire, simonpj, austin, bgamari Subscribers: dfeuer, thomie Differential Revision: https://phabricator.haskell.org/D2961 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9821#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler