overlapping instances in 7.10.1
Dear GHC developers, This request overrides my previous one of "7.10.1-err..." (it is simpler and more precise). The archive http://www.botik.ru/pub/local/Mechveliani/ghcQuest/7.10.1-errReport-may23-20... presents a question about ghc-7.10.1. Make it, please, with ghc-7.10.1 by ghc $doconCpOpt -O --make Main , $doconCpOpt = -fwarn-unused-matches -fwarn-unused-binds -fwarn-unused-imports -fno-warn-overlapping-patterns -XRecordWildCards -XNamedFieldPuns -XFlexibleContexts -XMultiParamTypeClasses -XUndecidableInstances -XTypeSynonymInstances -XFlexibleInstances -fcontext-stack=30 as it is written there in README.txt. README.txt explains which two instances are wrongly resolved -- as I expect. In ghc-7.8.2 they are resolved in a correct way (and there is a different pragma syntax). I conclude this from running the test in docon-2.12. Am I missing something? Please, advise, ------ Sergei
Sergei I finally found time to look into what is happening here. It's a good illustration of the dangers of overlapping instances. Here is the setup: * Module ResEuc_ * Contains instance (...) => Ring (ResidueE a) <---- (A) instance (..., Ring (ResidueE a)) => Field (ResidueE a) <---- (B) * Module PFact__ * Imports Pgcd_, which imports ResEuc_ * Contains code that needs (Field (ResidueE (UPol (ResidueE Integer)))) <------ (X) * To solve (X) we use instance (B) from ResEuc_ * And hence we need to solve (Ring (ResidueE (UPol (ResidueE Integer)))) which we do using (A) but not (C) * Module RsePol_ * Imports PFact__ * Contains the specialised instance instance (...) => Ring (ResidueE (UPol a)) <------ (C) which overlaps instance (A) * Module Main * Needs an instance for Field (ResidueE (UPol (Residue Integer))) <------ (Y) * Although GHC *could* generate this by instance declarations, which it would do using (B) and then using (C), instead GHC cleverly sees that it has generated it before, in module PFact__, and so uses the one from PFact__. And that is what gives rise to your error So the real problem is that in PFact__, we make an instance (X) that does not see the specialised instance (C). It *cannot* see that instance because RsePol_ imports PFact__. So whatever code uses (X) is not going to see the specialised instance. I bet that this is not what you intend. This may be a latent bug in DoCon. I solved the problem by combining PFact__ and RsePol_ into a single module. Then everything works fine. What are the general lessons here? * GHC generally assumes that if it generates (C T) in one place, then it can use that anywhere in the program that (C T) is needed. That is, there is only one (C T) dictionary. * But suppose you have overlapping instance in different modules; say module A where instance C [a] module B where import A; instance C [Maybe a] If you use (C [Maybe Int]) in A, then of course we won't see the instance in B. So you'll get a different dictionary than if you compute C [Maybe Int] in module B. In short, overlapping instances are OK, but it's best to put them in the same module as the instances they overlap. Could GHC behave as if all instances were calculated afresh in the module being compiled. Yes, of course it could, but at the cost of losing the benefit of cross-module specialisation. An overloaded function specialised at, say, [Int] in one module could not be re-used in another in case the instances changed. Simon | -----Original Message----- | From: ghc-tickets [mailto:ghc-tickets-bounces@haskell.org] On Behalf Of | Sergei Meshveliani | Sent: 23 May 2015 22:08 | To: glasgow-haskell-users@haskell.org | Cc: glasgow-haskell-bugs@haskell.org | Subject: overlapping instances in 7.10.1 | | Dear GHC developers, | | This request overrides my previous one of "7.10.1-err..." | (it is simpler and more precise). | The archive | | http://www.botik.ru/pub/local/Mechveliani/ghcQuest/7.10.1-errReport-http://www.botik.ru/pub/local/Mechveliani/ghcQuest/7.10.1-errReport-may23-20... | may23-2015.ziphttp://www.botik.ru/pub/local/Mechveliani/ghcQuest/7.10.1-errReport-may23-20... | | presents a question about ghc-7.10.1. | | Make it, please, with ghc-7.10.1 by | | ghc $doconCpOpt -O --make Main | , | $doconCpOpt = | -fwarn-unused-matches -fwarn-unused-binds -fwarn-unused-imports | -fno-warn-overlapping-patterns -XRecordWildCards -XNamedFieldPuns | -XFlexibleContexts -XMultiParamTypeClasses -XUndecidableInstances | -XTypeSynonymInstances -XFlexibleInstances -fcontext-stack=30 | | | as it is written there in README.txt. | | README.txt explains which two instances are wrongly resolved | -- as I expect. | | In ghc-7.8.2 they are resolved in a correct way | (and there is a different pragma syntax). | I conclude this from running the test in docon-2.12. | | Am I missing something? | | Please, advise, | | ------ | Sergei | | | | | _______________________________________________ | ghc-tickets mailing list | ghc-tickets@haskell.orgmailto:ghc-tickets@haskell.org | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
On Sat, 2015-06-13 at 23:07 +0000, Simon Peyton Jones wrote: (I reformat this text a bit)
[..] I finally found time to look into what is happening here. It’s a good illustration of the dangers of overlapping instances. Here is the setup:
* Module ResEuc_
* Contains instance (...) => Ring (ResidueE a) <---- (A) instance (..., Ring (ResidueE a)) => Field (ResidueE a) <---- (B)
* Module PFact__
* Imports Pgcd_, which imports ResEuc_
* Contains code that needs
(Field (ResidueE (UPol (ResidueE Integer)))) <------ (X)
* To solve (X) we use instance (B) from ResEuc_ * And hence we need to solve (Ring (ResidueE (UPol (ResidueE Integer))))
which we do using (A) but not (C)
* Module RsePol_
* Imports PFact__ * Contains the specialised instance
instance (...) => Ring (ResidueE (UPol a)) <------ (C)
which overlaps instance (A)
* Module Main * Needs an instance for
Field (ResidueE (UPol (Residue Integer))) <------ (Y)
* Although GHC *could* generate this by instance declarations, which it would do using (B) and then using (C),
instead GHC cleverly sees that it has generated it before, in module PFact__, and so uses the one from PFact__. And that is what gives rise to your error
So the real problem is that in PFact__, we make an instance (X) that does not see the specialised instance (C). It *cannot* see that instance because RsePol_ imports PFact__.
(is is called Pfact__). Yes, I intended this. And this is not a problem. This is because Pfact_ uses somewhat a smaller knowledge library than there may use further modules which import Pfact_.
So whatever code uses (X) is not going to see the specialised instance.
Why? ghc-7.8.3 treats overlapping instances differently. My intention was (and is) as follows. Compiling Main, the compiler sees the two solutions for (Y): the one imported from Pfact__ -- which implementation (dictionary?) is ready, and another that can be generated using (C) from RsePol_. By the meaning of what are overlapping instances, the compiler must resolve between these two. And this is resolved by the rule of checking the substitutional instance between instances. `Main' imports more knowledge about instances than Pfact_ does, so it is supposed to use an additional knowledge. The fact that one of the overlapping instances is already generated must not be a sufficient reason to set it in Main. This is why I think that ghc-7.8.3 treats the OI notion in a more natural way than ghc-7.10.1 does. May be, ghc-7.10.1 has a better technical tool for this, but ghc-7.8.3 corresponds to a natural notion of OI. Can GHC return to a natural OI notion? Or am I missing something?
I bet that this is not what you intend. This may be a latent bug in DoCon.
If I am not missing anything in my above discourse, then DoCon is all right at this point.
I solved the problem by combining PFact__ and RsePol_ into a single module. Then everything works fine.
I think that this approach will generally lead to great difficulties in composing an application. Please, consider my above explanation and tell me whether I am missing something. Regards, ------ Sergei
What are the general lessons here?
GHC generally assumes that if it generates (C T) in one place, then it can use that anywhere in the program that (C T) is needed. That is, there is only one (C T) dictionary.
· But suppose you have overlapping instance in different modules; say
module A where instance C [a]
module B where import A; instance C [Maybe a]
If you use (C [Maybe Int]) in A, then of course we won’t see the instance in B. So you’ll get a different dictionary than if you compute C [Maybe Int] in module B.
In short, overlapping instances are OK, but it’s best to put them in the same module as the instances they overlap.
Could GHC behave as if all instances were calculated afresh in the module being compiled. Yes, of course it could, but at the cost of losing the benefit of cross-module specialisation. An overloaded function specialised at, say, [Int] in one module could not be re-used in another in case the instances changed.
Simon
| -----Original Message----- | From: ghc-tickets [mailto:ghc-tickets-bounces@haskell.org] On Behalf Of | Sergei Meshveliani | Sent: 23 May 2015 22:08 | To: glasgow-haskell-users@haskell.org | Cc: glasgow-haskell-bugs@haskell.org | Subject: overlapping instances in 7.10.1
| Dear GHC developers, | | This request overrides my previous one of "7.10.1-err..." | (it is simpler and more precise). | The archive
| http://www.botik.ru/pub/local/Mechveliani/ghcQuest/7.10.1-errReport- | may23-2015.zip | | presents a question about ghc-7.10.1.
| Make it, please, with ghc-7.10.1 by
| ghc $doconCpOpt -O --make Main | , | $doconCpOpt =
| -fwarn-unused-matches -fwarn-unused-binds -fwarn-unused-imports | -fno-warn-overlapping-patterns -XRecordWildCards -XNamedFieldPuns | -XFlexibleContexts -XMultiParamTypeClasses -XUndecidableInstances | -XTypeSynonymInstances -XFlexibleInstances -fcontext-stack=30
| | as it is written there in README.txt. | README.txt explains which two instances are wrongly resolved | -- as I expect.
| In ghc-7.8.2 they are resolved in a correct way | (and there is a different pragma syntax). | I conclude this from running the test in docon-2.12.
| Am I missing something? | Please, advise,
| ------ | Sergei
| This is why I think that ghc-7.8.3 treats the OI notion in a more | natural way than ghc-7.10.1 does. | May be, ghc-7.10.1 has a better technical tool for this, but ghc- | 7.8.3 corresponds to a natural notion of OI. | | Can GHC return to a natural OI notion? | Or am I missing something? Well it all depends what you mean by "natural". To me it is profoundly un-natural to deliberately have the same type-class constraint solved in two different ways in the same program! To require this would prevent cross-module specialisation. If I have f :: C a => a -> a in one module, and I specialise it to f_spec :: [Int] -> [Int] in one module, I want to be free to re-use that specialisation in other modules. But under your "natural" story, I cannot do that, because (C [Int]) might be resolved differently there. I'd be interested in what others think. I've started https://ghc.haskell.org/trac/ghc/ticket/10526 to collect these points. Meanwhile, Sergei, it's clear how to fix docon, so you are no longer stuck. Simon | | | | > I bet that this is not what you intend. This may be a latent bug | in | > DoCon. | | If I am not missing anything in my above discourse, then DoCon is all | right at this point. | | | > I solved the problem by combining PFact__ and RsePol_ into a single | > module. Then everything works fine. | | I think that this approach will generally lead to great difficulties | in | composing an application. | | Please, consider my above explanation and tell me whether I am missing | something. | | Regards, | | ------ | Sergei | | | | > What are the general lessons here? | > | > GHC generally assumes that if it generates (C T) in one | place, | > then it can use that anywhere in the program that (C T) is needed. | > That is, there is only one (C T) dictionary. | > | > | > · But suppose you have overlapping instance in different | > modules; say | > | > module A where instance C [a] | > | > module B where import A; instance C [Maybe a] | > | > If you use (C [Maybe Int]) in A, then of course we won’t see the | > instance in B. So you’ll get a different dictionary than if you | > compute C [Maybe Int] in module B. | > | > | > | > In short, overlapping instances are OK, but it’s best to put them in | > the same module as the instances they overlap. | > | > | > | > Could GHC behave as if all instances were calculated afresh in the | > module being compiled. Yes, of course it could, but at the cost of | > losing the benefit of cross-module specialisation. An overloaded | > function specialised at, say, [Int] in one module could not be re- | used | > in another in case the instances changed. | > | > | > | > Simon | > | > | | > | -----Original Message----- | > | From: ghc-tickets [mailto:ghc-tickets-bounces@haskell.org] On | Behalf | > Of | > | Sergei Meshveliani | > | Sent: 23 May 2015 22:08 | > | To: glasgow-haskell-users@haskell.org | > | Cc: glasgow-haskell-bugs@haskell.org | > | Subject: overlapping instances in 7.10.1 | | > | Dear GHC developers, | > | | > | This request overrides my previous one of "7.10.1-err..." | > | (it is simpler and more precise). | > | The archive | | > | http://www.botik.ru/pub/local/Mechveliani/ghcQuest/7.10.1- | errReport- | > | may23-2015.zip | > | | > | presents a question about ghc-7.10.1. | > | > | Make it, please, with ghc-7.10.1 by | | > | ghc $doconCpOpt -O --make Main | > | , | > | $doconCpOpt = | | > | -fwarn-unused-matches -fwarn-unused-binds -fwarn-unused-imports | > | -fno-warn-overlapping-patterns -XRecordWildCards -XNamedFieldPuns | > | -XFlexibleContexts -XMultiParamTypeClasses -XUndecidableInstances | > | -XTypeSynonymInstances -XFlexibleInstances -fcontext-stack=30 | > | > | | > | as it is written there in README.txt. | > | README.txt explains which two instances are wrongly resolved | > | -- as I expect. | | > | In ghc-7.8.2 they are resolved in a correct way | > | (and there is a different pragma syntax). | > | I conclude this from running the test in docon-2.12. | | > | Am I missing something? | > | Please, advise, | | > | ------ | > | Sergei |
On Mon, 2015-06-15 at 09:29 +0000, Simon Peyton Jones wrote:
| This is why I think that ghc-7.8.3 treats the OI notion in a more | natural way than ghc-7.10.1 does. | May be, ghc-7.10.1 has a better technical tool for this, but ghc- | 7.8.3 corresponds to a natural notion of OI. | | Can GHC return to a natural OI notion? | Or am I missing something?
Well it all depends what you mean by "natural". To me it is profoundly un-natural to deliberately have the same type-class constraint solved in two different ways in the same program!
To require this would prevent cross-module specialisation. If I have f :: C a => a -> a in one module, and I specialise it to f_spec :: [Int] -> [Int] in one module, I want to be free to re-use that specialisation in other modules. But under your "natural" story, I cannot do that, because (C [Int]) might be resolved differently there.
I'd be interested in what others think. I've started https://ghc.haskell.org/trac/ghc/ticket/10526 to collect these points.
Meanwhile, Sergei, it's clear how to fix docon, so you are no longer stuck.
I hope DoCon is not stuck -- because it relies (so far) on ghc-7.8.3. But I need to understand the subject of overlapping instances (OI). ---------- I am writing this _here_ because to clicking at https://ghc.haskell.org/trac/ghc/ticket/10526 my browser responds Problem occurred while loading the URL https://ghc.haskell.org/trac/ghc/ticket/10526 SSL handshake failed: A TLS fatal alert has been received. Then, I start searching from https://ghc.haskell.org/trac/ghc/ and cannot find #10526 Also I had earlier a certain difficulty with registering to this bug tracker, and suspect that registration was not fully successful, so that I doubt that I can write there. May I, please, discuss the subject here? ------------------- The report of 7.10.1-errReport-may23-2015.zip shows that 7.8.3 and 7.10.1 treat OI differently, But this report is rather complex. Instead consider the following simple example. -------------------------------------------- module A where class Size a where size :: a -> Int instance {-# OVERLAPPING #-} Size [a] where size = length f :: Size a => a -> Int f = size g :: [Maybe Int] -> Int g = size ---------------------------------------- module Main where import A instance {-# OVERLAPPING #-} Size [Maybe a] where size _ = -1 mbs = [] :: [Maybe Int] main = putStr (shows (f mbs) "\n") -- I -1 -- putStr (shows (g mbs) "\n") -- II 0 -------------------------------------------- Making and running in ghc-7.10.1 : ghc --make -XFlexibleInstances Main ./Main It prints "-1" for the line (I) for main, and "0" for the line (II). To compare to ghc-7.8.3, comment out the pragma and add -XOverlappingInstances to the ghc call. And this yields the same results -1 and -0 respectively. Please, how to change (a bit) the above simple example in order to see the difference between 7.8.3 and 7.10.1 ? Thanks, ------ Sergei
On Mon, 2015-06-15 at 09:29 +0000, Simon Peyton Jones wrote:
| This is why I think that ghc-7.8.3 treats the OI notion in a more | natural way than ghc-7.10.1 does. | May be, ghc-7.10.1 has a better technical tool for this, but ghc- | 7.8.3 corresponds to a natural notion of OI. | | Can GHC return to a natural OI notion? | Or am I missing something?
Well it all depends what you mean by "natural". To me it is profoundly un-natural to deliberately have the same type-class constraint solved in two different ways in the same program!
To require this would prevent cross-module specialisation. If I have f :: C a => a -> a in one module, and I specialise it to f_spec :: [Int] -> [Int] in one module, I want to be free to re-use that specialisation in other modules. But under your "natural" story, I cannot do that, because (C [Int]) might be resolved differently there.
Now, I give a simple (and a very contrived) example illustrating of how overlapping instances (OI) are used in DoCon. Also this example is made after the sample that Simon has given in his recent letter:
GHC generally assumes that if it generates (the instance) (C T) in one place, then it can use that anywhere in the program that (C T) is needed. That is, there is only one (C T) dictionary.
But suppose you have overlapping instance in different modules; say
module A where instance C [a] module B where import A; instance C [Maybe a]
If you use (C [Maybe Int]) in A, then of course we won’t see the instance in B. So you’ll get a different dictionary than if you compute C [Maybe Int] in module B.
In short, overlapping instances are OK, but it’s best to put them in the same module as the instances they overlap.
My example: ------------------------------------------------------------ module A where class Att a where att :: a -> (Int , Maybe Int) instance {-# OVERLAPPING #-} Att [a] where att xs = (length (reverse xs) , Nothing) f :: [Maybe Int] -> (Int , Maybe Int) f = att ----------------- module Main where import A instance {-# OVERLAPPING #-} Att [Maybe a] where att mbs = (length mbs , Just 1) mbs = [] :: [Maybe Int] main = putStr (shows (f mbs) "\n") -- (I) (0 , Nothing) -- putStr (shows (att mbs) "\n") -- (II) (0, Just 1) ---------------------------------------------------------------------- Att stands for the class C of Simon's letter. It means "certain attributes of a value, and also of its type". The value Nothing for the second part of (att a) means that the second component of attributes is not definitely known for this particular instance. length (reverse xs) imitates a non-optimal method for computing att in the instance for a => [a]. This instance is very general, so that the first component of attributes is evaluated in-efficiently (but correct), the second component has a correct value, but highly indefinite (few information derived). The function f uses the instance Att [Maybe Int], and it is satisfied with the generic instance given in A.hs. Because a) at this stage of the project there is not enough functionality to implement a better special method b) for this particular case in this module the generic instance is sufficient. Main.hs defines a special instance of Att for [Maybe a]. This instance in more special than the one defined in A.hs. And the value (att mbs) has the first component evaluated more efficiently than for the generic (Att [a]) instance. But the value is the same. The second component even has a different value in the result. But it is considered as correct (this is similar to the situation of: "generally the speed is 2 < s < 8, and in this the special case it is 3"). The call (att mbs) in `main' uses a different dictionary for Att than the call A.f (mbs). Right? Both ghc-7.8.3 (with -XFlexibleInstnce -XOverlappingInstances in the call) and ghc-7.10.1 (with -XFlexibleInstnce in the call) give the same results in 'main': (0, Nothing) for the line (I) and (0, Just 1) for the line (II). And I thought that everything is set naturally in this program. Simon, you would state that as A.f and (Main.att mbs) use the instances of Att for the same type [Maybe Int], "overlapping instances are OK, but it’s best to put them in the same module as the instances they overlap". I do not understand the grammar of the phrase in quotes, either it has a typo or this is too difficult English for me. Anyway: does this mean for this particular example, that it is highly desirable to set the two above instance declarations in the same module ? 1) What if they are in different modules, like in the above example. What unnatural may happen -- for example? 2) At least ghc-7.8.3 and ghc-7.10.1 do the same in this example. May be, you can change this example a bit to make ghc-7.8.3 and ghc-7.10.1 diverse, so that my example bug becomes visible? (they diverse on 7.10.1-errReport-may23-2015.zip but this bunch of modules is too complex). 3) It is not practical to join the above A.hs and Main.hs into one module. Because generally, between A.hs and Main.hs there are added many modules with much functionality, and the instance implementation in Main uses this functionality. This will be similar as putting all the developed library into one module. Please, advise, ------ Sergei
On Tue, 2015-06-16 at 00:03 +0400, Sergei Meshveliani wrote:
[..] 2) At least ghc-7.8.3 and ghc-7.10.1 do the same in this example. May be, you can change this example a bit to make ghc-7.8.3 and ghc-7.10.1 diverse, so that my example bug becomes visible? (they diverse on 7.10.1-errReport-may23-2015.zip but this bunch of modules is too complex).
(this is in the end of my last letter). I am trying now to reduce the report of 7.10.1-errReport-may23-2015.zip into something simple. And discover that Main yields a different result for -Onot. Probably, DoCon-2.12.1 will run correct under -Onot. For me, this is not important, because -O is desirable anyway. But for the GHC developers this effect may present some information. Regards, ------ Sergei
On Mon, 2015-06-15 at 09:29 +0000, Simon Peyton Jones wrote:
| This is why I think that ghc-7.8.3 treats the OI notion in a more | natural way than ghc-7.10.1 does. | May be, ghc-7.10.1 has a better technical tool for this, but ghc- | 7.8.3 corresponds to a natural notion of OI. | | Can GHC return to a natural OI notion? | Or am I missing something?
Well it all depends what you mean by "natural". To me it is profoundly un-natural to deliberately have the same type-class constraint solved in two different ways in the same program!
To require this would prevent cross-module specialisation. If I have f :: C a => a -> a in one module, and I specialise it to f_spec :: [Int] -> [Int] in one module, I want to be free to re-use that specialisation in other modules. But under your "natural" story, I cannot do that, because (C [Int]) might be resolved differently there.
Now, I give a simple (and a very contrived) example illustrating of how overlapping instances (OI) are used in DoCon. Also this example is made after the sample that Simon has given in his recent letter:
[..]
2) At least ghc-7.8.3 and ghc-7.10.1 do the same in this example. May be, you can change this example a bit to make ghc-7.8.3 and ghc-7.10.1 diverse, so that my example bug becomes visible? (they diverse on 7.10.1-errReport-may23-2015.zip but this bunch of modules is too complex). [..]
participants (2)
-
Sergei Meshveliani -
Simon Peyton Jones