Overlapping and incoherent instances

Friends One of GHC's more widely-used features is overlapping (and sometimes incoherent) instances. The user-manual documentation is herehttp://www.haskell.org/ghc/docs/latest/html/users_guide/type-class-extension.... The use of overlapping/incoherent instances is controlled by LANGUAGE pragmas: OverlappingInstances and IncoherentInstances respectively. However the overlap/incoherent-ness is a property of the *instance declaration* itself, and has been for a long time. Using LANGUAGE OverlappingInstances simply sets the "I am an overlapping instance" flag for every instance declaration in that module. This is a Big Hammer. It give no clue about *which* particular instances the programmer is expecting to be overlapped, nor which are doing the overlapping. It brutally applies to every instance in the module. Moreover, when looking at an instance declaration, there is no nearby clue that it might be overlapped. The clue might be in the command line that compiles that module! Iavor has recently implemented per-instance-declaration pragmas, so you can say instance {-# OVERLAPPABLE #-} Show a => Show [a] where ... instance {-# OVERLAPPING #-} Show [Char] where ... This is much more precise (it affects only those specific instances) and it is much clearer (you see it when you see the instance declaration). This new feature will be in GHC 7.10 and I'm sure you will be happy about that. But I propose also to deprecate the LANGUAGE pragmas OverlappingInstances and IncoherentInstances, as way to encourage everyone to use the new feature instead of the old big hammer. The old LANGUAGE pragmas will continue to work, of course, for at least another complete release cycle. We could make that two cycles if it was helpful. However, if you want deprecation-free libraries, it will entail a wave of library updates. This email is just to warn you, and to let you yell if you think this is a bad idea. It would actually not be difficult to retain the old LANGUAGE pragmas indefinitely - it just seems wrong not to actively push authors in the right direction. These deprecations of course popped up in the test suite, so I've been replacing them with per-instance pragmas there too. Interestingly in some cases, when looking for which instances needed the pragmas, I found...none. So OverlappingInstances was entirely unnecessary. Maybe library authors will find that too! Simon

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
instance {-# OVERLAPPABLE #-} Show a => Show [a] where ...
instance {-# OVERLAPPING #-} Show [Char] where ...
This, to me, is an admission that developers are not going to want to turn overlapping on globally in general, and so the current language extensions would not make sense to get adopted into the core language at any point. I agree with this idea, and so would second the proposal mentioned at http://www.reddit.com/r/haskell/comments/2c136i/xoverlappinginstances_and_xi... that a language extension that adds actual keywords to tag instances that should be allowed to overlap be added, instead of resorting to pragmas. This seems like an approach that could be useful in general and one could imagine moving "past" an extension to the core language at some point, potentially. - -- Stephen Paul Weber, @singpolyma See http://singpolyma.net for how I prefer to be contacted edition right joseph -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQIcBAEBCAAGBQJT2DFBAAoJENEcKRHOUZzemmIQAJEbSjPyx745UI6mkuhBVhKl LQWJlpu0/kzBHaFJl/mWcghKxoBFWwU+pCTh/Pr2oj0rp/KGskBLlplIqB4btZTA ov2MpPrsHm1M37MuGyMtiBhs57UJE+saKKuvcH3qzLZyBCHorE3lFcKAFbNupBrL e/vgNblQ70KGmDRAKqbAQHm9anoGeZUJPgQ9ylVEH4nBYLDo0YSNo/zTeB7fK2yv xBE+Ul3YGfhzf82cLJhYNQOpi5wJ3JEDBevKXcGRzr4Mhzn2Lke+26tu0tx6sOSN snPX2REoeQD1AfXvuNKSxV7BL+CQeyAOOmm2Isj3vW/oO3gkqpfRjCFc+ZSPEjlG XQ3S7L7cgNB34rd6sOFzTv83PXvsH0a0d5RqKUM2kN/qGEjSbAQ1FVyJEUcaEmzr jBnVnWq+abCOSOBg4joGfTxjq0zufdjxkzScEJVDVZ4pIXoxej7HJBi8UfIvo3Jo EDMGGsLSedt4tR2LzYf/5up7GPfuBsFQQzuIcdgMG8/zYca7zWPJgyunlXAPcbzr RvM/gf63SCBuVaQjrtv2Zhzp3PucWOL94NEmLYONU3uuEmo6bq1VO42fOUcl7X/1 UyhFbtoV7s/7PVClxdD4Ag9PumtSfl/CSvN0BA8AzDwTuHWNOPdThAHFqfAtRsD0 GyNzVRNOGuze9SqkLc4T =YeKu -----END PGP SIGNATURE-----

On 29-07-2014 20:41, Stephen Paul Weber wrote:
instance {-# OVERLAPPABLE #-} Show a => Show [a] where ...
instance {-# OVERLAPPING #-} Show [Char] where ...
This, to me, is an admission that developers are not going to want to turn overlapping on globally in general, and so the current language extensions would not make sense to get adopted into the core language at any point. I agree with this idea, and so would second the proposal mentioned at http://www.reddit.com/r/haskell/comments/2c136i/xoverlappinginstances_and_xi... that a language extension that adds actual keywords to tag instances that should be allowed to overlap be added, instead of resorting to pragmas. This seems like an approach that could be useful in general and one could imagine moving "past" an extension to the core language at some point, potentially.
OTOH, the pragma is mostly harmless for older GHC versions, while the keyword approach needs a preprocessor. -- Felipe.

Somebody signing messages as Felipe Lessa wrote:
OTOH, the pragma is mostly harmless for older GHC versions, while the keyword approach needs a preprocessor.
Only if *both* the old LANGUAGE pragma and the new pragma were employed, which will generate a deprecation warning for awhile and then eventually (likely) be rejected by newer GHCs, thus requiring a prepropcessor in either case. -- Stephen Paul Weber, @singpolyma See http://singpolyma.net for how I prefer to be contacted edition right joseph

Simon Peyton Jones
writes: Tue Jul 29 09:11:05 UTC 2014 ... This is a Big Hammer.
I agree with Simon's motivation that the whole-module overlap pragma is often too brutal. But I think that what Iavor has developed is still too brutal. (Sorry, and I hadn't known about these instance-level pragmas before now.) For my 2d, I think Andreas made an important distinction:
Andreas Abel andreas.abel at ifi.lmu.de Wed Jul 30 12:07:01 UTC 2014 The German equivalent of "overlap", ..., is used exclusively in a symmetrical fashion. It's like in English, if I say "our interests overlap", then it is pointless to ask whether my interest are overlapping yours or are overlapped by yours.
I'd say that the English "overlap" is also used in a symmetrical fashion (outside of specialist Haskell instances usage). There's a difference between: - this instance is nec. narrower than some other instance (IOW anything that's a substitution for this instance, is ipso facto a substitution for some wider) - vs. a partial overlap (some substitutions will fit this instance only, some will fit another instance, not this one, some will fit both) In my experience, unintended partial overlaps are the nastiest to diagnose. And partial overlaps are very seldom needed in practice. They're often a symptom that two separately-developed libraries are clashing. (For example the HList libraries -- as originally released -- used overlap extensively, but no partial overlaps.) So I would like the pragmas to be able to say: OK for this instance to subsumes or be subsumed by some other instance but it must not partially overlap any instance. (I guess this is beyond the question Simon's OP asked.) AntC

Friends, in sending my message below, I should also have sent a link to https://ghc.haskell.org/trac/ghc/ticket/9242#comment:25 Comment 25 describes the semantics of OVERLAPPING/OVERLAPPABLE etc, which I signally failed to do in my message below, leading to confusion in the follow up messages. My apologies for that. Some key points: * There is a useful distinction between overlapping and overlappable, but if you don't want to be bothered with it you can just say OVERLAPS (which means both). * Overlap between two candidate instances is allowed if either has the relevant property. This is a bit sloppy, but reduces the annotation burden. Actually, with this per-instance stuff I think it'd be perfectly defensible to require both to be annotated, but that's a different discussion. I hope that helps clarify. I'm really pretty certain that the basic proposal here is good: it implements the current semantics in a more fine-grained manner. My main motivation was to signal the proposed deprecation of the global per-module flag -XoverlappingInstances. Happily people generally seem fine with this. It is, after all, precisely what deprecations are for ("the old thing still works for now, but it won't do so for ever, and you should change as soon as is convenient"). Thanks Simon From: Libraries [mailto:libraries-bounces@haskell.org] On Behalf Of Simon Peyton Jones Sent: 29 July 2014 10:11 To: ghc-devs; GHC users; Haskell Libraries (libraries@haskell.org) Subject: Overlapping and incoherent instances Friends One of GHC's more widely-used features is overlapping (and sometimes incoherent) instances. The user-manual documentation is herehttp://www.haskell.org/ghc/docs/latest/html/users_guide/type-class-extension.... The use of overlapping/incoherent instances is controlled by LANGUAGE pragmas: OverlappingInstances and IncoherentInstances respectively. However the overlap/incoherent-ness is a property of the *instance declaration* itself, and has been for a long time. Using LANGUAGE OverlappingInstances simply sets the "I am an overlapping instance" flag for every instance declaration in that module. This is a Big Hammer. It give no clue about *which* particular instances the programmer is expecting to be overlapped, nor which are doing the overlapping. It brutally applies to every instance in the module. Moreover, when looking at an instance declaration, there is no nearby clue that it might be overlapped. The clue might be in the command line that compiles that module! Iavor has recently implemented per-instance-declaration pragmas, so you can say instance {-# OVERLAPPABLE #-} Show a => Show [a] where ... instance {-# OVERLAPPING #-} Show [Char] where ... This is much more precise (it affects only those specific instances) and it is much clearer (you see it when you see the instance declaration). This new feature will be in GHC 7.10 and I'm sure you will be happy about that. But I propose also to deprecate the LANGUAGE pragmas OverlappingInstances and IncoherentInstances, as way to encourage everyone to use the new feature instead of the old big hammer. The old LANGUAGE pragmas will continue to work, of course, for at least another complete release cycle. We could make that two cycles if it was helpful. However, if you want deprecation-free libraries, it will entail a wave of library updates. This email is just to warn you, and to let you yell if you think this is a bad idea. It would actually not be difficult to retain the old LANGUAGE pragmas indefinitely - it just seems wrong not to actively push authors in the right direction. These deprecations of course popped up in the test suite, so I've been replacing them with per-instance pragmas there too. Interestingly in some cases, when looking for which instances needed the pragmas, I found...none. So OverlappingInstances was entirely unnecessary. Maybe library authors will find that too! Simon

This is great! It's nice to see something finer grain.
We'd of course like to bring Safe Haskell into the picture though! Our
concern with the new design as it stands is the OVERLAPS flag. We'd
prefer it to be eliminated in favor of requiring developers specify
both OVERLAPPABLE and OVERLAPS if that truly is their intention.
# Why?
## Long Version
https://ghc.haskell.org/trac/ghc/wiki/SafeHaskell/NewOverlappingInstances#Ne...
## Short Version (kind-of)
The security implications of OVERLAPPABLE vs. OVERLAPPING are fairly
different. Remember, in Safe Haskell we apply a policy of only
allowing instances from a module M compiled with `-XSafe` to overlap
other instances from module M. If it overlaps (and is the most
specific overlap) instances from modules other than M then we don't
allow this to succeed. This is done to ensure that untrusted code
compiled with `-XSafe` can't alter the behavior of existing code, some
of which may be part of the TCB and security critical.
Brining the new finer grained pragmas into the story we get the following:
* OVERLAPPABLE is the programmer communicating that they can be
overlapped, an open instance if you will. We want to relax the above
restriction and allow instances from `-XSafe` modules to overlap
instances from their own module AND instances declared OVERLAPPABLE
that reside in any module.
* OVERLAPPING is the programming simply declaring they may overlap
less specific instances. We want to keep the above restriction for
these instances. That is, a instance I1 from a `-XSafe` module M won't
be able to overlap as the most specific instance, a instance I2 from
another module if I2 is marked as OVERLAPPING.
This distinction enables new encodings in Safe Haskell by allowing
security library authors to distinguish how untrusted code can overlap
their instances. In some way giving them open vs closed instances.
This distinction is subtle and important. Having a pragma OVERLAPS
that implies both glosses over this and will encourage developers to
use this without much thought.
## Safe Inference
We can also safely infer a module that only has OVERLAPPABLE instances
as safe, while ones that contain OVERLAPPING or OVERLAPS instances
must be regarded as unsafe since there is a difference in semantics of
these pragmas under Safe vs Unsafe.
So we also have an advantage if developers are more specific about
what they want, than just defaulting to OVERLAPS.
Cheers,
David
On 29 July 2014 02:11, Simon Peyton Jones
Friends
One of GHC’s more widely-used features is overlapping (and sometimes incoherent) instances. The user-manual documentation is here.
The use of overlapping/incoherent instances is controlled by LANGUAGE pragmas: OverlappingInstances and IncoherentInstances respectively.
However the overlap/incoherent-ness is a property of the *instance declaration* itself, and has been for a long time. Using LANGUAGE OverlappingInstances simply sets the “I am an overlapping instance” flag for every instance declaration in that module.
This is a Big Hammer. It give no clue about *which* particular instances the programmer is expecting to be overlapped, nor which are doing the overlapping. It brutally applies to every instance in the module. Moreover, when looking at an instance declaration, there is no nearby clue that it might be overlapped. The clue might be in the command line that compiles that module!
Iavor has recently implemented per-instance-declaration pragmas, so you can say
instance {-# OVERLAPPABLE #-} Show a => Show [a] where …
instance {-# OVERLAPPING #-} Show [Char] where …
This is much more precise (it affects only those specific instances) and it is much clearer (you see it when you see the instance declaration).
This new feature will be in GHC 7.10 and I’m sure you will be happy about that. But I propose also to deprecate the LANGUAGE pragmas OverlappingInstances and IncoherentInstances, as way to encourage everyone to use the new feature instead of the old big hammer. The old LANGUAGE pragmas will continue to work, of course, for at least another complete release cycle. We could make that two cycles if it was helpful.
However, if you want deprecation-free libraries, it will entail a wave of library updates.
This email is just to warn you, and to let you yell if you think this is a bad idea. It would actually not be difficult to retain the old LANGUAGE pragmas indefinitely – it just seems wrong not to actively push authors in the right direction.
These deprecations of course popped up in the test suite, so I’ve been replacing them with per-instance pragmas there too. Interestingly in some cases, when looking for which instances needed the pragmas, I found…none. So OverlappingInstances was entirely unnecessary. Maybe library authors will find that too!
Simon
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

To me, perhaps naively, IncoherentInstances is way more scary than OverlappingInstances. What behavior do these new pragmas have? In particular, will it be an error if there is no single most specific instance? And can the user decide whether it is an error? Twan On 29/07/14 11:11, Simon Peyton Jones wrote:
Friends
One of GHC’s more widely-used features is overlapping (and sometimes incoherent) instances. The user-manual documentation is here http://www.haskell.org/ghc/docs/latest/html/users_guide/type-class-extension....
The use of overlapping/incoherent instances is controlled by LANGUAGE pragmas: OverlappingInstances and IncoherentInstances respectively.
However the overlap/incoherent-ness is a property of the **instance declaration** itself, and has been for a long time. Using LANGUAGE OverlappingInstances simply sets the “I am an overlapping instance” flag for every instance declaration in that module.
This is a Big Hammer. It give no clue about **which** particular instances the programmer is expecting to be overlapped, nor which are doing the overlapping. It brutally applies to every instance in the module. Moreover, when looking at an instance declaration, there is no nearby clue that it might be overlapped. The clue might be in the command line that compiles that module!
Iavor has recently implemented per-instance-declaration pragmas, so you can say
instance {-# OVERLAPPABLE #-} Show a => Show [a] where …
instance {-# OVERLAPPING #-} Show [Char] where …
This is much more precise (it affects only those specific instances) and it is much clearer (you see it when you see the instance declaration).
This new feature will be in GHC 7.10 and I’m sure you will be happy about that. *But I propose also to deprecate the LANGUAGE pragmas OverlappingInstances and IncoherentInstances*, as way to encourage everyone to use the new feature instead of the old big hammer. The old LANGUAGE pragmas will continue to work, of course, for at least another complete release cycle. We could make that two cycles if it was helpful.
However, if you want deprecation-free libraries, it will entail a wave of library updates.
This email is just to warn you, and to let you yell if you think this is a bad idea. It would actually not be difficult to retain the old LANGUAGE pragmas indefinitely – it just seems wrong not to actively push authors in the right direction.
These deprecations of course popped up in the test suite, so I’ve been replacing them with per-instance pragmas there too. Interestingly in some cases, when looking for which instances needed the pragmas, I found…none. So OverlappingInstances was entirely unnecessary. Maybe library authors will find that too!
Simon
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

On Mon, Aug 11, 2014 at 11:36 AM, Twan van Laarhoven
To me, perhaps naively, IncoherentInstances is way more scary than OverlappingInstances.
It might be a bit naive. Most things that incoherent instances would allow are allowed with overlapping instances so long as you partition your code into two modules. So unless such a partitioning is impossible, overlapping instances are almost as scary as incoherent instances (unless the module separation somehow makes it less scary). And actually, with the way GHC handles instances, you can get more incoherent behavior than incoherent instances allow without enabling any extensions, just using modules: module A where class Foo a where foo :: a module B where import A instance F oo Int where foo = 5 bar :: Int ; bar = foo module C where import A instance Foo Int where foo = 6 baz :: Int ; baz = foo module D where import B import C quux = bar + baz -- 11 -- Dan

Hello,
this is clearly a bug in GHC: where `B` and `C` are imported, there should
have been an error, saying that there is a duplicate instance of `Foo Int`.
If there is no ticket for this already, could you please add one?
-Iavor
On Mon, Aug 11, 2014 at 12:35 PM, Dan Doel
On Mon, Aug 11, 2014 at 11:36 AM, Twan van Laarhoven
wrote: To me, perhaps naively, IncoherentInstances is way more scary than OverlappingInstances.
It might be a bit naive. Most things that incoherent instances would allow are allowed with overlapping instances so long as you partition your code into two modules. So unless such a partitioning is impossible, overlapping instances are almost as scary as incoherent instances (unless the module separation somehow makes it less scary).
And actually, with the way GHC handles instances, you can get more incoherent behavior than incoherent instances allow without enabling any extensions, just using modules:
module A where class Foo a where foo :: a
module B where import A instance F oo Int where foo = 5 bar :: Int ; bar = foo
module C where import A instance Foo Int where foo = 6 baz :: Int ; baz = foo
module D where import B import C
quux = bar + baz -- 11
-- Dan
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

This has been reported: https://ghc.haskell.org/trac/ghc/ticket/8338
But it's really not clear what the solution is!
Richard
On Aug 11, 2014, at 9:27 PM, Iavor Diatchki
Hello,
this is clearly a bug in GHC: where `B` and `C` are imported, there should have been an error, saying that there is a duplicate instance of `Foo Int`. If there is no ticket for this already, could you please add one?
-Iavor
On Mon, Aug 11, 2014 at 12:35 PM, Dan Doel
wrote: On Mon, Aug 11, 2014 at 11:36 AM, Twan van Laarhoven wrote: To me, perhaps naively, IncoherentInstances is way more scary than OverlappingInstances. It might be a bit naive. Most things that incoherent instances would allow are allowed with overlapping instances so long as you partition your code into two modules. So unless such a partitioning is impossible, overlapping instances are almost as scary as incoherent instances (unless the module separation somehow makes it less scary).
And actually, with the way GHC handles instances, you can get more incoherent behavior than incoherent instances allow without enabling any extensions, just using modules:
module A where class Foo a where foo :: a
module B where import A instance Foo Int where foo = 5 bar :: Int ; bar = foo
module C where import A instance Foo Int where foo = 6 baz :: Int ; baz = foo
module D where import B import C
quux = bar + baz -- 11
-- Dan
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
participants (9)
-
AntC
-
Dan Doel
-
David Terei
-
Felipe Lessa
-
Iavor Diatchki
-
Richard Eisenberg
-
Simon Peyton Jones
-
Stephen Paul Weber
-
Twan van Laarhoven