Hi Dominique, When implementing roles, I was indeed worried about the problem you're addressing: that code that previously worked with GND now won't. However, it turns out that few people have really complained about this. IIRC, in all of Hackage, only 3 packages needed to be changed because of this. If there were a larger impact to the GND breakage, I think your suggestion would be a good one. The problem I'm adressing in this thread is different: that library authors have been given a new, not-backward-compatible way of preventing abuses of their datatypes, and no proposal I have seen really addresses all of the problems here. I'm hoping my no-role-annots package might be helpful, but it doesn't fully resolve the issues. Richard On Mar 31, 2014, at 2:51 AM, Dominique Devriese wrote:

Richard,

(re-posting because I first used an address that is not subscribed to the lists)

I've been wondering about the following: it seems like the main problem in this situation is that the GeneralizedNewtypeDeriving extension changed meaning from "just coerce everything while deriving" to "only coerce stuff if it's allowed by the relevant role annotations". Would it not be an alternative solution to split up the GND extension into

* a backwards-compatible one (called GeneralizedNewtypeDeriving for backwards compatibility ;)) that ignores role annotations (as before) and uses unsafeCoerce whereever necessary * a safe one (called e.g. SafeNewtypeDeriving) that respects role annotations

The first one could then be deprecated and removed in a release or two. That might give library maintainers time to move their packages to SafeNewtypeDeriving when they have tested that everything works...

Regards, Dominique

P.S.: The above is based on a limited understanding of the problem, so I'm sorry if it misses some aspect of the problem...

2014-03-31 2:14 GMT+02:00 Richard Eisenberg :

...

I spent some time thinking about what, precisely, can be done here to make folks happier. (See the thread beginning here: http://www.haskell.org/pipermail/libraries/2014-March/022321.html) And, the answer seemed to all be in the concrete syntax. The only logical alternative (that I could think of) to having roles is to disallow GND, and I don't think anyone is proposing that. And, it is impossible to infer an author's desired roles for a datatype. The heuristics mentioned here all seem far too fragile and hard to predict to become a lasting feature of GHC (in my opinion). What's left? Concrete syntax.

So, I have written and uploaded no-role-annots-1.0, a backward-compatible alternative to role annotations -- no CPP required. It's not as principled as proper role annotations, but it should do the job for most users.

Here are two examples:

1. Datatypes:

...

import Language.Haskell.RoleAnnots

data Map k v = (Nominal k, Representational v) => MkMap [(k,v)]

The constraints (which need be put on only one data constructor, if there are many) will get the built-in role inference mechanism to do what the user requests. In this example, the `Representational v` is actually redundant, but causes no harm. Because these classes have universal instances ("instance Nominal a") and have no methods, they should have no run-time significance. The only downside I can see is that the code above needs -XGADTs or -XExistentialQuantification to work, though it is neither a GADT nor has existentials. (Pattern-matching on such a definition needs no extensions.)

2. Newtypes:

Newtype constructors cannot be constrained, unfortunately. So, we have to resort to Template Haskell:

...

import Language.Haskell.RoleAnnots

roleAnnot [NominalR, RepresentationalR] [d| newtype Map k v = MkMap [(k, v)] |]

This is clearly worse, but I was able to come up with no other solution that worked for newtypes. Note that, in the example, I used the fact that Template Haskell interprets a bare top-level expression as a Template Haskell splice. We could also wrap that line in $( ... ) to be more explicit about the use of TH. Also problematic here is that the code above requires -XRoleAnnotations in GHC 7.8. To get this extension enabled without using CPP, put it in a condition chunk in your .cabal file, like this:

...

if impl(ghc >= 7.8) default-extensions: RoleAnnotations

I hope this is helpful to everyone. Please feel free to post issues/pull requests to my github repo at github.com/goldfirere/no-role-annots.

Thanks, Richard

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Ah -- I see. Interesting suggestion. As we're not thinking much now about the change to a nominal default (definitely too late for 7.8), I've recorded this idea on the roles wiki page (https://ghc.haskell.org/trac/ghc/wiki/Roles) for easy recovery in the future. I think the idea has merit, but there's no concrete action to take on it, at the moment. Thanks, Richard On Mar 31, 2014, at 2:12 PM, Dominique Devriese wrote:

Richard,

Right, but I was thinking about the debate between "nominal/non-parametric-by-default" or "representational/parametric-by-default" for parameters of data types that aren't forced to nominal from inspecting the datatype implementation. As I understand it, representational-by-default (currently used) may leave libraries that don't add role annotations open for abuse, but won't unnecessarily break library users' code and nominal-by-default prevents all abuse but may unnecessarily break code that uses libraries that have not added proper role annotations.

What I was wondering about is if the dilemma could be solved by choosing nominal-by-default in the long term for the role inference (so that library writers cannot accidentally leave abstraction holes open by forgetting to add role annotations) and use them in the long-term-supported SafeNewtypeDeriving extension, but provide a deprecated not-quite-as-safe GND extension for helping out users of libraries that have not yet added role annotations. I would fancy that this not-quite-as-safe GND could use unsafeCoerce wherever the safe one would give an error about annotated roles.

Regards, Dominique

2014-03-31 17:05 GMT+02:00 Richard Eisenberg :

...

Hi Dominique,

When implementing roles, I was indeed worried about the problem you're addressing: that code that previously worked with GND now won't. However, it turns out that few people have really complained about this. IIRC, in all of Hackage, only 3 packages needed to be changed because of this. If there were a larger impact to the GND breakage, I think your suggestion would be a good one.

The problem I'm adressing in this thread is different: that library authors have been given a new, not-backward-compatible way of preventing abuses of their datatypes, and no proposal I have seen really addresses all of the problems here. I'm hoping my no-role-annots package might be helpful, but it doesn't fully resolve the issues.

Richard

On Mar 31, 2014, at 2:51 AM, Dominique Devriese wrote:

...

Richard,

(re-posting because I first used an address that is not subscribed to the lists)

I've been wondering about the following: it seems like the main problem in this situation is that the GeneralizedNewtypeDeriving extension changed meaning from "just coerce everything while deriving" to "only coerce stuff if it's allowed by the relevant role annotations". Would it not be an alternative solution to split up the GND extension into

* a backwards-compatible one (called GeneralizedNewtypeDeriving for backwards compatibility ;)) that ignores role annotations (as before) and uses unsafeCoerce whereever necessary * a safe one (called e.g. SafeNewtypeDeriving) that respects role annotations

The first one could then be deprecated and removed in a release or two. That might give library maintainers time to move their packages to SafeNewtypeDeriving when they have tested that everything works...

Regards, Dominique

P.S.: The above is based on a limited understanding of the problem, so I'm sorry if it misses some aspect of the problem...

2014-03-31 2:14 GMT+02:00 Richard Eisenberg :

...

I spent some time thinking about what, precisely, can be done here to make folks happier. (See the thread beginning here: http://www.haskell.org/pipermail/libraries/2014-March/022321.html) And, the answer seemed to all be in the concrete syntax. The only logical alternative (that I could think of) to having roles is to disallow GND, and I don't think anyone is proposing that. And, it is impossible to infer an author's desired roles for a datatype. The heuristics mentioned here all seem far too fragile and hard to predict to become a lasting feature of GHC (in my opinion). What's left? Concrete syntax.

So, I have written and uploaded no-role-annots-1.0, a backward-compatible alternative to role annotations -- no CPP required. It's not as principled as proper role annotations, but it should do the job for most users.

Here are two examples:

1. Datatypes:

...

import Language.Haskell.RoleAnnots

data Map k v = (Nominal k, Representational v) => MkMap [(k,v)]

The constraints (which need be put on only one data constructor, if there are many) will get the built-in role inference mechanism to do what the user requests. In this example, the `Representational v` is actually redundant, but causes no harm. Because these classes have universal instances ("instance Nominal a") and have no methods, they should have no run-time significance. The only downside I can see is that the code above needs -XGADTs or -XExistentialQuantification to work, though it is neither a GADT nor has existentials. (Pattern-matching on such a definition needs no extensions.)

2. Newtypes:

Newtype constructors cannot be constrained, unfortunately. So, we have to resort to Template Haskell:

...

import Language.Haskell.RoleAnnots

roleAnnot [NominalR, RepresentationalR] [d| newtype Map k v = MkMap [(k, v)] |]

This is clearly worse, but I was able to come up with no other solution that worked for newtypes. Note that, in the example, I used the fact that Template Haskell interprets a bare top-level expression as a Template Haskell splice. We could also wrap that line in $( ... ) to be more explicit about the use of TH. Also problematic here is that the code above requires -XRoleAnnotations in GHC 7.8. To get this extension enabled without using CPP, put it in a condition chunk in your .cabal file, like this:

...

if impl(ghc >= 7.8) default-extensions: RoleAnnotations

I hope this is helpful to everyone. Please feel free to post issues/pull requests to my github repo at github.com/goldfirere/no-role-annots.

Thanks, Richard

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

We indeed had unexported "internal coercions" as a desideratum during the design process, but we didn't have a specific use case and it didn't mesh with our eventual decision to use class instances. Now that the whole thing is implemented, Coercible instances are actually fully magical inside of GHC. Thus, there would be nothing especially difficult, from an implementation standpoint, of adding the feature described below in a later release. There is always the niggling issue of concrete syntax. So, if you (or others) hit upon a nice, real use case desiring this feature and want to come up with a design, please propose it! I do think it would likely be easy to make happen. Richard On Apr 8, 2014, at 10:28 PM, Isaac Dupree wrote:

I've been wondering whether there are any cases where the ability to coerce a parameter should not be exported (as for Set) but where the implementation would like to use some coercions (for example, if it internally wanted to coerce the contained type to a specific newtype, to change some of the contained type's instances while being able to guarantee that the invariant-related instances stayed safe).

This would call for coercibility to be something you can import and export. What if, in the long term, there were a flag ExplicitRoleExports (probably rarely used):

- with ExplicitRoleExports on, role signatures would be listed in the export list and must be at least as restrictive as the roles visible within the module. Exported types without role signatures in the export list would default to being exported as all-nominal.

- with ExplicitRoleExports off, any types the modules export would be exported with the same roles visible inside the module.

As is currently, the default role at a type's definition site would allow as many coercions as is segfault-safe.

-Isaac

P.S. I think this particular proposal likely has issues. If you import (T :: * -> * -> *) as "nominal, representational" and from elsewhere as "representational, nominal", do those roles combine somehow? What about contravariant [or is it covariant] roles as in (T :: (* -> *) -> *)? I haven't thought these through. Also I haven't thought of any concrete use cases for export-controlled roles; perhaps there aren't any.

On 03/31/2014 02:12 PM, Dominique Devriese wrote:

...

Richard,

Right, but I was thinking about the debate between "nominal/non-parametric-by-default" or "representational/parametric-by-default" for parameters of data types that aren't forced to nominal from inspecting the datatype implementation. As I understand it, representational-by-default (currently used) may leave libraries that don't add role annotations open for abuse, but won't unnecessarily break library users' code and nominal-by-default prevents all abuse but may unnecessarily break code that uses libraries that have not added proper role annotations.

What I was wondering about is if the dilemma could be solved by choosing nominal-by-default in the long term for the role inference (so that library writers cannot accidentally leave abstraction holes open by forgetting to add role annotations) and use them in the long-term-supported SafeNewtypeDeriving extension, but provide a deprecated not-quite-as-safe GND extension for helping out users of libraries that have not yet added role annotations. I would fancy that this not-quite-as-safe GND could use unsafeCoerce wherever the safe one would give an error about annotated roles.

Regards, Dominique

2014-03-31 17:05 GMT+02:00 Richard Eisenberg :

...

Hi Dominique,

When implementing roles, I was indeed worried about the problem you're addressing: that code that previously worked with GND now won't. However, it turns out that few people have really complained about this. IIRC, in all of Hackage, only 3 packages needed to be changed because of this. If there were a larger impact to the GND breakage, I think your suggestion would be a good one.

The problem I'm adressing in this thread is different: that library authors have been given a new, not-backward-compatible way of preventing abuses of their datatypes, and no proposal I have seen really addresses all of the problems here. I'm hoping my no-role-annots package might be helpful, but it doesn't fully resolve the issues.

Richard

On Mar 31, 2014, at 2:51 AM, Dominique Devriese wrote:

...

Richard,

(re-posting because I first used an address that is not subscribed to the lists)

I've been wondering about the following: it seems like the main problem in this situation is that the GeneralizedNewtypeDeriving extension changed meaning from "just coerce everything while deriving" to "only coerce stuff if it's allowed by the relevant role annotations". Would it not be an alternative solution to split up the GND extension into

* a backwards-compatible one (called GeneralizedNewtypeDeriving for backwards compatibility ;)) that ignores role annotations (as before) and uses unsafeCoerce whereever necessary * a safe one (called e.g. SafeNewtypeDeriving) that respects role annotations

The first one could then be deprecated and removed in a release or two. That might give library maintainers time to move their packages to SafeNewtypeDeriving when they have tested that everything works...

Regards, Dominique

P.S.: The above is based on a limited understanding of the problem, so I'm sorry if it misses some aspect of the problem...

2014-03-31 2:14 GMT+02:00 Richard Eisenberg :

...

I spent some time thinking about what, precisely, can be done here to make folks happier. (See the thread beginning here: http://www.haskell.org/pipermail/libraries/2014-March/022321.html) And, the answer seemed to all be in the concrete syntax. The only logical alternative (that I could think of) to having roles is to disallow GND, and I don't think anyone is proposing that. And, it is impossible to infer an author's desired roles for a datatype. The heuristics mentioned here all seem far too fragile and hard to predict to become a lasting feature of GHC (in my opinion). What's left? Concrete syntax.

So, I have written and uploaded no-role-annots-1.0, a backward-compatible alternative to role annotations -- no CPP required. It's not as principled as proper role annotations, but it should do the job for most users.

Here are two examples:

1. Datatypes:

...

import Language.Haskell.RoleAnnots

data Map k v = (Nominal k, Representational v) => MkMap [(k,v)]

The constraints (which need be put on only one data constructor, if there are many) will get the built-in role inference mechanism to do what the user requests. In this example, the `Representational v` is actually redundant, but causes no harm. Because these classes have universal instances ("instance Nominal a") and have no methods, they should have no run-time significance. The only downside I can see is that the code above needs -XGADTs or -XExistentialQuantification to work, though it is neither a GADT nor has existentials. (Pattern-matching on such a definition needs no extensions.)

2. Newtypes:

Newtype constructors cannot be constrained, unfortunately. So, we have to resort to Template Haskell:

...

import Language.Haskell.RoleAnnots

roleAnnot [NominalR, RepresentationalR] [d| newtype Map k v = MkMap [(k, v)] |]

This is clearly worse, but I was able to come up with no other solution that worked for newtypes. Note that, in the example, I used the fact that Template Haskell interprets a bare top-level expression as a Template Haskell splice. We could also wrap that line in $( ... ) to be more explicit about the use of TH. Also problematic here is that the code above requires -XRoleAnnotations in GHC 7.8. To get this extension enabled without using CPP, put it in a condition chunk in your .cabal file, like this:

...

if impl(ghc >= 7.8) default-extensions: RoleAnnotations

I hope this is helpful to everyone. Please feel free to post issues/pull requests to my github repo at github.com/goldfirere/no-role-annots.

Thanks, Richard

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs