The degree to which we believe this "partially" is important: If we are willing to believe that the only library writers we care about doing this are those in the core, then fine. In this case we shouldn't feel compelled to suggest to library writers that they annotate, ever. I'm good with this. If the team here thinks otherwise, that we need to start a campaign to get every library writer to eventually annotate, then I have deep objections. The situation today is that · A client of a library can use GND to do bad things to the library (e.g. change the “key” type of (Map key value)). · Role annotations allow the library author to prevent that happening. Would you say that means that we are “compelled to suggest to library writers that they annotate”? I would have thought that it would indeed be good to suggest to them that a new opportunity exists for them to make their library more robust to clients. They are free to do nothing, or to take advantage of the suggestion. It’s an upside-only situation. Looking further ahead, when you say that “there can be no migration from representational-by-default”, do you have data to support that? Notably, any client not using GND could not observe a change. So simply seeing how many library modules use GND would be an upper bound on how many libraries would fail to compile you were to ask us to change the default. Is that 1% of Hackage modules? 10%? 0.1%? I don’t know. The awkward bit is that if a client is using GND, which fails after a change to nominal-by-default, the fix is to change the library, not the client, and I can see that is awkward. Simon From: Libraries [mailto:libraries-bounces@haskell.org] On Behalf Of Mark Lentczner Sent: 25 March 2014 15:10 To: libraries@haskell.org Libraries; ghc-devs@haskell.org Subject: Re: We need to add role annotations for 7.8 Thank you to everyone who has been helping me understand this issue in greater depth. tl;dr: As long as we don't expect any libraries beyond to core to annotate, I'm cool. This presumes that the extra safety isn't, in practice, dependent on transitive adoption by libraries. It also implies that representational is the only possible default, and that there can be no migration from it. My approach to thinking about this is guided by thinking about supporting an eco-system with 1000s of libraries (hackage), a few dozen of which are heavily promoted (the platform), and a small set that are closely tied to the compiler (the core). The availability, speed of release, motivation, and even skill of the the developers varies widely over that range. I also think about the various "stances" of different developers: * End developer: makes use of libraries, but just builds apps * Internal developer: makes libraries for internal use in a project * Casual library writer: makes libraries, primarily for their own needs, but distributed on hackage * Popular library writer: actively maintains libraries which are widely used * Core library writer: maintainer of a core package that stays in lock step with the compiler Then, I think about, for each of these, what is the effect on a new feature on them, their existing code, and future code? Does it affect them only if they are using the feature? If they aren't using the feature? For library writers, how does the feature affect clients? If a client wants to use a feature, under what conditions does the library need to do something? This last issue of the "transitivity" the feature is often the biggest concern. Given that... onto type roles: The default of representational is the only option, because a default of nominal would require far too many developers to have to update their code. I don't believe that we can ever migrate to nominal as default. The feature implies that any abstract data type that uses a type parameter in certain ways needs annotate to get the full safety afforded now afforded. However, without annotation, the data type is still no worse off than it was before (there is added safety, but not perhaps relevant to the stand point of the library writer). Further, this (pre-existing) non-safety isn't likely a huge concern. Making sure the docs take the tone that most developers need to nothing, and when developers need to be concerned seems like an important way to ensure the right outcome. A key question here is transitivity: Is it possible for module A to not annotate a type, and then have module B by a different author use the type in A in another abstract type, that is annotated, and get the benefit. Seems the answer is "partially". If the answer were "no", then use of the feature would be dependent on transitive adoption, and that is where the big burden on developers comes from. The degree to which we believe this "partially" is important: If we are willing to believe that the only library writers we care about doing this are those in the core, then fine. In this case we shouldn't feel compelled to suggest to library writers that they annotate, ever. I'm good with this. If the team here thinks otherwise, that we need to start a campaign to get every library writer to eventually annotate, then I have deep objections. I read the paper, and understand how the authors felt the syntax options were all less than perfect, and choose what they did. But that choice, perhaps unwittingly, the implication that it forces -XCPP on all libraries except perhaps some of the core. This is because they all need to support previous compilers. So, a one line annotation has turned into an ugly beast, and perhaps added -XCPP where there was none, which is really unfortunate. (I, like many, consider it a defeat when one has to resort to -XCPP.) It seems to me that the paper didn't really consider less-perfect, heuristic solutions. It might have had significantly less impact on library writers were some heuristic (no constructors exported? has any type constraint on the parameter? etc..) might have allowed most data types to go without annotation at the cost of a few (where nominal was incorrectly inferred) requiring immediate action. In this situation, a non-language feature (pragma or other device) might have been more palatable. Finally, on the choice of terms, nominal, representational, and phantom all seem like clear, self-explanatory choices to me. - Mark

Hi Mark, I appreciate your analysis in terms of classes of users -- I think that is helpful for framing the discussion. About transitivity: I think we're in the clear here. Let's say package A exports types missing role annotations. If package B imports package A and wants to have the full safety afforded by roles, that is no problem whatsoever. Package B has annotations on its types (which may use package A's types) that may restrict certain parameters to be nominal, as appropriate. If package A had role annotations, it's quite possible that package B could omit some annotations (as role inference propagates nominal roles), but there is no problem inherent in this. (Indeed, if package A adds annotations in the future, package B would have redundant, but harmless, annotations.) So, I disagree with Mark's "partially" below -- I think we're fully OK in this regard. About heuristics: we briefly considered some, though there's no documentation of this anywhere. Specifically, we thought about giving nominal roles to parameters used in class constraints. The problem is, in the actual datatype definition, the constraints tend not to appear? Should we look around for other functions with constraints? That seems likely to be more confusing than helpful. Furthermore, I strongly don't like the idea of using heuristics to infer a feature such as this -- it can cause strange behavior and is hard to specify. Richard On Mar 25, 2014, at 11:09 AM, Mark Lentczner wrote:

Thank you to everyone who has been helping me understand this issue in greater depth.

tl;dr: As long as we don't expect any libraries beyond to core to annotate, I'm cool. This presumes that the extra safety isn't, in practice, dependent on transitive adoption by libraries. It also implies that representational is the only possible default, and that there can be no migration from it.

My approach to thinking about this is guided by thinking about supporting an eco-system with 1000s of libraries (hackage), a few dozen of which are heavily promoted (the platform), and a small set that are closely tied to the compiler (the core). The availability, speed of release, motivation, and even skill of the the developers varies widely over that range.

I also think about the various "stances" of different developers: End developer: makes use of libraries, but just builds apps Internal developer: makes libraries for internal use in a project Casual library writer: makes libraries, primarily for their own needs, but distributed on hackage Popular library writer: actively maintains libraries which are widely used Core library writer: maintainer of a core package that stays in lock step with the compiler Then, I think about, for each of these, what is the effect on a new feature on them, their existing code, and future code? Does it affect them only if they are using the feature? If they aren't using the feature? For library writers, how does the feature affect clients? If a client wants to use a feature, under what conditions does the library need to do something? This last issue of the "transitivity" the feature is often the biggest concern.

Given that... onto type roles:

The default of representational is the only option, because a default of nominal would require far too many developers to have to update their code. I don't believe that we can ever migrate to nominal as default.

The feature implies that any abstract data type that uses a type parameter in certain ways needs annotate to get the full safety afforded now afforded. However, without annotation, the data type is still no worse off than it was before (there is added safety, but not perhaps relevant to the stand point of the library writer). Further, this (pre-existing) non-safety isn't likely a huge concern. Making sure the docs take the tone that most developers need to nothing, and when developers need to be concerned seems like an important way to ensure the right outcome.

A key question here is transitivity: Is it possible for module A to not annotate a type, and then have module B by a different author use the type in A in another abstract type, that is annotated, and get the benefit. Seems the answer is "partially". If the answer were "no", then use of the feature would be dependent on transitive adoption, and that is where the big burden on developers comes from.

The degree to which we believe this "partially" is important: If we are willing to believe that the only library writers we care about doing this are those in the core, then fine. In this case we shouldn't feel compelled to suggest to library writers that they annotate, ever. I'm good with this. If the team here thinks otherwise, that we need to start a campaign to get every library writer to eventually annotate, then I have deep objections.

I read the paper, and understand how the authors felt the syntax options were all less than perfect, and choose what they did. But that choice, perhaps unwittingly, the implication that it forces -XCPP on all libraries except perhaps some of the core. This is because they all need to support previous compilers. So, a one line annotation has turned into an ugly beast, and perhaps added -XCPP where there was none, which is really unfortunate. (I, like many, consider it a defeat when one has to resort to -XCPP.)

It seems to me that the paper didn't really consider less-perfect, heuristic solutions. It might have had significantly less impact on library writers were some heuristic (no constructors exported? has any type constraint on the parameter? etc..) might have allowed most data types to go without annotation at the cost of a few (where nominal was incorrectly inferred) requiring immediate action. In this situation, a non-language feature (pragma or other device) might have been more palatable.

Finally, on the choice of terms, nominal, representational, and phantom all seem like clear, self-explanatory choices to me.

- Mark

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs