one thing i'm confused by, and this wasn't properly addressed in the prior threads, is for a type like data Annotated t ann = MkAnn t ann would you consider the following unsafe? instance Eq t => Eq ( Annotated t ann) (==) (MkAnn t1 _) (MkAnn t2 _ ) = t1 == t2 instance Ord t => Ord (Annotated t ann) compare (MkAnn t1 _) (MkAnn t2 _) = compare t1 t2 by the rubric you've proposed these might be *BAD*, right? But theres many cases where you're doing computation on data, and you wish to track origin, time, etc, but these annotations are *irrelevant* for the actual values computed... It sounds like the proposal you want would rule out such instances! am i not understanding your proposal? cheers -Carter On Sun, Oct 6, 2013 at 5:46 PM, Johan Tibell wrote:

What would break if we changed it to Unsafe?

On Sun, Oct 6, 2013 at 2:32 PM, Ryan Newton wrote:

...

Hi all,

We had a discussion on haskell-cafe, which only confirmed the unreliability of "Eq" and "Ord". Fine. But if I instead want to define a SafeEq class and instances based on GHC.Generics, then I can't have users writing Generic instances that lie about the structure of their types.

But as you can see this module is currently marked "Trustworthy":

http://www.haskell.org/ghc/docs/7.4.1/html/libraries/ghc-prim-0.2.0.0/GHC-Ge...

...

I simply propose that this is changed to "Unsafe"! [1]

Context: The bottom line is that I believe in the Safe-Haskell mission here, and I want to be able to deliver practical libraries that give a strong

...

that types can't be broken. Currently, the LVish library has one big hole in it because of this Eq/Ord limitation; the problem is documented here. If we can provide incontrovertible Safe-Haskel guarantees, this is a huge, qualitative difference compared to what's possible in the C++, Java's and MLs of the world. There are plenty of libraries that provide guarantees like "deterministic parallelism" IFF the user does everything right and breaks no rules (CnC, Pochoir, etc). But we can do better!

Best, -Ryan

[1] Small detail... some of these bindings, like the class name "Generic" itself, will still need to be accessible from a Trustworthy library. I propose GHC.Generics.Safe for that, following existing precedent.

On Sun, Oct 6, 2013 at 5:18 PM, Ryan Newton wrote:

...

Thanks for the responses all.

I'm afraid the point about GHC.Generics got lost here. I'll respond and then rename this as a specific library proposal.

I don't want to fix the world's Eq instances, but I am ok with requiring that people "derive Generic" for any data they want to put in an LVar container. (From which I can give them a SafeEq instance.)

It's not just LVish that is in this boat.... any library that tries to provide deterministic parallelism outside of the IO monad has some very

fine

...

lines to walk. Take a look at Accelerate. It is deterministic (as long as you run only with the CUDA backend and only on one specific GPU... otherwise fold topologies may look different and non-associative folds may leak). Besides, "runAcc" does a huge amount of implicit IO (writing to disk, calling nvcc, etc)! At the very least this could fail if the disk if full. But then again, regular "pure" computations fail when memory runs out... so I'm ok grouping these in the same bucket for now. "Determinism modulo available resources."

...

A possible problem with marking "instance Eq" as an unsafe feature is that many modules would be only Trustworthy instead of Safe.

My proposal is actually more narrow than that. My proposal is to mark GHC.Generics as Unsafe.

That way I can define my own SafeEq, and know that someone can't break it by making a Generic instance that lies. It is very hard for me to see why people should be able to make their own Generic instances (that might

...

...

about the structure of the type), in Safe-Haskell.

...

That would go against my "every purely functional module is

automatically

...

safe because the compiler checks that it cannot launch the missiles" understanding of Safe Haskell.

Heh, that may already be violated by the fact that you can't use other extensions like OverlappingInstances, or provide your own Typeable instances.

...

Actually, Eq instances are not unsafe per se, but only if I also use

some

...

other module that assumes certain properties about all Eq instances in scope. So in order to check safety, two independent modules (the

guarantee lie provider

...

...

...

and the consumer of the Eq instance) would have to cooperate.

I've found, that this is a very common problem that we have when trying to make our libraries Safe-Haskell compliant -- often we want to permit and deny combinations of modules. I don't have a solution I'm afraid.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Let me tweak that -- I think there will be places where "import GHC.Generics" must be changed to ""import GHC.Generics.Safe" to get the Generic symbol. Alternatively, some essential piece, like the class Generic's members could be hidden from GHC.Generics and moved into GHC.Generics.Internal (and that would be the Unsafe one). That would probably be a good idea for minimizing impact on current SafeHaskell code.... but it might have more impact on non-safehaskell code, which would never have detected this status switch in the first place. Also, .Safe is more in line with the existing conventions. On Sun, Oct 6, 2013 at 11:18 PM, Ryan Newton wrote:

I can't know for certain but I think I would bet money on "nothing".

Edward & David may know more about what actual use SafeHaskell is getting?

On Sun, Oct 6, 2013 at 5:46 PM, Johan Tibell wrote:

...

What would break if we changed it to Unsafe?

On Sun, Oct 6, 2013 at 2:32 PM, Ryan Newton wrote:

...

Hi all,

We had a discussion on haskell-cafe, which only confirmed the unreliability of "Eq" and "Ord". Fine. But if I instead want to define a SafeEq class and instances based on GHC.Generics, then I can't have users writing Generic instances that lie about the structure of their types.

But as you can see this module is currently marked "Trustworthy":

http://www.haskell.org/ghc/docs/7.4.1/html/libraries/ghc-prim-0.2.0.0/GHC-Ge...

...

I simply propose that this is changed to "Unsafe"! [1]

Context: The bottom line is that I believe in the Safe-Haskell mission here, and

...

want to be able to deliver practical libraries that give a strong guarantee that types can't be broken. Currently, the LVish library has one big hole in it because of this Eq/Ord limitation; the problem is documented here. If we can provide incontrovertible Safe-Haskel guarantees, this is a huge, qualitative difference compared to what's possible in the C++, Java's and MLs of the world. There are plenty of libraries that provide guarantees like "deterministic parallelism" IFF the user does everything right and breaks no rules (CnC, Pochoir, etc). But we can do better!

Best, -Ryan

[1] Small detail... some of these bindings, like the class name "Generic" itself, will still need to be accessible from a Trustworthy library. I propose GHC.Generics.Safe for that, following existing precedent.

On Sun, Oct 6, 2013 at 5:18 PM, Ryan Newton wrote:

...

Thanks for the responses all.

I'm afraid the point about GHC.Generics got lost here. I'll respond

and

...

then rename this as a specific library proposal.

I don't want to fix the world's Eq instances, but I am ok with requiring that people "derive Generic" for any data they want to put in an LVar container. (From which I can give them a SafeEq instance.)

It's not just LVish that is in this boat.... any library that tries to provide deterministic parallelism outside of the IO monad has some very fine lines to walk. Take a look at Accelerate. It is deterministic (as long as you run only with the CUDA backend and only on one specific GPU... otherwise fold topologies may look different and non-associative folds may leak). Besides, "runAcc" does a huge amount of implicit IO (writing to disk, calling nvcc, etc)! At the very least this could fail if the disk if full. But then again, regular "pure" computations fail when memory runs out... so I'm ok grouping these in the same bucket for now. "Determinism modulo available resources."

...

A possible problem with marking "instance Eq" as an unsafe feature is that many modules would be only Trustworthy instead of Safe.

My proposal is actually more narrow than that. My proposal is to mark GHC.Generics as Unsafe.

That way I can define my own SafeEq, and know that someone can't break it by making a Generic instance that lies. It is very hard for me to see why people should be able to make their own Generic instances (that might

...

...

about the structure of the type), in Safe-Haskell.

...

That would go against my "every purely functional module is

automatically

...

safe because the compiler checks that it cannot launch the missiles" understanding of Safe Haskell.

Heh, that may already be violated by the fact that you can't use other extensions like OverlappingInstances, or provide your own Typeable instances.

...

Actually, Eq instances are not unsafe per se, but only if I also use

some

...

other module that assumes certain properties about all Eq instances in scope. So in order to check safety, two independent modules (the

...

...

...

and the consumer of the Eq instance) would have to cooperate.

I've found, that this is a very common problem that we have when

I lie provider trying to

...

...

make our libraries Safe-Haskell compliant -- often we want to permit and deny combinations of modules. I don't have a solution I'm afraid.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett wrote:

Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself?

Exactly.

Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead?

The new rule would be that if you hand-implemented Generic or Generic1 in

your module it isn't Safe.

Ah... so you're proposing adding an extra rule to the GHC compiler itself? Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really want you can mark your module as TrustWorthy even with manual instances. (And then, yes, we'd need a GHC.Generics.Safe to pull the Generic symbol itself from when we just want to derive it.) That would make it so that derived Generic, Generic1 would be considered

Safe, and you wouldn't break literally every user of the library who care about Safe Haskell.

Isn't that also accomplished just by "import GHC.Generic.Safe (Generic)"? And the handful of hypothetical people that are using this right now can make that one-line fix. I will *personally* make that fix for all those people if they would come forward and share the link to their code ;-). As it stands the things are damn-near impossible to get right instantiating

them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them!

Exactly!

Thanks Edward! That provides a real example of someone who uses both SafeHaskell and GHC.Generics in their libraries! I'm glad to know such a person exists but that you don't depend on handwritten Generic instances. (I'm also glad to hear that you care about SafeHaskell in the first place!) I totally agree that #ifdef'ing on GHC version is a pain, and all that code would need to conditionally import GHC.Generics or GHC.Generics.Safe to retain -XSafe status. I'm perfectly happy with the GHC-internal fix if someone can point me to how to do it / where it should go. I guess I'll start by greping for Typeable in the compiler. Is this something you can do trivially Edward? (Or others?) Just for reference here is a small patch to Base representing my library-based way of making the change discussed in this thread: https://github.com/rrnewton/packages-base/commit/23f8f464a0b46ca17c655c4144f... Again, my use-case (closed/safe Eq & Ord instances) is a few steps removed from this Generics business and I don't feel strongly about how it gets done. But having *SOME* way of doing it will be a massive boon to my lab's particular effort at maximizing the abilities of guaranteed-Safe deterministic parallelism in Haskell. Further, I think anyone else who tries to do something in SafeHaskell that depends on guaranteeing a closed/controlled set of instances (SafeEq, SafeOrd but also others) based on GHC.Generics will *also* find this change useful. Best, -Ryan On Mon, Oct 7, 2013 at 12:44 PM, Edward Kmett wrote:

I'd much rather have code that can compile as Safe rather than Trustworthy. Trustworthy code is a pain to annotate the safety of correctly.

A one line change to GHC to ensure the safety of GHC.Generics and to maximize the number of operations you can use safely seems entirely reasonable.

Otherwise I have to mark perfectly Safe-by-construction code Trustworthy, enlarging the trusted code base considerably. Using properly constructed Generic instances is perfectly safe.

That library level fix has knock-on effects across the entire ecosystem for anyone who cares about SafeHaskell. I tend to instantiate GHC.Generics for any type in most of my packages. A quick grep finds 209 modules broken in just the projects I have open in my working directory. That is a mixture of public and private repos. The majority of which are public. So, I first have to change a couple hundred import statements... conditionally on compiler version, since I maintain wide support windows. This fix would literally break SafeHaskell for me silently across almost everything I have written, forcing me to run through tons of packages trying to figure out what I have to change to get it back. I've done this song and dance before trying to get lambdabot to build, since it uses lens these days. If I don't get every one of them right lens builds as unsafe. There isn't a good debugging process for this, as the only real way I have to know it worked is to go through each one and make sure I didn't compromise safe haskell by looking at the resulting haddocks for each case.

The code that *uses* GHC.Generics uses their internals and is safe by construction. It is only the instance construction that is unsafe, so lets mark it as such.

-Edward

On Mon, Oct 7, 2013 at 11:11 AM, Ryan Newton wrote:

...

On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett wrote:

...

Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself?

Exactly.

...

Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead?

The new rule would be that if you hand-implemented Generic or Generic1 in

...

your module it isn't Safe.

Ah... so you're proposing adding an extra rule to the GHC compiler itself?

Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really want you can mark your module as TrustWorthy even with manual instances. (And then, yes, we'd need a GHC.Generics.Safe to pull the Generic symbol itself from when we just want to derive it.)

That would make it so that derived Generic, Generic1 would be considered

...

Safe, and you wouldn't break literally every user of the library who care about Safe Haskell.

Isn't that also accomplished just by "import GHC.Generic.Safe (Generic)"? And the handful of hypothetical people that are using this right now can make that one-line fix. I will *personally* make that fix for all those people if they would come forward and share the link to their code ;-).

As it stands the things are damn-near impossible to get right

...

instantiating them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them!

Exactly!