one thing i'm confused by, and this wasn't properly addressed in the prior threads, is for a type like data Annotated t ann = MkAnn t ann would you consider the following unsafe? instance Eq t => Eq ( Annotated t ann) (==) (MkAnn t1 _) (MkAnn t2 _ ) = t1 == t2 instance Ord t => Ord (Annotated t ann) compare (MkAnn t1 _) (MkAnn t2 _) = compare t1 t2 by the rubric you've proposed these might be *BAD*, right? But theres many cases where you're doing computation on data, and you wish to track origin, time, etc, but these annotations are *irrelevant* for the actual values computed... It sounds like the proposal you want would rule out such instances! am i not understanding your proposal? cheers -Carter On Sun, Oct 6, 2013 at 5:46 PM, Johan Tibell wrote:

What would break if we changed it to Unsafe?

On Sun, Oct 6, 2013 at 2:32 PM, Ryan Newton wrote:

...

Hi all,

We had a discussion on haskell-cafe, which only confirmed the unreliability of "Eq" and "Ord". Fine. But if I instead want to define a SafeEq class and instances based on GHC.Generics, then I can't have users writing Generic instances that lie about the structure of their types.

But as you can see this module is currently marked "Trustworthy":

http://www.haskell.org/ghc/docs/7.4.1/html/libraries/ghc-prim-0.2.0.0/GHC-Ge...

...

I simply propose that this is changed to "Unsafe"! [1]

Context: The bottom line is that I believe in the Safe-Haskell mission here, and I want to be able to deliver practical libraries that give a strong

...

that types can't be broken. Currently, the LVish library has one big hole in it because of this Eq/Ord limitation; the problem is documented here. If we can provide incontrovertible Safe-Haskel guarantees, this is a huge, qualitative difference compared to what's possible in the C++, Java's and MLs of the world. There are plenty of libraries that provide guarantees like "deterministic parallelism" IFF the user does everything right and breaks no rules (CnC, Pochoir, etc). But we can do better!

Best, -Ryan

[1] Small detail... some of these bindings, like the class name "Generic" itself, will still need to be accessible from a Trustworthy library. I propose GHC.Generics.Safe for that, following existing precedent.

On Sun, Oct 6, 2013 at 5:18 PM, Ryan Newton wrote:

...

Thanks for the responses all.

I'm afraid the point about GHC.Generics got lost here. I'll respond and then rename this as a specific library proposal.

I don't want to fix the world's Eq instances, but I am ok with requiring that people "derive Generic" for any data they want to put in an LVar container. (From which I can give them a SafeEq instance.)

It's not just LVish that is in this boat.... any library that tries to provide deterministic parallelism outside of the IO monad has some very

fine

...

lines to walk. Take a look at Accelerate. It is deterministic (as long as you run only with the CUDA backend and only on one specific GPU... otherwise fold topologies may look different and non-associative folds may leak). Besides, "runAcc" does a huge amount of implicit IO (writing to disk, calling nvcc, etc)! At the very least this could fail if the disk if full. But then again, regular "pure" computations fail when memory runs out... so I'm ok grouping these in the same bucket for now. "Determinism modulo available resources."

...

A possible problem with marking "instance Eq" as an unsafe feature is that many modules would be only Trustworthy instead of Safe.

My proposal is actually more narrow than that. My proposal is to mark GHC.Generics as Unsafe.

That way I can define my own SafeEq, and know that someone can't break it by making a Generic instance that lies. It is very hard for me to see why people should be able to make their own Generic instances (that might

...

...

about the structure of the type), in Safe-Haskell.

...

That would go against my "every purely functional module is

automatically

...

safe because the compiler checks that it cannot launch the missiles" understanding of Safe Haskell.

Heh, that may already be violated by the fact that you can't use other extensions like OverlappingInstances, or provide your own Typeable instances.

...

Actually, Eq instances are not unsafe per se, but only if I also use

some

...

other module that assumes certain properties about all Eq instances in scope. So in order to check safety, two independent modules (the

guarantee lie provider

...

...

...

and the consumer of the Eq instance) would have to cooperate.

I've found, that this is a very common problem that we have when trying to make our libraries Safe-Haskell compliant -- often we want to permit and deny combinations of modules. I don't have a solution I'm afraid.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Let me tweak that -- I think there will be places where "import GHC.Generics" must be changed to ""import GHC.Generics.Safe" to get the Generic symbol. Alternatively, some essential piece, like the class Generic's members could be hidden from GHC.Generics and moved into GHC.Generics.Internal (and that would be the Unsafe one). That would probably be a good idea for minimizing impact on current SafeHaskell code.... but it might have more impact on non-safehaskell code, which would never have detected this status switch in the first place. Also, .Safe is more in line with the existing conventions. On Sun, Oct 6, 2013 at 11:18 PM, Ryan Newton wrote:

I can't know for certain but I think I would bet money on "nothing".

Edward & David may know more about what actual use SafeHaskell is getting?

On Sun, Oct 6, 2013 at 5:46 PM, Johan Tibell wrote:

...

What would break if we changed it to Unsafe?

On Sun, Oct 6, 2013 at 2:32 PM, Ryan Newton wrote:

...

Hi all,

We had a discussion on haskell-cafe, which only confirmed the unreliability of "Eq" and "Ord". Fine. But if I instead want to define a SafeEq class and instances based on GHC.Generics, then I can't have users writing Generic instances that lie about the structure of their types.

But as you can see this module is currently marked "Trustworthy":

http://www.haskell.org/ghc/docs/7.4.1/html/libraries/ghc-prim-0.2.0.0/GHC-Ge...

...

I simply propose that this is changed to "Unsafe"! [1]

Context: The bottom line is that I believe in the Safe-Haskell mission here, and

...

want to be able to deliver practical libraries that give a strong guarantee that types can't be broken. Currently, the LVish library has one big hole in it because of this Eq/Ord limitation; the problem is documented here. If we can provide incontrovertible Safe-Haskel guarantees, this is a huge, qualitative difference compared to what's possible in the C++, Java's and MLs of the world. There are plenty of libraries that provide guarantees like "deterministic parallelism" IFF the user does everything right and breaks no rules (CnC, Pochoir, etc). But we can do better!

Best, -Ryan

[1] Small detail... some of these bindings, like the class name "Generic" itself, will still need to be accessible from a Trustworthy library. I propose GHC.Generics.Safe for that, following existing precedent.

On Sun, Oct 6, 2013 at 5:18 PM, Ryan Newton wrote:

...

Thanks for the responses all.

I'm afraid the point about GHC.Generics got lost here. I'll respond

and

...

then rename this as a specific library proposal.

I don't want to fix the world's Eq instances, but I am ok with requiring that people "derive Generic" for any data they want to put in an LVar container. (From which I can give them a SafeEq instance.)

It's not just LVish that is in this boat.... any library that tries to provide deterministic parallelism outside of the IO monad has some very fine lines to walk. Take a look at Accelerate. It is deterministic (as long as you run only with the CUDA backend and only on one specific GPU... otherwise fold topologies may look different and non-associative folds may leak). Besides, "runAcc" does a huge amount of implicit IO (writing to disk, calling nvcc, etc)! At the very least this could fail if the disk if full. But then again, regular "pure" computations fail when memory runs out... so I'm ok grouping these in the same bucket for now. "Determinism modulo available resources."

...

A possible problem with marking "instance Eq" as an unsafe feature is that many modules would be only Trustworthy instead of Safe.

My proposal is actually more narrow than that. My proposal is to mark GHC.Generics as Unsafe.

That way I can define my own SafeEq, and know that someone can't break it by making a Generic instance that lies. It is very hard for me to see why people should be able to make their own Generic instances (that might

...

...

about the structure of the type), in Safe-Haskell.

...

That would go against my "every purely functional module is

automatically

...

safe because the compiler checks that it cannot launch the missiles" understanding of Safe Haskell.

Heh, that may already be violated by the fact that you can't use other extensions like OverlappingInstances, or provide your own Typeable instances.

...

Actually, Eq instances are not unsafe per se, but only if I also use

some

...

other module that assumes certain properties about all Eq instances in scope. So in order to check safety, two independent modules (the

...

...

...

and the consumer of the Eq instance) would have to cooperate.

I've found, that this is a very common problem that we have when

I lie provider trying to

...

...

make our libraries Safe-Haskell compliant -- often we want to permit and deny combinations of modules. I don't have a solution I'm afraid.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Sun, Oct 6, 2013 at 6:28 PM, Ganesh Sittampalam wrote:

- Referential transparency: e.g. no unsafePerformIO

- Module boundary control: no abstraction violation like Template

Haskell and GeneralizedNewtypeDeriving - Semantic consistency: importing a safe module can't change existing code, so no OverlappingInstances and the like

Is this change necessary to preserve the existing properties, or are you

hoping to add a new one?

I'm not currently aware of ways to break these invariants *just* with GHC.Generics. Hmm, but I would like to know why it is marked trustworthy and not inferred-safe... My argument is more that it is a violation in spirit to write bad Generic instances that in turn lead to bad fromRep and toRep conversions. (In a similar way to bad Typeable insteances leading to bad type conversions). Second, the usage intent of GHC.Generics as far as I can see, is that users 99.9% of the time will be using -XDeriveGeneric. Third, without making this change I know of no way to provide safe/correct Eq and Ord instances in a way that extends to user datatypes. And in LVish we have one example of what I think could be a valuable Safe-Haskell compliant parallel programming library (see the POPL'14 paper for more detailshttp://www.cs.indiana.edu/~rrnewton/papers/2013_07_LVish_quasiDet_working_dr...), but which will never be fully water-tight without vetted Eq and Ord instances.

I also understand that you want to require 'standard' Generic instances on types - will that mean that module authors are forced to expose internals to use your library?

Great point! My intent was to short-circuit that by providing "TrustWorthy" SafeEq and SafeOrd instances for standard types. If you know any better ways to go forward, let me know! Also, if anyone uses SafeHaskell with non-derived Generic instances, please speak up! Cheers, -Ryan

Indeed! That seems to be a direct violation of this language in the manual: *"An important part of this is that safe compiled code is not able to examine or create data values using data constructors that it cannot import" * And the funny part is that that is unrelated to my particular problem of the user making their own Generic instances and is instead related to simply exposing "to"... Thanks for putting that together. On Mon, Oct 7, 2013 at 1:13 AM, John Lato wrote:

On Sun, Oct 6, 2013 at 10:14 PM, Ryan Newton wrote:

...

On Sun, Oct 6, 2013 at 6:28 PM, Ganesh Sittampalam wrote:

...

- Referential transparency: e.g. no unsafePerformIO

- Module boundary control: no abstraction violation like Template

...

Haskell and GeneralizedNewtypeDeriving - Semantic consistency: importing a safe module can't change existing code, so no OverlappingInstances and the like

Is this change necessary to preserve the existing properties, or are you

...

hoping to add a new one?

I'm not currently aware of ways to break these invariants *just* with GHC.Generics. Hmm, but I would like to know why it is marked trustworthy and not inferred-safe...

How about this demo repo? https://github.com/JohnLato/safe-bugtest

I'm really not a safe haskell expert, but I believe this is a demonstration of using GHC.Generics to violate a module's abstraction boundaries with SafeHaskell enabled.

If I'm incorrect, I would appreciate if somebody could explain my error. If, however, I'm correct, then I think that Ryan's proposal of marking GHC.Generics Unsafe is the best way to remedy the problem.

A possible stumbling block may involve base and package-trust, but I'm not certain of the current status.

John L.

i assume https://github.com/JohnLato/safe-bugtest/blob/master/Main.hs#L13should say putStrLn "Should print \"Pos (2)\"" rather than -2? On Mon, Oct 7, 2013 at 1:23 AM, Carter Schonwald

wrote:

ooo, thats illuminating.

thanks for cooking that up

On Mon, Oct 7, 2013 at 1:13 AM, John Lato wrote:

...

On Sun, Oct 6, 2013 at 10:14 PM, Ryan Newton wrote:

...

On Sun, Oct 6, 2013 at 6:28 PM, Ganesh Sittampalam wrote:

...

- Referential transparency: e.g. no unsafePerformIO

- Module boundary control: no abstraction violation like Template

...

Haskell and GeneralizedNewtypeDeriving - Semantic consistency: importing a safe module can't change existing code, so no OverlappingInstances and the like

Is this change necessary to preserve the existing properties, or are you

...

hoping to add a new one?

I'm not currently aware of ways to break these invariants *just* with GHC.Generics. Hmm, but I would like to know why it is marked trustworthy and not inferred-safe...

How about this demo repo? https://github.com/JohnLato/safe-bugtest

I'm really not a safe haskell expert, but I believe this is a demonstration of using GHC.Generics to violate a module's abstraction boundaries with SafeHaskell enabled.

If I'm incorrect, I would appreciate if somebody could explain my error. If, however, I'm correct, then I think that Ryan's proposal of marking GHC.Generics Unsafe is the best way to remedy the problem.

A possible stumbling block may involve base and package-trust, but I'm not certain of the current status.

John L.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Also, I'm not really sure this is a bug, per se. In my opinion when you allow for some sort of generic operations (whether via GHC.Generics or Data), it's roughly equivalent to exporting all the constructors. I don't see how it would work otherwise. But since there's precedent for Typeable, maybe Generics should be restricted in SafeHaskell as well. On Mon, Oct 7, 2013 at 1:05 AM, John Lato wrote:

Well, no. Presumably the example shouldn't compile at all. That message is more an indication that the demonstration is working as intended.

On Mon, Oct 7, 2013 at 12:31 AM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

i assume https://github.com/JohnLato/safe-bugtest/blob/master/Main.hs#L13should say putStrLn "Should print \"Pos (2)\""

rather than -2?

On Mon, Oct 7, 2013 at 1:23 AM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

ooo, thats illuminating.

thanks for cooking that up

On Mon, Oct 7, 2013 at 1:13 AM, John Lato wrote:

...

On Sun, Oct 6, 2013 at 10:14 PM, Ryan Newton wrote:

...

On Sun, Oct 6, 2013 at 6:28 PM, Ganesh Sittampalam wrote:

...

- Referential transparency: e.g. no unsafePerformIO

- Module boundary control: no abstraction violation like Template

...

Haskell and GeneralizedNewtypeDeriving - Semantic consistency: importing a safe module can't change existing code, so no OverlappingInstances and the like

Is this change necessary to preserve the existing properties, or are

...

you hoping to add a new one?

I'm not currently aware of ways to break these invariants *just* with GHC.Generics. Hmm, but I would like to know why it is marked trustworthy and not inferred-safe...

How about this demo repo? https://github.com/JohnLato/safe-bugtest

I'm really not a safe haskell expert, but I believe this is a demonstration of using GHC.Generics to violate a module's abstraction boundaries with SafeHaskell enabled.

If I'm incorrect, I would appreciate if somebody could explain my error. If, however, I'm correct, then I think that Ryan's proposal of marking GHC.Generics Unsafe is the best way to remedy the problem.

A possible stumbling block may involve base and package-trust, but I'm not certain of the current status.

John L.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Andres is right that it's not as evil as defining your own Typeable. The crux of the matter is that Generic essentially allows full access to the data type. Unfortunately it's easy to forget this... On Mon, Oct 7, 2013 at 1:43 AM, Andres Löh wrote:

While I understand you all feel uncomfortable with this, I do not think the problem demonstrated by John has anything to do with Generic.

I've made a fork here

https://github.com/kosmikus/safe-bugtest

that shows an (IMHO) similar problem using Show and Read instead of Generic. (And something slightly different could certainly also produced using Enum).

If you're deriving Generic, then yes, you gain the functionality of that class, which is to inspect the structure of the type, and then yes, you can in principle construct any value of that type. So deriving Generic for types that should be abstract is always going to be risky. But this is no different than deriving any other class, only that Generic gives you particularly fine-grained access to the internals of a type.

Also, at least in my opinion, it is entirely valid to define your own Generic instances. It's more work, and while I haven't used it often so far, I can imagine that there are good use cases. I don't think it's anywhere near as evil as defining your own Typeable instances.

Cheers, Andres

-- Andres Löh, Haskell Consultant Well-Typed LLP, http://www.well-typed.com

Here‘s a thought: doesn’t Generic already have an unused phantom type that's only there “just in case we need to add something in the future”? How about using that to track whether an instance was derived or not, and enforce *that* with SafeHaskell? Your SafeEq can then use a Generic Derived constraint or whatever.

Ah this sounds promising. You mean that 'x' param in from/to? So to use that, whereever I call to/from I would have to constrain that x to be "Derived". But do I understand correctly that GHC would have to modified so that its derived instances of Generic force that x param to be Derived?

On Tue, Oct 8, 2013 at 9:48 AM, José Pedro Magalhães wrote:

On Mon, Oct 7, 2013 at 10:32 AM, Dag Odenhall wrote:

...

Here‘s a thought: doesn’t Generic already have an unused phantom type that's only there “just in case we need to add something in the future”?

No, it doesn't.

Ah, I had misunderstood, then. :-) Just a thought: what if we had a type family

type family Derives (t :: k1) (c :: k2) :: Bool

which would automatically be instantiated by GHC appropriately? E.g., if the user had the following code:

data MyData = MyData deriving (Eq, Generic) deriving instance Show MyData instance Ord MyData

GHC would automatically instantiate:

type instance Derives MyData Eq = True type instance Derives MyData Generic = True type instance Derives MyData Show = True type instance Derives MyData Ord = False

Would this be something Ryan could use for detecting safe instances for LVish?

Yeah that looks like a much better plan to me. A slight issue with this is GHC's error messages for this are kinda bad (couldn't match 'False with 'True with little or no context) but that's probably fixable?

The formulation as a type family seems to conflict with the open-world principle. Would a class-based encoding instead be sufficient? -- the proposed, special wired-in class class Derives (t :: k1) (c :: k2) -- GHC would infer these from Pedro's example's declarations instance Derives MyData Eq instance Derives MyData Generic instance Derives MyData Show NB that there is no instance Derives MyData Ord, but standalone deriving could yield one "later" On Mon, Oct 14, 2013 at 10:02 PM, Ryan Newton wrote:

Hey, that's an awesome formulation! Thanks Pedro.

Any idea how much work this would be to implement in GHC, if it did garner approval?

On Tue, Oct 8, 2013 at 3:48 AM, José Pedro Magalhães wrote:

...

Hi,

On Mon, Oct 7, 2013 at 10:32 AM, Dag Odenhall wrote:

...

Here‘s a thought: doesn’t Generic already have an unused phantom type that's only there “just in case we need to add something in the future”?

No, it doesn't.

Just a thought: what if we had a type family

type family Derives (t :: k1) (c :: k2) :: Bool

which would automatically be instantiated by GHC appropriately? E.g., if the user had the following code:

data MyData = MyData deriving (Eq, Generic) deriving instance Show MyData instance Ord MyData

GHC would automatically instantiate:

type instance Derives MyData Eq = True type instance Derives MyData Generic = True type instance Derives MyData Show = True type instance Derives MyData Ord = False

Would this be something Ryan could use for detecting safe instances for LVish?

Cheers, Pedro

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

The issue with such an explicit false is that it requires more magic on behalf of the compiler. It would have to be filled in whenever an explicit `instance Eq Blah` was written. Recall that deriving instance Eq Blah can occur after the data type declaration site and may have to be used for many more complicated recursive data types, so it isn't sufficient to fill in at the data type declaration. -Edward On Mon, Oct 14, 2013 at 11:41 PM, Ryan Newton wrote:

But what stops the user from defining their own instances if they in fact did not derive it?

The explicit "False" in Pedro's formulation seems to serve this purpose.

On Mon, Oct 14, 2013 at 11:20 PM, Nicolas Frisby

...

wrote:

...

The formulation as a type family seems to conflict with the open-world principle. Would a class-based encoding instead be sufficient?

-- the proposed, special wired-in class class Derives (t :: k1) (c :: k2)

-- GHC would infer these from Pedro's example's declarations instance Derives MyData Eq instance Derives MyData Generic instance Derives MyData Show

NB that there is no instance Derives MyData Ord, but standalone deriving could yield one "later"

On Mon, Oct 14, 2013 at 10:02 PM, Ryan Newton wrote:

...

Hey, that's an awesome formulation! Thanks Pedro.

Any idea how much work this would be to implement in GHC, if it did garner approval?

On Tue, Oct 8, 2013 at 3:48 AM, José Pedro Magalhães wrote:

...

Hi,

On Mon, Oct 7, 2013 at 10:32 AM, Dag Odenhall wrote:

...

Here‘s a thought: doesn’t Generic already have an unused phantom type that's only there “just in case we need to add something in the future”?

No, it doesn't.

Just a thought: what if we had a type family

type family Derives (t :: k1) (c :: k2) :: Bool

which would automatically be instantiated by GHC appropriately? E.g., if the user had the following code:

data MyData = MyData deriving (Eq, Generic) deriving instance Show MyData instance Ord MyData

GHC would automatically instantiate:

type instance Derives MyData Eq = True type instance Derives MyData Generic = True type instance Derives MyData Show = True type instance Derives MyData Ord = False

Would this be something Ryan could use for detecting safe instances for LVish?

Cheers, Pedro

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Ah, I see. I had implicitly assumed we were talking about yet another "magic" class. This is snow balling in my head right now, so I'll hold off on further suggestions. My main concern was the relationship between: * an irreducible application of the Derives type family * an application that reduces to False * an insoluble constraint for the Derives class On Oct 15, 2013 7:28 AM, "José Pedro Magalhães" wrote:

On Tue, Oct 15, 2013 at 4:49 AM, Edward Kmett wrote:

...

The issue with such an explicit false is that it requires more magic on behalf of the compiler.

It would have to be filled in whenever an explicit `instance Eq Blah` was written.

Recall that

deriving instance Eq Blah

can occur after the data type declaration site and may have to be used for many more complicated recursive data types, so it isn't sufficient to fill in at the data type declaration.

Yes, I know. But I don't think this would require _more_ magic than the class instances. And I don't see how class instances solve the problem; how can the user be prevented from defining an |instance Derives MyData Eq| if the instance wasn't indeed derived? Only through compiler magic...

Pedro

...

-Edward

On Mon, Oct 14, 2013 at 11:41 PM, Ryan Newton wrote:

...

But what stops the user from defining their own instances if they in fact did not derive it?

The explicit "False" in Pedro's formulation seems to serve this purpose.

On Mon, Oct 14, 2013 at 11:20 PM, Nicolas Frisby < nicolas.frisby@gmail.com> wrote:

...

The formulation as a type family seems to conflict with the open-world principle. Would a class-based encoding instead be sufficient?

-- the proposed, special wired-in class class Derives (t :: k1) (c :: k2)

-- GHC would infer these from Pedro's example's declarations instance Derives MyData Eq instance Derives MyData Generic instance Derives MyData Show

NB that there is no instance Derives MyData Ord, but standalone deriving could yield one "later"

On Mon, Oct 14, 2013 at 10:02 PM, Ryan Newton wrote:

...

Hey, that's an awesome formulation! Thanks Pedro.

Any idea how much work this would be to implement in GHC, if it did garner approval?

On Tue, Oct 8, 2013 at 3:48 AM, José Pedro Magalhães < dreixel@gmail.com> wrote:

...

Hi,

On Mon, Oct 7, 2013 at 10:32 AM, Dag Odenhall wrote:

> Here‘s a thought: doesn’t Generic already have an unused phantom > type that's only there “just in case we need to add something in the > future”? > No, it doesn't.

Just a thought: what if we had a type family

type family Derives (t :: k1) (c :: k2) :: Bool

which would automatically be instantiated by GHC appropriately? E.g., if the user had the following code:

data MyData = MyData deriving (Eq, Generic) deriving instance Show MyData instance Ord MyData

GHC would automatically instantiate:

type instance Derives MyData Eq = True type instance Derives MyData Generic = True type instance Derives MyData Show = True type instance Derives MyData Ord = False

Would this be something Ryan could use for detecting safe instances for LVish?

Cheers, Pedro

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Hi, On 07/10/2013 04:14, Ryan Newton wrote:

On Sun, Oct 6, 2013 at 6:28 PM, Ganesh Sittampalam mailto:ganesh@earth.li> wrote:

- Referential transparency: e.g. no unsafePerformIO

- Module boundary control: no abstraction violation like Template Haskell and GeneralizedNewtypeDeriving - Semantic consistency: importing a safe module can't change existing code, so no OverlappingInstances and the like

Is this change necessary to preserve the existing properties, or are you hoping to add a new one?

I'm not currently aware of ways to break these invariants *just* with GHC.Generics. Hmm, but I would like to know why it is marked trustworthy and not inferred-safe...

What actually happens if you take off the annotation?

My argument is more that it is a violation in spirit to write bad Generic instances that in turn lead to bad fromRep and toRep conversions. (In a similar way to bad Typeable insteances leading to bad type conversions).

I think the problem with "in spirit" arguments is that different people will have different views on what should be safe or not. With a clearly defined set of properties that SafeHaskell does enforce - and by implication nothing else - everyone knows where they stands. Making LVish work seems like a a worthy goal, and I can see a general case for having guaranteed-correct Eq and Ord instances. I wonder if there's anything to be shared with Conal Elliott's unamb, which I think also has some unchecked safety requirements: http://conal.net/blog/posts/merging-partial-values

Second, the usage intent of GHC.Generics as far as I can see, is that users 99.9% of the time will be using -XDeriveGeneric.

I don't think that's entirely true; if you want to get data abstraction then you really would need to define your own instances. Forcing people to give up on their data abstraction actually goes against one of the properties of SafeHaskell above (in spirit :-).

I also understand that you want to require 'standard' Generic instances on types - will that mean that module authors are forced to expose internals to use your library?

Great point! My intent was to short-circuit that by providing "TrustWorthy" SafeEq and SafeOrd instances for standard types.

That doesn't help for arbitrary user-written types though, does it? Ganesh

On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett wrote:

Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself?

Exactly.

Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead?

The new rule would be that if you hand-implemented Generic or Generic1 in

your module it isn't Safe.

Ah... so you're proposing adding an extra rule to the GHC compiler itself? Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really want you can mark your module as TrustWorthy even with manual instances. (And then, yes, we'd need a GHC.Generics.Safe to pull the Generic symbol itself from when we just want to derive it.) That would make it so that derived Generic, Generic1 would be considered

Safe, and you wouldn't break literally every user of the library who care about Safe Haskell.

Isn't that also accomplished just by "import GHC.Generic.Safe (Generic)"? And the handful of hypothetical people that are using this right now can make that one-line fix. I will *personally* make that fix for all those people if they would come forward and share the link to their code ;-). As it stands the things are damn-near impossible to get right instantiating

them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them!

Exactly!

Thanks Edward! That provides a real example of someone who uses both SafeHaskell and GHC.Generics in their libraries! I'm glad to know such a person exists but that you don't depend on handwritten Generic instances. (I'm also glad to hear that you care about SafeHaskell in the first place!) I totally agree that #ifdef'ing on GHC version is a pain, and all that code would need to conditionally import GHC.Generics or GHC.Generics.Safe to retain -XSafe status. I'm perfectly happy with the GHC-internal fix if someone can point me to how to do it / where it should go. I guess I'll start by greping for Typeable in the compiler. Is this something you can do trivially Edward? (Or others?) Just for reference here is a small patch to Base representing my library-based way of making the change discussed in this thread: https://github.com/rrnewton/packages-base/commit/23f8f464a0b46ca17c655c4144f... Again, my use-case (closed/safe Eq & Ord instances) is a few steps removed from this Generics business and I don't feel strongly about how it gets done. But having *SOME* way of doing it will be a massive boon to my lab's particular effort at maximizing the abilities of guaranteed-Safe deterministic parallelism in Haskell. Further, I think anyone else who tries to do something in SafeHaskell that depends on guaranteeing a closed/controlled set of instances (SafeEq, SafeOrd but also others) based on GHC.Generics will *also* find this change useful. Best, -Ryan On Mon, Oct 7, 2013 at 12:44 PM, Edward Kmett wrote:

I'd much rather have code that can compile as Safe rather than Trustworthy. Trustworthy code is a pain to annotate the safety of correctly.

A one line change to GHC to ensure the safety of GHC.Generics and to maximize the number of operations you can use safely seems entirely reasonable.

Otherwise I have to mark perfectly Safe-by-construction code Trustworthy, enlarging the trusted code base considerably. Using properly constructed Generic instances is perfectly safe.

That library level fix has knock-on effects across the entire ecosystem for anyone who cares about SafeHaskell. I tend to instantiate GHC.Generics for any type in most of my packages. A quick grep finds 209 modules broken in just the projects I have open in my working directory. That is a mixture of public and private repos. The majority of which are public. So, I first have to change a couple hundred import statements... conditionally on compiler version, since I maintain wide support windows. This fix would literally break SafeHaskell for me silently across almost everything I have written, forcing me to run through tons of packages trying to figure out what I have to change to get it back. I've done this song and dance before trying to get lambdabot to build, since it uses lens these days. If I don't get every one of them right lens builds as unsafe. There isn't a good debugging process for this, as the only real way I have to know it worked is to go through each one and make sure I didn't compromise safe haskell by looking at the resulting haddocks for each case.

The code that *uses* GHC.Generics uses their internals and is safe by construction. It is only the instance construction that is unsafe, so lets mark it as such.

-Edward

On Mon, Oct 7, 2013 at 11:11 AM, Ryan Newton wrote:

...

On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett wrote:

...

Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself?

Exactly.

...

Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead?

The new rule would be that if you hand-implemented Generic or Generic1 in

...

your module it isn't Safe.

Ah... so you're proposing adding an extra rule to the GHC compiler itself?

Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really want you can mark your module as TrustWorthy even with manual instances. (And then, yes, we'd need a GHC.Generics.Safe to pull the Generic symbol itself from when we just want to derive it.)

That would make it so that derived Generic, Generic1 would be considered

...

Safe, and you wouldn't break literally every user of the library who care about Safe Haskell.

Isn't that also accomplished just by "import GHC.Generic.Safe (Generic)"? And the handful of hypothetical people that are using this right now can make that one-line fix. I will *personally* make that fix for all those people if they would come forward and share the link to their code ;-).

As it stands the things are damn-near impossible to get right

...

instantiating them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them!

Exactly!

On Mon, Oct 7, 2013 at 1:33 PM, Ryan Newton wrote:

Again, my use-case (closed/safe Eq & Ord instances) is a few steps removed from this Generics business and I don't feel strongly about how it gets done. But having *SOME* way of doing it will be a massive boon to my lab's particular effort at maximizing the abilities of guaranteed-Safe deterministic parallelism in Haskell. Further, I think anyone else who tries to do something in SafeHaskell that depends on guaranteeing a closed/controlled set of instances (SafeEq, SafeOrd but also others) based on GHC.Generics will *also* find this change useful.

Hi, In containers we have Data instances "This instance preserves data abstraction at the cost of inefficiency". Generic is a more modern way to address the same problem. This Generic instance for Set is just a more convenient way to use Set.toList/Set.fromList http://lpaste.net/94028, which are -XSafe functions right? For your problem I suggest using template-haskell to write your "SafeEq" "SafeOrd" instances. It will let you access unexported constructors too, which means people using LVish can maintain their abstractions for everybody except LVish. My understanding is that the proposed changes to ghc/Generic would mean that all data used with LVish would have to expose it's implementation to everybody through the derived Generic instance. Regards, Adam

My understanding is that the

proposed changes to ghc/Generic would mean that all data used with LVish would have to expose it's implementation to everybody through the derived Generic instance.

Either that or they would have use TrustWorthy to mark a module containing custom equality/ordering operations as safe according to our definition, and take the proof obligations that come with that. In fact, this is what we would plan to do to "opt in" ADTs like bytestring, vector, containers. We can trust as many people/libraries as we like, but we want to retain the ability to run untrusted code with the right set of flags to ensure that it can't break our library. (And that's what we can't do without locking down the Generic backdoor.)