#9220: type roles for unboxed arrays --------------------------------------+------------------------------------ Reporter: rwbarton | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries (other) | Version: 7.8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Unknown/Multiple Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: --------------------------------------+------------------------------------ Comment (by goldfire): This is an area (arrays in Haskell) I know little about. How does this request compare to #9163, asking for `Ptr` to have a phantom role? Perhaps the difference is that `Ptr` refers to foreign objects but arrays store proper Haskell values? Sorry -- just not terribly familiar over here. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: libraries | Version: 7.8.1 (other) | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by rwbarton): As I see it "representational" has to do with the representation of Haskell values as closures (usually on the heap). That's different from the representation of the "underlying value" in an unboxed or storable array. For example, if I needed to process packed arrays of 24-bit integers, I might create a newtype of Int and write a Storable instance for it with `sizeOf _ = 3`. (Admittedly this is a somewhat contrived scenario.) The same considerations also apply to `Ptr`. (Indeed, a storable array is little more than a `Ptr` under the hood.) But `Ptr` is an inherently unsafe/unmanaged sort of thing. If you want to know how many items' worth of space is allocated in an area pointed to by a `Ptr`, you have to keep track of it yourself. Unboxed/storable arrays are supposed to be a safe abstraction on top of raw memory areas. That's why I think it's reasonable to have different defaults between unboxed/storable arrays and `Ptr`s. (And it is in some sense a matter of defaults, since there is still `unsafeCoerce` when you happen to know that the memory layouts match up properly.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: libraries | Version: 7.8.1 (other) | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by simonpj): * owner: => goldfire Comment: I'm OK with `Ptr` being different. But if I have a `UArray (Int,Int) Age`, surely it's OK to cast it to `UArray (Int,Int) Int`, where {{{ newtype Age = MkAge Int }}} After all, `Int` and `Age` are represented identically, and it might be useful to make such a bulk conversion. Yes there might be different instances somewhere, but that's true of ''all'' uses of `coerce`. So I vote for the element parameter of these arrays being representational, rather than nominal. Unless someone has other arguments to make. Richard (or Joachim), might you do this, in due course? Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: libraries | Version: 7.8.1 (other) | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by carter): @Richard, in Reid's example, that would be for Storable rather than Unboxed arrays (at least if I'm understanding this thread correctly, and thus because Storables are just wrapped up pointers, it falls under the same onus as Ptr's rather than Unboxed arrays?) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: libraries | Version: 7.8.1 (other) | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by goldfire): Ah -- that makes good sense then. Thanks, rwbarton. I'll take care of this in my next round of GHC hacking, sometime after ICFP. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Just to be clear, what exactly is "the fix" you mention? * What roles are you proposing for arrays of various sorts? * What goes wrong with the above instance? Is it rejecting a program that is bogus, or is it rejecting a program that "ought" to be ok? Incidentally, I think it should ''definitely'' be part of the contract for `Storable` that `sizeOf` should return the same result for a newtype as for the type it wraps. Thus this would be BAD: {{{ newtype N = MkN T instance Storable T where sizeOf _ = 4 instance Storable N where sizeOf _ = 8 }}} If `N` and `T` are representationally equaly, how could they take a different number of bytes to represent. That should be part of the documentation for `Storable`. I believe this is what Reid was saying in comment:8. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Well I argued in comment:5 for a representational (not nominal!) role; and Reid agreed in comment:8. Specifically, which types are "all the types mentioned in the initial report"? What roles do other array types have? You didn't answer my second question in comment:12. Any thoughts on that? Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by goldfire): Replying to [comment:14 simonpj]:

Specifically, which types are "all the types mentioned in the initial report"? What roles do other array types have?

{{{ Data.Array.Unboxed.UArray Data.Array.IO.IOUArray Data.Array.ST.STUArray Data.Array.Storable.StorableArray }}} I don't know anything in particular about the roles of other array types. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Here's my summary * Plan A: * If an unboxed array of `Age` stores (unboxed) ages in a 16-bit field, but an unboxed array of `Int` stores (unboxed) ints in a 32-bit field, then it would be wrong to coerce from unboxed-array-of-`Age` to unboxed- array-of-`Int`. (The field width is described by `sizeOf` in the `Storable` class.) * This is not an unreasonable scenario. Perhaps you invented the newtype precisely because ages have a narrower range of values than ints. * In that case, unboxed arrays should have nominal role for their element type, and the GND example in comment:11 should rightfully fail. * Plan B: * An alternative is to insist that the `sizeOf` method should yield the same result for a newtype as for its underlying representation type. * Then a representational role for the element type would be justified. * Which would allow more expressive coercions; notably the GND in comment:11 I was originally voting for Plan B. But (a) it relies on an unenforced user convention about `Storable` instances, and (b) there are reasonable situations in which you might want a different width for a newtype. So on reflection I think Plan A is probably right. If we adopt it, we should add a digest of this thread (e.g. the above Plan A/B notes) as a Note with the role declaration. The core-libraries committee is already cc'd. Edward, we need a decision for 7.10. Thanks. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by ekmett): Simon, I have code in production where the contract for a newtype and the wrapper you want to have enforced here doesn't hold -- in particular cases where the newtype _solely exists_ to create a Storable with different alignment and sizing properties. This comes up for all sorts of things, packed colors, bit arrays, etc, structures folks are marshaling for std140 compliance to/from OpenGL uniform buffers... It seems dangerous to me to tie a contract about the behavior of a user defineable class to representability. If we're letting users define Storable instances is seems strange to me to conflate the two notions of representation (bit representation of Storable) and the heap representation for GHC. This puts me very much in the Plan A camp, personally. Plan B it seems could just lead to segfaults for existing user instances out there. With Ptr we're basically handing the user a live grenade, they know handling it wrong can lead to segfaults, and they do so with due care. With an Array I don't think there is any such expectation of unsafety, and folks are expecting things to 'just work'. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by ekmett): (Wrote the bulk of this before you replied, figured I'd send it anyways) Hrmm, actually one could argue a stronger statement, it seems to me that Plan B is inconsistent. How would you even construct a type for, say, endian swapping, or going to/from network order for a Word32 upon storage, you'd still want the underlying type to be a newtype, rather than just randomly throw in a bottom with an unnecessary 'data' for no reason, and I've seen both of these as instances in the wild as part of larger Storable types. Also merely having sizeOf agree isn't sufficient to ensure correctness. Consider the above case of order swapping on storage, sizeOf gives the same answer in the above case, and if you coerced a boxed array nothing changes, but if you coerced an unboxed array you'd get all the characters spontaneously byte reversing due to effectively going through a "reinterpret_cast<>". It strikes me that two very different things would be happening semantically there, so having UArray have a nominal element type, even though Array is able to (and should be) representational makes sense to me. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by ekmett): Boxed arrays are representational (just by the default roles) though both `Array` and `UArray` are currently ''also'' `representational` in the first argument. =/ That is dangerous for much the same reason as relying on the semantics of `Storable` not changing to a newtype, it is often the very point that the newtype would change such an instance. In fact `UArray` is currently even worse and gets inferred as {{{ type role UArray representational phantom -- ack! }}} because the underlying constructor doesn't mention the element type at all. We basically just ignored roles in the `array` package when shipping 7.8.x -- It strikes me that we should have {{{ type role Array nominal representational type role UArray nominal nominal }}} The first argument is formed nominal by depending upon the chosen `Ix` instance, and in the case of `UArray` the second argument is forced nominal by depending upon the chosen `Storable` instance. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by ekmett): Ok, lets go through Array with a fine toothed comb: {{{ type role Array nominal representational type role IOArray nominal representational }}} That lets users case the element type in the array. This is safe because e.g. `instance IArray Array e` is fully polymorphic in `e` with no constraints. The index has to be nominal as it is tied to the semantics of the internal bounds, the underlying array size, etc. Users can coerce boxed arrays to change the element types. {{{ type role UArray nominal nominal type role IOUArray nominal nominal type role StorableArray nominal nominal }}} The index is nominal for the same reason as above, the element type is nominal because we'd otherwise silently be re-interpreting the bit representation of the underlying storables in the latter case, and the interpretation of the array size, etc. changes based on the same sort of information in `UArray` and `IOUArray`. {{{ type role STArray nominal nominal representational type role STUArray nominal nominal nominal }}} The region parameter is nominal lest users easily subvert `ST` safety, otherwise same as `Array` and `UArray` respectively. It turns out there isn't a lot of representational material here, just the element type in `Array`, `STArray` and `IOArray`. Every other role in the entire `array` package is necessarily `nominal`. Does that give enough to proceed? That should be every type used by the package, though, I think the bulk comes from `GHC.Arr`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#9220: type roles for unboxed arrays -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: goldfire Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Core | Version: 7.8.1 Libraries | Keywords: Resolution: | Architecture: Unknown/Multiple Operating System: | Difficulty: Unknown Unknown/Multiple | Blocked By: Type of failure: | Related Tickets: None/Unknown | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by Richard Eisenberg ): In [changeset:"e394e74df5ca8081c0ffd147d34e788d290fb21a/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="e394e74df5ca8081c0ffd147d34e788d290fb21a" Fix #9220 by adding role annotations. This includes a submodule update for `array`. There is also an added test in libraries/array/tests/T9220. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9220#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler