Removing the .Safe modules from vector (was: Haskell Platform proposal: Add the vector package)

Hi all, After reading the Safe Haskell paper today, I got the impression that no one actually wants the .Safe modules currently in vector. If vector was to be made Safe Haskell friendly, we should instead add .Unsafe modules (and have the rest of the modules declared Trustworthy). Having .Unsafe modules is better than having .Safe modules, because * there are many more safe functions than unsafe functions, and * Haskell is by default safe, so having modules called .Safe is a bit like having modules called .Pure. There's precedence for having .Unsafe modules in e.g. bytestring. If that's the case, and if Roman agrees, I suggest we release a new major version that * removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location. I suggest that the deprecation doesn't involve an actual deprecation pragma in this release [2], but instead just a comment. A future major release could add the deprecation pragma and another major release after that could remove the actual functions. Actually removing functions causes huge Hackage churn so I suggest that we don't do that until the new .Unsafe modules have been around for a long time. 1. Normally I would suggest a deprecation period before removing functions from an API, but I just searched through all of Hackage and there's only a single package (bitvec) that makes use of the .Safe API 2. While deprecation pragmas sound good in theory there isn't much library authors can do to make the warning go away. An author can migrate to the new API, but in practice authors need to support a couple of releases of each library. Cheers, Johan

+1
On Wed, Aug 29, 2012 at 9:33 PM, Johan Tibell
Hi all,
After reading the Safe Haskell paper today, I got the impression that no one actually wants the .Safe modules currently in vector. If vector was to be made Safe Haskell friendly, we should instead add .Unsafe modules (and have the rest of the modules declared Trustworthy). Having .Unsafe modules is better than having .Safe modules, because
* there are many more safe functions than unsafe functions, and * Haskell is by default safe, so having modules called .Safe is a bit like having modules called .Pure. There's precedence for having .Unsafe modules in e.g. bytestring.
If that's the case, and if Roman agrees, I suggest we release a new major version that
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
I suggest that the deprecation doesn't involve an actual deprecation pragma in this release [2], but instead just a comment. A future major release could add the deprecation pragma and another major release after that could remove the actual functions.
Actually removing functions causes huge Hackage churn so I suggest that we don't do that until the new .Unsafe modules have been around for a long time.
1. Normally I would suggest a deprecation period before removing functions from an API, but I just searched through all of Hackage and there's only a single package (bitvec) that makes use of the .Safe API
2. While deprecation pragmas sound good in theory there isn't much library authors can do to make the warning go away. An author can migrate to the new API, but in practice authors need to support a couple of releases of each library.
Cheers, Johan
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

+1 Cheers, Edward Excerpts from Johan Tibell's message of Wed Aug 29 21:33:44 -0400 2012:
Hi all,
After reading the Safe Haskell paper today, I got the impression that no one actually wants the .Safe modules currently in vector. If vector was to be made Safe Haskell friendly, we should instead add .Unsafe modules (and have the rest of the modules declared Trustworthy). Having .Unsafe modules is better than having .Safe modules, because
* there are many more safe functions than unsafe functions, and * Haskell is by default safe, so having modules called .Safe is a bit like having modules called .Pure. There's precedence for having .Unsafe modules in e.g. bytestring.
If that's the case, and if Roman agrees, I suggest we release a new major version that
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
I suggest that the deprecation doesn't involve an actual deprecation pragma in this release [2], but instead just a comment. A future major release could add the deprecation pragma and another major release after that could remove the actual functions.
Actually removing functions causes huge Hackage churn so I suggest that we don't do that until the new .Unsafe modules have been around for a long time.
1. Normally I would suggest a deprecation period before removing functions from an API, but I just searched through all of Hackage and there's only a single package (bitvec) that makes use of the .Safe API
2. While deprecation pragmas sound good in theory there isn't much library authors can do to make the warning go away. An author can migrate to the new API, but in practice authors need to support a couple of releases of each library.
Cheers, Johan

On Wed, 29 Aug 2012, Johan Tibell wrote:
Hi all,
After reading the Safe Haskell paper today, I got the impression that no one actually wants the .Safe modules currently in vector. If vector was to be made Safe Haskell friendly, we should instead add .Unsafe modules (and have the rest of the modules declared Trustworthy). Having .Unsafe modules is better than having .Safe modules, because
* there are many more safe functions than unsafe functions, and * Haskell is by default safe, so having modules called .Safe is a bit like having modules called .Pure. There's precedence for having .Unsafe modules in e.g. bytestring.
If that's the case, and if Roman agrees, I suggest we release a new major version that
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
+1

Hi,
After reading the Safe Haskell paper today, I got the impression that no one actually wants the .Safe modules currently in vector. If vector was to be made Safe Haskell friendly, we should instead add .Unsafe modules (and have the rest of the modules declared Trustworthy). Having .Unsafe modules is better than having .Safe modules, because
* there are many more safe functions than unsafe functions, and * Haskell is by default safe, so having modules called .Safe is a bit like having modules called .Pure. There's precedence for having .Unsafe modules in e.g. bytestring.
If that's the case, and if Roman agrees, I suggest we release a new major version that
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
+1 from me. Milan

Johan Tibell wrote:
If that's the case, and if Roman agrees, I suggest we release a new major version that
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
I suggest that the deprecation doesn't involve an actual deprecation pragma in this release [2], but instead just a comment. A future major release could add the deprecation pragma and another major release after that could remove the actual functions.
As I said earlier, I will definitely remove the .Safe modules. I'm not particularly keen on adding .Unsafe modules but in the scheme you're proposing, they'll just reexport a few functions and are (hopefully) easy to generate automatically so if there really is demand for it, I'll add them. Although I'd like to point out that with these scheme, vector won't have any SafeHaskell-safe modules so I'm not entirely sure what the point is, given that the entire discussion was because people objected to removing support for SafeHaskell. A caveat: I will only add the .Unsafe modules if they are generated automatically. I *hope* that's easy to do but can't make any promises as to how long it will take at this point. If someone else wants to have a go at automatic generation, I'll gladly accept patches. However, I'm not sure how to add the deprecation comments. I'll end up with this: module Foo (unsafeFoo) where unsafeFoo :: Very Long Type unsafeFoo = ... module Foo.Unsafe(unsafeFoo) where import Foo How do I make Haddock say that unsafeFoo is deprecated in the docs for Foo but not in the docs for Foo.Unsafe? Or is this impossible and will I have to do this? module Foo (unsafeFoo) where -- | (Deprecated) Very long comment unsafeFoo :: Very Long Type unsafeFoo = ... module Foo.Unsafe(unsafeFoo) where import qualified Foo -- | Very long comment unsafeFoo :: Very Long Type {-# INLINE unsafeFoo #-} unsafeFoo = Foo.unsafeFoo Last question: do the +1 comments mean "yes, sounds good" or "yes, this is something I will use"? Roman

On Thu, Aug 30, 2012 at 3:09 AM, Roman Leshchinskiy
As I said earlier, I will definitely remove the .Safe modules. I'm not particularly keen on adding .Unsafe modules but in the scheme you're proposing, they'll just reexport a few functions and are (hopefully) easy to generate automatically so if there really is demand for it, I'll add them. Although I'd like to point out that with these scheme, vector won't have any SafeHaskell-safe modules so I'm not entirely sure what the point is, given that the entire discussion was because people objected to removing support for SafeHaskell.
My main interest is having the .Safe modules removed. I wanted to present a compromise that'd allow people that care about Safe Haskell to still use vector. I'm hoping that having .Unsafe modules will be less work than having .Safe modules. My intention was to move from: * M - Unsafe * M.Safe - Trustworthy to: * M - Trustworthy * M.Unsafe - Unsafe I just had a look at Data.Vector and the haddock docs claim that it's Safe-inferred. So I guess I don't understand the current setup even. Why aren't the M modules Unsafe, given that they use IO in their implementation?
A caveat: I will only add the .Unsafe modules if they are generated automatically. I *hope* that's easy to do but can't make any promises as to how long it will take at this point. If someone else wants to have a go at automatic generation, I'll gladly accept patches.
Fair enough. Could you remove the .Safe modules before that?
Or is this impossible and will I have to do this?
module Foo (unsafeFoo) where
-- | (Deprecated) Very long comment unsafeFoo :: Very Long Type unsafeFoo = ...
module Foo.Unsafe(unsafeFoo) where import qualified Foo
-- | Very long comment unsafeFoo :: Very Long Type {-# INLINE unsafeFoo #-} unsafeFoo = Foo.unsafeFoo
I'm afraid so.
Last question: do the +1 comments mean "yes, sounds good" or "yes, this is something I will use"?
For me it is "removed the .Safe modules please". -- Johan

On 8/30/12 10:16 AM, Johan Tibell wrote:
On Thu, Aug 30, 2012 at 3:09 AM, Roman Leshchinskiy
wrote: As I said earlier, I will definitely remove the .Safe modules. I'm not particularly keen on adding .Unsafe modules but in the scheme you're proposing, they'll just reexport a few functions and are (hopefully) easy to generate automatically so if there really is demand for it, I'll add them. Although I'd like to point out that with these scheme, vector won't have any SafeHaskell-safe modules so I'm not entirely sure what the point is, given that the entire discussion was because people objected to removing support for SafeHaskell.
My main interest is having the .Safe modules removed.
I'm +1 to this part. As you say, having .Safe modules is like having .Pure modules: utter foolishness for Haskell. I'm all for momentum towards Safe Haskell. But I'd really rather not revisit our previous discussion on that matter. I'd say we should: (1) add vector to the Platform, sans .Safe modules (2) move the less objective/objectionable packages towards Safe Haskell, in order to instill that as a virtue and to get some empirical data about how users interact with Safe Haskell (3) revisit the Safe Haskell + vector issues once Safe Haskell is more popular and we have some better ideas what the practical arguments are (as opposed to just the theoretical ones). -- Live well, ~wren

On Thu, Aug 30, 2012 at 6:04 PM, wren ng thornton
I'm all for momentum towards Safe Haskell. But I'd really rather not revisit our previous discussion on that matter. I'd say we should:
(1) add vector to the Platform, sans .Safe modules (2) move the less objective/objectionable packages towards Safe Haskell, in order to instill that as a virtue and to get some empirical data about how users interact with Safe Haskell (3) revisit the Safe Haskell + vector issues once Safe Haskell is more popular and we have some better ideas what the practical arguments are (as opposed to just the theoretical ones).
Agreed. Let's get vector into the Platform as if Safe Haskell never existed, and revisit it later if it becomes something that we need.

On 30/08/2012 15:16, Johan Tibell wrote:
On Thu, Aug 30, 2012 at 3:09 AM, Roman Leshchinskiy
wrote: As I said earlier, I will definitely remove the .Safe modules. I'm not particularly keen on adding .Unsafe modules but in the scheme you're proposing, they'll just reexport a few functions and are (hopefully) easy to generate automatically so if there really is demand for it, I'll add them. Although I'd like to point out that with these scheme, vector won't have any SafeHaskell-safe modules so I'm not entirely sure what the point is, given that the entire discussion was because people objected to removing support for SafeHaskell.
My main interest is having the .Safe modules removed. I wanted to present a compromise that'd allow people that care about Safe Haskell to still use vector. I'm hoping that having .Unsafe modules will be less work than having .Safe modules.
My intention was to move from:
* M - Unsafe * M.Safe - Trustworthy
to:
* M - Trustworthy * M.Unsafe - Unsafe
I just had a look at Data.Vector and the haddock docs claim that it's Safe-inferred.
I think that's a bug in Haddock that has since been fixed. Whoever generated that documentation needs to update to a newer version of Haddock. Cheers, Simon

On 30/08/2012 02:33, Johan Tibell wrote:
Hi all,
After reading the Safe Haskell paper today, I got the impression that no one actually wants the .Safe modules currently in vector. If vector was to be made Safe Haskell friendly, we should instead add .Unsafe modules (and have the rest of the modules declared Trustworthy). Having .Unsafe modules is better than having .Safe modules, because
* there are many more safe functions than unsafe functions, and * Haskell is by default safe, so having modules called .Safe is a bit like having modules called .Pure. There's precedence for having .Unsafe modules in e.g. bytestring.
If that's the case, and if Roman agrees, I suggest we release a new major version that
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
+1 from me, although when we discussed doing this previously Roman was not keen.
I suggest that the deprecation doesn't involve an actual deprecation pragma in this release [2], but instead just a comment. A future major release could add the deprecation pragma and another major release after that could remove the actual functions.
Why not a deprecation pragma initially? That's like adding another level of "soft deprecation"; how long before we also need "extra-soft deprecation" and "ultra-soft deprecation"? (I feel a bog-roll analogy coming on :-)
1. Normally I would suggest a deprecation period before removing functions from an API, but I just searched through all of Hackage and there's only a single package (bitvec) that makes use of the .Safe API
We also have: Data.Array.IO.Safe Control.Monad.ST.Safe Foreign.Marshal.Safe and I think all of these are in the same state: the unsafe APIs in the non-Safe modules are currently deprecated, and the .Safe modules can be removed at some time in the future. Cheers, Simon

On Fri, Aug 31, 2012 at 5:29 AM, Simon Marlow
Why not a deprecation pragma initially? That's like adding another level of "soft deprecation"; how long before we also need "extra-soft deprecation" and "ultra-soft deprecation"? (I feel a bog-roll analogy coming on :-)
Because libraries need to support not only the last release of vector but also a few versions back. The reason is the following: If * Package P1 depends on package P2 <= A.B.C and P3 <= X.Y.Z. * Package P2 depends on package P3 <= X.Y.Z. * A new major release of package P3, X.Y.Z+1, that moves some functions to a new module and leaves behind deprecated re-exports, is released. * A new release of P2, A.B.C+1, that adds some new functions (unrelated to the changes in P3) is made. This release uses the new P3 API in order to get rid of deprecation messages, and thus have to depend on P3 == X.Y.Z+1. * P1 wants to use the new P2 API and thus depends on P2 == A.B.C+1. * P1 no longer builds due to its dependency chain contains incompatible constraints on P3. We've seen this on Hackage in the past. Introducing lower bounds on the latest release of a package creates a "but" through Hackage between packages that has an upper bound that excludes the new release and packages that have a lower bound that includes it. Breakage ensues. This last happened with the deepseq package I think. Authors can work around the problem with CPP, but it's ugly and annoying. My preference is to have a soft deprecation (i.e. comments) of an API, wait for a period of time, add pragmas to catch the last non-updaters, make the braking change. -- Johan

On 30/08/12 03:33, Johan Tibell wrote:
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
Is there a particular reason for first deprecating the functions exported from Data.Vector, instead of just removing them from their original location? By removing the Data.Vector.Safe module you already break some existing code, with the easy fix of moving to import Data.Vector. Moving the the unsafe functions to Data.Vector.Unsafe would introduce similar breakage, with a similarly easy fix of adding an import declaration. Twan

On Fri, Aug 31, 2012 at 6:05 AM, Twan van Laarhoven
On 30/08/12 03:33, Johan Tibell wrote:
* removes all the .Safe modules [1], * adds new .Unsafe modules, and * marks the functions that are now exported through the .Unsafe modules deprecated in their original (non-.Unsafe) location.
Is there a particular reason for first deprecating the functions exported from Data.Vector, instead of just removing them from their original location? By removing the Data.Vector.Safe module you already break some existing code, with the easy fix of moving to import Data.Vector. Moving the the unsafe functions to Data.Vector.Unsafe would introduce similar breakage, with a similarly easy fix of adding an import declaration.
See my answer to Simon about deprecation. The difference between removing the .Safe modules and moving to .Unsafe is that no one uses the Safe API but plenty of people use the unsafe* functions. -- Johan
participants (11)
-
Ben Millwood
-
Edward Kmett
-
Edward Z. Yang
-
Henning Thielemann
-
Johan Tibell
-
Mark Lentczner
-
Milan Straka
-
Roman Leshchinskiy
-
Simon Marlow
-
Twan van Laarhoven
-
wren ng thornton