September 2018 - Libraries

Deprecating fromIntegral
by Niklas Hambüchen 13 Aug '20

13 Aug '20

This is not a joke. I just found a bug in base (https://ghc.haskell.org/trac/ghc/ticket/14262) import System.IO hWaitForInput stdin 4294968296 This code should wait for 49.something days, but it waits for 1 second. It is caused by a quick use of `fromIntegral` to "convert" `Int -> CInt` (where CInt = Int32), causing an overflow. I argue that `fromIntegral` causes unnoticed data corruption without any warning, and thus are worse than partial functions. In this case, this data corruption occurs in the most fundamental code that we all use and have to rely upon (`base`). So I propose that we add a deprecation pragma to `fromIntegral` (however, keeping it forever, like Java does, as there's no benefit to removing it, we just want that people use safer functions over time), and instead provide a `safeFromIntegral` that can only widen ranges, and one or more `unsafeFromIntegral` or appropriately named functions that can error, wrap, return Maybe, or have whatever desired behaviour when the input value doesn't fit into the output type. For some inspiration, `foundation` provides ideas in that direction: http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-From.html http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-IntegralCon… It is a bit funny how we argue that Haskell is good for high assurance programming, and we build all sorts of fancy constructs to eliminate more run-time errors, when in fact we do worse than C in detecting errors in the most basic types a computer can use, such as converting between 64-bit and 32-bit integers. Let's fix that. It avoids real problems in real production systems that are difficult to debug when they happen (who writes a unit test checking that a timeout of 50 days still works? This is the kind of stuff where you have to rely on -- very basic -- types). I'd like to probe support for such a change before doing any work on it. Niklas

18 58

Deprecate First and Last in Data.Monoid
by Andrew Martin 13 Jun '20

13 Jun '20

I propose that the data types First and Last, provided by Data.Monoid, be marked deprecated in GHC 8.6 and removed in GHC 8.10. The Semigroup-Monoid Proposal (SMP) brought about changes that, I will argue, make these data types (1) unneeded and (2) confusing. Why are they no longer needed? Data.Semigroup provides two identically named data types (First and Last). These have Semigroup instances matches what their names suggest. Additionally, SMP corrects the Monoid instance for Maybe so that it now lifts Semigroup instances instead of Monoid instances. What this means is that, assuming the following imports: import qualified Data.Monoid as M import qualified Data.Semigroup as S We have the following equivalences: M.First a === Maybe (M.First a) M.Last a === Maybe (M.Last a) When I say "equivalence", I mean that their Monoid instances have the same behavior. Their Show/Read instances are certainly different, and I’m not sure about their Ord instances. Getting back to the second reason, removing these data types would eliminate confusion. It’s strange that Semigroup is a superclass of Monoid, yet they have Data types with conflicting names defined in their respective modules. When you see the data type First, you cannot tell what someone is talking additional context. In general, base strives to avoid duplicate identifiers. The only other exception to this rule I can think of is that Bifunctor and Arrow both use the identifiers first and second. But, the situation with Data.Semigroup and Data.Monoid is higher impact since people are more likely to import these modules unqualified together. (As an aside, and the confusion I’m talking about here isn’t entirely hypothetical. I got tripped up by this when I first started using the semigroups library. I had to train myself to stop importing Data.Monoid unqualified, because it kept stealing these identifiers) Again, the two reasons are that these are no longer necessary and that they cause confusion. I would appreciate any feedback, including simple nods of agreement or disagreement. Thanks. -- -Andrew Thaddeus Martin

7 8

Proposal: Rename HashMap.lookupDefault to HashMap.findWithDefault
by Matt Renaud 06 Jan '19

06 Jan '19

*What* *====* Rename unordered-container's Data.HashMap.lookupDefault <https://hackage.haskell.org/package/unordered-containers-0.2.8.0/docs/Data-…> to findWithDefault. findWithDefault :: (Eq k, Hashable k) => v -> k -> HashMap k v -> v Note: There are no functionality changes, this is purely a rename. *Why* *===* This aligns the Data.HashMap API with containers' Data.Map.findWithDefault <https://hackage.haskell.org/package/containers-0.5.11.0/docs/Data-Map-Stric…> . Data.Map.findWithDefault :: Ord k => a -> k -> Map k a -> a The map APIs provided by the two packages are *almost* drop in replacement compatible if you aren't using any of the implementation specific functions (like ordering based functions from Data.Map), this change brings us one step closer. API consistency reduces the cognitive overhead when learning a new package. Having to learn different function names for the same functionality depending on which "map" implementation you're using is a poor developer experience. We chose the containers' name findWithDefault over unordered-containers' lookupDefault for two reasons: 1. Existing lookupX functions returns a Maybe value, while findX functions return a non-wrapped value. 2. The containers package ships with GHC and is a "core" package. *Pros:* *-----* - Consistent API between different "map" implementations (Data.Map, Data.IntMap, Data.HashMap). This makes switching implementations an import change. - Naming matches other similar functions (lookupX return Maybe-wrapped values) *Cons:* *-----* - API change requires users to update their code + unordered-containers has A LOT of users: 358815 total (13325 in the last 30 days) *How* *===* https://github.com/tibbe/unordered-containers/pull/176/commits/152f8818ee13… *Code Changes:* *-------------* - Rename the function in Data.HashMap.Base (and expose it from Strict and Lazy modules) - Make lookupDefault an INLINE alias of findWithDefault - Add DEPRECATION notice to lookupDefault - Bump unordered-containers version to 0.2.9.0 *Migration - Option 1:* *---------------------* *- *Announce on Haskell communication channels (haskell-cafe@, haskell-community@, #haskell on Twitter, Reddit thread, etc.) - Users of unordered-containers >= 0.2.9.0 receive warning about deprecated function - Code can be updated by find and replace: s/lookupDefault/findWithDefault/ - lookupDefault with deprecation notice remains for 1 year (subject to change) - after 1 year the lookupDefault function is removed, unordered-containers version bumped to 0.3.0.0 (major version bump due to breaking change) *Migration - Option 2:* *---------------------* - Announce on Haskell communication channels (haskell-cafe@, haskell-community@, #haskell on Twitter, Reddit thread, etc.) - Users of unordered-containers >= 0.2.9.0 receive warning about deprecated function - Code can be updated by find and replace: s/lookupDefault/findWithDefault/ - lookupDefault function is never removed *Discussion: * *===========* I would like to get some comments on this proposal. In particular: - Is the small API churn worth the increase in consistency? - Should migration option 1 (completely remove the old function) or 2 (keep old function indefinitely) be taken? We can punt on this and go with option 2 to start and revisit later if desired. I hope a decision about the proposal can be reached by 2018-02-09. Thanks!

6 10

Proposal: Add HasCallStack to the partial functions in base
by Daniel Cartwright 28 Sep '18

28 Sep '18

I propose to add `HasCallStack` to all partial functions in base, e.g. `fromJust`, `fromRight`, `fromLeft`, etc. Relevant discussion here: https://ghc.haskell.org/trac/ghc/ticket/15559

4 4

Touching unlifted values
by Andrew Martin 21 Sep '18

21 Sep '18

The touch# primitive accepts a levity-polymorphic argument. I am wondering if there is ever any difference between using it on a lifted value and an unlifted value. Consider the following: module Lifted where import Control.Monad.ST (runST) import Control.Monad.Primitive (touch) import Data.Int (Int64) import Data.Primitive (newPinnedByteArray,mutableByteArrayContents,readOffAddr,writeOffAddr) computation :: Int64 computation = runST $ do arr <- newPinnedByteArray 8 let addr = mutableByteArrayContents arr writeOffAddr addr 0 (42 :: Int64) i <- readOffAddr addr 0 touch arr return i Calling touch on the mutable byte array is necessary to make sure that the memory that the Addr points doesn't get GCed while we are writing and reading to and from it. Here is the relevant GHC core (compiled with -O2): -- RHS size: {terms: 32, types: 67, coercions: 19, joins: 0/1} computation1 computation1 = \ s1_a24R -> case newPinnedByteArray# 8# (s1_a24R `cast` <Co:4>) of { (# ipv_a246, ipv1_a247 #) -> let { addr_s273 addr_s273 = byteArrayContents# (ipv1_a247 `cast` <Co:9>) } in case writeInt64OffAddr# addr_s273 0# 42# ipv_a246 of s'#_a24p { __DEFAULT -> case readInt64OffAddr# addr_s273 0# s'#_a24p of { (# ipv2_a267, ipv3_a268 #) -> case touch# ((MutableByteArray ipv1_a247) `cast` <Co:3>) (ipv2_a267 `cast` <Co:3>) of s'_a24K { __DEFAULT -> (# s'_a24K, I64# ipv3_a268 #) } } } } -- RHS size: {terms: 5, types: 30, coercions: 0, joins: 0/0} computation computation = case runRW# computation1 of { (# ipv_a239, ipv1_a23a #) -> ipv1_a23a } Instead, what if we touched the underlying unlifted MutableByteArray#? Here is the code for doing this: {-# language MagicHash #-} {-# language UnboxedTuples #-} module Unlifted ( computation ) where import System.IO.Unsafe (unsafeDupablePerformIO) import Control.Monad.Primitive (unsafePrimToPrim,primitive,PrimState,PrimMonad) import Data.Int (Int64) import Data.Primitive (MutableByteArray(..),newPinnedByteArray,mutableByteArrayContents,readOffAddr,writeOffAddr) import GHC.Exts (touch#,MutableByteArray#) computation :: Int64 computation = unsafeDupablePerformIO $ do arr@(MutableByteArray arr#) <- newPinnedByteArray 8 let addr = mutableByteArrayContents arr writeOffAddr addr 0 (42 :: Int64) i <- readOffAddr addr 0 touchUnlifted arr# return i touchUnlifted :: PrimMonad m => MutableByteArray# (PrimState m) -> m () touchUnlifted x = unsafePrimToPrim $ (primitive (\s -> case touch# x s of { s' -> (# s', () #) }) :: IO ()) GHC generates the following core for this module (again, omitting irrelevant parts): computation1 computation1 = \ s_a49h -> case newPinnedByteArray# 8# (s_a49h `cast` <Co:3>) of { (# ipv_a48a, ipv1_a48b #) -> let { addr_s4aY addr_s4aY = byteArrayContents# (ipv1_a48b `cast` <Co:8>) } in case writeInt64OffAddr# addr_s4aY 0# 42# ipv_a48a of s'#_a48A { __DEFAULT -> case readInt64OffAddr# addr_s4aY 0# s'#_a48A of { (# ipv2_a4aq, ipv3_a4ar #) -> case touch# ipv1_a48b (ipv2_a4aq `cast` <Co:2>) of s'_a2xn { __DEFAULT -> (# s'_a2xn, I64# ipv3_a4ar #) } } } } -- RHS size: {terms: 5, types: 30, coercions: 0, joins: 0/0} computation computation = case runRW# computation1 of { (# ipv_a47X, ipv1_a47Y #) -> ipv1_a47Y } I feel confident that both of these are semantically equivalent. Both uses of touch# should keep the MutableByteArray# alive until we are done using the pointer we extracted from it. What I'm less sure about is whether or not the first one actually does an alloctation for the MutableByteArray data constructor when it calls touch. Is this eliminated in some other stage of compilation? -- -Andrew Thaddeus Martin

2 1

Fwd: Generalize `E{0,1,2,3,6,9,12}` from `Data.Fixed`
by andrew.lelechenko＠gmail.com 20 Sep '18

20 Sep '18

(Alexandre asked me to forward the email below on his behalf, because he is experiencing technical difficulties with this mail list. — Andrew) Greetings CLC; I'm writing this email to propose a change to `Data.Fixed`. Full credit for this idea goes to Bhavik Mehta (@b-mehta on GitHub), who implemented it in this PR for `exact-pi`. In `Data.Fixed` there are several `E`-prefixed datatypes used to represent a certain number of digits of precision in fixed-precision arithmetic. For example, `E1` has 1 decimal place, `E12` has 12. Each of them, `E{0,1,2,3,6,9,12}` is hardcoded. If more precision types are to be provided, they have to be hardcoded as well, and all of these types resemble each other. I think there is room for improvement here. Instead of having ``` data E0 instance HasResolution E0 where resolution _ = 1 ``` and repeating it as many times as there are `E` datatypes, I propose to add the following type: ``` {-# LANGUAGE DataKinds #-} {-# LANGUAGE KindSignatures #-} import GHC.TypeLits (Nat, KnownNat, natVal) data E (n :: Nat) ``` and then do ``` instance KnownNat n => HasResolution (E n) where resolution _ = 10^natVal (undefined :: E n) ``` just once, replacing `data E0` with `type E0 = E 0` (and the same for the rest of them) to continue reexporting these types. `E` should also be exported. I have created a Trac feature request ticket with the same contents as this email, and made a PR to GHC’s repository on GitHub. To finalize, there are a few topics I’d like to raise regarding this change. 1. Does the community find this change beneficial in general? 2. Does the community approve of using DataKinds in a mundane section of the base package? 3. Does everyone accept a small breaking change of E0, E1, etc. from a data type to a type synonym? Or should we go the conservative way and just add E without refactoring E0, E1, etc.? Regards, Alexandre

4 3

Fwd: Add fixity for (==) and (/=)
by Dannyu NDos 18 Sep '18

18 Sep '18

---------- Forwarded message --------- From: Dannyu NDos <ndospark320(a)gmail.com> Date: 2018년 9월 17일 (월) 오후 6:18 Subject: Re: Add fixity for (==) and (/=) To: <david.feuer(a)gmail.com> Proof by truth table (F is False, T is True): p q r (p == q) (q == r) ((p == q) == r) (p == (q == r)) F F F T T F F F F T T F T T F T F F F T T F T T F T F F T F F F T T T T F T F F F F T T F T F F F T T T T T T T That proves associativity of (==). Also note that p /= q ≡ not p == q. Proof: p q (p /= q) (not p) (not p == q) F F F T F F T T T T T F T F T T T F F F And by symmetry of (/=), p /= q ≡ p == not q. Then: (p /= q) /= r ≡ (not p == q) == not r ≡ not p == (q == not r) ≡ p /= (q /= r). Hence (/=) is associative. Q.E.D.

5 7

Add fixity for (==) and (/=)
by Dannyu NDos 18 Sep '18

18 Sep '18

Given that they are instantized for `Bool`s, they are associative, so it seems approvable to give them a fixity. (Sidenote: For the monoid over (==) I suggested on last May, I stated that it determines if there is an even number of `True`s, but that's wrong. It determines if there is an odd number of `False`s.)

3 2

GC-Managed ByteArray Slicing
by Andrew Martin 09 Sep '18

09 Sep '18

In GHC Haskell, there are two common options for working with bytes in memory: data ByteArray = ByteArray ByteArray# data ByteSlice = ByteSlice ByteArray# Int# Int# The second one, ByteSlice, is roughly what the bytestring library does (but with a ForeignPtr instead of a ByteArray# inside of it). What's unfortunate is that it's difficult to know which of these two types is going to provide better performance for a given application. These heuristics are helpful for guiding the process: 1. The longer the data is going to be around, the better the unsliced variant will be. This point is about space efficiency. The sliced variant needs two extra machine words to hold those indices, plus it holds on to extra bytes from the array that it is slicing into. 2. The shorter the data is going to be around, the better the sliced variant is. This point is about speed of execution. The cost of the unsliced variant is typically a memcpy. If the data is getting GCed quickly anyway, the unsliced copy doesn't save you any space. 3. The shorter the data is, the better the unsliced variant is. 4. The more sharing of the data there is, the better the sliced variant is. But, users have to commit to which one they want to use in their data type definitions, and the data type may be used in different contexts where one or the other is preferable. For example, consider the following: data Student = Student { name :: {-# UNPACK #-} !BytesType , grade :: {-# UNPACK #-} !Int } Which of the two types should we use for BytesType? (For this example, ignore the incorrectness of using bytes to represent text. With either of the two bytes types, it's easy to put a newtype on top that communicates that what's inside is guaranteed to be UTF-8 encoded text). It depends on the usage. For example, consider that we are parsing students from this xml file: <students> <student> <name>Erica Chang</name> <grade>87</grade> </student> <student> <name>Lizbeth Cruz</name> <grade>91</grade> </student> ... </students> Let's say we've got about 1GB of this, and we run a parser on 64KB chunks as they are read from disk. There are two contasting scenarios to consider. If the file is parsed at startup and the resulting array of students is kept in memory for a long time afterward, then the unsliced variant is better. It allows us to throw away all of the bytes corresponding to the xml nodes, which are no longer relevant. The sliced variant, by constrast, would keep the entire contents of the original file in memory. However, if the file is parsed by a SAX-style streaming parser, the students being parsed, processed, and then discarded as they encountered, the sliced variant is better. Slicing wins here because the chunks from the file are discarded shortly after they are encountered. Read "shortly" as "before the next minor GC". In this example, Student was an application-specific data type, so the application author could figure out what they were actually doing with the data and define the data type accordingly. However, this isn't always a solution. What if they needed to use the same type differently in different contexts? What if the type was provided by a library and the library author doesn't know how it will be used? So far, this has just been an explanation of a problem that I've noticed. I understand the problem well, but now I'm going to propose a solution. There are a lot of holes in my understanding of GHC's runtime, so I don't understand if what I'm proposing is actually plausible. What if the GHC's runtime handled the copy decisions for us? What if we had these primitives: data Bytes# sliceBytes# :: Bytes# -> Int# -> Int# -> Bytes# lengthBytes# :: Bytes# -> Int# Slicing causes the runtime to track that there is an additional reference to the byte array but that the offset and length are different. I'm not sure where this information would be stored though. When GC runs, it would copy slices from bytes if the there were no longer any references to the entire array. So a single type could serve the both purposes discussed in the scenario above. I'm not sure what kind of impact this might have on GC time. -- -Andrew Thaddeus Martin

2 1

Proposal: Add mapAccumLM and mapAccumRM to Data.Traversable
by Reed Mullanix 03 Sep '18

03 Sep '18

I propose adding a pair of functions to Data.Traversable: mapAccumLM and mapAccumRM with the type '(Traversable t, Monad m) => (a -> b -> m (a,c)) -> a -> t b -> m (a, t c)'. These would behave exactly the same as mapAccumL and mapAccumR, but would allow the addition of monadic effects. This would allow the traversal of structures with an accumulator, without resorting to using foldlM or foldrM, both of which require the extra boilerplate of reconstructing the structure after applying the action, which can be somewhat frustrating and error-prone. A possible implementation would be to add transformer counterparts to StateL/StateR in Data.Functor.Util: (gist: https://gist.github.com/TOTBWF/dc6020be28df7b00372ab8e507aa54b7) newtype StateLT s m a = StateLT { runStateLT :: s -> m (s,a) } instance (Functor m) => Functor (StateLT s m) where fmap f (StateLT k) = StateLT $ \s -> fmap (\(s',a) -> (s', f a)) $ k s instance Monad m => Applicative (StateLT s m) where pure a = StateLT $ \s -> return (s, a) StateLT kf <*> StateLT kv = StateLT $ \s -> do (s', f) <- kf s (s'', v) <- kv s' return (s'', f v) liftA2 f (StateLT kx) (StateLT ky) = StateLT $ \s -> do (s', x) <- kx s (s'', y) <- ky s' return (s'', f x y) mapAccumLM :: (Monad m, Traversable t) => (a -> b -> m (a,c)) -> a -> t b -> m (a, t c) mapAccumLM f s t = runStateLT (traverse (StateLT . flip f) t) s newtype StateRT s m a = StateRT { runStateRT :: s -> m (s,a) } type StateR s = StateRT s Identity instance (Functor m) => Functor (StateRT s m) where fmap f (StateRT k) = StateRT $ \s -> fmap (\(s',a) -> (s', f a)) $ k s instance Monad m => Applicative (StateRT s m) where pure a = StateRT $ \s -> return (s, a) StateRT kf <*> StateRT kv = StateRT $ \s -> do (s', v) <- kv s (s'', f) <- kf s' return (s'', f v) liftA2 f (StateRT kx) (StateRT ky) = StateRT $ \s -> do (s', y) <- ky s (s'', x) <- kx s' return (s'', f x y) mapAccumRM :: (Monad m, Traversable t) => (a -> b -> m (a,c)) -> a -> t b -> m (a, t c) mapAccumRM f s t = runStateRT (traverse (StateRT . flip f) t) s

2 1