July 2018 - Libraries

Deprecating fromIntegral
by Niklas Hambüchen 12 Aug '20

12 Aug '20

This is not a joke. I just found a bug in base (https://ghc.haskell.org/trac/ghc/ticket/14262) import System.IO hWaitForInput stdin 4294968296 This code should wait for 49.something days, but it waits for 1 second. It is caused by a quick use of `fromIntegral` to "convert" `Int -> CInt` (where CInt = Int32), causing an overflow. I argue that `fromIntegral` causes unnoticed data corruption without any warning, and thus are worse than partial functions. In this case, this data corruption occurs in the most fundamental code that we all use and have to rely upon (`base`). So I propose that we add a deprecation pragma to `fromIntegral` (however, keeping it forever, like Java does, as there's no benefit to removing it, we just want that people use safer functions over time), and instead provide a `safeFromIntegral` that can only widen ranges, and one or more `unsafeFromIntegral` or appropriately named functions that can error, wrap, return Maybe, or have whatever desired behaviour when the input value doesn't fit into the output type. For some inspiration, `foundation` provides ideas in that direction: http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-From.html http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-IntegralCon… It is a bit funny how we argue that Haskell is good for high assurance programming, and we build all sorts of fancy constructs to eliminate more run-time errors, when in fact we do worse than C in detecting errors in the most basic types a computer can use, such as converting between 64-bit and 32-bit integers. Let's fix that. It avoids real problems in real production systems that are difficult to debug when they happen (who writes a unit test checking that a timeout of 50 days still works? This is the kind of stuff where you have to rely on -- very basic -- types). I'd like to probe support for such a change before doing any work on it. Niklas

18 58

Deprecate First and Last in Data.Monoid
by Andrew Martin 13 Jun '20

13 Jun '20

I propose that the data types First and Last, provided by Data.Monoid, be marked deprecated in GHC 8.6 and removed in GHC 8.10. The Semigroup-Monoid Proposal (SMP) brought about changes that, I will argue, make these data types (1) unneeded and (2) confusing. Why are they no longer needed? Data.Semigroup provides two identically named data types (First and Last). These have Semigroup instances matches what their names suggest. Additionally, SMP corrects the Monoid instance for Maybe so that it now lifts Semigroup instances instead of Monoid instances. What this means is that, assuming the following imports: import qualified Data.Monoid as M import qualified Data.Semigroup as S We have the following equivalences: M.First a === Maybe (M.First a) M.Last a === Maybe (M.Last a) When I say "equivalence", I mean that their Monoid instances have the same behavior. Their Show/Read instances are certainly different, and I’m not sure about their Ord instances. Getting back to the second reason, removing these data types would eliminate confusion. It’s strange that Semigroup is a superclass of Monoid, yet they have Data types with conflicting names defined in their respective modules. When you see the data type First, you cannot tell what someone is talking additional context. In general, base strives to avoid duplicate identifiers. The only other exception to this rule I can think of is that Bifunctor and Arrow both use the identifiers first and second. But, the situation with Data.Semigroup and Data.Monoid is higher impact since people are more likely to import these modules unqualified together. (As an aside, and the confusion I’m talking about here isn’t entirely hypothetical. I got tripped up by this when I first started using the semigroups library. I had to train myself to stop importing Data.Monoid unqualified, because it kept stealing these identifiers) Again, the two reasons are that these are no longer necessary and that they cause confusion. I would appreciate any feedback, including simple nods of agreement or disagreement. Thanks. -- -Andrew Thaddeus Martin

7 8

Proposal: Rename HashMap.lookupDefault to HashMap.findWithDefault
by Matt Renaud 05 Jan '19

05 Jan '19

*What* *====* Rename unordered-container's Data.HashMap.lookupDefault <https://hackage.haskell.org/package/unordered-containers-0.2.8.0/docs/Data-…> to findWithDefault. findWithDefault :: (Eq k, Hashable k) => v -> k -> HashMap k v -> v Note: There are no functionality changes, this is purely a rename. *Why* *===* This aligns the Data.HashMap API with containers' Data.Map.findWithDefault <https://hackage.haskell.org/package/containers-0.5.11.0/docs/Data-Map-Stric…> . Data.Map.findWithDefault :: Ord k => a -> k -> Map k a -> a The map APIs provided by the two packages are *almost* drop in replacement compatible if you aren't using any of the implementation specific functions (like ordering based functions from Data.Map), this change brings us one step closer. API consistency reduces the cognitive overhead when learning a new package. Having to learn different function names for the same functionality depending on which "map" implementation you're using is a poor developer experience. We chose the containers' name findWithDefault over unordered-containers' lookupDefault for two reasons: 1. Existing lookupX functions returns a Maybe value, while findX functions return a non-wrapped value. 2. The containers package ships with GHC and is a "core" package. *Pros:* *-----* - Consistent API between different "map" implementations (Data.Map, Data.IntMap, Data.HashMap). This makes switching implementations an import change. - Naming matches other similar functions (lookupX return Maybe-wrapped values) *Cons:* *-----* - API change requires users to update their code + unordered-containers has A LOT of users: 358815 total (13325 in the last 30 days) *How* *===* https://github.com/tibbe/unordered-containers/pull/176/commits/152f8818ee13… *Code Changes:* *-------------* - Rename the function in Data.HashMap.Base (and expose it from Strict and Lazy modules) - Make lookupDefault an INLINE alias of findWithDefault - Add DEPRECATION notice to lookupDefault - Bump unordered-containers version to 0.2.9.0 *Migration - Option 1:* *---------------------* *- *Announce on Haskell communication channels (haskell-cafe@, haskell-community@, #haskell on Twitter, Reddit thread, etc.) - Users of unordered-containers >= 0.2.9.0 receive warning about deprecated function - Code can be updated by find and replace: s/lookupDefault/findWithDefault/ - lookupDefault with deprecation notice remains for 1 year (subject to change) - after 1 year the lookupDefault function is removed, unordered-containers version bumped to 0.3.0.0 (major version bump due to breaking change) *Migration - Option 2:* *---------------------* - Announce on Haskell communication channels (haskell-cafe@, haskell-community@, #haskell on Twitter, Reddit thread, etc.) - Users of unordered-containers >= 0.2.9.0 receive warning about deprecated function - Code can be updated by find and replace: s/lookupDefault/findWithDefault/ - lookupDefault function is never removed *Discussion: * *===========* I would like to get some comments on this proposal. In particular: - Is the small API churn worth the increase in consistency? - Should migration option 1 (completely remove the old function) or 2 (keep old function indefinitely) be taken? We can punt on this and go with option 2 to start and revisit later if desired. I hope a decision about the proposal can be reached by 2018-02-09. Thanks!

6 10

Extensions to the module name system in H2020
by Marco Zocca 31 Jul '18

31 Jul '18

Hi all, I was wondering if there are plans to extend/revisit/tidy up the module name system (https://wiki.haskell.org/Hierarchical_module_names) in view of Haskell 2020. I'm mostly concerned with scientific/numerical applications, where I find the current state of things to be a bit chaotic (see Numeric/Numerical/Optimisation/Optimization etc.). I would be glad to help out, and gather intelligence from the community as well via e.g. a poll. Best, Marco (github.com/ocramz)

6 12

Map-to-integer for ciphers?
by 박신환 29 Jul '18

29 Jul '18

For use of ciphers (SHA-256, RSA-2048, etc.), a type must be able to be injectively mapped to integers. It seems `Enum` is currently the closest thing that does this. But `Enum` is supposed to be for arithmetic sequences, so it seems better to define a new typeclass. (Here named `Cipherable`) There are some types that aren't members of `Enum`. For example, `Maybe`, `[]`, etc. They seem okay to be Cipherable. Hence: {-# LANGUAGE ScopedTypeVariables #-} instance Cipherable a => Cipherable (Maybe a) where toEnum 0 = Nothing toEnum n = Just (toEnum n) fromEnum Nothing = 0 fromEnum (Just x) = 1 + fromEnum x instance forall a. (Cipherable a, Bounded a) => Cipherable [a] where toEnum 0 = [] toEnum n = let (q,r) = (n-1) `quotRem` (1 + fromEnum (maxBound :: a)) in toEnum r : toEnum q fromEnum [] = 0 fromEnum (x:xs) = 1 + fromEnum x + (1 + fromEnum (maxBound :: a)) * fromEnum xs instance Cipherable Void where toEnum = errorWithoutStackTrace "Cipher.Cipherable.Void.toEnum" fromEnum = absurd (Besides, it is possible to re-write that of `[]` without ScopedTypeVariables? I see no way...)

2 3

atomicModifyIORef' strictness
by David Feuer 13 Jul '18

13 Jul '18

In the process of replacing atomicModifyMutVar#, I noticed that atomicModifyIORef' is just a little bit lazy: atomicModifyIORef' ref (\x -> (x + 1, undefined)) will increment the reference and then throw an exception in the calling thread. We could sometimes avoid creating thunks by changing this, so the above would install undefined in the IORef as well. For example, suppose I write atomicModifyIORef' ref (\x -> (3, x + 5)) This will end up producing a thunk for the second component. That could be avoided using something like atomicModifyIORef' ref (\ !x -> (3, x + 5)) but it seems a little surprising to have to do that in a function that focuses on being strict. I don't really care too much one way or the other, but I figured I should point this out and see what you all think.

1 0

Re: Discussion: New atomic IORef functions
by David Feuer 12 Jul '18

12 Jul '18

On Wed, Jul 11, 2018, 11:32 PM winter <drkoster(a)qq.com> wrote: > OK then, here's some subtle things i want to shout out if you're going to > make the change ; ) > > 1. The proposed naming is just not great, how about these: > > atomicExchangeIORef :: IORef a -> (a -> (a, b)) -> IO (a, (a, b)) > atomicExchangeIORef' :: IORef a -> (a -> (a, b)) -> IO (a, (a, b)) > > atomicApplyIORef :: IORef a -> (a -> a) -> IO (a, a) > atomicApplyIORef :: IORef a -> (a -> a) -> IO (a, a) > > It may not that great but it's more informative IMO. > I agree that my names aren't great, but I think yours are worse. > 3. atomicExchangeIORef should not force the tuple, e.g. don't pattern > match on the tuple result, instead let users choose to force or not. (May > worth adding some document, I'd be happy to help) > I generally disagree. Being lazy in the tuple is pretty much an invitation to memory leaks. I think the lazy version should ideally also be available (it's what the new primop will provide, after all), but I don't think it should be the obvious thing for people to reach for. > 4. atomicExchangeIORef' will not only force the tuple, but also force both > `a` and `b` part to match atomicModifyIORef' 's semantics. > I don't see the need for this. Users can easily force exactly what they want using atomicModifyIORef2 (or if they really want, atomicModifyIORef2Lazy). > 5. atomicApplyIORef(and atomicApplyIORef') deserve its own primop, since > we can skip building selector thunks all together. > Yes, I've implemented that primop already, and it will be added. I'm still looking for an expert to help implement the simplest of the lot, atomicSwapMutVar#, as efficiently as possible.

2 1

Re: Discussion: New atomic IORef functions
by Dr.Koster 11 Jul '18

11 Jul '18

But if you force the pair result, you have to evaluate modifying function isn't it? That's sometime unwanted when you have an very expensive f here, let's say a deep binary search which might not depend previous result. I think that is also why current atomicModifyIORef is designed this way, so I'd rather keep it the old way. 发自我的iPhone ------------------ Original ------------------ From: David Feuer <david.feuer(a)gmail.com> Date: Thu,Jul 12,2018 0:51 AM To: winter <drkoster(a)qq.com> Cc: Haskell Libraries <libraries(a)haskell.org> Subject: Re: Discussion: New atomic IORef functions All the fundamental functions I've defined can be usefully used in the base library. I think that's one good reason to put them there. Another is that while the basic array operations have traditionally been exposed through array, vector, and primitive, and the basic TVar operations have been exposed through stm, the basic IORef and MVar operations have been exposed through base (except, for some reason, casMutVar#). I see no reason to change that. I never proposed a function that forces the previous value unnecessarily, so I don't know why you're complaining about that. The extra laziness I don't like is in the pair result; none of the uses I've seen thus can far make intentional use of that. That's why I tend to think atomicModifyIORef (as it exists today) is almost never what people actually want. Squeezing into a second component leads to extra allocation in what may be a performance-critical function; that said, I'm willing to hold off on higher tuples for now. On Sun, Jul 8, 2018, 3:36 AM winter <drkoster(a)qq.com> wrote: I believe new variations should always be motivated by use-case if there're too many choices, the lazy behavior of old `atomicModifyIORef` is justified by some cases the modifying functions are lazy in its argument, thus a lazy version could win by not forcing previous thunks, we'd want to keep its behavior as how it's documented. As for tuples more than pairs, they're not really needed, user can always squeeze their product into `b` component. IMHO only the addition of `atomicModifyIORef_` is sensible in the context of base, other APIs may go to package like primitives. But if you have a motivated use case with `atomicModifyIORef2`, etc. Please tell me. On 2018年07月08日 03:09, David Feuer wrote: Whoops! I left out the proposal link: https://github.com/ghc-proposals/ghc-proposals/pull/149 Also, what I called atomicModifyIORef_ below should really be called something like atomicModifyIORef'_, since it forces a polymorphic value. Another thing to note: the underlying atomicModifyMutVar2# primop actually supports more than just pairs. It can handle triples, solos, and any other record types whose first components are lifted: atomicModifyIORefSoloLazy :: IORef a -> (a -> Solo a) -> IO (Solo a) atomicModifyIORefSolo :: IORef a -> (a -> Solo a) -> IO a atomicModifyIORef3, atomicModifyIORef3Lazy :: IORef a -> (a -> (a, b, c)) -> IO (a, b, c) etc. Should we add any such? On Sat, Jul 7, 2018, 2:35 PM David Feuer <david.feuer(a)gmail.com> wrote: I have proposed[1] the replacement of the atomicModifyMutVar# primop, and the addition of two cheaper but less capable ones. It seems likely that the proposal will succeed, but that the GHC steering committee will leave the question of user interface changes to the libraries list. I would like to open the discussion here. The new primops lead naturally to several thin wrappers: -- Atomically replace the IORef contents -- with the first component of the result of -- applying the function to the old contents. -- Return the old value and the result of -- applying the function, without forcing the latter. -- -- atomicModifyIORef ref f = do -- (_old, ~(_new, res)) <- atomicModifyIORef2Lazy ref f -- return res atomicModifyIORef2Lazy :: IORef a -> (a -> (a, b)) -> IO (a, (a, b)) -- Atomically replace the IORef contents -- with the result of applying the function -- to the old contents. Return the old and -- new contents without forcing the latter. atomicModifyIORefLazy_ :: IORef a -> (a -> a) -> IO (a, a) -- Atomically replace the IORef contents -- with the given value and return the old -- contents. -- -- atomicWriteIORef ref x = void (atomicSwapIORef ref x) atomicSwapIORef :: IORef a -> a -> IO a Based on the code I've read that uses atomicModifyIORef, I believe that the complete laziness of atomicModifyIORef2Lazy and atomicModifyIORefLazy_ is very rarely desirable. I therefore believe we should also (or perhaps instead?) offer stricter versions: atomicModifyIORef2 :: IORef a -> (a -> (a, b)) -> IO (a, (a, b)) atomicModifyIORef2 ref f = do r@(_old, (_new, _res)) <- atomicModifyIORef2Lazy ref f return r atomicModifyIORef_ :: IORef a -> (a -> a) -> IO (a, a) atomicModifyIORef_ ref f = do r@(_old, !_new) <- atomicModifyIORefLazy_ ref f return r The classic atomicModifyIORef also admits a less gratuitously lazy version: atomicModifyIORefNGL :: IORef a -> (a -> (a,b)) -> IO b atomicModifyIORefNGL ref f = do (_old, (_new, res)) <- atomicModifyIORef2 ref f return res Should we add that as well (with a better name)? Should we even consider *replacing* the current atomicModifyIORef with that version? That could theoretically break existing code, but I suspect it would do so very rarely. If we don't change the existing atomicModifyIORef now, I think we should consider deprecating it: it's very easy to accidentally use it too lazily. _______________________________________________ Libraries mailing list Libraries(a)haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries _______________________________________________ Libraries mailing list Libraries(a)haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

3 2

Re: Discussion: New atomic IORef functions
by David Feuer 11 Jul '18

11 Jul '18

All the fundamental functions I've defined can be usefully used in the base library. I think that's one good reason to put them there. Another is that while the basic array operations have traditionally been exposed through array, vector, and primitive, and the basic TVar operations have been exposed through stm, the basic IORef and MVar operations have been exposed through base (except, for some reason, casMutVar#). I see no reason to change that. I never proposed a function that forces the previous value unnecessarily, so I don't know why you're complaining about that. The extra laziness I don't like is in the pair result; none of the uses I've seen thus can far make intentional use of that. That's why I tend to think atomicModifyIORef (as it exists today) is almost never what people actually want. Squeezing into a second component leads to extra allocation in what may be a performance-critical function; that said, I'm willing to hold off on higher tuples for now. On Sun, Jul 8, 2018, 3:36 AM winter <drkoster(a)qq.com> wrote: > I believe new variations should always be motivated by use-case if > there're too many choices, the lazy behavior of old `atomicModifyIORef` is > justified by some cases the modifying functions are lazy in its argument, > thus a lazy version could win by not forcing previous thunks, we'd want to > keep its behavior as how it's documented. > > As for tuples more than pairs, they're not really needed, user can always > squeeze their product into `b` component. > > IMHO only the addition of `atomicModifyIORef_` is sensible in the context > of base, other APIs may go to package like primitives. But if you have a > motivated use case with `atomicModifyIORef2`, etc. Please tell me. > > On 2018年07月08日 03:09, David Feuer wrote: > > Whoops! I left out the proposal link: > > https://github.com/ghc-proposals/ghc-proposals/pull/149 > > Also, what I called atomicModifyIORef_ below should really be called > something like atomicModifyIORef'_, since it forces a polymorphic value. > > Another thing to note: the underlying atomicModifyMutVar2# primop actually > supports more than just pairs. It can handle triples, solos, and any other > record types whose first components are lifted: > > atomicModifyIORefSoloLazy > :: IORef a -> (a -> Solo a) -> IO (Solo a) > > atomicModifyIORefSolo > :: IORef a -> (a -> Solo a) -> IO a > > atomicModifyIORef3, atomicModifyIORef3Lazy > :: IORef a -> (a -> (a, b, c)) -> IO (a, b, c) > > etc. > > Should we add any such? > > On Sat, Jul 7, 2018, 2:35 PM David Feuer <david.feuer(a)gmail.com> wrote: > >> I have proposed[1] the replacement of the atomicModifyMutVar# primop, and >> the addition of two cheaper but less capable ones. It seems likely that the >> proposal will succeed, but that the GHC steering committee will leave the >> question of user interface changes to the libraries list. I would like to >> open the discussion here. >> >> The new primops lead naturally to several thin wrappers: >> >> -- Atomically replace the IORef contents >> -- with the first component of the result of >> -- applying the function to the old contents. >> -- Return the old value and the result of >> -- applying the function, without forcing the latter. >> -- >> -- atomicModifyIORef ref f = do >> -- (_old, ~(_new, res)) <- atomicModifyIORef2Lazy ref f >> -- return res >> atomicModifyIORef2Lazy >> :: IORef a -> (a -> (a, b)) -> IO (a, (a, b)) >> >> -- Atomically replace the IORef contents >> -- with the result of applying the function >> -- to the old contents. Return the old and >> -- new contents without forcing the latter. >> atomicModifyIORefLazy_ >> :: IORef a -> (a -> a) -> IO (a, a) >> >> -- Atomically replace the IORef contents >> -- with the given value and return the old >> -- contents. >> -- >> -- atomicWriteIORef ref x = void (atomicSwapIORef ref x) >> atomicSwapIORef >> :: IORef a -> a -> IO a >> >> Based on the code I've read that uses atomicModifyIORef, I believe that >> the complete laziness of atomicModifyIORef2Lazy and >> atomicModifyIORefLazy_ is very rarely desirable. I therefore believe we >> should also (or perhaps instead?) offer stricter versions: >> >> atomicModifyIORef2 >> :: IORef a -> (a -> (a, b)) -> IO (a, (a, b)) >> atomicModifyIORef2 ref f = do >> r@(_old, (_new, _res)) <- atomicModifyIORef2Lazy ref f >> return r >> >> atomicModifyIORef_ >> :: IORef a -> (a -> a) -> IO (a, a) >> atomicModifyIORef_ ref f = do >> r@(_old, !_new) <- atomicModifyIORefLazy_ ref f >> return r >> >> The classic atomicModifyIORef also admits a less gratuitously lazy >> version: >> >> atomicModifyIORefNGL >> :: IORef a -> (a -> (a,b)) -> IO b >> atomicModifyIORefNGL ref f = do >> (_old, (_new, res)) <- atomicModifyIORef2 ref f >> return res >> >> Should we add that as well (with a better name)? Should we even consider >> *replacing* the current atomicModifyIORef with that version? That could >> theoretically break existing code, but I suspect it would do so very >> rarely. If we don't change the existing atomicModifyIORef now, I think we >> should consider deprecating it: it's very easy to accidentally use it too >> lazily. >> > > > _______________________________________________ > Libraries mailing listLibraries@haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > > > _______________________________________________ > Libraries mailing list > Libraries(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >

1 0

Better casing functions (German ß, etc.)
by 박신환 11 Jul '18

11 Jul '18

Current Haskell has 'simple' `Char`-to-`Char` casing functions (as specified by Unicode), namely `toUpper`, `toLower` and `toTitle`. So to convert cases of a `String`, Haskell intends `fmap toUpper`, etc. But this has some bugs. Case 1. German ß (Eszett) 'ß' (U+00DF), Latin Small Letter Sharp S, is a lowercase letter itself, but Unicode doesn't specify its 'simple' uppercase counterpart. It's because its uppercase counterpart is not a single character, but two characters, "SS". Case 2. Turkish İ and ı Rather than the common 'I' and 'i' case pair, Turkish language has the 'İ' (U+0130) and 'i' pair and the 'I' and 'ı' (U+0131) pair. Those are, dotted I pair and dotless I pair, respectively. Case 3. Greek Σ (Sigma) Greek 'Σ' (U+03A3) must be lowercase mapped to 'ς' (U+03C2) if followed by a whitespace, rather than normal 'σ' (U+03C3). Case 4. Greek iota subscript (Ypogegrammeni) Greek 'Capital' letters with iota subscripts (for example, 'ᾈ' (U+1F88)), though they are the 'simple' uppercase counterpart of their lowercase counterpart, they themselves are actually treated as titlecase characters. For example, the actual uppercase counterpart of 'ᾀ' (U+1F80) is "ἈΙ" (U+1F08 U+0399). That is, an actual capital iota instead of the iota subscript. Case 5. Precomposed letters without upper/lowercase counterpart For example, ΐ (U+03B0) doesn't have precomposed uppercase counterpart. It must be effectively mapped to "Ϊ́" (U+03AA U+0301). In Summary, we need more elaborated casing functions which are `String`-to-`String`. Bibliography: The Unicode Standard Version 11.0 – Core Specification, Section 5.18.

4 3