- Libraries - Haskell.org

joinAtomicallyWithMask, and a potential resource leak in http-client
by Leon Smith 12 Dec '19

12 Dec '19

I'm going to propose four new functions to add to the STM library, and would be interested in getting some feedback for this idea. The basic idea is that we implement these two functions, along with their WithUninterruptibleMask counterparts: {-# LANGUAGE RankNTypes, ImpredicativeTypes #-} import Control.Monad(join) import Control.Concurrent.STM(STM, atomically) import Control.Exception(mask, mask_) joinAtomicallyWithMask :: STM ((forall a. (IO a -> IO a)) -> IO b) -> IO b joinAtomicallyWithMask transaction = mask $ \unmask -> do join . atomically $ do ioCont <- transaction return $ do ioCont unmask joinAtomicallyWithMask_ :: STM (IO b) -> IO b joinAtomicallyWithMask_ = mask_ . join . atomically However, the entire point of this endeavor is to implement these functions better than this. What we really want to do is to mask asynchronous exceptions in conjunction with an STM commit: a conservative implementation would upgrade the masking state shortly before we know if the transaction will commit or retry: if it retries, we restore the old masking state; if we commit, then we leave it alone. A perhaps more daring implementation would find a place immediately after commit before async exceptions could possibly appear, but this probably isn't necessary if the approach turns out to be too difficult/problematic. joinAtomicallyWithUninterruptibleMask actually allows us to express some programs that we couldn't before: will complete the STM transaction in an uninterruptible masking state, but we don't have to worry about blocking in the transaction because we won't have upgraded the masking state. I actually noticed this pattern twice recently: once in some code I was writing, and a second time within a week or two later, I noticed this would fix a potential resource leak in http-client, namely: https://github.com/snoyberg/http-client/blob/master/http-client/Data/KeyedP… reap destroy var = loop where loop = do threadDelay (5 * 1000 * 1000) join $ atomically $ do m'' <- readTVar var case m'' of PoolClosed -> return (return ()) PoolOpen idleCount m | Map.null m -> retry | otherwise -> do (m', toDestroy) <- findStale idleCount m writeTVar var m' return $ do mask_ (mapM_ (ignoreExceptions . destroy) toDestroy) loop It looks like to me that we really want to ensure that we call "destroy" on every resource taken out of the pool, but it would appear that the mask_ is somewhat misplaced: while perhaps it removes the most likely places that an asynchronous exception could break that invariant, it looks like there's still a possibility that an asynchronous exception that happens after the transaction commits but before the call to `mask_`. And lastly, I know I independently came up with the join . atomically idiom, but I know I'm not the first nor the last to do so. It's an idiom that does deserve to be more widely appreciated, and this could be a way to do it. Best, Leon

1 0

popCount is a method of Bits, not a FiniteBits
by Oleg Grenrus 03 Dec '19

03 Dec '19

I propose to change it to be member of FiniteBits I recall, there was a proposal to remove bitSize from Bits, so it's an opportunity to introduce another small, yet breaking change at the same time. Discussion time 2 week. - Oleg

6 11

Newtype wrapper for Float/Double with sensible Eq/Ord instances
by Zemyla 02 Dec '19

02 Dec '19

There should be a newtype wrapper that wraps a RealFloat, keeps the Enum/Num/Fractional/so on instances, but gives it sensible ones for Eq and Ord, with the following order: NaN < -Inf < negative < -0 < +0 < positive < +Inf It may not belong in base, but if nothing else it definitely belongs in containers, because Set and Map depend critically on sensible instances of Ord. (Also, it should newtype derive Read and Show, so that people using them with GHCi see "fromList [(NaN, yadda), (-1.5, yadda), (2.7, blah), (+Inf, blah)]" instead of "fromList [(FloatOrd { unwrapFloat :: NaN }, yadda), ...".)

3 3

Add NonEmptyMap and NonEmptySet to containers
by V.Liepelt 25 Nov '19

25 Nov '19

Hi John, I saw your thread about non-empty containers today. Why not call them `Set1`, `Map1`, etc. in analogy to `many`/`many1` from parser combinators? I think most agree that `NonEmpty` is not a great name—I couldn’t summarise it better than Andreas: "NonEmpty what?”. Hence I always do the following: ``` import Data.List.NonEmpty (NonEmpty) import qualified Data.List.NonEmpty as List1 type List1 = NonEmpty ``` In fact I just hoogled for this and noticed that the rebase package seems to do this too. More anecdotal precedent: recently a colleague wrote a datatype isomorphic to `(a, a, [a])` and called it something not quite as ridiculous as `ListWithAtLeastTwoElements`, so I suggested `List2` and everybody involved was happy with that. Vilem

12 12

Improve Data.Functor documentation
by chessai . 24 Nov '19

24 Nov '19

A beginner learning about parametricity and the relationship between Functorial composition and Functor identity brought up a good point to me: the documentation for Bifunctor[1] and Profunctor[2] both explain the relationship between parametricity and their laws, particularly what constitutes a minimum "burden" of proof. I don't have an exact wording yet, but I think it would be a good idea to update the Functor documentation accordingly. A separate point of issue: The line "The instances of Functor for lists, Maybe and IO satisfy these laws." This statement is redundant at best when you consider that Haskellers aim for instances of Functor to be lawful. At worst it could leave beginners confused and with more questions. I could imagine someone interpreting this statement as saying there might exist some unlawful instances. I think this line should just be removed. Let me know what you think. Thanks

3 3

Applicative ((->) c) methods
by Keith 20 Nov '19

20 Nov '19

Just thinking for Applicative ((->) c) (<*) = const (*>) = flip const -- or \ _ x -> x Is it because we can rely on GHC to produce the same code from the current definitions? Also wondering why Monad uses f >>= k = \ r -> k (f r) r but Applicative uses (<*>) f g x = f x (g x) liftA2 q f g x = q (f x) (g x) Seems like if it's style/legibility, it should be consistent between (>>=) and (<*>). And if it's inlining issue, we'd want lambdas for all three. --Keith -- Sent from my phone with K-9 Mail.

4 6

Add Data.Foldable1 to base
by Oleg Grenrus 19 Nov '19

19 Nov '19

I propose adding `Foldable1` type-class into `base1`. Add Foldable1 ============= Motivation ---------- It's regularly asked whether `Foldable1` could be added to `base` (e.g. on reddit[^ref1], there're also GHC issue[^ref2] and old phabricator diff[^ref3]) Also there's work towards non-empty maps and sets[^ref4], which would benefit from `Foldable1`. As commented on reddit, `Foldable1` could be added without any pain to the `base` as it's pure addition - no modifications needed in existing modules. [^ref1]: https://www.reddit.com/r/haskell/comments/6d0vgt/could_we_have_foldable1_an… [^ref2]: https://gitlab.haskell.org/ghc/ghc/issues/13573 [^ref3]: https://phabricator.haskell.org/D4812 [^ref4]: https://github.com/haskell/containers/pull/616 Change ------ The change exist as merge request[^ref4] on gitlab.haskell.org. Importantly, this change **doesn't change** anything in other modules of `base`, except of adding a `Foldable` instance to `Data.Complex`. In particular, `foldl1` and `foldr1` in `Data.Foldable` remain partial, etc. My version of `Foldable1` class is big, so I'll comment the motivation for each member ```haskell class Foldable t => Foldable1 t where {-# MINIMAL foldMap1 | toNonEmpty | foldr1map #-} -- the defining member, like foldMap but only asking for Semigroup foldMap1 :: Semigroup m => (a -> m) -> t a -> m fold1 :: Semigroup m => t m -> m# -- strict foldMap1, cf foldMap' foldMap1' :: Semigroup m => (a -> m) -> t a -> m -- analogue of toList toNonEmpty :: t a -> NonEmpty a -- left&right, strict&non-strict folds foldr1 :: (a -> a -> a) -> t a -> a foldr1' :: (a -> a -> a) -> t a -> a foldl1 :: (a -> a -> a) -> t a -> a foldl1' :: (a -> a -> a) -> t a -> a -- these can have efficient implementation for NonEmptySet maximum1 :: forall a. Ord a => t a -> a minimum1 :: forall a. Ord a => t a -> a -- head1 have efficient implementation for NonEmpty and Tree -- last1 for symmetry head1 :: t a -> a last1 :: t a -> a -- fold variants with premap. -- Without this map, we cannot implement foldl using foldr etc. foldr1map :: (a -> b) -> (b -> b -> b) -> t a -> b foldl1'map :: (a -> b) -> (b -> b -> b) -> t a -> b foldl1map :: (a -> b) -> (b -> b -> b) -> t a -> b foldr1'map :: (a -> b) -> (b -> b -> b) -> t a -> b ``` The merge request also adds instances for everything non-empty in `base`. I propose the `Data.Foldable1` as the module name. `semigroupoids`[^ref6] uses `Data.Semigroup.Foldable`, but it's confusing; and using different name could help migration. The module contains five top-level functions, which should be self-explanatory: ```haskell intercalate1 :: (Foldable1 t, Semigroup m) => m -> t m -> m foldrM1 :: (Foldable1 t, Monad m) => (a -> a -> m a) -> t a -> m a foldlM1 :: (Foldable1 t, Monad m) => (a -> a -> m a) -> t a -> m a maximum1By :: Foldable1 t => (a -> a -> Ordering) -> t a -> a minimum1By :: Foldable1 t => (a -> a -> Ordering) -> t a -> a ``` This is less than in `Data.Semigroup.Foldable`[^ref9], as without `Apply` they don't make sense. For example: ```haskell -- needs Apply, not in Data.Foldable1 traverse1_ :: (Foldable1 t, Apply f) => (a -> f b) -> t a -> f () ``` And if we relax `Apply` to `Applicative`, we get `traverse_`. [^ref5]: https://gitlab.haskell.org/ghc/ghc/merge_requests/1973 [^ref9]: https://hackage.haskell.org/package/semigroupoids-5.3.3/docs/Data-Semigroup… [^ref5]: https://gitlab.haskell.org/ghc/ghc/merge_requests/1973 Compatibility & migration ------------------------- I drafted a compatibility package `foldable1` (github[^ref6], haddocks[^ref7]), which I hope could be maintained under github.com/haskell organization. I can act as a maintainer, with a hope that there won't be a lot of changes happening in `Data.Foldable1`. To my surprise, there's already a package with this name on Hackage[^ref8] by M Farkas-Dyck (cc'd). I hope they would donate the name to Haskell.org / CLC; the package won't have any other good use when `Data.Foldable1` is in `base`, then act as a compat package. `Data.Foldable1` contains also instances for `Lift`, `Backwards` and `Reverse` data types from `transformers`. Perfectly, the `transformers` bundled with GHC with this change would implement the instances as well. This change should propage to `transformers-compat` too. Similarly, `containers` would have an instance for `Tree` (and non-empty `Set` and `Map` when they are added). `semigroupoids` would need a bit of work, to depend on `foldable1`, yet the public changes can be kept quite minimal. I don't think that anything in reverse dependencies of `lens` will be broken by this change, if "new" `Foldable1` is re-exported from `semigroupoids`' `Data.Semigroup.Foldable`[^ref9] Other "compat" packages -- like `tagged`, `bifunctors` -- have to be dealt with case by case. For example whether they should depend on `foldable1` or other way around. [^ref6]: https://github.com/phadej/foldable1 [^ref7]: https://oleg.fi/haddocks/foldable1/ [^ref8]: https://hackage.haskell.org/package/foldable1 [^ref9]: https://hackage.haskell.org/package/semigroupoids-5.3.3/docs/Data-Semigroup… Unresolved questions -------------------- - Should we add `Bifoldable1` too. That should be trivial. - GHC-8.10 freeze is quite soon, is targeting GHC-8.12/base-4.15 more realistic?

6 12

Re: Proposal: Laws for mtl classes
by Li-yao Xia 18 Nov '19

18 Nov '19

Hello Libraries, Some time ago I proposed some laws for the most common mtl classes (State, Reader, Error): https://mail.haskell.org/pipermail/libraries/2019-April/029549.html To address some concerns about completeness I reorganized the laws into groups describing more high-level properties, and formalized them in Coq to ensure the laws are strong enough to lift themselves through common transformers (StateT, ExceptT, ContT; ReaderT and WriterT are similar to StateT, but formalizing that relationship also makes the laws interact in interesting ways). I'm open to suggestions for better ways to verify the "completeness" of the laws. However, as long there aren't any objections to the existing laws themselves, it still seems worth having some documentation earlier rather than never. In the end the actual changes to the initial proposal were quite minor. Feel free to weigh in on the following pull requests: https://github.com/haskell/mtl/pull/61 (MonadReader, MonadState) https://github.com/haskell/mtl/pull/62 (MonadError) --- # Changes 1. Three laws were added, In MonadReader, > local id = id to complete the property that local is a monoid homomorphism (rather than only semigroup); thanks to Andreas Abel for pointing out its absence in the initial proposal. > local f u = ask >>= \s -> local (\_ -> f s) u This law was necessary to verify ContT's MonadReader instance. ContT is not actually lawful, but a certain subset of it seems to be (the elements of ContT that satisfy the commutativity of ask). I did not manage to find out whether it's implied by the other laws. In MonadError, a naturality law > fmap f (catchError u h) = catchError (fmap f u) (fmap f . h) whose need arose when describing the relationship between StateT's and ReaderT's instances. 2. Another finding is the fact that, much like the laws of MonadState are equivalent to saying that 'state' is a monad morphism, the 'ask' fragment of MonadReader (which to many *is* MonadReader) can also be characterized by a monad morphism, which cannot be 'reader', as it only yields two of the 'ask' laws. 3. There were two mistakes in the original proposal. One MonadError law was too strong: > catchError (m >>= k) h = tryError m >>= either h k -- broken by StateT And local should flip the order of composition > local g . local f = local (g . f) -- wrong > local g . local f = local (f . g) -- right --- # Updated proposal ## MonadState > get >>= put = pure () > put s >> get = put s >> pure s > put s1 >> put s2 = put s2 Those three laws imply the following equations expressing that get has no side effects: > get >> m = m > get >>= \s1 -> get >>= \s2 -> k s1 s2 = get >>= \s -> k s s state must be equivalent to its default definition in terms of get and put, and conversely. Under that last condition, a property which is equivalent to the laws above is that state must be a monad morphism, from State s to m. --- ## MonadReader ask has no side effects, and produces the same result at any time. > ask >> m = m > ask >>= \s1 -> ask >>= \s2 -> k s1 s2 = ask >>= \s -> k s s > > m <*> ask = ask <**> m local f applies f to the environment produced by ask. > local f ask = f <$> ask > local f u = ask >>= \s -> local (\_ -> f s) u local is a monoid morphism from (r -> r) to (reversed) (m a -> m a) (i.e., (Endo r -> Dual (Endo (m a)))). > local id = id > local g . local f = local (f . g) local is a monad morphism from m to m. > local f (pure x) = pure x > local f (a >>= k) = local f a >>= \x -> local f (k x) reader must be equivalent to its default definition in terms of ask, and conversely. Under that last condition, a property which is equivalent to the first two laws is that reader must be a monad morphism from Reader r to m. Another property equivalent to the first three laws is that there is a monad morphism phi :: forall a. ReaderT r m a -> m a such that phi ask = ask and phi . lift = id. --- ## MonadError See also Exceptionally Monadic Error Handling, by Jan Malakhovski: https://arxiv.org/abs/1810.13430 catchError and throwError form a monad, with (>>=) interpreted as catchError and pure as throwError. > catchError (throwError e) h = h e > catchError m throwError = m > catchError (catchError m k) h = catchError m (\e -> catchError (k e) h) pure and throwError are left zeros for catchError and (>>=) respectively. > catchError (pure a) h = pure a > throwError e >>= k = throwError e catchError commutes with fmap (it is a natual transformation). > fmap f (catchError u h) = catchError (fmap f u) (fmap f . h) --- Li-yao Xia

3 3

A bug with Sum (as in Monoid)
by Dannyu NDos 12 Nov '19

12 Nov '19

Sum has bug with floating points. Current definition states +0 as the identity element, while the actual identity is -0 since +0 + -0 = +0.

5 7

Exposed # kinded variables + polykinded Prelude classes?
by Zemyla 06 Nov '19

06 Nov '19

I'm wondering if there would be a benefit, if not to the average programmer, then to the ones working on deeper/faster code, to allow some of the # kinded types (mostly Int#, Word#, Char#, Float#, Double#) to be used in Safe code, and to have typeclasses able to work with them. For instance, the definition of Show would become: class Show (a :: TYPE r) where show :: a -> String default show :: (r ~ 'LiftedRep) => a -> String show x = showsPrec 0 x "" showsPrec :: Int -> a -> ShowS default showsPrec :: (r ~ 'LiftedRep) => Int -> a -> ShowS showsPrec _ x s = show x ++ s showList :: (r ~ 'LiftedRep) => [a] -> ShowS showList ls s = showList__ shows ls s The fact that the defaults only work when the type is a LiftedRep is a nonissue, because there's only a finite number of non-lifted types we'd be defining it for. You could do the same with Eq, Ord, Num, Real, Integral, Fractional, Floating, RealFrac, RealFloat, Semigroup, Monoid, Bits, FiniteBits, and probably several others I can't think of right now. However, with the functions that return pairs, you'd need a version that returns an unboxed pair instead. Assuming you changed ReadPrec, you could even do the same with Read: newtype ReadP (a :: RuntimeRep r) = ReadP (forall b. (a -> R b) -> R b) newtype ReadPrec (a :: RuntimeRep r) = ReadPrec (Int -> ReadP a) IO, ST, and STM could be made polykinded the same way, and would open up Storable. However, how to do a definition for Monad that works for polykinded monads is an issue. I do know that RebindableSyntax handles it easily when there's just one monad that can operate on multiple kinds, though. As for which # types could be exposed, I'm thinking that Char#, Int#, Word#, Float#, Double#, and Proxy# wouldn't be able to break out of Safe code. Int64# and Word64# would work as well, and for 64-bit machines would just be type aliases for Int# and Word# respectively. For types which have functions with undefined behavior for some arguments, you can just make wrappers that check the arguments and error out for the bad values. MutVar#, MVar#, TVar#, and StableName# don't open up any functions that would be unsuitable for safe code, either. I'm pretty sure that Array# and MutableArray# would also be safe, as long as all functions were length-checked and threw errors instead of having undefined behavior. As for why this would be a desirable thing? Mostly for the sake of convenience and generality, I think. I find myself working with unboxed values from time to time, and it's a pain to always remember to use (+#) for Int# and plusWord# for Word#. In addition, having typeclasses that can return unboxed values (like a hypothetical sized# :: Sized a => a -> Int# vs sized :: Sized a => a -> Int) can improve the generated code when the code using the typeclass doesn't get specialized. The module to import these would have to explain the differences between # kinded types and * kinded ones: # values aren't lazy; they can't be top-level definitions; you can't use unboxed tuples or sums with GHCi; and with a few exceptions, you can't place them in containers (you can't have an [Int#], for instance).

5 5