June 2020 - Libraries

Generalize IsString
by Dannyu NDos 08 Jun '20

08 Jun '20

As we already have IsList, shouldn't we just do this? class (IsList l, Item l ~ Char) => IsString l instance (a ~ Char) => IsString [a] instance (a ~ Char) => IsString (NonEmpty a) instance (a ~ Char) => IsString (ZipList a)

4 4

Sliding windows for streaming
by David Feuer 07 Jun '20

07 Jun '20

I'm looking for a bit of help with a library design choice. The streaming package currently offers a slidingWindow function converting a stream into a stream of fixed-size windows of that stream[1]: slidingWindow :: Monad m => Int -- Window size -> Stream (Of a) m b -> Stream (Of (Seq a)) m b This is based directly on a similar function in conduit. Using a rough translation into the world of lists, we have slidingWindow 3 "abcdef" = ["abc","bcd","cde","def"] The awkward case where the stream is shorter than the window is handled by potentially producing a short sequence at the end: slidingWindow 3 "ab" = ["ab"] slidingWindow 3 "" = [""] I recently merged a pull request that adds variations on sliding window maxima and minima using what's apparently a "folklore" algorithm. For example slidingWindowMax 3 "abcbab" = "abcccb" This is basically like slidingWindowMax k = map maximum . slidingWindow k except that an empty stream doesn't yield anything, to avoid undefined values. The big advantage of these specialized functions is that rather than having to take a maximum over a sequence of length `k` at each step, they only do a constant (amortized) amount of work at each step. Nice! But not very general. Suppose we want to take a moving average of some sort, like an arithmetic mean, geometric mean, harmonic mean, or median? That thought leads quite naturally to a data structure: a queue holding elements of some arbitrary *semigroup* that efficiently keeps track of the sum of all the elements in the queue[2]. While the choice of *data structure* is moderately obvious, the choice of *sliding window function* is less so. The tricky bit is, again, what happens when the stream is too short for the window. If you work in the Sum semigroup and divide the results by the window size to get a moving average, then a too-short stream will give a (single) result that's completely wrong! Oof. What would be the most useful way to deal with this? The streams in `streaming` give us the option of producing a distinguished "return" value that comes after all the yields. Would it make sense to *return* the incomplete sum, and the number of elements that went into it, instead of *yielding* it into the result stream? That seems flexible, but maybe a tad annoying. What do y'all think? [1] https://hackage.haskell.org/package/streaming-0.2.3.0/docs/Streaming-Prelud… [2] See the AnnotatedQueue in https://github.com/haskell-streaming/streaming/pull/99/files which basically modifies Okasaki's implicit queues using some of the basic ideas that appear in Hinze-Paterson 2–3 trees.

2 2

Re: [core libraries] Adding Backwards to base, re-export from Control.Applicative
by Baldur Blöndal 07 Jun '20

07 Jun '20

I'll CC Haskell Libraries þri., 12. nóv. 2019 kl. 22:14 skrifaði Ryan Scott <ryan.gl.scott(a)gmail.com>: > This discussion definitely belongs on the mailing list, yes. > > Ryan > > On Tue, Nov 12, 2019 at 11:59 AM chessai . <chessai1996(a)gmail.com> wrote: > >> I support this. +1 >> >> Should we copy in the libraries mailing list? >> >> On Tue, Nov 12, 2019, 5:56 AM Baldur Blöndal <baldurpet(a)gmail.com> wrote: >> >>> Is it time to move Control.Applicative.Backwards module from >>> transformers to base, and maybe re-export it from Control.Applicative. >>> >>> It's an interesting, important, simple property of Applicative that they >>> can be run backwards. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "haskell-core-libraries" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to haskell-core-libraries+unsubscribe(a)googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/haskell-core-libraries/CAK9DwL9vPwA%3DJ%3… >>> <https://groups.google.com/d/msgid/haskell-core-libraries/CAK9DwL9vPwA%3DJ%3…> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "haskell-core-libraries" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to haskell-core-libraries+unsubscribe(a)googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/haskell-core-libraries/CAD34_kK6QgQ%2BQmg… >> <https://groups.google.com/d/msgid/haskell-core-libraries/CAD34_kK6QgQ%2BQmg…> >> . >> >

8 7

Ann: let’s create better norms for ensuring all important libraries have several active maintainers
by Carter Schonwald 06 Jun '20

06 Jun '20

I’m very happy that we now agree to have several active comaintainers for all core/systematically . (though the specifics of how it was done this week i view as anthetical to what our community represents and values.) There’s ways of doing that can be done in a good faith way. I’m definitely not perfect. I also have spent much of the past five years trying to learn to manage my somewhat aggressive and personal flavor of generalized anxiety disorder (certain flavors of anxiety or depression in men present as aggressive behavior ). Some of my approaches on tickets that people find frustrating are my attempt to avoid getting fixated on getting into Interpersonal conflicts rather than trying to shut those voices down. Also I find it hard to have those dialogues in that communication format. For asynchronous conversations in different time zones I fully welcome discord or what’s app or signal or freenode irc. Conversations with emotional dimensions are challenging in email. Let alone with issue trackers! In the past month I was working with Simon Jakobi and Andrew lelechenko as a sort of pilot for a new Haskell action team (HAT) with them as the initial leadership for 2020, to have some folks we all recognize to have excellent taste and engineering to help support and triage all maintainers and efforts. We spent a bit of time helping out on bytestring and I think that was a success. and I hoping they and other members of HAT can make a big difference across all the important libraries we have. I look forward to support HAT via my role as CLC. More broadly, I was also privately in discussion with some folks before this week's tornado about how to move to make it the norm for all core / important libraries have 3-5 variously active and diversely different comaintainers. What happened this week is not how id have liked it to be rolled out, and I’m in the best of head spaces at the moment, but I think it is VERY good that we collectively agree there should be a norm of actively making sure there are several diverse co maintainers for every such library. how it was handled this week is not how it should have been handled, i pray that no one ever views this as grounds for similar treatment of people inour community who have been trying to be in their own way, good caretakers of important community resources In terms of what i like and appreciate about this community, I would like to challenge us all to think about how we can turn frustration into concrete and positively actionable feedback that supports each other in a good faith positive way. I feel that we all failed colectively, and thats fine, but i hope we can learn to engage in this differently. perhaps most importantly, i think its GENUINELY important to strive to support and enable every project to have a genuinely diverse maintainer set for all of our important libraries, both in terms of age, creed and whatever. Haskell has failed to do a good job of that in the past, and i challenge us all to work to address that starting with our core going forward On Fri, Jun 5, 2020 at 8:32 AM Mathieu Boespflug <m(a)tweag.io> wrote: > Zemyla, > > I would expect more charitable reception of anything said by anyone, but > especially when the original author stated clearly in the very same message > that they are not a native English speaker. And even if the words are to be > taken on face value, do note that no one has called anyone a tyrant. > Calling out specific behaviour (that predates the current pandemic) as > "tyrannous" (or "tyrannical") and naming someone a tyrant is simply not the > same thing. > > > On Fri, Jun 05, 2020 at 13:06:20, Zemyla <zemyla(a)gmail.com> wrote: > >> So you're saying that the fact that there's basically a civil war and >> pandemic going on is no excuse to respond to e-mails. And you're calling >> his behavior "tyrannous" when there are literal tyrants shooting tear gas >> at civilians. >> >> I gotta admit, this makes me disinclined to believe you're acting in good >> faith. Sometimes shit happens to good people, and someone who can't show a >> little compassion in these (pardon the cliche) unprecedented times is not >> someone I want anywhere involved in package management. >> >>

4 6

Add instance Monad ZipList
by Dannyu NDos 06 Jun '20

06 Jun '20

instance Monad ZipList where ZipList [] >>= _ = ZipList [] ZipList (x:xs) >>= f = ZipList $ do let ZipList y' = f x guard (not (null y')) let ZipList ys = ZipList xs >>= ZipList . join . maybeToList . fmap snd . uncons . getZipList . f head y' : ys instance MonadFail ZipList where fail _ = empty instance MonadPlus ZipList

9 19

random library: Request for Carter Tazio Schonwald (@cartazio) to step down as maintainer with immediate effect
by Profpatsch 06 Jun '20

06 Jun '20

I posted this comment on the thread of https://github.com/haskell/random/pull/62#issuecomment-638088258 Since it was deleted and the comment thread locked by Carter, I am escalating this request to the mailing list, where it cannot be deleted. Profpatsch <notifications(a)github.com> writes: > @cartazio Please take even a fleeting look at https://github.com/idontgetoutmuch/random/pulls?q=is%3Apr+is%3Aclosed, notice the amount of discussion and the timeline. > > This is the full-time work of a team of half a dozen of experts, probably breaching one man-year of effort, easily. > It cannot just be discarded out of hand. > > I have been following the discussions (spread over mailinglists, issue trackers and elsewhere), and I propose you hand over maintainership of `random` as a base package. A parallel issue with relevant discussion can be found at https://github.com/haskell/random/pull/61 I am writing this email from my personal account, because I fear for the health of the Haskell ecosystem, and I think Carter’s behaviour is extremely detrimental to it. I think a lot of other people feel the same way, so I encourage you to add your voice publicly. ~ Profpatsch

11 14

Hackage trustee selection process
by David Feuer 04 Jun '20

04 Jun '20

At present, there is no process for selecting Hackage trustees, there are no designated terms for them, and there is no process for removing them. This strikes me as most unfortunate. We trust that these folks will manage Hackage for the benefit of the community, but there is no process the community can rely on for ensuring the right people have trustee powers. Can we fix this situation? A couple specific suggestions: 1. Perhaps the Hackage trustees can adopt organizational standards similar to those of the Core Library Committee to get things going quickly. 2. Perhaps the CLC should be able to designate a certain number of its own members as Hackage trustees.

4 4

Bifoldable instance for Map
by Joseph C. Sible 04 Jun '20

04 Jun '20

I'd like to propose a change to the containers package: adding a Bifoldable instance to Map. I briefly mentioned this on Reddit [1] and no obvious problems were brought up. I submitted a PR implementing it [2]. This seems like an obvious and straightforward instance to me. Thoughts? Joseph C. Sible [1]: https://old.reddit.com/r/haskell/comments/fsgqd6/monthly_hask_anything_apri… [2]: https://github.com/haskell/containers/pull/714

5 18

Re: Improving Random
by dominic＠steinitz.org 04 Jun '20

04 Jun '20

Ben, We invited anyone interested including Carter to participate in the design back in February. He did not do so. How much longer do you think we should give him? No serious work has been done on it since September 2014 and numerous promises to do something have never materialised. Is there any part of Carter’s critique that in your view has not been addressed by replies in this thread? We should be welcoming contributions and improve libraries when the opportunities arise and not wait another 5 years for perfection (which may never arrive). Dominic Steinitz dominic(a)steinitz.org http://idontgetoutmuch.org Twitter: @idontgetoutmuch > Message: 4 > Date: Wed, 3 Jun 2020 13:11:08 +0200 > From: Ben Franksen <ben.franksen(a)online.de> > To: libraries(a)haskell.org > Subject: Re: Improving Random > Message-ID: <rb80gc$3ong$1(a)ciao.gmane.io> > Content-Type: text/plain; charset=utf-8 > > I agree that communication is an important aspect of maintainership. Not > responding to PRs for a long period of time is bad because it lets > people wonder if anything is being done about it at all. That can be > quite frustrating as I know from my own experience. > > That said, Carter's critique of the proposed API (the classes) looks > valid to me. I understand that coming up with a better API design that > accommodates all the cool permormance improvements may take its time. > > @Carter: why don't you try to eleborate your critique of the proposed > API on the tracker and present a rough sketch of where you think things > should go instead. Then you guys can all work together to flesh this out. > > My experience is that writing things down helps a lot to get them > clearer in my head. If the side effect is people feel that their > contributions are respected and actually worked on, so much the better. > > Cheers > Ben > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Libraries mailing list > Libraries(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > > > ------------------------------ > > End of Libraries Digest, Vol 202, Issue 4 > *****************************************

4 3

Improving Random
by dominic＠steinitz.org 03 Jun '20

03 Jun '20

Hello Libraries, You may recall that following the blog post <https://alexey.kuleshevi.ch/blog/2019/12/21/random-benchmarks/> by @lehins, a group of us (@curiousleo, @lehins and me) invited participation in February <https://mail.haskell.org/pipermail/libraries/2020-February/030261.html> to take this work and apply it to improving the current random library. Our proximate goals were to fix #25 <https://github.com/haskell/random/issues/25> (filed in 2015) and #51 <https://github.com/haskell/random/issues/51> (filed in 2018). After a lot of discussion and experimentation, we have a proposal that addresses both these issues and also: #26 <https://github.com/haskell/random/issues/26>, #44 <https://github.com/haskell/random/issues/44>, #53 <https://github.com/haskell/random/issues/53>, #55 <https://github.com/haskell/random/issues/55>, #58 <https://github.com/haskell/random/issues/58> and #59 <https://github.com/haskell/random/issues/59>. For backwards compatibility, the proposal retains the old style classes and enhances them. Thus in 1.1 we have class RandomGen g where next :: g -> (Int, g) genRange :: g -> (Int, Int) split :: g -> (g, g) {-# MINIMAL next, split #-} and in 1.2 we have class RandomGen g where next :: g -> (Int, g) genWord8 :: g -> (Word8, g) genWord16 :: g -> (Word16, g) genWord32 :: g -> (Word32, g) genWord64 :: g -> (Word64, g) genWord32R :: Word32 -> g -> (Word32, g) genWord64R :: Word64 -> g -> (Word64, g) genShortByteString :: Int -> g -> (Data.ByteString.Short.Internal.ShortByteString, g) genRange :: g -> (Int, Int) split :: g -> (g, g) {-# MINIMAL split, (genWord32 | genWord64 | next, genRange) #-} and next and genRange are deprecated. This interface is what allows the significantly faster performance as no longer is everything forced to go via Integer. Several new interfaces are introduced and it is recommended that new applications use these and, where feasible, existing applications migrate to using them. The major API addition in this PR is the definition of a new class MonadRandom: -- | 'MonadRandom' is an interface to monadic pseudo-random number generators. class Monad m => MonadRandom g s m | g m -> s where {-# MINIMAL freezeGen,thawGen,(uniformWord32|uniformWord64) #-} type Frozen g = (f :: Type) | f -> g freezeGen :: g s -> m (Frozen g) thawGen :: Frozen g -> m (g s) uniformWord32 :: g s -> m Word32 -- default implementation in terms of uniformWord64 uniformWord64 :: g s -> m Word64 -- default implementation in terms of uniformWord32 -- plus methods for other word sizes and for byte strings -- all have default implementations so the MINIMAL pragma holds Conceptually, in MonadRandom g s m, g s is the type of the generator, s is the state type, and m the underlying monad. Via the functional dependency g m -> s, the state type is determined by the generator and monad. Frozen is the type of the generator's state "at rest". It is defined as an injective type family via f -> g, so there is no ambiguity as to which g any Frozen g belongs to. This definition is generic enough to accommodate, for example, the Gen type from mwc-random, which itself abstracts over the underlying primitive monad and state token. This is the full instance declaration (provided here as an example - this instance is not part of random as random does not depend on mwc-random): instance (s ~ PrimState m, PrimMonad m) => MonadRandom MWC.Gen s m where type Frozen MWC.Gen = MWC.Seed freezeGen = MWC.save thawGen = MWC.restore uniformWord8 = MWC.uniform uniformWord16 = MWC.uniform uniformWord32 = MWC.uniform uniformWord64 = MWC.uniform uniformShortByteString n g = unsafeSTToPrim (genShortByteStringST n (MWC.uniform g)) Pure random number generators can also be made instances of this class providing a uniform interface to both pure and stateful random number generators. An instance for the standard number generator StdGen is provided. The Random typeclass has conceptually been split into Uniform and UniformRange. The Random typeclass is still included for backwards compatibility. Uniform is for types where it is possible to sample from the type's entire domain; UniformRange is for types where one can sample from a specified range: class Uniform a where uniformM :: MonadRandom g s m => g s -> m a class UniformRange a where uniformRM :: MonadRandom g s m => (a, a) -> g s -> m a The proposal is a breaking change but the changes are not very intrusive and we have PRs ready for the affected downstream libraries: requires base >= 4.10 (GHC-8.2) StdGen is no longer an instance of Read randomIO and randomRIO were extracted from the Random class into separate functions In addition, there may be import clashes with new functions, e.g. uniform and uniformR. Further explanatory details may be found here <https://github.com/idontgetoutmuch/random/blob/v1.2-release-notes/RELEASE-N…> and the PR for the proposed new version is here <https://github.com/haskell/random/pull/61>. Here are some benchmarks run on a 3.1 GHz Intel Core i7. The full benchmarks can be run using e.g. stack bench. The benchmarks are measured in milliseconds per 100,000 generations. In some cases, the performance is over x1000(!) times better; the minimum performance increase for the types listed below is more than x35. | Name | Mean (1.1) | Mean (1.2) | Improvement| | ----------------------- | ---------- | ---------- | ---------- | | pure/random/Float | 30 | 0.03 | 1038| | pure/random/Double | 52 | 0.03 | 1672| | pure/random/Integer | 43 | 0.33 | 131| | pure/uniform/Word8 | 14 | 0.03 | 422| | pure/uniform/Word16 | 13 | 0.03 | 375| | pure/uniform/Word32 | 21 | 0.03 | 594| | pure/uniform/Word64 | 42 | 0.03 | 1283| | pure/uniform/Word | 44 | 0.03 | 1491| | pure/uniform/Int8 | 15 | 0.03 | 511| | pure/uniform/Int16 | 15 | 0.03 | 507| | pure/uniform/Int32 | 22 | 0.03 | 749| | pure/uniform/Int64 | 44 | 0.03 | 1405| | pure/uniform/Int | 43 | 0.03 | 1512| | pure/uniform/Char | 17 | 0.49 | 35| | pure/uniform/Bool | 18 | 0.03 | 618| | pure/uniform/CChar | 14 | 0.03 | 485| | pure/uniform/CSChar | 14 | 0.03 | 455| | pure/uniform/CUChar | 13 | 0.03 | 448| | pure/uniform/CShort | 14 | 0.03 | 473| | pure/uniform/CUShort | 13 | 0.03 | 457| | pure/uniform/CInt | 21 | 0.03 | 737| | pure/uniform/CUInt | 21 | 0.03 | 742| | pure/uniform/CLong | 43 | 0.03 | 1544| | pure/uniform/CULong | 42 | 0.03 | 1460| | pure/uniform/CPtrdiff | 43 | 0.03 | 1494| | pure/uniform/CSize | 43 | 0.03 | 1475| | pure/uniform/CWchar | 22 | 0.03 | 785| | pure/uniform/CSigAtomic | 21 | 0.03 | 749| | pure/uniform/CLLong | 43 | 0.03 | 1554| | pure/uniform/CULLong | 42 | 0.03 | 1505| | pure/uniform/CIntPtr | 43 | 0.03 | 1476| | pure/uniform/CUIntPtr | 42 | 0.03 | 1463| | pure/uniform/CIntMax | 43 | 0.03 | 1535| | pure/uniform/CUIntMax | 42 | 0.03 | 1493| Dominic Steinitz dominic(a)steinitz.org http://idontgetoutmuch.org Twitter: @idontgetoutmuch

11 21