That's very different since it changes how desugaring works. With NoImplicitPrelude desugaring uses unqualified names. It's a rather fragile pragma. -----Original Message----- From: Erik Hesselink [mailto:hesselink@gmail.com] Sent: 27 January 2015 11:11 To: Augustsson, Lennart Cc: libraries@haskell.org Subject: Re: Drastic Prelude changes imminent On Tue, Jan 27, 2015 at 11:32 AM, Augustsson, Lennart wrote:

* Add a new pragma {-# LANGUAGE Prelude=AlternativePrelude #-} * Semantics is name-space only: import Prelude (); import AlternativePrelude

I don't have any opinion on the full proposal yet, but I usually use "{-# LANGUAGE NoImplicitPrelude #-}" when using an alternate prelude. Is there any different between that and "import Prelude ()"? Erik This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at http://www.standardchartered.com/en/incorporation-details.html Insofar as this communication contains any market commentary, the market commentary has been prepared by a sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied for any other purpose, and is subject to the relevant disclaimers available at http://wholesalebanking.standardchartered.com/en/utility/Pages/d-mkt.aspx. Please visit http://wholesalebanking.standardchartered.com/en/capabilities/financialmarke... for important information with respect to derivative products.

On 27/01/2015 11:26, Augustsson, Lennart wrote:

That's very different since it changes how desugaring works. With NoImplicitPrelude desugaring uses unqualified names. It's a rather fragile pragma.

You're thinking of RebindableSyntax. NoImplicitPrelude has no effect on desugaring. Cheers, Simon

-----Original Message----- From: Erik Hesselink [mailto:hesselink@gmail.com] Sent: 27 January 2015 11:11 To: Augustsson, Lennart Cc: libraries@haskell.org Subject: Re: Drastic Prelude changes imminent

On Tue, Jan 27, 2015 at 11:32 AM, Augustsson, Lennart wrote:

...

* Add a new pragma {-# LANGUAGE Prelude=AlternativePrelude #-} * Semantics is name-space only: import Prelude (); import AlternativePrelude

I don't have any opinion on the full proposal yet, but I usually use "{-# LANGUAGE NoImplicitPrelude #-}" when using an alternate prelude. Is there any different between that and "import Prelude ()"?

Erik

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at http://www.standardchartered.com/en/incorporation-details.html

Insofar as this communication contains any market commentary, the market commentary has been prepared by a sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied for any other purpose, and is subject to the relevant disclaimers available at http://wholesalebanking.standardchartered.com/en/utility/Pages/d-mkt.aspx.

Please visit http://wholesalebanking.standardchartered.com/en/capabilities/financialmarke... for important information with respect to derivative products. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

That doesn't really sound feasible, as there would be name clashes when importing both the Prelude (with generalized functions) and Data.List (with monomorphic functions). The only way to get generalized functions and minimize breakage is to also generalize Data.List. Erik On Tue, Jan 27, 2015 at 5:03 PM, Johan Tibell wrote:

Looking at the generalizations in Data.List I find them pretty odd now when I think about it. I'd expect Data.List functions to be monomorphic to lists, just like I expect functions in Data.Map to be monomorphic to maps. Now there might be generalized versions of these functions in e.g. the Prelude, but generalizing Data.List means that I don't even have the monomorphic versions available if I want to resolve ambiguity by using them*.

* It turns out resolving ambiguity is pretty annoying. The type signatures are awkward to write so I end having to write something akin to

mymap :: (a -> b) -> [a] -> [b] mymap = map

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

I don't see why from your email, but my only point was that having both polymorphic versions of functions in the Prelude, and monomorphic ones in Data.List, will probably be worse than either of the other two options (the fully polymorphic version being prepared, and the status quo you want to keep). Erik On Tue, Jan 27, 2015 at 5:10 PM, Augustsson, Lennart wrote:

But having Data.List export a non-list version of foldr is a terrible idea.

-----Original Message----- From: Erik Hesselink [mailto:hesselink@gmail.com] Sent: 27 January 2015 16:07 To: Johan Tibell Cc: Augustsson, Lennart; libraries@haskell.org Subject: Re: Drastic Prelude changes imminent

That doesn't really sound feasible, as there would be name clashes when importing both the Prelude (with generalized functions) and Data.List (with monomorphic functions). The only way to get generalized functions and minimize breakage is to also generalize Data.List.

Erik

On Tue, Jan 27, 2015 at 5:03 PM, Johan Tibell wrote:

...

Looking at the generalizations in Data.List I find them pretty odd now when I think about it. I'd expect Data.List functions to be monomorphic to lists, just like I expect functions in Data.Map to be monomorphic to maps. Now there might be generalized versions of these functions in e.g. the Prelude, but generalizing Data.List means that I don't even have the monomorphic versions available if I want to resolve ambiguity by using them*.

* It turns out resolving ambiguity is pretty annoying. The type signatures are awkward to write so I end having to write something akin to

mymap :: (a -> b) -> [a] -> [b] mymap = map

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at http://www.standardchartered.com/en/incorporation-details.html

Insofar as this communication contains any market commentary, the market commentary has been prepared by a sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied for any other purpose, and is subject to the relevant disclaimers available at http://wholesalebanking.standardchartered.com/en/utility/Pages/d-mkt.aspx.

Please visit http://wholesalebanking.standardchartered.com/en/capabilities/financialmarke... for important information with respect to derivative products.

There's precedence for this, I think: Data.Map, Data.Vector, Data.List all export monomorphic functions. It's just we don't import Data.List as L because lists are given special treatment by Haskell 2010. It makes sense for Prelude to be polymorphic to me.

On 2015-01-27 at 17:31:39 +0100, Erik Hesselink wrote: [...]

The only thing I can think of is to generalize Data.List when generalizing the prelude, but also introduce e.g. Data.List.Mono which has all the monomorphic variants.

I moved the original `Data.List` into `Data.OldList` early on during the refactoring, and it was exposed for some time to allow packages like haskell98/2010 to re-export the list-specialised versions. However, since the haskell98/2010 packages were dropped from GHC 7.10 (as implementing for swapping out the prelude would have required more effort to do properly), `Data.OldList` was hidden from `base`. We could still expose it again as `Data.List.Mono` (or some other name) if it helps anybody. For details see also http://git.haskell.org/ghc.git/commitdiff/e888b943396c21db74ba2fc69bf3a89e24... -- hvr

I'm trying to understand why people with (serious, legitimate) concerns about BBP didn't come out of the woodwork until RC2. I fear it's likely that nothing can be done to change this until 7.12 in any case. For the record, I'm not a huge fan of the Foldable abstraction myself; I just don't think now is a good time for that discussion. On Tue, Jan 27, 2015 at 11:35 AM, Yitzchak Gale wrote:

First of all, I am very strongly in favor of Lennart's proposal of slowing down the BBP.

This discussion is now moving in the direction of changes or alternatives to BBP itself. That is a worthwhile discussion, but it's not practical until we first come to a conclusion about Lennart's original proposal. For now, what is on the table is to find a way to swap BBP, exactly as it is now, in or out using some simple mechanism like a pragma.

The proposed syntax LANGUAGE Prelude=AlternativePrelude is nice, but the difficulty in finding a Proposal 3 shows that this syntax is not good enough. We need a combined Proposal 1 and Proposal 3 that works something like this:

The base package comes with two versions of each of the affected modules. Something like this:

Prelude.Traditional Prelude.Alternative

Data.List.Traditional Data.List.Alternative

etc.

Then we need some simple mechanism - a LANGUAGE pragma, or whatever - that swaps all at once which of the two sets are re-exported by the standard modules.

-Yitz _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

I think that _if_ there had been a better organized, more clarifying discussion on the scope of BBP earlier (one was promised after Neil raised concerns, but I think it fell by the wayside), then we might not be in this boat. Now, whatever the “right” thing is eventually, it does feel like we need to apply brakes until people can settle in further. I know people will be disappointed no matter what happens, and they will feel this stuff has moved slowly enough for long enough. However, without having had that discussion, I don’t see how we can move forward. The proposal Lennart and Simon have therefore seems good and necessary, save for the issue of how BBP infects Data.List in particular as well.  I think we should go with it, and _also_ just have Data.List always export the monomorphic functions for now. I recognize that this means even with AlternativePrelude on, people will not get the full niceness of the “BBP experience”. However, it seems as close as we can get, to my mind, without creating greater headaches and issues all around. We could perhaps also insert a warning encouraging people to import Data.List qualified, either in this GHC of the next release. This would allow BBP to proceed in a relatively breakage-free way in the future without needing to generalize the Data.List module. Cheers, Gershom On January 27, 2015 at 12:14:03 PM, Augustsson, Lennart (lennart.augustsson@sc.com) wrote:

Perhaps because a lot of people didn't know about it? Simon PJ didn't know until we told him on Friday, so I don't think the change has been that well advertised.

-----Original Message----- From: Libraries [mailto:libraries-bounces@haskell.org] On Behalf Of David Feuer Sent: 27 January 2015 17:07 To: Yitzchak Gale Cc: libraries@haskell.org Subject: Re: Drastic Prelude changes imminent

I'm trying to understand why people with (serious, legitimate) concerns about BBP didn't come out of the woodwork until RC2. I fear it's likely that nothing can be done to change this until 7.12 in any case.

For the record, I'm not a huge fan of the Foldable abstraction myself; I just don't think now is a good time for that discussion.

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at http://www.standardchartered.com/en/incorporation-details.html

Insofar as this communication contains any market commentary, the market commentary has been prepared by a sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied for any other purpose, and is subject to the relevant disclaimers available at http://wholesalebanking.standardchartered.com/en/utility/Pages/d-mkt.aspx.

Please visit http://wholesalebanking.standardchartered.com/en/capabilities/financialmarke... for important information with respect to derivative products. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

If memory serves, what became the BBP was initially introduced at the Haskell Implementors' Workshop in Boston in 2013. (Or, at least, that's when I became aware of it.) I know I'm a special case of a Haskell user, but I've seen mention of the BBP aplenty. However, I do think we should learn from this experience and perhaps establish a very-low-traffic list where we announce breaking language changes. As for the BBP itself, my 1¢ is that I've never liked it much, but I don't like the status quo either. (Too much `hiding` and `qualified`!) Having neither a good alternative nor time/energy to develop one has kept me relatively quiet in the discussion, and is why I'm offering only 1¢ right now instead of 2¢. (My chief concern is how it affects newcomers to Haskell, which is admittedly a different concern than others have expressed. It may be worth noting that my concern was addressed by suggesting newcomers use the `haskell2010` package... which is now defunct, due to fallout from the AMP.) As a much more minor point, I believe

{-# LANGUAGE NoImplicitPrelude #-} and import Prelude () should have different behavior. Doesn't the latter import any Prelude instances? Might these be different (and conflicting with!) instances in an alternate prelude? So, I think we must mimic the pragma version as opposed to the empty import list.

Richard On Jan 27, 2015, at 12:27 PM, Brandon Allbery wrote:

On Tue, Jan 27, 2015 at 12:25 PM, Gershom B wrote: I think that _if_ there had been a better organized, more clarifying discussion on the scope of BBP earlier (one was promised after Neil raised concerns, but I think it fell by the wayside), then we might not be in this boat.

Agreed. I knew this was coming from various mentions, but there was not a good overview of what was going on and what effects it would have until this thread. This makes me think that it might be wise to back off until 7.12.

-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

A lot of people might have opinions about what is good for beginners, but the only opinions that matter are from those who have actually taught a large number of beginners. I'm not one of them, so I'll keep my beginner opinions to myself. -----Original Message----- From: Libraries [mailto:libraries-bounces@haskell.org] On Behalf Of Neil Mitchell Sent: 27 January 2015 17:44 To: Richard Eisenberg Cc: cvs-libraries@haskell.org Subject: Re: Drastic Prelude changes imminent

now instead of 2¢. (My chief concern is how it affects newcomers to Haskell, which is admittedly a different concern than others have expressed. It may be worth noting that my concern was addressed by suggesting newcomers use the `haskell2010` package... which is now defunct, due to fallout from the AMP.)

The haskell2010 package has always been unusable. Initially haskell98 reexposed a subset of base, and they worked together. Then some of the modules were cloned and tweaked, and any time I put them in the same package it all went terribly wrong. I also have concerns about beginners, but it's clear everyone has radically different opinions about what is good or bad for beginners. I raised my concerns as soon as I found out about the changes (October 2014), argued my case, made a few blog posts, but seemingly lost. At that point, much like Simon Marlow, I decided to live with it. I have a feeling we might rehash the same discussion every time a new larger set of people find out about the changes, especially as those people are less likely to be connected to the cutting edge of libraries development. Thanks, Neil _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at http://www.standardchartered.com/en/incorporation-details.html Insofar as this communication contains any market commentary, the market commentary has been prepared by a sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied for any other purpose, and is subject to the relevant disclaimers available at http://wholesalebanking.standardchartered.com/en/utility/Pages/d-mkt.aspx. Please visit http://wholesalebanking.standardchartered.com/en/capabilities/financialmarke... for important information with respect to derivative products.

Augustsson, Lennart wrote

A lot of people might have opinions about what is good for beginners, but the only opinions that matter are from those who have actually taught a large number of beginners. I'm not one of them, so I'll keep my beginner opinions to myself.

When I was a beginner, I don't recall having trouble understanding the polymorphic versions, but I was soundly confounded by the apparent redundancy of map/fmap etc. Particularly when the answer was "to make things easier for beginners"! -- View this message in context: http://haskell.1045720.n5.nabble.com/Drastic-Prelude-changes-imminent-tp5764... Sent from the Haskell - Libraries mailing list archive at Nabble.com.

+1 On Tue, Jan 27, 2015 at 2:06 PM, Christopher Allen wrote:

This has been the experience of many (not all) beginners I've worked with as well.

Confusion with redundancy more often than the concept embedded in the polymorphic version. It should be said that Functor is usually a titch easier to learn for new people than Foldable or Traversable even if they're not complicated to experienced Haskellers.

Another problem is that most books for learning Haskell don't incorporate much of the typeclasses under discussion, partly because of how new they are.

I don't think these caveats are a good reason to block or mutilate the BBP.

--- Chris

On Tue, Jan 27, 2015 at 12:57 PM, harry wrote:

...

Augustsson, Lennart wrote

...

A lot of people might have opinions about what is good for beginners, but the only opinions that matter are from those who have actually taught a large number of beginners. I'm not one of them, so I'll keep my beginner opinions to myself.

When I was a beginner, I don't recall having trouble understanding the polymorphic versions, but I was soundly confounded by the apparent redundancy of map/fmap etc. Particularly when the answer was "to make things easier for beginners"!

-- View this message in context: http://haskell.1045720.n5.nabble.com/Drastic-Prelude-changes-imminent-tp5764... Sent from the Haskell - Libraries mailing list archive at Nabble.com. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Tue, 27 Jan 2015, Gershom B wrote:

We could perhaps also insert a warning encouraging people to import Data.List qualified, either in this GHC of the next release.

+10 for qualification. Btw. you can already get a warning for every import using the option: -fwarn-missing-import-lists

On Tue, 27 Jan 2015, Herbert Valerio Riedel wrote:

Ironically, the AMP-related warnings in GHC 7.8 weren't very effective at getting noticed timely in order to have Hackage ready for the actual AMP-changes occurring in GHC 7.10. By far the most breakages[1] I encountered (and reported) over the last weeks in popularly-depended-upon packages were due to missing Functor/Applicative instances. It'd be interesting to know why those warnings weren't acted upon during 2014.

Otoh, I don't have numbers how many packages were actually fixed up *thanks* to the GHC 7.8 warnings...

I have added Applicative instances whereever suggested by GHC-7.8, but I have not recompiled all my packages since then and I have not released all updated packages. Maybe a year is just too short?

Hi, Simon PJ mentions the Core Library committee in this context, but – being a member of that – I must admit that even I were not aware of the changes until they were underway. After I saw the commits coming in I searched for information on the proposal, but only found the ticket #9586 titled “Implement Traversable/Foldable-Burning-Bridges Proposal”, which made me expect some actual proposal somewhere usually in the wiki, but it seems there was none. There is a thread on libraries on Burning Bridges from May 2013, but it was not clear to me that that turned into something concrete. I concluded that I simply missed some discussion somewhere but everything went it’s orderly way. I was uneasy about these changes, but if in doubt I trust those who invest their time. Somewhere in the current thread it was mentioned that it was at the Haskell Implementors Workshop 2013 where this plan was created. I was not there, so that’d explain why I wasn’t aware of the changes at that time. What lessens can we learn from the BBP process? * Personal meetings are great, but please remember to actively communicate results. * Proposal wiki pages, with motivation, examples, alternatives, have served us well. I’m not sure why that was skipped here. We should have them in the future. Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

We started work on the Foldable/Traversable issue during 7.8, right after the BBP process burned down and it was clear there was a strong desire to see motion on that topic, and the core libraries committee was formed, but decided not to act on anything during 7.8, as 7.8's release seemed immanent at the time. Little did we know it'd drag on for another 7 months. Overall, that proved to be a good thing, as we spent much of that time trying to understand how to retie the Gordian knot that is the set of hs boot files that makes up base. But there is definitely a process concern. We've now had proposals that 'passed' a year ago or six months ago, that we're able now to implement, now that we finally have folks in place that know how to work through the existing process. Most of these are small fixes, broadly agreed upon. Do we make someone who already jumped through all the hoops to get a coherent proposal through the process run and jump through them all again? Once 7.10 started we set to work in earnest. A large part of the committee work that has happened in 7.10 has been un-log-jamming issues that have been stuck in trac on the libraries@ process with or without a patch. Some of the bugs that had made it into trac had languished for years. Most of the complications in the current design came about because of things that only became obvious once the framework was in place. I do agree that it is quite difficult to keep abreast of all that is happening in GHC. Part of that is a consequence of having an order of magnitude more cooks in the kitchen these days. At the same time you have core libraries work going on, we have a bunch of independent work going on on the front of how to deal with SIMD instructions, swizzling, folks who care deeply about the fate of DPH, a ton of exciting work on the cabal side that centers around how to deal with finally solving 'cabal hell', complications induced by considering the impact of Kilpatrick and Yang's work on Backpack, work on the RTS and garbage collector, etc. There are a lot more people active now. There were something like 40+ people actively contributing code at the beginning of last year, and that number can only have grown since. When it was just the two Simons and whatever couple of graduate students happened to stumble in, things proceeded at a much more gentle pace. But with Simon Marlow moving on to facebook, a lot of folks decided to pitch in to pick up the slack. It is easy to keep track of the head-space of a smaller pool of contributors. A whole new toolchain has been built up around using phabricator for code review that has greatly improved the quality of the patches going into GHC. This has helped a great deal, but if you think the libraries@ process is hard to keep up with, well, there have been several hundred phabricator patch reviews since the process started last year, mostly dealing with the minutiae of how to integrate pre-existing trac tickets raised by libraries@ proposals or flat out ghc bug reports. ;) We absolutely do need to work out more effective ways to communicate what is going on. There will be growing pains, there will be surprises, but we are all pulling towards a common goal, that of making GHC be the best compiler it can be for Haskell, and making continuing to strive towards making Haskell be the best version of itself that it can be. -Edward On Wed, Jan 28, 2015 at 3:09 PM, Ben Gamari wrote:

Joachim Breitner writes:

...

Hi,

snip

...

* Proposal wiki pages, with motivation, examples, alternatives, have served us well. I’m not sure why that was skipped here. We should have them in the future.

Perhaps just as important is to maintain a Wiki page listing the proposals being considered by the CLC as well as the release milestones for adopted proposals. As someone who followed the BBP proposal fairly carefully in 2013, I was also a bit surprised when patches started showing up this cycle.

Cheers,

- Ben

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

AMP started at the same time as the BBP, both were started by threads of roughly the same size, with the same level of active participation. The major difference is no warnings were added for the Foldable/Traversable proposal as at the time, generalizing types was seen as pretty simple. e.g. foldMap comes into scope without a warning in 7.10 due to BBP, but then (<*>) also comes into scope in the AMP and we didn't warn for that, either. I do fully agree that the BBP could have gone in a bit cleaner if we'd realized the full scope of the issues we would face during the 7.8 timeline. Most of the hiccups only came clear after Herbert had spent months refactoring the internals of base to even make the change possible. Just an aside, though, it does appear objectively that those warnings actually did very little good in terms of getting folks to fix their code. By far the vast majority of the 7.10 related breakage we've seen has come from folks who still don't have Applicative instances for their Monad. -Edward On Wed, Jan 28, 2015 at 4:20 PM, Joachim Breitner wrote:

Hi,

Am Mittwoch, den 28.01.2015, 15:45 -0500 schrieb Edward Kmett:

...

We absolutely do need to work out more effective ways to communicate what is going on.

I’d like to point out that the AMP proposal and process can serve as a positive example. It is a change of comparable impact, but has sired much less noise. My impression was that there were more people aware of it, and sooner in the process, and that has helped.

Maybe every possibly far-reaching change should be advertised with a comic from now on? https://ro-che.info/ccc/21 :-)

Greetings, Joachim

...

-- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On 2015-01-28 at 21:09:33 +0100, Ben Gamari wrote: [...]

Perhaps just as important is to maintain a Wiki page listing the proposals being considered by the CLC as well as the release milestones for adopted proposals. As someone who followed the BBP proposal fairly carefully in 2013, I was also a bit surprised when patches started showing up this cycle.

As a side-note: personally, I became aware of the core-lib-committee's plan of targetting AMP + FTP for GHC 7.10 through the HaskellCast[1] with Edward back in 2013, in which Edward talks about the CLC's roadmap for the upcoming future major GHC releases. Maintaining a written representation of such information on a Wiki or similiar is obviously something the CLC should seriously take into consideration in the future... :-) Cheers, hvr [1]: http://youtu.be/6GNDzrgFhGM?t=10m25s

I have gone ahead and created a wiki page with a draft FAQ on the Foldable/Traversable proposal: https://wiki.haskell.org/Foldable_Traversable_In_Prelude I've attempted to hit the highlights that I can think of, but I'm sure there's stuff either wrong or missing, or perhaps that could just be marked up better. Please, everyone, feel free to expand, clarify, or make minor or formatting edits as you desire. If you feel there is something substantial that is unaddressed (especially of the form "Why not X?", please email me off-list and I'll try to work to cook up a proper addition to the page. Or, if you have any edits that you think should happen but aren't equipped to edit the wiki at the moment, I'l try to act on that too :-) Hopefully, this can serve as a resource not only for the current discussion, but to point people to regarding the final form of the proposal as it actually takes shape in GHC 7.10. --Gershom On Wed, Jan 28, 2015 at 11:02 AM, Gershom B wrote:

On January 28, 2015 at 5:57:56 AM, Simon Peyton Jones ( simonpj@microsoft.com) wrote:

...

...

Just to be clear, this isn't my proposal: it's from Lennart and Neil. They did come to see me; I had indeed entirely missed the fact that the Prelude was changing so much; I did encourage them to explicitly and clearly air their concerns. But I don't feel ready to take a personal position on how best to move forward.

Thanks for the clarity Simon. Apologies for contributing to noise by a sloppy reading of earlier messages in this thread.

I’ve been advocating that it is important that a FAQ be written up on the BBP to make the maze of (careful) choices more understandable to a lay audience. That will at least help the discussion to not repeatedly cover the same ground. If nobody else manages to step in and do it, I’ll see what I can get done myself in the next few days.

Cheers, Gershom

...

I very much value what the Core Libraries Committee does. Good libraries are a foundational component of a language and, although it's not very glamourous, working on making the libraries (particularly the core ones) work smoothly is incredibly valuable. So we all owe huge thank you to the Core Libraries Committee.

It's unfortunate that all this has arisen so late in the 7.10 release cycle. Perhaps I should have been paying more attention. Perhaps the CLC should have published the proposal, and with clearer signposting. Perhaps Lennart and Neil should have been more vocal earlier. But it's unproductive to discuss who to blame, and we probably all carry some responsibility. (We might want to learn some lessons for the future, though; one being that changes to the Prelude need particular care and advertising.)

More important is what to do next. Let's talk about how to resolve this during this week. A small group of us (including Edward K and Lennart) plan to have a Skype call on Monday. But the more we have a clear set of alternatives, the better.

Simon

On Tue, 27 Jan 2015, Augustsson, Lennart wrote:

Perhaps because a lot of people didn't know about it? Simon PJ didn't know until we told him on Friday, so I don't think the change has been that well advertised.

When BBP was discussed my impression was that many people affected by the proposal did not take part in the discussion. E.g. I was concerned about Haskell beginners and they are certainly not subscribed to libraries@haskell.org.

There were several fixes offered: 1.) Data.List could cease to re-export Prelude types This would work, but it would cause code breakage. 2.) Data.List could export monomorphic types, while the Prelude types could be generalized. Now everyone who imports Data.List would get conflicts with Prelude types, causing code breakage. 3.) Data.List could continue to needlessly re-export Prelude types but have those generalized to match the Prelude. This is the only option where existing code more or less universally continues to work. We can't actually deprecate a re-export at present, so we don't have a path to #1 at the moment, but it'd be a more ideal situation in the long run. -Edward On Tue, Jan 27, 2015 at 11:03 AM, Johan Tibell wrote:

Looking at the generalizations in Data.List I find them pretty odd now when I think about it. I'd expect Data.List functions to be monomorphic to lists, just like I expect functions in Data.Map to be monomorphic to maps. Now there might be generalized versions of these functions in e.g. the Prelude, but generalizing Data.List means that I don't even have the monomorphic versions available if I want to resolve ambiguity by using them*.

* It turns out resolving ambiguity is pretty annoying. The type signatures are awkward to write so I end having to write something akin to

mymap :: (a -> b) -> [a] -> [b] mymap = map

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

The current step was to generalize them so that code that imported Data.List so that they don't conflict with the Prelude versions. The intent was that once we gained the ability to deprecate re-exports that we'd remove them entirely. We now have a patch that enables us to do such things. It turns out that if you look at hackage, for the most part folks are actually importing Data.List unqualified to bring into scope foldl' or sort. Almost nobody invokes foldr from it. The "generalization and subsequent re-export deprecation" path breaks less code than the "leaving them monomorphic and breaking everyone who uses Data.List unqualified" path, even if it leads to an intermediate state that is a bit awkward, and doesn't lead easily to a final state where monomorphic versions of those combinators are available. -Edward On Wed, Feb 4, 2015 at 11:13 AM, Christopher Done wrote:

...

Looking at the generalizations in Data.List I find them pretty odd now when I think about it. I'd expect Data.List functions to be monomorphic to lists, just like I expect functions in Data.Map to be monomorphic to maps. Now there might be generalized versions of these functions in e.g. the Prelude, but generalizing Data.List means that I don't even have the monomorphic versions available if I want to resolve ambiguity by using them*.

Sums up my feelings exactly. Data.Map, Data.List, etc. should be monomorphic. Adding generalized functions in Data.List is a little baffling. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

The original intent had been to have a discussion about deprecating the generalized combinators in Data.List in 7.12 and removing them in 7.14 once we had the ability to deprecate a re-export. This path gives a 2 year window for the much smaller pool of users who import Data.List qualified and also use List.foldr to drop "List." from their code, and a way to gracefully provide warnings, without immediately breaking the larger pool of users who import Data.List unqualified and just reach for members that aren't in the Prelude and rely on the lack of conflicts today. I don't see a path that results in monomorphic members of Data.List for those particular combinators that doesn't require a sharp shock in which a bunch of code breaks for a large pool of users without us being able to warn them first. -Edward -Edward On Wed, Feb 4, 2015 at 12:05 PM, Johan Tibell wrote:

On Wed, Feb 4, 2015 at 9:01 AM, Edward Kmett wrote:

...

The current step was to generalize them so that code that imported Data.List so that they don't conflict with the Prelude versions.

The intent was that once we gained the ability to deprecate re-exports that we'd remove them entirely. We now have a patch that enables us to do such things.

It turns out that if you look at hackage, for the most part folks are actually importing Data.List unqualified to bring into scope foldl' or sort. Almost nobody invokes foldr from it.

The "generalization and subsequent re-export deprecation" path breaks less code than the "leaving them monomorphic and breaking everyone who uses Data.List unqualified" path, even if it leads to an intermediate state that is a bit awkward, and doesn't lead easily to a final state where monomorphic versions of those combinators are available.

I didn't quite follow this. Does this mean that the Data.List functions will be generalized for now but eventually monomorphized again?

...

-Edward

On Wed, Feb 4, 2015 at 11:13 AM, Christopher Done wrote:

...

...

Looking at the generalizations in Data.List I find them pretty odd now when I think about it. I'd expect Data.List functions to be monomorphic to lists, just like I expect functions in Data.Map to be monomorphic to maps. Now there might be generalized versions of these functions in e.g. the Prelude, but generalizing Data.List means that I don't even have the monomorphic versions available if I want to resolve ambiguity by using them*.

Sums up my feelings exactly. Data.Map, Data.List, etc. should be monomorphic. Adding generalized functions in Data.List is a little baffling. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Again, the intention was to *have a discussion* about doing so early in 7.12. * Keeping foldr, etc. in Data.List in 7.10 monomorphic in the face of a generalized Prelude would break an enormous amount of code. It would force users who currently do not use qualified imports to do so and increase conflicts over all. As it is, even Neil and Lennart on the agreed on a call the other day that the Foldable/Traversable changes break almost no code. * Keeping those combinators in there and generalized indefinitely breaks basically no code, but leads to flare ups like the current one where folks look at the illogical-seeming intermediate state and get upset. * Removing those generalized, redundant combinators after a deprecation cycle gives a path by which in the annoyance at the state of affairs in the second option can fade, admittedly at the expense that there is no place for the monomorphic versions to live. This gives two years for folks to adapt a *much* smaller set of code, replete with smoothly handleable warnings. * By way of contrast, just reverting them to be monomorphic in 7.12 or 7.14 would break just as much code a year or two later. It is an option folks could advocate for, but it is an option that has no smooth upgrade path from the status quo, making it a very awkward option to enact at best. It has the benefit that the monomorphic versions of those combinators would have a place to live, but getting there requires much higher transitional pain in the absence of language extensions to make it possible. Now, one could envision a language pragma or the like that would enable us to go down this path. e.g. Lennart's WEAK pragma or by crafting a technical solution whereby you can export with restricted signatures. In the short term we took such technical fixes off the table as far too messy to address the issue, and when such proposals have come up in the past they've been fairly controversial. So of those four options, I'd prefer to go down the third path in the absence of folks agreeing on extending the language in ways to make the fourth option viable, but there is definitely room for debate. -Edward On Wed, Feb 4, 2015 at 12:42 PM, Johan Tibell wrote:

On Wed, Feb 4, 2015 at 9:22 AM, Edward Kmett wrote:

...

The original intent had been to have a discussion about deprecating the generalized combinators in Data.List in 7.12 and removing them in 7.14 once we had the ability to deprecate a re-export.

This path gives a 2 year window for the much smaller pool of users who import Data.List qualified and also use List.foldr to drop "List." from their code, and a way to gracefully provide warnings, without immediately breaking the larger pool of users who import Data.List unqualified and just reach for members that aren't in the Prelude and rely on the lack of conflicts today.

Wait, are we removing foldr from Data.List entirely?

...

I don't see a path that results in monomorphic members of Data.List for those particular combinators that doesn't require a sharp shock in which a bunch of code breaks for a large pool of users without us being able to warn them first.

-Edward

-Edward

On Wed, Feb 4, 2015 at 12:05 PM, Johan Tibell wrote:

...

On Wed, Feb 4, 2015 at 9:01 AM, Edward Kmett wrote:

...

The current step was to generalize them so that code that imported Data.List so that they don't conflict with the Prelude versions.

The intent was that once we gained the ability to deprecate re-exports that we'd remove them entirely. We now have a patch that enables us to do such things.

It turns out that if you look at hackage, for the most part folks are actually importing Data.List unqualified to bring into scope foldl' or sort. Almost nobody invokes foldr from it.

The "generalization and subsequent re-export deprecation" path breaks less code than the "leaving them monomorphic and breaking everyone who uses Data.List unqualified" path, even if it leads to an intermediate state that is a bit awkward, and doesn't lead easily to a final state where monomorphic versions of those combinators are available.

I didn't quite follow this. Does this mean that the Data.List functions will be generalized for now but eventually monomorphized again?

...

-Edward

On Wed, Feb 4, 2015 at 11:13 AM, Christopher Done wrote:

...

...

Looking at the generalizations in Data.List I find them pretty odd now when I think about it. I'd expect Data.List functions to be monomorphic to lists, just like I expect functions in Data.Map to be monomorphic to maps. Now there might be generalized versions of these functions in e.g. the Prelude, but generalizing Data.List means that I don't even have the monomorphic versions available if I want to resolve ambiguity by using them*.

Sums up my feelings exactly. Data.Map, Data.List, etc. should be monomorphic. Adding generalized functions in Data.List is a little baffling. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Tue, Jan 27, 2015 at 3:18 PM, Eric Mertens wrote:

I'm also uncomfortable with reducing Foldable to a mere "toList" method as I find that that solution ends up being too "left-biased" There might be a discussion to be had about what is or isn't in Foldable, but I don't think it's anything blocking.

As one of the people who was involved in making some of these changes, I should warn you that things are still quite biased in certain ways. Some things, like concatMap, lack really sane unbiased defaults.

On Tue, 27 Jan 2015, Malcolm Wallace wrote:

I do think that the current social mechanisms for changing the Haskell base library are less than ideal. It sometimes feels like someone can send a message to a mailing list with a "bright idea", and so long as only a few people see it, and those that do see it do not object, the proposal is automatically accepted. That would be fine for any library _except_ base, which I think ought to be much more stable.

Maybe we should try a voting scheme that entirely relies on rejection rather than acceptance. In such a scheme the solution that causes the minimal overall resistance would win.

I think that all breaking changes that are approved on this mailing list should be migrated to a much more visible Github issue (i.e. https://github.com/ghc/ghc) before final approval. You can still do all actual pull requests and coding via Phabricator, but the discussion should be on Github, because the barrier for new people to join in on the discussion is lower. You will get a lot more discussion and visibility into these changes purely by using Github as the central forum. This mailing list is not an appropriate forum for discussion for several reasons: A) The volume of email is too high. Only people who are experts at managing their inbox subscribe here. B) There is too little signal to noise (for a beginner). Why would a beginner want to join a mailing list that debates such dry topics as whether or not to add Data.Intersperse.sequence? C) There's no (easy) way to subscribe to (or mute) a particular thread (unlike Github where you can easily watch or mute specific issues) Pretty much everybody on this mailing list is a die-hard Haskell expert and that's not representative of the Haskell community at large. Migrating important topics to Github will give us a more representative sampling of the community and will also help centralize the discussion better. On 1/27/15, 12:54 PM, Johan Tibell wrote:

A point about process: I've been aware of this change for quite some time but I wasn't aware of the details (e.g. the changes to Data.List). It's easier to judge proposals like this if the complete list of changes are listed in some proposal page.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Whether or not we use Github, you have to admit that the development process for core libraries is not very transparent in its current form. We have evidence for this right now: lots of people have been complaining that they never realized that this change got into GHC-7.10. You don't have to adopt my specific suggestion, but you can't just keep doing business as usual either and pretend that there isn't a problem. If you reject my Github suggestion then propose a better solution for increasing the transparency and visibility of the process. On 1/27/15, 1:40 PM, David Feuer wrote:

Github is not and has never been a focus for GHC development. It seems a bit odd to separate things into "noise" on Trac and the mailing lists and, theoretically, "signal" on GitHub. I agree that important community discussions should not be happening on Phabricator, but they generally aren't anyway.

On Jan 27, 2015 4:09 PM, "Gabriel Gonzalez" mailto:gabriel439@gmail.com> wrote:

I think that all breaking changes that are approved on this mailing list should be migrated to a much more visible Github issue (i.e. https://github.com/ghc/ghc) before final approval. You can still do all actual pull requests and coding via Phabricator, but the discussion should be on Github, because the barrier for new people to join in on the discussion is lower. You will get a lot more discussion and visibility into these changes purely by using Github as the central forum.

This mailing list is not an appropriate forum for discussion for several reasons:

A) The volume of email is too high. Only people who are experts at managing their inbox subscribe here. B) There is too little signal to noise (for a beginner). Why would a beginner want to join a mailing list that debates such dry topics as whether or not to add Data.Intersperse.sequence? C) There's no (easy) way to subscribe to (or mute) a particular thread (unlike Github where you can easily watch or mute specific issues)

Pretty much everybody on this mailing list is a die-hard Haskell expert and that's not representative of the Haskell community at large. Migrating important topics to Github will give us a more representative sampling of the community and will also help centralize the discussion better.

On 1/27/15, 12:54 PM, Johan Tibell wrote:

...

A point about process: I've been aware of this change for quite some time but I wasn't aware of the details (e.g. the changes to Data.List). It's easier to judge proposals like this if the complete list of changes are listed in some proposal page.

_______________________________________________ Libraries mailing list Libraries@haskell.org mailto:Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org mailto:Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Could we not do both? Link to the Github issue in the mailing list post? The mailing lists aren't as visible to the wider community as you might think anyway. On Tue, Jan 27, 2015 at 3:59 PM, Henning Thielemann < lemming@henning-thielemann.de> wrote:

On Tue, 27 Jan 2015, Gabriel Gonzalez wrote:

Whether or not we use Github, you have to admit that the development

...

process for core libraries is not very transparent in its current form. We have evidence for this right now: lots of people have been complaining that they never realized that this change got into GHC-7.10.

You don't have to adopt my specific suggestion, but you can't just keep doing business as usual either and pretend that there isn't a problem. If you reject my Github suggestion then propose a better solution for increasing the transparency and visibility of the process.

We could warn about such a discussion in other mailing lists such as haskell, haskell-cafe, haskell-beginner, ghc-dev. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

We already have a process that works pretty well. Create a wiki page that specifies the implementation. Notifying every Haskell user is pointless if there is no specification for them to look at. I have no idea why the specification step was skipped for this change. On Tue, Jan 27, 2015 at 2:02 PM, Gabriel Gonzalez wrote:

On 1/27/15, 1:59 PM, Henning Thielemann wrote:

...

We could warn about such a discussion in other mailing lists such as haskell, haskell-cafe, haskell-beginner, ghc-dev.

That's a good start, but I'd prefer to also add non-email solutions for promoting visibility (i.e. social networks like Twitter, Google+, and Reddit). For example, most of the reason this issue got a lot of visibility was the post to /r/haskell earlier today.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Hello Gabriel, On 2015-01-27 at 22:53:35 +0100, Gabriel Gonzalez wrote:

Whether or not we use Github, you have to admit that the development process for core libraries is not very transparent in its current form. We have evidence for this right now: lots of people have been complaining that they never realized that this change got into GHC-7.10.

Fwiw, this is what I commented already a couple of months ago addressing a complaint that the changes were hidden from the public (although the complaint was then revised to refer to the decisions being non-transparent): http://www.reddit.com/r/haskell/comments/2if0fu/on_concerns_about_haskells_p...

You don't have to adopt my specific suggestion, but you can't just keep doing business as usual either and pretend that there isn't a problem. If you reject my Github suggestion then propose a better solution for increasing the transparency and visibility of the process.

Just a thought (I heard elsewhere): We already have "GHC Weekly News", what about "Haskell Libraries Weekly News" to cover the highlights (especially library proposal) of this mailing list (if we can find a volunteer to do the weekly write-up)?

On 1/27/15, 1:40 PM, David Feuer wrote:

[...]

On Tue, Jan 27, 2015 at 3:18 PM, Eric Mertens wrote:

I can certainly see the reasoning in wanting Data.List.foldr to actually be the list-specific one, and this would be consistent with other cases like Data.Map. Changing this might break some code, but it might be a more reasonable situation in the end. If we're going to consider this it would probably be better in its own thread so that it doesn't get drowned out by the rest of the BBP discussion.

The main reason this wasn't done is that we ran quick trial balloon to see which style broke less code. Looking through hackage for usage of Data.List, quite a bit of it is done unqualified. Monomorphizing Data.List combinators would break code that is written as import Data.List foo = ... foldr ... Removing them would break code that is written as import qualified Data.List as List foo = ... List.foldr ... Both of these styles were fairly common, though the former more so. In an ideal world we'd have never put them in Data.List in the first place, same with mapM etc. in Control.Monad, etc. and then something like Lennart's proposal here would have been viable. The main thing we can learn from this is that having modules that "conveniently" re-export stuff is a dangerous thing in terms of long term API design.

Am 27.01.2015 um 23:17 schrieb Edward Kmett:

The main thing we can learn from this is that having modules that "conveniently" re-export stuff is a dangerous thing in terms of long term API design.

I think that, too, for some time now. My conclusion so far was, that I should not assume that X.a is just re-exported Y.a when importing 'a' (maybe unless the equality is explicitly documented).

Lennart, On 2015-01-28 at 11:14:50 +0100, Augustsson, Lennart wrote:

I've updated the Wiki page (https://ghc.haskell.org/trac/ghc/wiki/BurningBridgesSlowly) with what I think would be a sensible course of actions:

[..] While I'm still in the process of thinking through the plans/proposals layed out on that wiki page, I'm somewhat missing what actual problem(s) those proposals are trying to solve. What's the actual problem/risk we're running into, if we instead go with a "Proposal 0" which consists in `base-4.8` as currently implemented in GHC 7.10.1RC2 going forward becoming part of GHC 7.10.1? It's not like having something implemented in GHC makes it magically become written into stone for the next Haskell Report. We've been already preparing Hackage for this change in `base` since even before GHC 7.10.1RC1 was released last year[1], with the implicit assumption that the release-candidate would be very representative of what GHC 7.10.1 is going to look like so Hackage upstream authors could adapt their packages with confidence. When RC1 came out, Michael was so kind to throw the Stackage-machinery at testing GHC 7.10 and informing even more package-authors in case their packages needed attention. Reverting BBP may cause additional work for those package authors that adapted their packages by removing now redundant imports to achieve perfect `-Wall`-hygiene. So what are the tangible problems with the current BBP? Code-breakage doesn't seem to be the issue here. To the contrary, you seem to rather suggest to actively cause a major compatibility breakage in order to do BBP "right" (as you consider generalising the signatures in `Data.List` to be "wrong"[2] -- it would help if you could elaborate more concretely what the actual problem is, so I can actually try to address that concern). -- hvr. [1]: Edward states in https://www.reddit.com/r/haskell/comments/2ttvsj/major_prelude_changes_propo... that 569 packages of the Stackage-set are building with GHC 7.10.1 just fine [2]: https://www.reddit.com/r/haskell/comments/2ttvsj/major_prelude_changes_propo...

Here is a rough idea for how monomorphic list functions could be handled 1) export monorphic list functions from a differently name module. One possibility is a `List` module (no `Data` prefix) 2) at some point (perhaps the next GHC release), have a deprecation warning for the Data.List module On Wed, Jan 28, 2015 at 5:56 AM, Augustsson, Lennart < Lennart.Augustsson@sc.com> wrote:

I see some problems:

* The changes of types to Data.List functions. This is very counter-intuitive. * I don't think the methods in Foldable are quite the right ones (e.g., sum, product should not be there). This would give the rest of us time to experiment with BBP since it would be very easy use (import Prelude(); import PreludeBBP). * Peace of mind. A lot of people (even some in the libraries committee) found these changes a surprise. A ghc warning about Data.List will ensure that everyone knows about them.

-----Original Message----- From: Herbert Valerio Riedel [mailto:hvr@gnu.org] Sent: 28 January 2015 13:46 To: Augustsson, Lennart Cc: libraries@haskell.org Subject: Re: Drastic Prelude changes imminent

Lennart,

On 2015-01-28 at 11:14:50 +0100, Augustsson, Lennart wrote:

...

I've updated the Wiki page (https://ghc.haskell.org/trac/ghc/wiki/BurningBridgesSlowly) with what I think would be a sensible course of actions:

[..]

While I'm still in the process of thinking through the plans/proposals layed out on that wiki page, I'm somewhat missing what actual problem(s) those proposals are trying to solve.

What's the actual problem/risk we're running into, if we instead go with a "Proposal 0" which consists in `base-4.8` as currently implemented in GHC 7.10.1RC2 going forward becoming part of GHC 7.10.1? It's not like having something implemented in GHC makes it magically become written into stone for the next Haskell Report.

We've been already preparing Hackage for this change in `base` since even before GHC 7.10.1RC1 was released last year[1], with the implicit assumption that the release-candidate would be very representative of what GHC 7.10.1 is going to look like so Hackage upstream authors could adapt their packages with confidence. When RC1 came out, Michael was so kind to throw the Stackage-machinery at testing GHC 7.10 and informing even more package-authors in case their packages needed attention.

Reverting BBP may cause additional work for those package authors that adapted their packages by removing now redundant imports to achieve perfect `-Wall`-hygiene.

So what are the tangible problems with the current BBP? Code-breakage doesn't seem to be the issue here. To the contrary, you seem to rather suggest to actively cause a major compatibility breakage in order to do BBP "right" (as you consider generalising the signatures in `Data.List` to be "wrong"[2] -- it would help if you could elaborate more concretely what the actual problem is, so I can actually try to address that concern).

-- hvr.

[1]: Edward states in

https://www.reddit.com/r/haskell/comments/2ttvsj/major_prelude_changes_propo...

that 569 packages of the Stackage-set are building with GHC 7.10.1 just fine

[2]: https://www.reddit.com/r/haskell/comments/2ttvsj/major_prelude_changes_propo...

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at http://www.standardchartered.com/en/incorporation-details.html

Insofar as this communication contains any market commentary, the market commentary has been prepared by a sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied for any other purpose, and is subject to the relevant disclaimers available at http://wholesalebanking.standardchartered.com/en/utility/Pages/d-mkt.aspx.

Please visit http://wholesalebanking.standardchartered.com/en/capabilities/financialmarke... for important information with respect to derivative products. _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Here is the seed of the current actions w.r.t. generalizing the combinators in Data.List. http://osdir.com/ml/libraries@haskell.org/2013-05/msg00324.html Notice how a few posts in I'd mentioned that if we could deprecate the export it'd yield a better final state which I still happen to believe. On Wed, Jan 28, 2015 at 12:35 PM, Edward Kmett wrote:

On Wed, Jan 28, 2015 at 5:14 AM, Augustsson, Lennart < Lennart.Augustsson@sc.com> wrote:

...

I've updated the Wiki page ( https://ghc.haskell.org/trac/ghc/wiki/BurningBridgesSlowly)

with what I think would be a sensible course of actions:

Simon Marlow rightly points out that adding a new LANGUAGE pragma is full of backwards compatibility problems.

So, here's a different plan how to introduce BBP.

* Leave Data.List type signatures and exports alone.

* Make ghc 7.10 warn when Data.List has been imported in a way that will break with BBP.

* No changes to Prelude signatures, but ship ghc 7.10 with PreludeBBP (which uses Foldable).

* Make PreludeBBP the default in ghc 7.12 (or later).

This will give us time to discuss exactly what should go into Foldable.

The signature changes in Control.Monad and other modules are unrelated to BBP and should be in ghc 7.10.

The ghc warning for importing Data.List should be as helpful as possible, something like:

Foo:5:1 Warning: Data.List imported unqualified, this will conflict with BBP.

Instead use 'import Data.List(maximumBy, sortBy)'

Upon reflection, I am prepared to seriously consider this approach.

There is an aspect of this proposal that I quite like.

Deprecating the re-exports in Data.List and Control.Monad is definitely a good thing in the long term.

There is, however, a question of whether it makes more sense to leave the monomorphic combinators in Data.List that collide with the Prelude at all after such a deprecation cycle. The warning you give by way of example there says that 'importing this module unqualified will conflict with the BBP' on the other hand, looking forward at the state that results in the least long term confusion, we might prefer to say that if we import the module explicitly listing a combinator that is already exported by the Prelude but which will be generalized that it will conflict with the BBP.

e.g. it is

import Data.List (foldr)

It seems to me that in the long term removing the redundant export entirely from base is the avenue that leads to the least confusion.

I've been eyeballing deprecating the re-exports of combinators from the mtl for a long time, but have lacked a mechanism in the language for expressing it.

Even if we don't implement the rest of the proposal, I would love to have the ability to express these things.

*However,* I'd like you to seriously consider an alternative viewpoint, which is that I strongly believe that what you are asking for actually forces considerably more work for more people over the course of the next year for few reasons:

1.) A large number of folks have been proactive in responding to the release candidates for 7.10.1 by updating their packages to be 7.10-ready. Several hundred packages have already been patched and uploaded to hackage, most of the packages in stackage are 7.10 ready today.

https://github.com/fpco/stackage/issues/378

A significant subset of those would be broken by your proposal. This means that there would now be hundreds of packages on hackage by our most active contributors with broken versions that need to be manually made uninstallable by an arcane process using the package management tools.

2.) An _alarming_ amount of code imports Control.Monad and then uses mapM or imports Data.List unqualified. This is why we didn't remove those combinators or ask for tools for deprecating them earlier in 7.10. The deprecation cycle you are proposing affects easily an order of magnitude more code than the proposed changes in 7.10, and worse it is often the kind of code that is written by those least prepared to understand what is happening. This is why above, I mentioned that it might be better to consider a version of the proposal where the explicit import of a combinator we'd like to remove as a re-export is flagged.

Ultimately the ability to deprecate a re-export would be useful as a way to mitigate future changes, even if we decided to let BBP roll on in 7.10 due to issue #1 above. We could also apply it to Data.List and the re-exported combinators in Control.Monad (and those in the mtl) to get to a state where future language changes can go through something much closer to the process you have envisioned in this article.

So as a concrete alternative proposal, I'd like to suggest that we consider:

Shipping the BBP as planned.

Adding to GHC the ability to complain if a given re-export is the only way a symbol finds its way into scope that you use, and apply those flags to Control.Monad.mapM and a bunch of combinators in Data.List.*.

This would build a path that would enable us to in the future enact release plans closer to what you have here, but would avoid forcing a huge pile of double-work on all the folks who have already adapted their code.

On the other hand, I'm happy to revisit through the libraries@ process if we want to head back to a 7.8-style Foldable. It strikes me, though, that the better time to do that would have been 3 months ago when we had long threads on the topic on the mailing list, rather than after the release candidate.

If we did revisit the current design, it wouldn't come without a cost: If we do that it would means that a number of key combinators in Data.Foldable will necessarily change their stack behavior as they fold in different directions currently than the Prelude combinators.

Adding the members enabled the [] instance to retain the current semantics, and allowed users who don't like the behavior of 'sum' and the like to fix it on a one-off basis on their own containers.

It seems that pushing for such a change needs to reach a much higher bar of acceptance, as it results in existing code yielding different answers with very different stack behavior.

-Edward

...

...

Just to be clear, this isn't my proposal: it's from Lennart and Neil.

The ghc warning for importing Data.List should be as helpful as possible, something like:

Foo:5:1 Warning: Data.List imported unqualified, this will conflict with BBP.

Instead use 'import Data.List(maximumBy, sortBy)'

Upon reflection, I am prepared to seriously consider this approach.

It should be noted that while I was there with Lennart when we talked about the problems, my viewpoints and Lennart's are not identical. I think the decision was insufficiently communicated (as seems to be universally agreed), I think it's probably a bad idea to ever make this change (I may well be in the minority here), and that the change was done in the wrong way (see http://neilmitchell.blogspot.co.uk/2014/10/how-to-rewrite-prelude.html for how I think it should have been approached). However, I'm not a fan of importing List qualified (I find that deeply ugly), and a warning that I can't disable compatibly in older version of GHC without CPP is pretty annoying - the GHC warning checker will then be dictating personal style, and (in my very personal opinion) be in the wrong. I raised this issue in October, I fought my corner through blog posts and reddit, and I lost - I considered that the end. The Core libraries committee have done the work, and at this stage, to back out will be a fantastic amount of work. I don't have the time to put in that amount of work, and I think it unfair to demand those who are for BBP to do the work for me. At the same time, I have more than once picked the "easy option" in the short term when coding, been warned by Lennart that I was making a mistake, and then discovered down the line that Lennart was right. Making mistakes with Prelude is going to be costly. Thanks, Neil

That is the one. =) My previous thinking on this had been to figure out how to get deprecated exports implemented early in 7.12, so we could put in a proposal to libraries@ to use it to clean up the nonsensical generalizations of the Data.List combinators, and go back and address several Control.Monad re-exports like, mapM and the mtl Monad re-exports in 7.14. That would make it easier to use push towards a world where we _can_ implement something like an alternate Prelude as mostly an import-only decision in the future. For all that I don't see a good way to deal with Foldable/Traversable changes in such a manner at this point, I happen to agree in general with the approach that Neil espouses about how to evolve the language, and removing barriers such as these to eventually following that model would be more or less an unmitigatedly good thing. Now, if we could get deprecated exports earlier we could theoretically put those deprecations in place now. But it seems to me that putting even that mechanism in place and agreeing on a plan of action around how to use it would be a lot to cram in to try to add here at the very end of 7.10 and risks missing cases, like we did when we rushed out the AMP warnings at the end of 7.8. The benefit of going down that road rather than a WEAK pragma is that it avoids introducing a new mechanic for how name resolution works that complicates the language, a deprecated export pragma by way of comparison is generally useful and can be ignored by compilers that don't understand it, even when used in third-party code. The benefits of a WEAK pragma is that it _could_ be shored up and made a part of the language, but it is admittedly a rather odd pragma. Adding such a construction is a topic that has come up a few times before with lukewarm reception. Most existing pragmas can be sensibly ignored by compilers that don't know about them. By way of comparison existing pragmas like SPECIALIZE, SOURCE, [NO]INLINE, UNPACK, WARNING, DEPRECATED, etc. don't change the way names resolve for the entire program. It seems a messy solution to rush a language extension in that we may regret later and simultaneously break all of the code that has already been made 7.10-ready. -Edward On Wed, Jan 28, 2015 at 5:33 PM, Herbert Valerio Riedel wrote:

On 2015-01-28 at 18:35:01 +0100, Edward Kmett wrote:

[...]

...

Adding to GHC the ability to complain if a given re-export is the only way a symbol finds its way into scope that you use, and apply those flags to Control.Monad.mapM and a bunch of combinators in Data.List.*.

...I think the 4-year-old #4879 GHC ticket covers that ability?

[1]: https://ghc.haskell.org/trac/ghc/ticket/4879

On Sat, Jan 31, 2015 at 1:18 AM, Mark Lentczner wrote:

I'm sorry for being so behind the times, and now have to write this at this stage...

BBP deeply saddens me. While I understand the desire for a more generalized Prelude (surely this is one of the seven stages of Haskell, along with "need to write a Monad tutorial"....), I don't think the ends come anywhere close to justifying the means.

It is very painful the prelude isn't more generalized. Its the first thing that bit me learning Haskell and its pained me the entire time I've used it.

I realize I'm often (always?) the voice of stodginess. But I'll tell where I'm coming from: I want to get people to adopt Haskell for serious software development. But that means I have get them to buy into an ecosystem and for them to do so they need to feel it will be stable on the order decades. They need to feel that before they're going to be willing to invest in the tooling, the process development, and committing their work to Haskell.

Personally, I want a language thats good to use. That means constant improvement. My life gets better by far more then it costs me to update some code in fairly trivial ways when the language or libraries change. In fact, Haskell makes updates easy. Why should we instead build up the need to replace the language with a new one, how is that better? We can have constant gradual improvement, or force an entire change of language. I for one will take a constant improvement every time.

In 1990 most shops wouldn't ship software in C++ because none of us trusted the compilers, or that language (already C++ 2.0) wouldn't change in ways that would be unpredictable and negatively affect our work. It took almost another decade. In 2005 many places still wouldn't ship things using STL. It takes a long time for people to trust that an ecosystem has stabilized.

I think they're exactly who not to take our queues from. I think they're the perfect example of how to make bad software and I really don't want to be them.

"Burning Bridges". The name says it all. We cannot do this. We have to look well beyond how a change like this affects compiling a bunch of code form stackage... We have to think beyond the discussions on our mailing lists... Think of the text books, the corse syllabuses, the in-house trainings, think of work flow and deployment processes.. think of vast amounts of code we can't see... We cannot undermine the Prelude and Data.List that they are all built on.

Alternative we must do it to keep advancing.

If we make a change like this, we've just told the world to wait another 10 years before considering Haskell.

Or told people who should consider Haskell. Perhaps that is good. The sadder thing is that I don't think the proposal really adds much to

Haskell:

The value of generalizing the list functions greatly over estimated: There have been alternate, more generalized Preludes for ages... and I don't see much use of them. Sure, they require some futzing with imports to use - but that isn't that high of a bar, and yet clearly their value rarely merits it in people's minds. Maybe I don't write the most clever code (code golf excepted), but I rarely pine for a more generalized length. Or for all to work over Foldable... etc.

For the few times when I need something like Foldable I don't see what the issue is with importing qualified and using prefixes. People here seem to feel that a "F." is somehow an ugly wart on their code... but that is just the way it is with most software: You need a lot of things in scope, you use namespaces of one sort or another.

There's also a whole issue of commitment to forward and backward compatibility that we as a community don't yet have. Littering our code with CPP is not really acceptable. And all production code insists on clean compilation with -Wall, so we can't be spewing warnings. Or for that matter playing tricks with import orders. This stuff is tricky and hard... no doubt, but it is the price of a bid at adoption.

And we shouldn't have it. Its good we don't have it. Its why I'm here frankly. The constant drive for improvement is how its solve the problems that make it worth coding in. Lets instead of being adopted have the best language possible and the most pleasant time programming in it. Lets not tear our hair out because someone made a bad decision in the 1980s and now we're stuck with it. I've done that. It was terrible. The world doesn't need another language that makes that mistake. All in all, what saddens me here is that this whole episode, like the

similar one around roles and 7.8, is indicative that on the whole, we as a community, are not ready to hunker down and work toward making Haskell a solid tool for software development. We are, instead, too fascinated with making it more perfect.

Hiya Mark. Your note was pretty passionate, so I hope you’ll forgive me if mine is a bit strongly worded as well. There’s a phenomenon, especially on the internet, where people don’t make arguments of the form “I don’t like” or “I don’t want” or even “I think” but instead of “Personally, I’m fine with X, but I can see how see how _others_ could…”.  I tend to think of this argument as a form of ‘concern trolling’ even though I know that is an overloaded term that maybe has too many other connotations. In any case, I have seen a lot of this form of argumentation in the Foldable/Traversable discussions. “I understand this stuff perfectly well, but I can see how  a beginner could…”. “I of course can fix up my libraries in an afternoon, but I can see how this would cause trouble for others…”. “Patching my codebase to handle the change isn’t bad, but I can see how others who might want to adopt Haskell could…” So what’s the problem here? Bluntly, if somebody has an objection, we have a chance of convincing them. If imaginary rhetorical strawmen in somebody’s head have an objection, we don’t have a chance of convincing them, or even having a proper dialogue with them, because nobody can speak to these imaginary people outside of the person that is imagining them. In this case, I know that you have a tremendous amount more industry experience than me, and in fact than many others of us, but I nonetheless think you’re engaging in a form of this advocacy on the part of invented parties. Let me explain why. In the course of writing “industrial Haskell” I have discovered that one of the _first_ things that many Haskell shops do is to settle on a standardized, “custom” import list for their stuff. Often, this list omits the partial functions from the import of Prelude. Sometimes, it is packaged up as a convenience module, sometimes it is just a chunk of import statements. These imports include chunks of Prelude, chunks of Base, and often, chunks of whatever homegrown “enterprise glue” utilities that a given shop has come to rely on — logging, parsing, configuration, sometimes communication, etc. So what affect does a change in Prelude have in such systems? Often, much less than if there were not such standards. And furthermore, if you’re using a custom convenience module anyway, you have a mechanism for managing the impact of changes in Prelude in a fairly modular way. I’m not saying every “Industrial Haskell” shop now does something like this — but honestly, those that don’t should, and this is the sort of thing that, as industrial Haskell adoption grows, will clearly get adopted as a “best practice.” On top of this, industrial shops, unlike library authors, have a much stronger set of controls on the environment they expect their code to work in. You can set a companywide standard for a version of GHC, and cutover to new versions on a controlled schedule of your choice, in a coordinated fashion. You can lock down package version numbers strongly and migrate them all at once. Etc. Again, even though some shops may not do this uniformly, it is certainly a best practice with many benefits, even absent any changes in the Prelude. And its implementation again strongly mitigates the effects of such changes. In any case, let’s not set our sights too high. It may be that “the 80%” of large shops won’t adopt Haskell anytime soon. Frankly, I don’t think that Prelude instability will be the reason why. How many developers at how many shops have heard of Haskell and take it seriously as more than a curiosity regardless? How many, even when they hear about Google or Facebook adoption say, “Ah so it is a fancy language for big problems faced on the bleeding edge. Me? I’m just writing some typical stuff, who cares?” Let’s have some proportion. For all our adoption, for Haskell to become even a “2%” language across industry would be a few orders of magnitude growth from where we are now. As far as I know, no other language with our current user base has felt the need to lock things down so tightly at this point on the growth curve. If, at this stage in its career, Java can get lambdas, that almost seems a constructive disproof in itself of the “widely used things shouldn’t evolve” principle you seem to be describing. From my standpoint, the ironic thing is that we have endured far more seriously and wide-ragingly breaking changes over the course of prior GHC releases for years, but they have not generated this much heat because they have _looked_ innocuous. So we finally have a core libraries committee that is tasked with trying to make changes in a smooth way. And the changes here proposed are _very_ smooth! But now, of course, the care that was taken to make sure everything works cleanly and smoothly itself has now generated backlash, because it makes the surface area of this change _look_ larger. Honestly? The Prelude is just the set of stuff in scope by default. If the entire prelude consisted of one function that drew an ASCII fish and nothing else, I could live with that. My main inclination here is to trust the core libraries committee to do its job, and to see what benefits that brings over time, rather than killing off this entire direction of work before it even has a chance to begin to prove itself. Cheers, Gershom On January 31, 2015 at 1:19:18 AM, Mark Lentczner (mark.lentczner@gmail.com) wrote:

I'm sorry for being so behind the times, and now have to write this at this stage...

BBP deeply saddens me. While I understand the desire for a more generalized Prelude (surely this is one of the seven stages of Haskell, along with "need to write a Monad tutorial"....), I don't think the ends come anywhere close to justifying the means.

I realize I'm often (always?) the voice of stodginess. But I'll tell where I'm coming from: I want to get people to adopt Haskell for serious software development. But that means I have get them to buy into an ecosystem and for them to do so they need to feel it will be stable on the order decades. They need to feel that before they're going to be willing to invest in the tooling, the process development, and committing their work to Haskell.

In 1990 most shops wouldn't ship software in C++ because none of us trusted the compilers, or that language (already C++ 2.0) wouldn't change in ways that would be unpredictable and negatively affect our work. It took almost another decade. In 2005 many places still wouldn't ship things using STL. It takes a long time for people to trust that an ecosystem has stabilized.

"Burning Bridges". The name says it all. We cannot do this. We have to look well beyond how a change like this affects compiling a bunch of code form stackage... We have to think beyond the discussions on our mailing lists... Think of the text books, the corse syllabuses, the in-house trainings, think of work flow and deployment processes.. think of vast amounts of code we can't see... We cannot undermine the Prelude and Data.List that they are all built on.

If we make a change like this, we've just told the world to wait another 10 years before considering Haskell.

The sadder thing is that I don't think the proposal really adds much to Haskell:

The value of generalizing the list functions greatly over estimated: There have been alternate, more generalized Preludes for ages... and I don't see much use of them. Sure, they require some futzing with imports to use - but that isn't that high of a bar, and yet clearly their value rarely merits it in people's minds. Maybe I don't write the most clever code (code golf excepted), but I rarely pine for a more generalized length. Or for all to work over Foldable... etc.

For the few times when I need something like Foldable I don't see what the issue is with importing qualified and using prefixes. People here seem to feel that a "F." is somehow an ugly wart on their code... but that is just the way it is with most software: You need a lot of things in scope, you use namespaces of one sort or another.

There's also a whole issue of commitment to forward and backward compatibility that we as a community don't yet have. Littering our code with CPP is not really acceptable. And all production code insists on clean compilation with -Wall, so we can't be spewing warnings. Or for that matter playing tricks with import orders. This stuff is tricky and hard... no doubt, but it is the price of a bid at adoption.

All in all, what saddens me here is that this whole episode, like the similar one around roles and 7.8, is indicative that on the whole, we as a community, are not ready to hunker down and work toward making Haskell a solid tool for software development. We are, instead, too fascinated with making it more perfect.

- Mark _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Sat, 2015-01-31 at 09:59 -0800, Johan Tibell wrote:

I re-read the goals* of the proposal: https://wiki.haskell.org/Foldable_Traversable_In_Prelude

"One goal here is that people should be able to use methods from these modules without the need of a qualified import -- i.e. to prevent clash in the namespace, by resolving such in favor of the more generalized versions. Additionally, there are some new methods added to the Foldable class because it seems to be the "right" thing."

Before FTP, I would have to write this to use Foldable/Traversable:

import qualified Data.Foldable as F

The import is qualified as to not collide with the Prelude.

If I have code that needs to be compatible with more than one GHC release (I typically need compatibility with the last 3 major releases), what do I have to write post-FTP? Since I cannot rely on the Prelude being generalized (because I might be compiling with a pre-FTP compiler) I need to either write:

#if MIN_VERSION_base(x,y,z) -- Get Foldable etc from Prelude #else import Data.Foldable (...) import Prelude hiding (...) -- the same #endif

You just create custom Prelude.hs and place all the ugly CPP inside it. Then you continue writing code assuming the generalized prelude for all `base` versions (or specialized prelude if you prefer.) I already use custom prelude when I need portability because older base exports incompatible `catch` from Prelude. That works OK for me so far, and I don't see why it will not work for FTP. (I personally hate FTP, but I probably will be able to live with it.)

Which is terrible. Alternatively I can write

import qualified Data.Foldable as F

but now nothing is gained over the pre-FTP state. Only after 3+ years (at the current GHC release phase) I can drop that one extra import. One out of perhaps 20. That seems quite a small gain given that we will then have made Data.List a very confusing module (it's essentially Data.Foldable under a different name), broken some code due to added type ambiguity, and also removed one of the simpler ways to remove that ambiguity, which would have been to import one of the monomorphic list functions that no longer exist.

* People tell me that there are other goals, but they aren't really stated clearly anywhere.

I'd like to known other goals. Right now it looks for me like saving few keystrokes.

-- Johan _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On Sun, Feb 1, 2015 at 4:35 AM, Bardur Arantsson wrote:

However, I certainly agree with Gershom B that the "enterprise" people are exactly the kind of people who have the means to prevent these problems affecting *them* too much. They probably should be (or are already) doing due dilligence to that effect.

I was actually going to use the same argument but in the opposite way! In all my larger projects I wind up with a project-specific mini-Prelude. So I'm pretty much isolated from Prelude changes. Therefore, I don't mind if the Prelude never changes again. So I'd prefer to err on the side that makes life easier for library maintainers. Of course it also means I don't mind the changes very much. They just don't seem like such an obvious improvement either. I've actually backed away from generalizations before due to the poorer type inference and error messages that came out of surprising places, as well as lack of sufficient usefulness. The main disconnect for me is that I occasionally use Foldable and Traversable. But I use a lot more lists, and I actually kind of like the less general functions. When I need the general ones, I know where they are, and I don't mind importing. I have a tool that adds qualified imports automatically, but I import Foldable and Traversable orders of magnitude less frequently than, say, Data.Map. I don't mind qualified imports. I know some people find them ugly, but when someone says "ugly" about an aesthetic thing (i.e. names, layout, etc.) I translate that to "I'm not used to it." This is from personal experience where I found certain things ugly (e.g. camelCase), but when I got used to them they look just fine. I went and read through part of the original BPP thread, and the principles in support seem reasonable. The only doc I can find for BBP is https://wiki.haskell.org/Foldable_Traversable_In_Prelude, and it doesn't have a "motivation" section. It would be interesting to see some specific examples about how the current situation is very painful, and how the generalization would make it better. Like I said, I'm not really against it, but maybe I can learn something from those who support it so passionately.

On Sun, Feb 1, 2015 at 4:09 AM, Herbert Valerio Riedel wrote:

Fwiw, the 3+ year argument applies to almost any `base`-augmentation if you need such backward-compatibility. By that argument there's little incentive to add anything to `base` at all.

That isn't true. Careful additions of things to existing modules that are generally imported qualified are fine. Additions of new modules are fine. Yes, a side effect of being 'base' is that it is harder to change and must consider backward and forward compatibility over a longer time span. So pre-FTP (in GHC 7.8), the following is needed to more or less emulate

the post-FTP state:

*[many line of ugly imports]* That isn't the right tradeoff. Without FTP, one imports qualified: import qualified Data.Foldable as F import qualified Data.Traversable as T If importing qualified is too much a barrier to use of Foldable and Traversable - then I think we should re-consider how useful they are. If we

don't intend to make Foldable/Traversable a first-class citizen of Haskell to help increase its adoption, why have it in `base` in the first place?

"first-class citizen" better not equate to "you get it with the Prelude" nor "you can always import it unqualified". If we use either of those as the bar, then we will end up with huge number of things in default scope, and we'll namespace it by prefix strings *alá PHP.*

Moreover, how do we justify (other than by historic accident) that the more generalised synonyms are stored away in modules … Do lists really require such a special privilege?

Yup, historic accident *and* lists do get special privilege. We could easily turn your question around: Why do Foldable and Traversable deserve to be not only importable unqualified, but by all programs as part of the Prelude? Why not rather place the more general versions into the default scope

… This results in a more symmetric situation.

It might be more symmetric. But I don't see universal acclaim that Foldable and Traversable are a generalization to be preferred over lists in the majority of code. And the "why not" is because it is a big change to the Prelude with negative consequences judged significant by many of us. The design-choices of the FTP implementation were motivated to achieve

its goal while breaking as little existing code as possible. We assumed this to be the better compromise rather than introducing a severe breakage of the majority existing Haskell code which don't qualify-import `Data.List`.

In the evaluation, we need to consider at least a third choice: Not to do FTP at all. - Mark

Mark, On 2015-02-01 at 15:00:33 +0100, Mark Lentczner wrote: [...]

...

If we don't intend to make Foldable/Traversable a first-class citizen of Haskell to help increase its adoption, why have it in `base` in the first place?

"first-class citizen" better not equate to "you get it with the Prelude" nor "you can always import it unqualified".

If we use either of those as the bar, then we will end up with huge number of things in default scope, and we'll namespace it by prefix strings *alá PHP.*

That's bordering on slippery-slope argumentation and misrepresenting what the changes due to FTP actually are. FTP is mostly about replacing *existing* symbols in the Prelude with their generalised counterparts because they were *already* there (otherwise they wouldn't cause conflict in the first place). The few newly added symbols to the `Prelude` scope (specifically: Foldable(foldMap), Traversable(traverse), and Monoid(..) ) were done to satisfy the expectation to be able define instances as well as type-signatures without requiring additional imports, make the Prelude self-contained/closed. This particular design choice is not mandatory for the FTP.

...

Moreover, how do we justify (other than by historic accident) that the more generalised synonyms are stored away in modules … Do lists really require such a special privilege?

Yup, historic accident *and* lists do get special privilege. We could easily turn your question around: Why do Foldable and Traversable deserve to be not only importable unqualified, but by all programs as part of the Prelude?

Because they are more general and more powerful than their list-specialised counterparts. They can (almost) always be used in place of their more specialised versions. The converse, however, does not generally hold. Moreover, having them available by default in scope encourages users to think in terms of them and look for opportunities (i.e. recognize common shapes/patterns) to levarage the abstraction.

...

Why not rather place the more general versions into the default scope … This results in a more symmetric situation.

It might be more symmetric. But I don't see universal acclaim that Foldable and Traversable are a generalization to be preferred over lists in the majority of code. And the "why not" is because it is a big change to the Prelude with negative consequences judged significant by many of us.

Could you please spell out the significant negative consequences you're referring to? Otherwise it's too vague and denies me the chance to address your concern (and we are considering technical solutions to make the FTP more palatable to all concerned parties, but this requires being able exactly identify the pain points). -- hvr

I happen to support the vast majority of my packages for 3 major GHC releases, not just platforms, and for the current stackage build as well. The idea that we can't personally benefit from things for ~3 years because we happen to have packages with long support cycles leads to a fallacy by which by induction no progress can ever be made to benefit the larger pool of Haskell users we will have in the future because we can never pay a price in the short term to benefit them. Regarding Foldable, there is a chain of reasoning that begins with accepting the Applicative-Monad Proposal: Once we accept Applicative as a superclass of Monad then mapM is _never_ the right abstraction. Why? mapM is needlessly constrained to Monad, so you really start needing the ability to traverse with Applicative. mapM actively gets in the way of making code work with the reduced Applicative constraints that are possible post AMP. Dead End #1. We can't just generalize the signature of mapM in a way that makes mapM work with Applicative without a whole host of other issues. (mapM is a member of the Traversable class alongside traverse, there exists a corner case where Traversable.mapM can have better termination behavior on a weirdly designed container, so having Prelude mapM and Traversable.mapM generalized differently would be bad.) Dead End #2. To go the other way and put in a monomorphized traverse into Prelude would break insane amounts of code across the entire ecosystem. To expose traverse is to expose Traversable. Exposing Traversable from the Prelude then really wants to drag in the superclasses of Traversable lest you have a class that is exposed in the Prelude with a superclass you can't define. Dead End #3. We could have added Applicative as a superclass of Monad without adding it to the Prelude, but it would have been a bad way to proceed. Now users who want to either benefit from the change or define a Monad, ever, would have to add an extra import Control.Applicative line, what is the probem. This is the same argument against forcing Foldable into hiding while possibly exposing Traversable, just mutatis mutandis for Foldable/Applicative and Traversable/Monad. Consequently, exposing the pieces you need to define classes that are exposed through Prelude seems to be the direction that in the long term is the most defensible in terms of how to explain the state of the world to newcomers, once we get the hurdle of transition. The current state of affairs was quite eloquently summarized by this transcript from Nick Partridge about the process of describing the AMP to his coworkers: In a conversation with co-workers about these changes, I fired up a ghci session to show the difference betweenmapM and traverse and what it would mean to generalize the functions in Prelude, and how traverse was just a more general mapM. Here is how that went down:

:i traverse

import Data.Traversable :i traverse class (Functor t, Data.Foldable.Foldable t) => Traversable (t :: * -> *) where

Top level: Not in scope: ‘traverse’ Perhaps you meant ‘reverse’ (imported from Prelude) Ah, of course. I forgot the import. traverse :: Control.Applicative.Applicative f => (a -> f b) -> t a -> f (t b) ... -- Defined in ‘Data.Traversable’ Great. Now let's look at mapM:

:i mapM

:i Prelude.mapM Prelude.mapM :: Monad m => (a -> m b) -> [a] -> m [b] -- Defined in ‘Control.Monad’ :t traverse

Top level: Ambiguous occurrence ‘mapM’ It could refer to either ‘Data.Traversable.mapM’, imported from ‘Data.Traversable’ or ‘Prelude.mapM’, imported from ‘Prelude’ (and originally defined in ‘Control.Monad’) Oh. Okay we'll check out the one in Prelude by prefixing it. traverse :: (Traversable t, Control.Applicative.Applicative f) => (a -> f b) -> t a -> f (t b) Now let's specialise traverse for Monad and List, to show that it's equivalent.

:t (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

<interactive>:1:2: Could not deduce (Control.Applicative.Applicative m1) arising from a use of ‘traverse’ from the context (Monad m) bound by the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] at Top level or from (Monad m1) bound by an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] at <interactive>:1:2-50 Possible fix: add (Control.Applicative.Applicative m1) to the context of an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] or the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] In the expression: (traverse :: Monad m => (a -> m b) -> [a] -> m [b]) Curses! Looking forward to AMP. Insert explanation about how Applicative should be a superclass of Monad. Pretend that it really is. Wave hands about.

:t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b])

<interactive>:1:14: Not in scope: type constructor or class ‘Applicative’ Okay this is getting silly.

import Control.Applicative :t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) :: Applicative m => (a -> m b) -> [a] -> m [b]

This is *really* *really* common teaching/learning scenario using the current state of the Prelude. On Sat, Jan 31, 2015 at 12:59 PM, Johan Tibell wrote:

I re-read the goals* of the proposal: https://wiki.haskell.org/Foldable_Traversable_In_Prelude

"One goal here is that people should be able to use methods from these modules without the need of a qualified import -- i.e. to prevent clash in the namespace, by resolving such in favor of the more generalized versions. Additionally, there are some new methods added to the Foldable class because it seems to be the "right" thing."

Before FTP, I would have to write this to use Foldable/Traversable:

import qualified Data.Foldable as F

The import is qualified as to not collide with the Prelude.

If I have code that needs to be compatible with more than one GHC release (I typically need compatibility with the last 3 major releases), what do I have to write post-FTP? Since I cannot rely on the Prelude being generalized (because I might be compiling with a pre-FTP compiler) I need to either write:

#if MIN_VERSION_base(x,y,z) -- Get Foldable etc from Prelude #else import Data.Foldable (...) import Prelude hiding (...) -- the same #endif

Which is terrible. Alternatively I can write

import qualified Data.Foldable as F

but now nothing is gained over the pre-FTP state. Only after 3+ years (at the current GHC release phase) I can drop that one extra import. One out of perhaps 20. That seems quite a small gain given that we will then have made Data.List a very confusing module (it's essentially Data.Foldable under a different name), broken some code due to added type ambiguity, and also removed one of the simpler ways to remove that ambiguity, which would have been to import one of the monomorphic list functions that no longer exist.

* People tell me that there are other goals, but they aren't really stated clearly anywhere.

-- Johan

However, it'd be nice if we could come up with a design that, more or less, the whole community thinks is a good one.

If we only ever added things to the language that everyone could completely agree on we would end up with something as conservative as Java in terms of adding new features. Haskell should strive to improve while it is still obscure enough that we can change things like this. On Sun, Feb 1, 2015 at 8:59 PM, Iavor Diatchki wrote:

Hello,

I think that "dead end #3" is the right choice, and keep Foldable etc. in their own modules. We have a module system, we should use it. In general, I think if we are to "burn bridges" it should be by moving things out of the Prelude, not adding more abstractions there.

For the sake of concreteness, here are some of the things I don't like about the current design:

- the design of 'Foldable'. When you ask GHCi about what are the members of the class, you are presented with a long list of more or less random methods. This does not seem like a clean abstraction (e.g., why are 'sum' and 'product' there??).

- the 'Monoid' class. Monoids are a useful abstraction, but encoding them as a Haskell class is very much a compromise (in my mind, anyway). Keeping them in their own module seems like a good idea.

- the state of Data.List: I generally prefer that, when possible, a datatype should provide non-overloaded access to its functionality, and in addition, there can be instances to expose the same functionality via an overloaded API.

Of course, I can generally work around all of those, so not a big deal. However, it'd be nice if we could come up with a design that, more or less, the whole community thinks is a good one.

-Iavor

I happen to support the vast majority of my packages for 3 major GHC releases, not just platforms, and for the current stackage build as well.

The idea that we can't personally benefit from things for ~3 years because we happen to have packages with long support cycles leads to a fallacy by which by induction no progress can ever be made to benefit the larger pool of Haskell users we will have in the future because we can never pay a price in the short term to benefit them.

Regarding Foldable, there is a chain of reasoning that begins with accepting the Applicative-Monad Proposal:

Once we accept Applicative as a superclass of Monad then mapM is _never_ the right abstraction.

Why?

mapM is needlessly constrained to Monad, so you really start needing the ability to traverse with Applicative.

mapM actively gets in the way of making code work with the reduced Applicative constraints that are possible post AMP.

Dead End #1. We can't just generalize the signature of mapM in a way that makes mapM work with Applicative without a whole host of other issues. (mapM is a member of the Traversable class alongside traverse, there exists a corner case where Traversable.mapM can have better termination behavior on a weirdly designed container, so having Prelude mapM and Traversable.mapM generalized differently would be bad.)

Dead End #2. To go the other way and put in a monomorphized traverse into Prelude would break insane amounts of code across the entire ecosystem.

To expose traverse is to expose Traversable.

Exposing Traversable from the Prelude then really wants to drag in the superclasses of Traversable lest you have a class that is exposed in the Prelude with a superclass you can't define.

Dead End #3. We could have added Applicative as a superclass of Monad without adding it to the Prelude, but it would have been a bad way to proceed. Now users who want to either benefit from the change or define a Monad, ever, would have to add an extra import Control.Applicative line, what is the probem. This is the same argument against forcing Foldable into hiding while possibly exposing Traversable, just mutatis mutandis for Foldable/Applicative and Traversable/Monad.

Consequently, exposing the pieces you need to define classes that are exposed through Prelude seems to be the direction that in the long term is the most defensible in terms of how to explain the state of the world to newcomers, once we get the hurdle of transition.

The current state of affairs was quite eloquently summarized by this transcript from Nick Partridge about the process of describing the AMP to his coworkers:

In a conversation with co-workers about these changes, I fired up a ghci session to show the difference betweenmapM and traverse and what it would mean to generalize the functions in Prelude, and how traverse was just a more general mapM.

Here is how that went down:

...

:i traverse

Top level: Not in scope: ‘traverse’ Perhaps you meant ‘reverse’ (imported from Prelude)

Ah, of course. I forgot the import.

...

import Data.Traversable :i traverse class (Functor t, Data.Foldable.Foldable t) => Traversable (t :: * -> *) where traverse :: Control.Applicative.Applicative f => (a -> f b) -> t a -> f (t b) ... -- Defined in ‘Data.Traversable’

Great. Now let's look at mapM:

...

:i mapM

Top level: Ambiguous occurrence ‘mapM’ It could refer to either ‘Data.Traversable.mapM’, imported from ‘Data.Traversable’ or ‘Prelude.mapM’, imported from ‘Prelude’ (and originally defined in ‘Control.Monad’)

Oh. Okay we'll check out the one in Prelude by prefixing it.

...

:i Prelude.mapM Prelude.mapM :: Monad m => (a -> m b) -> [a] -> m [b] -- Defined in ‘Control.Monad’ :t traverse traverse :: (Traversable t, Control.Applicative.Applicative f) => (a -> f b) -> t a -> f (t b)

Now let's specialise traverse for Monad and List, to show that it's equivalent.

...

:t (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

<interactive>:1:2: Could not deduce (Control.Applicative.Applicative m1) arising from a use of ‘traverse’ from the context (Monad m) bound by the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] at Top level or from (Monad m1) bound by an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] at <interactive>:1:2-50 Possible fix: add (Control.Applicative.Applicative m1) to the context of an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] or the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] In the expression: (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

Curses! Looking forward to AMP. Insert explanation about how Applicative should be a superclass of Monad. Pretend that it really is. Wave hands about.

...

:t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b])

<interactive>:1:14: Not in scope: type constructor or class ‘Applicative’

Okay this is getting silly.

...

import Control.Applicative :t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) :: Applicative m => (a -> m b) -> [a] -> m [b]

This is really really common teaching/learning scenario using the current state of the Prelude.

On Sat, Jan 31, 2015 at 12:59 PM, Johan Tibell wrote:

...

I re-read the goals* of the proposal: https://wiki.haskell.org/Foldable_Traversable_In_Prelude

"One goal here is that people should be able to use methods from these modules without the need of a qualified import -- i.e. to prevent clash in the namespace, by resolving such in favor of the more generalized versions. Additionally, there are some new methods added to the Foldable class because it seems to be the "right" thing."

Before FTP, I would have to write this to use Foldable/Traversable:

import qualified Data.Foldable as F

The import is qualified as to not collide with the Prelude.

If I have code that needs to be compatible with more than one GHC release (I typically need compatibility with the last 3 major releases), what do I have to write post-FTP? Since I cannot rely on the Prelude being generalized (because I might be compiling with a pre-FTP compiler) I need to either write:

#if MIN_VERSION_base(x,y,z) -- Get Foldable etc from Prelude #else import Data.Foldable (...) import Prelude hiding (...) -- the same #endif

Which is terrible. Alternatively I can write

import qualified Data.Foldable as F

but now nothing is gained over the pre-FTP state. Only after 3+ years (at the current GHC release phase) I can drop that one extra import. One out of perhaps 20. That seems quite a small gain given that we will then have made Data.List a very confusing module (it's essentially Data.Foldable under a different name), broken some code due to added type ambiguity, and also removed one of the simpler ways to remove that ambiguity, which would have been to import one of the monomorphic list functions that no longer exist.

* People tell me that there are other goals, but they aren't really stated clearly anywhere.

-- Johan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Hi Doaitse, The changes discussed here are mostly backwards compatible. The Applicative-Monad changes are less so, all Monads now also need an Applicative instance. If you want to test with 7.10's release candidate, I've made a Mac build available at [1]. This is RC1, I'll try to make RC2 available as well. Regards, Erik [1] http://www.reddit.com/r/haskell/comments/2rims6/ghc_7100rc1_build_for_mac_os... On Sun, Feb 1, 2015 at 10:38 PM, S. Doaitse Swierstra wrote:

As a humble maintainer of a few libraries I would appreciate if some guidance could be given as to how to keep my libraries working with older versions of GHC and this new Prelude. Thus far the number of options and flags to achieve interoperability seems to grow and grow.

Besides that I understand that the new version is not yet available on my Mac, so I do not even dare to change my GHC implementation to find out what the consequences are of this new proposal. Too often I have had to throw away my complete Haskell infrastructure since I got lost in cabal hell, versions mismatch, etc, .

Doaitse

...

On 01 Feb 2015, at 21:11 , Matthias Hörmann wrote:

...

However, it'd be nice if we could come up with a design that, more or less, the whole community thinks is a good one.

If we only ever added things to the language that everyone could completely agree on we would end up with something as conservative as Java in terms of adding new features. Haskell should strive to improve while it is still obscure enough that we can change things like this.

On Sun, Feb 1, 2015 at 8:59 PM, Iavor Diatchki wrote:

...

Hello,

I think that "dead end #3" is the right choice, and keep Foldable etc. in their own modules. We have a module system, we should use it. In general, I think if we are to "burn bridges" it should be by moving things out of the Prelude, not adding more abstractions there.

For the sake of concreteness, here are some of the things I don't like about the current design:

- the design of 'Foldable'. When you ask GHCi about what are the members of the class, you are presented with a long list of more or less random methods. This does not seem like a clean abstraction (e.g., why are 'sum' and 'product' there??).

- the 'Monoid' class. Monoids are a useful abstraction, but encoding them as a Haskell class is very much a compromise (in my mind, anyway). Keeping them in their own module seems like a good idea.

- the state of Data.List: I generally prefer that, when possible, a datatype should provide non-overloaded access to its functionality, and in addition, there can be instances to expose the same functionality via an overloaded API.

Of course, I can generally work around all of those, so not a big deal. However, it'd be nice if we could come up with a design that, more or less, the whole community thinks is a good one.

-Iavor

I happen to support the vast majority of my packages for 3 major GHC releases, not just platforms, and for the current stackage build as well.

The idea that we can't personally benefit from things for ~3 years because we happen to have packages with long support cycles leads to a fallacy by which by induction no progress can ever be made to benefit the larger pool of Haskell users we will have in the future because we can never pay a price in the short term to benefit them.

Regarding Foldable, there is a chain of reasoning that begins with accepting the Applicative-Monad Proposal:

Once we accept Applicative as a superclass of Monad then mapM is _never_ the right abstraction.

Why?

mapM is needlessly constrained to Monad, so you really start needing the ability to traverse with Applicative.

mapM actively gets in the way of making code work with the reduced Applicative constraints that are possible post AMP.

Dead End #1. We can't just generalize the signature of mapM in a way that makes mapM work with Applicative without a whole host of other issues. (mapM is a member of the Traversable class alongside traverse, there exists a corner case where Traversable.mapM can have better termination behavior on a weirdly designed container, so having Prelude mapM and Traversable.mapM generalized differently would be bad.)

Dead End #2. To go the other way and put in a monomorphized traverse into Prelude would break insane amounts of code across the entire ecosystem.

To expose traverse is to expose Traversable.

Exposing Traversable from the Prelude then really wants to drag in the superclasses of Traversable lest you have a class that is exposed in the Prelude with a superclass you can't define.

Dead End #3. We could have added Applicative as a superclass of Monad without adding it to the Prelude, but it would have been a bad way to proceed. Now users who want to either benefit from the change or define a Monad, ever, would have to add an extra import Control.Applicative line, what is the probem. This is the same argument against forcing Foldable into hiding while possibly exposing Traversable, just mutatis mutandis for Foldable/Applicative and Traversable/Monad.

Consequently, exposing the pieces you need to define classes that are exposed through Prelude seems to be the direction that in the long term is the most defensible in terms of how to explain the state of the world to newcomers, once we get the hurdle of transition.

The current state of affairs was quite eloquently summarized by this transcript from Nick Partridge about the process of describing the AMP to his coworkers:

In a conversation with co-workers about these changes, I fired up a ghci session to show the difference betweenmapM and traverse and what it would mean to generalize the functions in Prelude, and how traverse was just a more general mapM.

Here is how that went down:

...

:i traverse

Top level: Not in scope: ‘traverse’ Perhaps you meant ‘reverse’ (imported from Prelude)

Ah, of course. I forgot the import.

...

import Data.Traversable :i traverse class (Functor t, Data.Foldable.Foldable t) => Traversable (t :: * -> *) where traverse :: Control.Applicative.Applicative f => (a -> f b) -> t a -> f (t b) ... -- Defined in ‘Data.Traversable’

Great. Now let's look at mapM:

...

:i mapM

Top level: Ambiguous occurrence ‘mapM’ It could refer to either ‘Data.Traversable.mapM’, imported from ‘Data.Traversable’ or ‘Prelude.mapM’, imported from ‘Prelude’ (and originally defined in ‘Control.Monad’)

Oh. Okay we'll check out the one in Prelude by prefixing it.

...

:i Prelude.mapM Prelude.mapM :: Monad m => (a -> m b) -> [a] -> m [b] -- Defined in ‘Control.Monad’ :t traverse traverse :: (Traversable t, Control.Applicative.Applicative f) => (a -> f b) -> t a -> f (t b)

Now let's specialise traverse for Monad and List, to show that it's equivalent.

...

:t (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

<interactive>:1:2: Could not deduce (Control.Applicative.Applicative m1) arising from a use of ‘traverse’ from the context (Monad m) bound by the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] at Top level or from (Monad m1) bound by an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] at <interactive>:1:2-50 Possible fix: add (Control.Applicative.Applicative m1) to the context of an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] or the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] In the expression: (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

Curses! Looking forward to AMP. Insert explanation about how Applicative should be a superclass of Monad. Pretend that it really is. Wave hands about.

...

:t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b])

<interactive>:1:14: Not in scope: type constructor or class ‘Applicative’

Okay this is getting silly.

...

import Control.Applicative :t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) :: Applicative m => (a -> m b) -> [a] -> m [b]

This is really really common teaching/learning scenario using the current state of the Prelude.

On Sat, Jan 31, 2015 at 12:59 PM, Johan Tibell wrote:

...

I re-read the goals* of the proposal: https://wiki.haskell.org/Foldable_Traversable_In_Prelude

"One goal here is that people should be able to use methods from these modules without the need of a qualified import -- i.e. to prevent clash in the namespace, by resolving such in favor of the more generalized versions. Additionally, there are some new methods added to the Foldable class because it seems to be the "right" thing."

Before FTP, I would have to write this to use Foldable/Traversable:

import qualified Data.Foldable as F

The import is qualified as to not collide with the Prelude.

If I have code that needs to be compatible with more than one GHC release (I typically need compatibility with the last 3 major releases), what do I have to write post-FTP? Since I cannot rely on the Prelude being generalized (because I might be compiling with a pre-FTP compiler) I need to either write:

#if MIN_VERSION_base(x,y,z) -- Get Foldable etc from Prelude #else import Data.Foldable (...) import Prelude hiding (...) -- the same #endif

Which is terrible. Alternatively I can write

import qualified Data.Foldable as F

but now nothing is gained over the pre-FTP state. Only after 3+ years (at the current GHC release phase) I can drop that one extra import. One out of perhaps 20. That seems quite a small gain given that we will then have made Data.List a very confusing module (it's essentially Data.Foldable under a different name), broken some code due to added type ambiguity, and also removed one of the simpler ways to remove that ambiguity, which would have been to import one of the monomorphic list functions that no longer exist.

* People tell me that there are other goals, but they aren't really stated clearly anywhere.

-- Johan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Hello, On Sun, Feb 1, 2015 at 12:25 PM, Austin Seipp wrote:

- the 'Monoid' class. Monoids are a useful abstraction, but encoding

...

them as a Haskell class is very much a compromise (in my mind, anyway). Keeping them in their own module seems like a good idea.

Monoid in Prelude is a result of the AMP, and that's not going to change without breaking code even more, or introducing orphans. Neither of these are going to happen: BBP has nothing to do with it.

Really? This is not obvious at all. Is it because of the odd `Applicative` instance for pairs? I am saying 'odd' because there are no similar instances for tuples of higher arity; there are also no instance for `Monad` for the same type, which makes as much sense, I guess. Obviously this is just an opinion, but I don't think these instances fit pairs naturally, and make for confusing looking code. Of course, I can generally work around all of those, so not a big deal.

...

However, it'd be nice if we could come up with a design that, more or less, the whole community thinks is a good one.

I understand why you feel that way, but it's simply impossible to do in any scalable or sensible way in general. Why do we need a unanimous vote here? Or, by this reasoning, when would we *not* need a unanimous vote?

Even 80% in agreement is literally a supermajority beyond what you need in almost any liberal democracy in the world to affect change, for example. If we set our standards that high, it's not only slightly ridiculous, it's tantamount to making sure nothing ever changes.

Pleasing all the people all the time is just never going to be possible.

Agreed. The benefit of getting a design that is understood and accepted by most of the community is 1) when we use the language we feel it is awesome, and not an uneasy compromise; 2) when we introduce new folks to Haskell, we can explain and defend the various design decisions, 3) when newcomers look for examples of how to do things, they will look in the standard libraries. I wouldn't give `Foldable` as an example of how to design a type-class. I'd be much happier with changes that actually improve the overall design even at the price of breaking code. I feel that the BBP does not improve the overall state of the design, and just reshuffles things a bit which, for me, has no benefit. -Iavor

...

-Iavor I happen to support the vast majority of my packages for 3 major GHC releases, not just platforms, and for the current stackage build as well.

The idea that we can't personally benefit from things for ~3 years because we happen to have packages with long support cycles leads to a fallacy by which by induction no progress can ever be made to benefit the larger pool of Haskell users we will have in the future because we can never pay a price in the short term to benefit them.

Regarding Foldable, there is a chain of reasoning that begins with accepting the Applicative-Monad Proposal:

Once we accept Applicative as a superclass of Monad then mapM is _never_ the right abstraction.

Why?

mapM is needlessly constrained to Monad, so you really start needing the ability to traverse with Applicative.

mapM actively gets in the way of making code work with the reduced Applicative constraints that are possible post AMP.

Dead End #1. We can't just generalize the signature of mapM in a way that makes mapM work with Applicative without a whole host of other issues. (mapM is a member of the Traversable class alongside traverse, there exists a corner case where Traversable.mapM can have better termination behavior on a weirdly designed container, so having Prelude mapM and Traversable.mapM generalized differently would be bad.)

Dead End #2. To go the other way and put in a monomorphized traverse into Prelude would break insane amounts of code across the entire ecosystem.

To expose traverse is to expose Traversable.

Exposing Traversable from the Prelude then really wants to drag in the superclasses of Traversable lest you have a class that is exposed in the Prelude with a superclass you can't define.

Dead End #3. We could have added Applicative as a superclass of Monad without adding it to the Prelude, but it would have been a bad way to proceed. Now users who want to either benefit from the change or define a Monad, ever, would have to add an extra import Control.Applicative line, what is the probem. This is the same argument against forcing Foldable into hiding while possibly exposing Traversable, just mutatis mutandis for Foldable/Applicative and Traversable/Monad.

Consequently, exposing the pieces you need to define classes that are exposed through Prelude seems to be the direction that in the long term is the most defensible in terms of how to explain the state of the world to newcomers, once we get the hurdle of transition.

The current state of affairs was quite eloquently summarized by this transcript from Nick Partridge about the process of describing the AMP to his coworkers:

In a conversation with co-workers about these changes, I fired up a ghci session to show the difference betweenmapM and traverse and what it would mean to generalize the functions in Prelude, and how traverse was just a more general mapM.

Here is how that went down:

...

:i traverse

Top level: Not in scope: ‘traverse’ Perhaps you meant ‘reverse’ (imported from Prelude)

Ah, of course. I forgot the import.

...

import Data.Traversable :i traverse class (Functor t, Data.Foldable.Foldable t) => Traversable (t :: * -> *) where traverse :: Control.Applicative.Applicative f => (a -> f b) -> t a -> f (t b) ... -- Defined in ‘Data.Traversable’

Great. Now let's look at mapM:

...

:i mapM

Top level: Ambiguous occurrence ‘mapM’ It could refer to either ‘Data.Traversable.mapM’, imported from ‘Data.Traversable’ or ‘Prelude.mapM’, imported from ‘Prelude’ (and originally defined in ‘Control.Monad’)

Oh. Okay we'll check out the one in Prelude by prefixing it.

...

:i Prelude.mapM Prelude.mapM :: Monad m => (a -> m b) -> [a] -> m [b] -- Defined in ‘Control.Monad’ :t traverse traverse :: (Traversable t, Control.Applicative.Applicative f) => (a -> f b) -> t a -> f (t b)

Now let's specialise traverse for Monad and List, to show that it's equivalent.

...

:t (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

<interactive>:1:2: Could not deduce (Control.Applicative.Applicative m1) arising from a use of ‘traverse’ from the context (Monad m) bound by the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] at Top level or from (Monad m1) bound by an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] at <interactive>:1:2-50 Possible fix: add (Control.Applicative.Applicative m1) to the context of an expression type signature: Monad m1 => (a1 -> m1 b1) -> [a1] -> m1 [b1] or the inferred type of it :: Monad m => (a -> m b) -> [a] -> m [b] In the expression: (traverse :: Monad m => (a -> m b) -> [a] -> m [b])

Curses! Looking forward to AMP. Insert explanation about how Applicative should be a superclass of Monad. Pretend that it really is. Wave hands about.

...

:t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b])

<interactive>:1:14: Not in scope: type constructor or class ‘Applicative’

Okay this is getting silly.

...

import Control.Applicative :t (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) (traverse :: Applicative m => (a -> m b) -> [a] -> m [b]) :: Applicative m => (a -> m b) -> [a] -> m [b]

This is *really* *really* common teaching/learning scenario using the current state of the Prelude.

On Sat, Jan 31, 2015 at 12:59 PM, Johan Tibell wrote:

...

I re-read the goals* of the proposal: https://wiki.haskell.org/Foldable_Traversable_In_Prelude

"One goal here is that people should be able to use methods from these modules without the need of a qualified import -- i.e. to prevent clash in the namespace, by resolving such in favor of the more generalized versions. Additionally, there are some new methods added to the Foldable class because it seems to be the "right" thing."

Before FTP, I would have to write this to use Foldable/Traversable:

import qualified Data.Foldable as F

The import is qualified as to not collide with the Prelude.

If I have code that needs to be compatible with more than one GHC release (I typically need compatibility with the last 3 major releases), what do I have to write post-FTP? Since I cannot rely on the Prelude being generalized (because I might be compiling with a pre-FTP compiler) I need to either write:

#if MIN_VERSION_base(x,y,z) -- Get Foldable etc from Prelude #else import Data.Foldable (...) import Prelude hiding (...) -- the same #endif

Which is terrible. Alternatively I can write

import qualified Data.Foldable as F

but now nothing is gained over the pre-FTP state. Only after 3+ years (at the current GHC release phase) I can drop that one extra import. One out of perhaps 20. That seems quite a small gain given that we will then have made Data.List a very confusing module (it's essentially Data.Foldable under a different name), broken some code due to added type ambiguity, and also removed one of the simpler ways to remove that ambiguity, which would have been to import one of the monomorphic list functions that no longer exist.

* People tell me that there are other goals, but they aren't really stated clearly anywhere.

-- Johan

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

-- Regards,

Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Edward Kmett wrote

*"The Prelude now causes no import conflicts with any names anywhere in base."*

For me, the advantage of Foldable/Traversable is more about managing the import list in my head than the import list in my code (although it's worth it to clear up that mess alone). Multiple functions with the same name and intent, but independently defined for each concrete data type, do not make for a pleasant learning or programming experience. -- View this message in context: http://haskell.1045720.n5.nabble.com/Drastic-Prelude-changes-imminent-tp5764... Sent from the Haskell - Libraries mailing list archive at Nabble.com.

Actually, I don't consider this instance a hack. One of the things that convinced me of the elegance of this `Writer`-like instance for tuples is that you can use it to implement another instance: instance (Monoid a, Monoid b) => Monoid (a, b) where mempty = pure mempty mappend = liftA2 mappend This is a special case of the more general pattern of lifting `Monoid`s over `Applicative`s: instance (Applicative f, Monoid b) => Monoid (f b) where mempty = pure mempty mappend = liftA2 mappend On 2/11/15 1:01 PM, Henning Thielemann wrote:

Am 01.02.2015 um 23:02 schrieb Edward Kmett:

...

On Sun, Feb 1, 2015 at 4:39 PM, Iavor Diatchki mailto:iavor.diatchki@gmail.com> wrote:

Really? This is not obvious at all. Is it because of the odd `Applicative` instance for pairs? I am saying 'odd' because there are no similar instances for tuples of higher arity; there are also no instance for `Monad` for the same type, which makes as much sense, I guess. Obviously this is just an opinion, but I don't think these instances fit pairs naturally, and make for confusing looking code.

instance Monoid m => Monad ((,) m)

is the "unnamed" writer monad that corresponds to the

That looks like a hack which requires another hack (namely moving Monoid to Prelude). Would it have been too difficult to use the Writer monad from transformers or mtl? Btw. I think if I use this instance this is certainly by accident. I prefer to be explicit about my programming intentions and would use the Writer monad.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Hello,

I think that "dead end #3" is the right choice, and keep Foldable etc. in their own modules. We have a module system, we should use it. In general, I think if we are to "burn bridges" it should be by moving things out of the Prelude, not adding more abstractions there.

It'd strike me as incredibly arbitrary to wind up with some weird half-way

On Sun, Feb 1, 2015 at 2:59 PM, Iavor Diatchki wrote: point where mapM and traverse generalize but where _obviously_ related combinators such as mapM_ and traverse_ don't. For the sake of concreteness, here are some of the things I don't like

about the current design:

(e.g., why are 'sum' and 'product' there??).

We have a number of competing concerns to balance in Foldable. 1.) We absolutely cannot have rampant performance regressions. 2.) Existing code should continue to as large an extent as much possible. 3.) We shouldn't randomly change the semantics of existing code. 4.) We want a minimal class. Removing `sum` and `product` from the class serves desire #4 perfectly. However, it incurs problems for desire #1 and #3. There are really 3 camps about how Data.Foldable.sum should be implemented. Camp 1) sum = foldl (+) 0 is the implementation that is in the Haskell Report. This implementation is actually pretty much maximally bad for all usecases. It has uses foldl so it leaks all sorts of memory while building up the structure that it finally folds. Camp 2) sum = foldl' (+) 0 is the implementation that folks like Duncan would rather see. This kills most of the space and performance problems with the standard implementation, but switching could introduce bottoms in existing programs. Camp 3) sum = getSum . foldMap sum is the implementation that ensures that it doesn't destroy the asymptotics of the number of uses of 'mappend' in foldMap. The right container can readily fold 2^20th a's with 20 mappends. The best the camp 2 solution can do is still 1 million steps, but at least those steps aren't randomly leaking space. The downside of this is that this implementation can use more stack in the cases where ghc was smart enough to monomorphize the original Prelude solution to a number type where it could figure out via strictness that it could do an efficient left fold. Similarly consider maximum from the Report: maximum, minimum :: (Ord a) => [a] -> a maximum [] = error "Prelude.maximum: empty list" maximum xs = foldl1 max xs vs. maximum from the Data.Foldable: -- | The largest element of a non-empty structure. maximum :: (Foldable t, Ord a) => t a -> a maximum = foldr1 max There are two concerns that caused us to decide to include these as members in the class as part of 7.10. * Picking one *silently changes the semantics of existing programs* with consequences that are hard to predict. * A separate libraries@ proposal was filed by David Feuer and Reid Barton was filed *and passed* to add a number of additional members to Data.Foldable for yet another concern, which is to allow these operations to work with better asymptotics for many containers. Foldable only ever consumes 'f' in negative position, it never produces an 'f', which means that unlike every other class we have that works parametrically over something of kind * -> *, we can know something about 'a'. This enables folks to construct containers where many of these operations are O(log n) or O(1). Included, perhaps gratuitously, in that proposal were the additional generalizations for null and length. Those who proposed these extensions had a desire to allow users to improve the asymptotics of these operations, and nobody who is now declaiming the size of the class objected back in September when the proposal went through on the mailing list to a number of enthusiastic +1's and a couple of lukewarm dislikes. Personally I'm on the fence about the latter set of additions that exist purely to improve the asymptotics of some operations over and above what you should be able to get by computing directly with a given choice of monoid, but I'm also not particularly fond of saying that what we're going to do with everything that passes through the proposal process is wait 3 months until the midnight hour before a release and then recolor the bikeshed.

- the state of Data.List: I generally prefer that, when possible, a datatype should provide non-overloaded access to its functionality, and in addition, there can be instances to expose the same functionality via an overloaded API.

Personally, I'd just as soon remove these members from this module entirely. They act as an active barrier against proposals such as Neil and Lennart's that for changes like this -- or much more practically, for other changes in the future -- that we consider exposing an alternate Prelude. Data.List derives most of its members as a renaming of PreludeList from the standard report. The problem is the *vast majority* of the users of Data.List, do so by importing it unqualified simply to get 'sort'. They aren't actively seeking to introduce a conflicting import of foldr! The benefit of offering a monomorphic version of these combinators in there compares badly against the level of code breakage that it would introduce. At an earlier stage in the design we had a Data.OldList module, which contained the original signatures, but having it required us to tie the import graph in base in knots. Reintroducing such a module is definitely a viable option, the pain involved in adopting is mostly borne internally by base.

Of course, I can generally work around all of those, so not a big deal. However, it'd be nice if we could come up with a design that, more or less, the whole community thinks is a good one.

The very reason why a core libraries committee was formed was because there was a great deal of frustration with the inability to make headway on these sorts of issues by universal consent. We had a few cases like Ian unilaterally removing Eq and Show as a superclass of Num, but really it was just "who had commit access" that determined what could happen, and nobody who felt like they had the responsibility to ensure the stability of base was balanced against the increasing pressures on it from different fronts. As the Haskell community has grown larger and larger getting every one of thousands of contributors to the ecosystem to agree on every fine detail when you have issues where you have a very large design surface has become progressively more and more improbable. Unanimity is a laudable goal, but often unattainable. That said, I very much believe in establishing as broad of a consensus as possible. -Edward

I didn't say arbitrary. At the risk of an overly complicated example, just because I have it handy: In the following type: rep 1000 "a" folds in logarithmic time, lots of operations actually get to exploit the obliviousness of (*>) and (<*) and (>>) to one argument or the other's values to exploit replication heavily for parts of the result as well. This is needed to match the asymptotics of Fritz Henglein's linear time table joins in a final encoding for instance. data Table a = Table { count :: {-# UNPACK #-} !Int , runTable :: forall r. Monoid r => (a -> r) -> r } instance IsList (Table a) where type Item (Table a) = a fromList = foldMap pure toList = Foldable.toList fromListN n xs = Table n (`foldMap` xs) instance Functor Table where fmap f (Table i m) = Table i $ \k -> m (k.f) instance Foldable Table where foldMap f (Table _ m) = m f foldr f z (Table _ m) = m (Endo . f) `appEndo` z instance Monoid (Table a) where mempty = Table 0 $ \_ -> mempty mappend (Table i m) (Table j n) = Table (i + j) $ \k -> m k `mappend` n k newtype Ap f a = Ap { runAp :: f a } instance (Applicative f, Monoid a) => Monoid (Ap f a) where mempty = Ap (pure mempty) mappend (Ap m) (Ap n) = Ap (liftA2 mappend m n) instance Traversable Table where -- this reassembles the result with sharing! traverse f = runAp . foldMap (Ap . fmap pure . f) instance Applicative Table where pure a = Table 1 $ \k -> k a Table n as <*> Table m bs = Table (n * m) $ \k -> as $ \f -> bs (k . f) Table n as <* Table m _ = Table (n * m) $ \k -> as (rep m . k) Table n _ *> Table m bs = Table (n * m) $ rep n . bs -- peasant multiplication rep :: Monoid m => Int -> m -> m rep y0 x0 | y0 <= 0 = mempty | otherwise = f x0 y0 where f x y | even y = f (mappend x x) (quot y 2) | y == 1 = x | otherwise = g (mappend x x) (quot (y - 1) 2) x g x y z | even y = g (mappend x x) (quot y 2) z | y == 1 = mappend x z | otherwise = g (mappend x x) (quot (y - 1) 2) (mappend x z) bag :: (forall m. Monoid m => (a -> m) -> m) -> Table a bag k = Table (getSum $ k $ \_ -> Sum 1) k instance Monad Table where return a = Table 1 $ \k -> k a as >>= f = bag $ \k -> runTable as $ \a -> runTable (f a) k (>>) = (*>) fail _ = empty instance MonadZip Table where -- we can handle this in a smarter fashion now mzipWith k m n = foldMap pure $ mzipWith k (Foldable.toList m) (Foldable.toList n) munzip m = (fmap fst m, fmap snd m) instance Alternative Table where empty = Table 0 $ \_ -> mempty Table m as <|> Table n bs = Table (m + n) $ \k -> as k `mappend` bs k instance MonadPlus Table where mzero = Table 0 $ \_ -> mempty Table m as `mplus` Table n bs = Table (m + n) $ \k -> as k `mappend` bs k instance MonadFix Table where mfix a2ba = foldMap pure $ mfix (Foldable.toList . a2ba) etc. On Sun, Feb 1, 2015 at 4:17 PM, Roman Cheplyaka wrote:

On 01/02/15 22:56, Edward Kmett wrote:

...

sum = getSum . foldMap sum

is the implementation that ensures that it doesn't destroy the asymptotics of the number of uses of 'mappend' in foldMap.

The right container can readily fold 2^20th a's with 20 mappends.

Fold 2^20 *arbitrary* a's? What kind of container is that?

Roman

On Tue Feb 03 2015 at 6:26:58 PM Erik Hesselink wrote:

On Tue, Feb 3, 2015 at 4:45 PM, Michael Snoyman wrote:

...

Now that I actually write this all out, I'm beginning to think I *am* opposed to the proposal overall, though not by a wide margin. It seems to fall into the "too little, too late" category: the benefits it brings are not massive due to my three points above, and the disruption introduced to the ecosystem is potential massive.

What is the potential massive disruption?

Erik

1. Breakage of existing code due to lack of type inference 2. Breakage of existing tutorials/documentation 3. Confusing error messages I didn't really want to drill into it because that seems to be the point of discussion of this entire thread.

On Tue, Feb 3, 2015 at 2:20 PM, Erik Hesselink wrote:

Some problems like the weirdness of Data.List can be fixed with deprecations later.

Herbert just ported forward a patch to allow deprecated exports, which is needed to do such generalizations, so at least as a consequence of the current brouhaha, we now have the tools to make that particular complaint eventually go away.

Erik _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

On 2015-02-03 at 16:45:22 +0100, Michael Snoyman wrote: [...]

* I have many other complaints about the standard Prelude: partial functions like `head` and lazy I/O being a recommended default being primary in that list.

I think most of us are unsatisfied with the current status quo, which is why the BBP manifesto got so much attention (of which the FTP is just one of the laundry-list items). But we have to start somewhere and rather incrementally, like e.g. low-hanging fruits such as the FTP, before addressing the more complicated items on that manifesto.

* I've personally moved on from the default Prelude and regularly use alternate preludes in my own code (classy-prelude being the most common example, but others existing too). I'm not the only person who does this, and it solves a lot of the problems.

So basically you don't have any stake in the default Prelude anymore? ;) More seriously, there's one downside to the proliferation of alternate Preludes I'm worried about: While they allow for easy experimentation (which is great but hardly useful to base production code on), they also fragment the Haskell-language, as each alternate Prelude is actually a different dialect of Haskell. Having a common Prelude providing the common vocabulary that allows all sub-cultures to co-exist while balancing all/most needs helps keeping the language united. The more the default Prelude fails to satisfy its job of keeping up with the times, the less incentive there is not to resort to create alternative Preludes and suffer from Haskell's version of the Tower-of-Babel. So when in future scenario like that you talk about "Haskell", you'll have to specify which prelude-version of Haskell you mean, and also make sure you pick the respective literature that covers that specific prelude-dialect. And each dialect can potentially lock you into different sub-ecosystem of packages. I'm aware this is extrapolating quite a bit, and it doesn't necessarily have to come to such extremes. Cheers, hvr

It seems to fall into the "too little, too late" category: the benefits it brings are not massive due to my three points above, and the disruption introduced to the ecosystem is potential massive.

I agree that it is too little. There is a lot of brain damage in the prelude that really should be fixed. I don't think its too late though, we can start making forward progress with changes like this. Yes we need many more, but I think there is hope for redemption. We will want to work on our approaches to the upgrade path but several good ideas have already been put forward and tooling can be improved for the people who insist on long backwards compatibility periods. Personally, I'd like to see a cleanup of each section of the prelude, its seriously holding us back.