Proposal for removing transformers dependency

Hello. I'm coming with a proposal for removing transformers dependency from ghc library. The reason for this proposal that it's not possible to build consistent environment where a modern libraries (that depend on a newer transformers or mtl-2.2) and libraries that use ghc API are used together. And often people are tend to use version that is bundled with ghc, even if newer are available. As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers. I have prepared a proof of concept [1], however it doesn't look very clean and it's quite possible that will require some generalization, for example introduction of the ghc-transformers-instances package that will have all required instances. Should I continue doing this? Are there any things to consider and fix? [1] https://github.com/qnikst/ghc/compare/wip/remove-tf -- Alexander

On 2015-01-21 at 16:51:55 +0100, Alexander V Vershilov wrote: [...]
As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers.
...but doesn't this effectively mean that the instances provided by the `ghc` package are of little use or rather can't be easily combined with code that builds on top of the original `transformers` classes everyone else uses? Cheers, hvr

I thought about providing package ghc-transformers-instances, that will
provide instances for transformers's type classes for user. So
ghc-tf-instances will depend on current ghc, and current transformers that
could be provided by user environment.
On Jan 21, 2015 7:15 PM, "Herbert Valerio Riedel"
On 2015-01-21 at 16:51:55 +0100, Alexander V Vershilov wrote:
[...]
As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers.
...but doesn't this effectively mean that the instances provided by the `ghc` package are of little use or rather can't be easily combined with code that builds on top of the original `transformers` classes everyone else uses?
Cheers, hvr

On 2015-01-21 at 17:19:42 +0100, Alexander V Vershilov wrote:
I thought about providing package ghc-transformers-instances, that will provide instances for transformers's type classes for user. So ghc-tf-instances will depend on current ghc, and current transformers that could be provided by user environment.
So the price is seems to be mostly to tolerate orphan instances (and potentially having to copy more code from transformers into ghc, if one wants to use more features in ghc's code). One would have to make sure to keep that package updated everytime a new transformers or ghc package version is released, as well as making sure to always test all supported combinations of ghc/transformers versions before making a new releases. One thing to keep in mind though is that then 'haskeline' (which is needed by GHCi) still remains a consumer of 'transformers', so we'd still have to bundle a 'transformers' package version with GHC even if `ghc` doesn't depend on it anymore. Somewhat related, the `ghc` -> `Cabal` dependency was broken up in GHC 7.10 but we'll still bundle `Cabal` with GHC 7.10. I'm not sure how much this helps Stackage which I assume would constraint transformers to a single version, and therefore force a reinstall of the haskeline version shipped with GHC with a different version of transformers. Fwiw, I welcome decoupling libraries from the GHC distribution as every exposed library adds to the synchronisation-with-upstream-overhead when preparing new GHC releases, in addition to adding implicit version constraints to the package database. With a GHC release-management hat on though: As for GHC 7.10.1, it's rather late... at the very least it needs to get into RC2 (whose cut off is tomorrow) for that to happen. Cheers, hvr

Hi, Am Donnerstag, den 22.01.2015, 08:37 +0100 schrieb Herbert Valerio Riedel:
One thing to keep in mind though is that then 'haskeline' (which is needed by GHCi) still remains a consumer of 'transformers', so we'd still have to bundle a 'transformers' package version with GHC even if `ghc` doesn't depend on it anymore. Somewhat related, the `ghc` -> `Cabal` dependency was broken up in GHC 7.10 but we'll still bundle `Cabal` with GHC 7.10.
although there has been talk about only shipping the .so file with a clash-free name or path... was this not done simply because noone bothered enough to actually do it, or was there a bigger problem? Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

On 22 January 2015 at 09:34, Joachim Breitner
Hi,
Am Donnerstag, den 22.01.2015, 08:37 +0100 schrieb Herbert Valerio Riedel:
One thing to keep in mind though is that then 'haskeline' (which is needed by GHCi) still remains a consumer of 'transformers', so we'd still have to bundle a 'transformers' package version with GHC even if `ghc` doesn't depend on it anymore. Somewhat related, the `ghc` -> `Cabal` dependency was broken up in GHC 7.10 but we'll still bundle `Cabal` with GHC 7.10.
although there has been talk about only shipping the .so file with a clash-free name or path... was this not done simply because noone bothered enough to actually do it, or was there a bigger problem?
There was an email thread about this very problem started by Michael Snoyman last month. As noted later in that thread, https://www.haskell.org/pipermail/ghc-devs/2014-December/007826.html the packages mentioned above and others were added to the ghc-pkg database in order to address https://ghc.haskell.org/trac/ghc/ticket/8919, hence making the problem worse. It sounds to me like Alexander's patch, plus the solution alluded by Joachim above for "invisible" packages that don't clash with ones registered in the ghc-pkg db, would allow us to avoid having any of the following packages leaking into all sandboxes for all users of GHC 7.10 and following: * haskeline * transformers * xhtml * terminfo Perhaps others. That would be a big win. Here's a more ambitious change: if we could make installation of the ghc library itself optional, then we could avoid forcing the installation of a whole swathe of other packages into the global db, including transformers. Best, Mathieu

On 2015-01-22 at 12:40:05 +0100, Boespflug, Mathieu wrote: [...]
It sounds to me like Alexander's patch, plus the solution alluded by Joachim above for "invisible" packages that don't clash with ones registered in the ghc-pkg db, would allow us to avoid having any of the following packages leaking into all sandboxes for all users of GHC 7.10 and following:
* haskeline * transformers * xhtml * terminfo
Perhaps others. That would be a big win.
Btw, there's an alternative hack that could be used. Since the remaining use of haskeline/terminfo/transformers would then be mostly in GHCi (if haven't missed anything), which is luckily not a library, we don't actually need those packages to be turned into DSOs. We could do something similiar to what `haddock` inside the GHC tree does w/ e.g. its attoparsec dependency: just embed those 4 packages into the ghci executable as if they weren't separate packages. Cheers, hvr

On 22 January 2015 at 10:37, Herbert Valerio Riedel
On 2015-01-21 at 17:19:42 +0100, Alexander V Vershilov wrote:
I thought about providing package ghc-transformers-instances, that will provide instances for transformers's type classes for user. So ghc-tf-instances will depend on current ghc, and current transformers that could be provided by user environment.
So the price is seems to be mostly to tolerate orphan instances (and potentially having to copy more code from transformers into ghc, if one wants to use more features in ghc's code).
Yes, if I didn't miss any other relevant issue. It's possible that we will need some coercion between Strict State that comes with GHC and one that is in transformers.
One would have to make sure to keep that package updated everytime a new transformers or ghc package version is released, as well as making sure to always test all supported combinations of ghc/transformers versions before making a new releases.
One thing to keep in mind though is that then 'haskeline' (which is needed by GHCi) still remains a consumer of 'transformers', so we'd still have to bundle a 'transformers' package version with GHC even if `ghc` doesn't depend on it anymore. Somewhat related, the `ghc` -> `Cabal` dependency was broken up in GHC 7.10 but we'll still bundle `Cabal` with GHC 7.10.
From a distro-developer perspective (gentoo) the only relevant library is ghc library, and ghc-bin is an executable, so in the worst case user will end up with having 2 different transformers package. But there will be no transformers madness as no library pulls concrete version.
I'm not sure how much this helps Stackage which I assume would constraint transformers to a single version, and therefore force a reinstall of the haskeline version shipped with GHC with a different version of transformers.
I don't know about a Stackage but at least Gentoo allow to install additional version for packages that are shipped with ghc and ghc library do not depend on it, it also handles a case when versions are equal, and then installation is a noop. Thinking of haskeline user will have problems only in case if user's haskeline version is equal to shipped with GHC but depends on a different tf version. I think that in this case force installing will lead to some level of breakage, however all executables (ghc-bin) and new libraries will continue working. Workaround is to make a minor version bump in haskeline and this can be done purely on a distro-level.
Fwiw, I welcome decoupling libraries from the GHC distribution as every exposed library adds to the synchronisation-with-upstream-overhead when preparing new GHC releases, in addition to adding implicit version constraints to the package database.
With a GHC release-management hat on though: As for GHC 7.10.1, it's rather late... at the very least it needs to get into RC2 (whose cut off is tomorrow) for that to happen.
Great. I'll try to cleanup code, and prepare ghc-tf-instances today, test of the packages that depend on the ghc library in order to understand if ghc-tf-instances solution works and then send diff to a Phab. Every tips, suggestions, or reminders of things that I should check will be gladly accepter.
Cheers, hvr
-- Alexander

| One thing to keep in mind though is that then 'haskeline' (which is | needed by GHCi) still remains a consumer of 'transformers', so we'd | still have to bundle a 'transformers' package version with GHC even if | `ghc` doesn't depend on it anymore. Somewhat related, the `ghc` -> couldn't we invite Judah to remove the transformers dependency for haskeline? It all seems a bit late for 7.10. Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Herbert Valerio Riedel | Sent: 22 January 2015 07:37 | To: Alexander V Vershilov | Cc: ghc-devs@haskell.org | Subject: Re: Proposal for removing transformers dependency | | On 2015-01-21 at 17:19:42 +0100, Alexander V Vershilov wrote: | > I thought about providing package ghc-transformers-instances, that | > will provide instances for transformers's type classes for user. So | > ghc-tf-instances will depend on current ghc, and current | transformers | > that could be provided by user environment. | | So the price is seems to be mostly to tolerate orphan instances (and | potentially having to copy more code from transformers into ghc, if | one wants to use more features in ghc's code). | | One would have to make sure to keep that package updated everytime a | new transformers or ghc package version is released, as well as making | sure to always test all supported combinations of ghc/transformers | versions before making a new releases. | | One thing to keep in mind though is that then 'haskeline' (which is | needed by GHCi) still remains a consumer of 'transformers', so we'd | still have to bundle a 'transformers' package version with GHC even if | `ghc` doesn't depend on it anymore. Somewhat related, the `ghc` -> | `Cabal` dependency was broken up in GHC 7.10 but we'll still bundle | `Cabal` with GHC 7.10. | | I'm not sure how much this helps Stackage which I assume would | constraint transformers to a single version, and therefore force a | reinstall of the haskeline version shipped with GHC with a different | version of transformers. | | Fwiw, I welcome decoupling libraries from the GHC distribution as | every exposed library adds to the synchronisation-with-upstream- | overhead when preparing new GHC releases, in addition to adding | implicit version constraints to the package database. | | With a GHC release-management hat on though: As for GHC 7.10.1, it's | rather late... at the very least it needs to get into RC2 (whose cut | off is tomorrow) for that to happen. | | Cheers, | hvr | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs

Hi, Am Mittwoch, den 21.01.2015, 19:51 +0400 schrieb Alexander V Vershilov:
Should I continue doing this?
with my Debian packaging maintainer hat on: Yes, please do! Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org

Huge +1 from a Stackage standpoint. The transformers dependency (and other
such dependencies) causes me a huge amount of pain.
On Wed Jan 21 2015 at 8:52:57 AM Joachim Breitner
Hi,
Am Mittwoch, den 21.01.2015, 19:51 +0400 schrieb Alexander V Vershilov:
Should I continue doing this?
with my Debian packaging maintainer hat on: Yes, please do!
Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Hi, I also hope that this is integrated into GHC 7.10. --Kazu
Hello.
I'm coming with a proposal for removing transformers dependency from ghc library. The reason for this proposal that it's not possible to build consistent environment where a modern libraries (that depend on a newer transformers or mtl-2.2) and libraries that use ghc API are used together. And often people are tend to use version that is bundled with ghc, even if newer are available.
As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers.
I have prepared a proof of concept [1], however it doesn't look very clean and it's quite possible that will require some generalization, for example introduction of the ghc-transformers-instances package that will have all required instances.
Should I continue doing this? Are there any things to consider and fix?
[1] https://github.com/qnikst/ghc/compare/wip/remove-tf -- Alexander

Ok. In this case, I'm interested in main users of ghc library,
as I have to make sure that they will not be broken by
any changes that I'm introduce? I can take few from hackage
but maybe there are ones that users care about. So I can
validate my solution.
--
Alexander
On 22 January 2015 at 03:11, Kazu Yamamoto
Hi,
I also hope that this is integrated into GHC 7.10.
--Kazu
Hello.
I'm coming with a proposal for removing transformers dependency from ghc library. The reason for this proposal that it's not possible to build consistent environment where a modern libraries (that depend on a newer transformers or mtl-2.2) and libraries that use ghc API are used together. And often people are tend to use version that is bundled with ghc, even if newer are available.
As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers.
I have prepared a proof of concept [1], however it doesn't look very clean and it's quite possible that will require some generalization, for example introduction of the ghc-transformers-instances package that will have all required instances.
Should I continue doing this? Are there any things to consider and fix?
[1] https://github.com/qnikst/ghc/compare/wip/remove-tf -- Alexander
ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Alexander

Don't worry, we've never given any promise about GHC API stability across major versions (or minor versions, for that matter.) Edward Excerpts from Alexander V Vershilov's message of 2015-01-21 22:16:11 -0800:
Ok. In this case, I'm interested in main users of ghc library, as I have to make sure that they will not be broken by any changes that I'm introduce? I can take few from hackage but maybe there are ones that users care about. So I can validate my solution.
-- Alexander
On 22 January 2015 at 03:11, Kazu Yamamoto
wrote: Hi,
I also hope that this is integrated into GHC 7.10.
--Kazu
Hello.
I'm coming with a proposal for removing transformers dependency from ghc library. The reason for this proposal that it's not possible to build consistent environment where a modern libraries (that depend on a newer transformers or mtl-2.2) and libraries that use ghc API are used together. And often people are tend to use version that is bundled with ghc, even if newer are available.
As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers.
I have prepared a proof of concept [1], however it doesn't look very clean and it's quite possible that will require some generalization, for example introduction of the ghc-transformers-instances package that will have all required instances.
Should I continue doing this? Are there any things to consider and fix?
[1] https://github.com/qnikst/ghc/compare/wip/remove-tf -- Alexander
ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

As a result of this thread I have created Phab Diff [1]. It contains
required fixes in compiler and ghc-transformers-instance package.
I have tested result on 2 following usecases:
1. I have installed in a sandox transformers-0.3 and ghc-heap-view
that uses them and ghc-lib. (Doesn't require ghc-tf-instances)
2. I have installed in a sandbox ghc-tf-instaces and ghc-mtl
that uses ghc-lib and ghc-tf-instances. Some fixes were required
but they all were pretty straighforward to implement.
[1] https://phabricator.haskell.org/D626
On 21 January 2015 at 18:51, Alexander V Vershilov
Hello.
I'm coming with a proposal for removing transformers dependency from ghc library. The reason for this proposal that it's not possible to build consistent environment where a modern libraries (that depend on a newer transformers or mtl-2.2) and libraries that use ghc API are used together. And often people are tend to use version that is bundled with ghc, even if newer are available.
As transformers usage are quite limited in ghc, and it's really relevant in ghc-bin, it's possible to duplicate the code, and provide required fixes in ghc-bin.cabal. As a result ghc uses it's own MonadIO, MonadControl and Strict State, and ghc-bin.cabal (ghc/*) uses ones from transformers.
I have prepared a proof of concept [1], however it doesn't look very clean and it's quite possible that will require some generalization, for example introduction of the ghc-transformers-instances package that will have all required instances.
Should I continue doing this? Are there any things to consider and fix?
[1] https://github.com/qnikst/ghc/compare/wip/remove-tf -- Alexander
-- Alexander
participants (8)
-
Alexander V Vershilov
-
Boespflug, Mathieu
-
Edward Z. Yang
-
Herbert Valerio Riedel
-
Joachim Breitner
-
Kazu Yamamoto
-
Michael Snoyman
-
Simon Peyton Jones