Hi Simon, As a bit of context here, CLC issue 105 was definitely part of the discussions leading to the refinement of this proposal (and we also brought some of that over from the discussions that you participated in in the Stability Working Group). Please feel free to get in touch with any questions or concerns - as the ones closest to GHC, you're of course best placed to see things that we may have overlooked. Thanks! David On Thu, Mar 16, 2023 at 10:58 AM Simon Peyton Jones < simon.peytonjones@gmail.com> wrote:

Thanks Laurent,

In fact this fits with our thinking in https://github.com/haskell/core-libraries-committee/issues/105. We'll discuss and Ben/Matthew will get back to you.

Simon

On Sun, 12 Mar 2023 at 13:39, Laurent P. René de Cotret < laurent.decotret@outlook.com> wrote:

...

Dear GHC developers,

In recent weeks, John Ericson has fine-tuned a Haskell Foundation Technical Proposal to split `base` into two libraries: `ghc-base` and `base`, the latter simply re-exporting everything for `ghc-base` (for now). You can read about the rationale and specifics more in details in the proposal itself: https://github.com/haskellfoundation/tech-proposals/pull/47

Note that this proposal has recently been streamlined into a form which is more focused than its initial state, and might be worth a re-read.

The Haskell Foundation Technical Working Group has reached a consensus that this work will benefit the Haskell community. Moreover, the Haskell Foundation has agreed to spend some of its resources to implement this proposal, which would start by ensuring the completion of MR7898 ( https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7898).

This work will affect GHC developers. Therefore, the Technical Working Group would like to get buy-in from the GHC developers before formally accepting this proposal.

Best regards, Laurent P. René de Cotret on behalf of the Haskell Foundation Technical Working Group _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Would this include making those modules not hidden in ghc base? There’s been a few times where that status made it quite hard to build documentation for those modules! On Tue, Mar 21, 2023 at 1:16 PM Ben Gamari wrote:

Laurent P. René de Cotret writes:

...

Dear GHC developers,

In recent weeks, John Ericson has fine-tuned a Haskell Foundation Technical Proposal to split `base` into two libraries: `ghc-base` and `base`, the latter simply re-exporting everything for `ghc-base` (for now). You can read about the rationale and specifics more in details in the proposal itself: https://github.com/haskellfoundation/tech-proposals/pull/47

Note that this proposal has recently been streamlined into a form which is more focused than its initial state, and might be worth a re-read.

The Haskell Foundation Technical Working Group has reached a consensus that this work will benefit the Haskell community. Moreover, the Haskell Foundation has agreed to spend some of its resources to implement this proposal, which would start by ensuring the completion of MR7898 (https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7898).

This work will affect GHC developers. Therefore, the Technical Working Group would like to get buy-in from the GHC developers before formally accepting this proposal.

Hi Laurent,

In general I am quite supportive of this proposal. I have discussed the idea with John on several occassions and agree that separating the implementation of `base` from its user-facing interfaces with a package boundary would simplify life for both users and GHC's maintainers (c.f. [1]).

I also threw together my own implementation of the idea in a few hours some weeks back (having forgotten about John's effort); this can be found in the wip/ghc-base branch [2]. From that experience I have no doubts that this idea is feasible. The only issues that I am slightly unsure of are:

* whether/how to prevent `ghc-base` references from seeping into error messages.

* which interfaces should be re-exposed from `base`. In [2] we propose that a fair number of interfaces be marked as GHC-internal. Those which are marked [3] as "hidden" should likely be exposed only via `ghc-base`. However, for compatibility reasons we may decide to continue exporting some subset of "internal" modules (with frozen export lists) from `base`.

Regardless, I am very happy to see this split move forward and am grateful to John for his work in this direction.

Cheers,

- Ben

[1] https://github.com/haskell/core-libraries-committee/issues/146 [2] https://gitlab.haskell.org/ghc/ghc/-/tree/wip/ghc-base [3] https://docs.google.com/spreadsheets/d/1WmyYLbJIMk9Q-vK4No5qvKIIdIZwhhFFlw6i... _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

i mean exactly the hidden status which means you cant directly go to the highlighted source. My recollection is that the only justification for that was as a mechanism for "foot cannons here," which itself predates the more modern *.Internal module norms we hew to today. On Tue, Mar 21, 2023 at 8:21 PM Ben Gamari wrote:

Carter Schonwald writes:

...

Would this include making those modules not hidden in ghc base? There’s been a few times where that status made it quite hard to build documentation for those modules!

What do you mean concretely? In most cases modules would remain exposed (that is, importable). However, I doubt there will be any change in their Haddock status (e.g. not-home or hidden).

Cheers,

- Ben

Hi all, In case it's useful, I made my best attempt to summarize the state of stakeholder discussion here: https://github.com/haskellfoundation/tech-proposals/pull/47#issuecomment-147... All the best, David

I switched all the HADDOCK hide to not-home in base a couple of years ago, but I see a couple of new ones have snuck in in the meantime. I would suggest adding a lint against hiding in GHC CI. Adam On Wed, 22 Mar 2023, at 08:37, David Christiansen via ghc-devs wrote:

...

i mean exactly the hidden status which means you cant directly go to the highlighted source.

Isn't this more of a limitation in the tool that generates the highlighted source? It seems to me to be more reasonable to make that tool show hidden modules' source, rather than to change the API of packages to accommodate the tool.

David

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hi, Though I'm no longer a very active GHC developer I do have some questions. Overall I'm in support of this but I think I'd rather see an outright split from the start rather than first having a forwarder library. The reason for this is that I'm afraid of the performance impact of having this intermediate layer. For statically linked programs this means at least an additional load and branch on every call to a standard library. This would for instance affect Windows quite heavily. I'm not sure the impact is mitigated by branch prediction and prefetching. At the least it'll polute your L2 cache much more than before. For dynamically linked we could potentially use symbol preemption to remove the forwarding or on Windows redirect using import libraries. Now maybe I'm overestimating the impact this would have, but I'd very much like to see some numbers on a small-ish experiment to see what impact (if any) there are and what mitigation we can do. Typically it's quite hard to optimize after the fact. Maybe I've missed it in there. Proposal, but can the compiler remove the forwarding? i.e. Can the calls be specialized directly to the definition one? If so it'll break having alternative standard libs at runtime? Kind regards, Tamar On Fri, Mar 24, 2023, 04:47 Ben Gamari wrote:

"Adam Sandberg Eriksson" writes:

...

I switched all the HADDOCK hide to not-home in base a couple of years ago, but I see a couple of new ones have snuck in in the meantime. I would suggest adding a lint against hiding in GHC CI.

Sounds reasonable to me. Perhaps someone could open a ticket to track this?

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Phyx writes:

Hi,

Though I'm no longer a very active GHC developer I do have some questions.

Overall I'm in support of this but I think I'd rather see an outright split from the start rather than first having a forwarder library.

The reason for this is that I'm afraid of the performance impact of having this intermediate layer.

For statically linked programs this means at least an additional load and branch on every call to a standard library. This would for instance affect Windows quite heavily. I'm not sure the impact is mitigated by branch prediction and prefetching. At the least it'll polute your L2 cache much more than before.

For dynamically linked we could potentially use symbol preemption to remove the forwarding or on Windows redirect using import libraries.

Now maybe I'm overestimating the impact this would have, but I'd very much like to see some numbers on a small-ish experiment to see what impact (if any) there are and what mitigation we can do.

Typically it's quite hard to optimize after the fact. Maybe I've missed it in there. Proposal, but can the compiler remove the forwarding? i.e. Can the calls be specialized directly to the definition one? If so it'll break having alternative standard libs at runtime?

I highly doubt that this split will have any measurable overhead. Reexporting a definition defined in one module from another module via an export list does not produce any code at all; importing such a declaration is equivalent to importing the definition from the defining module. If for some reason we can't in some cases directly reexport then we would likely rather have a some very trivial bindings that GHC would be quite eager to inline. Cheers, - Ben

Phyx writes:

...

I highly doubt that this split will have any measurable overhead. Reexporting a definition defined in one module from another module via an export list does not produce any code at all; importing such a declaration is equivalent to importing the definition from the defining module.

Ah right, I can see how that's true at the Haskell level but..

...

If for some reason we can't in some cases directly reexport then we would likely rather have a some very trivial bindings that GHC would be quite eager to inline.

Sure, I can see how you'd inline based on the haskell contract, I can't see how you avoid the compile time overhead when compiling the library. If you have a haskell library

module Test (Control.Monad.when, Control.Applicative.many) where

import Control.Monad(when) import Control.Applicative(many)

compiling it:

ghc test.hs [1 of 1] Compiling Test ( test.hs, test.o )

which still contains the closure for the library.

I'm a bit lost. When I compile this module I get an object which contains no references to `when` or `many`. The only references to these two symbols should be in the interface file. What am I missing? Cheers, - Ben

Phyx writes:

...

I highly doubt that this split will have any measurable overhead. Reexporting a definition defined in one module from another module via an export list does not produce any code at all; importing such a declaration is equivalent to importing the definition from the defining module.

Ah right, I can see how that's true at the Haskell level but..

...

If for some reason we can't in some cases directly reexport then we would likely rather have a some very trivial bindings that GHC would be quite eager to inline.

Sure, I can see how you'd inline based on the haskell contract, I can't see how you avoid the compile time overhead when compiling the library. If you have a haskell library

module Test (Control.Monad.when, Control.Applicative.many) where

import Control.Monad(when) import Control.Applicative(many)

compiling it:

ghc test.hs [1 of 1] Compiling Test ( test.hs, test.o )

To clarify, what is happening in this program is not inlining. Rather when you re-export `Control.Monad.when` from `Test` GHC merely records this fact in the interface file it produces for `Test`. No code is generated for `when` in `Test`. Cheers, - Ben