June 2021 - ghc-devs - Haskell.org

The future of Phabricator
by Ben Gamari 19 Sep '21

19 Sep '21

TL;DR. For several reasons I think we should consider alternatives to Phabricator. My view is that GitLab seems like the best option. Hello everyone, Over the past year I have been growing increasingly weary of our continued dependence on Phabricator. Without a doubt, its code review interface is the best I have used. However, for a myriad of reasons I am recently questioning whether it is still the best tool for GHC's needs. # The problem There are a number of reasons why I am currently uncertain about Phabricator. For one, at this point we have no options for support in the event that something goes wrong as the company responsible for Phabricator, Phacility, has closed their support channels to non-paying customers. Furthermore, in the past year or two Phacility has been placing their development resources in the parts their customers pay them for, which appear to be much different that the parts that we actively use. For this reason, some parts that we rely on seem oddly half-finished. This concern was recently underscored by some rather unfortunate misfeatures in Harbormaster which resulted in broken CI after the Hadrian merge and now apparent bugs which have made it difficult to migrate to the CircleCI integration we previously prepared. Perhaps most importantly, in our recent development priorities survey our use of Phabricator was the most common complaint by a fair margin, both in case of respondents who have contributed patches and those who have not. On the whole, past contributors and potential future contributors alike have strongly indicated that they want a more Git-like experience. Of course, this wasn't terribly surprising; this is just the most recent case where contributors have made this preference known. Frankly, in a sense it is hard to blame them. The fact that users need to install a PHP tool, Arcanist, to contribute anything but documentation patches has always seemed like unnecessary friction to me and I would be quite happy to be rid of it. Indeed we have had a quite healthy number of GitHub documentation patches since we started accepting them. This makes me thing that there may indeed be potential contributoes that we are leaving on the table. # What to do With Rackspace support ending at the end of year, now may be a good time to consider whether we really want to continue on this road. Phabricator is great at code review but I am less and less certain that it is worth the maintenance uncertainty and potential lost contributors that it costs. Moreover, good alternatives seem closer at-hand than they were when we deployed Phabricator. ## Move to GitHub When people complain about our infrastructure, they often use GitHub as the example of what they would like to see. However, realistically I have a hard time seeing GitHub as a viable option. Its feature set is simply insufficient enough to handle the needs of a larger project like GHC without significant external tooling (as seen in the case of Rust-lang). The concrete reasons have been well-documented in previous discussions but, to summarize, * its review functionality is extremely painful to use with larger patches * rebased patches lead to extreme confusion and lost review comments * it lacks support for pre-receive hooks, which serve as a last line of defense to prevent unintended submodule changes * its inability to deal with external tickets is problematic * there is essentially no possibility that we could eventually migrate GHC's tickets to GitHub's ticket tracker without considerable data loss (much less manage the ticket load that would result), meaning that we would forever be stuck with maintaining Trac. * on a personal note, its search functionality has often left me empty-handed On the whole, these issues seem pretty hard to surmount. ## Move to GitLab In using GitLab for another project over the past months I have been positively surprised by its quality. It handles rebased merge requests far better than GitHub, has functional search, and quite a usable review interface. Furthermore, upstream has been extremely responsive to suggestions for improvement [1]. Even out-of-the-box it seems to be flexible enough to accommodate our needs, supporting integration with external issue trackers, having reasonable release management features, and support for code owners to automate review triaging (replacing much of the functionality of Phabricator's Herald). Finally, other FOSS projects' [3] recent migrations from Phabrictor to GitLab have shown that GitLab-the-company is quite willing to offer help when needed. I took some time this weekend to try setting up a quick GHC instance [2] to play around with. Even after just a few hours of playing around I think the result is surprisingly usable. Out of curiosity I also played around with importing some tickets from Trac (building on Matt Pickering's Trac-to-Maniphest migration tool). With relatively little effort I was even able to get nearly all of our tickets (as of 9 months ago) imported while preserving ticket numbers (although there are naturally a few wrinkles that would need to be worked out). Naturally, I think we should treat the question of ticket tracker migration as an orthogonal one to code review, but it is good to know that this is possible. ## Continue with Phabricator Continuing with Phabricator is of course an option. Its review functionality is great and it has served us reasonably well. However, compared to GitLab and even GitHub of today its features seem less distinguished than they once did. Moreover, the prospect of having to maintain a largely stagnant product with no support strikes me as a slightly dangerous game to play. Working around the issues we have recently encountered has already cost a non-negligible amount of time. # The bottom line If it wasn't clear already, I think that we should strongly consider a move to GitLab. At this point it seems clear that it isn't going to vanish, has a steady pace of development, is featureful, and available. However, these are just my thoughts. What do you think? Cheers, - Ben [1] 11.4 will ship with a file tree view in the code review interface, which I reported (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is one of the Phabricator features I missed the most during review [2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641 [3] The GNOME and freedesktop.org projects have recently migrated, the former from a hodge-podge of self-hosted services and the latter from Phabricator

17 40

New implementation for `ImpredicativeTypes`
by Alejandro Serrano Mena 08 Sep '21

08 Sep '21

Hi all, As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a… you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards, Alejandro

8 23

Potential improvement in compacting GC
by Ömer Sinan Ağacan 03 Sep '21

03 Sep '21

Hello, I recently implemented the algorithm used by GHC's compacting GC in another project. The algorithm (after marking) makes two passes over the heap /generation. In GHC, these passes are implemented in [1] and in the next function. In my implementation I tried 3 ways of implementing these passes, one of which is the same as GHC's code, and benchmarked each version. I found that the fastest implementation is not what's used in GHC, but it could easily be used. I should mention that my code targets Wasm, and I benchmarked Wasm instructions executed. Not CPU cycles, CPU instructions, or anything like that. It's very likely that the results will be different when benchmarking code running on actual hardware. Secondly, in my case the heap is mostly dead (residency is low). In GHC, compaction for the oldest generation is enabled when residency passes a threshold, so the assumption is the heap is mostly live. I'm guessing this should also make some difference. Anyway, the first implementation I tried was similar to GHC's scan, but I did scan += object_size(scan); instead of bumping scan by one, as GHC does in [2]. This was the slowest version. Second implementation did the same as GHC (bumped scan by one). This was faster, but still slower than the next version. What I found to be the best is scanning the bitmap, not the heap. The bitmap iterator reads one word at a time. In each iteration it checks if the bitmap word is 0. In GHC, in the best case this can skip 16 words on heap on 32-bit systems, and 32 words on 64-bit. Reminder: we need two bits per object in the bitmap, see [3]. (this is not the case in my implementation so the payoff is better) When the bitmap word is not 0 I use "count trailing zeros" (or "count leading zeros" depending on the bitmap implementation) to get the number of words to skip. This is a single instruction on Wasm and x86 (TZCNT or LZCNT, available via __builtin_ctzl and __builtin_clzl in gcc). So instead of skipping one word at a time, this can potentially skip 16 words (or 32 on 64-bit architectures). When that's not possible, it can still skip multiple words by using ctz/clz. Ömer [1]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt… [2]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt… [3]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt…

5 6

Is simplified subsumption really necessary?
by Chris Smith 24 Jul '21

24 Jul '21

This might be in the "ship has sailed" territory, but I'd like to bring it up anyway. https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-s… says: Suppose GHC lacked all four features, and someone proposed adding them. > That proposal would never leave the launchpad. > Let's test that hypothesis. I've been spending increasing amounts of time fighting against simplified subsumption while porting Haskell code to GHC 9.0. It's not that any specific instance of this problem is hard to fix; rather, it's that it completely screws up my intuition about what should be valid Haskell. It doesn't help that HLS still requires 8.10.4, so I usually don't find out I've broken my libraries for GHC 9.0 until continuous integration kicks in. At this point, it's become fairly routine that my code that works fine with 8.10.4 is broken with 9.0, and this makes me sad. Understandably, eta expansion reducing the strictness of terms is bad. But wouldn't it be possible to choose a desugaring with seq that doesn't do so?

4 6

Rewriting plugins: request for feedback
by Sam Derbyshire 23 Jul '21

23 Jul '21

Hi everyone, Recent refactors to the constraint solver have complicated the story for authors of type-checking plugins. Following a suggestion by Christiaan Baaij, and aided by Richard Eisenberg, I'm working on adding the ability for type-checking plugins to rewrite type family applications directly (hooking straight into GHC's type-family rewriting code). The goal is to ease migration for the plugin authors, hopefully simplifying the common operation of rewriting type family applications. Let me now describe the current API for this new functionality, for which I would like feedback. -------------------------------------------------------------------------- A new field is added to the TcPlugin data type, namely tcPluginRewrite :: s -> UniqFM TyCon TcPluginRewriter Here s is the existential plugin state that users can choose as they wish for their own plugin. Plugins thus register which type family TyCons they are interested in rewriting, like so: myTcPluginRewrite :: MyPluginState -> UniqFM TyCon TcPluginRewriter myTcPluginRewrite s@(MyPluginState {myFam1TyCon, myFam2TyCon}) = listToUFM [ (myFam1TyCon, myFam1Rewriter s), (myFam2TyCon, myFam2Rewriter s) ] For each type family TyCon, the plugin should provide a rewriting function (corresponding to myFam1Rewriter and myFam2Rewriter above), of type type TcPluginRewriter = [Ct] -> [TcType] -> TcPluginM TcPluginRewriteResult This means that a rewriting function is supplied with the current Given constraints and the arguments of the (guaranteed to be saturated) type-family application, and should return a result of type: data TcPluginRewriteResult where TcPluginRewriteError :: (Diagnostic a, Typeable a) => a -> TcPluginRewriteResult TcPluginNoRewrite :: TcPluginRewriteResult TcPluginRewriteTo :: { rewriteTo :: TcType , rewriteEvidence :: TcCoercion } -> TcPluginRewriteResult That is, the plugin can specify what the type-family application should be rewritten to, throw an error, or do nothing. -------------------------------------------------------------------------- I'm hoping that the components of existing type-checking plugins concerned with type-family applications can instead use this mechanism and achieve more robust results by doing so, with less effort. I would like to hear from plugin authors whether this API suits their needs. If you know of someone not on this list that might be interested, please invite them to join the conversation. The current state of this patch can be found in the GHC MR 5909: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5909 Thanks, Sam

3 9

Breaking changes to the base library
by Harendra Kumar 04 Jul '21

04 Jul '21

I see the following errors when compiling with ghc head version: $ ghc-stage2 --version The Glorious Glasgow Haskell Compilation System, version 9.3.20210608 $ cabal build --with-compiler ghc-stage2 --allow-newer Data/Colour/CIE.hs:80:12: error: Ambiguous occurrence ‘sum’ It could refer to either ‘Prelude.sum’, imported from ‘Prelude’ at Data/Colour/CIE.hs:25:8-22 (and originally defined in ‘Data.Foldable’) or ‘Data.List.sum’, imported from ‘Data.List’ at Data/Colour/CIE.hs:41:1-16 (and originally defined in ‘GHC.List’) | 80 | total = sum $ map fst l | ^^^ Can someone briefly describe this change and what's the recommended way of fixing this? Just hide the Data.List definition? I do not see this mentioned in the release notes of 9.2/9.4 here: https://ghc.gitlab.haskell.org/ghc/doc/users_guide/9.2.1-notes.html -harendra

8 8

Loading a typechecked module and then using it immediately as a package
by Erdi, Gergo 29 Jun '21

29 Jun '21

PUBLIC Hi, I have the following to .hs files: 1. MyLib.hs: module MyLib where ... 2. Test.hs: {-# LANGUAGE PackageImports #-} module Test where import "my-pkg" MyLib ... I would like to parse/typecheck/load MyLib.hs into some Unit "my-unit", then add that to the package "my-pkg", and then typecheck Test.hs, all in-proc using the GHC API, without putting any other files on disk. How do I do that? What I tried is loading MyLib.hs after setting the homeUnitId in the DynFlags to "my-unit", then editing the packageNameMap in the unitState of the DynFlags to may "my-pkg" to "my-unit": setHomeUnit :: (GhcMonad m) => UnitId -> m () setHomeUnit unitId = do dflags <- getSessionDynFlags modifySession $ \h -> h{ hsc_dflags = dflags{ homeUnitId = unitId } } registerUnit :: (GhcMonad m) => PackageName -> UnitId -> m () registerUnit pkg unitId = modifySession $ \h -> h{ hsc_dflags = addUnit $ hsc_dflags h } where addUnit dflags = dflags { unitState = let us = unitState dflags in us { packageNameMap = M.insert pkg (Indefinite unitId Nothing) $ packageNameMap us } } pipeline = do setHomeUnit myUnit loadModule =<< typecheckModule =<< parseModule =<< modSumarryFor "MyLib" registerUnit myPkg myUnit setHomeUnit mainUnitId typecheckModule =<< parseModule =<< modSumarryFor "Test" Alas, this doesn't work: the import of `MyLib` from `my-pkg` fails with: input/linking/Test.hs:5:1: error: Could not find module 'MyLib' It is not a module in the current program, or in any known package. TBH I'm not very surprised that it didn't work - that registerUnit function is doing some pretty deep surgery on the unitState that probably breaks several invariants. Still, I wasn't able to find a better way - all the functions in GHC.Unit.State seem to be for querying only. Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

4 5

Re: Loading a typechecked module and then using it immediately as a package
by Erdi, Gergo 29 Jun '21

29 Jun '21

PUBLIC Should I? OK, I just tried calling `flushFinderCaches` after I change the home unit to `mainUnitId`, but I still get exactly the same behaviour: `findInstalledHomeModule` returns `InstalledFound` and things go downhill from there. -----Original Message----- From: Matthew Pickering <matthewtpickering(a)gmail.com> Sent: Tuesday, June 29, 2021 6:34 PM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Cc: ghc-devs(a)haskell.org Subject: [External] Re: Loading a typechecked module and then using it immediately as a package ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. Are you clearing the FinderCache? On Tue, Jun 29, 2021 at 11:14 AM Erdi, Gergo <Gergo.Erdi(a)sc.com> wrote: > > PUBLIC > > I don't know yet what's going on, but one thing I did notice is that `findInstalledHomeModule` returns `InstalledFound` for `MyLib`, which doesn't sound right to me -- `MyLib` should come from the "fake-uid" unit, and `Test` is typechecked in the `mainUnitId`. > > -----Original Message----- > From: Erdi, Gergo > Sent: Tuesday, June 29, 2021 5:51 PM > To: Matthew Pickering <matthewtpickering(a)gmail.com> > Subject: Re: Loading a typechecked module and then using it immediately as a package > > PUBLIC > > I tried moving `MyLib.hs` into a directory different than `Test.sh`, but the error message still refers to its correct location! So this error is not guessing the file name of where `MyLib.hs` could be loaded from; instead, it seems to refer correctly to where the module (previously loaded) was. Hmm. > > -----Original Message----- > From: Matthew Pickering <matthewtpickering(a)gmail.com> > Sent: Tuesday, June 29, 2021 5:04 PM > To: Erdi, Gergo <Gergo.Erdi(a)sc.com> > Subject: [External] Re: Loading a typechecked module and then using it immediately as a package > > > Do you have a `MyLib.hs` source file? If you move that somewhere else (another folder) then things might work? > > On Tue, Jun 29, 2021 at 9:41 AM Erdi, Gergo <Gergo.Erdi(a)sc.com> wrote: > > > > PUBLIC > > > > What's weird about it is that if I print the `moduleNameProvidersMap`, I can see `MyLib` inside, and it looks no different than any other module from e.g. `ghc-prim` that I can import into `Test.hs` without any package qualification. Also, why does the error message refer to the file name `input/MyLib.hs`? Why does GHC even know that that is where it would have to be loaded from, if it weren't to be used from an already loaded package? > > > > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations > > Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm > > Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. > > Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. > > Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. > > Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

1 0

RE: Loading a typechecked module and then using it immediately as a package
by Erdi, Gergo 29 Jun '21

29 Jun '21

PUBLIC I don't know yet what's going on, but one thing I did notice is that `findInstalledHomeModule` returns `InstalledFound` for `MyLib`, which doesn't sound right to me -- `MyLib` should come from the "fake-uid" unit, and `Test` is typechecked in the `mainUnitId`. -----Original Message----- From: Erdi, Gergo Sent: Tuesday, June 29, 2021 5:51 PM To: Matthew Pickering <matthewtpickering(a)gmail.com> Subject: Re: Loading a typechecked module and then using it immediately as a package PUBLIC I tried moving `MyLib.hs` into a directory different than `Test.sh`, but the error message still refers to its correct location! So this error is not guessing the file name of where `MyLib.hs` could be loaded from; instead, it seems to refer correctly to where the module (previously loaded) was. Hmm. -----Original Message----- From: Matthew Pickering <matthewtpickering(a)gmail.com> Sent: Tuesday, June 29, 2021 5:04 PM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Subject: [External] Re: Loading a typechecked module and then using it immediately as a package Do you have a `MyLib.hs` source file? If you move that somewhere else (another folder) then things might work? On Tue, Jun 29, 2021 at 9:41 AM Erdi, Gergo <Gergo.Erdi(a)sc.com> wrote: > > PUBLIC > > What's weird about it is that if I print the `moduleNameProvidersMap`, I can see `MyLib` inside, and it looks no different than any other module from e.g. `ghc-prim` that I can import into `Test.hs` without any package qualification. Also, why does the error message refer to the file name `input/MyLib.hs`? Why does GHC even know that that is where it would have to be loaded from, if it weren't to be used from an already loaded package? > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

2 1

Re: Loading a typechecked module and then using it immediately as a package
by Erdi, Gergo 29 Jun '21

29 Jun '21

PUBLIC What's weird about it is that if I print the `moduleNameProvidersMap`, I can see `MyLib` inside, and it looks no different than any other module from e.g. `ghc-prim` that I can import into `Test.hs` without any package qualification. Also, why does the error message refer to the file name `input/MyLib.hs`? Why does GHC even know that that is where it would have to be loaded from, if it weren't to be used from an already loaded package? -----Original Message----- From: Erdi, Gergo Sent: Monday, June 28, 2021 3:04 PM To: 'Matthew Pickering' <matthewtpickering(a)gmail.com> Cc: 'ghc-devs(a)haskell.org' <ghc-devs(a)haskell.org> Subject: RE: [External] Re: Loading a typechecked module and then using it immediately as a package INTERNAL Oh, and one more question. This all works if in `Test.hs` I use a package import for `MyLib`. However, if I remove that, I get an error attempting to use module ‘main:MyLib’ (input/MyLib.hs) which is not loaded So I tried adding an ExposePackage to the DynFlags's packageFlags between loading the library and typechecking `Test.hs`, but I couldn't get that to work: 1. If I just set it in the `hsc_dflags`, it doesn't take. 2. If I use `initUnits` after setting it in `hsc_dflags`, I get a new error: cannot satisfy my-pkg -----Original Message----- From: Erdi, Gergo Sent: Monday, June 28, 2021 2:41 PM To: Matthew Pickering <matthewtpickering(a)gmail.com> Cc: ghc-devs(a)haskell.org Subject: RE: [External] Re: Loading a typechecked module and then using it immediately as a package INTERNAL Thanks, I got this working (after changing it slightly to use the GHC 9.0 API to the UnitState). However, I have some followup questions: 1. In the fakeUnitInfo, what is the relationship between unitLibraries and unitExposedModules? 2. In the fakeUnitInfo, what is the unitImportDirs used for? 3. What is the difference between a PackageId and a PackageName? Other than that, the code registers MyLib into the my-pkg package by letting `load` do the actual heavy lifting. For my purposes, I will need to control this loading myself: only `MyLib`'s typechecked interface should be used, with no actual compilation of definitions. So I wrote some code that makes a ModIface from a TcGblEnv. I seemingly got the whole thing working just by setting mi_exports from the TcGblEnv and leaving everything else with trivial values (fingerprint0 for fingerprints, empty lists for instances, etc.). If `MyLib` has no external dependencies and doesn't define any typeclasses, instances, or type families, is this enough? What is the role of `mi_globals`, should I set that from `tcg_rdr_env`, or should I leave it empty? Thanks, Gergo -----Original Message----- From: Matthew Pickering <matthewtpickering(a)gmail.com> Sent: Friday, June 25, 2021 6:54 PM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Cc: ghc-devs(a)haskell.org Subject: [External] Re: Loading a typechecked module and then using it immediately as a package ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. Hi Gergo, Please see a minimal example in this gist. https://clicktime.symantec.com/3Eb2qXqu8Yp6VtdK9d5pNmL7Vc?u=https%3A%2F%2Fg… Cheers, Matt On Fri, Jun 25, 2021 at 10:20 AM Erdi, Gergo via ghc-devs <ghc-devs(a)haskell.org> wrote: > > PUBLIC > > > Hi, > > > > I have the following to .hs files: > > > > MyLib.hs: > > module MyLib where > … > > Test.hs: > > {-# LANGUAGE PackageImports #-} > module Test where > import “my-pkg” MyLib > … > > > > I would like to parse/typecheck/load MyLib.hs into some Unit “my-unit”, then add that to the package “my-pkg”, and then typecheck Test.hs, all in-proc using the GHC API, without putting any other files on disk. How do I do that? > > > > What I tried is loading MyLib.hs after setting the homeUnitId in the DynFlags to “my-unit”, then editing the packageNameMap in the unitState of the DynFlags to may “my-pkg” to “my-unit”: > > setHomeUnit :: (GhcMonad m) => UnitId -> m () > > setHomeUnit unitId = do > > dflags <- getSessionDynFlags > > modifySession $ \h -> h{ hsc_dflags = dflags{ homeUnitId = unitId > } } > > > > registerUnit :: (GhcMonad m) => PackageName -> UnitId -> m () > > registerUnit pkg unitId = modifySession $ \h -> h{ hsc_dflags = > addUnit $ hsc_dflags h } > > where > > addUnit dflags = dflags > > { unitState = let us = unitState dflags in us > > { packageNameMap = M.insert pkg (Indefinite unitId > Nothing) $ packageNameMap us > > } > > } > > > > pipeline = do > > setHomeUnit myUnit > > loadModule =<< typecheckModule =<< parseModule =<< modSumarryFor “MyLib” > > registerUnit myPkg myUnit > > > > setHomeUnit mainUnitId > > typecheckModule =<< parseModule =<< modSumarryFor “Test” > > > > > > Alas, this doesn’t work: the import of `MyLib` from `my-pkg` fails with: > > > > input/linking/Test.hs:5:1: error: > > Could not find module ‘MyLib’ > > It is not a module in the current program, or in any known package. > > > > TBH I’m not very surprised that it didn’t work – that registerUnit function is doing some pretty deep surgery on the unitState that probably breaks several invariants. Still, I wasn’t able to find a better way – all the functions in GHC.Unit.State seem to be for querying only. > > > > Thanks, > > Gergo > > > This email and any attachments are confidential and may also be > privileged. If you are not the intended recipient, please delete all > copies and notify the sender immediately. You may wish to refer to the > incorporation details of Standard Chartered PLC, Standard Chartered > Bank and their subsidiaries at https: //www.sc.com/en/our-locations > > Where you have a Financial Markets relationship with Standard > Chartered PLC, Standard Chartered Bank and their subsidiaries (the > "Group"), information on the regulatory standards we adhere to and how > it may affect you can be found in our Regulatory Compliance Statement > at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at > http: //www.sc.com/rcs/fm > > Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. > > Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. > > Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. > > Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. > _______________________________________________ > ghc-devs mailing list > ghc-devs(a)haskell.org > https://clicktime.symantec.com/3A3FModesGCHWyu3Lvr5tp57Vc?u=http%3A%2F > %2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

1 0