- ghc-devs - Haskell.org

Re: Migration guide for multiple home units
by Erdi, Gergo 20 Jun '22

20 Jun '22

PUBLIC I managed to get this working, but I would like some feedback from Matt or others on whether this is the intended way of doing this. 1. The `extendMG` change can be brute-forced just to keep things going, by looking up dependency `ModSummary`s by `ModName` (we don't have full `Module`s at this point yet!): registerModSummary :: (GhcMonad m) => ModSummary -> m () registerModSummary ms = modifySession \env -> let mg = hsc_mod_graph env -- TODO: of course with a bit more housekeeping we can do better than this... edges = [ NodeKey_Module $ msKey ms' | dep <- deps, ms' <- mgModSummaries mg, ms_mod_name ms' == dep ] mg' = extendMG mg edges ms in env{ hsc_mod_graph = mg' } where deps = map (unLoc . snd) $ ms_textual_imps ms 2. `registerModule` can be kept mostly as-is, since it only uses `modifyUnitState` to change the active unit: registerModule :: (GhcMonad m) => ModDetails -> ModIface -> m () registerModule details iface = modifySession $ extendHpt . addModule where hmi = HomeModInfo iface details Nothing mod = mi_module iface modOrig = ModOrigin (Just True) [] [] True addModule = modifyUnitState $ \us -> us { moduleNameProvidersMap = M.insert (moduleName mod) (M.singleton mod modOrig) $ moduleNameProvidersMap us } extendHpt = hscUpdateHUG $ addHomeModInfoToHug hmi modifyUnitState :: (UnitState -> UnitState) -> HscEnv -> HscEnv modifyUnitState f env = env { hsc_unit_env = let ue = hsc_unit_env env in let units = ue_units ue units' = f units in ue_setUnits units' ue } 3. The tricky part is getting `addUnit` right. For this, based on the implementation of `initUnits`, I came up with the following: modifyUnitEnv :: (UnitEnv -> UnitEnv) -> HscEnv -> HscEnv modifyUnitEnv f env = env { hsc_unit_env = let ue = hsc_unit_env env in f ue } addUnit :: DynFlags -> UnitId -> HscEnv -> HscEnv addUnit dflags unitId = modifyUnitEnv $ \ue -> let dbs = ue_unit_dbs ue unit_state = ue_units ue home_unit = ue_homeUnit ue in flip updateHug ue $ unitEnv_insert unitId $ HomeUnitEnv { homeUnitEnv_units = unit_state , homeUnitEnv_unit_dbs = dbs , homeUnitEnv_dflags = dflags , homeUnitEnv_hpt = emptyHomePackageTable , homeUnitEnv_home_unit = home_unit } setCurrentUnit :: UnitId -> HscEnv -> HscEnv setCurrentUnit unitId = modifyUnitEnv $ ue_setActiveUnit unitId So my questions about this: 1. How does setting the home unit make sense? By doing this, I am effectively setting the home unit to `main` for all units, since that's the initial `ue_homeUnit` of the initial unit environment. Or does it not matter because after `addUnit`, I call `setCurrentUnit` anyway? I've found that I can't use the unit I am just adding as its own home unit, because that then leads to module name resolution problems in `main`: every imported module from `main` is searched for in `main` instead of its correct unit. 2. Speaking of `main`, why is it that when adding units, I have to skip `mainUnitId`, otherwise module resolution breaks again? 3. Unlike the previous version, I am no longer creating and putting `UnitInfo`s anywhere. Where is this going to bite me? Where (if anywhere) should I put `UnitInfo`s with the new setup? Thanks, Gergo -----Original Message----- From: ÉRDI Gergő <gergo(a)erdi.hu> Sent: Sunday, June 19, 2022 12:32 PM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Cc: GHC Devs <ghc-devs(a)haskell.org>; Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com> Subject: [External] Re: Migration guide for multiple home units On Thu, 16 Jun 2022, Erdi, Gergo via ghc-devs wrote: > Is there a migration guide for GHC API clients for the new “multiple home units” > feature? OK so in concrete terms, please see my attached program which is a heavily cut-down, standalone version of my real program. On commit fd42ab5fa1df847a6b595dfe4b63d9c7eecbf400^ (i.e. 3219610e3ba6cb6a5cd1f4e32e2b4befea5bd384) it compiles and works as expected. On commit fd42ab5fa1df847a6b595dfe4b63d9c7eecbf400 onwards, two problems pop up: 1. `extendMG` has changed and now requires manually specifying outgoing dependency edges. I thought the whole point of `summariseFile` was to collect this information? The reason I need to `extendMG` at that point is to get intra-unit orphan instances working. 2. `modifyUnitState` and its two uses (`addUnit` and `registerModule`) need to be updated to the new API. I think it makes sense that these need changing, since they touch exactly on the issue of which units are being compiled right now. However, I don't know how to update these. Also, I guess `setHomeUnit` should change the `CurrentUnit` instead of the `HomeUnit` now? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

2 2

Migration guide for multiple home units
by Erdi, Gergo 19 Jun '22

19 Jun '22

PUBLIC Hi, Is there a migration guide for GHC API clients for the new "multiple home units" feature? In particular, I have the following two functions in my code that used to interact with related features of GHC: addUnit :: UnitInfo -> HscEnv -> HscEnv addUnit unitInfo@GenericUnitInfo{..}<mailto:unitInfo@GenericUnitInfo%7b..%7d> = modifyUnitState $ \us -> us { packageNameMap = addToUFM (packageNameMap us) unitPackageName unitId , unitInfoMap = M.insert unitId unitInfo $ unitInfoMap us } registerModule :: (GhcMonad m) => ModDetails -> ModIface -> m () registerModule details iface = modifySession $ extendHpt . addModule where hmi = HomeModInfo iface details Nothing mod = mi_module iface modOrig = ModOrigin (Just True) [] [] True addModule = modifyUnitState $ \us -> us { moduleNameProvidersMap = M.insert (moduleName mod) (M.singleton mod modOrig) $ moduleNameProvidersMap us } extendHpt env = env { hsc_unit_env = let ue = hsc_unit_env env in ue { ue_hpt = addHomeModInfoToHpt hmi (hsc_HPT env) } } I implemented these using the following utility function: modifyUnitState :: (UnitState -> UnitState) -> HscEnv -> HscEnv modifyUnitState f env = env { hsc_unit_env = let ue = hsc_unit_env env in ue { ue_units = f (ue_units ue) } } With the recent changes to GHC, `modifyUnitState` doesn't work anymore because there is no single `UnitEnv` in the `HscEnv`. I tried updating the `HomeUnitEnv` of the current unit: modifyUnitState :: (UnitState -> UnitState) -> HscEnv -> HscEnv modifyUnitState f env = env { hsc_unit_env = let ue = hsc_unit_env env in ue_updateHomeUnitEnv f' (ue_currentUnit ue) ue } where f' hue = let units = homeUnitEnv_units hue in hue { homeUnitEnv_units = f units } but this leads to a panic: GHC version 9.5.20220613: Unit unknown to the internal unit environment unit (cortex-prim) pprInternalUnitMap main (flags: main, Just main) -> Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/GHC/Utils/Panic.hs:182:37 in ghc-lib-0.20220613-HQTH0nHhxeOCnJy5jYkZiX:GHC.Utils.Panic pprPanic, called at compiler/GHC/Unit/Env.hs:450:14 in ghc-lib-0.20220613-HQTH0nHhxeOCnJy5jYkZiX:GHC.Unit.Env ue_findHomeUnitEnv, called at compiler/GHC/Unit/Env.hs:394:48 in ghc-lib-0.20220613-HQTH0nHhxeOCnJy5jYkZiX:GHC.Unit.Env ue_unitFlags, called at compiler/GHC/Driver/Env.hs:421:18 in ghc-lib-0.20220613-HQTH0nHhxeOCnJy5jYkZiX:GHC.Driver.Env hscSetActiveUnitId, called at compiler/GHC/Driver/Env.hs:416:34 in ghc-lib-0.20220613-HQTH0nHhxeOCnJy5jYkZiX:GHC.Driver.Env hscSetActiveHomeUnit, called at compiler/GHC/Driver/Make.hs:1853:15 in ghc-lib-0.20220613-HQTH0nHhxeOCnJy5jYkZiX:GHC.Driver.Make What is the new way of registering a new unit or new module with GHC? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

3 2

Re: GHC code size
by Ben Gamari 14 Jun '22

14 Jun '22

Rodrigo Mesquita <r.mesquita(a)campus.fct.unl.pt> writes: > The visualisation is very cool. I’m left wondering: > > - What triggered this “explosion” in 2012? That is a good question. I'll admit, I'm not entirely sure as I myself only really started contributing to GHC in 2012. > - Do we have statistics on the number of contributors over time? > Yes, we usually report these during the "State of the compiler" talk that is given during the Haskell Implementor's Workshop. I use this script [1] to generate these. It's current output is: "1 year ago" 768 71 "2 year ago" 695 73 "3 year ago" 611 84 "4 year ago" 532 79 "5 year ago" 467 65 "6 year ago" 403 64 "7 year ago" 340 63 "8 year ago" 272 68 "9 year ago" 223 49 "10 year ago" 199 24 Where: * the first column is the beginning of the measured period * the second is the total number of unique contributors in GHC at the end of that period * the third is the number of *new* contributors during that period Cheers, - Ben [1] https://gitlab.haskell.org/bgamari/ghc-utils/-/blob/master/analytics/new-co…

1 0

Re: GHC code size
by Bryan Richter 14 Jun '22

14 Jun '22

A quick googling discovered https://githubnext.com/projects/repo-visualization, which has some of the desired features. :) (CC'ing the author and team mentioned in the article, too.) Has: * very visual * subdirectory breakdown * filetype breakdown Doesn't have: * Separating code from comments * History is listed under "future work" * Drop-in support for gitlab (it's presented as a GitHub Action) The article has an interactive widget you can point at a repo. I pointed it at ghc/ghc, and although my browser is still churning ten minutes later, here's a preview. It's pretty cool! On 14/06/2022 16:20, Hécate wrote: > > I'm taking the liberty of forwarding this to Bryan, as he's in a > unique position to help on this front. :) > > Le 14/06/2022 à 16:18, Simon Peyton Jones a écrit : >> Thanks Hecate. I used your figures in my talk. Really helpful. >> >> A note to all ghc-devs: it's be lovely to have a regularly-updated >> summary visualisation of GHC's source code: >> >> * Separating code from comments >> * Broken up by sub-directory >> * As visual as possible >> * Ideally with some kind of historical time-line ability >> >> This can't be new. Zillions of GitHub repositories could be >> visualised like this. There must be prior art; probably a lot of >> it. Can we just press a button and get it? >> >> Simon >> >> On Fri, 10 Jun 2022 at 17:45, Hécate <hecate(a)glitchbra.in> wrote: >> >> If you don't have a nix shell handy, here is what I'm getting: >> >> ❯ cloc compiler rts driver >> 1148 text files. >> 1137 unique files. >> 108 files ignored. >> >> github.com/AlDanial/cloc <http://github.com/AlDanial/cloc> v >> 1.88 T=1.31 s (794.3 files/s, 431269.4 lines/s) >> --------------------------------------------------------------------------------------- >> Language files blank >> comment code >> --------------------------------------------------------------------------------------- >> Haskell 635 68541 >> 140216 231567 >> C 158 10529 >> 16953 51162 >> C/C++ Header 209 4329 >> 8984 14536 >> yacc 2 971 >> 10 5024 >> Logos 3 530 >> 0 3642 >> Pascal 1 661 >> 936 2312 >> make 14 252 >> 409 850 >> Windows Module Definition 7 27 >> 0 489 >> Assembly 5 76 >> 269 478 >> Puppet 1 106 >> 0 445 >> Python 1 32 >> 19 162 >> D 1 16 >> 42 60 >> YAML 1 6 10 18 >> Lisp 1 2 4 7 >> Windows Resource File 1 0 0 1 >> --------------------------------------------------------------------------------------- >> SUM: 1040 86078 >> 167852 310753 >> --------------------------------------------------------------------------------------- >> >> Le 10/06/2022 à 18:29, chessai a écrit : >>> You might be able to do something with cloc and a shell script >>> for a rough estimate. >>> >>> ``` >>> $ cd ghc >>> $ nix-shell -p clock --run "cloc ." >>> ``` >>> >>> will output a detailed report of the loc and language breakdown >>> of the top level ghc directory (it is comment-aware and aware of >>> many languages). there might be a way to get cloc or a similar >>> tool to output something more inspect able (eg json), and then >>> use a shell script to gather everything from the appropriate >>> directories/files. >>> >>> I suspect something could be hacked up in less than a day, but >>> it would require a bit of research. Hopefully this is helpful >>> and gets you going - I'd be happy to hear of better solutions. >>> >>> Thanks >>> >>> On Fri, Jun 10, 2022, 11:20 Simon Peyton Jones >>> <simon.peytonjones(a)gmail.com> wrote: >>> >>> Dear GHC devs >>> >>> Is it possible to get a "lines-of-code" summary of GHC these >>> days? Like the one below, from 2011. >>> >>> It needs more than `wc` because it's helpful to split lines >>> of code from lines of comments and notes. >>> >>> We used to have `count_lines` but I'm not sure whether it is >>> still extant. >>> >>> I'm giving a talk at Zurihac on Sunday morning, about the >>> internals of GHC. Any data before then, preferably in a >>> form comparable to that below, would be terrific. >>> >>> But you have a lot else to do. This isn't do-or-die, just >>> nice to have. >>> >>> Thanks >>> >>> Simon >>> >>> image.png >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs(a)haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs(a)haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> -- >> Hécate ✨ >> 🐦: @TechnoEmpress >> IRC: Hecate >> WWW:https://glitchbra.in >> RUN: BSD >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -- > Hécate ✨ > 🐦: @TechnoEmpress > IRC: Hecate > WWW:https://glitchbra.in > RUN: BSD

6 5

Type vs Constraint
by Simon Peyton Jones 13 Jun '22

13 Jun '22

Dear GHC devs and Core Libraries folk The tension between Type and Constraint (are they equal or not?) has plaugued GHC for years. This GHC proposal #518 <https://github.com/ghc-proposals/ghc-proposals/pull/518> suggests how to fix it. Please do offer your thoughts. Simon

1 0

GHC code size
by Simon Peyton Jones 10 Jun '22

10 Jun '22

Dear GHC devs Is it possible to get a "lines-of-code" summary of GHC these days? Like the one below, from 2011. It needs more than `wc` because it's helpful to split lines of code from lines of comments and notes. We used to have `count_lines` but I'm not sure whether it is still extant. I'm giving a talk at Zurihac on Sunday morning, about the internals of GHC. Any data before then, preferably in a form comparable to that below, would be terrific. But you have a lot else to do. This isn't do-or-die, just nice to have. Thanks Simon [image: image.png]

3 2

GitLab bullets
by Simon Peyton Jones 10 Jun '22

10 Jun '22

Ben GitLab's rendering has become broken. See this ticket https://gitlab.haskell.org/ghc/ghc/-/issues/20686 - look under "Common ground...". The bullets all have a newline after them - Ditto the numbered points under "Generalisation" - But some bullets under "Generalisation" are OK. - Bullets have a handle that seems to let you drag the bullet around. But if you do so, it messa up the format completely! Ben, can you revert this Gitlab change? Simon

3 3

Haddock needs help
by Hécate 08 Jun '22

08 Jun '22

Hi everyone, Haddock needs help. Since I joined the Haddock team to help triage the tickets and interact with the community, we lost all of our experts, and I didn't have time to level up quickly enough to handle the mass of incoming tickets, let alone actually reduce the number of tickets to number below two hundred. As things stand now, the Haddock code base is in a disastrous state, largely not understood and its CI is in shambles. There are things that we can improve on the short and longer term – see https://github.com/haskell/haddock/issues/1465 – but the greater lack of expertise means that any project involving some core business logic is bound to be utterly and unnecessarily painful. The Hi Haddock GSOC proposal, whilst fully implemented in GHC, cannot be brought in Haddock at this moment in a reasonable timeline without any help. At present time, I need: * People who can refactor the code base, following modern software engineering practices, like domain-driven design and test-driven development. * UI developers, proficient in CSS and web accessibility. If you feel like you fit some of these criteria, please do contact me at this address. If your company can spare some engineering hours for you to give a hand, you're most welcome to do so. Just so we are clear, I am immensely grateful to the people who have submitted fixes and patches these past months, but this situation is untenable. Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD

3 3

GitLab upgrade starting shortly
by Ben Gamari 08 Jun '22

08 Jun '22

Hello all, I will be starting the GitLab upgrade shortly. Expect approximately 30 minutes of downtime. Cheers, - Ben

2 5

Two-factor authentication enforced on gitlab.haskell.org
by Ben Gamari 08 Jun '22

08 Jun '22

Hi all, Due to a recent up-tick in spam activity, we have started enforcing two-factor authentication on gitlab.haskell.org. We hope that this isn't too much of a burden, but do let us know if so and we can evaluate other options. Cheers, - Ben

1 1