- ghc-devs - Haskell.org

Question about specialization
by Harendra Kumar 07 Sep '21

07 Sep '21

Hi GHC devs, I have a simple program using the streamly library, as follows, the whole code is in the same module: {-# INLINE iterateState #-} {-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-} iterateState :: MonadState Int m => Int -> SerialT m Int iterateState n = do x <- get if x > n then do put (x - 1) iterateState n else return x main :: IO () main = do State.evalStateT (S.drain (iterateState 0)) 100000 Earlier the SPECIALIZE pragma was not required on iterateState, but after some refactoring in the library (the monad bind of SerialT changed a bit), this program now requires a SPECIALIZE on iterateState to trigger specialization, just INLINE also does not help. My question is whether this may be expected in some conditions or is this something which can be considered a bug in the compiler? I am also curious what specifically could have made the compiler not specialize this anymore, is it the size of the function or some other threshold? Thanks, Harendra

2 2

Potential improvement in compacting GC
by Ömer Sinan Ağacan 02 Sep '21

02 Sep '21

Hello, I recently implemented the algorithm used by GHC's compacting GC in another project. The algorithm (after marking) makes two passes over the heap /generation. In GHC, these passes are implemented in [1] and in the next function. In my implementation I tried 3 ways of implementing these passes, one of which is the same as GHC's code, and benchmarked each version. I found that the fastest implementation is not what's used in GHC, but it could easily be used. I should mention that my code targets Wasm, and I benchmarked Wasm instructions executed. Not CPU cycles, CPU instructions, or anything like that. It's very likely that the results will be different when benchmarking code running on actual hardware. Secondly, in my case the heap is mostly dead (residency is low). In GHC, compaction for the oldest generation is enabled when residency passes a threshold, so the assumption is the heap is mostly live. I'm guessing this should also make some difference. Anyway, the first implementation I tried was similar to GHC's scan, but I did scan += object_size(scan); instead of bumping scan by one, as GHC does in [2]. This was the slowest version. Second implementation did the same as GHC (bumped scan by one). This was faster, but still slower than the next version. What I found to be the best is scanning the bitmap, not the heap. The bitmap iterator reads one word at a time. In each iteration it checks if the bitmap word is 0. In GHC, in the best case this can skip 16 words on heap on 32-bit systems, and 32 words on 64-bit. Reminder: we need two bits per object in the bitmap, see [3]. (this is not the case in my implementation so the payoff is better) When the bitmap word is not 0 I use "count trailing zeros" (or "count leading zeros" depending on the bitmap implementation) to get the number of words to skip. This is a single instruction on Wasm and x86 (TZCNT or LZCNT, available via __builtin_ctzl and __builtin_clzl in gcc). So instead of skipping one word at a time, this can potentially skip 16 words (or 32 on 64-bit architectures). When that's not possible, it can still skip multiple words by using ctz/clz. Ömer [1]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt… [2]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt… [3]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rt…

5 6

Trying to create a regression test for #20261
by Alexis Praga 01 Sep '21

01 Sep '21

Hi, As a intermediate beginner in Haskell, I want to try to tackle a small issue for contributing to GHC. I've looked into https://gitlab.haskell.org/ghc/ghc/-/issues/20261, but it's not clear how to integrate a running time in a regression test. >From what I understand, regression tests should be compared against sucess/failure or a given output. Should there be a hard-coded limit ? If that's the case, how could I create such a test ? I could not find a similar test in the source code. Any help is appreciated, thanks ! -- Alexis Praga

2 2

GHC plugin authors?
by Richard Eisenberg 01 Sep '21

01 Sep '21

Hi all, I have seen a few posts from Sam Derbyshire here asking for feedback about plugin API design, and the responses have been minimal. This poses a design challenge, because the GHC folk who design the interface are sometimes distinct from the people who use the interface. We're trying to be good, seeking feedback from real, live clients. Is there a better way to do so than this mailing list? Example: we could create a Category on discourse.haskell.org, if that would reach the audience better. Or we could make a repo with issue trackers somewhere simply to track plugin design. What would work? (I recognize that I'm asking in a perhaps-ineffective channel for advice, but I really don't have a better idea right now. Maybe some of you plugin authors are here and will point us in a better direction.) Thanks, Richard

3 3

Inter-module inlining doesn't work, Id unfolding seems to be empty -- why?
by Erdi, Gergo 01 Sep '21

01 Sep '21

PUBLIC Hi, The attached program uses the GHC 9.0.1 API to typecheck and compile a single module that contains a single definition marked as INLINE: module Inline where data A = A1 | A2 data B = B1 | B2 {-# INLINE f #-} f :: A ->B f A1 = B1 f A2 = B2 In my bigger program, I noticed that usages of `f` in another module aren't inlined, so I made the attached experiment: I compile this to desugared, simplified, tidied and prep'd Core, and then use updateModDetailsIdInfos to fill in the unfolding of `f`. At least, that's what I thought I do. However, if I print the unfolding of `f` after this process, I get `OtherCon []` instead of what I expected to be a CoreUnfolding. What am I doing wrong? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

1 1

Re: Failed to build latest stable GHC on FreeBSD with Hadrian
by Jeffrey Young 28 Aug '21

28 Aug '21

Hi Alexis, This line shows the error: > | Configure package 'ghc-bin' > # cabal-configure (for _build/stage0/utils/haddock/setup-config) > # cabal-configure (for _build/stage0/ghc/setup-config) > hadrian: Encountered missing or private dependencies: > ghc ==9.3.* A package had a constraint on the ghc api to only accept ghc == 9.3.*. Since you are trying to build ghc 9.2 this fails. The package in question is printed by hadrian in this bit of your log: > src/Development/Shake/Internal/Rules/Oracle.hs:159:32 in > shake-0.19.4-9f51719be46f95213676da51d6a6626f69f0cfa15957aab44867ff78b757df38:Development.Shake.Internal.Rules.Oracle > * Depends on: OracleQ (ContextDataKey (Context {stage = Stage0, package = > Package {pkgType = Program, pkgName = "haddock", pkgPath = > "utils/haddock"}, way = v})) > at need, called at src/Hadrian/Oracles/Cabal/Rules.hs:54:9 in Notice the `pkgName` field in the `Package` record in the error that hadrian produced. It shows us that haddock was the cuplrit. Not sure why/how that cabal file got mixed up but I bet if you went to `${GHC_HOME}/utils/haddock/haddock.cabal` you'll find the constraint in question and should be able to switch it to `ghc==9.2.*` and rebuild according to the wiki. Hope this helps, Jeff On 8/27/21 10:08 AM, ghc-devs-request(a)haskell.org wrote: > Send ghc-devs mailing list submissions to > ghc-devs(a)haskell.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > or, via email, send a message with subject or body 'help' to > ghc-devs-request(a)haskell.org > > You can reach the person managing the list at > ghc-devs-owner(a)haskell.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of ghc-devs digest..." > > > Today's Topics: > > 1. Failed to build latest stable GHC on FreeBSD with Hadrian > (Alexis Praga) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 27 Aug 2021 19:15:26 +0200 > From: Alexis Praga <alexis.praga(a)gmail.com> > To: ghc-devs(a)haskell.org > Subject: Failed to build latest stable GHC on FreeBSD with Hadrian > Message-ID: > <CAH3o9gzh99J8SWPebkKsykEf+UD59S7-bSZi9OPQY1WxC_tE=w(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, > > As a complete beginner in regards to GHC, I tried to build GHC 9.2 as it > looked like the latest stable from git.I failed to build 9.0.1 before that. > > After checking out the ghc-9.2 branch, I ran (following the wiki): > >> ./boot >> set LOCALBASE=/usr/local >> ./configure --with-gmp-includes=$LOCALBASE/include > --with-gmp-libraries=$LOCALBASE/lib --disable-large-address-space >> hadrian/build -j > This failed with : > >> | Configure package 'rts' >> | Configure package 'haddock' >> | Configure package 'ghc-bin' >> # cabal-configure (for _build/stage0/utils/haddock/setup-config) >> # cabal-configure (for _build/stage0/ghc/setup-config) >> hadrian: Encountered missing or private dependencies: >> ghc ==9.3.* >> >> Error when running Shake build system: >> at action, called at src/Rules.hs:40:19 in main:Rules >> at need, called at src/Rules.hs:62:5 in main:Rules >> * Depends on: _build/stage0/bin/haddock >> at apply1, called at > src/Development/Shake/Internal/Rules/Oracle.hs:159:32 in > shake-0.19.4-9f51719be46f95213676da51d6a6626f69f0cfa15957aab44867ff78b757df38:Development.Shake.Internal.Rules.Oracle >> * Depends on: OracleQ (ContextDataKey (Context {stage = Stage0, package = > Package {pkgType = Program, pkgName = "haddock", pkgPath = > "utils/haddock"}, way = v})) >> at need, called at src/Hadrian/Oracles/Cabal/Rules.hs:54:9 in > main:Hadrian.Oracles.Cabal.Rules >> * Depends on: _build/stage0/utils/haddock/setup-config >> * Raised the exception: >> ExitFailure 1 > The config.log is attached. > > Any tips on how to proceed would be appreciated, thanks ! >

3 2

Failed to build latest stable GHC on FreeBSD with Hadrian
by Alexis Praga 27 Aug '21

27 Aug '21

Hi, As a complete beginner in regards to GHC, I tried to build GHC 9.2 as it looked like the latest stable from git.I failed to build 9.0.1 before that. After checking out the ghc-9.2 branch, I ran (following the wiki): > ./boot > set LOCALBASE=/usr/local > ./configure --with-gmp-includes=$LOCALBASE/include --with-gmp-libraries=$LOCALBASE/lib --disable-large-address-space > hadrian/build -j This failed with : > | Configure package 'rts' > | Configure package 'haddock' > | Configure package 'ghc-bin' > # cabal-configure (for _build/stage0/utils/haddock/setup-config) > # cabal-configure (for _build/stage0/ghc/setup-config) > hadrian: Encountered missing or private dependencies: > ghc ==9.3.* > > Error when running Shake build system: > at action, called at src/Rules.hs:40:19 in main:Rules > at need, called at src/Rules.hs:62:5 in main:Rules > * Depends on: _build/stage0/bin/haddock > at apply1, called at src/Development/Shake/Internal/Rules/Oracle.hs:159:32 in shake-0.19.4-9f51719be46f95213676da51d6a6626f69f0cfa15957aab44867ff78b757df38:Development.Shake.Internal.Rules.Oracle > * Depends on: OracleQ (ContextDataKey (Context {stage = Stage0, package = Package {pkgType = Program, pkgName = "haddock", pkgPath = "utils/haddock"}, way = v})) > at need, called at src/Hadrian/Oracles/Cabal/Rules.hs:54:9 in main:Hadrian.Oracles.Cabal.Rules > * Depends on: _build/stage0/utils/haddock/setup-config > * Raised the exception: > ExitFailure 1 The config.log is attached. Any tips on how to proceed would be appreciated, thanks ! -- Alexis Praga

2 1

Registering orphan instances and filling the `deps` field of ModIface [RE: Where (else) do I need to register instances from loaded modules?]
by Erdi, Gergo 27 Aug '21

27 Aug '21

PUBLIC Re: #1., I've simplified the code by removing all the Unit shuffling: everything is now in mainUnit. Unfortunately, this doesn't change anything other than cutting a cool 100 lines from the MWE code; I'm attaching the new smaller & simpler version. It is based on GHC 9.0.1 with an extra patch to export the first half of mkIfaceTc, just before calling mkFullIface. With this change, it really should be filling everything in ModIface the same way as the real GHC (so e.g. `deps` should be properly filled), modulo the fingerprints. Unfortunately, the `deps` field of the `ModIface`s created this way still doesn't contain any orphan instance modules: `Class.src`'s deps are empty (which is what I'd expect), `Instance.src` has a dependency in dep_mods (but not in dep_orphs) on `Class {-# SORUCE #-}` (I'm not sure why it's marked as a source import when `Class` module was already loaded into the module providers map before typechecking `Instance`...), and of course `Use.src` still fails so I can't get to its dependencies. It is confusing to me that the problem should be because of the dep_orphs, because `Instance.src` does NOT use any orphan instances from `Class.src; merely, it provides a new orphan instance for other modules to depend on. Can someone explain this part to me? Thanks, Gergo -----Original Message----- From: Erdi, Gergo Sent: Wednesday, August 25, 2021 10:37 AM To: Matthew Pickering <matthewtpickering(a)gmail.com> Cc: GHC <ghc-devs(a)haskell.org>; gergo(a)erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com> Subject: Re: Where (else) do I need to register instances from loaded modules? Hi, Thanks Matt! Unfortunately, I couldn't get it working even with these suggestions. 1. I changed registerModule so that it only changes the HPT when toUnitId (moduleUnit mod) == homeUnitId (hsc_dflags env). But I'm pretty sure this will be vacuously true, since the unit of the module comes from prepareModule setting the home unit before the call to `downsweep`. TBH I don't understand the whole idea behind the home package, and maybe all my pain comes from misusing packages in the first place. As you can see in the original code I've sent, I'm loading modules from different units; in my real use case, I'd even like to do things like load module1 from unit1, then module2 from unit2, then module3 from unit1 again -- basically just using the units as a cheap way to classify modules so that later downstream, I can make decisions based on the unit of the module of the source of a given definition. Should I give up on this idea, is this going to lead to problems later on? 2. I also changed mkModIface to fill in the `deps` and while I'm at it, the `usages` field, by copy-pasting more code from mkIfaceTc. I'm really starting to think that I'd be better off just changing mkIfaceTc to return a PartialModIface... Anyway, the new definitions of `deps` and `usages` are: let pluginModules = map lpModule (cachedPlugins (hsc_dflags hsc_env)) unit_id = homeUnitId (hsc_dflags hsc_env) deps <- mkDependencies unit_id (map mi_module pluginModules) tcg dep_files <- readIORef dependent_files let used_names = mkUsedNames tcg usages <- mkUsageInfo hsc_env this_mod (imp_mods imports) used_names dep_files merged pluginModules With these changes applied, I am still seeing the same error. Printing shows that `dep_orphs deps` is empty for Instance.src; further tracing, in turn, shows that `imp_orphs . tcg_imports $ tcg` is also empty! I have also found the wording in the documentation regarding dep_orphs confusing: it sounds like, in my case, this would mean that Use has a dependency on Instance, because it is using an orphan instance from there. But my problems are way before I am making a ModIface for Use, since it fails already during typechecking. But you saying the problem is the incomplete deps in ModIface must mean that dep_orphs needs to be filled in the ModIface for Instance (or, oh god I hope not, Class), right? If I sound somewhat confused, rest assured that in reality I am even MORE confused! Any further advice welcome, Gergo -----Original Message----- From: Matthew Pickering <matthewtpickering(a)gmail.com> Sent: Tuesday, August 24, 2021 5:16 PM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Cc: GHC <ghc-devs(a)haskell.org>; gergo(a)erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com> Subject: [External] Re: Where (else) do I need to register instances from loaded modules? Hi Gergo, Some things I noticed: * The HPT only contains modules from the current home unit. It seems that registerModule ignores this so you break this invariant. * It seems that `Instance` contains an orphan, but you don't fill in the `deps` field when making a `ModIface`, which should contain all the orphan modules below the module you are compiling. I think that if you fill in `deps` correctly then things may work out. `Dependencies` eventually becomes `ImportAvail`, which gets consulted by `tcVisibleOrphanMods` which is called in `tcGetInstEnvs` which is called by `matchInstEnv`. Matt This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

1 0

[Haskell] [ANNOUNCE] GHC 8.10.7 released
by Zubin Duggal 26 Aug '21

26 Aug '21

The GHC team is very pleased to announce the availability of GHC 8.10.7. Source and binary distributions are available at the [usual place](https://downloads.haskell.org/ghc/8.10.7/). Download Page: https://www.haskell.org/ghc/download_ghc_8_10_7.html Blog Post: https://www.haskell.org/ghc/blog/20210827-ghc-8.10.7-released.html This is a small bugfix release, fixing one linking portability issue (#19950) present in GHC 8.10.5 and GHC 8.10.6 on some x86_64 macOS toolchains, which resulted in undefined symbol errors for `___darwin_check_fd_set_overflow`. Issue #19950 is caused by a bug in newer Apple toolchains (specifically XCode 12) where programs compiled with affected versions of XCode are not backwards compatible with configurations running older version of XCode (certain versions of XCode 11). We claimed to have fixed this in GHC 8.10.6, but alas this wasn't the case. The fix was originally tested on the master branch, which uses a different build configuration from the 8.10 branch. We have now tested the fix on the GHC 8.10 branch and finally squashed the bug. We would like to thank Microsoft Research, GitHub, IOHK, the Zw3rk stake pool, Tweag I/O, Serokell, Equinix, SimSpace, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. We would also like to thank the hundreds of open-source contributors whose work makes GHC possible. A complete list of bug fixes and improvements can be found in the [release notes](https://downloads.haskell.org/ghc/8.10.7/docs/html/users_guide/8. 10.7-notes.html). As always, feel free to report any issues you encounter via [gitlab.haskell.org](https://gitlab.haskell.org/ghc/ghc/-/issues/new).

1 0

primitive (byte) string literal with length?
by Viktor Dukhovni 25 Aug '21

25 Aug '21

Is there any GHC syntax for constructing a primitive string literal with a known (not hand coded) byte count? With `"some bytes"#` I get just the `Addr#` pointer, but not the size. If there's nothing available, would it be reasonable to introduce a new syntax? Perhaps: "some bytes"## :: (# Addr#, Int# #) -- Viktor.

3 5