- ghc-devs - Haskell.org

Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?)
by Erdi, Gergo 30 Dec '21

30 Dec '21

PUBLIC Ah, at least I've figured out why exactly the simplifier does this (regardless of floating out due to full laziness or not): it is because the definition of the default implementation of (>>) is inlined. I have (>>) defined as such: ma >> mb = ma >>= \_ -> mb So when the simplifier encounters a call site "foo >> bar", it needs to bind "foo" and "bar" to avoid duplication (since "ma" and "mb" could occur multiple times on the right-hand side of (>>)'s definition). This means that the best I can hope for would be to avoid floating out "ma" and "mb", but the variable bindings themselves are unavoidable in the face of inlining (and of course I do want inlining). So I'll need to rethink my approach anyway. -----Original Message----- From: Erdi, Gergo Sent: Thursday, December 30, 2021 12:56 PM To: Matthew Pickering <matthewtpickering(a)gmail.com> Cc: Joachim Breitner <mail(a)joachim-breitner.de>; ghc-devs(a)haskell.org Subject: RE: [External] Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) PUBLIC Turning on various Opt_D_dump flags, I can see that this tranformation happens in the first iteration of the simplifier. Looking at GHC.Core.Opt.getCoreToDo, I would guess this corresponds to this line: -- initial simplify: mk specialiser happy: minimum effort please runWhen do_presimplify simpl_gently, If I wrote my own optimisation pass, I'm not sure what it could do to inhibit GHC from doing this transformation later on in its pipeline. And I don't want to give up on the simplifier pipeline completely, since there are parts of it that I very much need. One of them is, unfortunately, the specializer itself -- and that comment in getCoreToDo makes me worried that if I somehow managed to get rid of this transformation (or if I skipped the simplifier pipeline completely and just ran the specializer out-of-band), I would break specialization anyway... -----Original Message----- From: Matthew Pickering <matthewtpickering(a)gmail.com> Sent: Thursday, December 30, 2021 10:42 AM To: Erdi, Gergo <Gergo.Erdi(a)sc.com> Cc: Joachim Breitner <mail(a)joachim-breitner.de>; ghc-devs(a)haskell.org Subject: [External] Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) Hi Gergo, Sounds like you might be better off writing your own optimisation pass rather than relying on making GHC do what you want. Cheers Matt On Thu, Dec 30, 2021 at 9:05 AM Erdi, Gergo via ghc-devs <ghc-devs(a)haskell.org> wrote: > > PUBLIC > > Hi Joachim, > > Thanks for the hints! > > > Hi Gergo, > > > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via > > ghc- > > devs: > > > PUBLIC > > > > phew > > Yeah obviously I'm sitting here not only adding these tags, but also > coming up with the automated systems and also the company policies > forcing the usage of said systems ;) > > > didn't investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > Unfortunately, it's hard to make a small example because this all > lives in bizarro world where "IO" isn't IO, "String" isn't [Char] etc, > so I can't just take the input program and pass it to vanilla GHC. > > > > > * It is generally preferable to turn local lambda expressions > > into top-level functions. This way, instead of dynamically > > allocating a FUN heap object, it's just a static function. > > > > * sat_sKv is an IO expression? Then it is actually a function in a way > > (taking the "State token" as an argument). So the above applies. > > This is "IO" but not GHC's IO, just another type with an opaque > definition and a Monad instance. There's no reason for GHC to think > that sat_sKv would be a function. > > Of course, sat_sKw is necessarily a function since it is the second > argument to bind. But only now I am noticing that even sat_sKw's > definition includes *yet another* floated-out variable: > > sat_sKv = some complicated expression 1 sat_sKu = some complicated > expression 2 sat_sKw = \_ -> sat_sKu main = bindIO sat_sKv sat_sKw > > Here, I don't see why GHC should think that sat_sKv and sat_sKu are > functions. For example, here's the definition of sat_sKu: > > sat_sKu :: IO () > [LclId] > sat_sKu > = let { > sat_sRy [Occ=Once1] :: String > [LclId] > sat_sRy > = let { > sat_sRx [Occ=Once1] :: String > [LclId] > sat_sRx = unpackCStringUtf8# "\n"# } in > let { > sat_sRw [Occ=Once1] :: String > [LclId] > sat_sRw > = case foobar True of { > False -> unpackCStringUtf8# "False"#; > True -> unpackCStringUtf8# "True"# > } } in > sApp# sat_sRw sat_sRx } in > putStrOut sat_sRy > > Here, putStrOut is so opaque that it doesn't even have a definition, > it is merely a name registered into GHC's name tables. So I really > don't see why this looks "function-y" to GHC. > > The reason this is all a problem for me is because I would like to > then interpret all these let-bindings, including the toplevel ones, as > eagerly defined variables. But then there is of course a difference > between > > main = > let a1 = someIOAction1 > a2 = someOtherIOAction2 > a2' = \_ -> a2 > in bindIO a1 a2' > > and > > main = > let a1 = someIOAction1 > a2 = \_ -> someIOAction2 > in bindIO a1 a2 > > because if the *definition* of "a2" throws (not its action), then the > first version will throw immediately whereas the second will only > throw when "main" is *run*, after running "a1". > > > * I think this is the FloatOut pass. You can turn it out using > > -fno-full-laziness. Not sure if some others passes might > > do similar things, though. > > I tried turning off Opt_FullLaziness, but unfortunately I am still > getting the same result. I was also hopeful when I saw there's an > Opt_FloatIn flag, but alas that doesn't change this behaviour either > (and it's turned on by default anyway...) > > I'll try to make a self-contained minimal example next week. > > Thanks, > Gergo > > -----Original Message----- > From: ghc-devs <ghc-devs-bounces(a)haskell.org> On Behalf Of Joachim > Breitner > Sent: Wednesday, December 29, 2021 8:39 PM > To: ghc-devs(a)haskell.org > Subject: [External] Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? > > > Hi Gergo, > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via > ghc- > devs: > > PUBLIC > > phew > > > I’m seeing ‘do’ blocks getting taking apart into top-level > > definitions, so e.g. > > > > main = do > > some complicated expression 1 > > some complicated expression 2 > > > > is compiled into > > > > sat_sKv = some complicated expression 1 sat_sKw = \_ -> some > > complicated expression 2 main = bindIO sat_sKv sat_sKw > > > > This seems to happen regardless of any common subexpressions, i.e. > > it is not the case that sat_sKv or sat_sKw are used anywhere else. > > > > What is the intended benefit of this floating-out? Is there a > > particular Core-to-Core pass that causes this? Is it possible to > > turn it off? > > > didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > * It is generally preferable to turn local lambda expressions > into top-level functions. This way, instead of dynamically > allocating a FUN heap object, it’s just a static function. > > * sat_sKv is an IO expression? Then it is actually a function in a way > (taking the “State token” as an argument). So the above applies. > > * I think this is the FloatOut pass. You can turn it out using > -fno-full-laziness. Not sure if some others passes might > do similar things, though. > > Cheers, > Joachim > > > -- > Joachim Breitner > mail(a)joachim-breitner.de > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

1 0

Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?)
by Erdi, Gergo 30 Dec '21

30 Dec '21

PUBLIC Hi Joachim, Thanks for the hints! > Hi Gergo, > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- > devs: > > PUBLIC > > phew Yeah obviously I'm sitting here not only adding these tags, but also coming up with the automated systems and also the company policies forcing the usage of said systems ;) > didn't investigate deeper (maybe if you provide a small example I would), but just from looking at this: Unfortunately, it's hard to make a small example because this all lives in bizarro world where "IO" isn't IO, "String" isn't [Char] etc, so I can't just take the input program and pass it to vanilla GHC. > > * It is generally preferable to turn local lambda expressions > into top-level functions. This way, instead of dynamically > allocating a FUN heap object, it's just a static function. > > * sat_sKv is an IO expression? Then it is actually a function in a way > (taking the "State token" as an argument). So the above applies. This is "IO" but not GHC's IO, just another type with an opaque definition and a Monad instance. There's no reason for GHC to think that sat_sKv would be a function. Of course, sat_sKw is necessarily a function since it is the second argument to bind. But only now I am noticing that even sat_sKw's definition includes *yet another* floated-out variable: sat_sKv = some complicated expression 1 sat_sKu = some complicated expression 2 sat_sKw = \_ -> sat_sKu main = bindIO sat_sKv sat_sKw Here, I don't see why GHC should think that sat_sKv and sat_sKu are functions. For example, here's the definition of sat_sKu: sat_sKu :: IO () [LclId] sat_sKu = let { sat_sRy [Occ=Once1] :: String [LclId] sat_sRy = let { sat_sRx [Occ=Once1] :: String [LclId] sat_sRx = unpackCStringUtf8# "\n"# } in let { sat_sRw [Occ=Once1] :: String [LclId] sat_sRw = case foobar True of { False -> unpackCStringUtf8# "False"#; True -> unpackCStringUtf8# "True"# } } in sApp# sat_sRw sat_sRx } in putStrOut sat_sRy Here, putStrOut is so opaque that it doesn't even have a definition, it is merely a name registered into GHC's name tables. So I really don't see why this looks "function-y" to GHC. The reason this is all a problem for me is because I would like to then interpret all these let-bindings, including the toplevel ones, as eagerly defined variables. But then there is of course a difference between main = let a1 = someIOAction1 a2 = someOtherIOAction2 a2' = \_ -> a2 in bindIO a1 a2' and main = let a1 = someIOAction1 a2 = \_ -> someIOAction2 in bindIO a1 a2 because if the *definition* of "a2" throws (not its action), then the first version will throw immediately whereas the second will only throw when "main" is *run*, after running "a1". > * I think this is the FloatOut pass. You can turn it out using > -fno-full-laziness. Not sure if some others passes might > do similar things, though. I tried turning off Opt_FullLaziness, but unfortunately I am still getting the same result. I was also hopeful when I saw there's an Opt_FloatIn flag, but alas that doesn't change this behaviour either (and it's turned on by default anyway...) I'll try to make a self-contained minimal example next week. Thanks, Gergo -----Original Message----- From: ghc-devs <ghc-devs-bounces(a)haskell.org> On Behalf Of Joachim Breitner Sent: Wednesday, December 29, 2021 8:39 PM To: ghc-devs(a)haskell.org Subject: [External] Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? [You don't often get email from mail(a)joachim-breitner.de. Learn why this is important at http://aka.ms/LearnAboutSenderIdentification.] ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. Hi Gergo, Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- devs: > PUBLIC phew > I’m seeing ‘do’ blocks getting taking apart into top-level > definitions, so e.g. > > main = do > some complicated expression 1 > some complicated expression 2 > > is compiled into > > sat_sKv = some complicated expression 1 sat_sKw = \_ -> some > complicated expression 2 main = bindIO sat_sKv sat_sKw > > This seems to happen regardless of any common subexpressions, i.e. it > is not the case that sat_sKv or sat_sKw are used anywhere else. > > What is the intended benefit of this floating-out? Is there a > particular Core-to-Core pass that causes this? Is it possible to turn > it off? didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: * It is generally preferable to turn local lambda expressions into top-level functions. This way, instead of dynamically allocating a FUN heap object, it’s just a static function. * sat_sKv is an IO expression? Then it is actually a function in a way (taking the “State token” as an argument). So the above applies. * I think this is the FloatOut pass. You can turn it out using -fno-full-laziness. Not sure if some others passes might do similar things, though. Cheers, Joachim -- Joachim Breitner mail(a)joachim-breitner.de https://clicktime.symantec.com/33oTTj3Bye1kkqsmxZWi3QZ6xU?u=http%3A%2F%2Fww… _______________________________________________ ghc-devs mailing list ghc-devs(a)haskell.org https://clicktime.symantec.com/3bo4KFUFPcVSdMe4B1CUzr6xU?u=http%3A%2F%2Fmai… This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

2 2

What's the benefit of taking "do" blocks apart? Is there a way to turn that off?
by Erdi, Gergo 29 Dec '21

29 Dec '21

PUBLIC Hi, I'm seeing 'do' blocks getting taking apart into top-level definitions, so e.g. main = do some complicated expression 1 some complicated expression 2 is compiled into sat_sKv = some complicated expression 1 sat_sKw = \_ -> some complicated expression 2 main = bindIO sat_sKv sat_sKw This seems to happen regardless of any common subexpressions, i.e. it is not the case that sat_sKv or sat_sKw are used anywhere else. What is the intended benefit of this floating-out? Is there a particular Core-to-Core pass that causes this? Is it possible to turn it off? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

2 1

Output language of typechecking pass?
by Benjamin Redelings 28 Dec '21

28 Dec '21

Hi, I have been looking for info on what actually comes out of the type-checking pass in GHC. This is mostly because it seems like the "Type classes in Haskell" paper implements both type checking and translation to dictionary-passing in one pass, whereas it seems like GHC separates this into (i) type checking and (ii) desugaring. Questions: 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? 2. Does the output of type checking contain type lambdas? 3. Does the type checking pass determine where to add dictionary arguments? 4. Are there any other resources I should be looking at? I am confused about #3, because the `-ddump-tc` output doesn't seem to show any type dictionaries in function bodies themselves, but it does seem to contain some kind of info about dictionaries as "evidence" -- but I am not sure what "evidence" is, or how it links into the AST for a function body. I did briefly look at `-ddump-tc-ast`, but not in detail yet. -BenRI

4 7

Measuring thread cpu time (using bound threads)
by Harendra Kumar 28 Dec '21

28 Dec '21

Hi GHC devs, I want to measure the CPU time spent in a particular Haskell thread across thread yield points. I am assuming Haskell threads do not keep track of per thread cpu time. Therefore, to measure it I have to use the thread cpu clock (CLOCK_THREAD_CPUTIME_ID) provided by the OS (Linux). But Haskell threads can keep jumping across multiple OS threads so the OS thread's CPU time cannot be used reliably. To solve that issue I thought I could use bound threads. From the documentation it sounded like bound threads are exclusively bound to a particular OS thread so we can get thread cpu time from the OS and use it. However, if I print the thread id for a bound thread using "gettid" (OS thread id on Linux) or using "pthread_self" (pthread id), both these thread ids are changing for the same Haskell thread which is a bound thread. Is that expected, or am I doing something wrong? My questions: * Are bound threads really bound to a particular OS thread or they can use multiple OS threads for executing Haskell code, and only use a fixed OS thread for FFI code? * Is there a way to make this work? Can we obtain the thread cpu time reliably across thread yield points? Is there a way to really bind Haskell threads to OS threads? * Is there a better or alternative way to get the thread CPU time? Thanks, Harendra

1 0

Source locations from Core
by Erdi, Gergo 28 Dec '21

28 Dec '21

PUBLIC Hi, I'm looking for ways to map Core fragments back to source locations. I see there is an annotated version of Core in `GHC/Core.hs` called `AnnExpr`, which I could see being useful for this if I set the annotation type to `SrcSpan`, but that's not what I get out of GHC's desugarer, simplifier or tidier. If there's no built-in mechanism for this, my only idea would be to create a HsExpr-to-HsExpr transformation that wraps every node in a call that is opaque enough to be persisted through Core-to-Core transformations but still transparent enough that it doesn't block optimization opportunities. Is that even possible? Alternatively, would it make it easer if I was content with only getting source locations for variable occurrences? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.

2 2

CI: Choice of base commit for perf comparisons
by Joachim Breitner 22 Dec '21

22 Dec '21

Hi, the new (or “new”?) handling of perf numbers, where CI just magically records and compares them, without us having to manually edit the `all.T` files, is a big improvement, thanks! However, I found the choice of the base commit to compare against unhelpful. Assume master is at commit M, and I start a feature branch and MR with commit A. CI runs, and tells me about a performance regressions, and CI is red. I now fix the issue and push commit B to the branch. CI runs, but it picks A to compare against, and now it is red because of an seemingly unexpected performance improvement! I would have expected that all CI runs for this MR to compare the performance against the base branch on master, and to look for perf change notices in all commit messages in between. I see these advantages: * The reported perf changes correspond to the changes shown on the MR page * Green CI = the MR is ready (after squashing) * CI will have numbers for the base commit more reliably (else, if I push commit C quickly after B, then the job for B might be cancelled and Ci will report changes of C against A instead of B, which is unexpected). I have used this logic of reporting perf changes (or any other “differential CI”) against the base branch in the Motoko project and it was quite natural. Would it be desirable and possible for us here, too? (A possible rebuttal might be: we don’t push new commits to feature branches, but always squash and rebase, as that’s what we have to do before merging anyways. If that’s the case then ok, although I generally lean to having chronological commits on feature branches and a nice squashed commit on master.) Cheers, Joachim -- Joachim Breitner mail(a)joachim-breitner.de http://www.joachim-breitner.de/

2 2

Merge request merge timing
by Ben Gamari 21 Dec '21

21 Dec '21

Hello all, Recently there have been a few instances where merge requests were merged extremely quickly (e.g. within an hour of the MR being opened). While prompt review is a good thing, we also want to make sure that all potentially interested reviewers have an opportunity to comment. In this vein, I propose that we wait at least 24 hours from the time an MR is opened before sending it to Marge. I have documented this to the merge-request review guidelines on the Wiki [1]. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/gitlab/merge-requests

1 0

[PSA] Patches to Haddock regarding GHC development move to GitLab!
by Hécate 17 Dec '21

17 Dec '21

Hi everyone, just a PSA from the Haddock team and GHC release managers to warn everyone that, effective immediately, patches to Haddock needed for GHC merge requests are not to be directed to GitHub anymore. Those patches are to be sent to the GitLab mirror: https://gitlab.haskell.org/ghc/haddock. I will try and warn folks who currently have PRs opened on GitHub targeting ghc-head, but please know that they'll be closed in the next few days. Have a great end of the week, Hécate, for the Haddock Team. -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD

2 1

Transparently hooking into the STG stack to validate an escape analysis
by Sebastian Graf 17 Dec '21

17 Dec '21

Hi Devs, my master's student Sebastian and I (also Sebastian :)) are working on an escape analysis in STG, see https://gitlab.haskell.org/ghc/ghc/-/issues/16891#note_347903. We have a prototype for the escape analysis that we want to validate/exploit now. The original plan was to write the transformation that allocates non-escaping objects on the STG stack. But that is quite tricky for many reasons, one of them being treatment of the GC. This mail is rather lengthy, so I suggest you skip to "What we hope could work" and see if you can answer it without the context I provide below. If you can't, I would be very grateful if you were willing to suffer through the exposition. # Instrumentation So instead we thought about doing a (easily changed and thus versatile) instrumentation-based approach: Assign a sequence number to every instantiation (a term I will use to mean "allocation of g's closure") of things that we our analysis determines as escaping or non-escaping, such as STG's let bindings (focusing on non-let-no-escape functions for now). (One sequence number *per allocation* of the let binding's closure, not based on its syntactic identity.) Then, we set a bit in a (dynamically growing) global bit vector whenever the let "RHS is entered" and then unset it when we "leave the let-body". Example: f = \x y -> let { g = [y] \z -> y + z; } in g x Here, our analysis could see that no instantiation (which I say instead of "allocation of g's closure") of g will ever escape its scope within f. Our validation would give a fresh sequence number to the instantiation of g whenever f is called and store it in g's closure (which we arrange by piggy-backing on -prof and adding an additional field to the profiling header). Then, when g's RHS is entered, we set the bit in the global bit vector, indicating "this instantiation of g might escape". After leaving the RHS of g, we also leave the body of the defining let, which means we unset the bit in the bit vector, meaning "every use so far wasn't in an escaping scenario". So far so good. Modifying the code upon entering g takes a bit of tinkering but can be done by building on TickyTicky in StgToCmm. But what is not done so easily is inserting the continuation upon entering the let that will unset the bit! # What doesn't work: Modifying the Sequel At first, we tried to modify the sequel <https://gitlab.haskell.org/ghc/ghc/-/blob/4c6985cc91b9cf89b393f69c9a603e8df…> of the let-body to an `AssignTo`. That requires us to know the registers in which the let-body will return its results, which in turn means we have to know the representation of those results, so we have to write a function `stgExprPrimRep :: GenStgExpr p -> [PrimRep]`. Urgh! We were very surprised that there was no such function. And while we tested our function, we very soon knew why. Consider the following pattern synonym matcher: GHC.Natural.$mNatJ# :: forall {rep :: GHC.Types.RuntimeRep} {r :: TYPE rep}. GHC.Num.Natural.Natural -> (GHC.Num.BigNat.BigNat -> r) -> ((# #) -> r) -> r = {} \r [scrut_sBE cont_sBF fail_sBG] case scrut_sBE of { GHC.Num.Natural.NS _ -> fail_sBG GHC.Prim.(##); GHC.Num.Natural.NB ds_sBJ -> let { sat_sBK :: GHC.Num.BigNat.BigNat = CCCS GHC.Num.BigNat.BN#! [ds_sBJ]; } in cont_sBF sat_sBK; }; Note how its result is representation-polymorphic! It only works because our machine implementation allows tail-calls. It's obvious in hindsight that we could never write `stgExprPrimRep` in such a way that it will work on the expression `cont_sBF sat_sBK`. So the sequel approach seems infeasible. # What we hope could work: A special stack frame The other alternative would be to insert a special continuation frame on the stack when we enter the let-body (inspired by stg_restore_cccs). This continuation frame would simply push all registers (FP regs, GP regs, Vec regs, ...) to the C stack, do its work (unsetting the bit), then pop all registers again and jump to the topmost continuation on the STG stack. Example: f :: forall rep (r :: TYPE rep). Int# -> (Int# -> r) -> r f = \x g -> let { h = [x] \a -> x + a; } in case h x of b { __DEFAULT -> g b } We are only interested in unsetting the bit for h here. Consider the stack when entering the body of h. caller_of_f_cont_info <- Sp Now push our special continuation frame: caller_of_f_cont_info seq_h unset_bit_stk_info <- Sp E.g., the stack frame contains the info pointer and the sequence number. (Btw., I hope I got the stack layout about right and this is even possible) Then, after we entered the continuation of the __DEFAULT alt, we do a jump to g. Plot twist: g returns an unboxed 8-tuple of `Int#`s (as caller_of_f_cont_info knows, but f certainly doesn't!), so before it returns it will push two args on the stack (correct?): caller_of_f_cont_info seq_h unset_bit_stk_info unboxed tuple component 7 unboxed tuple component 8 <- Sp And then `g` jumps directly to the entry code for `unset_bit_stk_info` (which does the register saving I mentioned), which absolutely can't figure out from Sp alone where seq_h is. Darn! I think Luite's recent work on the StgToByteCode had to work around similar issues, I found this wiki page <https://gitlab.haskell.org/ghc/ghc-wiki-mirror/-/blob/master/Unboxed-Tuples…>. But we aren't in a position where we know the representation of `r` *at all*! So our idea was to scan the stack, beginning from `Sp`, until I find `unset_bit_stk_info`, giving us the following situation: caller_of_f_cont_info seq_h unset_bit_stk_info <- Bp unboxed tuple component 7 unboxed tuple component 8 <- Sp I suggestively named the register in which we store the result Bp in analogy to the traditional base pointer. This information would be enough to unset the bit at index seq_h and then copy the unboxed tuple components 7 and 8 up by two words: caller_of_f_cont_info unboxed tuple component 7 unboxed tuple component 8 <- Sp Then jump to caller_of_f_cont_info, which knows what to make of the stack and the register state. The stack scanning is horrible and too brittle and slow for a production setting, as any of the unboxed tuple components could have the same bit pattern as `unset_bit_stk_info`. We simply hope that is never the case and it's fine for the purposes of a quick-and-dirty instrumentation. QUESTION 1: What do you think? Could this work? Can you anticipate pit falls of this approach? # What about Thunks and StgRhsCon? QUESTION 2: The instrumentation as described won't work for Thunks (which are only entered once) and constructor applications (like sat_sBK in the BigNat matcher above). Not sure how to change that without incurring huge performance hits (by deactivating memoisation). Ideas are welcome here. Thanks for reading this far. I hope you could follow and are now fizzing with excitement because you have a much better idea of how to do this and will let me know :) Sebastian

2 4