PUBLIC
PUBLIC
OK I now have a standalone demonstrator that shows, at least, that the default method implementation is not specialized. With the attached input programs, the resulting Core (using GHC e46edfcf47d674731935b2ea1443cc7927e071fb) is as follows (only showing the relevant parts):
seq :: forall (m :: * -> *) a b. Monad m => m a -> m b -> m b
seq
= \ (@(m :: * -> *)) (v_srS [Occ=Once1!] :: Monad m) ->
case v_srS of { C:Monad _ [Occ=Dead] v_srV [Occ=Once1] -> v_srV }
$dmseq :: forall (m :: * -> *) a b. Monad m => m a -> m b -> m b
$dmseq
= \ (@(m :: * -> *))
($dMonad [Occ=Once1] :: Monad m)
(@a)
(@b)
(ma [Occ=Once1] :: m a)
(mb [Occ=OnceL1] :: m b) ->
let {
sat_ss0 [Occ=Once1] :: a -> m b
[LclId]
sat_ss0 = \ _ [Occ=Dead] -> mb } in
bind @m $dMonad @a @b ma sat_ss0
$fMonadIO :: Monad IO
$fMonadIO = C:Monad @IO bindIO $fMonadIO_$cseq;
$fMonadIO_$cseq :: forall a b. IO a -> IO b -> IO b
$fMonadIO_$cseq = \ (@a) (@b) -> $dmseq @IO $fMonadIO @a @b;
foo :: IO ()
foo = seq @IO $fMonadIO @() @() ioA ioA
If I turn on Opt_D_dump_spec, I can see that specializer *is* running, it just doesn't *do* anything.
From: Erdi, Gergo
Sent: Monday, October 11, 2021 4:00 PM
To: Simon Peyton Jones <simonpj(a)microsoft.com>; Matthew Pickering <matthewtpickering(a)gmail.com>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com>; 'GHC' <ghc-devs(a)haskell.org>
Subject: RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)
PUBLIC
Trust me when I say I understand your frustration. It is even more frustrating for me not to be able to just send a Github repo link containing my code...
I'll try to make an MWE that demonstrates the problem but it will probably take quite some time. I was hoping that maybe there's some known gotcha that I'm not aware of - for example (see my other thread), I just discovered that setting optimization flags one by one isn't equal to setting them wholesale with -On, so I was *not* running specialisation in my normal (per-module) pipeline at all! Unfortunately, now that I've discovered this and made sure optLevel is set to at least 1, I am still seeing the polymorphic default implementation of >> as the only one :/
I also tried to be cheeky about the binding order and put the whole collected CoreProgram into a single Rec binder to test your guess, since that should make the actual textual order irrelevant. Unfortunately, that sill doesn't change anything :/
From: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>>
Sent: Monday, October 11, 2021 3:33 PM
To: Erdi, Gergo <Gergo.Erdi(a)sc.com<mailto:Gergo.Erdi@sc.com>>; Matthew Pickering <matthewtpickering(a)gmail.com<mailto:matthewtpickering@gmail.com>>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com<mailto:Raphael.Montelatici@sc.com>>; 'GHC' <ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>>
Subject: [External] RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)
PUBLIC
ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients.
It's incredibly hard to debug this sort of thing remotely, without the ability to reproduce it. First, you are using a variant of GHC, with changes that we can only guess at. Second, even with unmodified GHC I often find that unexpected things happen - until I dig deeper and it becomes obvious!
There is one odd thing about your dump: it seems to be in reverse dependency order, with functions being defined before they are used, rather than after. That would certainly stop the specialiser from working. The occurrence analyser would sort this out (literally). But that's a total guess.
Simon
PS: I am leaving Microsoft at the end of November 2021, at which point simonpj(a)microsoft.com<mailto:simonpj@microsoft.com> will cease to work. Use simon.peytonjones(a)gmail.com<mailto:simon.peytonjones@gmail.com> instead. (For now, it just forwards to simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>.)
From: Erdi, Gergo <Gergo.Erdi(a)sc.com<mailto:Gergo.Erdi@sc.com>>
Sent: 11 October 2021 03:58
To: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>>; Matthew Pickering <matthewtpickering(a)gmail.com<mailto:matthewtpickering@gmail.com>>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com<mailto:Raphael.Montelatici@sc.com>>; 'GHC' <ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>>
Subject: RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)
PUBLIC
PUBLIC
Hi Simon, Matt & others,
It took me until now to be able to try out GHC HEAD, mostly because I had to adapt to all the GHC.Unit.* refactorings. However, now I am on a466b02492f73a43c6cb9ce69491fc85234b9559 which includes the commit Simon pointed out. My original plan was to expose the first half of specProgram, i.e. the part that calls `go binds`. I did that because I want to apply specialisation on collected whole-program Core, not just whatever would be in scope for a Core-to-Core plugin pass, so I am not writing a CoreM and I don't even have a ModGuts at hand.
However, I found out from Matt's email on this thread that this is not going to be enough and eventually I'll need to figure out how I intend to apply the rewrite rules that come out of this. So for now, I am just turning on specialization in the normal pipeline by setting Opt_Specialise, Opt_SpecialiseAggressively, and Opt_CrossModuleSpecialise. And I'm still not seeing $dm>> being specialized.
Is this because I define each of "class Monad", "data IO a", "instance Monad IO", and "main", in four distinct modules? In other words, is this something I will not be able to try out until I figure out how to make a fake ModGuts and run a CoreM from outside the normal compilation pipeline, feeding it the whole-program collected CoreBinds? But if so, why is it that when I feed my whole program to just specBinds (which I can try easily since it has no ModGuts/CoreM dependency other than a uniq supply and the CoreProgram), I get back an empty UsageDetails?
Thanks,
Gergo
For reference, the relevant definitions dumped from GHC with specialization (supposedly) turned on:
main = $fMonadIO_$c>> @() @() sat_sJg xmain
$fMonadIO_$c>> :: forall a b. IO a -> IO b -> IO b
$fMonadIO_$c>> = \ (@a_aF9) (@b_aFa) -> $dm>> @IO $fMonadIO @a_aF9 @b_aFa;
$dm>> :: forall (m :: Type -> Type) a b. Monad m => m a -> m b -> m b
$dm>>
= \ (@(m_ani :: Type -> Type))
($dMonad_sIi [Occ=Once1] :: Monad m_ani)
(@a_ar4)
(@b_ar5)
(ma_sIj [Occ=Once1] :: m_ani a_ar4)
(mb_sIk [Occ=OnceL1] :: m_ani b_ar5) ->
let {
sat_sIm [Occ=Once1] :: a_ar4 -> m_ani b_ar5
[LclId]
sat_sIm = \ _ [Occ=Dead] -> mb_sIk } in
>>= @m_ani $dMonad_sIi @a_ar4 @b_ar5 ma_sIj sat_sIm
From: Erdi, Gergo
Sent: Thursday, October 7, 2021 9:30 AM
To: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com<mailto:Raphael.Montelatici@sc.com>>; GHC <ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>>
Subject: RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)
PUBLIC
Indeed, I am using 9.0.1. I'll try upgrading. Thanks!
From: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>>
Sent: Wednesday, October 6, 2021 6:12 PM
To: Erdi, Gergo <Gergo.Erdi(a)sc.com<mailto:Gergo.Erdi@sc.com>>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici(a)sc.com<mailto:Raphael.Montelatici@sc.com>>; GHC <ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>>
Subject: [External] RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)
Grego,
Yes I think that should auto-specialise.
Which version of GHC are you using? Do you have this patch?
commit ef0135934fe32da5b5bb730dbce74262e23e72e8
Author: Simon Peyton Jones simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>
Date: Thu Apr 8 22:42:31 2021 +0100
Make the specialiser handle polymorphic specialisation
Here's why I ask. The call
$fMonadIO_$c>> = \ (@a) (@b) -> $dm>> @IO $fMonadIO @a @b
indeed applies $dm>> to $fMonadIO, but it also applies it to a and b. In the version of GHC you have, maybe that stops the call from floating up to the definition site, and being used to specialise it.
Can you make a repro case without your plugin?
Simon
PS: I am leaving Microsoft at the end of November 2021, at which point simonpj(a)microsoft.com<mailto:simonpj@microsoft.com> will cease to work. Use simon.peytonjones(a)gmail.com<mailto:simon.peytonjones@gmail.com> instead. (For now, it just forwards to simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>.)
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations
Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm
Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.
Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions.
Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.
Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations
Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm
Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.
Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions.
Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.
Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
