[GHC] #11179: Allow plugins to access "dead code"
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- GHC removes dead-bindings quite early in the compilation process, which makes perfect sense from a pure compilation point of view. But with user- plugins, some of the dead-bindings can be interesting: They might be properties or other code that instructs the plugin to do a certain action, even if the binding isn't otherwise used anywhere else or exported. (Think of embedded properties.) Here's a discussion about the issue, where SimonPJ asked for a ticket to be filed: [http://mail.haskell.org/pipermail/ghc- devs/2015-December/010708.html] It appears the desugarer removes some of the dead-code. One option could be to stop the desugarer from doing that, and leaving it to the later optimizer passes so plugins can still see all the bindings (if they run early enough), or require the user to put in an "KeepAlive" pragma on bindings that she cares about. While keeping all-dead code in the desugarer would be the simplest thing to do, requiring the user to put in a pragma isn't a terrible solution either. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by gridaphobe): * cc: gridaphobe (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I'm happy to advise. It's actually the occurrence analyser that's dropping the dead code, incidentally. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ak3n Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by ak3n): * owner: => ak3n -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ak3n Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ak3n): Which solution is better? Move out dead code elimination from `simpleOptPgm` to later pass after plugins? Or add `KeepAlive` pragma with flags? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by ak3n): * owner: ak3n => -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * related: => #10823 Comment: #10823 refers to the same issue (which I just stumbled over myself). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * related: #10823 => Comment: I have his this problem as well now. I don’t like any solution that requires unwieldy interaction from the user. That leaves * not removing dead code in the desugarer or * adding a hook to the plugins interface to interact with the code before desugaring (and maybe mark bindings as “please keep”) While the second one would surely be useful for many other things, I wonder why we not simply do the first one. We remove dead code later anyways (do we?), and the extra cost of desugaring dead code seems to be neglectible, as there usually is no dead code. (And if there is, then most likely for a good reason – e.g. to be picked up by plugins). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by gridaphobe): I looked into this briefly a while ago, and IIRC the first option may be as simple as removing the call to `simpleOptPgm` inside `Desugar.deSugar`. Or, rather, moving it to an initial Core2Core pass, which would preserve the current pipeline but cleanly allow plugins access to the fully unoptimized program. There's a warning right before the call though, that suggests the unoptimized program may be excessively large. I don't know how much of a concern this would be in practice. The other thing to consider is whether removing the initial optimization pass may break existing Core2Core plugins. If a plugin inserts itself at the beginning of the Core2Core pipeline, the overall order would change from {{{ simpleOptPgm > plugin > ... }}} to {{{ plugin > simpleOptPgm > ... }}} For some plugins, e.g. Levent's or LiquidHaskell, this is exactly what is desired, but it could break others. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Rather than mess with `simpleOptPgm` I think it might be better just to ask the occurrence analyser (which `simplOptPgm` uses) not to drop dead code, at least not at top level. Tha can't be hard. Just pass a boolean into `occAnalBind`. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * status: new => patch * differential: => Phab:D3073 Comment: Done it. Also had to make sure no binders are maked as `Dead`. See Phab:D3073. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * related: => #10823 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by gridaphobe): I'm not sure that keeping dead top-level binders is sufficient. I believe that `simpleOptPgm` does other things that cause trouble for analysis tools. For example, IIRC it will automatically inline single-use, unexported binders. It would be nice if plugins could see a "faithful" representation of the source code, i.e. as close to the parsed HsSyn as possible. I realize this is a bit out of scope with respect to the original ticket, happy to file a separate ticket with a concrete example. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): ha, here is a hack to keep things alive: {{{ module Test () where foo = foo {-# RULES "keeps foo alive" id foo = foo #-} }}} I will try revise Phab:D3073 with a nicer approach, but for people stuck with older versions of GHC, this is might help. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata):
I realize this is a bit out of scope with respect to the original ticket, happy to file a separate ticket with a concrete example.
I guess that’s worth a separate ticket. Maybe we want plugins that run on the type-checked code before desugaring? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by gridaphobe): After a closer I don't think it's actually worth opening a separate ticket, as the optimizations the desugarer performs are truly simple. It's just 1. the dead code removal this ticket already aims to address, and 2. inlining single-use functions, which is also problematic for analysis tools (we may have annotated the function with some internal invariant!). As an example, {{{ foo :: Int -> Int foo x = g x where -- g is only used once so it is immediately inlined by -- CoreSubst.simple_opt_expr g y = y + 1 }}} Before `simpleOptPgm`: {{{ foo :: Int -> Int foo = \ (x :: Int) -> letrec { g :: Int -> Int g = letrec { g :: Int -> Int g = \ (y :: Int) -> + y 1; } in g; } in g x }}} After: {{{ foo :: Int -> Int foo = \ (x :: Int) -> (\ (y :: Int) -> + y 1) x }}} Granted, the nested definition of `g` pre-optimization is weird, but LiquidHaskell (at least) already handles this pattern without any special effort on our end. I think a cleaner solution would be my suggestion to move `simpleOptPgm` from the desugarer to an initial Core2Core pass (perhaps guarded by a flag so it doesn't bug regular users). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by gridaphobe): (I'm happy to implement my suggestion, it would also let me finish the languishing Phab:D1565 and remove the desugarer fork we use in LiquidHaskell) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): (I would not object a different implementation, as long as it gets the job done; preferably without requiring the user to add any flags or annotations.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
It would be nice if plugins could see a "faithful" representation of the
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I'm getting lost here * First: source code I have no idea what that means, but by all means open a new feature request with a proposed design. * Second: should `simpleOptPgm` be a separate pass? The reason it's put in the back of the desugarer is not adequately documented. So it'd be worth a try if there is some benefit (what is the benefit?). You'd also need to ensure that Lint did not check the output of the desugarer, only the output of that first pass. `simpleOptPgm` really discards a LOT of junk sometimes! * Third, returning to the original question of the ticket, how can we keep all the binders alive in the output of `simpleOptPgm`? For that I like the approach you are taking in Phab:D3073; having a keep-alive set is nicer than messing with the guts of the analysis. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#11179: Allow plugins to access "dead code" -------------------------------------+------------------------------------- Reporter: lerkok | Owner: (none) Type: feature request | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #10823 | Differential Rev(s): Phab:D3073 Wiki Page: | -------------------------------------+------------------------------------- Comment (by abakst): Here is one benefit of moving `simpleOptPgm` to a separate pass: I am currently writing a static analysis as a CoreToCore plugin pass. Consider the following: {{{ {-# OPTIONS_GHC -fplugin Analysis.Plugin -fplugin-opt Analysis.Plugin:dataNodeProcess #-} module DataNode where import Control.Distributed.Process.ManagedProcess dataNodeProcess :: ProcessDefinition Int dataNodeProcess = defaultProcess { apiHandlers = [handleCall $ \i () -> reply () i] } }}} The associated core *without* optimizations (`ghc -O0 DataNode.hs -ddump- ds`) is {{{ dataNodeProcess dataNodeProcess = case defaultProcess of _ { ProcessDefinition _ ds_d4tk ds_d4tl ds_d4tm ds_d4tn ds_d4to -> ProcessDefinition (: ($ (handleCall $dSerializable_a462 $dSerializable_a462) (\ i_a30r ds_d4tf -> case ds_d4tf of _ { () -> reply $dSerializable_a462 () i_a30r })) []) ds_d4tk ds_d4tl ds_d4tm ds_d4tn ds_d4to } }}} The detail isn't important, but since `defaultProcess` is exported by`ManagedProcess`, the analysis knows what to do. On the other hand, if I run `ghc -O2 DataNode.hs -ddump-ds`, this is (part of) the resulting core: {{{ dataNodeProcess dataNodeProcess = ProcessDefinition (build (\ @ a_d4y4 c_d4y5 n_d4y6 -> c_d4y5 ($ (handleCall $dSerializable_a4an $dSerializable_a4an) (\ i_a33Z ds_d4y0 -> case ds_d4y0 of _ { () -> reply $dSerializable_a4an () i_a33Z })) n_d4y6)) [] [] (defaultProcess2 `cast` ...) (defaultProcess1 `cast` ...) Terminate }}} As `defaultProcess1` and `defaultProcess2` are not exported by `ManagedProcess`, the analysis has no idea what do (which can be quite annoying if it's in a place where we want precision). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11179#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC