[GHC] #12733: Orphan hashes are not being propagated correctly

#12733: Orphan hashes are not being propagated correctly -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Driver | Version: 8.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In Phab:D2607, simonpj asked why we didn't also put class instances in the export hash. Why indeed. In the course of investigating, I discovered another bug in how we handle orphan recompilation. Unfortunately, this bug needs two packages to manifest. {{{ -- p/P.hs module P where instance Show (a -> b) where --instance Show (IO a) where -- q/Q.hs module Q where import P -- q/Q2.hs module Q2 where import Q }}} Put in some default Cabal files, with q depending on p, and then build the entire kaboodle. Then edit `p/P.hs` to uncomment the second orphan instance. Rebuild. {{{ ezyang@sabre:~/Dev/labs/orph$ cabal new-build q -w ghc-head In order, the following will be built (use -v for more details): - p-0.1.0.0 (lib) (file P.hs changed) - q-0.1.0.0 (lib) (dependency rebuilt) Preprocessing library p-0.1.0.0... [1 of 1] Compiling P ( P.hs, /srv/code/labs/orph/dist- newstyle/build/x86_64-linux/ghc-8.1.20161016/p-0.1.0.0/build/P.o ) P.hs:2:10: warning: [-Wmissing-methods] • No explicit implementation for either ‘showsPrec’ or ‘show’ • In the instance declaration for ‘Show (a -> b)’ P.hs:3:10: warning: [-Wmissing-methods] • No explicit implementation for either ‘showsPrec’ or ‘show’ • In the instance declaration for ‘Show (IO a)’ module Q2 where Preprocessing library q-0.1.0.0... [1 of 2] Compiling Q ( Q.hs, /srv/code/labs/orph/dist- newstyle/build/x86_64-linux/ghc-8.1.20161016/q-0.1.0.0/build/Q.o ) [P changed] ezyang@sabre:~/Dev/labs/orph$ }}} Notice: Q2 never got rebuilt. This is wrong wrong wrong: P changed, and we need to rebuild Q2! Here are the hashes for Q.hi before and after: {{{ -- Before interface Q 80120161016 interface hash: 2ffc06a1552565a73c888110ab4cf8df ABI hash: b2768862866252ea5a63401d2aa05aaa export-list hash: 092acc280445a8bcc96904afb8b766b0 orphan hash: 693e9af84d3dfcc71e640e005bdc5e2e flag hash: f01bb78d3c47152bc5253f256b2a60d3 -- After interface Q 80120161016 interface hash: 009db9e5ecbe49e36b7bf38e02087e75 ABI hash: b2768862866252ea5a63401d2aa05aaa export-list hash: 092acc280445a8bcc96904afb8b766b0 orphan hash: 693e9af84d3dfcc71e640e005bdc5e2e flag hash: f01bb78d3c47152bc5253f256b2a60d3 }}} The orphan hash didn't get updated. At the moment, when we compute the orphan hash, we only consider the orphan hashes from the home package. Here's the comment justifying it: {{{ -- the export hash of a module depends on the orphan hashes of the -- orphan modules below us in the dependency tree. This is the way -- that changes in orphans get propagated all the way up the -- dependency tree. We only care about orphan modules in the current -- package, because changes to orphans outside this package will be -- tracked by the usage on the ABI hash of package modules that we import. }}} It is true that we know to rebuild Q.hs when P.hs changes. But that is not much solace for Q2.hs... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12733 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12733: Orphan hashes are not being propagated correctly -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Driver | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by ezyang: @@ -75,1 +75,1 @@ - The orphan hash didn't get updated. + The ABI hash didn't get updated. @@ -77,1 +77,1 @@ - At the moment, when we compute the orphan hash, we only consider the + At the moment, when we compute the export hash, we only consider the New description: In Phab:D2607, simonpj asked why we didn't also put class instances in the export hash. Why indeed. In the course of investigating, I discovered another bug in how we handle orphan recompilation. Unfortunately, this bug needs two packages to manifest. {{{ -- p/P.hs module P where instance Show (a -> b) where --instance Show (IO a) where -- q/Q.hs module Q where import P -- q/Q2.hs module Q2 where import Q }}} Put in some default Cabal files, with q depending on p, and then build the entire kaboodle. Then edit `p/P.hs` to uncomment the second orphan instance. Rebuild. {{{ ezyang@sabre:~/Dev/labs/orph$ cabal new-build q -w ghc-head In order, the following will be built (use -v for more details): - p-0.1.0.0 (lib) (file P.hs changed) - q-0.1.0.0 (lib) (dependency rebuilt) Preprocessing library p-0.1.0.0... [1 of 1] Compiling P ( P.hs, /srv/code/labs/orph/dist- newstyle/build/x86_64-linux/ghc-8.1.20161016/p-0.1.0.0/build/P.o ) P.hs:2:10: warning: [-Wmissing-methods] • No explicit implementation for either ‘showsPrec’ or ‘show’ • In the instance declaration for ‘Show (a -> b)’ P.hs:3:10: warning: [-Wmissing-methods] • No explicit implementation for either ‘showsPrec’ or ‘show’ • In the instance declaration for ‘Show (IO a)’ module Q2 where Preprocessing library q-0.1.0.0... [1 of 2] Compiling Q ( Q.hs, /srv/code/labs/orph/dist- newstyle/build/x86_64-linux/ghc-8.1.20161016/q-0.1.0.0/build/Q.o ) [P changed] ezyang@sabre:~/Dev/labs/orph$ }}} Notice: Q2 never got rebuilt. This is wrong wrong wrong: P changed, and we need to rebuild Q2! Here are the hashes for Q.hi before and after: {{{ -- Before interface Q 80120161016 interface hash: 2ffc06a1552565a73c888110ab4cf8df ABI hash: b2768862866252ea5a63401d2aa05aaa export-list hash: 092acc280445a8bcc96904afb8b766b0 orphan hash: 693e9af84d3dfcc71e640e005bdc5e2e flag hash: f01bb78d3c47152bc5253f256b2a60d3 -- After interface Q 80120161016 interface hash: 009db9e5ecbe49e36b7bf38e02087e75 ABI hash: b2768862866252ea5a63401d2aa05aaa export-list hash: 092acc280445a8bcc96904afb8b766b0 orphan hash: 693e9af84d3dfcc71e640e005bdc5e2e flag hash: f01bb78d3c47152bc5253f256b2a60d3 }}} The ABI hash didn't get updated. At the moment, when we compute the export hash, we only consider the orphan hashes from the home package. Here's the comment justifying it: {{{ -- the export hash of a module depends on the orphan hashes of the -- orphan modules below us in the dependency tree. This is the way -- that changes in orphans get propagated all the way up the -- dependency tree. We only care about orphan modules in the current -- package, because changes to orphans outside this package will be -- tracked by the usage on the ABI hash of package modules that we import. }}} It is true that we know to rebuild Q.hs when P.hs changes. But that is not much solace for Q2.hs... -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12733#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12733: Orphan hashes are not being propagated correctly -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Driver | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by ezyang): Suggested fix: compute the export hash by hashing together the orphan hashes of ALL transitively reachable orphan modules, not just the ones from the home package. But perhaps this will have some negative performance implication? (Perhaps not too bad; we only pay for orphans, and there are not too many of them. We pay a lot more for family instances.) Another possibility would be to maintain a "transitive orphan hash" which is the orphan hash and the orphan hashes of all transitively imported orphan modules in a module. Then, for imports for external packages, we only put the orphan hash of direct imports into our export hash. Then we wouldn't have to probe all transitively imported orphans; just direct imports--the downside is that if any of these transitive orphan hashes change, our export hash would change, even if the final effective orphans in scope stayed the same. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12733#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12733: Orphan hashes are not being propagated correctly -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Driver | Version: 8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2610 Wiki Page: | -------------------------------------+------------------------------------- Changes (by ezyang): * status: new => patch * differential: => Phab:D2610 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12733#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12733: Orphan hashes are not being propagated correctly
-------------------------------------+-------------------------------------
Reporter: ezyang | Owner:
Type: bug | Status: patch
Priority: normal | Milestone:
Component: Driver | Version: 8.1
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2610
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#12733: Orphan hashes are not being propagated correctly -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Driver | Version: 8.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2610 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12733#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12733: Orphan hashes are not being propagated correctly -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: bug | Status: closed Priority: normal | Milestone: 8.2.1 Component: Driver | Version: 8.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2610 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: => 8.2.1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12733#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC