[GHC] #10424: Build path leaks into ABI hashes
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- The build path of GHC leaks into the ABI hashes, i.e. if you build GHC in /tmp/fooADDF and then in /tmp/fooSDFS, you get different ABI hashes. This is a serious problem for distributions, as it means that we cannot rebuild the GHC package without rebuilding all Haskell packages as well. The (or at least one) problem is include paths. We have {{{ include-dirs: /home/jojo/build/haskell/ghc/rts/dist/build /home/jojo/build/haskell/ghc/includes /home/jojo/build/haskell/ghc/includes/dist- derivedconstants/header }}} in `inplace/lib/package.conf.d/builtin_rts.conf`, so when running the preprocessor, GHC passes this full path to the preprocessor: {{{ /usr/bin/gcc -E -undef -traditional -DOPTIMISE_INTEGER_GCD_LCM -include libraries/base/dist-install/build/autogen/cabal_macros.h -I libraries/base /dist-install/build -I libraries/base/dist-install/build -I libraries/base /dist-install/build/autogen -I libraries/base/include -I /home/jojo/build/haskell/ghc/libraries/integer-gmp/include -I /home/jojo/build/haskell/ghc/rts/dist/build -I /home/jojo/build/haskell/ghc/includes -I /home/jojo/build/haskell/ghc/includes/dist-derivedconstants/header '-D__GLASGOW_HASKELL__=711' -include /home/jojo/build/haskell/ghc/includes/ghcversion.h '-Dlinux_BUILD_OS=1' '-Dx86_64_BUILD_ARCH=1' '-Dlinux_HOST_OS=1' '-Dx86_64_HOST_ARCH=1' '-D__GLASGOW_HASKELL_TH__=YES' '-D__SSE__=1' '-D__SSE2__=1' -x assembler- with-cpp libraries/base/System/Info.hs -o /tmp/ghc14804_0/ghc14804_1.hscpp }}} It then reads the included files from the `hscpp` file : {{{ $ grep home /tmp/ghc14830_0/ghc14830_1.hscpp # 1 "/home/jojo/build/haskell/ghc/includes/ghcversion.h" 1 # 1 "/home/jojo/build/haskell/ghc/includes/ghcplatform.h" 1 }}} and puts it into the interface, where it becomes part of the hash: {{{ $ ./inplace/bin/ghc-stage1 --show-iface ./libraries/base/dist- install/build/System/Info.hi |grep Dep addDependentFile "/home/jojo/build/haskell/ghc/includes/ghcplatform.h" addDependentFile "/home/jojo/build/haskell/ghc/includes/ghcversion.h" addDependentFile "libraries/base/dist- install/build/autogen/cabal_macros.h" addDependentFile "/usr/include/stdc-predef.h" }}} Clearly, the end result is undesirable (changing hashes due to path names) and actually useless, as the build path will not be there later anyways. I’m not sure what the best way to fix this is. Here are a few ideas: * Do not include the path of `addDependentFile` entries in the hash, but only the file hash. Should solve the ABI hash. * Use relative paths in `inplace/lib/package.conf.d/builtin_rts.conf`. Not sure what breaks. This bug appeared in 7.8.4 and is present in both 7.10 and current HEAD. Any other ideas? (Debian bugreport at http://bugs.debian.org/785282) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #4012 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by nomeata): * related: => #4012 Comment: #4012 is related, but not fully: I guess #4012 would be fixed if compilation is deterministic in the same environment. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #4012 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by nomeata): Here is a possible fix: {{{ diff --git a/compiler/iface/MkIface.hs b/compiler/iface/MkIface.hs index 9a2cd35..180742f 100644 --- a/compiler/iface/MkIface.hs +++ b/compiler/iface/MkIface.hs @@ -622,7 +622,7 @@ addFingerprints hsc_env mb_old_fingerprint iface0 new_decls iface_hash <- computeFingerprint putNameLiterally (mod_hash, ann_fn (mkVarOcc "module"), -- See mkIfaceAnnCache - mi_usages iface0, + usages, sorted_deps, mi_hpc iface0) @@ -655,6 +655,9 @@ addFingerprints hsc_env mb_old_fingerprint iface0 new_decls (non_orph_fis, orph_fis) = mkOrphMap ifFamInstOrph (mi_fam_insts iface0) fix_fn = mi_fix_fn iface0 ann_fn = mkIfaceAnnCache (mi_anns iface0) + -- Do not allow filenames to affect the interface + usages = [ case u of UsageFile _ fp -> UsageFile "" fp; _ -> u | u <- mi_usages iface0 ] + getOrphanHashes :: HscEnv -> [Module] -> IO [Fingerprint] getOrphanHashes hsc_env mods = do }}} This excludes the actual filename from the hash generation. It seems to solve the problem for Debian (so likely we will use it). But I also think it is generally fine to have. It would be bad if the hash would not change although we should recompile depending modules. But even if the actual filenames change for some reason, as long as their content was the same (and the file’s hash is part of the module hash), I don’t think we have to recompile the depending modules. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #4012 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): Looks sensible to me, although I am not deep into ABI hashes. '''Is there a single Note where the desired semantics of ABI hashes are explained?''' Including the consequences for distros, which led to this ticket being created? Then we could refer to that Note from here. The comment "Do not allow filenames to affect the interface" just isn't enough! Even an inadequate Note would be better than none. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #4012 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): JFTR, the Debian package of 7.10.2 in experimental is including this patch since a while: https://sources.debian.net/src/ghc/7.10.2-2/debian/patches /buildpath-abi-stability.patch/ So far, no bugs were reported, but I’m not sure how many users of the 7.10 package are out there. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #4012 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * cc: niteria (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: niteria Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #4012 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by niteria): * owner: (none) => niteria -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: niteria Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #4012 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by juhpetersen): Any reason why this was never merged? We have also used the patch in Fedora without problems. Or has this been addressed already in another way? The latest patch is still in later debian and Fedora packages: https://salsa.debian.org/haskell- team/DHG_packages/blob/experimental/p/ghc/debian/patches/buildpath-abi- stability.patch -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: niteria Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #4012 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata):
Any reason why this was never merged? We have also used the patch in Fedora without problems.
The only reason is that likely only Debian and Fedora care, and both of us have the patch … :-) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10424: Build path leaks into ABI hashes -------------------------------------+------------------------------------- Reporter: nomeata | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.4 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #4012 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by niteria): * owner: niteria => (none) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10424#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC