[GHC] #16368: Boot GHC's libffi installation path leaks into stage1 build
#16368: Boot GHC's libffi installation path leaks into stage1 build -------------------------------------+------------------------------------- Reporter: sgraf | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: ⊥ Component: Build System | Version: 8.6.3 (Hadrian) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Starting yesterday, I'm having problems building GHC from a `nix-shell --pure https://github.com/alpmestan/ghc.nix/archive/master.tar.gz`. Build comes to a halt after/while configuring the RTS, because it can't find libffi.h in a global nix store installation that I can't find anywhere in my `env`. That nix-store path certainly exists and has an installation of `libffi`, but without an include path (probably since recently, haven't had problems before). After some debugging, I figured out that this path leaks in through `ghc- cabal` crawling through the Package DB of the host GHC. These are the libraries it finds out about this way (dumped by inserting `putStrLn (unlines (forDeps Installed.includeDirs))` in line 388 of `utils/ghc- cabal/Main.hs`: {{{ /nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/base-4.12.0.0/include /nix/store/5c9pfgazxid22ik3smh8zi805cp1i03y-gmp-6.1.2-dev/include /nix/store/7874h075nf8yikvr47642xqrwqwyv99s-ghc-8.6.3/lib/ghc-8.6.3 /integer-gmp-1.0.2.0/include /nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/include /nix/store/karxq4hlfmfj0c3yk4wv5mfaz06p70k8-libffi-3.2.1/include }}} They are then passed on to `DEP_INCLUDE_DIRS_SINGLE_QUOTED` and wreak havoc from there. Which of these paths are vital? The `libffi` path at least seems to shadow the local tarballs for me. This concerns Hadrian and the Make-based build system. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16368 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16368: Boot GHC's libffi installation path leaks into stage1 build -------------------------------------+------------------------------------- Reporter: sgraf | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: ⊥ Component: Build System | Version: 8.6.3 (Hadrian) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by sgraf): * cc: bgamari (added) Comment: The `libffi` bit seems to be pulled in from `rts`, which itself isn't mentioned as include dir: {{{ {sourcePackageId = PackageIdentifier {pkgName = PackageName "rts", pkgVersion = mkVersion [1,0]}, sourceLibName = LMainLibName, installedComponentId_ = ComponentId "", installedUnitId = UnitId "rts", instantiatedWith = [], compatPackageKey = "rts", license = Left (License (ELicense (ELicenseId BSD_3_Clause) Nothing)), copyright = "", maintainer = "glasgow-haskell-users@haskell.org", author = "", stability = "", homepage = "", pkgUrl = "", synopsis = "", description = "", category = "", abiHash = AbiHash "", indefinite = False, exposed = True, exposedModules = [], hiddenModules = [], trusted = False, importDirs = [], libraryDirs = ["/nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/rts","/nix/store/karxq4hlfmfj0c3yk4wv5mfaz06p70k8-libffi-3.2.1/lib"], libraryDynDirs = [], dataDir = "", hsLibraries = ["HSrts"], extraLibraries = ["m","rt","dl","ffi","pthread"], extraGHCiLibraries = [], includeDirs = ["/nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/include","/nix/store/karxq4hlfmfj0c3yk4wv5mfaz06p70k8-libffi-3.2.1/include"], includes = ["Stg.h"], depends = [], abiDepends = [], ccOptions = [], cxxOptions = [], ldOptions = [], frameworkDirs = [], frameworks = [], haddockInterfaces = [], haddockHTMLs = [], pkgRoot = Just "/nix/store /23ndnrb11m3k7zh4p84vjmdgand0p45b-ghc-8.6.3-with-packages/lib/ghc-8.6.3", libVisibility = LibraryVisibilityPrivate} }}} I could probably fix this by the following patch: {{{#!diff diff --git a/utils/ghc-cabal/Main.hs b/utils/ghc-cabal/Main.hs index 8b776499fd..f4a2313194 100644 --- a/utils/ghc-cabal/Main.hs +++ b/utils/ghc-cabal/Main.hs @@ -338,14 +338,22 @@ generate directory distdir config_args [(_,[rts])] -> PackageIndex.insert rts{ Installed.ldOptions = [], - Installed.libraryDirs = filter (not . ("gcc-lib" `isSuffixOf`)) (Installed.libraryDirs rts)} index - -- GHC <= 6.12 had $topdir/gcc-lib in their - -- library-dirs for the rts package, which causes - -- problems when we try to use the in-tree mingw, - -- due to accidentally picking up the incompatible - -- libraries there. So we filter out gcc-lib from - -- the RTS's library-dirs here. + Installed.libraryDirs = filter_dirs (Installed.libraryDirs rts), + Installed.includeDirs = filter_dirs (Installed.includeDirs rts) + } index _ -> error "No (or multiple) ghc rts package is registered!!" + filter_dirs = filter (\dir -> not (or [is_gcc_lib dir, is_libffi dir])) + -- GHC <= 6.12 had $topdir/gcc-lib in their + -- library-dirs for the rts package, which causes + -- problems when we try to use the in-tree mingw, + -- due to accidentally picking up the incompatible + -- libraries there. So we filter out gcc-lib from + -- the RTS's library-dirs here. + is_gcc_lib = ("gcc-lib" `isSuffixOf`) + -- In Trac #16368, we noticed that libffi paths + -- from the boot GHC shadow the local libffi tarballs + -- in a similar manner. + is_libffi = ("libffi" `isInfixOf`) dep_ids = map snd (externalPackageDeps lbi) deps = map display dep_ids }}} Are there any other packages in the above list that look from to you? bgamari, perhaps? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16368#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16368: Boot GHC's libffi installation path leaks into stage1 build -------------------------------------+------------------------------------- Reporter: sgraf | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: ⊥ Component: Build System | Version: 8.6.3 (Hadrian) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by sgraf: Old description:
Starting yesterday, I'm having problems building GHC from a `nix-shell --pure https://github.com/alpmestan/ghc.nix/archive/master.tar.gz`.
Build comes to a halt after/while configuring the RTS, because it can't find libffi.h in a global nix store installation that I can't find anywhere in my `env`. That nix-store path certainly exists and has an installation of `libffi`, but without an include path (probably since recently, haven't had problems before).
After some debugging, I figured out that this path leaks in through `ghc- cabal` crawling through the Package DB of the host GHC. These are the libraries it finds out about this way (dumped by inserting `putStrLn (unlines (forDeps Installed.includeDirs))` in line 388 of `utils/ghc- cabal/Main.hs`:
{{{ /nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/base-4.12.0.0/include /nix/store/5c9pfgazxid22ik3smh8zi805cp1i03y-gmp-6.1.2-dev/include /nix/store/7874h075nf8yikvr47642xqrwqwyv99s-ghc-8.6.3/lib/ghc-8.6.3 /integer-gmp-1.0.2.0/include /nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/include /nix/store/karxq4hlfmfj0c3yk4wv5mfaz06p70k8-libffi-3.2.1/include }}}
They are then passed on to `DEP_INCLUDE_DIRS_SINGLE_QUOTED` and wreak havoc from there.
Which of these paths are vital? The `libffi` path at least seems to shadow the local tarballs for me. This concerns Hadrian and the Make- based build system.
New description: Starting yesterday, I'm having problems building GHC from a `nix-shell --pure https://github.com/alpmestan/ghc.nix/archive/master.tar.gz`. Build comes to a halt after/while configuring the RTS, because it can't find libffi.h in a global nix store installation that I can't find anywhere in my `env`. That nix-store path certainly exists and has an installation of `libffi`, but without an include path (probably since recently, haven't had problems before). After some debugging, I figured out that this path leaks in through `ghc- cabal` crawling through the Package DB of the host GHC. These are the libraries it finds out about this way (dumped by inserting `putStrLn (unlines (forDeps Installed.includeDirs))` in line 388 of `utils/ghc- cabal/Main.hs`: {{{ /nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/base-4.12.0.0/include /nix/store/5c9pfgazxid22ik3smh8zi805cp1i03y-gmp-6.1.2-dev/include /nix/store/7874h075nf8yikvr47642xqrwqwyv99s-ghc-8.6.3/lib/ghc-8.6.3 /integer-gmp-1.0.2.0/include /nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/include /nix/store/karxq4hlfmfj0c3yk4wv5mfaz06p70k8-libffi-3.2.1/include }}} They are then passed on to `DEP_INCLUDE_DIRS_SINGLE_QUOTED` and wreak havoc from there. Which of these paths are vital? The `libffi` path at least seems to shadow the local tarballs for me. This concerns Hadrian and the Make-based build system. For completeness, this is the error I'm eventually seeing: {{{ FFI.hsc:9:10: fatal error: ffi.h: No such file or directory compilation terminated. compiling _build/stage0/libraries/ghci/build/GHCi/FFI_hsc_make.c failed (exit code 1) command was: /nix/store/8zfm4i1aw4c3l5n6ay311ds6l8vd9983-gcc- wrapper-7.4.0/bin/cc -c _build/stage0/libraries/ghci/build/GHCi/FFI_hsc_make.c -o _build/stage0/libraries/ghci/build/GHCi/FFI_hsc_make.o -I/nix/store/891h83mar65k138156v41kzryc9ij0v3-ghc-build- environment/include -I_build/generated -I_build/stage0/libraries/ghci/build -I/nix/store/891h83mar65k138156v41kzryc9ij0v3-ghc-build- environment/include -I/nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/unix-2.7.2.2/include -I/nix/store /7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/time-1.8.0.2/include -I/nix/store /7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/bytestring-0.10.8.2/include -I/nix/store /7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/base-4.12.0.0/include -I/nix/store /5c9pfgazxid22ik3smh8zi805cp1i03y-gmp-6.1.2-dev/include -I/nix/store /7874h075nf8yikvr47642xqrwqwyv99s-ghc-8.6.3/lib/ghc-8.6.3/integer- gmp-1.0.2.0/include -I/nix/store/7874h075nf8yikvr47642xqrwqwyv99s- ghc-8.6.3/lib/ghc-8.6.3/include -I/nix/store/karxq4hlfmfj0c3yk4wv5mfaz06p70k8-libffi-3.2.1/include -Wall -Werror=unused-but-set-variable -Wno-error=inline -include _build/stage0/libraries/ghci/build/autogen/cabal_macros.h -Dx86_64_HOST_ARCH=1 -Dlinux_HOST_OS=1 -D__GLASGOW_HASKELL__=806 }}} -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16368#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16368: Boot GHC's libffi installation path leaks into stage1 build -------------------------------------+------------------------------------- Reporter: sgraf | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: ⊥ Component: Build System | Version: 8.6.3 (Hadrian) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by sgraf): That patch doesn't help at all. It seems we really depend on the libffi bundled with boot GHC(?). The problem is probably with my Nix shell and this recent change https://github.com/NixOS/nixpkgs/commit/a7c774300b7eab779afb9a81b3f02f3bed69.... So, Nix provides a boot GHC that is built and bundled with the system libffi. I still have to figure out where it stores libffi's headers. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16368#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16368: Boot GHC's libffi installation path leaks into stage1 build -------------------------------------+------------------------------------- Reporter: sgraf | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: ⊥ Component: Build System | Version: 8.6.3 (Hadrian) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by sgraf): Nix stores headers in /nix/store/xxx-libffi-3.2.1-dev/include, so it separates build artifacts from headers. So the problem is with the wrong setting [https://github.com/NixOS/nixpkgs/commit/a7c774300b7eab779afb9a81b3f02f3bed69... #diff-50972848d68a2300257f35df16c0fd3aR156 here]. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16368#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16368: Boot GHC's libffi installation path leaks into stage1 build -------------------------------------+------------------------------------- Reporter: sgraf | Owner: (none) Type: bug | Status: closed Priority: normal | Milestone: ⊥ Component: Build System | Version: 8.6.3 (Hadrian) | Resolution: invalid | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by sgraf): * status: new => closed * resolution: => invalid Comment: Fixed in https://github.com/NixOS/nixpkgs/pull/56573. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16368#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC