Build failure -- missing dependency? Help!

I'm getting this (with 'sh validate -legacy'). Oddly * It does not happen on HEAD * It does happen on wip/T19495, a tiny patch with one innocuous change to GHC.Tc.Gen.HsType I can't see how my patch could possible cause "missing files" in ghc-bignum! I'm guessing that there is a missing dependency that someone doesn't show up in master, but does in my branch, randomly. There's something funny about ghc-bignum; it doesn't seem to be a regular library Can anyone help? Thanks Simon "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id base-4.16.0.0 -hide-all-packages -package-env - -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -Ilibraries/base/dist-install/build/include -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id ghc-bignum-1.0 -package-id ghc-prim-0.8.0 -package-id rts -this-unit-id base -Wcompat -Wnoncanonical-monad-instances -XHaskell2010 -O -dcore-lint -ticky -Wwarn -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -outputdir libraries/base/dist-install/build -dynamic-too -c libraries/base/./GHC/Exception/Type.hs-boot -o libraries/base/dist-install/build/GHC/Exception/Type.o-boot -dyno libraries/base/dist-install/build/GHC/Exception/Type.dyn_o-boot Failed to load interface for 'GHC.Num.Integer' There are files missing in the 'ghc-bignum' package, try running 'ghc-pkg check'. Use -v (or `:set -v` in ghci) to see a list of the files searched for. make[1]: *** [libraries/base/ghc.mk:4: libraries/base/dist-install/build/GHC/Exception/Type.o-boot] Error 1 make[1]: *** Waiting for unfinished jobs....

On Mar 14, 2021, at 6:53 PM, Simon Peyton Jones via ghc-devs
wrote: I’m getting this (with ‘sh validate –legacy’). Oddly
• It does not happen on HEAD • It does happen on wip/T19495, a tiny patch with one innocuous change to GHC.Tc.Gen.HsType I can’t see how my patch could possible cause “missing files” in ghc-bignum!
I’m guessing that there is a missing dependency that someone doesn’t show up in master, but does in my branch, randomly.
There’s something funny about ghc-bignum; it doesn’t seem to be a regular library
Can anyone help?
I managed to reproduce the issue on my machine, and noticed that after: $ cd libraries/ghc-bignum/ $ gmake $ cd ../.. $ ./validate --legacy --no-clean the build continues OK. So it looks like the legacy parallel build has a missing dependency on the completion of the build of libraries/ghc-bignum at the point when it is trying to run: $ "inplace/bin/ghc-stage1" -v1 \ -hisuf hi \ -osuf o \ -hcsuf hc \ -static -O0 -H64m -Wall -fllvm-fill-undef-with-garbage -Werror \ -this-unit-id base-4.16.0.0 \ -hide-all-packages -package-env - -i \ -ilibraries/base/. \ -ilibraries/base/dist-install/build \ -Ilibraries/base/dist-install/build \ -ilibraries/base/dist-install/build/./autogen \ -Ilibraries/base/dist-install/build/./autogen \ -Ilibraries/base/include \ -Ilibraries/base/dist-install/build/include \ -optP-include \ -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h \ -package-id ghc-bignum-1.0 \ -package-id ghc-prim-0.8.0 \ -package-id rts \ -this-unit-id base \ -Wcompat -Wnoncanonical-monad-instances \ -XHaskell2010 -O \ -dcore-lint -dno-debug-output \ -no-user-package-db \ -rtsopts \ -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances \ -outputdirlibraries/base/dist-install/build \ -dynamic-too \ -c libraries/base/./GHC/Exception/Type.hs-boot \ -o libraries/base/dist-install/build/GHC/Exception/Type.o-boot \ -dyno libraries/base/dist-install/build/GHC/Exception/Type.dyn_o-boot My best guess is that the problem command fires via libraries/base/dist-install/package-data.mk which is created by cabal, and things get rather complicated from there... -- Viktor.

Hi Simon, The issue is that: 1. Make build system doesn't respect package dependencies, only module dependencies (afaik) 2. The build system isn't aware that most modules implicitly depend on GHC.Num.Integer/Natural (to desugar Integer/Natural literals) That's why we have several fake imports in `base` that look like:
import GHC.Num.Integer () -- See Note [Depend on GHC.Num.Integer] in GHC.Base
Note [Depend on GHC.Num.Integer] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Integer type is special because GHC.Iface.Tidy uses constructors in GHC.Num.Integer to construct Integer literal values. Currently it reads the interface file whether or not the current module *has* any Integer literals, so it's important that GHC.Num.Integer is compiled before any other module. (There's a hack in GHC to disable this for packages ghc-prim and ghc-bignum which aren't allowed to contain any Integer literals.) Likewise we implicitly need Integer when deriving things like Eq instances. The danger is that if the build system doesn't know about the dependency on Integer, it'll compile some base module before GHC.Num.Integer, resulting in: Failed to load interface for ‘GHC.Num.Integer’ There are files missing in the ‘ghc-bignum’ package, Bottom line: we make GHC.Base depend on GHC.Num.Integer; and everything else either depends on GHC.Base, or does not have NoImplicitPrelude (and hence depends on Prelude). Note: this is only a problem with the make-based build system. Hadrian doesn't seem to interleave compilation of modules from separate packages and respects the dependency between `base` and `ghc-bignum`. So we should add a similar fake import into libraries/base/GHC/Exception/Type.hs-boot. I will open a MR. Sylvain On 14/03/2021 21:53, Simon Peyton Jones via ghc-devs wrote:
I’m getting this (with ‘sh validate –legacy’). Oddly
* It does not happen on HEAD * It does happen on wip/T19495, a tiny patch with one innocuous change to GHC.Tc.Gen.HsType
I can’t see how my patch could possible cause “missing files” in ghc-bignum!
I’m guessing that there is a missing dependency that someone doesn’t show up in master, but does in my branch, randomly.
There’s something funny about ghc-bignum; it doesn’t seem to be a regular library
Can anyone help?
Thanks
Simon
"inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id base-4.16.0.0 -hide-all-packages -package-env - -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -Ilibraries/base/dist-install/build/include -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id ghc-bignum-1.0 -package-id ghc-prim-0.8.0 -package-id rts -this-unit-id base -Wcompat -Wnoncanonical-monad-instances -XHaskell2010 -O -dcore-lint -ticky -Wwarn -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -outputdir libraries/base/dist-install/build -dynamic-too -c libraries/base/./GHC/Exception/Type.hs-boot -o libraries/base/dist-install/build/GHC/Exception/Type.o-boot -dyno libraries/base/dist-install/build/GHC/Exception/Type.dyn_o-boot
Failed to load interface for ‘GHC.Num.Integer’
There are files missing in the ‘ghc-bignum’ package,
try running 'ghc-pkg check'.
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
make[1]: *** [libraries/base/ghc.mk:4: libraries/base/dist-install/build/GHC/Exception/Type.o-boot] Error 1
make[1]: *** Waiting for unfinished jobs....
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Thanks Sylvain
So we should add a similar fake import into libraries/base/GHC/Exception/Type.hs-boot. I will open a MR.
Thank you! Don't forget to comment it - especially because it is fake.
Make build system doesn't respect package dependencies, only module dependencies (afaik)
Does Hadrian suffer from this malady too? Are the fake imports needed? Or can we sweep them away when we sweep away make?
Simon
From: ghc-devs
import GHC.Num.Integer () -- See Note [Depend on GHC.Num.Integer] in GHC.Base
Note [Depend on GHC.Num.Integer] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Integer type is special because GHC.Iface.Tidy uses constructors in GHC.Num.Integer to construct Integer literal values. Currently it reads the interface file whether or not the current module *has* any Integer literals, so it's important that GHC.Num.Integer is compiled before any other module. (There's a hack in GHC to disable this for packages ghc-prim and ghc-bignum which aren't allowed to contain any Integer literals.) Likewise we implicitly need Integer when deriving things like Eq instances. The danger is that if the build system doesn't know about the dependency on Integer, it'll compile some base module before GHC.Num.Integer, resulting in: Failed to load interface for 'GHC.Num.Integer' There are files missing in the 'ghc-bignum' package, Bottom line: we make GHC.Base depend on GHC.Num.Integer; and everything else either depends on GHC.Base, or does not have NoImplicitPrelude (and hence depends on Prelude). Note: this is only a problem with the make-based build system. Hadrian doesn't seem to interleave compilation of modules from separate packages and respects the dependency between `base` and `ghc-bignum`. So we should add a similar fake import into libraries/base/GHC/Exception/Type.hs-boot. I will open a MR. Sylvain On 14/03/2021 21:53, Simon Peyton Jones via ghc-devs wrote: I'm getting this (with 'sh validate -legacy'). Oddly 1. It does not happen on HEAD 2. It does happen on wip/T19495, a tiny patch with one innocuous change to GHC.Tc.Gen.HsType I can't see how my patch could possible cause "missing files" in ghc-bignum! I'm guessing that there is a missing dependency that someone doesn't show up in master, but does in my branch, randomly. There's something funny about ghc-bignum; it doesn't seem to be a regular library Can anyone help? Thanks Simon "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id base-4.16.0.0 -hide-all-packages -package-env - -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -Ilibraries/base/dist-install/build/include -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id ghc-bignum-1.0 -package-id ghc-prim-0.8.0 -package-id rts -this-unit-id base -Wcompat -Wnoncanonical-monad-instances -XHaskell2010 -O -dcore-lint -ticky -Wwarn -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -outputdir libraries/base/dist-install/build -dynamic-too -c libraries/base/./GHC/Exception/Type.hs-boot -o libraries/base/dist-install/build/GHC/Exception/Type.o-boot -dyno libraries/base/dist-install/build/GHC/Exception/Type.dyn_o-boot Failed to load interface for 'GHC.Num.Integer' There are files missing in the 'ghc-bignum' package, try running 'ghc-pkg check'. Use -v (or `:set -v` in ghci) to see a list of the files searched for. make[1]: *** [libraries/base/ghc.mk:4: libraries/base/dist-install/build/GHC/Exception/Type.o-boot] Error 1 make[1]: *** Waiting for unfinished jobs.... _______________________________________________ ghc-devs mailing list ghc-devs@haskell.orgmailto:ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devshttps://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc275ff3bbab342f7a26508d8e78c9369%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637513938702730130%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=piDkvXfdWip%2FxOKiYwYUb1G%2FvCpliL9dBuV8ltIdE10%3D&reserved=0

Thank you! Don’t forget to comment it – especially because it is fake.
Done in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5265
Make build system doesn't respect package dependencies, only module dependencies (afaik)
Does Hadrian suffer from this malady too? Are the fake imports needed? Or can we sweep them away when we sweep away make?
No, Hadrian has other issues but not this one :) Sylvain

On Mon, Mar 15, 2021 at 09:46:35AM +0100, Sylvain Henry wrote:
Thank you! Don’t forget to comment it – especially because it is fake.
Done in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5265
Speaking of build failures with the legacy make system, I see a build
failure on FreeBSD 12.2 with "validate --legacy" that I don't see with
hadrian. It looks like the C compiler flags aren't quite the same and
warnings are more tolerated in the hadrian build.
The issue is that once "PosixSource.h" is included, FreeBSD (rightly I
believe) hides header prototypes of various non-POSIX extensions. In
particular pthread_setname_np(3), is not exposed from

On Mon, Mar 15, 2021 at 06:44:20AM -0400, Viktor Dukhovni wrote:
..., the FreeBSD "validate --legacy" successfully builds GHC. [ The tests seem to all be failing, perhaps the test driver scripts are not portable to FreeBSD, but previously the compiler was not building. ]
FWIW, the tests seem to fail for two reasons: 1. The "install dir" and "test space" directories don't appear to be handled correctly. I had to drop the spaces. 2. On FreeBSD many tests run into the dreaded: unhandled ELF relocation(RelA) type 19 Can anyone versed in Elf internals help with: https://gitlab.haskell.org/ghc/ghc/-/issues/19086 -- Viktor.

Hi Viktor,
- I believe the "test spaces" part is important and would need to be fixed,
if spaces break this is not desirable.
- For the Relocations part, I'm happy to offer guidance and help for anyone
who wants to take a stab at it, right
now I'm not in a position where I could take this on myself, I'm afraid.
Cheers,
Moritz
On Tue, Mar 16, 2021 at 12:29 AM Viktor Dukhovni
On Mon, Mar 15, 2021 at 06:44:20AM -0400, Viktor Dukhovni wrote:
..., the FreeBSD "validate --legacy" successfully builds GHC. [ The tests seem to all be failing, perhaps the test driver scripts are not portable to FreeBSD, but previously the compiler was not building. ]
FWIW, the tests seem to fail for two reasons:
1. The "install dir" and "test space" directories don't appear to be handled correctly. I had to drop the spaces.
2. On FreeBSD many tests run into the dreaded:
unhandled ELF relocation(RelA) type 19
Can anyone versed in Elf internals help with:
https://gitlab.haskell.org/ghc/ghc/-/issues/19086
-- Viktor. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
participants (4)
-
Moritz Angermann
-
Simon Peyton Jones
-
Sylvain Henry
-
Viktor Dukhovni