[GHC] #10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes)
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: | Owner: RyanGlScott | Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Keywords: | Operating System: Linux Architecture: x86_64 | Type of failure: Building GHC (amd64) | failed Test Case: | Blocked By: Blocking: | Related Tickets: #10682 Differential Revisions: | -------------------------------------+------------------------------------- Attempting to build GHC on an x86_64 Linux machine fails for me: {{{ $ make ===--- building phase 0 make --no-print-directory -f ghc.mk phase=0 phase_0_builds make[1]: Nothing to be done for `phase_0_builds'. ===--- building phase 1 make --no-print-directory -f ghc.mk phase=1 phase_1_builds make[1]: Nothing to be done for `phase_1_builds'. ===--- building final phase make --no-print-directory -f ghc.mk phase=final all inplace/bin/dll-split compiler/stage2/build/.depend-v-dyn.haskell "DynFlags" "Annotations ApiAnnotation Avail Bag BasicTypes Binary BooleanFormula BreakArray BufWrite Class CmdLineParser CmmType CoAxiom ConLike Coercion Config Constants CoreArity CoreFVs CoreSubst CoreSyn CoreTidy CoreUnfold CoreUtils CoreSeq CoreStats CostCentre Ctype DataCon Demand Digraph DriverPhases DynFlags Encoding ErrUtils Exception FamInstEnv FastFunctions FastMutInt FastString Fingerprint FiniteMap ForeignCall Hooks HsBinds HsDecls HsDoc HsExpr HsImpExp HsLit PlaceHolder HsPat HsSyn HsTypes HsUtils HscTypes IOEnv Id IdInfo IfaceSyn IfaceType InstEnv Kind Lexeme Lexer ListSetOps Literal Maybes MkCore MkId Module MonadUtils Name NameEnv NameSet OccName OccurAnal OptCoercion OrdList Outputable PackageConfig Packages Pair Panic PatSyn PipelineMonad Platform PlatformConstants PprCore PrelNames PrelRules Pretty PrimOp RdrName Rules Serialized SrcLoc StaticFlags StringBuffer TcEvidence TcRnTypes TcType TrieMap TyCon Type TypeRep TysPrim TysWiredIn Unify UniqFM UniqSet UniqSupply Unique Util Var VarEnv VarSet Bitmap BlockId ByteCodeAsm ByteCodeInstr ByteCodeItbls CLabel Cmm CmmCallConv CmmExpr CmmInfo CmmMachOp CmmNode CmmSwitch CmmUtils CodeGen.Platform CodeGen.Platform.ARM CodeGen.Platform.ARM64 CodeGen.Platform.NoRegs CodeGen.Platform.PPC CodeGen.Platform.PPC_Darwin CodeGen.Platform.SPARC CodeGen.Platform.X86 CodeGen.Platform.X86_64 Hoopl Hoopl.Dataflow InteractiveEvalTypes MkGraph PprCmm PprCmmDecl PprCmmExpr Reg RegClass SMRep StgCmmArgRep StgCmmClosure StgCmmEnv StgCmmLayout StgCmmMonad StgCmmProf StgCmmTicky StgCmmUtils StgSyn Stream" dll-split: out of memory (requested 1099512676352 bytes) make[1]: *** [compiler/stage2/dll-split.stamp] Error 1 make: *** [all] Error 2 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): That's odd. Works fine for me, and Phabricator is also x86_64. * Did you try 'make maintainer-clean && ./boot && ./configure && make'? * Maybe it is something specific to dll-split. You could try to apply the patch from Phab:D677. I don't think the patch will apply cleanly, but it's a small patch, so just try to fix the merge conflict. Only applying the change in DynFlags might also be enough. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by RyanGlScott): I agree that this is odd, since the build does work for me on other x86_64 machines. I'm not sure what's different about this one. `make maintainer-clean && ./boot && ./configure && make` doesn't work. I tried manually applying the changes from Phab:D677 and running that, but unfortunately, a similar issue creeps up: {{{ [rgscott@tank ghc]$ make ===--- building phase 0 make --no-print-directory -f ghc.mk phase=0 phase_0_builds make[1]: Nothing to be done for `phase_0_builds'. ===--- building phase 1 make --no-print-directory -f ghc.mk phase=1 phase_1_builds make[1]: Nothing to be done for `phase_1_builds'. ===--- building final phase make --no-print-directory -f ghc.mk phase=final all "inplace/bin/ghc-stage2" -hisuf dyn_hi -osuf dyn_o -hcsuf dyn_hc -fPIC -dynamic -O0 -H64m -hide-all-packages -i -iutils/ghctags/. -iutils/ghctags/dist-install/build -iutils/ghctags/dist- install/build/autogen -Iutils/ghctags/dist-install/build -Iutils/ghctags /dist-install/build/autogen -optP-include -optPutils/ghctags/dist- install/build/autogen/cabal_macros.h -package-id Cabal-1.23.0.0-inplace -package-id base-4.8.2.0-inplace -package-id containers-0.5.6.2-inplace -package-id ghc-7.11.20150916-inplace -XHaskell2010 -no-user-package-db -rtsopts -odir utils/ghctags/dist-install/build -hidir utils/ghctags /dist-install/build -stubdir utils/ghctags/dist-install/build -c utils/ghctags/./Main.hs -o utils/ghctags/dist-install/build/Main.dyn_o ghc-stage2: out of memory (requested 1099512676352 bytes) make[1]: *** [utils/ghctags/dist-install/build/Main.dyn_o] Error 1 make: *** [all] Error 2 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): Well it made it to ghc-stage2 at least. Could you try running the testsuite (run 'make test'). Do all tests fail, or just a few? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by RyanGlScott): All tests fail—I get the same out of memory error from `ghc-stage2`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thomie): * priority: normal => highest * cc: gcampax (added) * milestone: => 8.0.1 Comment: Well I have no idea, but that's a release blocker. Maybe @gcampax can debug this, as the co-author of 0d1a8d09f452977aadef7897aa12a8d41c7a4af0. You can use `./configure --disable-large-address-space` in the meantime. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): Presumably it is something special about the reporter's system. Something like overcommit being disabled (though in my preliminary tests this did not cause reserving large memory areas to fail). What distro is it? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by RyanGlScott): Tell me if you want more specific distro info than this: {{{ $ uname -a Linux tank.soic.indiana.edu 3.10.0-229.11.1.el7.x86_64 #1 SMP Wed Jul 22 12:06:11 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux $ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.1 (Maipo) $ cat /proc/meminfo MemTotal: 263928412 kB MemFree: 52989964 kB MemAvailable: 167374740 kB Buffers: 3540 kB Cached: 109364396 kB SwapCached: 52700 kB Active: 143228364 kB Inactive: 59588436 kB Active(anon): 87890144 kB Inactive(anon): 5694784 kB Active(file): 55338220 kB Inactive(file): 53893652 kB Unevictable: 32 kB Mlocked: 32 kB SwapTotal: 134217724 kB SwapFree: 134030536 kB Dirty: 28 kB Writeback: 0 kB AnonPages: 93435332 kB Mapped: 203232 kB Shmem: 136064 kB Slab: 5871708 kB SReclaimable: 5490788 kB SUnreclaim: 380920 kB KernelStack: 18336 kB PageTables: 230584 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 266181928 kB Committed_AS: 101990088 kB VmallocTotal: 34359738367 kB VmallocUsed: 720588 kB VmallocChunk: 34224804536 kB HardwareCorrupted: 0 kB AnonHugePages: 92712960 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 310208 kB DirectMap2M: 59412480 kB DirectMap1G: 210763776 kB }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by gcampax): Is it possible that you have an address space limit configured (ulimit -v or /etc/security/limits.conf )? If so, is it ok to disable it to run any Haskell code, or is it a policy to have it? Unfortunately the large address space support fails if the address space limit is low. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by RyanGlScott): Yes, it does appear that I have an address space limit: {{{ $ ulimit -v 67108864 }}} This isn't a machine I have root access on, so I don't believe this can be changed easily. Is 67.1 GB of address space not enough? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by bgamari): * cc: simonmar (added) Comment: simonmar, perhaps we should be looking at the ulimit and adjust our allocation request downwards if necessary? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonmar): Urgh, the problem is that making the size of the area dynamic imposes a runtime cost - we have to read that value every time the GC visits an object. But I can't think of an alternative. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by bgamari): Presumably it will sit in cache in this case, no? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: patch Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Rev(s): Phab:D1354 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * differential: => Phab:D1354 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: patch Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Rev(s): Phab:D1405 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * differential: Phab:D1354 => Phab:D1405 Comment: `getrlimit` and friends seem to be a bit finicky. The new approach (Phab:D1405) is just to successively reduce the allocation size until we find a size that the kernel will give us. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes)
-------------------------------------+-------------------------------------
Reporter: RyanGlScott | Owner:
Type: bug | Status: patch
Priority: highest | Milestone: 8.0.1
Component: Compiler | Version: 7.11
Resolution: | Keywords:
Operating System: Linux | Architecture: x86_64
Type of failure: Building GHC | (amd64)
failed | Test Case:
Blocked By: | Blocking:
Related Tickets: #10682 | Differential Rev(s): Phab:D1405
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes)
-------------------------------------+-------------------------------------
Reporter: RyanGlScott | Owner:
Type: bug | Status: patch
Priority: highest | Milestone: 8.0.1
Component: Compiler | Version: 7.11
Resolution: | Keywords:
Operating System: Linux | Architecture: x86_64
Type of failure: Building GHC | (amd64)
failed | Test Case:
Blocked By: | Blocking:
Related Tickets: #10682 | Differential Rev(s): Phab:D1405
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#10877: x86_64: dll-split: out of memory (requested 1099512676352 bytes) -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: closed Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.11 Resolution: fixed | Keywords: Operating System: Linux | Architecture: x86_64 Type of failure: Building GHC | (amd64) failed | Test Case: Blocked By: | Blocking: Related Tickets: #10682 | Differential Rev(s): Phab:D1405 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10877#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC