[GHC] #9771: Excessive memory usage compiling T3064
#9771: Excessive memory usage compiling T3064 -------------------------------------+------------------------------------- Reporter: hvr | Owner: simonpj Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: Compiler | Version: 7.9 Keywords: | Operating System: Architecture: Unknown/Multiple | Unknown/Multiple Difficulty: Unknown | Type of failure: Compile- Blocked By: | time performance bug Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- With 64dc4d1085a7db375f6532362bf7e23fac3a25eb the testsuite suffers from T3064 allocating an excessive amount of memory (several GiBs) and finally getting killed due to timeout and/or out-of-memory. {{{ =====> T3064(normal) 3054 of 4150 [0, 0, 0] cd ./perf/compiler && '/home/hvr/ghc/inplace/bin/ghc-stage2' -fforce- recomp -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno- ghci-history -c T3064.hs +RTS -V0 -tT3064.comp.stats --machine-readable -RTS >T3064.comp.stderr 2>&1 Compile failed (status 35072) errors were: Killed Failed to find field: peak_megabytes_allocated *** framework failure for T3064(normal) do_test exception Traceback (most recent call last): File "/home/hvr/ghc/testsuite/driver/testlib.py", line 793, in do_test result = func(*[name,way] + args) File "/home/hvr/ghc/testsuite/driver/testlib.py", line 995, in compile return do_compile( name, way, 0, '', [], extra_hc_opts ) File "/home/hvr/ghc/testsuite/driver/testlib.py", line 1024, in do_compile result = simple_build( name, way, extra_hc_opts, should_fail, top_mod, 0, 1, force, override_flags ) File "/home/hvr/ghc/testsuite/driver/testlib.py", line 1262, in simple_build statsResult = checkStats(name, way, stats_file, opts.compiler_stats_range_fields) File "/home/hvr/ghc/testsuite/driver/testlib.py", line 1136, in checkStats val = int(m.group(1)) AttributeError: 'NoneType' object has no attribute 'group' OVERALL SUMMARY for test run started at Wed Nov 5 08:29:10 2014 CET 0:04:14 spent to go through 4150 total tests, which gave rise to 16187 test cases, of which 16186 were skipped 0 had missing libraries 0 expected passes 0 expected failures 1 caused framework failures 0 unexpected passes 0 unexpected failures 0 unexpected stat failures make[1]: Leaving directory '/home/hvr/ghc/testsuite/tests' real 4m13.317s user 3m49.476s sys 0m5.831s }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9771> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#9771: Excessive memory usage compiling T3064 -------------------------------------+------------------------------------- Reporter: hvr | Owner: simonpj Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by Herbert Valerio Riedel <hvr@…>): In [changeset:"77f8221103a98b38384edd8c2caae6cc2c4ffd72/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="77f8221103a98b38384edd8c2caae6cc2c4ffd72" Temporarily disable T3064 (see #9771) This disables T3064 temporarily as it puts a strain on buildbots during validation exhausting all available memory. }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9771#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#9771: Excessive memory usage compiling T3064 -------------------------------------+------------------------------------- Reporter: hvr | Owner: simonpj Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Thanks Herbert. I'll look into this. There really ought to be a way to ensure that tests don't go AWOL and eat all resources, though. Simon -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9771#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#9771: Excessive memory usage compiling T3064 -------------------------------------+------------------------------------- Reporter: hvr | Owner: simonpj Type: bug | Status: new Priority: highest | Milestone: 7.10.1 Component: Compiler | Version: 7.9 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by hvr): Replying to [comment:2 simonpj]:
There really ought to be a way to ensure that tests don't go AWOL and eat all resources, though.
Well, one could either pass some +RTS options to ghc to limit its heap via `-M<size>` (which otoh distorts GHC's reported memory stats, and relies on the RTS to correctly enforce that limit), or (at least on Linux, so I'm not sure what to do for Windows buildbots) you can use `ulimit` to impose a memory allocation limit for a process (and its children). -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9771#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#9771: Excessive memory usage compiling T3064 -------------------------------------+------------------------------------- Reporter: hvr | Owner: simonpj Type: bug | Status: closed Priority: highest | Milestone: 7.10.1 Component: Compiler | Version: 7.9 Resolution: fixed | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by simonpj): * status: new => closed * resolution: => fixed Comment: I believe I have fixed the long compile time on T3064. I've re-enabled: {{{ commit c79cbacb6d9161c529ac13685ff29ac058a3ebfa Author: Simon Peyton Jones <simonpj@microsoft.com> Date: Thu Nov 6 15:46:11 2014 +0000 Re-enable T3064, which works now testsuite/tests/perf/compiler/all.T | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9771#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC