Re: [GHC] #9389: Full Test Suite Failures

22 Jan 2015

      #9389: Full Test Suite Failures
-------------------------------------+-------------------------------------
        Reporter:  jrp               |                   Owner:
            Type:  bug               |                  Status:  new
        Priority:  normal            |               Milestone:
       Component:  Test Suite        |                 Version:  7.8.3
      Resolution:                    |                Keywords:
Operating System:  MacOS X           |            Architecture:
 Type of failure:  Other             |  Unknown/Multiple
      Blocked By:                    |               Test Case:
 Related Tickets:                    |                Blocking:
  #9286,#8756,#7876,#7877            |  Differential Revisions:
-------------------------------------+-------------------------------------

Comment (by jrp):

 Today's results ({{{perf}}} BuildFlavour):

 {{{
 OVERALL SUMMARY for test run started at Thu Jan 22 19:36:52 2015 GMT
  0:34:34 spent to go through
     4405 total tests, which gave rise to
    18011 test cases, of which
     3603 were skipped

      328 had missing libraries
    13819 expected passes
      139 expected failures

        0 caused framework failures
        0 unexpected passes
      116 unexpected failures
        6 unexpected stat failures

 Unexpected failures:
    ../../libraries/base/tests     enum01 [exit code non-0]
 (normal,hpc,optasm,profasm,threaded1,threaded2,dyn,profthreaded)
    ../../libraries/base/tests     enum01 [bad stdout or stderr] (ghci)
    ../../libraries/base/tests     enum02 [exit code non-0]
 (normal,hpc,optasm,profasm,threaded1,threaded2,dyn,profthreaded)
    ../../libraries/base/tests     enum02 [bad stdout or stderr] (ghci)
    ../../libraries/base/tests     enum03 [exit code non-0]
 (normal,hpc,optasm,profasm,threaded1,threaded2,dyn,profthreaded)
    ../../libraries/base/tests     enum03 [bad stdout or stderr] (ghci)
    ../../libraries/base/tests     topHandler01 [bad exit code] (threaded1)
    ../../libraries/base/tests     topHandler01 [bad stdout or stderr]
 (ghci)
    ../../libraries/process/tests  process011 [bad exit code]
 (threaded1,threaded2,profthreaded)
    ../../libraries/process/tests  process011 [bad stdout or stderr] (ghci)
    ../../libraries/unix/tests     signals002 [bad stdout]
 (threaded1,threaded2,profthreaded)
    ../../libraries/unix/tests     signals002 [bad stdout or stderr] (ghci)
    annotations/should_run         annrun01 [bad stdout or stderr] (ghci)
    cabal/sigcabal01               sigcabal01 [bad stderr] (normal)
    codeGen/should_run             CgStaticPointers [bad stdout or stderr]
 (ghci)
    concurrent/should_run          T5611 [bad stdout] (profasm)
    concurrent/should_run          allocLimit4 [bad exit code] (ghci)
    concurrent/should_run          conc012 [bad exit code] (ghci)
    concurrent/should_run          conc059 [bad stderr]
 (threaded1,threaded2)
    deSugar/should_run             DsStaticPointers [bad stdout or stderr]
 (ghci)
    deSugar/should_run             dsrun014 [bad exit code] (ghci)
    driver/objc                    objc-hi [bad profile] (profthreaded)
    driver/objc                    objc-hi [bad heap profile] (profasm)
    driver/objc                    objcpp-hi [bad profile] (profthreaded)
    driver/objc                    objcpp-hi [bad heap profile] (profasm)
    driver/retc001                 retc001 [bad stderr] (normal)
    ghci/scripts                   ghci024 [bad stdout] (normal)
    numeric/should_run             add2 [bad exit code] (ghci)
    numeric/should_run             mul2 [bad exit code] (ghci)
    numeric/should_run             quotRem2 [bad exit code] (ghci)
    parser/should_run              readRun004 [bad exit code] (ghci)
    profiling/should_run           T2552 [bad profile]
 (profasm,profthreaded)
    profiling/should_run           callstack001 [bad stdout] (prof)
    profiling/should_run           ioprof [bad profile]
 (profasm,profthreaded)
    profiling/should_run           prof-doc-fib [bad profile]
 (profasm,profthreaded,prof)
    profiling/should_run           scc001 [bad profile] (prof)
    quasiquotation/qq007           qq007 [exit code non-0] (profasm)
    quasiquotation/qq008           qq008 [exit code non-0] (profasm)
    rts                            GcStaticPointers [bad stdout or stderr]
 (ghci)
    rts                            ListStaticPointers [bad stdout or
 stderr] (ghci)
    rts                            T7919 [bad exit code] (optasm,dyn)
    rts                            T7919 [exit code non-0]
 (profasm,profthreaded)
    rts                            T9078 [exit code non-0] (profasm)
    rts                            derefnull [bad profile]
 (profasm,profthreaded)
    rts                            divbyzero [bad profile]
 (profasm,profthreaded)
    rts                            linker_unload [bad exit code] (normal)
    rts                            overflow1 [bad profile]
 (profasm,profthreaded)
    rts                            overflow2 [bad profile]
 (profasm,profthreaded)
    rts                            overflow3 [bad profile]
 (profasm,profthreaded)
    simplCore/should_compile       T5550 [exit code non-0] (profasm)
    th                             TH_StaticPointers [bad stdout or stderr]
 (ghci)
    typecheck/should_run           T3500a [exit code non-0]
 (normal,hpc,optasm,profasm,threaded1,threaded2,dyn,profthreaded)
    typecheck/should_run           T3500a [bad stdout or stderr] (ghci)
    typecheck/should_run           T5751 [exit code non-0]
 (normal,hpc,optasm,profasm,threaded1,threaded2,dyn,profthreaded)
    typecheck/should_run           T5751 [bad stdout or stderr] (ghci)
    typecheck/should_run           T7126 [exit code non-0]
 (normal,hpc,optasm,profasm,threaded1,threaded2,dyn,profthreaded)
    typecheck/should_run           T7126 [bad stdout or stderr] (ghci)
    typecheck/should_run           tcrun045 [stderr mismatch] (normal)

 Unexpected stat failures:
    perf/compiler             T1969 [stat not good enough] (normal)
    perf/compiler             T5642 [stat not good enough] (normal)
    perf/compiler             T783 [stat not good enough] (normal)
    perf/compiler             T9675 [stat not good enough] (optasm)
    perf/should_run           T9203 [stat too good] (normal)
    simplCore/should_compile  T7702 [stat too good] (profasm)
 }}}

 and {{{validate --slow}}} gives:

 {{{
 OVERALL SUMMARY for test run started at Thu Jan 22 21:16:02 2015 GMT
  0:25:08 spent to go through
     4405 total tests, which gave rise to
    14234 test cases, of which
     2726 were skipped

      267 had missing libraries
    11018 expected passes
      149 expected failures

        0 caused framework failures
        0 unexpected passes
       74 unexpected failures
        0 unexpected stat failures

 Unexpected failures:
    ../../libraries/base/tests     enum01 [exit code non-0]
 (normal,hpc,optasm,threaded1,threaded2,dyn)
    ../../libraries/base/tests     enum01 [bad stdout or stderr] (ghci)
    ../../libraries/base/tests     enum02 [exit code non-0]
 (normal,hpc,optasm,threaded1,threaded2,dyn)
    ../../libraries/base/tests     enum02 [bad stdout or stderr] (ghci)
    ../../libraries/base/tests     enum03 [exit code non-0]
 (normal,hpc,optasm,threaded1,threaded2,dyn)
    ../../libraries/base/tests     enum03 [bad stdout or stderr] (ghci)
    ../../libraries/base/tests     topHandler01 [bad exit code] (threaded1)
    ../../libraries/base/tests     topHandler01 [bad stdout or stderr]
 (ghci)
    ../../libraries/base/tests     topHandler03 [bad exit code] (ghci)
    ../../libraries/process/tests  process011 [bad exit code]
 (threaded1,threaded2)
    ../../libraries/process/tests  process011 [bad stdout or stderr] (ghci)
    ../../libraries/unix/tests     signals002 [bad stdout]
 (threaded1,threaded2)
    ../../libraries/unix/tests     signals002 [bad stdout or stderr] (ghci)
    cabal/sigcabal01               sigcabal01 [bad stderr] (normal)
    codeGen/should_run             CgStaticPointers [bad stdout or stderr]
 (ghci)
    concurrent/should_run          allocLimit4 [bad exit code] (ghci)
    concurrent/should_run          conc012 [bad exit code] (ghci)
    concurrent/should_run          conc059 [bad stderr]
 (threaded1,threaded2)
    deSugar/should_run             DsStaticPointers [bad stdout or stderr]
 (ghci)
    deSugar/should_run             dsrun014 [bad exit code] (ghci)
    driver/retc001                 retc001 [bad exit code] (normal)
    driver/retc002                 retc002 [bad exit code] (normal)
    ghci/scripts                   ghci024 [bad stdout] (normal)
    numeric/should_run             add2 [bad exit code] (ghci)
    numeric/should_run             mul2 [bad exit code] (ghci)
    numeric/should_run             quotRem2 [bad exit code] (ghci)
    parser/should_run              readRun004 [bad exit code] (ghci)
    rts                            GcStaticPointers [bad stdout or stderr]
 (ghci)
    rts                            ListStaticPointers [bad stdout or
 stderr] (ghci)
    rts                            T7919 [bad exit code]
 (optasm,threaded2,dyn)
    rts                            linker_unload [bad exit code] (normal)
    th                             TH_StaticPointers [bad stdout or stderr]
 (ghci)
    typecheck/should_run           T3500a [exit code non-0]
 (normal,hpc,optasm,threaded1,threaded2,dyn)
    typecheck/should_run           T3500a [bad stdout or stderr] (ghci)
    typecheck/should_run           T5751 [exit code non-0]
 (normal,hpc,optasm,threaded1,threaded2,dyn)
    typecheck/should_run           T5751 [bad stdout or stderr] (ghci)
    typecheck/should_run           T7126 [exit code non-0]
 (normal,hpc,optasm,threaded1,threaded2,dyn)
    typecheck/should_run           T7126 [bad stdout or stderr] (ghci)
    typecheck/should_run           tcrun045 [stderr mismatch] (normal)
 }}}

 whereas {{{validate --fast}}} paints a rosier picture:

 {{{
 OVERALL SUMMARY for test run started at Thu Jan 22 22:10:03 2015 GMT
  0:06:07 spent to go through
     4405 total tests, which gave rise to
    14230 test cases, of which
    10172 were skipped

       52 had missing libraries
     3932 expected passes
       71 expected failures

        0 caused framework failures
        0 unexpected passes
        3 unexpected failures
        0 unexpected stat failures

 Unexpected failures:
    cabal/sigcabal01  sigcabal01 [bad stderr] (normal)
    driver/retc001    retc001 [bad stderr] (normal)
    rts               linker_unload [bad exit code] (normal)
 }}}

--
Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9389#comment:21
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler