Hi all, I happened to run ./validate --slow on my linux machine and I thought it would be useful to share the results. Results: Unexpected results from: TEST="EtaExpandLevPoly PatternSplice StrictPats T10508_api T11627b T12809 T12870a T12870b T12870c T12870d T12870e T12870f T12870g T12870h T12903 T12962 T13366 T13543 T13688 T13780c T13822 T13949 T14137 T2552 T2783 T3001-2 T4114c T4114d T4188 T4334 T5129 T5363 T5559 T5611 T680 T7411 T7837 T7944 T8025 T8089 T8542 TH_spliceE5_prof_ext UnsafeReenter compact_gc compact_share dsrun014 dynamic-paper haddock.Cabal hpc_fork prof-doc-fib prof-doc-last profinline001 read029 return_mem_to_os rn041 scc001 scc002 scc003 scc005 space_leak_001 tc165 tryReadMVar2" SUMMARY for test run started at Thu Sep 14 04:57:20 2017 PDT 0:12:27 spent to go through 6091 total tests, which gave rise to 24105 test cases, of which 4531 were skipped 143 had missing libraries 19120 expected passes 203 expected failures 0 caused framework failures 0 caused framework warnings 6 unexpected passes 97 unexpected failures 5 unexpected stat failures Unexpected passes: /tmp/ghctest-e17wqi8b/test spaces/./dependent/should_compile/dynamic-paper.run dynamic-paper [unexpected] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./indexed-types/should_compile/T7837.run T7837 [unexpected] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./parser/should_compile/read029.run read029 [unexpected] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./rename/should_compile/rn041.run rn041 [unexpected] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_fail/T7411.run T7411 [unexpected] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/tc165.run tc165 [unexpected] (optasm) Unexpected failures: /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (dyn) /tmp/ghctest-e17wqi8b/test spaces/./concurrent/should_run/T5611.run T5611 [bad stdout] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./concurrent/should_run/tryReadMVar2.run tryReadMVar2 [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (dyn) /tmp/ghctest-e17wqi8b/test spaces/./dependent/should_fail/T13780c.run T13780c [stderr mismatch] (normal) /tmp/ghctest-e17wqi8b/test spaces/./dependent/should_compile/dynamic-paper.run dynamic-paper [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./driver/T4114d.run T4114d [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./driver/T4114c.run T4114c [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./ghc-api/T10508_api.run T10508_api [bad exit code] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./numeric/should_compile/T8542.run T8542 [stderr mismatch] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./ghc-api/T10508_api.run T10508_api [bad exit code] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./partial-sigs/should_compile/PatternSplice.run PatternSplice [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./patsyn/should_run/T13688.run T13688 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./patsyn/should_run/T13688.run T13688 [exit code non-0] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T3001-2.run T3001-2 [exit code non-0] (prof_hb) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad profile] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad profile] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad profile] (prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad profile] (prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc001.run scc001 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc005.run scc005 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/profinline001.run profinline001 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T5559.run T5559 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hc_hb) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T680.run T680 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc002.run scc002 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/prof-doc-fib.run prof-doc-fib [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hb) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T2552.run T2552 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/prof-doc-last.run prof-doc-last [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hd) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T5363.run T5363 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hy) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hr) /tmp/ghctest-e17wqi8b/test spaces/./rts/T2783.run T2783 [bad exit code] (threaded1) /tmp/ghctest-e17wqi8b/test spaces/./rts/return_mem_to_os.run return_mem_to_os [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/T12903.run T12903 [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870a.run T12870a [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stdout] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870b.run T12870b [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870c.run T12870c [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stdout] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870d.run T12870d [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870b.run T12870b [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870c.run T12870c [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870g.run T12870g [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870h.run T12870h [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870h.run T12870h [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stdout] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stdout] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stdout] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stdout] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T7944.run T7944 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T13543.run T13543 [stderr mismatch] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T13543.run T13543 [stderr mismatch] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T13543.run T13543 [stderr mismatch] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T14137.run T14137 [stderr mismatch] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./th/TH_spliceE5_prof_ext.run TH_spliceE5_prof_ext [bad exit code] (normal) /tmp/ghctest-e17wqi8b/test spaces/./th/T4188.run T4188 [exit code non-0] (normal) /tmp/ghctest-e17wqi8b/test spaces/./th/T4188.run T4188 [exit code non-0] (ext-interp) /tmp/ghctest-e17wqi8b/test spaces/./th/T13366.run T13366 [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./th/should_compile/T8025/T8025.run T8025 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./th/should_compile/T13949/T13949.run T13949 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (normal) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/T12809.run T12809 [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/StrictPats.run StrictPats [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [exit code non-0] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (threaded1) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/hpc/tests/fork/hpc_fork.run hpc_fork [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_share.run compact_share [bad stdout] (profasm) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_share.run compact_share [bad stdout] (ghci) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_gc.run compact_gc [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_share.run compact_share [bad stdout] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./ffi/should_fail/UnsafeReenter.run UnsafeReenter [bad exit code] (threaded1) /tmp/ghctest-e17wqi8b/test spaces/./ffi/should_fail/UnsafeReenter.run UnsafeReenter [bad exit code] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./ffi/should_fail/UnsafeReenter.run UnsafeReenter [bad exit code] (profthreaded) Unexpected stat failures: /tmp/ghctest-e17wqi8b/test spaces/./perf/haddock/haddock.Cabal.run haddock.Cabal [stat not good enough] (normal) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/space_leak_001.run space_leak_001 [stat too good] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/space_leak_001.run space_leak_001 [stat too good] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/T4334.run T4334 [stat not good enough] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/space_leak_001.run space_leak_001 [stat too good] (dyn)
From my point of view, with this amount of failures it's difficult to tell if a new change introduces problems. Shall we mark them as broken and open tickets for follow ups?
Cheers, Bartosz
Wow... 97 unexpected failures is bad.
Ben/David/someone else: might you investigate/characterise them?
Simon
| -----Original Message-----
| From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Bartosz
| Nitka
| Sent: 14 September 2017 13:57
| To: ghc-devs Devs
Simon Peyton Jones via ghc-devs
Wow... 97 unexpected failures is bad.
Ben/David/someone else: might you investigate/characterise them?
I periodically run --slow and have opened a number of tickets in response in the past (#11819 being the one I was able to easily find). Bartosz, perhaps you could add a comment to that ticket with your results? David, do you think you could take a few hours to go through the failing tests and classify them thematically? Cheers, - Ben
participants (3)
-
Bartosz Nitka -
Ben Gamari -
Simon Peyton Jones