Re: [GHC] #15736: Testsuite failures from validate --slow

22 Nov 2018

      #15736: Testsuite failures from validate --slow
-------------------------------------+-------------------------------------
        Reporter:  jrp               |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Test Suite        |              Version:  8.6.1
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
                                     |  (amd64)
 Type of failure:  Incorrect result  |            Test Case:
  at runtime                         |  EtaExpandLevPoly T14936 T15349
                                     |  T2783 T4334 T7919 haddock.Cabal
                                     |  haddock.base haddock.compiler
                                     |  hpc_fork plugin-recomp-change
                                     |  plugin-recomp-change-prof recomp007
                                     |  space_leak_001
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by jrp):

 ... and just running {{{make test TEST=...}}}

 {{{
 =====> recomp007(normal) 1 of 11 [0, 0, 0]
 cd "driver/recomp007/recomp007.run" && $MAKE -s --no-print-directory
 recomp007
 Actual stderr output differs from expected:
 diff -uw "/dev/null"
 "driver/recomp007/recomp007.run/recomp007.run.stderr.normalised"
 --- /dev/null   2018-11-22 18:53:00.700826500 +0000
 +++ driver/recomp007/recomp007.run/recomp007.run.stderr.normalised
 2018-11-22 21:24:45.101279549 +0000
 @@ -0,0 +1,122 @@
 +WARNING: file compiler/utils/ListSetOps.hs, line 58
 +  []
 +  [Distribution.Backpack, Distribution.Backpack.FullUnitId,
 +   Distribution.Backpack.ModuleShape,
 +   Distribution.Backpack.PreModuleShape,
 +   Distribution.Backpack.ReadyComponent,
 +   Distribution.CabalSpecVersion, Distribution.Compat.Graph,
 +   Distribution.Compat.Semigroup, Distribution.Compiler,
 +   Distribution.License, Distribution.ModuleName,
 +   Distribution.Parsec.Common, Distribution.SPDX.License,
 +   Distribution.SPDX.LicenseExceptionId,
 +   Distribution.SPDX.LicenseExpression, Distribution.SPDX.LicenseId,
 +   Distribution.SPDX.LicenseReference,
 +   Distribution.Simple.BuildTarget, Distribution.Simple.Compiler,
 +   Distribution.Simple.Doctest, Distribution.Simple.Flag,
 +   Distribution.Simple.Haddock, Distribution.Simple.InstallDirs,
 +   Distribution.Simple.PackageIndex, Distribution.Simple.Program.Find,
 +   Distribution.Simple.Program.GHC, Distribution.Simple.Program.Types,
 +   Distribution.Simple.Setup, Distribution.System,
 +   Distribution.Types.AbiDependency, Distribution.Types.AbiHash,
 +   Distribution.Types.Benchmark,
 +   Distribution.Types.BenchmarkInterface,
 +   Distribution.Types.BenchmarkType, Distribution.Types.BuildInfo,
 +   Distribution.Types.BuildType, Distribution.Types.ComponentId,
 +   Distribution.Types.ComponentLocalBuildInfo,
 +   Distribution.Types.ComponentName,
 +   Distribution.Types.ComponentRequestedSpec,
 +   Distribution.Types.CondTree, Distribution.Types.Condition,
 +   Distribution.Types.Dependency, Distribution.Types.ExeDependency,
 +   Distribution.Types.Executable, Distribution.Types.ExecutableScope,
 +   Distribution.Types.ExposedModule, Distribution.Types.ForeignLib,
 +   Distribution.Types.ForeignLibOption,
 +   Distribution.Types.ForeignLibType,
 +   Distribution.Types.GenericPackageDescription,
 +   Distribution.Types.IncludeRenaming,
 +   Distribution.Types.InstalledPackageInfo,
 +   Distribution.Types.LegacyExeDependency, Distribution.Types.Library,
 +   Distribution.Types.LocalBuildInfo, Distribution.Types.Mixin,
 +   Distribution.Types.Module, Distribution.Types.ModuleReexport,
 +   Distribution.Types.ModuleRenaming,
 +   Distribution.Types.MungedPackageId,
 +   Distribution.Types.MungedPackageName,
 +   Distribution.Types.PackageDescription,
 +   Distribution.Types.PackageId, Distribution.Types.PackageName,
 +   Distribution.Types.PkgconfigDependency,
 +   Distribution.Types.PkgconfigName,
 +   Distribution.Types.SetupBuildInfo, Distribution.Types.SourceRepo,
 +   Distribution.Types.TargetInfo, Distribution.Types.TestSuite,
 +   Distribution.Types.TestSuiteInterface, Distribution.Types.TestType,
 +   Distribution.Types.UnitId, Distribution.Types.UnqualComponentName,
 +   Distribution.Types.Version, Distribution.Types.VersionRange,
 +   Distribution.Utils.ShortText, Distribution.Verbosity,
 +   Language.Haskell.Extension, Control.Applicative, Data.Complex,
 +   Data.Functor.Compose, Data.Functor.Const, Data.Functor.Identity,
 +   Data.Functor.Product, Data.Functor.Sum, Data.Monoid,
 +   Data.Semigroup, Data.Semigroup.Internal, Data.Version, Data.Void,
 +   GHC.Exts, GHC.Generics, GHC.IO.Exception, Data.Graph,
 +   Data.IntMap.Internal, Data.IntSet.Internal, Data.Map.Internal,
 +   Data.Sequence.Internal, Data.Set.Internal, Data.Tree,
 +   Text.PrettyPrint.Annotated.HughesPJ, Text.PrettyPrint.HughesPJ,
 +   Data.Text, Data.Text.Lazy]
 +WARNING: file compiler/utils/ListSetOps.hs, line 58
 +  [Distribution.Backpack, Distribution.Backpack.FullUnitId,
 +   Distribution.Backpack.ModuleShape,
 +   Distribution.Backpack.PreModuleShape,
 +   Distribution.Backpack.ReadyComponent,
 +   Distribution.CabalSpecVersion, Distribution.Compat.Graph,
 +   Distribution.Compat.Semigroup, Distribution.Compiler,
 +   Distribution.License, Distribution.ModuleName,
 +   Distribution.Parsec.Common, Distribution.SPDX.License,
 +   Distribution.SPDX.LicenseExceptionId,
 +   Distribution.SPDX.LicenseExpression, Distribution.SPDX.LicenseId,
 +   Distribution.SPDX.LicenseReference,
 +   Distribution.Simple.BuildTarget, Distribution.Simple.Compiler,
 +   Distribution.Simple.Doctest, Distribution.Simple.Flag,
 +   Distribution.Simple.Haddock, Distribution.Simple.InstallDirs,
 +   Distribution.Simple.PackageIndex, Distribution.Simple.Program.Find,
 +   Distribution.Simple.Program.GHC, Distribution.Simple.Program.Types,
 +   Distribution.Simple.Setup, Distribution.System,
 +   Distribution.Types.AbiDependency, Distribution.Types.AbiHash,
 +   Distribution.Types.Benchmark,
 +   Distribution.Types.BenchmarkInterface,
 +   Distribution.Types.BenchmarkType, Distribution.Types.BuildInfo,
 +   Distribution.Types.BuildType, Distribution.Types.ComponentId,
 +   Distribution.Types.ComponentLocalBuildInfo,
 +   Distribution.Types.ComponentName,
 +   Distribution.Types.ComponentRequestedSpec,
 +   Distribution.Types.CondTree, Distribution.Types.Condition,
 +   Distribution.Types.Dependency, Distribution.Types.ExeDependency,
 +   Distribution.Types.Executable, Distribution.Types.ExecutableScope,
 +   Distribution.Types.ExposedModule, Distribution.Types.ForeignLib,
 +   Distribution.Types.ForeignLibOption,
 +   Distribution.Types.ForeignLibType,
 +   Distribution.Types.GenericPackageDescription,
 +   Distribution.Types.IncludeRenaming,
 +   Distribution.Types.InstalledPackageInfo,
 +   Distribution.Types.LegacyExeDependency, Distribution.Types.Library,
 +   Distribution.Types.LocalBuildInfo, Distribution.Types.Mixin,
 +   Distribution.Types.Module, Distribution.Types.ModuleReexport,
 +   Distribution.Types.ModuleRenaming,
 +   Distribution.Types.MungedPackageId,
 +   Distribution.Types.MungedPackageName,
 +   Distribution.Types.PackageDescription,
 +   Distribution.Types.PackageId, Distribution.Types.PackageName,
 +   Distribution.Types.PkgconfigDependency,
 +   Distribution.Types.PkgconfigName,
 +   Distribution.Types.SetupBuildInfo, Distribution.Types.SourceRepo,
 +   Distribution.Types.TargetInfo, Distribution.Types.TestSuite,
 +   Distribution.Types.TestSuiteInterface, Distribution.Types.TestType,
 +   Distribution.Types.UnitId, Distribution.Types.UnqualComponentName,
 +   Distribution.Types.Version, Distribution.Types.VersionRange,
 +   Distribution.Utils.ShortText, Distribution.Verbosity,
 +   Language.Haskell.Extension, Control.Applicative, Data.Complex,
 +   Data.Functor.Compose, Data.Functor.Const, Data.Functor.Identity,
 +   Data.Functor.Product, Data.Functor.Sum, Data.Monoid,
 +   Data.Semigroup, Data.Semigroup.Internal, Data.Version, Data.Void,
 +   GHC.Exts, GHC.Generics, GHC.IO.Exception, Data.Graph,
 +   Data.IntMap.Internal, Data.IntSet.Internal, Data.Map.Internal,
 +   Data.Sequence.Internal, Data.Set.Internal, Data.Tree,
 +   Text.PrettyPrint.Annotated.HughesPJ, Text.PrettyPrint.HughesPJ,
 +   Data.Text, Data.Text.Lazy]
 +  []
 *** unexpected failure for recomp007(normal)
 =====> T15904(normal) 2 of 11 [0, 1, 0]
 cd "hp2ps/T15904.run" && $MAKE -s --no-print-directory T15904
 Actual stdout output differs from expected:
 diff -uw "hp2ps/T15904.run/T15904.stdout.normalised"
 "hp2ps/T15904.run/T15904.run.stdout.normalised"
 --- hp2ps/T15904.run/T15904.stdout.normalised   2018-11-22
 21:24:46.193284051 +0000
 +++ hp2ps/T15904.run/T15904.run.stdout.normalised       2018-11-22
 21:24:46.193284051 +0000
 @@ -1,6 +1,5 @@
  [1 of 1] Compiling T15904           ( T15904.hs, T15904.o )
  Linking "T15904" ...
  {"e": 2.72, "pi": 3.14}
 -
 -
 +$\n
  \
 *** unexpected failure for T15904(normal)
 =====> ArithInt16(normal) 3 of 11 [0, 2, 0]
 cd "primops/should_run/ArithInt16.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o ArithInt16
 ArithInt16.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 cd "primops/should_run/ArithInt16.run" && ./ArithInt16
 =====> ArithWord16(normal) 4 of 11 [0, 2, 0]
 cd "primops/should_run/ArithWord16.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o ArithWord16
 ArithWord16.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 cd "primops/should_run/ArithWord16.run" && ./ArithWord16
 =====> heapprof001(profasm) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -O -prof -static -fprof-auto
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -hc -p
 -RTS 7
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/hp2ps" heapprof001
 "gs" -dNODISPLAY -dBATCH -dQUIET -dNOPAUSE
 "profiling/should_run/heapprof001.run/heapprof001.ps"
 =====> heapprof001(prof) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -prof -static -fprof-auto -fasm
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -p -RTS 7
 =====> heapprof001(prof_hc_hb) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -O -prof -static -fprof-auto
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -hc
 -hbvoid -RTS 7
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/hp2ps" heapprof001
 "gs" -dNODISPLAY -dBATCH -dQUIET -dNOPAUSE
 "profiling/should_run/heapprof001.run/heapprof001.ps"
 =====> heapprof001(prof_hb) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -O -prof -static -fprof-auto
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -hb -RTS 7
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/hp2ps" heapprof001
 "gs" -dNODISPLAY -dBATCH -dQUIET -dNOPAUSE
 "profiling/should_run/heapprof001.run/heapprof001.ps"
 =====> heapprof001(prof_hd) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -O -prof -static -fprof-auto
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -hd -RTS 7
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/hp2ps" heapprof001
 "gs" -dNODISPLAY -dBATCH -dQUIET -dNOPAUSE
 "profiling/should_run/heapprof001.run/heapprof001.ps"
 =====> heapprof001(prof_hy) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -O -prof -static -fprof-auto
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -hy -RTS 7
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/hp2ps" heapprof001
 "gs" -dNODISPLAY -dBATCH -dQUIET -dNOPAUSE
 "profiling/should_run/heapprof001.run/heapprof001.ps"
 =====> heapprof001(prof_hr) 5 of 11 [0, 2, 0]
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o heapprof001
 heapprof001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-
 warn-missed-specialisations -fshow-warning-groups -fdiagnostics-
 color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output
 -O -prof -static -fprof-auto
 cd "profiling/should_run/heapprof001.run" && ./heapprof001 +RTS -hr -RTS 7
 cd "profiling/should_run/heapprof001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/hp2ps" heapprof001
 "gs" -dNODISPLAY -dBATCH -dQUIET -dNOPAUSE
 "profiling/should_run/heapprof001.run/heapprof001.ps"
 =====> T2783(normal) 6 of 11 [0, 2, 0]
 cd "rts/T2783.run" &&  "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-
 stage2" -o T2783 T2783.hs -dcore-lint -dcmm-lint -no-user-package-db
 -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups
 -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-
 debug-output
 cd "rts/T2783.run" && ./T2783
 =====> T7919(normal) 7 of 11 [0, 2, 0]
 cd "rts/T7919.run" &&  "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-
 stage2" -o T7919 T7919.hs -dcore-lint -dcmm-lint -no-user-package-db
 -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups
 -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-
 debug-output  -dynamic
 cd "rts/T7919.run" && ./T7919
 =====> T14761c(normal) 8 of 11 [0, 2, 0]
 cd "typecheck/should_compile/T14761c.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -c T14761c.hs
 -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-
 specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-
 diagnostics-show-caret -Werror=compat -dno-debug-output -fno-warn-
 incomplete-patterns
 =====> hpc_fork(normal) 9 of 11 [0, 2, 0]
 cd "libraries/hpc/tests/fork/hpc_fork.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o hpc_fork
 hpc_fork.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-
 missed-specialisations -fshow-warning-groups -fdiagnostics-color=never
 -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -fhpc
 cd "libraries/hpc/tests/fork/hpc_fork.run" && perl hpcrun.pl --clear
 --exeext= --hpc="/home/jrp/Projects/ghc/inplace/test   spaces/hpc"
 ./hpc_fork
 =====> user001(normal) 10 of 11 [0, 2, 0]
 cd "libraries/unix/tests/user001.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o user001
 user001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-
 missed-specialisations -fshow-warning-groups -fdiagnostics-color=never
 -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -package
 unix
 cd "libraries/unix/tests/user001.run" && ./user001
 Actual stdout output differs from expected:
 diff -uw "libraries/unix/tests/user001.run/user001.stdout.normalised"
 "libraries/unix/tests/user001.run/user001.run.stdout.normalised"
 --- libraries/unix/tests/user001.run/user001.stdout.normalised  2018-11-22
 21:25:17.853414712 +0000
 +++ libraries/unix/tests/user001.run/user001.run.stdout.normalised
 2018-11-22 21:25:17.853414712 +0000
 @@ -6,6 +6,6 @@
  getEffectiveUserName: OK
  getGroupEntryForID: OK
  getGroupEntryForName: OK
 -getAllGroupEntries: OK
 +getAllGroupEntries: ERROR: getAllGroupEntries: does not exist (No such
 file or directory)
  getUserEntryForID: OK
 -getAllUserEntries: OK
 +getAllUserEntries: ERROR: getAllUserEntries: does not exist (No such file
 or directory)
 *** unexpected failure for user001(normal)
 =====> T3816(normal) 11 of 11 [0, 3, 0]
 cd "libraries/unix/tests/T3816.run" &&
 "/home/jrp/Projects/ghc/inplace/test   spaces/ghc-stage2" -o T3816
 T3816.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-
 missed-specialisations -fshow-warning-groups -fdiagnostics-color=never
 -fno-diagnostics-show-caret -Werror=compat -dno-debug-output  -package
 unix
 cd "libraries/unix/tests/T3816.run" && ./T3816
 Wrong exit code for T3816(normal)(expected 0 , actual 1 )
 Stderr ( T3816 ):
 T3816: getAllGroupEntries: does not exist (No such file or directory)
 *** unexpected failure for T3816(normal)

 Unexpected results from:
 TEST="T15904 T3816 recomp007 user001"

 SUMMARY for test run started at Thu Nov 22 21:24:33 2018 GMT
  0:00:46 spent to go through
       11 total tests, which gave rise to
       93 test cases, of which
       76 were skipped

        0 had missing libraries
       13 expected passes
        0 expected failures

        0 caused framework failures
        0 caused framework warnings
        0 unexpected passes
        4 unexpected failures
        0 unexpected stat failures

 Unexpected failures:
    driver/recomp007/recomp007.run    recomp007 [bad stderr] (normal)
    hp2ps/T15904.run                  T15904 [bad stdout] (normal)
    libraries/unix/tests/user001.run  user001 [bad stdout] (normal)
    libraries/unix/tests/T3816.run    T3816 [bad exit code] (normal)

 Appending 0 stats to git notes.
 make[1]: *** [../mk/test.mk:342: test] Error 1
 make[1]: Leaving directory '/home/jrp/Projects/ghc/testsuite/tests'
 make: *** [Makefile:224: test] Error 2
 }}}

-- 
Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15736#comment:12
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler