I've removed the code, if you update it should be gone Yes, that’s great. The commit I made should have reduced the amount of failing tests to 0. framework failures are always quite unusual. Definitely not zero! I’m getting these failing tests Unexpected failures: plugins/plugins07.run plugins07 [bad exit code] (normal) plugins/plugins09.run plugins09 [bad stdout] (normal) plugins/plugins11.run plugins11 [bad stdout] (normal) plugins/T10420.run T10420 [bad exit code] (normal) plugins/T11244.run T11244 [bad stderr] (normal) plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) rts/stack002.run stack002 [exit code non-0] (normal) rts/T3236.run T3236 [exit code non-0] (normal) rts/testwsdeque.run testwsdeque [exit code non-0] (threaded1) /../libraries/Win32/tests/T4452.run T4452 [bad exit code] (normal) Unexpected stat failures: perf/compiler/T6048.run T6048 [stat not good enough] (optasm) perf/compiler/T12234.run T12234 [stat not good enough] (optasm) perf/compiler/T12150.run T12150 [stat not good enough] (optasm) perf/should_run/T15226.run T15226 [stat too good] (normal) perf/should_run/T15226a.run T15226a [stat too good] (normal) perf/compiler/MultiLayerModules.run MultiLayerModules [stat not good enough] (normal) Framework failures: plugins/plugins07.run plugins07 [normal] (pre_cmd failed: 2) plugins/T10420.run T10420 [normal] (pre_cmd failed: 2) plugins/T11244.run T11244 [normal] (pre_cmd failed: 2) plugins/plugin-recomp-pure.run plugin-recomp-pure [normal] (pre_cmd failed: 2) plugins/plugin-recomp-impure.run plugin-recomp-impure [normal] (pre_cmd failed: 2) plugins/plugin-recomp-flags.run plugin-recomp-flags [normal] (pre_cmd failed: 2) Framework warnings: . T13701 [numfield-no-expected] (No expected value found for bytes allocated in num_field check) I’ll send you the info you wanted for T10420 in a separate email. Thanks for helping! Simon From: Phyx <lonetiger@gmail.com> Sent: 13 June 2018 20:47 To: Simon Peyton Jones <simonpj@microsoft.com> Cc: ghc-devs@haskell.org Subject: Re: Strace Hi Simon, On Wed, Jun 13, 2018 at 5:24 PM, Simon Peyton Jones <simonpj@microsoft.com<mailto:simonpj@microsoft.com>> wrote: OK – so maybe the root cause is a framework failure – and indeed for the last few weeks I’ve seen Framework failures: plugins/plugins07.run plugins07 [normal] (pre_cmd failed: 2) plugins/T10420.run T10420 [normal] (pre_cmd failed: 2) plugins/T11244.run T11244 [normal] (pre_cmd failed: 2) I have just learned to live with these failures, because I knew you were working on making things better. But it sounds as if they are still taking place. The commit I made should have reduced the amount of failing tests to 0. framework failures are always quite unusual. So: * Yes, please make it not happen by default I've removed the code, if you update it should be gone. It was there and on by default because I was trying to debug failures on Harbormaster, I realized a switch isn't very useful as I won't be able to toggle it for Harbormaster anyway. * * If you don’t get these framework failures, can we work together to resolve them? These don't happen for me nor on Harbormaster, try picking a test, e.g T10420 run only that test to make sure it's not a threading issue: make TEST=T10420 test -C testsuite/tests If it still gives a framework error then do at the top level make VERBOSE=3 TEST=T10420 test -C testsuite/tests once it runs, the output should contain the command it ran as a pre_cmd, and the stdout and stderr from the pre_cmd output. Could you then send the error? if it doesn't show any of this, try make CLEANP=0 VERBOSE=3 TEST= T10420 test -C testsuite/tests --trace and copy and paste the pre_cmd command, which should just replay the action it did. Cheers, Tamar Thanks Simon From: Phyx <lonetiger@gmail.com<mailto:lonetiger@gmail.com>> Sent: 13 June 2018 17:19 To: Simon Peyton Jones <simonpj@microsoft.com<mailto:simonpj@microsoft.com>> Cc: ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> Subject: Re: Strace Hi Simon, The strace is only supposed to run when the normal test pre_cmd fails. If it's running that often it means your tests are all failing during pre_cmd with a framework failure https://git.haskell.org/ghc.git/blobdiff/4778cba1dbb6adf495930322d7f9e9db0af... But maybe I shouldn't turn this on my default. I'll pramaterize it when I get home. Tamar. On Wed, Jun 13, 2018, 17:09 Simon Peyton Jones <simonpj@microsoft.com<mailto:simonpj@microsoft.com>> wrote: Tamar I’m getting megabytes of output from ‘sh validate’ on windows. It looks like this 629 151745 [main] sh 2880 fhandler_base::fhaccess: returning 0 291 152036 [main] sh 2880 faccessat: returning 0 7757 159793 [main] sh 2880 fhandler_base_overlapped::wait_overlapped: wfres 0, wores 1, bytes 7 179457 1608947 [main] make 11484 fhandler_base_overlapped::wait_overlapped: wfres 0, wores 1, bytes 7 99 159892 [main] sh 2880 fhandler_base_overlapped::wait_overlapped: normal write, 7 bytes ispipe() 1 180 1609127 [main] make 11484 fhandler_base_overlapped::wait_overlapped: normal read, 7 bytes ispipe() 1 139 160031 [main] sh 2880 write: 7 = write(1, 0x6000396A0, 7) 142 1609269 [main] make 11484 fhandler_base::read: returning 7, binary mode 139 1609408 [main] make 11484 read: 7 = read(5, 0x60005B4B0, 7) 136 1609544 [main] make 11484 read: read(5, 0x60005B4B7, 193) blocking 4693 164724 [main] sh 2880 set_signal_mask: setmask 0, newmask 80000, mask_bits 0 but with hundreds of thousands of lines. (I have not counted) I believe that it may be the result of this line, earlier in the log cd "/c/Users/simonpj/AppData/Local/Temp/ghctest-8fa9s6rk/test spaces/./plugins/plugins07.run" && strace $MAKE -s --no-print-directory -C rule-defining-plugin package.plugins07 TOP=/c/code/HEAD/testsuite# Note the strace. That in turn was added in your commit commit 60fb2b2160aa16194b74262f4df8fad5af171b0f Author: Tamar Christina <tamar@zhox.com<mailto:tamar@zhox.com>> Date: Mon May 28 19:34:11 2018 +0100 Clean up Windows testsuite failures Summary: Another round and attempt at getting these down to 0. Could you perhaps have made a mistake here? Currently validate is unusable. Thanks! Simon