[GHC] #12554: Testsuite exibits large amount of framework failures
#12554: Testsuite exibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Keywords: | Operating System: Windows Architecture: | Type of failure: Incorrect result Unknown/Multiple | at runtime Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I've tried this on multiple computers and they all have the same issue. I get around 400 testsuite failures with the error: {{{ r:/temp/ghctest-0u4c8o/test spaces/./th/T11680.run T11680 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T11680.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T11797.run T11797 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T11797.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T10734.run T10734 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T10734.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T11345.run T11345 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T11345.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T10820.run T10820 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T10820.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T11484.run T11484 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T11484.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T9022.run T9022 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T9022.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T12130.run T12130 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T12130.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T8761.run T8761 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T8761.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T12407.run T12407 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T12407.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T11463.run T11463 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T11463.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T12478_4.run T12478_4 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T12478_4.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T12478_3.run T12478_3 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T12478_3.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T12513.run T12513 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T12513.run') r:/temp/ghctest-0u4c8o/test spaces/./th/T12530.run T12530 [ext-interp] ([Error 183] Cannot create a file when that file already exists: 'r:/temp/ghctest-0u4c8o/test spaces/./th/T12530.run') }}} What they all have in common is that they are all using the latest MSYS2 and tools: {{{ Tamar@Squid MINGW64 ~/ghc $ python3 --version Python 3.5.2 Tamar@Squid MINGW64 ~/ghc $ uname -a MINGW64_NT-10.0 Kino 2.5.2(0.297/5/3) 2016-07-15 08:31 x86_64 Msys }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Indeed I am also seeing this on the new Windows build bot. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): Taking a look again, but my Python foo is weak.. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I wonder if the testsuite driver is attempting to run the same test in multiple ways at concurrently, therefore attempting to copy the same file into place more than once. thomie, does this ring any bells? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Oh, right, we disable threading on Windows. Never mind then. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): Well, my TeamCity runs patch the testrunner to run in parallel. The weird thing is, it doesn't happen on my old msys installes, with python 3.4.3, I don't know if this started because of a change in python or the msys runtime. Another odd thing is that I would expect the tests that trigger this to be pretty random. But according to my teamcity data of the past ~1 month, the tests that fail are pretty stable. +- 3-4 tests each run, but all in the same category. Mostly the TH tests. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): I have my doubts about the threadsafety of the testsuite. In particular, the location of the function that's failing is right after a `rmtree` call that's supposed to ensure the path doesn't exist. Yet it does. Also I see that only relative paths are used in all directory name. I can only assume that it's relative to the `cwd` as explicit shell commands are issued to change the shell's `cwd`. Why the python call for this is not used I don't understand. I must be missing something since this is reliably working on Linux.. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Alright, I think I see what is going on here. But first some background. The testsuite driver runs each testcase in its own fresh directory to ensure a clean test environment. The code responsible for this can be found in `testlib.do_test()`. This looks something like the following, {{{#!python def do_test(name, way, func, args, files): opts = getTestOpts() shutil.rmtree(opts.testdir, ignore_errors=True) os.makedirs(opts.testdir) # Copy test files into testdir and run test ... }}} Note how we invoke `rmtree` with `ignore_errors=True`, presumably to account for the fact that `testdir` may not exist (e.g. if we are running in a fresh working directory). However, this unfortunately hides any **real** errors that we may encounter while trying to delete `testdir`, in which case we'll attempt to create `testdir` when it already exists. This appears to be precisely what happens some fraction of the time on Windows. Namely, if we rework `do_test` as follows, {{{#!python def do_test(name, way, func, args, files): opts = getTestOpts() if os.path.exists(opts.testdir): print('cleaning %s' % opts.testdir) shutil.rmtree(opts.testdir, ignore_errors=False) os.makedirs(opts.testdir) # Copy test files into testdir and run test ... }}} We find the following, {{{ =====> TH_nameSpace(ext-interp) 39 of 42 [0, 3, 12] cleaning ./th/TH_nameSpace.run *** framework failure for TH_nameSpace(ext-interp) [Error 5] Access is denied: './th/TH_nameSpace.run/TH_nameSpace.exe' }}} I suspect this is due to the fact that executable images of running programs are locked and therefore cannot be deleted. The strange thing here is that the process has presumably already terminated. It seems like the test process must still be in some sort of zombie state where it has terminated and we have its exit code, but its resources (e.g. executable image) have still not been released. As one would expect, adding a `time.sleep(0.1)` before the `rmtree` hides the issue. It's not clear to me what the proper solution is here. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by Phyx-): * Attachment "Logfile.CSV" added. I/O trace -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): That (though filtering on the python process, will make a wider filter now, seems to indicate, that at least the stderr is still locked when it tries to delete the folder.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): oh and I seem to get a different error from you {{{ =====> TH_repGuardOutput(ext-interp) 17 of 239 [0, 9, 8] cleaning ./TH_repGuardOutput.run cd "./TH_spliceE5_prof.run" && $MAKE -s --no-print-directory TH_spliceE5_prof *** framework failure for TH_repGuardOutput(ext-interp) [Error 32] The process cannot access the file because it is being used by another process: './TH_repGuardOutput.run/TH_repGuardOutput.comp.stderr' }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * Attachment "cgrun035.zip" added. Zipped ProcMon log of a failure of cgrun035 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * Attachment "ProcessName=cgrun035.exe.CSV" added. Different subset of the same trace, this time filtered on processname=cgrun035 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I spent quite a while today cleaning up the testsuite driver's Python 3 compatibility story and it seems that Python 3.5.2 is significantly less buggy than 2.7.11 with respect to this issue.. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Here is a fairly minimal testcase which reproduces the issue. Phyx-, can you verify that this doesn't crash in your working environment? {{{#!hs #!/usr/bin/python import sys import subprocess import os import os.path import time import shutil d = 'testdir' f = d + '/test.exe' log = d + '/log' if os.path.exists(d): shutil.rmtree(d) while True: os.makedirs(d) shutil.copy('test.exe', f) subprocess.check_call(f, stdout=open(log, 'w')) time.sleep(0.04) shutil.rmtree(d) print '.', sys.stdout.flush() }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): @bgamari, yes the script works fine for me as well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): that said, it also doesn't crash without the sleep in there. May be my test.exe is too simple? it's just "print 123" atm. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): Usage of `ignore_errors=True` was sloppy program, sorry about that. It would be better to revert back to the original (working and commented) code: {{{ def clean_full_path(name): try: # Remove files... os.remove(name) except OSError as e1: try: # ... and empty directories os.rmdir(name) except OSError as e2: # We don't want to fail here, but we do want to know # what went wrong, so print out the exceptions. # ENOENT isn't a problem, though, as we clean files # that don't necessarily exist. if e1.errno != errno.ENOENT: print(e1) if e2.errno != errno.ENOENT: print(e2) }}} On the real bug: you could try to eliminate any possible problems with `timeout`: {{{ - r = subprocess.call([timeout_prog, timeout, cmd], + r = subprocess.call(['sh', '-c', cmd], }}} On Windows the testsuite uses a `timeout` program written in Haskell, on all other platforms `timeout` is written in Python. I'm not quite sure why two separate versions exist, but all the code is in `testsuite/timeout`. Fwiw: I couldn't reproduce this issue myself, last time I tried. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I pushed some patches which I was testing with to Phab:D2598. They cover a few things including Python 3 compatibility, working around the `cleanup` issue described above, disabling usage of symlinks on Windows (since they require administrator privileges)n and marking a few tests as broken. Unfortunately the `cleanup` workaround only narrows the Window and does not close it entirely. Phyx- has isolated the `cleanup` problem as being a regression in the `msys2-runtime` package. 2.5.1 apparently works but 2.5.2 seems to fail. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Alright, so I've been trying to work out how to downgrade our Windows builder to a non-broken version of `msys2-runtime` (namely 2.5.1). It seems that the current installer executable ships with a 2.6 runtime, which is not compatible with the 2.5 series. Consequently it is necessary to use a release from July 2016 instead (http://repo.msys2.org/distrib/x86_64/msys2-base-x86_64-20160719.tar.xz). From this release you can downgrade the runtime with `pacman -U`. Namely I'm using http://repo.msys2.org/msys/x86_64/msys2-runtime-2.5.1-1-x86_64.pkg.tar.xz. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): So we're still at it. The current state of things is that, * msys `python-3.4.3` works with `msys2-runtime < 2.5.2` * msys `python-3.4.3`` does not work with runtime 2.5.1 nor 2.5.2 * mingw `python-2.7.11` does not work with either runtime In all cases the failure mode is error 183 during test directory deletion. In light of this it seems likely that there are two bugs: * CPython has a Windows compatibility bug (hence none of the mingw builds work) * `msys2-runtime >= 2.5.2` has a regression While the `msys2-runtime` issue is likely easier to track down since its a regression over a rather small diff, it's unfortunately not the issue we really care about. We'd far prefer to have mingw python working; unfortunately this is a much harder issue to debug. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Phyx- contributed another invaluable data point: * mingw `python-3.4.3` works with runtime 2.5.1 So it seems that there is very likely a Python regression between 3.4.3 and 3.5.2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Phyx- also tested mingw `python-3.4.3` with runtime 2.5.2. It failed, although perhaps not as frequently as with runtime 2.5.1. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Phyx said that his nightly server has been reliably running msys `python-3.4.3` and `msys2-runtime` 2.5.0 for quite some time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): Not my nightly, my main development machine. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Phyx- ran a run on his server to `msys2-runtime` 2.5.0 and mingw `python-3.4.3` and found that it too failed, despite comment:21. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Two runs with msys `python-3.4.3` and `msys-runtime-2.5.0` passed as clean. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * cc: RyanGlScott (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:25 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures
-------------------------------------+-------------------------------------
Reporter: Phyx- | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Test Suite | Version: 8.0.1
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#12554: Testsuite exhibits large amount of framework failures
-------------------------------------+-------------------------------------
Reporter: Phyx- | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Test Suite | Version: 8.0.1
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#12554: Testsuite exhibits large amount of framework failures
-------------------------------------+-------------------------------------
Reporter: Phyx- | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Test Suite | Version: 8.0.1
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2684 Wiki Page: | -------------------------------------+------------------------------------- Changes (by Phyx-): * status: new => patch * differential: => Phab:D2684 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:29 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures
-------------------------------------+-------------------------------------
Reporter: Phyx- | Owner:
Type: bug | Status: patch
Priority: normal | Milestone:
Component: Test Suite | Version: 8.0.1
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2684
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Test Suite | Version: 8.0.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2684 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Phyx has put in a herculean effort to try to fix up the testsuite driver in Phab:D2684. Unfortunately, while the result works on Harbormaster and RyanGlScott's machine, it still fails for me. However, it turns out that only mingw-w64 Python 2.7 is affected. Given that maintaining Python 2 support is already becoming rather tiresome and that Python 3 seems to work just fine, I say let's just deprecate Python 2 support in the testsuite driver. I've proposed this as Phab:D2766. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:31 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12554: Testsuite exhibits large amount of framework failures
-------------------------------------+-------------------------------------
Reporter: Phyx- | Owner:
Type: bug | Status: patch
Priority: normal | Milestone:
Component: Test Suite | Version: 8.0.1
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2684
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#12554: Testsuite exhibits large amount of framework failures
-------------------------------------+-------------------------------------
Reporter: Phyx- | Owner:
Type: bug | Status: patch
Priority: normal | Milestone:
Component: Test Suite | Version: 8.0.1
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2684
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#12554: Testsuite exhibits large amount of framework failures -------------------------------------+------------------------------------- Reporter: Phyx- | Owner: Type: bug | Status: closed Priority: normal | Milestone: 8.2.1 Component: Test Suite | Version: 8.0.1 Resolution: fixed | Keywords: Operating System: Windows | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2684 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed * milestone: => 8.2.1 Comment: This should at long last be fixed by the above two patches. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12554#comment:34 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC