#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: -------------------------------+------------------------------------------- Changes (by thomie): * related: => #10441 Comment: I guess this works as well? {{{ python run-tests.py -e "config.compiler='/c/programs/ghc/ghc'" }}} Can we try to not rely on this magic? See also #10441. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: -------------------------------+------------------------------------------- Description changed by Rufflewind: Old description:

Ever since commit 5258566ee5c89aa757b0cf1433169346319c018f it seems that out-of-tree tests will not run on MinGW anymore. Note that I am using MinGW tools with a native Windows build of Python, which is the main cause of the problem.

The apparent error is something along the lines of "cannot eval an empty string":

{{{ Traceback (most recent call last): File "testsuite/driver/runtests.py", line 196, in <module> get_compiler_info() File "<string>", line 166, in get_compiler_info File "<string>", line 0 ^ SyntaxError: unexpected EOF while parsing }}}

But this error is quite misleading. What really happened was that it tried to run GHC with the `--info` argument but failed silently and returned an empty string. If you apply D908, it becomes a lot more obvious: Python is failing to run `/c/programs/ghc/bin/ghc --info` because `/c/programs/ghc/bin/ghc` "does not exist" as Python does not understand MinGW-style paths.

The funny thing is that it worked just fine before 525856, so you have to wonder why quoting the paths caused this. It turns out that MinGW does some magic behind the scenes, automagically converting MinGW-style paths into ordinary Python paths when a native Windows program is invoked. In particular, it means running this in MinGW shell

{{{ python run-tests.py -e 'config.compiler=/c/programs/ghc/ghc' }}}

will actually invoke Python with `config.compiler=C:/programs/ghc/ghc`. However, if it's explicitly quoted like this:

{{{ python run-tests.py -e 'config.compiler="/c/programs/ghc/ghc"' }}}

the magic doesn't work anymore.

New description: Ever since commit 5258566ee5c89aa757b0cf1433169346319c018f it seems that out-of-tree tests will not run on MinGW anymore. Note that I am using MinGW tools with a native Windows build of Python, which is the main cause of the problem. The apparent error is something along the lines of "cannot eval an empty string": {{{ Traceback (most recent call last): File "testsuite/driver/runtests.py", line 196, in <module> get_compiler_info() File "<string>", line 166, in get_compiler_info File "<string>", line 0 ^ SyntaxError: unexpected EOF while parsing }}} But this error is quite misleading. What really happened was that it tried to run GHC with the `--info` argument but failed silently and returned an empty string. If you apply D908, it becomes a lot more obvious: Python is failing to run `/c/programs/ghc/bin/ghc --info` because `/c/programs/ghc/bin/ghc` "does not exist" as Python does not understand MinGW-style paths. The funny thing is that it worked just fine before 525856, so you have to wonder why quoting the paths caused this. It turns out that MinGW does some magic behind the scenes, automagically converting MinGW-style paths into ordinary Python paths when a native Windows program is invoked. In particular, it means running this in MinGW shell {{{ python run-tests.py -e 'config.compiler=/c/programs/ghc/ghc' }}} will actually invoke Python with `config.compiler=C:/programs/ghc/ghc`. However, if it's explicitly quoted (''edit:'' and also escaped with backslashes) like this {{{ python run-tests.py -e 'config.compiler="\"/c/programs/ghc/ghc\""' }}} the magic doesn't work anymore. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

But escaped single quotes are still ok? That's what was used before

#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: -------------------------------+------------------------------------------- Comment (by Rufflewind): Replying to [comment:4 thomie]: 525856. Before 525856, it was invoked like this (contains a double-quote only): {{{ … -e 'config.compiler="$(TEST_HC)"' }}} Now it's invoked like this (with a double-quote 'and an escaped double- quote'): {{{ … -e 'config.compiler="\"$(TEST_HC)\""' }}} The escaped double-quote is what's tripping up MinGW (an escaped single- quote would cause the same problem). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: -------------------------------+------------------------------------------- Comment (by thomie): Ok, now I understand, sorry for taking so long. The MINGW magic should happen *before* the string reaches Python, as you said. And now it doesn't, because the string is too heavily quoted. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: patch Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: Phab:D911 -------------------------------+------------------------------------------- Comment (by Rufflewind): Unfortunately, the `wrap` call still breaks the magic. The conversion seems to be documented here: http://www.mingw.org/wiki/Posix_path_conversion What is the reason for quoting `compiler.config`? A quick grep shows the following code use `compiler.config`: {{{ ./testsuite/config/ghc:config.compiler = 'ghc' ./testsuite/config/ghc: h = os.popen(config.compiler + ' --info', 'r') ./testsuite/config/ghc: h = os.popen(config.compiler + ' +RTS --info', 'r') ./testsuite/mk/test.mk: -e 'config.compiler=$(call quote_path,$(TEST_HC))' \ ./testsuite/driver/testglobals.py: self.compiler = '' ./testsuite/driver/runtests.py: # testframe -e 'config.compiler=ghc-5.04' }}} There might be others I've missed, but I think in many situations the quotes will either get stripped anyway, or can be rewritten using `subprocess` so as to avoid the need for them. If quotes are necessary, they could be just added at those specific places. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: patch Priority: normal | Milestone: 7.12.1 Component: Test Suite | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: Phab:D911 -------------------------------+------------------------------------------- Comment (by thomie): Replying to [comment:9 Rufflewind]:

What is the reason for quoting `compiler.config`?

Copy pasting from the comments: {{{ # Wrap non-empty program paths in quotes, because they may contain spaces. Do # it here, so we don't have to (and don't forget to do it) in the .T test # scripts (search for '{compiler}' or '{hpc}'). This may or may not be a good # idea. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10449: Out-of-tree tests broken on MinGW + native Python due to quoting of config.compiler -------------------------------+------------------------------------------- Reporter: Rufflewind | Owner: Type: bug | Status: closed Priority: normal | Milestone: 7.10.2 Component: Test Suite | Version: 7.10.1 Resolution: fixed | Keywords: Operating System: Windows | Architecture: Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: #10441 | Differential Revisions: Phab:D911 -------------------------------+------------------------------------------- Changes (by thoughtpolice): * status: merge => closed * resolution: => fixed Comment: Merged to `ghc-7.10`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10449#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler