#15614: Test suite should show reason for skipping a test -------------------------------------+------------------------------------- Reporter: osa1 | Owner: (none) Type: task | Status: new Priority: normal | Milestone: Component: Test Suite | Version: 8.5 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I want to add a GHCi test, but it's a bit tricky to run the test so I use a make rule and use `run_command`, like this: {{{ test('UnsafeReenterGhci', [only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])], run_command, ['$MAKE -s --no-print-directory UnsafeReenterGhci']) }}} The important part is `run_command` and `only_ways(['ghci'])`. The problem is that it's impossible to run this test, because by default a `run_command` test can only be run in normal way (see test_common_work() in testlib.py), but there's no way to know this without reading the source code. Having better documentation might help, but I think the test driver should print why a test is skipped (maybe only in verbose mode -- if such a mode exists). In our case it should print something like "Test is only run in 'normal' way" (because `run_command` tests only run in 'normal' way). Correct version of this test is: {{{ test('UnsafeReenterGhci', [extra_ways(['ghci']), only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])], run_command, ['$MAKE -s --no-print-directory UnsafeReenterGhci']) }}} The `extra_ways` makes the test driver consider running the test in `ghci` way, `only_ways` is to avoid running it in `normal` way (the default and only way for `run_command`). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15614 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler