Proposal: run GHC API tests on fast
Currently, most GHC API tests are not run on 'make fast', ostensibly because linking against the GHC API can take a while. I propose that change this, and run GHC API tests by default. Reasons: 1. The GHC API is closely tied a lot of internal structure of GHC, so it's very easy to make a change, track it through the rest of the compiler, but forget to update the tests/documentation. 2. We can boost this into poor man's testable documentation. The idea is to duplicate all GHC API examples in the manual in the test suite, and have a comment on all of the examples asking the developer to update the manual. (Or we could automatically extract the snippets from the manual, but that's work and this I could do in a few minutes.) 3. I don't think running these tests will add that much extra run time to the test suite; certainly interactively the time spent linking is unnoticeable. Let's set a one week discussion period for this proposal. Thanks, Edward
Fine by me. In practice tests that aren't part of fast tend to bitrot, so until we have better infrastructure to tell us about failing non-fast tests, it's better to move more of them into fast. On 30/05/2014 01:55, Edward Z. Yang wrote:
Currently, most GHC API tests are not run on 'make fast', ostensibly because linking against the GHC API can take a while. I propose that change this, and run GHC API tests by default. Reasons:
1. The GHC API is closely tied a lot of internal structure of GHC, so it's very easy to make a change, track it through the rest of the compiler, but forget to update the tests/documentation.
2. We can boost this into poor man's testable documentation. The idea is to duplicate all GHC API examples in the manual in the test suite, and have a comment on all of the examples asking the developer to update the manual. (Or we could automatically extract the snippets from the manual, but that's work and this I could do in a few minutes.)
3. I don't think running these tests will add that much extra run time to the test suite; certainly interactively the time spent linking is unnoticeable.
Let's set a one week discussion period for this proposal.
Thanks, Edward _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
OK, I've gone ahead and done this. Edward Excerpts from Edward Z. Yang's message of 2014-05-30 01:55:35 +0100:
Currently, most GHC API tests are not run on 'make fast', ostensibly because linking against the GHC API can take a while. I propose that change this, and run GHC API tests by default. Reasons:
1. The GHC API is closely tied a lot of internal structure of GHC, so it's very easy to make a change, track it through the rest of the compiler, but forget to update the tests/documentation.
2. We can boost this into poor man's testable documentation. The idea is to duplicate all GHC API examples in the manual in the test suite, and have a comment on all of the examples asking the developer to update the manual. (Or we could automatically extract the snippets from the manual, but that's work and this I could do in a few minutes.)
3. I don't think running these tests will add that much extra run time to the test suite; certainly interactively the time spent linking is unnoticeable.
Let's set a one week discussion period for this proposal.
Thanks, Edward
Hi Edward, Am Freitag, den 22.08.2014, 18:01 +0100 schrieb Edward Z.Yang:
OK, I've gone ahead and done this.
I’m seeing errors like this on travis, with DEBUG_STAGE2=YES, but with varying test cases from ghc-api: Compile failed (status 256) errors were: [1 of 1] Compiling Main ( ghcApi.hs, ghcApi.o ) Linking ghcApi ... /usr/bin/ld: reopening ghcApi.o: No such file or directory /usr/bin/ld:ghcApi.o: bfd_stat failed: No such file or directory /usr/bin/ld: reopening ghcApi.o: No such file or directory /usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.22 internal error, aborting at ../../bfd/merge.c line 873 in _bfd_merged_section_offset /usr/bin/ld: Please report this bug. collect2: ld returned 1 exit status *** unexpected failure for ghcApi(normal) Wrong exit code (expected 0 , actual 2 ) Stdout: Stderr: /usr/bin/ld: reopening T6145.o: No such file or directory /usr/bin/ld:T6145.o: bfd_stat failed: No such file or directory /usr/bin/ld: reopening T6145.o: No such file or directory /usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.22 internal error, aborting at ../../bfd/merge.c line 873 in _bfd_merged_section_offset /usr/bin/ld: Please report this bug. collect2: ld returned 1 exit status make[3]: *** [T6145] Error 1 *** unexpected failure for T6145(normal) e.g. https://s3.amazonaws.com/archive.travis-ci.org/jobs/33351170/log.txt https://s3.amazonaws.com/archive.travis-ci.org/jobs/33356796/log.txt Any idea? Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
I'll go ahead and try to reproduce. Sounds like a bug! Excerpts from Joachim Breitner's message of 2014-08-23 15:44:48 +0100:
Hi Edward,
Am Freitag, den 22.08.2014, 18:01 +0100 schrieb Edward Z.Yang:
OK, I've gone ahead and done this.
I’m seeing errors like this on travis, with DEBUG_STAGE2=YES, but with varying test cases from ghc-api:
Compile failed (status 256) errors were: [1 of 1] Compiling Main ( ghcApi.hs, ghcApi.o ) Linking ghcApi ... /usr/bin/ld: reopening ghcApi.o: No such file or directory
/usr/bin/ld:ghcApi.o: bfd_stat failed: No such file or directory /usr/bin/ld: reopening ghcApi.o: No such file or directory
/usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.22 internal error, aborting at ../../bfd/merge.c line 873 in _bfd_merged_section_offset
/usr/bin/ld: Please report this bug.
collect2: ld returned 1 exit status
*** unexpected failure for ghcApi(normal)
Wrong exit code (expected 0 , actual 2 ) Stdout:
Stderr: /usr/bin/ld: reopening T6145.o: No such file or directory
/usr/bin/ld:T6145.o: bfd_stat failed: No such file or directory /usr/bin/ld: reopening T6145.o: No such file or directory
/usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.22 internal error, aborting at ../../bfd/merge.c line 873 in _bfd_merged_section_offset
/usr/bin/ld: Please report this bug.
collect2: ld returned 1 exit status make[3]: *** [T6145] Error 1
*** unexpected failure for T6145(normal)
e.g. https://s3.amazonaws.com/archive.travis-ci.org/jobs/33351170/log.txt https://s3.amazonaws.com/archive.travis-ci.org/jobs/33356796/log.txt
Any idea?
Greetings, Joachim
I couldn't reproduce this error on x86_64 with BuildFlavour = devel2. Is perhaps parallelism involved? Edward Excerpts from Joachim Breitner's message of 2014-08-23 15:44:48 +0100:
Hi Edward,
Am Freitag, den 22.08.2014, 18:01 +0100 schrieb Edward Z.Yang:
OK, I've gone ahead and done this.
I’m seeing errors like this on travis, with DEBUG_STAGE2=YES, but with varying test cases from ghc-api:
Compile failed (status 256) errors were: [1 of 1] Compiling Main ( ghcApi.hs, ghcApi.o ) Linking ghcApi ... /usr/bin/ld: reopening ghcApi.o: No such file or directory
/usr/bin/ld:ghcApi.o: bfd_stat failed: No such file or directory /usr/bin/ld: reopening ghcApi.o: No such file or directory
/usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.22 internal error, aborting at ../../bfd/merge.c line 873 in _bfd_merged_section_offset
/usr/bin/ld: Please report this bug.
collect2: ld returned 1 exit status
*** unexpected failure for ghcApi(normal)
Wrong exit code (expected 0 , actual 2 ) Stdout:
Stderr: /usr/bin/ld: reopening T6145.o: No such file or directory
/usr/bin/ld:T6145.o: bfd_stat failed: No such file or directory /usr/bin/ld: reopening T6145.o: No such file or directory
/usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.22 internal error, aborting at ../../bfd/merge.c line 873 in _bfd_merged_section_offset
/usr/bin/ld: Please report this bug.
collect2: ld returned 1 exit status make[3]: *** [T6145] Error 1
*** unexpected failure for T6145(normal)
e.g. https://s3.amazonaws.com/archive.travis-ci.org/jobs/33351170/log.txt https://s3.amazonaws.com/archive.travis-ci.org/jobs/33356796/log.txt
Any idea?
Greetings, Joachim
Hi, Am Samstag, den 23.08.2014, 18:57 +0100 schrieb Edward Z.Yang:
I couldn't reproduce this error on x86_64 with BuildFlavour = devel2. Is perhaps parallelism involved?
likely. Did you try validating with "CPUS=2"? (But then, I believe that is actually the default). Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
I have seen this too just running "make THREADS=8". Looks like it's because the other tests in this directory are cleaning too aggressively. From the Makefile: ... clean: rm -f *.o *.hi T6145: clean '$(TEST_HC)' $(TEST_HC_OPTS) --make -v0 -package ghc T6145 ./T6145 "`'$(TEST_HC)' $(TEST_HC_OPTS) --print-libdir | tr -d '\r'`" ... so ghcApi.o is getting removed before the final link step, I would guess. Regards, Reid Barton
I pushed a fix for the tests in ghc-api. Edward Excerpts from Reid Barton's message of 2014-08-23 20:07:40 +0200:
I have seen this too just running "make THREADS=8". Looks like it's because the other tests in this directory are cleaning too aggressively. From the Makefile:
... clean: rm -f *.o *.hi
T6145: clean '$(TEST_HC)' $(TEST_HC_OPTS) --make -v0 -package ghc T6145 ./T6145 "`'$(TEST_HC)' $(TEST_HC_OPTS) --print-libdir | tr -d '\r'`" ...
so ghcApi.o is getting removed before the final link step, I would guess.
Regards, Reid Barton
Hi, Am Dienstag, den 02.09.2014, 09:08 +0200 schrieb Edward Z.Yang:
I pushed a fix for the tests in ghc-api.
thanks! Greetings, Joachim -- Joachim “nomeata” Breitner mail@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nomeata@debian.org
participants (4)
-
Edward Z. Yang -
Joachim Breitner -
Reid Barton -
Simon Marlow