
Hi devs, https://gitlab.haskell.org/rae/ghc/-/jobs/173504 https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test? The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate. Thanks, Richard

For what it's worth, I found two recent tickets documenting failures for that very same test: #16897 and #17269. On 09/10/2019 10:45, Richard Eisenberg wrote:
Hi devs,
https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test?
The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate.
Thanks, Richard
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-- Alp Mestanogullari, Haskell Consultant Well-Typed LLP, https://www.well-typed.com/ Registered in England and Wales, OC335890 118 Wymering Mansions, Wymering Road, London, W9 2NF, England

Unfortunately T13615 is indeed known to be fragile. This is tracked as #17269, ss you discovered. This is especially scary given that the test tests some rather subtle RTS logic. It's been on my list of things to investigate but unfortunately I've not been able to reproduce the failure locally so progress has been slow.
Given that it seems to fail fairly reliably on CI in some cases we probably ought to mark it as fragile.
Note that it's not quite true that fragile test outcomes are no longer tracked. We maintain a database of test outcomes from CI runs which includes both passing and failing ribs of fragile tests. I've written about this infrastructure here [1].
Cheers,
- Ben
[1] https://www.haskell.org/ghc/blog/20190708-testsuite-work.html
On October 9, 2019 4:45:57 AM EDT, Richard Eisenberg
Hi devs,
https://gitlab.haskell.org/rae/ghc/-/jobs/173504 https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test?
The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate.
Thanks, Richard
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
participants (3)
-
Alp Mestanogullari
-
Ben Gamari
-
Richard Eisenberg