questions about validating in the presence of known failures

[You can generalize this email from OS X to other architectures/operating systems.] Question 1: Is a list of "unexpecteds expected on OS X" maintained somewhere? The comments on #5757 seemed like a start, but they're 1.5 years old. The Trac "Custom Query" for OS X isn't helpful for my actual question of "can I ignore these wrt to validating my patch?". A wiki page collating tickets for any "known unexpecteds" for OS X (and other OSs) would be helpful. Question 2: Can we add another bit to unexpected results marking them as known/unknown? That would yield an immediacy for the validation workflow in the cases where we have long-standing known bugs, eg on various platforms. I think #5757 is on the right path, but marking a test that should pass as an expected failure seems disappointing. Is there a traditional workflow for handling this? Question 3: What's the status on the build bot farm? Thanks for your time, everyone.

On Thu, May 02, 2013 at 10:23:26AM -0500, Nicolas Frisby wrote:
Question 2: Can we add another bit to unexpected results marking them as known/unknown?
Unexpected results are all unknown. If they're known then they're expected results. I've just written this on how to deal with validate failures not caused by local patches: http://hackage.haskell.org/trac/ghc/wiki/TestingPatches#Validatehasfailingte...
Question 3: What's the status on the build bot farm?
I hope to do some work on this soon, e.g. add a way for build results to be uploaded to the server by the builders. However, I've been spending my time on more pressing matters recently. Thanks Ian

Your amendment to the wiki page is very helpful. In particular, it totally
supplants my suggestion about "known unexpecteds".
Thanks.
On Thu, May 2, 2013 at 11:25 AM, Ian Lynagh
On Thu, May 02, 2013 at 10:23:26AM -0500, Nicolas Frisby wrote:
Question 2: Can we add another bit to unexpected results marking them as known/unknown?
Unexpected results are all unknown. If they're known then they're expected results.
I've just written this on how to deal with validate failures not caused by local patches:
http://hackage.haskell.org/trac/ghc/wiki/TestingPatches#Validatehasfailingte...
Question 3: What's the status on the build bot farm?
I hope to do some work on this soon, e.g. add a way for build results to be uploaded to the server by the builders. However, I've been spending my time on more pressing matters recently.
Thanks Ian
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
participants (2)
-
Ian Lynagh
-
Nicolas Frisby