where's the appropriate place to put regression tests for compile bugs?
I've been having a bit of a poke about the regression tests, and all the tests seem to be for single Main files. One of my test cases that fails is for compiling a separate module - the bug doesn't manifest if you change it to Main. John, have you got a plan for expanding the test suite to cover this sort of thing? On another note, it'd be nice to have some basic integration/ functionality tests - ensure that a given jhc tree can build something using containers, for instance, or build a sensible dynamic library (my own personal bugbear). Have you thought about using one of the test frameworks for Haskell? Or even one of the more generic Perl test frameworks, given that regress.prl is in Perl already? Cheers mark (JAJH)
On Tue, Aug 25, 2009 at 04:19:10PM +1000, Mark Wotton wrote:
I've been having a bit of a poke about the regression tests, and all the tests seem to be for single Main files. One of my test cases that fails is for compiling a separate module - the bug doesn't manifest if you change it to Main. John, have you got a plan for expanding the test suite to cover this sort of thing?
I pretty much just add features to the regression driver as needed in an ad hoc way. Feel free to extend it.
On another note, it'd be nice to have some basic integration/ functionality tests - ensure that a given jhc tree can build something using containers, for instance, or build a sensible dynamic library (my own personal bugbear). Have you thought about using one of the test frameworks for Haskell? Or even one of the more generic Perl test frameworks, given that regress.prl is in Perl already?
I am not opposed to the idea of using a framework, but it would have to have some substantial benefit to justify another dependency. If something isn't clearly superior I'd rather just extend my regression driver some. It has some nice features, like recognizing when things are killed by SIGINT, setting processor limits, checking compile and run status independently. But yeah, having it support more types of tests (like, does this compile) would be useful. John -- John Meacham - ⑆repetae.net⑆john⑈ - http://notanumber.net/
On 25/08/2009, at 10:20 PM, John Meacham wrote:
On Tue, Aug 25, 2009 at 04:19:10PM +1000, Mark Wotton wrote:
I've been having a bit of a poke about the regression tests, and all the tests seem to be for single Main files. One of my test cases that fails is for compiling a separate module - the bug doesn't manifest if you change it to Main. John, have you got a plan for expanding the test suite to cover this sort of thing?
I pretty much just add features to the regression driver as needed in an ad hoc way. Feel free to extend it.
Righto, I'll have a hack.
On another note, it'd be nice to have some basic integration/ functionality tests - ensure that a given jhc tree can build something using containers, for instance, or build a sensible dynamic library (my own personal bugbear). Have you thought about using one of the test frameworks for Haskell? Or even one of the more generic Perl test frameworks, given that regress.prl is in Perl already?
I am not opposed to the idea of using a framework, but it would have to have some substantial benefit to justify another dependency. If something isn't clearly superior I'd rather just extend my regression driver some. It has some nice features, like recognizing when things are killed by SIGINT, setting processor limits, checking compile and run status independently. But yeah, having it support more types of tests (like, does this compile) would be useful.
I understand your desire to remove dependencies, but given that those most interested in running tests will be the developers, I'd argue that an extra package is not a big deal if it can be easily installed with CPAN or Cabal and isn't required for building and running jhc. still, i'll see what I can do within the framework first. mark
participants (2)
-
John Meacham -
Mark Wotton