ANNOUNCE: tasty, a new testing framework

I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained). Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of the box (through the standard packages), but it is very extensible, so you can write your own test providers. Please see the home page for more information: http://documentup.com/feuerbach/tasty Roman

On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
(which is unmaintained). Has this been confirmed by the author/maintainer? Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of the box (through the standard packages), but it is very extensible, so you can write your own test providers.
Please see the home page for more information: http://documentup.com/feuerbach/tasty Is it a drop-in replacement for test-framework, e.g. if I substitute "test-framework" for "tasty" in my .cabal files, will it work? If not, could you provide a quick guide for porting? Also, is the current version (0.1) recommended for general use?

Likewise, is test-framework explicitly unmaintained? I'd generally think a
testing tool should stabilize after a while... though I guess that
test-framework's author is pretty busy with some other work this year, but
I could be wrong.
On Mon, Aug 5, 2013 at 3:31 PM, Andrey Chudnov
On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
(which is unmaintained).
Has this been confirmed by the author/maintainer?
Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of
the box (through the standard packages), but it is very extensible, so you can write your own test providers.
Please see the home page for more information: http://documentup.com/**feuerbach/tastyhttp://documentup.com/feuerbach/tasty
Is it a drop-in replacement for test-framework, e.g. if I substitute "test-framework" for "tasty" in my .cabal files, will it work? If not, could you provide a quick guide for porting? Also, is the current version (0.1) recommended for general use?
______________________________**_________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe

* Andrey Chudnov
On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
(which is unmaintained). Has this been confirmed by the author/maintainer?
I've sent a couple of emails to Max (one in January, one in April) and haven't heard anything from him. My patches, which he applied in January, are still not released. That's why I regard test-framework as unmaintained. You can also make your own impression by browsing github (commit activity, outstanding pull requests, open issues...)
Is it a drop-in replacement for test-framework, e.g. if I substitute "test-framework" for "tasty" in my .cabal files, will it work? If not, could you provide a quick guide for porting?
Not quite. At the very least, you'll have to change module names (Test.Framework -> Test.Tasty, Test.Framework.Providers.HUnit -> Test.Tasty.HUnit, ...), and wrap the top-level list of tests into a testGroup. If you have type signatures, you'll need to rename Test to TestTree. That should be enough in most cases. If you use plusTestOptions, you'll need to look up appropriate functions from Test.Tasty.Options.
Also, is the current version (0.1) recommended for general use?
I'd love to see people using it. But you should treat this as beta software. I am in the process of migrating my own packages to use Tasty. Roman

Oh, a new testing framework - I'm always interested in that :)
At the very least, you'll have to change module names (Test.Framework -> Test.Tasty, Test.Framework.Providers.HUnit -> Test.Tasty.HUnit, ...), and wrap the top-level list of tests into a testGroup.
If you have type signatures, you'll need to rename Test to TestTree.
That should be enough in most cases. If you use plusTestOptions, you'll need to look up appropriate functions from Test.Tasty.Options. Making a tutorial would probably be a good idea here.
Janek

fair enough. I take it that you're also (implicitly) committing to
maintaining this for the next few years? :)
On Mon, Aug 5, 2013 at 4:02 PM, Roman Cheplyaka
* Andrey Chudnov
[2013-08-05 15:31:16-0400] On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
(which is unmaintained). Has this been confirmed by the author/maintainer?
I've sent a couple of emails to Max (one in January, one in April) and haven't heard anything from him. My patches, which he applied in January, are still not released. That's why I regard test-framework as unmaintained.
You can also make your own impression by browsing github (commit activity, outstanding pull requests, open issues...)
Is it a drop-in replacement for test-framework, e.g. if I substitute "test-framework" for "tasty" in my .cabal files, will it work? If not, could you provide a quick guide for porting?
Not quite.
At the very least, you'll have to change module names (Test.Framework -> Test.Tasty, Test.Framework.Providers.HUnit -> Test.Tasty.HUnit, ...), and wrap the top-level list of tests into a testGroup.
If you have type signatures, you'll need to rename Test to TestTree.
That should be enough in most cases. If you use plusTestOptions, you'll need to look up appropriate functions from Test.Tasty.Options.
Also, is the current version (0.1) recommended for general use?
I'd love to see people using it. But you should treat this as beta software. I am in the process of migrating my own packages to use Tasty.
Roman
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Could you add some documentation on how to use this with cabal? I've found
integrating tests with cabal unintuitive and poorly documented--to the
point where I haven't really bothered! I've gotten it working before, but I
would have to look it up again in the future. (I also didn't use a
framework.)
It would be awesome to see an example .cabal file along with your example
test cases.
thanks,
-tikhon
On Tue, Aug 6, 2013 at 1:53 AM, Roman Cheplyaka
* Carter Schonwald
[2013-08-05 16:58:37-0400] fair enough. I take it that you're also (implicitly) committing to maintaining this for the next few years? :)
That's correct.
Roman
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

You can find an example here:
https://github.com/feuerbach/regex-applicative/blob/master/regex-applicative...
If you'd like to contribute a short README section based on that, please
go ahead! :)
Roman
* Tikhon Jelvis
Could you add some documentation on how to use this with cabal? I've found integrating tests with cabal unintuitive and poorly documented--to the point where I haven't really bothered! I've gotten it working before, but I would have to look it up again in the future. (I also didn't use a framework.)
It would be awesome to see an example .cabal file along with your example test cases.
thanks, -tikhon
On Tue, Aug 6, 2013 at 1:53 AM, Roman Cheplyaka
wrote: * Carter Schonwald
[2013-08-05 16:58:37-0400] fair enough. I take it that you're also (implicitly) committing to maintaining this for the next few years? :)
That's correct.
Roman
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Awesome. Ill take a look at tasty sometime this month. Thanks for taking the time to patiently answer all of our questions. On Tuesday, August 6, 2013, Roman Cheplyaka wrote:
* Carter Schonwald
javascript:;> [2013-08-05 16:58:37-0400] fair enough. I take it that you're also (implicitly) committing to maintaining this for the next few years? :)
That's correct.
Roman

On 8/5/2013 2:48 PM, Roman Cheplyaka wrote:
I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of the box (through the standard packages), but it is very extensible, so you can write your own test providers.
Please see the home page for more information: http://documentup.com/feuerbach/tasty
Roman
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe Will tasty be available through yum?
*ducks*

Roman Cheplyaka
writes:
I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
It would be nice to see a comparison of the various test frameworks and why one might select one over another. I use hspec currently (which also integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what tasty might offer. And I particularly dislike writing tests inside of a gigantic list; I much prefer the "monadic" style of hspec. -- John Wiegley FP Complete Haskell tools, training and consulting http://fpcomplete.com johnw on #haskell/irc.freenode.net

* John Wiegley
Roman Cheplyaka
writes: I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
It would be nice to see a comparison of the various test frameworks and why one might select one over another. I use hspec currently (which also integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what tasty might offer. And I particularly dislike writing tests inside of a gigantic list; I much prefer the "monadic" style of hspec.
This has been discussed on reddit here: http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framewor... Roman

I admit I haven't yet had the time to try out testy, but there's one thing I'm curious about. QuickCheck can classify tests:

Sorry, my last email got sent too quickly. Anyway, continuing my thought. So QuickCheck can classify tests:
+++ OK, passed 100 tests (29% Short)
Can tasty display this classification info? That was a thing I missed a lot in test-framework and would probably motivate me to switch to tasty.
Janek
----- Oryginalna wiadomość -----
Od: "Roman Cheplyaka"
Roman Cheplyaka
writes: I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
It would be nice to see a comparison of the various test frameworks and why one might select one over another. I use hspec currently (which also integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what tasty might offer. And I particularly dislike writing tests inside of a gigantic list; I much prefer the "monadic" style of hspec.
This has been discussed on reddit here: http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framewor... Roman _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

It certainly can, but it doesn't do that yet. Should be very easy to
fix, though. Patches are welcome.
Roman
* Jan Stolarek
Sorry, my last email got sent too quickly. Anyway, continuing my thought. So QuickCheck can classify tests:
+++ OK, passed 100 tests (29% Short)
Can tasty display this classification info? That was a thing I missed a lot in test-framework and would probably motivate me to switch to tasty.
Janek
----- Oryginalna wiadomość ----- Od: "Roman Cheplyaka"
Do: haskell-cafe@haskell.org Wysłane: wtorek, 6 sierpień 2013 22:51:57 Temat: Re: [Haskell-cafe] ANNOUNCE: tasty, a new testing framework * John Wiegley
[2013-08-06 13:40:50-0500] > Roman Cheplyaka
writes: I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
It would be nice to see a comparison of the various test frameworks and why one might select one over another. I use hspec currently (which also integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what tasty might offer. And I particularly dislike writing tests inside of a gigantic list; I much prefer the "monadic" style of hspec.
This has been discussed on reddit here: http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framewor...
Roman
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

I'll add that as an issue on github then. I tried implementing this for test-framework and failed - the code was just too complicatd for me. It'll be intereseting to see whether tasty has simpler implementation :)
Janek
----- Oryginalna wiadomość -----
Od: "Roman Cheplyaka"
Sorry, my last email got sent too quickly. Anyway, continuing my thought. So QuickCheck can classify tests:
+++ OK, passed 100 tests (29% Short)
Can tasty display this classification info? That was a thing I missed a lot in test-framework and would probably motivate me to switch to tasty.
Janek
----- Oryginalna wiadomość ----- Od: "Roman Cheplyaka"
Do: haskell-cafe@haskell.org Wysłane: wtorek, 6 sierpień 2013 22:51:57 Temat: Re: [Haskell-cafe] ANNOUNCE: tasty, a new testing framework * John Wiegley
[2013-08-06 13:40:50-0500] > Roman Cheplyaka
writes: I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
It would be nice to see a comparison of the various test frameworks and why one might select one over another. I use hspec currently (which also integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what tasty might offer. And I particularly dislike writing tests inside of a gigantic list; I much prefer the "monadic" style of hspec.
This has been discussed on reddit here: http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framewor...
Roman
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Mon, Aug 05, 2013 at 09:48:39PM +0300, Roman Cheplyaka wrote:
I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of the box (through the standard packages), but it is very extensible, so you can write your own test providers.
Please see the home page for more information: http://documentup.com/feuerbach/tasty
Are there plans an equivalent of test-framework-th too? /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus I invented the term Object-Oriented, and I can tell you I did not have C++ in mind. -- Alan Kay

* Magnus Therning
On Mon, Aug 05, 2013 at 09:48:39PM +0300, Roman Cheplyaka wrote:
I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained).
Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of the box (through the standard packages), but it is very extensible, so you can write your own test providers.
Please see the home page for more information: http://documentup.com/feuerbach/tasty
Are there plans an equivalent of test-framework-th too?
I don't have any plans to do that myself, but I welcome anyone who cares to create and maintain such a package. donri on reddit said that porting test-framework-th should be a simple matter of changing one import. Roman
participants (8)
-
Andrey Chudnov
-
Carter Schonwald
-
Jan Stolarek
-
Joe Quinn
-
John Wiegley
-
Magnus Therning
-
Roman Cheplyaka
-
Tikhon Jelvis