* Tim Docker [2012-10-05 12:19:40+1000]

Hi Roman,

This sounds like a great idea. As you suggest, I've put this kind of thing together in scripts many times, without a consistent framework.

I haven't tried it yet, but have one question. When you call a function like

goldenVsString

I assume one generally provides a relative path. what is the file path relative to? How does one arrange a package of haskell code and associated golden files, and reference the golden files from within the haskell so that the tree works from any location?

The paths are naturally relative to the working directory of the test executable. In my projects, I typically have ./test.hs ./tests/case1.golden ./tests/case2.golden ./tests/case3.golden And so test.hs refers to test cases by relative paths like "tests/case1.golden". Of course, that'll only work if you run test.hs from its directory, but it is the case most of the time anyway. Also, since paths are relative to the directory containing the project.cabal file, "cabal test" works, too. If, for some reason, you want to be able to run tests from any location, you could provide the path to golden paths e.g. via an environment variable. Roman

On 05/10/12 07:55, Roman Cheplyaka wrote:

...

I am glad to announce the first public release of test-framework-golden — a golden testing library.

Hackage: http://hackage.haskell.org/package/test-framework-golden GitHub: https://github.com/feuerbach/test-framework-golden

Golden tests are similar to unit tests (as implemented in HUnit), but the idea is to store the expected result in files (called «golden» files).

I was introduced to the idea of golden testing by Bohdan Vlasyuk at ZuriHac in 2010. Since then I've discovered that this is a nice approach and it is already used in variety of projects (e.g. ghc, haddock).

Surprisingly, not much is written about golden testing, and I've been unable to find any golden testing libraries for Haskell or any other programming language. Every project has its own ad-hoc implementation in Haskell/Python/Shell/etc.

The closest match on the market is Simon Michael's shelltestrunner. But to use it you have to expose the tested functionality via command line, which may be inconvenient.

So this is my attempt at a general golden testing library.

It consists of two modules. Test.Golden has a simple interface that helps you get started very quickly. Test.Golden.Advanced provides a very generic testing function that you can use to implement the testing system you dream about.

The library is integrated with test-framework, so you can use golden tests in addition to SmallCheck/QuickCheck/HUnit tests.

In future there's a plan to support some golden test management capabilities.

Roman

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe