* Jan Stolarek
Hi *,
I've been trying to write a testsuite using tasty-golden and I've run into two technical problems.
I'll start with some background. I'm working on singletons library that provides Template Haskell functionality. I would like to tests whether some particular TH splices compile at all and if they compile whether they produce expected output. My plan is to attempt compilation of a file with GHC. If compilation fails then the test fails. If it succeedes then output is compared with a golden file. I am directly calling goldenTest function from Test.Tasty.Golden.Internal. I wrote this function to produce the test output:
compileWithGHC :: IO (Either BSL.ByteString BSL.ByteString)
It runs GHC with some specified options and returns Left stderr (if compilation fails) or Right stdout (if it succeedes). This function is passed as third parameter to goldenTest (using liftIO). Then I have a comparing function:
cmp :: Either BSL.ByteString BSL.ByteString -> Either BSL.ByteString BSL.ByteString -> IO (Maybe String)
If it gets two Rights then it compares them. If the second param is left then the function returns Just stderr.
I've run into following problems:
1. When I try to run my test I get:
message threw an exception: MY_GOLDEN_FILE: hGetBufSome: illegal operation (handle is closed)
I suspect that lazy IO is getting in the way but I have no idea why exactly things go wrong.
That's a plausible explanations. File handles are closed when the ValueGetter monad is run. Try to add something like Control.Exception.evaluate $ Control.DeepSeq.force cmpResult to your comparison function. (I think I should build this forcing behaviour right into tasty-golden itself.) (Filed as https://github.com/feuerbach/tasty-golden/issues/7)
2. I had to hardcode the path to my golden file. Otherwise the file could not be found. I suspect the reason is that test are run somewhere in dist/ subdirectory whereas my golden files are located alongside my test files. What can I do to have the golden files located without giving specific subdirectories in the source code?
Golden files are looked up in an ordinary way — i.e. relative paths are interpreted relative to the test program's CWD. So, for example, if your test files are under 'test/' subdirectory of your project's main directory, and you want to run the test suite from your project's main directory, then your paths must look like "test/...". (Filed as https://github.com/feuerbach/tasty-golden/issues/8)
A bonus question. I redefined vgReadFile from Test.Tasty.Golden.Internal to return Either:
vgReadFileE :: FilePath -> ValueGetter r (Either BSL.ByteString BSL.ByteString) vgReadFileE path = (fmap Right . liftIO . BSL.hGetContents =<<) $ ValueGetter $ ContT $ \k -> bracket (openBinaryFile path ReadMode) hClose k
The only modification is the "fmap Right .". How can I return Left if opening the file fails?
First off, you don't have to do that (unless you have some special needs — if so, what are they?). If the file doesn't exist, tasty will catch the exception and handle it gracefully. If you really want to do it (or are just curious), the easiest way is to revert your addition of 'fmap Right' and replace the 'bracket' code with 'try' code. Roman