Re: [Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

On Wed, Apr 28, 2010 at 1:24 AM, Richard G. wrote:

I think that formatted plain-text output would be much better than XML, something that is human-readable and relatively easy to parse via machine.  Something similar to the GHC error output would work well because developers are familiar with it.

I don't think we need to be limited to a single output format. It's a simple thing to have continuous integration (or cabal) invoke tests with a flag/option to output in a specific format. XML is useful because there are a number of mature tools that already expect xml -- we don't need to reinvent the wheel to get some of the capabilities that developers in other languages are enjoying if our tools use some of the same formats (despite the issues that may exist with those formats..). I like your suggestion for an emacs/dev-readable format, and it can coexist with xml and other "snazzier" outputs (such as the default format for test-framework, which uses many little tricks to draw and erase progress bars / etc.) --Rogan

Test <n>:<Result>    <Location>    <Error message>

E.g.,

Test 1:Passed    src/Some/File.hs:23

Test 2:Failed    src/Some/File.hs:27    Expecting `4'; received `5'.

Test 3:Error    src/Some/OtherFile.hs:39    Unexpected exception.

This would keep the complexity low in Cabal and allow for easy transformation to XML.

Richard G.

On 10-04-08 8:30 PM, Rogan Creswick wrote:

...

On Thu, Apr 8, 2010 at 5:53 AM, Duncan Coutts  wrote:

...

I think it's important to be able to convert into standard or custom formats. I've no idea if JUnit XML would make sense as the native format. It's plausible.

I hadn't really thought about cabal, itself, being a consumer for test results -- but I like your (Duncan's) points about defining a testing interface, and keeping it extensible.

For the record: I don't think junit xml is a good choice for a native format :), but I do think it's a good format to start with simply because there are many tools that can consume it already.

--Rogan

...

Duncan

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe