By 'test size reduction' I mean the 'shrink' function. It seems to me (as I said, first Haskell program and no experience with quickcheck) that it works nicely with a topdown generation of the test, but I don't see how to easily generate correct programs that way.

Even if you cannot release your tests, maybe you can help me with a very simple case. Consider a trivial AST. A program is a possibly nested block. Each block is a bunch of declarations of variables and some use of them

data Block = Block [Declaration] [Statement]

data Declaration = Var String String

data Statement = Statement Block | Use String

How one would generate things like:

Block [Var "a" "int"] [Use "a"] -- here a is declared in the same block

Block [] [Statement Block [Var "s" "int"] [Statement Block [] [Use "s"]]] -- here s is declared in some other visible scope

etc.

Or am I trying to approach the problem from the wrong angle?

 

On Mon, Feb 16, 2015 at 12:02 PM, Andrey Chudnov <achudnov@gmail.com> wrote:

I use QuichCheck for compiler testing where I generate random, but well-formed programs and check some high-level syntactic properties on results. The QuickCheck instance is open-source (see language-ecmascript), but the compiler-test code is closed-source at this time. Still, I found that it's not the ultimate answer: many properties are hard to formalize, so I have to resort to unit tests. I'm not sure what you mean by "how this would play with test size reduction". I think it's worth giving a try, but keep in mind that you might still need to use unit tests.

Let me know if you have any questions.

/Andrey

On 02/16/2015 11:53 AM, Maurizio Vitale wrote:

I'm starting to work on my first real haskell program (I've only RWH exercises under my belt) and wondering whether people use quickcheck at all for compiler testing.

I've seen uses of quickcheck for testing parsers, but I'm interested in generating well-formed programs (e.g. random programs with all declarations in reasonable random places). This could be used to test passes other than parsing (or even parsing, for languages that needs to distinguish identifiers, like the 'typedef' problem in C/C++).

The only thing I can think of, is to use quickcheck for randomly generating statements, go over them and figure out free variables (incl. functions) and generate declarations in random places for them. But I'm not sure how this would play with test size reduction and doesn't look like a nice solution anyhow.

Any idea or pointers to examples? or should I give up on quickcheck for this and just do direct testing?

Thanks,

  Maurizio

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe