Using QuickCheck to test against a database

Hi! I've just spent my evening trying to use QuickCheck properties to test the database accessing functions in a project I'm working on at the moment. I've got some stuff that's working, and I wanted to see what the cafe thought about it, and if anyone had any suggestions or critiscm that can help me make it even better! I got here by trying to figure out how I want to test my application. At first I saw 2 options: use a test database and assume it exists, or have each test setup the data it needs. I like the latter more, because it minimises dependence on the outside world. So I got down to writing some HUnit tests and then had a wave of inspiration... what if I didn't even define the data, but let QuickCheck do that for me? Here's what I came up with - note the code here is somewhat paraphrased for brevity: data DBState a = DBState { initDb :: IO () , entity :: a } This type stores an action that initialises the database with enough data such that it contains whatever 'entity' is. For example, 'DBState Person' knows how to insert a specific person into the database, and carries along an in-memory log. Continuing with the Person idea, I then went and made an Arbitrary instance for DBState Person: instance Arbitrary (DBState Person) where arbitrary = do l <- Person <$> arbitrary `suchThat` (not . null) return $ DBState (void $ insertPerson l) p where insertPerson p = doSql "INSERT INTO person (name) VALUES (?)" [ personName p l ] Then it's just a case of defining some properties: prop_allPersons = monadicIO $ do dbState <- pick arbitrary :: Gen [Person] fetched <- run $ initDb `mapM` dbState >> findAllPersons assert $ fetched `eqSet` (entity `map` dbState) where eqSet a b = all (`elem` a) b && all (`elem` b) a Voila! For any database, 'findAllPersons' returns all persons. Now... onto my own critiscisms with this. Firstly, way too much boiler plate. You have to remember to apply all of the states of your arbitrary instances, which is a pain, and guaranteed to be missed. At first I thought DBState looks like it's basically a writer/IO monad, but then I couldn't actually figure out the monoid to write to. In this case it's just [Person], but in more complex property we might need 2 types of entities (for example, to ensure a join worked correctly) which makes this list heterogenous. Secondly, the initDb action is sensitive to the order actions are sequenced. Presumably, I want to test against a database that's as accurate to production as possible, which means I want foreign keys enabled. If things are initialized in the wrong order, it violates the foreign key constraint, and it's a burden if people have to determine the correct order. One solution here is to make use of deferrable constraints, but support for these is flakey at best (and it doesn't work for bracketting tests with BEGIN; ROLLBACK). These problems just make the properties harder, but they don't seem to impede the quality or usefulness of the tests. I really want to make this work, because the idea of it is quite beautiful to me. Though that might be partly because this is my first time really using QuickCheck, so I'm getting the "oh wow this is sweeeet" effect at the same time ;) Would love to hear peoples thoughts! - Ollie

Hi!
This looks cool indeed.
On 2 December 2011 00:02, Oliver Charles
[snip] You have to remember to apply all of the states of your arbitrary instances, which is a pain, and guaranteed to be missed.
Why can't you define a helper function which runs the initDb action while picking the entity? Something like: pickDB = do DBState act e <- pick arbitrary; act; return e (placing calls to "monadicIO" and "run" appropriately, I am not familiar enough with the monadic API) Secondly, the initDb action is sensitive to the order actions are
sequenced.
Do you mean with respect to other initDb actions? Why is this? I thought you were using QuickCheck in order not to assume things about the state of the DB and that the necessary state is prepared solely by running the initDb action. Is this not the case then? Cheers, Ozgur

Secondly, the initDb action is sensitive to the order actions are sequenced.
Do you mean with respect to other initDb actions? Why is this? I thought you were using QuickCheck in order not to assume things about the state of the DB and that the necessary state is prepared solely by running the initDb action. Is this not the case then?
The initDb action is sensitive to the order in which *it* sequences things. With fk constraints there is a dependecy tree which must be respected. At least that's what I understood.

I just did cabal install cabal-install on a Mac running Mac OS 10.6.8 and got the eventual response [44 of 44] Compiling Main ( Main.hs, dist/build/cabal/cabal-tmp/Main.o ) Linking dist/build/cabal/cabal ... ld: warning: could not create compact unwind for .LFB3: non-standard register 5 being saved in prolog Installing executable(s) in /home/cshome/o/ok/.cabal/bin I also had this problem today: m% cabal install quickcheck Resolving dependencies... cabal: dependencies conflict: ghc-6.12.3 requires pretty ==1.0.1.2 however pretty-1.0.1.2 was excluded because ghc-6.12.3 requires pretty ==1.0.1.1 What's the procedure for wiping everything out and starting again?

The 'could not create compact unwind' message is a known (and still
outstanding) linking issue on OS X. It should be harmless - it refers
to the fact that OS X 10.6 uses "compact unwind info" for exceptions
instead of DWARF unwind information, when possible. The exact cause
isn't (yet) known. Generally this just means that the exception
information falls back to being DWARF-based.
If you want to blow away all your local packages, say:
$ rm -rf ~/.ghc
Note that will blow away every local package for every GHC version. If
you have multiple versions installed, you should look in that
directory first and then blow away the appropriate subdirectory (named
something like '<arch>-darwin-<ghc-ver>'.)
On Thu, Dec 1, 2011 at 7:07 PM, Richard O'Keefe
I just did cabal install cabal-install on a Mac running Mac OS 10.6.8 and got the eventual response [44 of 44] Compiling Main ( Main.hs, dist/build/cabal/cabal-tmp/Main.o ) Linking dist/build/cabal/cabal ... ld: warning: could not create compact unwind for .LFB3: non-standard register 5 being saved in prolog Installing executable(s) in /home/cshome/o/ok/.cabal/bin
I also had this problem today: m% cabal install quickcheck Resolving dependencies... cabal: dependencies conflict: ghc-6.12.3 requires pretty ==1.0.1.2 however pretty-1.0.1.2 was excluded because ghc-6.12.3 requires pretty ==1.0.1.1
What's the procedure for wiping everything out and starting again?
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Regards, Austin
participants (5)
-
austin seipp
-
Benjamin Edwards
-
Oliver Charles
-
Ozgur Akgun
-
Richard O'Keefe