On Sat, Mar 13, 2021 at 02:22:25PM -0700, amindfv@mailbox.org wrote:
On Sat, Mar 13, 2021 at 04:39:40PM +0500, Ignat Insarov wrote:
Note: I posted this [elsewhere] — I hope it is permissible to also post here. There seems to be no single right venue for Haskell questions at this time.
[elsewhere]: https://discourse.haskell.org/t/property-checks-and-postgresql/2078
So, I have a three layer stack: PostgreSQL data base, Haskell back end, PureScript front end. And there is a fault somewhere in this stack: sometimes some items present in the data base do not make it to the front end. A small fortune in programmer hours has been spent searching for the fault but there is no progress. No one can figure out even roughly where the fault is. Our last hope is to property check the whole thing, from end to end, verifying that any things put into the data base can be retrieved by the front end.
We have in place a fixture that creates a new PostgreSQL cluster, runs the back end on it and makes some scripted HTTP requests. We use it to run unit tests, such like _«if I make this `PUT` request and then that `GET` request to the same end point, I should get back the same thing as I put in»_. In principle it would not be a problem to make a property check out of this property. Practically, tearing down the cluster, building a new pristine one and initializing the data base anew takes seconds, so this way even a trivial property check would take minutes to run.
Do you know where the majority of the time is spent? I.e. can you just drop the schema without re-creating the cluster? I'd guess it'd be fast but I don't know your setup.
Another thing I heard being done is to selectively drop and re-create exactly the tables affected by a run of a property check. This seems challenging to automate though. How can I know in a general way if a table has been touched? And then, how can I re-create exactly the affected tables given that the data base initialization code is an opaque blob?
At $WORK we do this: property tests on PSQL data dropping data between tests, but we know which tables are modified so we can just truncate the affected ones.
I should add, too: it's not always desirable to truncate or start from a pristine database. Sometimes having data lying around that shouldn't affect your tests can be a good way of testing that that data truly doesn't affect the thing being tested. I.e. it's another method of fuzzing.
Tom