Many test techniques rely on your assumptions where bugs are likely to be. In practice its really difficult to be complete and correct in your assumptions, so we end-up with defects in our product. For software used at scale, those defects are likely to cause real problems.
Property Based Testing avoids relying on assumptions. Instead it uses randomization to exhaustively check your product always meets some ‘property’ It often finds defects in the code as well as ambiguities in specifications.
Frequently used terminology
|Property||Some fact or truth that always applies to your system. Some examples:
|Generator||Code that generates random input suitable for your test. Some examples are:
|Shrinker||Takes some generated input (usually one that caused a failure) and make it one size smaller. Test frameworks usually keep shrinking until they find the smallest input that still causes a failure. This really helps in analyzing the bug.|
Are my tests still deterministic if we use randomization?
Yes, you control the source of randomness. As long as you use the same source with the same seed, tests on your machine run exactly the same as elsewhere.
What frameworks help me get this up and running?
Fscheck for DotNet.
quick (no Shrinking) or gopter (with Shrinking) for Golang.
Whats a good place to have these type of tests?
These tests need to run fast. You’ll frequently see them in the build/unit-test stages. Sometimes in the component- or integration test stages of a pipeline.