Using step definition code from Given and When contexts

Have you ever wanted to use a single step definition like I create file '(.*)' from Given and When contexts in a feature file like this?

Scenario: CreateFile
    When I create file 'hello.txt'
    Then ...

Scenario DeleteFile
    Given I create  file 'hello.txt'
    When I delete 'hello.txt'

Here’s how to have 1 method that implements both the Given and the When I create file '...'

Create a class like this:

public class GivenWhenAttribute : StepDefinitionBaseAttribute
{
readonly static StepDefinitionType[] types = new[] { StepDefinitionType.Given, StepDefinitionType.When };
public GivenWhen() : this(null)  {  }
public GivenWhen(string regex) : base(regex, types ) { }
public GivenWhen(string regex, string culture) : this(regex) { Culture = culture; }
}

Use it in in the [Binding] classes like this:

[GivenWhen("I create file '(.*)'")]
public void CreateFile(String Name) { File.Create(Name); }

Adding PACT Contract Testing to an existing golang code base

Here’s how to add a Contract Test to a Go microservice, a provider in pact terminology, using pact-go This post uses v1 of pact-go, as the V2 version is stil in beta.

Install PACT cli tools on your dev machine

A linux or dev-container based environment can just use the same approach as the CI pipeline documented later on in this post.

For a Windows dev machine, install the pact-cli tools like this:

  1. Use browser to download https://github.com/pact-foundation/pact-ruby-standalone/releases/download/v2.0.2/pact-2.0.2-windows-x86.zip
  2. unzip to c:\Repos
  3. Change PATH to include c:\repos\pact\bin
  4. Restart any editors or terminals
  5. Run go install gopkg.in/pact-foundation/pact-go.v1

Create a unit test to validate the service meets the contract/pact

Add unit test like below. Notice these settings and how the environment variables affect them. This helps when tests need to run both on the local development machine as well as on CI/CD machines.

Setting Effect
PublishVerificationResults Controls if pact should publish the verification result to the broker. For my local dev machine I dont publish. From the CI pipeline I do publish
ProviderBranch The name of the branch for this provider version.
ProviderVersion The version of this provider. My CI pipeline ensures variable PACT_PROVIDER_VERSION contains the unique build number. On my local machine its just set to 0.0.0 
package app
import (
    "fmt"
    "bff/itemrepository"
    "os"
    "strconv"
    "strings"
    "testing"

    "github.com/pact-foundation/pact-go/dsl"
    "github.com/pact-foundation/pact-go/types"
    "github.com/stretchr/testify/assert"
)

func randomItem(env string, name string) itemrepository.item{
    return itemrepository.item{
        Environment:               env,
        Name:                      name,
    }
}

func getEnv(name string, defaultVal string) string {
    tmp := os.Getenv(name)
    if len(tmp) > 0 {
        return tmp
    } else {
        return defaultVal
    }
}

func getProviderPublishResults() bool {
    tmp, err := strconv.ParseBool(getEnv("PACT_PROVIDER_PUBLISH", "false"))
    if err != nil {
        panic(err)
    }
    return tmp
}

func TestProvider(t *testing.T) {
    //Arrange: Start the service in the background.
    port, repo, _ := startApp(false)
    pact := &dsl.Pact{ Consumer: "MyConsumer", Provider: "MyProvider", }

    //Act: Let pact spin-up a mock client to verifie our service.
    _, err := pact.VerifyProvider(t, types.VerifyRequest{
            ProviderBaseURL:            fmt.Sprintf("http://localhost:%d", port),
            BrokerURL:                  getEnv("PACT_BROKER_BASE_URL", ""),
            BrokerToken:                getEnv("PACT_BROKER_TOKEN", ""),
            PublishVerificationResults: getProviderPublishResults(),
            ProviderBranch:             getEnv("PACT_PROVIDER_BRANCH", ""),
            ProviderVersion:            getEnv("PACT_PROVIDER_VERSION", "0.0.0"),
            StateHandlers: types.StateHandlers{
                "I have a list of items": func() error {
                    repo.Set("env1", []itemrepository.item{randomItem("env1", "tenant1")})
                        return nil
                },
            },
        })
    pact.Teardown()

    //Assert
    assert.NoError(t, err)
}

Ensure the CI pipeline runs the test and publishes verification results

For my CI builds, I run this to make sure the CI machine has the pact-cli tools installed
cd /opt
curl -fsSL https://raw.githubusercontent.com/pact-foundation/pact-ruby-standalone/master/install.sh | bash
export PATH=$PATH:/opt/pact/bin
go install github.com/pact-foundation/pact-go@v1
...
pipeline already runs the unit tests
...

Check `can-i-deploy` in the CD pipeline

Change the CD pipeline to

  • verify this version of the service has passed the contract test, Otherwise do not deploy to production
  • After successful deployment, inform broker which new version is running in production.

My CD pipeline runs on different machines, so it again has to ensure PACT is installed, just like the CI pipeline:

# pipeline variables
PACT_PACTICIPANT=MyProvider
PACT_ENVIRONMENT=production
PACT_BROKER=https://yourtenant.pactflow.io
PACT_BROKER_TOKEN=...a read/write token...preferably a system user to represent CI/CD actions...

# task to install PACT
cd /opt
curl -fsSL https://raw.githubusercontent.com/pact-foundation/pact-ruby-standalone/master/install.sh | bash
echo "##vso[task.prependpath]/opt/pact/bin"
...
...
# Task to check if the version of the build can be deployed to production
pact-broker can-i-deploy --pacticipant $PACT_PACTICIPANT --version $BUILD_BUILDNUMBER --to-environment $PACT_ENVIRONMENT --broker-base-url $PACT_BROKER --broker-token $PACT_BROKER_TOKEN
...
#Do whatever tasks you need to deploy the build to the environment
...
...
#Task to record the deployment of this version of the producer to the environment
pact-broker record-deployment --environment $PACT_ENVIRONMENT --version $BUILD_BUILDNUMBER --pacticipant $PACT_PACTICIPANT --broker-base-url $PACT_BROKER --broker-token $PACT_BROKER_TOKEN

Adding PACT Contract Testing to an existing TypeScript code base

I like Contract Testing! I added a contract test with PACT-js and Jest for my consumer like this:

Installing PACT

  1. Disable the ignore-scripts setting: npm config set ignore-scripts false
  2. Ensure build chain is installed. Most linux based CI/CD agents have this out of the box. My local dev machine runs Windows; according to the installation guide for gyp the process is:
    1. Install Python from the MS App store. This takes about 5 minutes.
    2. Ensure the machine can build native code. My machine had Visual Studio already so I just added the ‘Desktop development with C++’ workload using the installer from ‘Tools -> Get Tools and Features’ This takes about 15 – 30 minutes
    3. npm install -g node-gyp
  3. Install the PACT js npmn package: npm i -S @pact-foundation/pact@latest
  4. Write a unit test using either the V3 or V2 of the PACT specification. See below for some examples.
  5. Update your CI build pipeline to publish the PACT like this: npx pact-broker publish ./pacts --consumer-app-version=$BUILD_BUILDNUMBER --auto-detect-version-properties --broker-base-url=$PACT_BROKER_BASE_URL --broker-token=$PACT_BROKER_TOKEN

A V3 version of a PACT unit test in Jest

//BffClient is the class implementing the logic to interact with the micro-service.
//the objective of this test is to:
//1. Define the PACT with the microservice
//2. Verify the class communicates according to the pact

import { PactV3, MatchersV3 } from '@pact-foundation/pact';
import path from 'path';
import { BffClient } from './BffClient';

// Create a 'pact' between the two applications in the integration we are testing
const provider = new PactV3({
    dir: path.resolve(process.cwd(), 'pacts'),
    consumer: 'MyConsumer',
    provider: 'MyProvider',
});

describe('GET /', () => {
    it('returns OK and an array of items', () => {
        const exampleData: any = { name: "my-name", environment: "my-environment", };

        // Arrange: Setup our expected interactions. Pact mocks the microservice for us.
        provider
            .given('I have a list of items')
            .uponReceiving('a request for all items')
            .withRequest({method: 'GET', path: '/', })
            .willRespondWith({
                status: 200,
                headers: { 'Content-Type': 'application/json' },
                body: MatchersV3.eachLike(exampleData),
            });
        return provider.executeTest(async (mockserver) => {
            // Act: trigger our BffClient client code to do its behavior 
            // we configured it to use the mock instead of needing some external thing to run
            const sut = new BffClient(mockserver.url, "");
            const response = await sut.get()

            // Assert: check the result
            expect(response.status).toEqual(200)
            const data:any[] = await response.json()
            expect(data).toEqual([exampleData]);
        });
    });
});

A V2 version

import { Pact, Matchers } from '@pact-foundation/pact';
import path from 'path';
import { BffClient } from './BffClient';

// Create a 'pact' between the two applications in the integration we are testing
const provider = new Pact({
    dir: path.resolve(process.cwd(), 'pacts'),
    consumer: 'MyConsumer',
    provider: 'MyProvider',
});

describe('GET', () => {
    afterEach(() => provider.verify());
    afterAll(() => provider.finalize());

    it('returns OK and array of items', async () => {
        const exampleData: any = { name: "my-name", environment: "my-environment", };
        // Arrange: Setup our expected interactions. Pact mocks the microservice for us.
        await provider.setup()
        await provider.addInteraction({
            state: 'I have a list of items',
            uponReceiving: 'a request for all items',
            withRequest: { method: 'GET', path: '/',  },
            willRespondWith: {
                status: 200,
                headers: { 'Content-Type': 'application/json' },
                body: Matchers.eachLike(exampleData),
            },
        })

        // Act: trigger our BffClient client code to do its behavior 
        // we configured it to use the mock instead of needing some external thing to run
        const sut= new BffClient(mockserver.url, "");
        const response = sut.get()
        
        // Assert: check the result
        expect(response.status).toEqual(200)
        const data: any[] = await response.json()
        expect(data).toEqual([exampleData]);
    });
});

Contract Testing helps you deploy faster with more confidence

Many modern systems are made-up of lots of small components such a microservices and frontends. Each independently built, released and maintained by a bunch of different teams. This is really useful for scaling out your organisation and increasing the speed at which changes can be delivered to your customers.

For people involved in producing this kind of software, it poses a few challenges:

  1. Will a new version of component X still work with its neighboring components?
  2. What does component Y even expect of component X? What is the structure of the data X returns? What status codes and headers?
  3. What version of component X is running in which environment. What version of the interface does X have there?
  4. How do we start work on X even though we have no clue when Y will be available?
  5. If I write a mock for Y does it truly simulate its interface or am I blinded by my own assumptions?
  6. How can I quickly deploy and test X without having to spin up its neighboring components? Can I even achieve that in a large product?

Contract Testing solves these problems for you. Lets assume we have some micro-service X and a web front-end Y that communicates with it. The tools that implement Contract Testing let component Y document its expectations of the interface with X in a machine readable contract, frequently called a PACT.

How it works from Y’s point-of-view:

The unit/integration tests of Y define the PACT. Maybe its the same as the previous version, maybe it different. Both is fine.

They let the test framework dynamically generate a mock for X based on this PACT. Instead of needing to talking to a ‘real’ X, they talk to the mock.

The mock and Y’s own unit test together verify that Y really works according to the PACT. If the test fails, the team of Y has some fixing work to do and this process restarts. If the test passes, Y uploads the PACT to a broker and informs it which version of Y just validated successfully.

Whenever Y is deployed to production its CI/CD pipeline asks the broker if the current version of X in production has been validated against this PACT version. If not, Y’s deployment fails. If its fine, the pipeline informs the broker which version of Y is now deployed to production.

How it works from X’s point-of-view:

The unit/integration tests of X Start an instance of X running locally on some reachable network location.

It retrieves the PACT from the broker and uses the testing framework to generate a mock of Y. This mock sends requests to X based on the PACT and validates X’s responses match the PACT.

If the test fails, the team of X has some fixing work to do and this process restarts. If the test passed, X communicates to the broker that it was able to work with this version of the PACT.

Whenever X is deployed to production its CI/CD pipeline asks the broker if the current version of Y in production has been validated against this PACT version. If not, X’s deployment fails. If its fine, the pipeline informs the broker which version of X is now deployed to production.

Generating custom random inputs for your property based test in golang

Golang’s quick testing package is great for generating random input for your property based test. Sometimes you want to control how the input values are generated. Here’s how you can create a custom generator for the input parameters to your property based test.

Lets assume you want to test the function IsValid(string) with many random inputs …. but …. the input may only contain characters in range a-z, A-Z, and 0-9. Here’s how to do it:

func randomAlphaNumericString(output []reflect.Value, rnd *rand.Rand) {
    alphabet := []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")
    size := rnd.Intn(8192)
    v := reflect.New(reflect.TypeOf("")).Elem()
    v.SetString(randStringOfLen(rnd, size, alphabet))
    output[0] = v
}

//Generates a string of len n containing random characters from alphabet
func randStringOfLen(rnd *rand.Rand, n int, alphabet[]rune) string {
    b := make([]rune, n)
    for i := range b {
        b[i] = alphabet[rnd.Intn(len(alphabet))]
    }
    return string(b)
}

func TestMyMethod(t *testing.T) {
    propertyTest := func(input string) bool { return true == IsValue(input) }
    c := quick.Config{MaxCount: 1000, Values: randomAlphaNumericString}
    if err := quick.Check(propertyTest , &c); err != nil {
        t.Error(err)
    }
}

Lets break it down into steps:

Firstly we define our propertyTest to check all input strings are valid. So far nothing special. This function takes a single string input parameter.

The quick.Config struct has the Values member. This member lets you supply a function to generate whatever parameters propertyTest needs. In our case the randomAlphaNumericString function does that job.

The randomAlphaNumericString function generates a suitable random string and it stores it as a reflect.Value in the output array at the position where the propertyTest expects to receive a string parameter.

TypeScript logo

Property based testing in TypeScript with fast-check

In a previous post, I explained property based testing. In this post we’ll see a simple example using fast-check

Let assume we’re building the same bank transfer code as described in the dotnet FsCheck earlier post. Here’s the TypeScript version of the test:

import BuggyBank from "../buggy-bank"
import * as fc from 'fast-check';

function transferProperties(startBalanceA: bigint
        , startBalanceB: bigint
        , amount: bigint) : void {

    const bank = new BuggyBank()
    bank.balanceOfA = startBalanceA
    bank.balanceOfB = startBalanceB
    try {
        bank.transfer(amount)
    } catch {
        //Transfer failed
        const balanceAUnchanged = bank.balanceOfA == startBalanceA
        const balanceBUnchanged = bank.balanceOfB == startBalanceB
        expect(balanceAUnchanged).toBeTruthy()
        expect(balanceBUnchanged).toBeTruthy()
        return
    }
    //Transfer succeeded
    const balanceAIsNonNegative = bank.balanceOfA >= 0
    const balanceAChanged = bank.balanceOfA != startBalanceA
    const balanceBChanged = bank.balanceOfB != startBalanceB
    expect(balanceAIsNonNegative).toBeTruthy()
    expect(balanceAChanged).toBeTruthy()
    expect(balanceBChanged).toBeTruthy()
}

test('properties of a bank transfer must be correct', () => {
    const config = { numRuns : 10000 }
    //use this if you need to control the seed for the random number generator
    //const config = { numRuns : 10000, seed:123 }
    const property = fc.property(fc.bigIntN(32)
            , fc.bigIntN(32)
            , fc.bigIntN(32)
            , transferProperties)
    fc.assert(property,config)
})

I created the quick-n-dirty example like this:

mkdir fast-check-example
cd fast-check-example
npm init --yes
npm install typescript ts-node
echo "{}" > tsconfig.json
npm install --save-dev jest ts-jest @types/jest
npm install --save-dev fast-check
mkdir src
#Start vscode
code

The first time I ran the test, it detected a defect: The code allowed transferring zero amounts:

Property failed after 1 tests
{ seed: -1444529403, path: "0:2:0:0:1:0:5:1:3:0:0:1:1:0:1:2:2:0:0:0:0:0", endOnFailure: true }
Counterexample: [0n,0n,0n]
Shrunk 21 time(s)
...stack trace to relevant expect() line in code ....

After fixing that bug it detected another defect: Transfers succeed even when the source account’s balance is insufficient:

Property failed after 4 tests
{ seed: 1922422813, path: "3:0:0:1:0", endOnFailure: true }
Counterexample: [0n,0n,1n]
Shrunk 4 time(s)
...
Golang logo

Property based testing in golang with quick

In a previous prost, I explained property based testing. In this post we’ll see a simple example using golang’s built-in quick package.

Let assume we’re building the same bank transfer code as described in the dotnet FsCheck earlier post.

Here’s the golang equivalent of the test:

package goquickcheckexample

import (
	"testing"
	"testing/quick"
)

func TestProperties(t *testing.T) {
	bank := BuggyBank{}
	properties := func(StartBalanceA int, StartBalanceB int, TransferAmount int) bool {
		bank.BalanceOfAccountA = StartBalanceA
		bank.BalanceOfAccountB = StartBalanceB
		err := bank.Transfer(TransferAmount)
		if err != nil {
			//Transfer failed
			balancesChanged := (bank.BalanceOfAccountA != StartBalanceA) || (bank.BalanceOfAccountB != StartBalanceB)
			if balancesChanged {
				t.Log("Balances changed on failed transfer")
			}
			return !balancesChanged
		}
		//Transfer succeeded
		balanceAIsNonNegative := bank.BalanceOfAccountA >= 0
		balanceAChanged := bank.BalanceOfAccountA != StartBalanceA
		balanceBchanged := bank.BalanceOfAccountB != StartBalanceB
		if !balanceAIsNonNegative {
			t.Log("Balance of A ended negative")
		}
		if !balanceAChanged {
			t.Log("Balance of A did not change")
		}
		if !balanceBchanged {
			t.Log("Balance of B did not change")
		}
		return balanceAIsNonNegative && balanceAChanged && balanceBchanged
	}

	c := quick.Config{MaxCount: 1000000}
	if err := quick.Check(properties, &c); err != nil {
		t.Error(err)
	}
}

Note: If you want all test runs to use the same set of random numbers then use: c := quick.Config{MaxCount: 1000000, Rand: rand.New(rand.NewSource(0))}

When I ran the test, it detected a defect: Transfers succeed even when the source account’s balance is insufficient:

bank_test.go:28: Balance of A ended negative
bank_test.go:41: #2: failed on input 6319534437456565100, -3125004238116898490, 8226184717426742479

After fixing that bug, it detected a defect: The code allowed transferring negative amounts:

bank_test.go:34: Balance of A ended negative
bank_test.go:47: #22: failed on input 5995030153294015290, -7891513722668943486, -3464545538278061921

While analyzing this defect we notice yet another one: This code is not safe against integer overflow.

Magnifying Glass

Custom assertions in Jmeter

Recently I needed load test an HTTP endpoint to verify:

  1. A few simple fields match expected values.
  2. Length of an array somewhere inside the JSON response matches the expected length.

The first check is trivial in Jmeter with a ‘Json Assertion’

The second one was a little more tricky because the Json Assertion doesn’t support the length() operator. Luckily the ‘BeanShell Assertion’ comes to the rescue. This type of assertion lets us write our own logic to decide the pass/fail outcome.

Here’s a short example:

import com.jayway.jsonpath.Criteria;
import com.jayway.jsonpath.Filter;
import com.jayway.jsonpath.JsonPath;

Object aStringWithJson = new String(SampleResult.getResponseDataAsString());
List items = JsonPath.read(aStringWithJson , "$.path.to.my.array", new Filter[] {});
int expectedSize = Integer.parseInt(Parameters.trim());
int actualSize = items.size();
if( actualSize != expectedSize ) {
    Failure = true;
    FailureMessage = "Got '" + actualSize + "' expected '" + expectedSize + "'";
}

The script has access to a bunch of interesting global variables:

Variable Meaning
SampleResult The result of the request.
Parameters the parameter provided to the assertion in the GUI.
Failure Setting it to true causes Jmeter to fail the request/response pair.
FailureMessage Jmeter reports this string a the reason why the request/response pair failed.

Property based testing in dotnet with FsCheck

In a previous post, I explained property based testing. In this post we’ll see a simple example in dotnet with Fscheck

Lets assume we built code to transfer money from account A to account B. Some properties that comes to mind are:

  1. Successful transfer changes account balances.
  2. Successful transfer leaves source balance as non-negative.
  3. Failed transfer do not change account balances.

Our first implementation looks like this (don’t judge! … its supposed to be buggy)

class BuggyBank
{
    public int BalanceOfAccountA {get; set;} 
    public int BalanceOfAccountB {get; set;}
    
    public void TranserToAccountB(int Amount) 
    {
        BalanceOfAccountA -= Amount;
        BalanceOfAccountB += Amount;
    }
}

First we setup an empty dotnet unit test project like this:

mkdir FsCheckExample
cd FsCheckExample
dotnet new mstest
dotnet add package FsCheck --version 3.0.0-beta2

Then we add one test to validate all properties like this. Its vital to use QuickCheckThrowOnFailure() instead of QuickCheck() otherwise the test runner never reports failures.

    [TestMethod]
    public void MultiplePropertiesInOneTest()
    {
        Prop.ForAll<int, int, int>((StartBalanceForA, StartBalanceForB, AmountToTransfer) => {
            BuggyBank T = new()
            {
                BalanceOfAccountA = StartBalanceForA,
                BalanceOfAccountB = StartBalanceForB
            };
            try 
            {
                T.TranserToAccountB(AmountToTransfer);
            } 
            catch 
            {
                //The transfer failed
                bool BalancesUnchanged = (T.BalanceOfAccountA == StartBalanceForA && T.BalanceOfAccountB == StartBalanceForB);
                return BalancesUnchanged.Label("Failed transfer do not changes account balances.");
            }

            //Transfer succeeded
            bool BalancesChanged = T.BalanceOfAccountA != StartBalanceForA && T.BalanceOfAccountB != StartBalanceForB;
            bool NonNegativeBalanceOfA = T.BalanceOfAccountA >= 0;
            return
                NonNegativeBalanceOfA.Label("Successful transfer leaves source balance as non-negative")
                .And(BalancesChanged).Label("Successful transfer change account balances.")
            ;
            
        }).QuickCheckThrowOnFailure();
    }

When we run this test, FsCheck discovers the code allows transfers of 0 which violate the property that balances must change after successful transfer.

  Failed MultiplePropertiesInOneTest [674 ms]
  Error Message:
   Test method FsCheckExample.FsCheckExample.MultiplePropertiesInOneTest threw exception: 
System.Exception: Falsifiable, after 1 test (0 shrinks) (10302727913982160643,12688396740290863899)
Last step was invoked with size of 2 and seed of (10747404355721025928,11972757671557555113):
Label of failing property: Successful transfer change account balances.
Original:
(0, 0, 0)
with exception:
System.Exception: Expected true, got false.

After fixing that bug, FsCheck finds yet another bug! Transfers succeed even when the source account’s balance is insufficient:

  Failed MultiplePropertiesInOneTest [444 ms]
  Error Message:
   Test method FsCheckExample.FsCheckExample.MultiplePropertiesInOneTest threw exception:
System.Exception: Falsifiable, after 8 tests (4 shrinks) (16420426895032412258,4100991820053463935)
Last step was invoked with size of 9 and seed of (13117456436209885594,15897597155289983093):
Labels of failing property (one or more is failing):
Successful transfer change account balances.
Successful transfer leaves source balance as non-negative
Original:
(-2, -2, 2)
Shrunk:
(0, 0, 1)
with exception:
System.Exception: Expected true, got false.

Property based testing

Many test techniques rely on your assumptions where bugs are likely to be. In practice its really difficult to be complete and correct in your assumptions, so we end-up with defects in our product. For software used at scale, those defects are likely to cause real problems.

Property Based Testing avoids relying on assumptions. Instead it uses randomization to exhaustively check your product always meets some ‘property’ It often finds defects in the code as well as ambiguities in specifications.

Frequently used terminology

Term Meaning
Property Some fact or truth that always applies to your system. Some examples:

  • The total amount of money in a set of bank accounts is constant regardless of what transfers are executed.
  • The result of serializing and deserializing some object is identical to the initial object.
  • The result of squaring a number is zero or more.
Generator Code that generates random input suitable for your test. Some examples are:

  • Generate a random integer.
  • Generate a random string.
  • Generate an object whose members have random values suitable for their type.
  • Generate a HTTP GET request with a random url.
  • Generate a randomly ordered set of command BUT it never starts with command X
Shrinker Takes some generated input (usually one that caused a failure) and make it one size smaller. Test frameworks usually keep shrinking until they find the smallest input that still causes a failure. This really helps in analyzing the bug.

Are my tests still deterministic if we use randomization?

Yes, you control the source of randomness. As long as you use the same source with the same seed, tests on your machine run exactly the same as elsewhere.

What frameworks help me get this up and running?

Fscheck for DotNet.
quick (no Shrinking) or gopter (with Shrinking) for Golang.
fast-check for JavaScript / TypeScript.

Whats a good place to have these type of tests?

These tests need to run fast. You’ll frequently see them in the build/unit-test stages. Sometimes in the component- or integration test stages of a pipeline.