Configuring the test runner

Changing the number of scenarii for each proof

By default BlackBox will generate 100 scenarii per proof. You may want to increase this number if you want it to be quicker to find a failing scenario. Or you may want to decrease it if you write functional tests as it would take too much time.

To do so:

use Innmind\BlackBox\{
    Application,
    Runner\Load,
};

Application::new([])
    ->scenariiPerProof(1_000)
    ->tryToProve(Load::everythingIn('proofs/'))
    ->exit();

Disabling the shrinking

The shrinking is the process by which BlackBox will try to find the smallest input that make a scenario fail. You may want to disable it in some cases like writing functional tests as it may take too much time.

To do so:

use Innmind\BlackBox\{
    Application,
    Runner\Load,
};

Application::new([])
    ->disableShrinking()
    ->tryToProve(Load::everythingIn('proofs/'))
    ->exit();

Use exhaustive shrinking

By default when BlackBox finds an error in a proof it will shrink the values in order to find the smallest values that fails. But if the shrunk values changes the type of error (1) compared to the original failure then it will stop the shrinking process and return the previous shrunk values.

1. meaning it's no longer the same assertion that fails

If you still want BlackBox to continue shrinking even if the type of error changes, then you can do:

use Innmind\BlackBox\{
    Application,
    Runner\Load,
};

Application::new([])
    ->useExhaustiveShrinking()
    ->tryToProve(Load::everythingIn('proofs/'))
    ->exit();

Code coverage

You can record the code covered by your proofs and dump the report to a file like this:

use Innmind\BlackBox\{
    Application,
    Runner\Load,
    Runner\CodeCoverage,
};

Application::new([])
    ->codeCoverage(
        CodeCoverage::of('src/')
            ->dumpTo('coverage.clover'),
    )
    ->tryToProve(Load::everythingIn('proofs/'))
    ->exit();

Disable GitHub Action output

When it detects it's run inside a GitHub Action the framework groups each proof output to make the output more compact for large suites. It also adds annotations to quickly jump to each failing proof.

You can disable such behaviour like this:

use Innmind\BlackBox\{
    Application,
    Runner\Load,
};

Application::new([])
    ->mapPrinter(static fn($printer) => $printer->disableGitHubOutput())
    ->tryToProve(Load::everythingIn('proofs/'))
    ->exit();

Allow proofs to not make any assertions

By default BlackBox will fail a proof when a scenario did not make any assertion. This is to make sure proof are correctly written and none that make no assertions goes unnoticed.

However if your style of making assertions may not always lead to a proof making one, then you can disable this feature this way:

use Innmind\BlackBox\{
    Application,
    Prove,
};

Application::new([])
    ->allowProofsToNotMakeAnyAssertions()
    ->tryToProve(static function(Prove $prove) {
        yield $prove
            ->proof('Some proof')
            ->given(Set::of('some input'))
            ->test(static function($assert, $input) {
                try {
                    doSomething($input);
                    $assert->fail('It should throw');
                } catch (\Exception $e) {
                    // expected behaviour
                }
            });
    })
    ->exit();