The sections below include information about executing tests in Ballerina.
Understand the test execution behavior
The following is the expected order of execution for setup and teardown functions of a test.
Understand the behavior during failures
BeforeSuitefunction fails, every other following function is skipped.
BeforeGroupsfunction fails, the tests belonging to that group, setup, and teardown functions specific to those tests will fail. The
AfterGroupsfunctions for that test group will be skipped. Other setup and teardown functions will be executed as expected. Tests belonging to other groups will not be affected.
BeforeEachfunction fails, every test will be skipped. Since
BeforeSuiteis already executed,
AfterSuitewill be executed. Even though the
BeforeGroupsfunction is executed prior to BeforeEach, the
AfterGroupsfunction will not be executed unless marked as
If a test function fails, none of the other functions get skipped.
beforetest function fails, the test function and the
afterfunction for that test will be skipped.
AfterEachfunction fails, every following
AfterEachand test function will get skipped.
alwaysRunproperty is enabled,
AfterSuitefunctions will run irrespective of the status of other functions.
Execute tests using CLI commands
Tests will be executed when you run the test command.
Execute all the tests in the current package with the following command.
$ bal test
Run tests for a group
List all the test groups in the package.
$ bal test --list-groups
Run only the tests belonging to the given group(s) in the current package.
$ bal test --groups <group_1>,<group_2>
Run the tests in the current package excluding the given group(s).
$ bal test --disable-groups <group_1>
Run selected tests
Run only the given test function(s) in the current package.
$ bal test --tests <test_function>
Run a given set of functions in the default module only.
$ bal test --tests PackageName:<test_function>
Run all the functions in the given module.
$ bal test --tests PackageName.<module_name>:*
Re-run failed tests
Run only the previously-failed test cases in the current package.
$ bal test --rerun-failed
Run selected data sets in data-driven tests
Data-driven tests can be executed using the
bal test command as any other test.
Run only the specified cases of a data set provided using the
# as the separator and append the case identifier to the end of the test function name.
$ bal test --tests <test_function>#Case1
Run only the previously-failed cases in a data set.
$ bal test --rerun-failed
Generate test report and code coverage
Generate an HTML test report without code coverage information. Also, dump the test results in the JSON format.
$ bal test --test-report
Dump only the test results in the JSON format.
$ bal test --code-coverage
Generate an HTML test report with code coverage information. Also, dump the test results in the JSON format.
$ bal test --test-report --code-coverage
Generate a JaCoCo XML test report with code coverage information. Also, dump the test results in the JSON format.
$ bal test --code-coverage --coverage-format=xml
Generate an HTML test report and a JaCoCo XML test report with code coverage information. Also, dump the test results in the JSON format.
$ bal test --test-report --code-coverage --coverage-format=xml
Exclude files or modules from the code coverage report. The exclusion list can be provided as a comma-separated list of absolute or relative paths. Glob patterns such as
** can be used in the paths.
The formats below can be used for excluding.
|./ or ./**||All the source files in the package.|
|./*||All the source files in the default module.|
|./generated or ./generated/**||All the source files under the |
|./modules or ./modules/**||All the source files under the |
|./modules/*/util.bal||All the |
|*.bal||All the Ballerina source files.|
|./*.bal||All the Ballerina source files in the default module.|
$ bal test --test-report --code-coverage --coverage-format=xml --excludes='./generated'
For more options of the test command, run the following.
$ bal test --help