SHTK_UNITTEST(3) BSD Library Functions Manual SHTK_UNITTEST(3)NAME
unittest — Utilities to implement test programs
LIBRARY
shtk_import unittest
DESCRIPTION
The unittest module provides a framework with which to implement test
programs.
A test program implemented with the unittest library is made up of a com‐
bination of standalone test cases and/or test cases within test fixtures.
The test program will exit with 0 (success) if and only if all tests
pass, and 1 (failure) if any single test fails. Skips and expected fail‐
ures are considered passes.
Program entry point
Test programs implemented using unittest must use the
shtk_unittest_main(3) function as their entry point. There are two mech‐
anisms to achieve this.
The first is to end the test program with the following definition of
main:
main() { shtk_unittest_main "${@}"; }
The second is to tell shtk(1) at build time to use shtk_unittest_main(3)
as the program's entry point:
$ shtk build -m shtk_unittest_main module_test.sh
In general, prefer the latter mechanism.
Asserts vs. expects
The unittest library provides a variety of helper check functions. One
such example is the ‘equal’ check, which allows the caller to compare two
values for equality and fail the test case with the appropriate error
message if the values differ.
Each check is offered in two flavors: an assert and an expect, which in
the previous example means we have access to the two corresponding func‐
tions assert_equal and expect_equal. Assert-style checks are fatal: when
the condition they test for is not met, the test case is immediately
aborted; instead, expect-style checks are not fatal: they record the cur‐
rently-running test case as failed but they let the test case continue
execution. Assert-style checks should be used to validate conditions
that must hold true in order to run the test (e.g. ensuring that the set‐
up of the test case succeeds); expect-style checks should be used for all
other cases.
Scoping
All functions provided by the unittest library are prefixed by
‘shtk_unittest_’ as you would expect from the coding practices of shtk.
However, such long prefix is inconvenient when writing test programs, as
test programs are a very special case of scripts in which a domain-spe‐
cific language makes sense.
Therefore, for simplicity reasons, test cases have access to shortened
names of the unittest functions without the ‘shtk_unittest_’ prefix. In
order words, a test case can simply call fail instead of having to call
shtk_unittest_fail.
Standalone test cases
Standalone test cases are defined by top-level functions whose name ends
with the ‘_test’ suffix and are registered as test cases with the
shtk_unittest_add_test(3) function. Such test cases are run indepen‐
dently of each other, with no common setup nor teardown code among them.
Test fixtures
Test fixtures are collections of related test cases that may share
optional setup and teardown methods. Test fixtures are defined by top-
level functions whose name ends with the ‘_fixture’ suffix and are regis‐
tered as fixtures with the shtk_unittest_add_fixture(3) function.
Within fixture fuctions, there may be a setup and a teardown method, and
there should be one or more test cases registered with
shtk_unittest_add_test(3). Each test within the fixture is executed
independently of each other, but the setup and teardown methods (if any)
are also executed with the test.
Test environment
The runtime engine (usually kyua(1)) is responsible for executing the
test program under a controlled directory and with a sanitized environ‐
ment.
However, due to the nature of shell-based test programs, unittest creates
one extra directory for each test case, and such directory becomes the
work directory of the test case. These directories are all created
within the directory in which the test program was executed. To ensure
that any files created by the test case remain within its own subdirec‐
tory as much as possible, the HOME and TMPDIR variables are set to point
to the individual test's work directory.
One time setup and teardown
Test programs may optionally define top-level one_time_setup and
one_time_teardown.
The code supplied in the one_time_setup function is executed exactly once
at the beginning of the test program and all state set up by the function
is shared across all tests.
The code supplied in the one_time_teardown function is executed exactly
once at the end of the test program and should be used to clean up any
state prepared by one_time_setup.
It is important to mention that these one-time setup and teardown rou‐
tines run in the parent directory of the executed test cases. In other
words: if the setup routine creates any file to be shared across all
tests, the tests will have access to such files by looking them up in
their parent directory.
Test results
Any test case can complete with one of the following results:
Expected failure (pass)
The test case detected a failure, but such failure is expected.
This condition is often use to denote test cases that are either
incomplete or test cases that exercise a known bug. See
shtk_unittest_set_expected_failure(3) for details.
Failure
The test case explicitly failed or it unexpectedly terminated. See
shtk_unittest_delayed_fail(3) and shtk_unittest_fail(3) for
details.
Pass The test case completed successfully without reporting any errors.
Skip (pass)
The test case determined that it cannot run through completion and
decided to abort early instead of reporting a false negative. See
shtk_unittest_skip(3) for details.
EXAMPLES
The most basic test program that can be written is the following, which
provides a simple test case for cp(1)'s -f flag:
shtk_import unittest
shtk_unittest_add_test cp_f_forces_override
cp_f_forces_override_test() {
echo "first" >first
echo "second" >second
chmod 555 second
# First make sure that cp without -f fails.
assert_command -s exit:1 -e ignore cp first second
# Now run a second attempt with -f and verify that the command
# exits successfully and it is silent.
assert_command cp -f first second
cmp -s first second || fail "source and destination do not match"
}
main() { shtk_unittest_main "${@}"; }
A more complex test program can use fixtures to group related test cases
and to provide common setup and teardown code for each of them:
shtk_import unittest
shtk_unittest_add_fixture cp
setup() {
# Create common files used by all test cases. Note that this runs
# once per test case, so state is not shared among them.
echo "first" >first
echo "second" >second
}
teardown() {
# Common cleanup to be executed at the end of the test case, no
# matter if it passes or fails.
rm -f first second
}
shtk_unittest_add_test override_existing_file
override_existing_file_test() {
assert_command cp first second
cmp -s first second || fail "source and destination do not match"
}
shtk_unittest_add_test f_forces_override
f_forces_override_test() {
chmod 555 second
assert_command -s exit:1 -e ignore cp first second
assert_command cp -f first second
cmp -s first second || fail "source and destination do not match"
}
}
main() { shtk_unittest_main "${@}"; }
Lastly, the most complex test program is depicted here, which includes a
combination of fixtures and test cases with one-time setup and teardown
routines:
shtk_import unittest
one_time_setup() {
... initialization code shared by all tests in the program ...
}
one_time_teardown() {
... clean up code shared by all tests in the program ...
}
shtk_unittest_add_fixture clients
clients_fixture() {
setup() {
... initialization code shared by all tests in the fixture ...
}
teardown() {
... cleanup code shared by all tests in the fixture ...
}
shtk_unittest_add_test add
add_test() {
... first test in the fixture ...
}
shtk_unittest_add_test modify
modify_test() {
... second test in the fixture ...
fail "And it fails"
}
}
shtk_unittest_add_test initialization
initialization_test() {
... standalone test not part of any fixture ...
skip "But cannot run due to some unsatisfied condition"
}
# Either do this or, preferably, pass -mshtk_unittest_main to
# "shtk build" when compiling the test program and don't define main.
main() { shtk_unittest_main "${@}"; }
SEE ALSOshtk(3), shtk_unittest_add_fixture(3), shtk_unittest_add_test(3),
shtk_unittest_delayed_fail(3), shtk_unittest_fail(3),
shtk_unittest_main(3), shtk_unittest_set_expected_failure(3),
shtk_unittest_skip(3)
Checks
shtk_unittest_assert_command(3), shtk_unittest_assert_equal(3),
shtk_unittest_assert_file(3), shtk_unittest_assert_not_equal(3),
shtk_unittest_expect_command(3), shtk_unittest_expect_equal(3),
shtk_unittest_expect_file(3), shtk_unittest_expect_not_equal(3)HISTORY
unittest first appeared in shtk 1.6.
BSD November 16, 2014 BSD
