| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This tutorial will use the JUnit
Test Infected article as a starting point. We will be creating a library
to represent money, libmoney, that allows conversions between
different currency types. The development style will be “test a
little, code a little”, with unit test writing preceding coding.
This constantly gives us insights into module usage, and also makes
sure we are constantly thinking about how to test our code.
| 3.1 How to Write a Test | ||
| 3.2 Setting Up the Money Build Using Autotools | ||
| 3.3 Setting Up the Money Build Using CMake | ||
| 3.4 Test a Little, Code a Little | ||
| 3.5 Creating a Suite | ||
| 3.6 SRunner Output |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Test writing using Check is very simple. The file in which the checks are defined must include ‘check.h’ as so:
#include <check.h> |
The basic unit test looks as follows:
START_TEST (test_name)
{
/* unit test code */
}
END_TEST
|
The START_TEST/END_TEST pair are macros that setup basic
structures to permit testing. It is a mistake to leave off the
END_TEST marker; doing so produces all sorts of strange errors
when the check is compiled.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Since we are creating a library to handle money, we will first create an interface in ‘money.h’, an implementation in ‘money.c’, and a place to store our unit tests, ‘check_money.c’. We want to integrate these core files into our build system, and will need some additional structure. To manage everything we’ll use Autoconf, Automake, and friends (collectively known as Autotools) for this example. Note that one could do something similar with ordinary Makefiles, or any other build system. It is in the authors’ opinion that it is generally easier to use Autotools than bare Makefiles, and they provide built-in support for running tests.
Note that this is not the place to explain how Autotools works. If you need help understanding what’s going on beyond the explanations here, the best place to start is probably Alexandre Duret-Lutz’s excellent Autotools tutorial.
The examples in this section are part of the Check distribution; you don’t need to spend time cutting and pasting or (worse) retyping them. Locate the Check documentation on your system and look in the ‘example’ directory. The standard directory for GNU/Linux distributions should be ‘/usr/share/doc/check/example’. This directory contains the final version reached the end of the tutorial. If you want to follow along, create backups of ‘money.h’, ‘money.c’, and ‘check_money.c’, and then delete the originals.
We set up a directory structure as follows:
.
|-- Makefile.am
|-- README
|-- configure.ac
|-- src
| |-- Makefile.am
| |-- main.c
| |-- money.c
| `-- money.h
`-- tests
|-- Makefile.am
`-- check_money.c
|
Note that this is the output of tree, a great directory
visualization tool. The top-level ‘Makefile.am’ is simple; it
merely tells Automake how to process sub-directories:
SUBDIRS = src . tests |
Note that tests comes last, because the code should be testing
an already compiled library. ‘configure.ac’ is standard Autoconf
boilerplate, as specified by the Autotools tutorial and as suggested
by autoscan.
‘src/Makefile.am’ builds ‘libmoney’ as a Libtool archive,
and links it to an application simply called main. The
application’s behavior is not important to this tutorial; what’s
important is that none of the functions we want to unit test appear in
‘main.c’; this probably means that the only function in
‘main.c’ should be main() itself. In order to test the
whole application, unit testing is not appropriate: you should use a
system testing tool like Autotest. If you really want to test
main() using Check, rename it to something like
_myproject_main() and write a wrapper around it.
The primary build instructions for our unit tests are in ‘tests/Makefile.am’:
|
TESTS tells Automake which test programs to run for
make check. Similarly, the check_ prefix in
check_PROGRAMS actually comes from Automake; it says to build
these programs only when make check is run. (Recall that
Automake’s check target is the origin of Check’s name.) The
check_money test is a program that we will build from
‘tests/check_money.c’, linking it against both
‘src/libmoney.la’ and the installed ‘libcheck.la’ on our
system. The appropriate compiler and linker flags for using Check are
found in @CHECK_CFLAGS@ and @CHECK_LIBS@, values
defined by the AM_PATH_CHECK macro.
Now that all this infrastructure is out of the way, we can get on with development. ‘src/money.h’ should only contain standard C header boilerplate:
|
‘src/money.c’ should be empty, and ‘tests/check_money.c’
should only contain an empty main() function:
|
Create the GNU Build System for the project and then build ‘main’ and ‘libmoney.la’ as follows:
$ autoreconf --install $ ./configure $ make |
(autoreconf determines which commands are needed in order
for configure to be created or brought up to date.
Previously one would use a script called autogen.sh or
bootstrap, but that practice is unnecessary now.)
Now build and run the check_money test with make
check. If all goes well, make should report that our tests
passed. No surprise, because there aren’t any tests to fail. If you
have problems, make sure to see Supported Build Systems.
This was tested on the isadora distribution of Linux Mint GNU/Linux in November 2012, using Autoconf 2.65, Automake 1.11.1, and Libtool 2.2.6b. Please report any problems to check-devel AT lists.sourceforge.net.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Since we are creating a library to handle money, we will first create an interface in ‘money.h’, an implementation in ‘money.c’, and a place to store our unit tests, ‘check_money.c’. We want to integrate these core files into our build system, and will need some additional structure. To manage everything we’ll use CMake for this example. Note that one could do something similar with ordinary Makefiles, or any other build system. It is in the authors’ opinion that it is generally easier to use CMake than bare Makefiles, and they provide built-in support for running tests.
Note that this is not the place to explain how CMake works. If you need help understanding what’s going on beyond the explanations here, the best place to start is probably the CMake project’s homepage.
The examples in this section are part of the Check distribution; you don’t need to spend time cutting and pasting or (worse) retyping them. Locate the Check documentation on your system and look in the ‘example’ directory, or look in the Check source. If on a GNU/Linux system the standard directory should be ‘/usr/share/doc/check/example’. This directory contains the final version reached the end of the tutorial. If you want to follow along, create backups of ‘money.h’, ‘money.c’, and ‘check_money.c’, and then delete the originals.
We set up a directory structure as follows:
.
|-- Makefile.am
|-- README
|-- CMakeLists.txt
|-- cmake
| |-- config.h.in
| |-- FindCheck.cmake
|-- src
| |-- CMakeLists.txt
| |-- main.c
| |-- money.c
| `-- money.h
`-- tests
|-- CMakeLists.txt
`-- check_money.c
|
The top-level ‘CMakeLists.txt’ contains the configuration checks for available libraries and types, and also defines sub-directories to process. The ‘cmake/FindCheck.cmake’ file contains instructions for locating Check on the system and setting up the build to use it. If the system does not have pkg-config installed, ‘cmake/FindCheck.cmake’ may not be able to locate Check successfully. In this case, the install directory of Check must be located manually, and the following line added to ‘tests/CMakeLists.txt’ (assuming Check was installed under C:\\Program Files\\check:
set(CHECK_INSTALL_DIR "C:/Program Files/check")
Note that tests comes last, because the code should be testing
an already compiled library.
‘src/CMakeLists.txt’ builds ‘libmoney’ as an archive,
and links it to an application simply called main. The
application’s behavior is not important to this tutorial; what’s
important is that none of the functions we want to unit test appear in
‘main.c’; this probably means that the only function in
‘main.c’ should be main() itself. In order to test the
whole application, unit testing is not appropriate: you should use a
system testing tool like Autotest. If you really want to test
main() using Check, rename it to something like
_myproject_main() and write a wrapper around it.
Now that all this infrastructure is out of the way, we can get on with development. ‘src/money.h’ should only contain standard C header boilerplate:
|
‘src/money.c’ should be empty, and ‘tests/check_money.c’
should only contain an empty main() function:
|
Create the CMake Build System for the project and then build ‘main’ and ‘libmoney.la’ as follows for Unix-compatible systems:
$ cmake . $ make |
and for MSVC on Windows:
$ cmake -G "NMake Makefiles" . $ nmake |
Now build and run the check_money test, with either make
test on a Unix-compatible system or nmake test if on Windows using MSVC.
If all goes well, the command should report that our tests
passed. No surprise, because there aren’t any tests to fail.
This was tested on Windows 7 using CMake 2.8.12.1 and MSVC 16.00.30319.01/ Visual Studios 10 in February 2014. Please report any problems to check-devel AT lists.sourceforge.net.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The Test Infected article starts out with a Money class, and so
will we. Of course, we can’t do classes with C, but we don’t really
need to. The Test Infected approach to writing code says that we
should write the unit test before we write the code, and in
this case, we will be even more dogmatic and doctrinaire than the
authors of Test Infected (who clearly don’t really get this stuff,
only being some of the originators of the Patterns approach to
software development and OO design).
Here are the changes to ‘check_money.c’ for our first unit test:
|
A unit test should just chug along and complete. If it exits early,
or is signaled, it will fail with a generic error message. (Note: it
is conceivable that you expect an early exit, or a signal and there is
functionality in Check to specifically assert that we should expect a
signal or an early exit.) If we want to get some information
about what failed, we need to use some calls that will point out a failure.
Two such calls are ck_assert_int_eq (used to determine if two integers
are equal) and ck_assert_str_eq (used to determine if two null terminated
strings are equal). Both of these functions (actually macros) will signal an error
if their arguments are not equal.
An alternative to using ck_assert_int_eq and ck_assert_str_eq
is to write the expression under test directly using ck_assert.
This takes one Boolean argument which must be True for the check to pass.
The second test could be rewritten as follows:
ck_assert(strcmp (money_currency (m), "USD") == 0); |
ck_assert will find and report failures, but will not print any
user supplied message in the unit test result. To print a user defined
message along with any failures found, use ck_assert_msg. The first
argument is a Boolean argument. The remaining arguments support varargs
and accept printf-style format strings and arguments. This is especially
useful while debugging. For example, the second test could be rewritten as:
ck_assert_msg(strcmp (money_currency (m), "USD") == 0,
"Was expecting a currency of USD, but found %s", money_currency (m));
|
If the Boolean argument is too complicated to elegantly express within
ck_assert(), there are the alternate functions ck_abort()
and ck_abort_msg() that unconditionally fail. The second test inside
test_money_create above could be rewritten as follows:
if (strcmp (money_currency (m), "USD") != 0)
{
ck_abort_msg ("Currency not set correctly on creation");
}
|
For your convenience ck_assert, which does not accept a user supplied message, substitutes a suitable message for you. (This is also equivalent to passing a NULL message to ck_assert_msg). So you could also write a test as follows:
ck_assert (money_amount (m) == 5); |
This is equivalent to:
ck_assert_msg (money_amount (m) == 5, NULL); |
which will print the file, line number, and the message
"Assertion 'money_amount (m) == 5' failed" if
money_amount (m) != 5.
When we try to compile and run the test suite now using make
check, we get a whole host of compilation errors. It may seem a bit
strange to deliberately write code that won’t compile, but notice what
we are doing: in creating the unit test, we are also defining
requirements for the money interface. Compilation errors are, in a
way, unit test failures of their own, telling us that the
implementation does not match the specification. If all we do is edit
the sources so that the unit test compiles, we are actually making
progress, guided by the unit tests, so that’s what we will now do.
We will patch our header ‘money.h’ as follows:
|
Our code compiles now, and again passes all of the tests. However,
once we try to use the functions in libmoney in the
main() of check_money, we’ll run into more problems, as
they haven’t actually been implemented yet.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
To run unit tests with Check, we must create some test cases,
aggregate them into a suite, and run them with a suite runner. That’s
a bit of overhead, but it is mostly one-off. Here’s a diff for the
new version of ‘check_money.c’. Note that we include stdlib.h to
get the definitions of EXIT_SUCCESS and EXIT_FAILURE.
|
Most of the money_suite() code should be self-explanatory. We are
creating a suite, creating a test case, adding the test case to the
suite, and adding the unit test we created above to the test case.
Why separate this off into a separate function, rather than inline it
in main()? Because any new tests will get added in
money_suite(), but nothing will need to change in main()
for the rest of this example, so main will stay relatively clean and
simple.
Unit tests are internally defined as static functions. This means that the code to add unit tests to test cases must be in the same compilation unit as the unit tests themselves. This provides another reason to put the creation of the test suite in a separate function: you may later want to keep one source file per suite; defining a uniquely named suite creation function allows you later to define a header file giving prototypes for all the suite creation functions, and encapsulate the details of where and how unit tests are defined behind those functions. See the test program defined for Check itself for an example of this strategy.
The code in main() bears some explanation. We are creating a
suite runner object of type SRunner from the Suite we
created in money_suite(). We then run the suite, using the
CK_NORMAL flag to specify that we should print a summary of the
run, and list any failures that may have occurred. We capture the
number of failures that occurred during the run, and use that to
decide how to return. The check target created by Automake
uses the return value to decide whether the tests passed or failed.
Now that the tests are actually being run by check_money, we
encounter linker errors again we try out make check. Try it
for yourself and see. The reason is that the ‘money.c’
implementation of the ‘money.h’ interface hasn’t been created
yet. Let’s go with the fastest solution possible and implement stubs
for each of the functions in money.c. Here is the diff:
|
Note that we #include <stdlib.h> to get the definition of
NULL. Now, the code compiles and links when we run make
check, but our unit test fails. Still, this is progress, and we can
focus on making the test pass.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The functions to run tests in an SRunner are defined as follows:
void srunner_run_all (SRunner * sr, enum print_output print_mode);
void srunner_run (SRunner *sr, const char *sname, const char *tcname,
enum print_output print_mode);
|
Those functions do two things:
srunner_run_all will run all the defined test cases of all
defined suites except if the environment variables CK_RUN_CASE
or CK_RUN_SUITE are defined. If defined, those variables shall
contain the name of a test suite or a test case, defining in that way
the selected suite/test case.
srunner_run will run the suite/case selected by the
sname and tcname parameters. A value of NULL
in some of those parameters means “any suite/case”.
print_mode specified.
For SRunners that have already been run, there is also a separate printing function defined as follows:
void srunner_print (SRunner *sr, enum print_output print_mode); |
The enumeration values of print_output defined in Check that
parameter print_mode can assume are as follows:
CK_SILENTSpecifies that no output is to be generated. If you use this flag, you either need to programmatically examine the SRunner object, print separately, or use test logging (see section Test Logging.)
CK_MINIMALOnly a summary of the test run will be printed (number run, passed, failed, errors).
CK_NORMALPrints the summary of the run, and prints one message per failed test.
CK_VERBOSEPrints the summary, and one message per test (passed or failed)
CK_ENVGets the print mode from the environment variable CK_VERBOSITY,
which can have the values "silent", "minimal", "normal", "verbose". If
the variable is not found or the value is not recognized, the print
mode is set to CK_NORMAL.
CK_SUBUNITPrints running progress through the subunit test runner protocol. See ’subunit support’ under the Advanced Features section for more information.
With the CK_NORMAL flag specified in our main(), let’s
rerun make check now. The output from the unit test is as follows:
Running suite(s): Money 0%: Checks: 1, Failures: 1, Errors: 0 check_money.c:9:F:Core:test_money_create:0: Assertion 'money_amount (m)==5' failed: money_amount (m)==0, 5==5 FAIL: check_money ===================================================== 1 of 1 test failed Please report to check-devel AT lists.sourceforge.net ===================================================== |
Note that the output from make check prior to Automake 1.13 will
be the output of the unit test program. Starting with 1.13 Automake will
run all unit test programs concurrently and store the output in
log files. The output listed above should be present in a log file.
The first number in the summary line tells us that 0% of our tests passed, and the rest of the line tells us that there was one check in total, and of those checks, one failure and zero errors. The next line tells us exactly where that failure occurred, and what kind of failure it was (P for pass, F for failure, E for error).
After that we have some higher level output generated by Automake: the
check_money program failed, and the bug-report address given in
‘configure.ac’ is printed.
Let’s implement the money_amount function, so that it will pass
its tests. We first have to create a Money structure to hold the
amount, and then implement the function to return the correct amount:
|
We will now rerun make check and… what’s this? The output is now as follows:
Running suite(s): Money 0%: Checks: 1, Failures: 0, Errors: 1 check_money.c:5:E:Core:test_money_create:0: (after this point) Received signal 11 (Segmentation fault) |
What does this mean? Note that we now have an error, rather than a
failure. This means that our unit test either exited early, or was
signaled. Next note that the failure message says “after this
point”; This means that somewhere after the point noted
(‘check_money.c’, line 5) there was a problem: signal 11 (a.k.a.
segmentation fault). The last point reached is set on entry to the
unit test, and after every call to the ck_assert(),
ck_abort(), ck_assert_int_*(), ck_assert_str_*(),
or the special function mark_point(). For example, if we wrote some test
code as follows:
stuff_that_works (); mark_point (); stuff_that_dies (); |
then the point returned will be that marked by mark_point().
The reason our test failed so horribly is that we haven’t implemented
money_create() to create any Money. We’ll go ahead and
implement that, the symmetric money_free(), and
money_currency() too, in order to make our unit test pass again,
here is a diff:
|
| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This document was generated by brarcher on August 2, 2015 using texi2html 1.82.