Using the BoostTest Framework

Test Organization

The Boost Unit Testing Framework (UTF) organizes unit tests suites and test cases. Test cases are meant to verify the functionality of a small portion of a class or module, i.e. a constructor of a class or initialization function of a module. Test suites contain any number of test cases, ideally all pertaining to the same class or module being tested. For example, a suite should be constructed to organize all of the individual test cases of a specific class. Each function of that class should have a corresponding test case within that suite. UTF automatically handles registering tests to suites and executing tests within suites. Ideally, every source file written should have a corresponding test suite or at least every module.

Test files should be named test_suite_name.cpp, where suite_name is the corresponding test suite. For example, test_contain.cpp is a unit test file that contains the test suite contain. This eases the management of adding tests within CMakeLists.txt.

Simple Test Suite Implementation

The Boost UTF provides a number of macros to ease the burden of creating and organizing tests. The BOOST_AUTO_TEST_SUITE macro creates a new test suite with a given name. In the example below, BOOST_AUTO_TEST_SUITE( suite_name ) creates a test suite named "suite_name". Note that there are no quotation marks around the name, contradictory to defining strings in CPP. The macro already has the quotation marks included in its definition.

Similar to creating a suite, test cases are created through the BOOST_AUTO_TEST_CASE macro. In the example below, BOOST_AUTO_TEST_CASE( test_one ) creates a new test case within "suite_name" called "test_one". After the test case macro curly braces are necessary to enclose the scope of that test case, similar to a function definition. All variables created within a test case are local so make sure to free any dynamic memory before the test case completes.

The actual testing assertions are implemented through additional macros, which must be called within test cases. The Boost UTF provides three levels of assertions; warning, check, and requirement. If a warning assertion fails, a corresponding message is displayed depending on the macro used, but the test case will NOT fail. Check assertions will cause a test case to fail, but will proceed executing code afterwards. Requirement assertions will fail a test case and immediately return from the test case, instead of proceeding with the test case execution.

Common Test Macros

Macro	Arguments	Example
BOOST_CHECK	any predicate	BOOST_CHECK( some_var 10 );
BOOST_CHECK_EQUAL	any two comparable objects	BOOST_CHECK_EQUAL( some_var, another_var)
BOOST_CHECK_CLOSE	two floats and a tolerance	BOOST_CHECK_CLOSE( v1, v2, 0.0001 )
BOOST_CHECK_MESSAGE	predicate and a string	BOOST_CHECK_MESSAGE( i 1, "i should be 1, but is " << i )

The above macros, if failed, will cause a test case to fail as they are CHECK assertions. To change the level of any assertion, simply change CHECK to WARN or REQUIRE, for warning and requirement levels respectively. Special care should be taken with REQUIRE assertions, as if any dynamic memory is used it will not be free'd after a fail. The requirement assertions immediately return from the test case from which it was called. The full listing of available test macros can be found at http://www.boost.org/doc/libs/1_39_0/libs/test/doc/html/utf/testing-tools/reference.html.

Example:

#include <boost/test/unit_test.hpp>
/******************************************************************************
*                        "suite_name" BOOST TEST SUITE
*******************************************************************************
*   Tests:
*       suite_name/test_one
*       suite_name/test_two
******************************************************************************/

BOOST_AUTO_TEST_SUITE( suite_name )

BOOST_AUTO_TEST_CASE( test_one )
{
    int i=2;
    BOOST_WARN( sizeof(int) == sizeof(short) );
    BOOST_CHECK( i == 1 );
    BOOST_REQUIRE( i > 5 );
    BOOST_CHECK( i == 6 ); // will never reach this check
}

BOOST_AUTO_TEST_CASE( test_two )
{

}

BOOST_AUTO_TEST_SUITE_END()

/******************************************************************************
*                        "suite_name" BOOST TEST SUITE
******************************************************************************/

Compiling with CMake

The autotest folder in the repository contains a file called test_main.cpp, which every unit test should be compiled with. This method decreases the number of executables that are generated during the build. Globals and functions that should only be executed once (for all test suites) can be placed within test_main.cpp to ensure this.

Only a few modifications are necessary to ensure a new test suite is built properly. Within CMakeLists.txt, there is a section called Test Compilation which handles what test files will be built. The example code below builds test_contain.cpp and test_strm.cpp against the main test file, test_main.cpp. If a new file should be added, simply add the name of it to the list of already included files.

# *****************************************************************************
# Test Compilation
# *****************************************************************************

ADD_EXECUTABLE(test_main test_main.cpp 
                     test_contain.cpp 
                     test_strm.cpp )

Once a file has been added to the executable build list, the test suites or test cases need to be added through ADD_TEST. Boost UTF provides command line options to run individual suites or test cases. CTest captures all output from executing tests, so individually running tests is required to identify which tests fail. For example, if CMakeList.txt only contained ADD_TEST(test_main ${EXECUTABLE_OUTPUT_PATH}/test_main), all of the test suites compiled would run, but we couldn't identify which test cases fail. CMake sees this as a single test.

The example below contains one test suite (contain) and three test cases (points, polygon, polygon2). Each test needs a unique name which should reflect which test case is being run. The general format for naming a test should be test_suiteName_caseName. Individual test suites and cases are added by using the command-line parameter --run_test supplied to the test executable test_main. This argument requires a specific format where the suite name precedes the test case name by a forward slash, ex. --run_test=suiteName/caseName.

# *****************************************************************************
# General Unit Test Section
# *****************************************************************************

# contain Test Suite
ADD_TEST(test_contain_point     
         ${EXECUTABLE_OUTPUT_PATH}/test_main --run_test=contain/points )
ADD_TEST(test_contain_polygon   
         ${EXECUTABLE_OUTPUT_PATH}/test_main --run_test=contain/polygon )
ADD_TEST(test_contain_polygon2  
         ${EXECUTABLE_OUTPUT_PATH}/test_main --run_test=contain/polygon2 )