Unit Testing

Unit tests for Vector2d

Note

There are a lot of unit testing frameworks for C++ programs (for example, Catch2). We are just going to something simple here.

We can use the basic assert statement together with a makefile target to automate testing.

Tip

assert will end the execution at the first failure. We could instead keep track of the number of failures and explicitly return that as the return value of main, with 0 signifying all tests passing.

We’ll do this with our Vector2d class.

The basic structure we will use is:

int main() {

    // create some vectors v1 and v2

    // do some operations on v1 and v2

    // assert the result

}

Ideally, we’d like to be able to capture the success of our test in a Bash script, so let’s start by checking what the return value of a program that failed an assert is

Consider the following:

#include <cassert>

int main() {

   assert(false);

}

Let’s build and run that, and then check the return code via:

echo $?

On my machine, I get 134, but the actual code may vary depending on the compiler and OS. The key however is that it is non-zero. This means that we can test for a non-zero return in a script.

Comparing vectors

To make life easier, we’d like to be able to check if two vectors are equal, using ==, and not equal, using !=.

So we want to overload those operators and add them to our class:

bool operator== (const Vector2d& vec) const {
    return x == vec.x && y == vec.y;
}

bool operator!= (const Vector2d& vec) const {
    return x != vec.x || y != vec.y;
}

The const after the argument list allows these to be used on a const Vector2d object.

These should be added directly to the class in vector2d.H.

Writing tests

Let’s write some basic tests:

Listing 60 unit_test_vector2d.cpp
#include "vector2d.H"
#include <cassert>

int main() {

    auto v1 = Vector2d(2, 4);
    auto v2 = Vector2d(-1, 7);

    // test addition

    std::cout << "testing addition" << std::endl;

    assert(v1 + v2 == Vector2d(1, 11));

    // test subtraction

    std::cout << "testing subraction" << std::endl;

    assert(v1 - v2 == Vector2d(3, -3));

    // test unary minus

    std::cout << "testing unary minus" << std::endl;

    assert(-v1 == Vector2d(-2, -4));

    // test not equal

    std::cout << "testing inequality" << std::endl;

    assert(v1 != v2);

    // test the setters and the default constructor

    v1.set_x(0.0);
    v1.set_y(0.0);

    std::cout << "testing setters" << std::endl;

    assert(v1 == Vector2d());

    std::cout << "all tests passed" << std::endl;
}

These tests cover most of the functionality we explicitly implemented in vector2d.H. As written, they should all pass.

Automating with make

We’d like to provide a way to automate these tests. Our first method will be via make that we can run on our own computer.

Let’s start with the general GNUmakefile we developed in our Makefiles section.

Note

There is a GNU make shell function that looks like it might work, but that actually expands the command and then tries to execute the output.

Here’s what the rule would look like:

testing: unit_test_vector2d
     ./unit_test_vector2d > /dev/null
     @if [ $$? == 0 ]; then echo "tests passed"; fi

After we run our command (we redirect stdout to /dev/null to make it quiet) we use an if-then statement to check the output. Note the @ at the start of the line makes it so make doesn’t print the command itself to the screen, just the output.

Here’s the full version:

Listing 61 GNUmakefile
# by default, make will try to build the first target it encounters.
# here we make up a dummy name "ALL" (note: this is not a special make
# name, it is just commonly used).

ALL: unit_test_vector2d

# find all of the source files and header files

SOURCES := $(wildcard *.cpp)
HEADERS := $(wildcard *.H)

# create a list of object files by replacing .cpp with .o

OBJECTS := $(SOURCES:.cpp=.o)

# a recipe for making an object file from a .cpp file
# Note: this makes every header file a dependency of every object file,
# which is not ideal, but it is safe.

%.o : %.cpp ${HEADERS}
	g++ -c $<

# explicitly write the rule for linking together the executable

unit_test_vector2d: ${OBJECTS}
	g++ -o $@ ${OBJECTS}

testing: unit_test_vector2d
	./unit_test_vector2d > /dev/null
	@if [ $$? -eq 0 ]; then echo "tests passed"; fi

try it…

Let’s make a test fail in our unit_test_vector2d.cpp and make sure that this still works.

Continuous integration and github

For a large project, with multiple developers, we often want to ensure that any changes that they make to the code still pass all of the tests before we merge it into the main git branch. This is called continuous integration. Github has a feature called Github actions that allow us to set up simple scripts that run on all pull requests.

We’ll look at how this can work – this is a high level overview of the process that we will experiment with. Usually one of the main developers of a large project will have already implemented the testing.

We’ll use this repository: https://github.com/phy504-sbu/unit_tests

There are a few places to look:

try it…

Let’s break the code and issue a pull request to see the action in action…

Tip

A Github action is used to build this lecture notes from the ReST source in the lecture note repo: https://github.com/zingale/phy504

This is that action: https://github.com/zingale/phy504/blob/main/.github/workflows/gh-pages.yml