At the time of writing this document, we have 60 integration tests which tests abrt, libreport and satyr functionality in real world scenarios.
These tests are run nightly for each of the currently supported releases of Fedora and Red Hat Enterprise Linux. You can find the results on our public mirror.
Caution!
Don't run the test suite on your machine directly, use virtual machine or machine dedicated for the testing.
- In your virtual machine or dedicated machine:
- clone the ABRT repository:
git clone git://git.fedorahosted.org/abrt.git
- go to
abrt/tests/runtest/
- run
./run
- clone the ABRT repository:
This will run all the tests in order specified in ./aux/test_order
. When the testing
is done, the output can be found in /tmp/abrt-testsuite/
.
To override the destination
directory or the other options, you can edit ./aux/config.sh
.
To run only one of the tests, you have to run it via ./aux/runner.sh
.
This script prepares your system by restarting required services
and cleaning up possible stale files.
For example, to run dbus-api
test, you have to pass the
whole path to runtest.sh
to runner.sh
script:
./aux/runner.sh dbus-api/runtest.sh
To run only part of the tests, edit ./aux/test_order
file and comment out the tests you don't want to run.
For example, when using the test suite to test already installed packages,
you don't need to build and install git versions.
In this case, you can comment out abrt-nightly-build
test
and all the \*-make-check
tests. These are now disabled by default.
- To create a new test you have to:
- create a directory reflecting its name
- create two files in that directory:
- executable
runtest.sh
file PURPOSE
file
- executable
- add the test to
./aux/test_order
The tests are written in bash using BeakerLib library. BeakerLib Quick Reference [PDF] is what you need.
It's always a good idea to copy one of the existing tests as a base for your test. Good candidates are:
run-abrtd
which basically serves as an example,ccpp-plugin
if you need to test crash handling (if you require crash directory),dbus-api
if you need to test DBus functionality.
Please provide an explanation for complex tests. Either explain certain sections of the
test in comments in runtest.sh
or create README
file inside the directory of the test.
To avoid painful digging in several distinct files logging is now divided into several logical levels:
$OUTPUT_ROOT
variable is defined in aux/config.sh
which represents the root
for all logs.
For each stage of the test suite run there is corresponding directory in root directory. It contains stage.log file which contains output of script governing the stage.
/tmp/abrt-testsuite ├── format/ ├── post/ ├── pre/ ├── report/ ├── results └── test/
PRE
- install required packages, collect log files, collect system information
TEST
- run all the tests
FORMAT
- format the results for REPORT stage
REPORT
- report the results
POST
- collect logs, cleanup
For TEST
stage there is an additional subdirectory for each test case:
/tmp/abrt-testsuite/test/ ├── abrt-make-check ├── abrt-nightly-build ├── abrt-should-return-rating-0-on-fail ├── blacklisted-package ...
Each directory contains several files:
/tmp/abrt-testsuite/test/systemd-init/ ├── dmesg ├── avc ├── fail.log ├── full.log ├── messages └── protocol.log
Only full.log
is mandatory. It contains stdout and stderr of the test run.
protocol.log
only contains the protocol generated by BeakerLib. If the test
fails with FATAL error, protocol.log
is not generated. In case of other failures,
these are extracted to fail.log
along with line numbers pointing to lines in full.log
.
dmesg
, messages
and avc
each contains log file messages written during the test run.