We have "installed tests", but don't provide an easy way to run them.
The protocol is very simple: each test must return 0 for success, 77 means
"skipped", anything else is an error. In addition, we want to print test
output only if the test failed.
I wrote this simple script. It is pretty basic, but implements the functions
listed above. Since it is written in python it should be easy to add option
parsing (like running only specific tests, or running unsafe tests, etc.)
I looked at the following alternatives:
- Ubuntu root-unittests: this works, but just dumps all output to the terminal,
has no coloring.
- @ssahani's test runner [2]
It uses the unittest library and the test suite was implented as a class, and
doesn't implement any of the functions listed above.
- cram [3,4]
cram runs our tests, but does not understand the "ignore the output" part,
has not support for our magic skip code (it uses hardcoded 80 instead),
and seems dead upstream.
- meson test
Here the idea would be to provide an almost-empty meson.build file under
/usr/lib/systemd/tests/ that would just define all the tests. This would
allow us to reuse the test runner we use normally. Unfortunately meson requires
a build directory and configuration to be done before running tests. This
would be possible, but seems a lot of effort to just run a few binaries.
[1] 242c96addb/debian/tests/root-unittests
[2] https://github.com/systemd/systemd-fedora-ci/blob/master/upstream/systemd-upstream-tests.py
[3] https://bitheap.org/cram/
[4] https://pypi.org/project/pytest-cram/Fixes#10069.
The extended testsuite only works with uid=0. It contains of several
subdirectories named "test/TEST-??-*", which are run one by one.
To run the extended testsuite do the following:
$ make all # Avoid the "sudo make" below building anything as root
$ cd test
$ sudo make clean check
...
make[1]: Entering directory `/mnt/data/harald/git/systemd/test/TEST-01-BASIC'
Making all in .
Making all in po
TEST: Basic systemd setup [OK]
make[1]: Leaving directory `/mnt/data/harald/git/systemd/test/TEST-01-BASIC'
...
If one of the tests fails, then $subdir/test.log contains the log file of
the test.
To debug a special testcase of the testsuite do:
$ make all
$ cd test/TEST-01-BASIC
$ sudo make clean setup run
QEMU
====
If you want to log in the testsuite virtual machine, you can specify
additional kernel command line parameter with $KERNEL_APPEND.
$ sudo make KERNEL_APPEND="systemd.unit=multi-user.target" clean setup run
you can even skip the "clean" and "setup" if you want to run the machine again.
$ sudo make KERNEL_APPEND="systemd.unit=multi-user.target" run
You can specify a different kernel and initramfs with $KERNEL_BIN and $INITRD.
(Fedora's or Debian's default kernel path and initramfs are used by default)
$ sudo make KERNEL_BIN=/boot/vmlinuz-foo INITRD=/boot/initramfs-bar clean check
A script will try to find your QEMU binary. If you want to specify a different
one you can use $QEMU_BIN.
$ sudo make QEMU_BIN=/path/to/qemu/qemu-kvm clean check
3762f8e316
