Systemd/test
Zbigniew Jędrzejewski-Szmek a412ec5714 meson: fix git ls-files invocations during rebase
Normally ls-files prints the full path to files from the repo root. But when
$GIT_WORK_TREE is set, ls-files prints paths relative to the current
directory. When rebasing, $GIT_WORK_TREE is set in the commands executed from
'rebase -x'. This causes problems if meson config is touched and the meson
reconfigures itself. ($GIT_WORK_TREE shouldn't be relevant, since the paths that
ls-files reports don't depend on the work tree, but whatever.) Let's unset
GIT_WORK_TREE to avoid the issue.

$ (cd test; git --git-dir=$PWD/../.git ls-files ':/test/dmidecode-dumps/*.bin')
test/dmidecode-dumps/HP-Z600.bin
test/dmidecode-dumps/Lenovo-ThinkPad-X280.bin
test/dmidecode-dumps/Lenovo-Thinkcentre-m720s.bin

$ (cd test; GIT_WORK_TREE=$PWD/.. git --git-dir=$PWD/../.git ls-files ':/test/dmidecode-dumps/*.bin')
dmidecode-dumps/HP-Z600.bin
dmidecode-dumps/Lenovo-ThinkPad-X280.bin
dmidecode-dumps/Lenovo-Thinkcentre-m720s.bin

Fixes #18148.
2021-01-09 09:22:26 +09:00
..
TEST-01-BASIC test: update test_create_image 2021-01-01 21:33:16 +00:00
TEST-02-UNITTESTS test: switch TEST-02-CRYPTSETUP and TEST-24-UNITTESTS 2020-09-24 17:02:20 +02:00
TEST-03-JOBS test: unify quotation around image names 2020-05-17 10:43:08 +02:00
TEST-04-JOURNAL test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-05-RLIMITS test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-06-SELINUX TEST-06-*: also try the installation path for Debian 2020-03-31 14:37:20 +02:00
TEST-07-ISSUE-1981 test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-08-ISSUE-2730 test: rework how images are created 2020-03-28 11:51:29 +01:00
TEST-09-ISSUE-2691 test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-10-ISSUE-2467 test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-11-ISSUE-3166 test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-12-ISSUE-3171 test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-13-NSPAWN-SMOKE nspawn: implement container host os-release interface 2020-06-23 12:58:21 +01:00
TEST-14-MACHINE-ID test: unify quotation around image names 2020-05-17 10:43:08 +02:00
TEST-15-DROPIN test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-16-EXTEND-TIMEOUT test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-17-UDEV-WANTS test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-18-FAILUREACTION test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-19-DELEGATE test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-20-MAINPIDGAMES test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-22-TMPFILES test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-23-TYPE-EXEC test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-24-CRYPTSETUP test: switch TEST-02-CRYPTSETUP and TEST-24-UNITTESTS 2020-09-24 17:02:20 +02:00
TEST-25-IMPORT test: convert ubuntu-ci to use deny-list 2020-08-01 15:36:35 +02:00
TEST-26-SETENV test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-27-STDOUTFILE test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-28-PERCENTJ-WANTEDBY test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-29-UDEV-ID_RENAMING test: add test for device renaming issue #16967 2020-09-15 11:29:38 +09:00
TEST-30-ONCLOCKCHANGE test: convert ubuntu-ci to use deny-list 2020-08-01 15:36:35 +02:00
TEST-31-DEVICE-ENUMERATION test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-32-OOMPOLICY test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-33-CLEAN-UNIT test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-34-DYNAMICUSERMIGRATE test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-36-NUMAPOLICY test: s/MUMA/NUMA/ 2020-12-16 16:38:44 +01:00
TEST-37-RUNTIMEDIRECTORYPRESERVE test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-38-FREEZER test: add test for cgroup v2 freezer support 2020-04-30 19:02:55 +02:00
TEST-39-EXECRELOAD test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-40-EXEC-COMMAND-EX test: convert ubuntu-ci to use deny-list 2020-08-01 15:36:35 +02:00
TEST-41-ONESHOT-RESTART test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-42-EXECSTOPPOST test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-43-PRIVATEUSER-UNPRIV test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-44-LOG-NAMESPACE test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-46-HOMED test: de-duplicate test_setup() functions 2020-03-28 11:51:11 +01:00
TEST-47-ISSUE-14566 tree-wide: fix spelling errors 2020-04-21 23:21:08 +02:00
TEST-48-START-STOP-NO-RELOAD core: store timestamps of unit load attempts 2020-06-30 16:50:00 +02:00
TEST-49-UDEV-EVENT-TIMEOUT test: add integration test for udev event timeout 2020-06-05 11:09:21 +02:00
TEST-50-DISSECT test: fix TEST-50-DISSECT build on Ubuntu CI 2020-10-17 17:58:40 +01:00
TEST-51-ISSUE-16115 core: don't consider SERVICE_SKIP_CONDITION for abnormal or failure restarts 2020-06-10 17:12:55 +02:00
TEST-52-HONORFIRSTSHUTDOWN test: Add "finish" rule to .PHONY 2020-07-15 08:58:46 +02:00
TEST-53-ISSUE-16347 timer: Adjust calendar timers based on monotonic timer instead of realtime 2020-07-15 09:23:09 +02:00
TEST-54-CREDS test: add test suite for new credentials logic 2020-08-25 19:46:36 +02:00
TEST-55-UDEV-TAGS test: add test for new "sticky" tags logic 2020-09-01 17:40:12 +02:00
TEST-56-OOMD test: add basic memory pressure extended test for oomd 2020-10-22 17:04:22 -07:00
dmidecode-dumps test: add one more dmidecode dump 2020-12-17 09:21:31 +01:00
fuzz meson: fix git ls-files invocations during rebase 2021-01-09 09:22:26 +09:00
hwdb.d Rename udev's hwdb/ to hwdb.d/ 2019-10-10 00:53:09 +01:00
journal-data test-journal-importer: add a test case with broken input 2017-02-15 00:31:55 -05:00
mocks treewide: more portable bash shebangs 2020-03-05 17:27:07 +01:00
test-execute test-execute: make sure shell execs the child 2020-11-06 15:20:34 +01:00
test-network test-network: add tests for route advmss 2021-01-09 00:01:53 +09:00
test-network-generator-conversion test: convert TEST-35-NETWORK-GENERATOR into a unit test 2020-03-28 11:46:48 +01:00
test-path license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
test-path-util core/execute: fall back to execve() for scripts 2020-11-06 15:14:13 +01:00
test-resolve test-resolve: add a bunch more packets for testing 2020-04-17 18:29:15 +02:00
test-sysusers sysusers: look at login.defs when setting the default range to allocate users 2020-10-01 19:53:45 +02:00
test-umount test-umount: add a simple test for swap_list_get() 2018-03-16 10:12:50 +01:00
testsuite-04.units tests: replace the few remaining Type=simple with Type=exec 2020-10-22 11:05:17 +02:00
testsuite-06.units test: move most of TEST-06-* setup to static files 2020-03-28 11:46:47 +01:00
testsuite-08.units test: move TEST-08-ISSUE-2730 setup to static files and meson scripts 2020-03-28 11:46:47 +01:00
testsuite-10.units test: move TEST-10-ISSUE-2467 setup to static files 2020-03-28 11:46:47 +01:00
testsuite-11.units tests: replace the few remaining Type=simple with Type=exec 2020-10-22 11:05:17 +02:00
testsuite-16.units test/TEST-16: simplify how variables are set 2020-03-28 11:46:47 +01:00
testsuite-28.units test/TEST-28: avoid race in handling of /testok 2020-03-28 11:51:29 +01:00
testsuite-30.units/systemd-timedated.service.d test: move TEST-30-ONCLOCKCHANGE setup to static files 2020-03-28 11:46:47 +01:00
testsuite-52.units test: remove executable bit from testsuite-52.service 2020-08-27 12:13:16 +02:00
units tree-wide: fix typo 2020-12-16 18:30:15 +01:00
.gitignore
README.testsuite tests: add TEST_PREFER_NSPAWN variable to run as many as possible under nspawn 2021-01-01 21:33:16 +00:00
create-busybox-container tests/TEST-50: support the case when /etc/os-release is present 2020-08-27 13:08:14 +02:00
create-sys-script.py license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
hwdb-test.sh license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
meson.build meson: fix git ls-files invocations during rebase 2021-01-09 09:22:26 +09:00
mkosi.build.networkd-test networkd-test: add mkosi snippet for building and running networkd-tets.py in a network namespaced container 2018-12-07 12:25:24 +01:00
mkosi.default.networkd-test mkosi: make sure our mkosi files work with f33 2020-11-25 13:18:36 +01:00
mkosi.nspawn.networkd-test networkd-test: add mkosi snippet for building and running networkd-tets.py in a network namespaced container 2018-12-07 12:25:24 +01:00
networkd-test.py networkd-test: add final newlines in generated files, use .format() 2020-12-10 14:31:39 +09:00
rule-syntax-check.py license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
run-integration-tests.sh Do not run ninja in run-integration-tests.sh 2021-01-01 21:33:16 +00:00
run-unit-tests.py run-unit-tests: add option to run unsafe tests too 2018-09-24 15:42:50 +02:00
sd-script.py tree-wide: fix typo 2020-12-14 12:05:55 +00:00
splash.bmp
sys-script.py Merge pull request #17431 from msekletar/udev-link-update-race 2020-11-10 09:41:57 +01:00
sysv-generator-test.py license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
test-efi-create-disk.sh treewide: more portable bash shebangs 2020-03-05 17:27:07 +01:00
test-exec-deserialization.py license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
test-functions tests: add TEST_PREFER_NSPAWN variable to run as many as possible under nspawn 2021-01-01 21:33:16 +00:00
test-network-generator-conversion.sh tests: when creating temp dirs, include test name in path 2020-10-01 19:53:45 +02:00
test-sysusers.sh.in sysusers: look at login.defs when setting the default range to allocate users 2020-10-01 19:53:45 +02:00
testdata test: convert TEST-35-NETWORK-GENERATOR into a unit test 2020-03-28 11:46:48 +01:00
udev-dmi-memory-id-test.sh meson: make each dmidecode a separate test 2020-12-17 09:21:29 +01:00
udev-test.pl test/udev-test: gracefully exit when imports fail 2020-11-18 03:50:17 +09:00

README.testsuite

The extended testsuite only works with UID=0. It contains of several
subdirectories named "test/TEST-??-*", which are run one by one.

To run the extended testsuite do the following:

$ ninja -C build  # Avoid building anything as root later
$ sudo test/run-integration-tests.sh
ninja: Entering directory `/home/zbyszek/src/systemd/build'
ninja: no work to do.
--x-- Running TEST-01-BASIC --x--
+ make -C TEST-01-BASIC BUILD_DIR=/home/zbyszek/src/systemd/build clean setup run
make: Entering directory '/home/zbyszek/src/systemd/test/TEST-01-BASIC'
TEST-01-BASIC CLEANUP: Basic systemd setup
TEST-01-BASIC SETUP: Basic systemd setup
...
TEST-01-BASIC RUN: Basic systemd setup [OK]
make: Leaving directory '/home/zbyszek/src/systemd/test/TEST-01-BASIC'
--x-- Result of TEST-01-BASIC: 0 --x--
--x-- Running TEST-02-CRYPTSETUP --x--
+ make -C TEST-02-CRYPTSETUP BUILD_DIR=/home/zbyszek/src/systemd/build clean setup run

If one of the tests fails, then $subdir/test.log contains the log file of
the test.

To run just one of the cases:

$ sudo make -C test/TEST-01-BASIC clean setup run

Specifying the build directory
==============================

If the build directory is not detected automatically, it can be specified
with BUILD_DIR=:

$ sudo BUILD_DIR=some-other-build/ test/run-integration-tests

or

$ sudo make -C test/TEST-01-BASIC BUILD_DIR=../../some-other-build/ ...

Note that in the second case, the path is relative to the test case directory.
An absolute path may also be used in both cases.

Configuration variables
=======================

TEST_NO_QEMU=1
    Don't run tests under QEMU

TEST_QEMU_ONLY=1
    Run only tests that require QEMU

TEST_NO_NSPAWN=1
    Don't run tests under systemd-nspawn

TEST_PREFER_NSPAWN=1
    Run all tests that do not require qemu under systemd-nspawn

TEST_NO_KVM=1
    Disable QEMU KVM auto-detection (may be necessary when you're trying to run the
    *vanilla* QEMU and have both qemu and qemu-kvm installed)

TEST_NESTED_KVM=1
    Allow tests to run with nested KVM. By default, the testsuite disables
    nested KVM if the host machine already runs under KVM. Setting this
    variable disables such checks

QEMU_MEM=512M
    Configure amount of memory for QEMU VMs (defaults to 512M)

QEMU_SMP=1
    Configure number of CPUs for QEMU VMs (defaults to 1)

KERNEL_APPEND='...'
    Append additional parameters to the kernel command line

NSPAWN_ARGUMENTS='...'
    Specify additional arguments for systemd-nspawn

QEMU_TIMEOUT=infinity
    Set a timeout for tests under QEMU (defaults to infinity)

NSPAWN_TIMEOUT=infinity
    Set a timeout for tests under systemd-nspawn (defaults to infinity)

INTERACTIVE_DEBUG=1
    Configure the machine to be more *user-friendly* for interactive debuggung
    (e.g. by setting a usable default terminal, suppressing the shutdown after
    the test, etc.)

The kernel and initramfs can be specified with $KERNEL_BIN and $INITRD.
(Fedora's or Debian's default kernel path and initramfs are used by default)

A script will try to find your QEMU binary. If you want to specify a different
one with $QEMU_BIN.

Debugging the qemu image
========================

If you want to log in the testsuite virtual machine, you can specify additional
kernel command line parameter with $KERNEL_APPEND and then log in as root.

$ sudo make -C test/TEST-01-BASIC KERNEL_APPEND="systemd.unit=multi-user.target" run

Root password is empty.

Ubuntu CI
=========

New PR submitted to the project are run through regression tests, and one set
of those is the 'autopkgtest' runs for several different architectures, called
'Ubuntu CI'.  Part of that testing is to run all these tests.  Sometimes these
tests are temporarily deny-listed from running in the 'autopkgtest' tests while
debugging a flaky test; that is done by creating a file in the test directory
named 'deny-list-ubuntu-ci', for example to prevent the TEST-01-BASIC test from
running in the 'autopkgtest' runs, create the file
'TEST-01-BASIC/deny-list-ubuntu-ci'.

The tests may be disabled only for specific archs, by creating a deny-list file
with the arch name at the end, e.g.
'TEST-01-BASIC/deny-list-ubuntu-ci-arm64' to disable the TEST-01-BASIC test
only on test runs for the 'arm64' architecture.

Note the arch naming is not from 'uname -m', it is Debian arch names:
https://wiki.debian.org/ArchitectureSpecificsMemo

For PRs that fix a currently deny-listed test, the PR should include removal
of the deny-list file.

In case a test fails, the full set of artifacts, including the journal of the
failed run, can be downloaded from the artifacts.tar.gz archive which will be
reachable in the same URL parent directory as the logs.gz that gets linked on
the Github CI status.

To add new dependencies or new binaries to the packages used during the tests,
a merge request can be sent to: https://salsa.debian.org/systemd-team/systemd
targeting the 'upstream-ci' branch.