Nested KVM is very flaky as we learnt from our CI. Hence, let's avoid
KVM whenever we detect we are already running inside of KVM.
Maybe one day nested KVM is fixed, at which point we can turn this on
again, but for now let's simply avoid nested KVM, since reliable CI is
more important than quick CI, I guess.
And yes, avoiding KVM for our qemu runs does make things substantially
slower, but I think it's not a complete loss.
Inspired by @evverx' findings in:
https://github.com/systemd/systemd/pull/8701#issuecomment-380213302
This makes the script wait for the newly created partition to
show up before trying to put a filesystem on it, which should
prevent the tests from failing with the following error:
```
New situation:
Disklabel type: dos
Disk identifier: 0x3541a0ec
Device Boot Start End Sectors Size Id Type
/dev/loop6p1 2048 800767 798720 390M 83 Linux
/dev/loop6p2 800768 819199 18432 9M 83 Linux
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
The file /dev/loop6p1 does not exist and no size was specified.
make: *** [setup] Error 1
F: Failed to mkfs -t ext4
Makefile:4: recipe for target 'setup' failed
```
Let's be more restrictive when validating PID files and MAINPID=
messages: don't accept PIDs that make no sense, and if the configuration
source is not trusted, don't accept out-of-cgroup PIDs. A configuratin
source is considered trusted when the PID file is owned by root, or the
message was received from root.
This should lock things down a bit, in case service authors write out
PID files from unprivileged code or use NotifyAccess=all with
unprivileged code. Note that doing so was always problematic, just now
it's a bit less problematic.
When we open the PID file we'll now use the CHASE_SAFE chase_symlinks()
logic, to ensure that we won't follow an unpriviled-owned symlink to a
privileged-owned file thinking this was a valid privileged PID file,
even though it really isn't.
Fixes: #6632
This catches errors like "ninja not found", missing programs etc. early,
instead of silently ignoring them and trying to boot a broken VM.
In install_config_files(), allow some distro specific files to be absent
(such as /etc/sysconfig/init).
All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu()
and test_cleanup(), so move them into test_functions and only override
them in TEST-02-CRYPTSETUP.
Also provide a common test_run() which by default assumes that both QEMU
and nspawn tests are run. Particular tests which don't support either
need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this
way around to avoid accidentally forgetting to opt in, and to encourage
test authors to at least always support nspawn.
Automatic rebuilding is removed: it doesn't play well with ninja, because
ninja always writes logs, and even if nothing needs to be built, it will
make the log file owned by root. So let's just remove this, and say that
the user must always do the build first.
I'm also keeping make for the tests, because ninja doesn't play well with
sudo.
Since the build directory is arbitrary, it needs to be specified, e.g.
sudo make BUILD_DIR=/home/zbyszek/src/systemd/build1 -C test/TEST-01-BASIC/
If run_qemu() exits with non-zero, this either meant that QEMU was not
available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was
set (which then should be a FAIL).
Limit the exit code of run_qemu() to QEMU availability only, and track timeouts
separately through the new $TIMED_OUT variable, which is then checked in
check_result_qemu().
Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work
similarly.
There are many cgroups-related changes (thanks, @htejun!)
This commit will simplify testing a bit.
Use:
make run UNIFIED_CGROUP_HIERARCHY=yes to enable cgroup-v2
make run UNIFIED_CGROUP_HIERARCHY=no to enable cgroup-v1
systemd-udev generated an insane amount of log output at debug level.
It would break TEST-02-CRYPTSETUP by filling the overflowing the disk
(which seems to be a bug in itself!).
WARNING: Image format was not specified for
'/var/tmp/systemd-test.tGi3od/rootdisk.img' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write
operations on block 0 will be restricted. Specify the 'raw' format
explicitly to remove the restrictions.
Also use unsafe caching mode, we don't care about data integrity here.
Fixes:
$ cd test/TEST-07-ISSUE-1981/
$ sudo make clean setup run
...
timeout: failed to run command ‘systemd-nspawn’: No such file or directory
...
TEST RUN: https://github.com/systemd/systemd/issues/1981 [FAILED]
Makefile:10: recipe for target 'run' failed
make: *** [run] Error 1
* Use $ROOTLIBDIR/systemd always
* Don't pass $ROOTLIBDIR/systemd as the first argument:
$ cat /proc/1/cmdline
/lib/systemd/systemd/lib/systemd/systemd...
Fixes backward/forward incompatibility errors on spawning.
For example:
$ sudo make run
...
Failed to register machine: Cannot set property TasksMax, or unknown
property.
$ ../../systemd-nspawn --version
systemd 228
$ systemd-nspawn --version
systemd 225
Sempaphore containers are not booted with systemd, so machined is not
available, which makes nspawn bail. Just skip nspawn tests in such
environments.
[ -d /run/systemd/system ] is esentially what sd_booted(3) is doing,
but on Ubuntu 15.05, without 'systemd-container' installed, we also
need to check for the presence of the systemd-machined binary.
Fixes:
systemd-testsuite systemd[34]: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth
systemd-testsuite systemd[34]: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth
systemd-testsuite systemd[34]: user@0.service: Failed at step PAM spawning /lib/systemd/systemd: Operation not permitted
...
on Debian, Ubuntu
Fixes:
systemd-testsuite login[31]: cannot open login definitions /etc/login.defs [No such file or directory]
systemd-testsuite systemd[1]: Received SIGCHLD from PID 31 (login).
systemd-testsuite systemd[1]: Child 31 (login) died (code=exited, status=1/FAILURE)
systemd-testsuite systemd[1]: console-getty.service: Child 31 belongs to console-getty.service
systemd-testsuite systemd[1]: console-getty.service: Main process exited, code=exited, status=1/FAILURE
systemd-testsuite systemd[1]: console-getty.service: Changed running -> dead
on Debian/Ubuntu
Useful on other distros
libpam_modules installs modules into /lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/security
on Debian
Fixes:
systemd-testsuite login[36]: PAM unable to dlopen(pam_group.so): /lib/security/pam_group.so: cannot open shared object file: No such file or directory
systemd-testsuite login[36]: PAM adding faulty module: pam_group.so
systemd-testsuite login[36]: PAM unable to dlopen(pam_limits.so): /lib/security/pam_limits.so: cannot open shared object file: No such file or directory
systemd-testsuite login[36]: PAM adding faulty module: pam_limits.so
...
etc
Fixes:
systemd[39]: systemd-exit.service: Executing: /bin/kill -s 58 29
systemd[39]: systemd-exit.service: Failed at step EXEC spawning /bin/kill: No such file or directory
systemd[29]: Received SIGCHLD from PID 39 ((kill)).
systemd[29]: Child 39 ((kill)) died (code=exited, status=203/EXEC)
* remove journal flushing (systemd-journal-flush.service runs journalctl --flush on boot)
* use sh -c and PATH instead of @SYSTEMCTL@ expansion
* remove unnecessary semicolons etc
The basic setup for the well-known system and session buses is
now done in read-only files in ${datadir} (normally /usr/share).
See the NEWS entry for 1.9.18 for details.
http://cgit.freedesktop.org/dbus/dbus/tree/NEWS
Prefer asking pkg-config for the rootlibdir, to also support systems with
--enable-split-usr. Fall back to the hardcoded /usr/lib/systemd if that fails.
It tries to find a suitable QEMU binary and will use KVM if present.
We can now configure QEMU from outside with 4 variables :
- $QEMU_BIN : path to QEMU's binary
- $KERNEL_APPEND : arguments appended to kernel cmdline
- $KERNEL_BIN : path to a kernel
Default /boot/vmlinuz-$KERNEL_VER
- $INITRD : path to an initramfs
Default /boot/initramfs-${KERNEL_VER}.img
- $QEMU_SMP : number of CPU simulated by QEMU.
Default 1
(from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
$ cd test
$ sudo make check
will run all tests in the TEST-* subdirectories
$ cd test/TEST-01-BASIC
$ sudo make clean setup run
will run the different stages of the test for debugging purposes
Evgeny Vereshchagin