Prompted by systemd/systemd#16111.
* check if /var is a mountpoint - if not, something went wrong. In case
of systemd/systemd#16111 the /failed file was created, because
systemd-cryptsetup failed, but it ended up being empty, making the result
check incorrectly pass
* forward journal messages to console - if we fail to mount /var,
journald won't flush logs to the persistent storage and we end up
empty handed and with no clue what went wrong
For example, without systemd/systemd#16111 and with this patch:
...
[FAILED] Failed to start systemd-cryptsetup@varcrypt.service.
See 'systemctl status systemd-cryptsetup@varcrypt.service' for details.
[DEPEND] Dependency failed for cryptsetup.target.
...
[ 3.882451] systemd-cryptsetup[581]: Key file /etc/varkey is world-readable. This is not a good idea!
[ 3.883946] systemd-cryptsetup[581]: WARNING: Locking directory /run/cryptsetup is missing!
[ 3.884846] systemd-cryptsetup[581]: Failed to load Bitlocker superblock on device /dev/disk/by-uuid/180ba5ef-873b-4018-9968-47c23431f71a: Invalid argument
...
[ 4.099451] sh[606]: + mountpoint /var
[ 4.100025] sh[603]: + systemctl poweroff --no-block
[ 4.101636] systemd[1]: Finished systemd-user-sessions.service.
[ 4.102598] sh[608]: /var is not a mountpoint
[FAILED] Failed to start testsuite-02.service.
dm-verity support in dissect-image at the moment is restricted to GPT
volumes.
If the image a single-filesystem type without a partition table (eg: squashfs)
and a roothash/verity file are passed, set the verity flag and mark as
read-only.
After today's Arch Linux image update, there seems to be a slight change
in delay reporting which breaks some asserts in
systemd-networkd-tests.py:
Expected:
limit 100 delay 50.0ms 10.0ms loss 20%
Current:
limit 100 delay 50ms 10ms loss 20%
Note that run_test() calls coredumpctl in a loop because in certain
environments (1 vCPU unaccelerated QEMU VM) it might take quite a
while to process the coredump.
This test runs fine locally (both on Qemu and nspawn) but sporadically fails on
autopkgtest for some reason.
Disable it while the issue is investigated to reduce noise.
The time-based cache allows starting a new unit without an expensive
daemon-reload, unless there was already a reference to it because of
a dependency or ordering from another unit.
If the cache is out of date, check again if we can load the
fragment.
In DHCPv6-PD environment, where WAN interface requests IPv6 via DHCPv6,
receives the address as well as delegated prefixes, with LAN interfaces
serving those delegated prefixes in their router advertisement messages.
The LAN interfaces on the router themselves do not have
the IPv6 addresses assigned by networkd from the prefix it
serves on that interface. Now this patch enables it.
Let's create new images public by default and then symlink/copy them
into the respective private directories afterwards, not the other way
around. This should fix a nasty race condition in parallel runs where
one tests attempts to copy the backing public image at the same moment
another test is already modifying it.
The test currently doesn't actually test configure-without-carrier since
it does have carrier for the entire test. It now forces carrier down
before starting the network portion of the test. Also, it tests to verify
the configuration is retained across future carrier losses/gains.
This allows users to configure a subnet id that should be used instead
of automatically (sequentially) assigned subnets. The previous attempt
had the downside that the subnet id would not be the same between
networkd restarts. In some setups it is desirable to have predictable
subnet ids across restarts of services and systems.
The code for the assignment had to be broken up into two pieces. One of
them is the old (sequential) assignment of prefixes and the other is the
new assignment based on configured subnet ids. The new assignment code
has to be executed first and has to be taken into account when (later
on) allocating the "old" subnets from the same pool.
Instead of having one iteration through the links we are now trying to
allocate a prefix for every link on every delegated prefix, unless they
received an assignment in a previous iteration.
Defines how link-local and autoconf addresses are generated.
0: generate address based on EUI64 (default)
1: do no generate a link-local address, use EUI64 for addresses generated
from autoconf
2: generate stable privacy addresses, using the secret from
stable_secret (RFC7217)
3: generate stable privacy addresses, using a random secret if unset
In the linked reproducer, m->fields_size == 0, and we calculate ri == -1, which
of course doesn't end well. Skip the whole calculation if m->fields_size == 0,
and also check that we don't go negative even if it is non-zero.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19446 and #15583.
Support running tests in parallel by switching to copying of the
base image instead of symlinking it..
This still requires some setup steps, like running `make setup` on tests
which have unique $IMAGE_NAME beforehand (and sequentially), otherwise
they'll all try to create the same base image when started in parallel,
leading to nasty issues. However, as running the integration tests in
parallel is such an unusual use case it should be good enough, for now.
Fixes: oss-fuzz#22208
```
test/fuzz/fuzz-calendarspec/oss-fuzz-22208... ../src/shared/calendarspec.c:666:48: runtime error: signed integer overflow: 2147000000 + 1000000 cannot be represented in type 'int'
#0 0x7f0b9f6cc56a in prepend_component ../src/shared/calendarspec.c:666
#1 0x7f0b9f6cd03a in parse_chain ../src/shared/calendarspec.c:718
#2 0x7f0b9f6cea1c in parse_calendar_time ../src/shared/calendarspec.c:845
#3 0x7f0b9f6d1397 in calendar_spec_from_string ../src/shared/calendarspec.c:1084
#4 0x401570 in LLVMFuzzerTestOneInput ../src/fuzz/fuzz-calendarspec.c:17
#5 0x401ae0 in main ../src/fuzz/fuzz-main.c:39
#6 0x7f0b9e31b1a2 in __libc_start_main (/lib64/libc.so.6+0x271a2)
#7 0x40122d in _start (/home/fsumsal/repos/systemd/build/fuzz-calendarspec+0x40122d)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../src/shared/calendarspec.c:666:48 in
```
Link groups are similar to port ranges found in managed switches.
You can add network interfaces to a numbered group and perform operations
on all the interfaces from that group at once.
Check:
- There is only 3 messages logged with type stdout
- Check all messages logged does not have new line: LINE_BREAK=eof
- Check that the 3 messages are logged from a different PID
- Check the 3 MESSAGE= content
As documented in systemd.path(5):
When a service unit triggered by a path unit terminates (regardless
whether it exited successfully or failed), monitored paths are
checked immediately again, and the service accordingly restarted
instantly.
This commit implements this behaviour for PathExists=, PathExistsGlob=,
and DirectoryNotEmpty=. These predicates are essentially
"level-triggered": the service should be activated whenever the
predicate is true. PathChanged= and PathModified=, on the other hand,
are "edge-triggered": the service should only be activated when the
predicate *becomes* true.
The behaviour has been broken since at least as far back as commit
8fca6944c2 ("path: stop watching path specs once we triggered the target
unit"). This commit had systemd stop monitoring inotify whenever the
triggered unit was activated. Unfortunately this meant it never updated
the ->inotify_triggered flag, so it never rechecked the path specs when
the triggered unit deactivated.
With this commit, systemd rechecks all paths specs whenever the
triggered unit deactivates. If any PathExists=, PathExistsGlob= or
DirectoryNotEmpty= predicate passes, the triggered unit is reactivated.
If the target unit is activated by something outside of the path unit,
the path unit immediately transitions to a running state. This ensures
the path unit stops monitoring inotify in this situation.
With this change in place, commit d7cf8c24d4 ("core/path: fix spurious
triggering of PathExists= on restart/reload") is no longer necessary.
The path unit (and its triggered unit) is now always active whenever
the PathExists= predicate passes, so there is no spurious restart when
systemd is reloaded or restarted.
Systems where a mount point is expected to be read-write needs a way to
fail mount units that fallback as read-only.
Add a property to allow setting the -w option when calling mount(8).
As Debian/Ubuntu use /lib/systemd instead of /usr/lib/systemd,
add systemd-journal-remote to the list of programs that test-functions
detects the correct path to, and replace its direct usage with
$SYSTEMD_JOURNAL_REMOTE
Also use $JOURNALCTL instead of journalctl.
Also minor correction in install_plymouth() to look in /lib/... as
well as /usr/lib/... and /etc/...
Remove the artifact files indicating test result (testok, failed, and
skipped) just before running the test so we always get the latest and
most relevant result instead of incorrectly consuming previous results.
Discovered in https://github.com/systemd/systemd/pull/15378#issuecomment-616801873
The disk attributes can take some time to update on certain filesystems,
so let's strip them from inputs of both `homectl` and `userdbctl` before
comparing them to avoid unexpected fails.
Also, switch from `cmp` to `diff` to make a potential test fail a bit more
debuggable.
Fixes: #14755
Let's append the date to the domain in the file name, to be able
to have multiple versions for the same domain.
There is no particular rhyme or reason to the domains being used:
I just pulled a few domains that happened to be present in issues reported
on github, even though the issues were not about pretty printing.
This doesn't really matter, since in non-/usr-merged systems plymouth
needs to be in /bin and on merged ones it doesn't matter, but it is
still prettier to insert the right path, and avoid /bin on merged
systems, since it's just a compat symlink.
Replaces: #15351
Give systemd a chance to process the stop event before checking if the
PID has indeed leaked. This should fix the intermittent test fails in CI
even with a fixed systemd version, like this one:
```
Apr 08 10:22:09 testsuite-47.sh[345]: ++ cat /leakedtestpid
Apr 08 10:22:09 testsuite-47.sh[334]: + leaked_pid=342
Apr 08 10:22:09 testsuite-47.sh[334]: + systemctl stop testsuite-47-repro
Apr 08 10:22:10 testsuite-47.sh[334]: + ps -p 342
Apr 08 10:22:10 testsuite-47.sh[348]: PID TTY TIME CMD
Apr 08 10:22:10 testsuite-47.sh[348]: 342 ? 00:00:00 sleep
Apr 08 10:22:10 testsuite-47.sh[334]: + exit 42
```
Followup to 197298ff9f
When doing 'make clean', we remove the cached image. So doing
'make -C TEST-NN-foo clean setup run clean-again' in a loop is very slow.
Let's filter out the 'clean' target (if specified), and do the cleaning
in the beginning, and then run other targets in a loop as before.
The test would fail when run again from the same image. So let's
rename the stuff we create to be more unique, and remove it before
running the test. (Removing it after would be more elegant, but it's
hard to make sure that everything is removed when things fail halfway.
Cleanup *before* tests is much more rebust.)
Using s-j-remote fixes the following issue: when coalescing files from multiple
inputs, simply copying all files with into the the same directory might
potentially mess things up, because a newer system.journal might overwrite an
older journal. This happens because we run multiple tests from the same image,
and need to clean out the directory after each run.
By using systemd-journal-remote, we nicely coalesce all files. This has the
advantage that if there aren't too many logs, we end up with just one journal
file.
ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are
copied to a separate directory to preserve after tests have been run. This
functionality can now be recreated by setting
ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
This was done downstream in debian and ubuntu [1]. I want to change the
downstream file to use run-integration-tests so we can change the way tests
work more easily. Let's start moving downstream functionality upstream.
$ sudo BLACKLIST_MARKERS='blacklist-ubuntu-ci-arm64 blacklist-ubuntu-ci' \
BUILD_DIR=build test/run-integration-tests.sh
[1] https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/upstream
It is more trouble than it is worth. The setup is of a loopback device
is very quick, so it's better to always create it when needed and
immediately drop afterwards.
This causes the unprivileged-nspawn-root directory to be removed
after running one test. The advantage is that we reduce the maximum
disk-space use quite a bit (47*400 MB → about 18GB).
has-overflow was a temporary hack that was removed in
844da987ef (Oct. 2016). All the makefiles
can be the same, and all the targets can be handled identically.
Before, we'd copy the test tree into nspawn-root, and run the tests from there.
This is OK, and doesn't actually take much extra time. But it uses quite a lot
of extra disk space. So let's make things a bit more efficient by running
directly from the image file.
We still run the unprivileged nspawn tests from a copy. Once the kernel
implements fs shift, we can do away with that too.
Before, we'd create a separate image for each test, in
/var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images
where very similar, except that each one had some unit files installed
specifically for the test. The installation of those custom unit files
was removed in previous commits (all the unit files are always installed).
The new approach is to only create as few distinct images as possible.
We have:
default.img: the "normal" image suitable for almost all the tests
basic.img: the same as default image but doesn't mask any services
cryptsetup.img: p2 is used for encrypted /var
badid.img: /etc/machine-id is overwritten with stuff
selinux.img: with selinux added for fun and fun
and a few others:
ls -l build/test/*img
lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img
lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img
lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img
lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img
lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img
lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img
lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img
I considered trying to use the same image everywhere. It would probably be
possible, but it would be very brittle. By using separate images where it is
necessary we keep various orthogonal modifications independent.
The way that images are cached is complicated by the fact that we still
want to keep them in /var/tmp. Thus, an image is created on first use and
linked to from build/test/ so it can be found by other tests.
Tests cannot be run in parallel. I think that is an acceptable limitation.
Creation of the images was probably taking more resources then the actual
tests, so we should be better off anyway.
We had an fstab for the sole purpose of remounting "/" rw. Mounting root ro
is a pointless excercise in obsolete approaches. More importantly, the nspawn
image is now the same as the qemu one.
The two timezone files are now installed in the global setup. I am not too
happy about this, but it still seems better than to create a completely
separate image just for this.
I picked the list of zone files to install by grepping through the code. This
is is a bit brittle, but installing all of them takes a while, and more
importantly, writes a lot of lines to the log.
Sometimes the test would fail there, nondeterministically. I'm not sure why,
but relying on PID1 not caching the file is clearly very brittle. Let's instead
call daemon-reload.
Unfortunately meson does not install symlinks, but copies the symlink
destination instead. So symlinks need to be created by a script.
This commit adds both symlinks in test/testsuite-08.units/ and meson
scriptlet calls. Strictly speaking, the first is not necessary, since nothing
reads stuff directly from the source tree.
I put SELINUX=disabled on my laptop, and the test fails with ENOENT when trying
to write to /sys/fs/selinux/enforce. It's a bit of a special case, but let's
avoid the failure.
The test currently fails in the check for LimitNOFILESoft/LimitNOFILE. I see
default values there. This doesn't seem to be related to the changes in the
test suite, but rather to the recent changes to pid1.
During installation, meson complains:
> Installing /home/zbyszek/src/systemd-work/test/units/sysinit.target to /var/tmp/systemd-test.Q1FSuj/root/usr/lib/systemd/tests/testdata/units
> Warning: trying to copy a symlink that points to a file. This will copy the file,
> but this will be changed in a future version of Meson to copy the symlink as is. Please update your
> build definitions so that it will not break when the change happens.
It *is* convenient to have those files as symlinks, but it is also confusing,
because symlinks create aliases, and it seems that in those cases we actually
don't want aliases (at least in the case of loopy*.service that'd make the test
pointless).
When specifying `DHCPv4.SendOption=`, it is used by systemd-networkd to
set the value of that option within the DHCP request that is sent out.
This differs to setting `DHCPServer.SendOption=`, which will place all
the options together as suboptions into the vendor-specific information
(code 43) option.
This commit adds two new config options, `DHCPv4.SendVendorOption=` and
`DHCPServer.SendVendorOption=`. These both have the behaviour of the old
`DHCPServer.SendOption=` flag, and set the value of the suboption in the
vendor-specific information option.
The behaviour of `DHCPServer.SendOption=` is then changed to reflect
that of `DHCPv4.SendOption=`. It will set the value of the corresponding
option in the DHCP request.
Proportional Integral controller-Enhanced (PIE) is a control
theoretic active queue management scheme. It is based on the
proportional integral controller but aims to control delay.
http://man7.org/linux/man-pages/man8/tc-pie.8.html
As in 2a5fcfae02
and in 3e67e5c992
using /usr/bin/env allows bash to be looked up in PATH
rather than being hard-coded.
As with the previous changes the same arguments apply
- distributions have scripts to rewrite shebangs on installation and
they know what locations to rely on.
- For tests/compilation we should rather rely on the user to have setup
there PATH correctly.
In particular this makes testing from git easier on NixOS where do not provide
/bin/bash to improve compose-ability.
The man pages state that the '+' prefix in Exec* directives should
ignore filesystem namespacing options such as PrivateTmp. Now it does.
This is very similar to #8842, just with PrivateTmp instead of
PrivateDevices.
To suppress the following warning:
---
Warning: Stopping systemd-udevd.service, but it can still be activated by:
systemd-udevd-control.socket
systemd-udevd-kernel.socket
In certain cases the expected enqueue-start-replace-continue
sequence would end up as enqueue-replace-start-continue which causes
unexpected fails even though the serialization/deserialization part
works as expected. As we can't use `--wait` in this case, let's give
sysetmd a second to actually start the unit before replacing it with
another one.
Also, switch from the single-letter test output to a bit verbose format.
Fixes: #14632
Frantisek Sumsal