Not sure how I missed this, but we of course need to wait for the
"systemd-run" commands to finish before we can check the output files
this generated.
It tests importing and exporting, and a few other machinectl commands.
It currently does not test pulling (i.e. http downloads), but we might
want to add that later on.
For netdev, config files are loaded twice, and the first time,
only Match and NetDev setions are read. So, the test given by
the previous commit covers only the second loading.
This adds another test that also covers the first loading.
systemd only uses functions that are as of Linux 4.15+ provided
externally to the CPU controller (currently usage_usec), so if we have a
new enough kernel, we don't need to set CGROUP_MASK_CPU for
CPUAccounting=true as the CPU controller does not need to necessarily be
enabled in this case.
Part of this patch is modelled on an earlier patch by Ryutaroh Matsumoto
(see PR #9665).
It's possible for sscanf to receive strings containing all three fields
and not matching the template at the same time. When this happens the
value of k doesn't change, which basically means that process_audit_string
tries to access memory randomly. Sometimes it works and sometimes it doesn't :-)
See also https://bugzilla.redhat.com/show_bug.cgi?id=1059314.
This removes the ability to configure which cgroup controllers to mount
together. Instead, we'll now hardcode that "cpu" and "cpuacct" are
mounted together as well as "net_cls" and "net_prio".
The concept of mounting controllers together has no future as it does
not exist to cgroupsv2. Moreover, the current logic is systematically
broken, as revealed by the discussions in #10507. Also, we surveyed Red
Hat customers and couldn't find a single user of the concept (which
isn't particularly surprising, as it is broken...)
This reduced the (already way too complex) cgroup handling for us, since
we now know whenever we make a change to a cgroup for one controller to
which other controllers it applies.
Fedora Rawhide renamed dbus.service to dbus-daemon.service - that
breaks tests which require working DBus (e.g. TEST-03-JOBS)
Excerpt from the dbus.spec:
The 'dbus' package is only retained for compatibility purposes. It will
eventually be removed and then replaced by 'Provides: dbus' in the
dbus-daemon package. It will then exclusively be used for other packages to
describe their dependency on a system and user bus. It does not pull in any
particular dbus *implementation*, nor any libraries. These should be pulled
in, if required, via explicit dependencies.
Unfortunately we can't check the 6rd attribute, because it's not
exposed in /sys or anywhere other than netlink... But at least we
can check that networkd brings up an interface that looks right.
This should help the fuzzers to discover code paths faster.
In case anyone is interested, they were generated with the following script
```
perl -aF'/[\s,]+/' -ne '
if (my ($s, $d) = ($F[0] =~ /^([^\s\.]+)\.([^\s\.]+)$/)) { $d{$s}{$d} = 1; }
END { while (my ($key, $value) = each %d) {
printf "[%s]\n%s\n", $key, join("\n", keys(%$value))
}}'
```
by passing src/network/networkd-network-gperf.gperf and
src/network/netdev/netdev-gperf.gperf to it.
It's not entirely impossible to screw something up playing with
kernel modules on a Saturday evening :-) This PR fixes a scenario
where a module has been loaded into the kernel but the module itself
has been removed from the disk.
```
$ lsmod | grep wireg
wireguard 225280 0
ip6_udp_tunnel 16384 1 wireguard
udp_tunnel 16384 1 wireguard
$ modprobe wireguard
modprobe: FATAL: Module wireguard not found in directory /lib/modules/4.18.16-200.fc28.x86_64
$ sudo ./systemd-networkd-tests.py NetworkdNetDevTests.test_wireguard
...
modprobe: FATAL: Module wireguard not found in directory /lib/modules/4.18.16-200.fc28.x86_64
test_wireguard (__main__.NetworkdNetDevTests) ... unexpected success
----------------------------------------------------------------------
Ran 1 test in 5.152s
FAILED (unexpected successes=1)
```
This is a follow-up to https://github.com/systemd/systemd/pull/10625.
After ec6a47044a, networkd starts
soon after removing links used by previous test. That causes
some racy situation of removing links and detecting links by networkd.
I went through my antique collection of fuzzers the other day
to see which ones I hadn't sent upstream yet. This one
seems to be nice to have and ready to be merged. As far as I can
tell, it hasn't managed to find anything useful yet,
but it's better to be safe than sorry especially when it comes to networking
code :-)
Currently, if I run the full "run-integration-tests.sh" script it will
fail on my machine because it fills up /var/tmp whith just too much
crap until the disk is full.
Let's make sure that "run-integration-tests.sh" cleans up after every
test. For that change the make targets to run from "clean setup run" to
"clean setup run clean" — except that that doesn't work since make is
smart enough to realize that the same target appears twice on the
command line and will only execute it once. Let's fix that by defining
another target "clean-again" which is just like "clean", but allows us
to be added to the same command line a second time. Then, let's build
with "clean setup run clean-again" and all is good.
While we are at it, let's also add .PHONY where appropriate, after all
these all are phony targets.
Add LogRateLimitIntervalSec= and LogRateLimitBurst= options for
services. If provided, these values get passed to the journald
client context, and those values are used in the rate limiting
function in the journal over the the journald.conf values.
Part of #10230
Those interfaces are created automatically when ip6_tunnel and ip6_gre loaded.
They break the test with exec-privatenetwork-yes.service.
C.f. 6b08180ca6.
We would read (-1), and then add 1 to it, call message_peek_body(..., 0, ...),
and when trying to make use of the data.
The fuzzer test case is just for one site, but they all look similar.
v2: fix two UINT8_MAX/UINT32_MAX mismatches founds by LGTM
We copied part of the string into a buffer that was off by two.
If the element signature had length one, we'd copy 0 bytes and crash when
looking at the "first" byte. Otherwise, we would crash because strncpy would
not terminate the string.
This is similar to the grandparent commit 'fix calculation of offsets table',
except that now the change is for array elements. Same story as before: we need
to make sure that the offsets increase enough taking alignment into account.
While at it, rename 'p' to 'previous' to match similar code in other places.
The offsets specify the ends of variable length data. We would trust the
incoming data, putting the offsets specified in our message
into the offsets tables after doing some superficial verification.
But when actually reading the data we apply alignment, so we would take
the previous offset, align it, making it bigger then current offset, and
then we'd try to read data of negative length.
In the attached example, the message specifies the following offsets:
[1, 4]
but the alignment of those items is
[1, 8]
so we'd calculate the second item as starting at 8 and ending at 4.
The alternative would be to treat gvariant and !gvariant messages differently.
But this is a problem because we check signatures is variuos places before we
have an actual message, for example in sd_bus_add_object_vtable(). It seems
better to treat things consistent (i.e. follow the lowest common denominator)
and disallow empty structures everywhere.
We didn't free one of the fields in two of the places.
$ valgrind --show-leak-kinds=all --leak-check=full \
build/fuzz-bus-message \
test/fuzz/fuzz-bus-message/leak-c09c0e2256d43bc5e2d02748c8d8760e7bc25d20
...
==14457== HEAP SUMMARY:
==14457== in use at exit: 3 bytes in 1 blocks
==14457== total heap usage: 509 allocs, 508 frees, 51,016 bytes allocated
==14457==
==14457== 3 bytes in 1 blocks are definitely lost in loss record 1 of 1
==14457== at 0x4C2EBAB: malloc (vg_replace_malloc.c:299)
==14457== by 0x53AFE79: strndup (in /usr/lib64/libc-2.27.so)
==14457== by 0x4F52EB8: free_and_strndup (string-util.c:1039)
==14457== by 0x4F8E1AB: sd_bus_message_peek_type (bus-message.c:4193)
==14457== by 0x4F76CB5: bus_message_dump (bus-dump.c:144)
==14457== by 0x108F12: LLVMFuzzerTestOneInput (fuzz-bus-message.c:24)
==14457== by 0x1090F7: main (fuzz-main.c:34)
==14457==
==14457== LEAK SUMMARY:
==14457== definitely lost: 3 bytes in 1 blocks
v2: fix error in free_and_strndup()
When the orignal and copied message were the same, but shorter than specified
length l, memory read past the end of the buffer would be performed. A test
case is included: a string that had an embedded NUL ("q\0") is used to replace
"q".
v3: Fix one more bug in free_and_strndup and add tests.
v4: Some style fixed based on review, one more use of free_and_replace, and
make the tests more comprehensive.
318/365 fuzz-bus-message:crash-26bba7182dedc8848939931d9fcefcb7922f2e56:address OK 0.03 s
319/365 fuzz-bus-message:crash-29ed3c202e0ffade3cad42c8bbeb6cc68a21eb8e:address OK 0.03 s
320/365 fuzz-bus-message:crash-b88ad9ecf4aacf4a0caca5b5543953265367f084:address OK 0.03 s
321/365 fuzz-bus-message:crash-c1b37b4729b42c0c05b23cba4eed5d8102498a1e:address OK 0.03 s
322/365 fuzz-bus-message:crash-d8f3941c74219b4c03532c9b244d5ea539c61af5:address OK 0.03 s
323/365 fuzz-bus-message:crash-e1b811da5ca494e494b77c6bd8e1c2f2989425c5:address OK 0.03 s
324/365 fuzz-bus-message:leak-c09c0e2256d43bc5e2d02748c8d8760e7bc25d20:address OK 0.04 s
325/365 fuzz-bus-message:message1:address OK 0.03 s
326/365 fuzz-bus-message:timeout-08ee8f6446a4064db064e8e0b3d220147f7d0b5b:address OK 0.03 s
327/365 fuzz-dhcp-server:discover-existing:address OK 0.04 s
328/365 fuzz-dhcp-server:discover-new:address OK 0.03 s
329/365 fuzz-dhcp-server:release:address OK 0.04 s
330/365 fuzz-dhcp-server:request-existing:address OK 0.03 s
331/365 fuzz-dhcp-server:request-new:address OK 0.03 s
332/365 fuzz-dhcp-server:request-reboot:address OK 0.03 s
333/365 fuzz-dhcp-server:request-renew:address OK 0.03 s
334/365 fuzz-dns-packet:issue-7888:address OK 0.03 s
335/365 fuzz-dns-packet:oss-fuzz-5465:address OK 0.03 s
336/365 fuzz-journal-remote:crash-5a8f03d4c3a46fcded39527084f437e8e4b54b76:address OK 0.06 s
337/365 fuzz-journal-remote:crash-96dee870ea66d03e89ac321eee28ea63a9b9aa45:address OK 0.04 s
338/365 fuzz-journal-remote:invalid-ts.txt:address OK 0.04 s
339/365 fuzz-journal-remote:oss-fuzz-8659:address OK 0.06 s
340/365 fuzz-journal-remote:oss-fuzz-8686:address OK 0.04 s
341/365 fuzz-journal-remote:sample.txt:address OK 0.07 s
342/365 fuzz-unit-file:directives.service:address OK 0.03 s
343/365 fuzz-unit-file:empty.scope:address OK 0.04 s
344/365 fuzz-unit-file:machine.slice:address OK 0.03 s
345/365 fuzz-unit-file:oss-fuzz-6884:address OK 0.05 s
346/365 fuzz-unit-file:oss-fuzz-6885:address OK 0.03 s
347/365 fuzz-unit-file:oss-fuzz-6886:address OK 0.04 s
348/365 fuzz-unit-file:oss-fuzz-6892:address OK 0.03 s
349/365 fuzz-unit-file:oss-fuzz-6897:address OK 0.05 s
350/365 fuzz-unit-file:oss-fuzz-6897-evverx:address OK 0.04 s
351/365 fuzz-unit-file:oss-fuzz-6908:address OK 0.05 s
352/365 fuzz-unit-file:oss-fuzz-6917:address OK 0.06 s
353/365 fuzz-unit-file:oss-fuzz-6977:address OK 0.08 s
354/365 fuzz-unit-file:oss-fuzz-6977-unminimized:address OK 0.10 s
355/365 fuzz-unit-file:oss-fuzz-7004:address OK 0.03 s
356/365 fuzz-unit-file:oss-fuzz-8064:address OK 0.03 s
357/365 fuzz-unit-file:oss-fuzz-8827:address OK 0.50 s
358/365 fuzz-unit-file:proc-sys-fs-binfmt_misc.automount:address OK 0.03 s
359/365 fuzz-unit-file:syslog.socket:address OK 0.03 s
360/365 fuzz-unit-file:systemd-ask-password-console.path:address OK 0.03 s
361/365 fuzz-unit-file:systemd-machined.service:address OK 0.03 s
362/365 fuzz-unit-file:systemd-resolved.service:address OK 0.03 s
363/365 fuzz-unit-file:systemd-tmpfiles-clean.timer:address OK 0.03 s
364/365 fuzz-unit-file:timers.target:address OK 0.03 s
365/365 fuzz-unit-file:var-lib-machines.mount:address OK 0.04 s
This gives us slightly nicer coverage in the normal test run.
When in a git repo, git ls-files is used to get a list of files known to git.
This mirrors what update-man-rules does for man files. Only looking at files
known to git makes it easier to not forget to commit the test file to git,
and also makes bisecting easier if some files are left in repo.
When outside of a git repo, we expect to be unpacked from a tarball, so just
using all files reported by ls is OK.
There isn't really much need to keep them separate. Anything which is a good
corpus entry can be used as a smoke test, and anything which which is a
regression test can just as well be inserted into the corpus.
The only functional difference from this patch (apart from different paths in
output) is that the regression tests are now zipped together with the rest of
the corpus.
$ meson configure build -Dslow-tests=true && ninja -C build test
...
307/325 fuzz-dns-packet:issue-7888:address OK 0.06 s
308/325 fuzz-dns-packet:oss-fuzz-5465:address OK 0.04 s
309/325 fuzz-journal-remote:crash-5a8f03d4c3a46fcded39527084f437e8e4b54b76:address OK 0.07 s
310/325 fuzz-journal-remote:crash-96dee870ea66d03e89ac321eee28ea63a9b9aa45:address OK 0.05 s
311/325 fuzz-journal-remote:oss-fuzz-8659:address OK 0.05 s
312/325 fuzz-journal-remote:oss-fuzz-8686:address OK 0.07 s
313/325 fuzz-unit-file:oss-fuzz-6884:address OK 0.06 s
314/325 fuzz-unit-file:oss-fuzz-6885:address OK 0.05 s
315/325 fuzz-unit-file:oss-fuzz-6886:address OK 0.05 s
316/325 fuzz-unit-file:oss-fuzz-6892:address OK 0.05 s
317/325 fuzz-unit-file:oss-fuzz-6897:address OK 0.05 s
318/325 fuzz-unit-file:oss-fuzz-6897-evverx:address OK 0.06 s
319/325 fuzz-unit-file:oss-fuzz-6908:address OK 0.07 s
320/325 fuzz-unit-file:oss-fuzz-6917:address OK 0.07 s
321/325 fuzz-unit-file:oss-fuzz-6977:address OK 0.13 s
322/325 fuzz-unit-file:oss-fuzz-6977-unminimized:address OK 0.12 s
323/325 fuzz-unit-file:oss-fuzz-7004:address OK 0.05 s
324/325 fuzz-unit-file:oss-fuzz-8064:address OK 0.05 s
325/325 fuzz-unit-file:oss-fuzz-8827:address OK 0.52 s
=0 ndisc_router_parse (rt=0x60d000000110) at ../src/libsystemd-network/ndisc-router.c:126
=1 0x000055555558dc67 in ndisc_handle_datagram (nd=0x608000000020, rt=0x60d000000110) at ../src/libsystemd-network/sd-ndisc.c:170
=2 0x000055555558e65d in ndisc_recv (s=0x611000000040, fd=4, revents=1, userdata=0x608000000020) at ../src/libsystemd-network/sd-ndisc.c:233
=3 0x00007ffff63913a8 in source_dispatch (s=0x611000000040) at ../src/libsystemd/sd-event/sd-event.c:3042
=4 0x00007ffff6395eab in sd_event_dispatch (e=0x617000000080) at ../src/libsystemd/sd-event/sd-event.c:3455
=5 0x00007ffff6396b12 in sd_event_run (e=0x617000000080, timeout=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:3512
=6 0x0000555555583f5c in LLVMFuzzerTestOneInput (data=0x6060000000e0 "\206", size=53) at ../src/fuzz/fuzz-ndisc-rs.c:422
=7 0x0000555555586356 in main (argc=2, argv=0x7fffffffe3d8) at ../src/fuzz/fuzz-main.c:33
Allows configuring the watchdog signal (with a default of SIGABRT).
This allows an alternative to SIGABRT when coredumps are not desirable.
Appropriate references to SIGABRT or aborting were renamed to reflect
more liberal watchdog signals.
Closes#8658
We have "installed tests", but don't provide an easy way to run them.
The protocol is very simple: each test must return 0 for success, 77 means
"skipped", anything else is an error. In addition, we want to print test
output only if the test failed.
I wrote this simple script. It is pretty basic, but implements the functions
listed above. Since it is written in python it should be easy to add option
parsing (like running only specific tests, or running unsafe tests, etc.)
I looked at the following alternatives:
- Ubuntu root-unittests: this works, but just dumps all output to the terminal,
has no coloring.
- @ssahani's test runner [2]
It uses the unittest library and the test suite was implented as a class, and
doesn't implement any of the functions listed above.
- cram [3,4]
cram runs our tests, but does not understand the "ignore the output" part,
has not support for our magic skip code (it uses hardcoded 80 instead),
and seems dead upstream.
- meson test
Here the idea would be to provide an almost-empty meson.build file under
/usr/lib/systemd/tests/ that would just define all the tests. This would
allow us to reuse the test runner we use normally. Unfortunately meson requires
a build directory and configuration to be done before running tests. This
would be possible, but seems a lot of effort to just run a few binaries.
[1] 242c96addb/debian/tests/root-unittests
[2] https://github.com/systemd/systemd-fedora-ci/blob/master/upstream/systemd-upstream-tests.py
[3] https://bitheap.org/cram/
[4] https://pypi.org/project/pytest-cram/Fixes#10069.
When parsing and installing binaries mentioned in Exec*= lines the
5ed0dcf4d5 commit added parsing logic to drop
prefixes, including handling duplicate exclamation marks. But this did not
handle arbitrary combination of multiple prefixes, ie. StartExec=+-/bin/sh was
parsed as -/bin/sh which then would fail to install.
Instead of using egrep and shell replacements, replace both with sed command
that does it all. This sed script extract a group of characters starting with a
/ up to the first space (if any) after the equals sign. This correctly handles
existing non-prefixed, prefixed, multiple-prefixed commands.
About half commands seem to repeat themself, thus sort -u cuts the list of
binaries to install about in half.
To validate change of behaviour both old and new functions were modified to
echo parsed binaries into separate files, and then diffed. The incorrect
-/bin/sh was missing in the new output.
Without this patch tests fail on default Ubuntu installs.
Back in 08318a2c5a, value "false" was enabled for
'-Dtests=', but various tests were not conditionalized properly. So even with
-Dtests=false -Dslow-tests=false we'd run 120 tests. Let's make this consistent.
... when no mount options are passed.
Change the code, to avoid the following failure in the newly added tests:
exec-temporaryfilesystem-rw.service: Executing: /usr/bin/sh -x -c
'[ "$(stat -c %a /var)" == 755 ]'
++ stat -c %a /var
+ '[' 1777 == 755 ']'
Received SIGCHLD from PID 30364 (sh).
Child 30364 (sh) died (code=exited, status=1/FAILURE)
(And I spotted an opportunity to use TAKE_PTR() at the end).
We can't remount the underlying superblocks, if we are inside a user
namespace and running Linux <= 4.17. We can only change the per-mount
flags (MS_REMOUNT | MS_BIND).
This type of mount() call can only change the per-mount flags, so we
don't have to worry about passing the right string options now.
Fixes#9914 ("Since 1beab8b was merged, systemd has been failing to start
systemd-resolved inside unprivileged containers" ... "Failed to re-mount
'/run/systemd/unit-root/dev' read-only: Operation not permitted").
> It's basically my fault :-). I pointed out we could remount read-only
> without MS_BIND when reviewing the PR that added TemporaryFilesystem=,
> and poettering suggested to change PrivateDevices= at the same time.
> I think it's safe to change back, and I don't expect anyone will notice
> a difference in behaviour.
>
> It just surprised me to realize that
> `TemporaryFilesystem=/tmp:size=10M,ro,nosuid` would not apply `ro` to the
> superblock (underlying filesystem), like mount -osize=10M,ro,nosuid does.
> Maybe a comment could note the kernel version (v4.18), that lets you
> remount without MS_BIND inside a user namespace.
This makes the code longer and I guess this function is still ugly, sorry.
One obstacle to cleaning it up is the interaction between
`PrivateDevices=yes` and `ReadOnlyPaths=/dev`. I've added a test for the
existing behaviour, which I think is now the correct behaviour.
A follow-up for commit 9d874aec45.
This patch makes "path" parameter mandatory in fd_set_*() helpers removing the
need to use fd_get_path() when NULL was passed. The caller is supposed to pass
the fd anyway so assuming that it also knows the path should be safe.
Actually, the only case where this was useful (or used) was when we were
walking through directory trees (in item_do()). But even in those cases the
paths could be constructed trivially, which is still better than relying on
fd_get_path() (which is an ugly API).
A very succinct test case is also added for 'z/Z' operators so the code dealing
with recursive operators is tested minimally.
The usage of an initrd made TEST-09-ISSUE-2691 more likely to fail with
a timeout, so increase the timeout by 90s and adjust TimeoutStopSec=
accordingly.
Not all distros support booting without an initrd. E.g. the Debian
kernel builds ext4 as a module and so relies on an initrd to
successfully start the QEMU-based images.
Mount tmpfses over the networkd and resolved config and state
directories, and stop the services beforehand. This ensures that the
test does not mess with an existing networkd/resolved setup. At least
for ethernet setups, this does not sever existing links, so is good
enough for the CI cases we are interested in (QEMU and LXC).
Relax the skip check to only skip the test when trying to run this on
real iron, but start running it in virtual machines now.
This allows us to run the test on Ubuntu 18.04 LTS in CI, which uses
both services by default.
Like s-networkd.service itself, it can happen that s-resolved.service
runs into restart limits. Don't enforce a successful call, as on
machines without resolved the unit might not be loaded.
- Reset systemd-networkd.service before each test run, to avoid running
into restart limits.
- Our networkd-test-router.service unit needs to run as root and thus
can't use `User=`; but networkd still insists on the
`systemd-network` system user to exist, so create it.
oss-fuzz flags this as:
==1==WARNING: MemorySanitizer: use-of-uninitialized-value
0. 0x7fce77519ca5 in ascii_is_valid systemd/src/basic/utf8.c:252:9
1. 0x7fce774d203c in ellipsize_mem systemd/src/basic/string-util.c:544:13
2. 0x7fce7730a299 in print_multiline systemd/src/shared/logs-show.c:244:37
3. 0x7fce772ffdf3 in output_short systemd/src/shared/logs-show.c:495:25
4. 0x7fce772f5a27 in show_journal_entry systemd/src/shared/logs-show.c:1077:15
5. 0x7fce772f66ad in show_journal systemd/src/shared/logs-show.c:1164:29
6. 0x4a2fa0 in LLVMFuzzerTestOneInput systemd/src/fuzz/fuzz-journal-remote.c:64:21
...
I didn't reproduce the issue, but this looks like an obvious error: the length
is specified, so we shouldn't use the string with any functions for normal
C-strings.
The test is heavily dependent on timeouts, and if we are run in
potentially very slow QEMU instances there's a good chance we'll miss
some which we normally wouldn't miss. Hence, let's test this one in
nspawn only. Given that the test is purely in service management it
shouldn't matter whether it runs in nspawn or qemu, hence keep running
it in nspawn, but don't bother with qemu.
Similar, do this for TEST-03-JOBS, too, which operates with relatively
short sleep times internally.
Fixes: #9123
As it turns out /usr/share/selinux/devel/ is now included in more RPMs
than just selinux-policy-devel (specifically container-selinux, which is
pulled in by various container related RPMs). Let's hence tighten the
dependency check a bit and look for systemd's .if file, which is what we
actually care about.
First, ellipsize() and ellipsize_mem() should not read past the input
buffer. Those functions take an explicit length for the input data, so they
should not assume that the buffer is terminated by a nul.
Second, ellipsization was off in various cases where wide on multi-byte
characters were used.
We had some basic test for ellipsize(), but apparently it wasn't enough to
catch more serious cases.
Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8686.
journalctl -o short would display those entries, but journalctl -o short-full
would refuse. If the entry is bad, just fall back to the receive-side realtime
timestamp like we would if it was completely missing.
We'd look for a '=' separator using memchr, i.e. ignoring any nul bytes in the
string, but then do a strndup, which would terminate on any nul byte, and then
again do a memcmp, which would access memory past the chunk allocated by strndup.
Of course, we probably shouldn't allow keys with nul bytes in them. But we
currently do, so there might be journal files like that out there. So let's fix
the journal-reading code first.
Something is wrong with the entry (probably a missing timestamp), so no point
in rotating. But suppress the error in process_source(), so that the processing
of the data stream continues.
Also, just return 0 from writer_write() on success, the only caller doesn't
care.
This corresponds nicely with the specifiers we already pass for
/var/lib, /var/cache, /run and so on.
This is particular useful to update the test-path service files to
operate without guessable files, thus allowing multiple parallel
test-path invocations to pass without issues (the idea is to set $TMPDIR
early on in the test to some private directory, and then only use the
new %T or %V specifier to refer to it).
Test the "[Bridge]" section keys
```
[Bridge]
UnicastFlood=true
HairPin=true
UseBPDU=true
FastLeave=true
AllowPortToBeRoot=true
Cost=555
Priority=23
```
```
test_bridge_init (__main__.BridgeTest) ... ok
test_bridge_port_priority (__main__.BridgeTest) ... ok
test_bridge_port_priority_set_zero (__main__.BridgeTest)
It should be possible to set the bridge port priority to 0 ... ok
test_bridge_port_property (__main__.BridgeTest)
Test the "[Bridge]" section keys ... ok
```
Yes, the output is sometimes annyoing, but /dev/null is not the right
place...
I figure this redirection was left in from some debugging session, let's
fix it, and make the setup_basic_environment invocation like in all
other test scripts.
Nested KVM is very flaky as we learnt from our CI. Hence, let's avoid
KVM whenever we detect we are already running inside of KVM.
Maybe one day nested KVM is fixed, at which point we can turn this on
again, but for now let's simply avoid nested KVM, since reliable CI is
more important than quick CI, I guess.
And yes, avoiding KVM for our qemu runs does make things substantially
slower, but I think it's not a complete loss.
Inspired by @evverx' findings in:
https://github.com/systemd/systemd/pull/8701#issuecomment-380213302
To indicate that the there're no more entries, these wrappers return false but
did leave the passed pointed unmodified.
However EOF is not an error and is a very common case so initialize the output
argument to NULL even in this case so callers don't need to do that.
Fixes: #8721
We go through the whole file system, so this test can take arbitrary time. But
this test is still quite useful, so let's at least try to make it more efficent
by not descending at all into the directories we would filter out later on
anyway.
Also increase the timeout, in case the previous step doesn't help enough.
Absolute paths make everything simple and quick, but sometimes this requirement
can be annoying. A good example is calling 'test', which will be located in
/usr/bin/ or /bin depending on the distro. The need the provide the full path
makes it harder a portable unit file in such cases.
This patch uses a fixed search path (DEFAULT_PATH which was already used as the
default value of $PATH), and if a non-absolute file name is found, it is
immediately resolved to a full path using this search path when the unit is
loaded. After that, everything behaves as if an absolute path was specified. In
particular, the executable must exist when the unit is loaded.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
This makes the script wait for the newly created partition to
show up before trying to put a filesystem on it, which should
prevent the tests from failing with the following error:
```
New situation:
Disklabel type: dos
Disk identifier: 0x3541a0ec
Device Boot Start End Sectors Size Id Type
/dev/loop6p1 2048 800767 798720 390M 83 Linux
/dev/loop6p2 800768 819199 18432 9M 83 Linux
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
The file /dev/loop6p1 does not exist and no size was specified.
make: *** [setup] Error 1
F: Failed to mkfs -t ext4
Makefile:4: recipe for target 'setup' failed
```
ubsan times out because we do too many allocations:
$ valgrind build/fuzz-unit-file test/fuzz-regressions/fuzz-unit-file/oss-fuzz-6977-full
...
test/fuzz-regressions/fuzz-unit-file/oss-fuzz-6977-full... ok
==1757==
==1757== HEAP SUMMARY:
==1757== in use at exit: 0 bytes in 0 blocks
==1757== total heap usage: 199,997 allocs, 199,997 frees, 90,045,318,585 bytes allocated
...
==3256== total heap usage: 100,120 allocs, 100,120 frees, 13,097,140 bytes allocated
https://oss-fuzz.com/v2/issue/4651449704251392/6977 should now be really fixed.
e3c3d6761b was the first attempt, but even with this change, e3c3d6761b
still makes sense.
With this "sudo ./run-integration-tests.sh" should work fully without
exception, even on systems lacking SELinux (in which case that test will
just be skipped)
It's always visible:
$ sudo modprobe sit
$ sudo unshare -n ip l
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
...
2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
...
grep already indicates if it matched anything by return value.
Additional advantage is then that if the test fails, the unexpected
matching lines are visible in the log output.
No need to go through the specifier_printf() if the path is already too long in
the unexpanded form (since specifiers increase the length of the string in all
practical cases).
In the oss-fuzz test case, valgrind reports:
total heap usage: 179,044 allocs, 179,044 frees, 72,687,755,703 bytes allocated
and the original config file is ~500kb. This isn't really a security issue,
since the config file has to be trusted any way, but just a matter of
preventing accidental resource exhaustion.
https://oss-fuzz.com/v2/issue/4651449704251392/6977
While at it, fix order of arguments in the neighbouring log_syntax() call.
msan doesn't understand sscanf with %ms, so it falsely reports unitialized
memory. Using sscanf with %ms is quite convenient in
socket_address_parse_netlink(), so let's just not run the fuzzer for
ListenNetlink= at all for now. If msan is fixed, we can remove this.
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=6884
The orignal reproducer from oss-fuzz depends on the hostname (via %H and %c).
The hostname needs a dash for msan to report this, so a simpler case from
@evverx with the dash hardcoded is also added.
The issue is a false positive from msan, which does not instruct stpncpy
(https://github.com/google/sanitizers/issues/926). Let's add a work-around
until this is fixed.
We have only three bits of space, i.e. 8 possible classes. Immediately reject
anything outside of that range. Add the fuzzer test case and an additional
unit test.
oss-fuzz #6908.
We probably should allow very deep calls of our recursive functions. Let's add
a limit to avoid resource exhaustion. 240 is 10 per hour (if somebody is using
this for time based triggers...), so it should be more than enough for most use
cases, and is conveniently below the 250 stack limit in msan.
oss-fuzz #6917.
Also fix one case where the presence of a newline was used to generate
an invalid environment assignment.
Tested: with mkosi, which builds the local tree and run ninja tests.
gmtime_r() will return NULL in that case, and we would crash.
I committed the reproducer case in fuzz-regressions/, even though we don't have
ubsan hooked up yet. Let's add it anyway in case it is useful in the future. We
actually crash anyway when compiled with asserts, so this can be easily
reproduced without ubsan.
oss-fuzz #6886.
This seems to be a false positive in msan:
https://github.com/google/sanitizers/issues/767.
I don't see anything wrong with the code either, and valgrind does not see the
issue. Anyway, let's add the test case.
We don't have msan hooked up yet, but hopefully we'll in the future.
oss-fuzz #6884.
We currently have just one sanitizer for tests, asan, but we may add more in
the future. So let's keep the loop over the sanitizers in meson.build, but
just enable all regression cases under all sanitizers. If it fails under one
of them, it might fail under a different one.
In subsequent commits I'll add test cases which might not fail under asan,
but it's good to commit them for future use.
The test names are made more verbose:
256/257 fuzz-dns-packet:oss-fuzz-5465:address OK 0.04 s
257/257 fuzz-dns-packet:issue-7888:address OK 0.03 s
The unit files for test-execute are named like
`exec-(setting-name-in-lower-character)-(optional-text).service`.
However, test units for AmbientCapabilities= are not following this.
So, let's rename them for the consistency.
This does not change anything in the functionality of the test.
This PR implements the first part of RFE #8046. I.e. this allows to
write:
```
u username -:300
```
Where the uid is chosen automatically but the gid is fixed.
The nobody user/group may not synthesized by systemd.
To run tests the functionalities in such situation, this adds tests
by user/group by daemon, as it is expected to exists all environments.
A runaway string should still be returned by the code that splits on
commas, so add a '?' to the regex so that the last '"?' in a string
still produces a valid block for the split code.
Tested:
ACTION=="remove\"GOTO=""
Which then produced:
$ test/rule-syntax-check.py src/login/70-uaccess.rules
# looking at src/login/70-uaccess.rules
Invalid line src/login/70-uaccess.rules:10: ACTION=="remove\"GOTO=""
clause: ACTION=="remove\"GOTO=""
Using a regex to match the groups is smarter than the split(',') that
would break in those cases.
Tested:
SUBSYSTEM=="usb", ENV{ID_USB_INTERFACES}=="*:060101:*,*:070202:*", TAG+="uaccess"
Rule checker doesn't break there after this commit.
Add support to backslash-escaped double quote inside a string.
Tested by modifying src/login/70-uaccess.rules to include:
ACTION=="remove" it", GOTO="uaccess_end"
And had the rule checker complain about it:
$ test/rule-syntax-check.py src/login/70-uaccess.rules
# looking at src/login/70-uaccess.rules
Invalid line src/login/70-uaccess.rules:10: ACTION=="remove" it", GOTO="uaccess_end"
clause: ACTION=="remove" it"
This is true since commit 7e760b79ad.
Note that the changes in the regex expressions relies on the fact that the
script assumes that the comma separator is mandatory.
Add a comment in the script to clarify this.
In udev man page, "PROGRAM" key is part of the keys which are used for
matching purposes so it should only be used with the compare operator "==".
Actually it doesn't really make sense to assign it a value.
udev code allows both "=" and "==" for PROGRAM and both are handled the same
way but for consistencies it's better to have only the compare operator allowed
by the rule syntax checker.
No rules shipped by systemd use PROGRAM key so nothing need to be changed in
our rule files.
`nobody` is a special user, whose credentials should be extracted with
`get_user_creds`. `getpwnam` called in `test-udev.pl` is a bit different,
which causes the test to fail with the following error:
```
device '/devices/pci0000:00/0000:00:1f.2/host0/target0:0:0/0:0:0:0/block/sda' expecting node/link 'node'
expected permissions are: nobody::0600
created permissions are : 65534:0:0600
permissions: error
add: ok
remove: ok
```
The ideal fix would probably be to implement `get_user_creds` in Perl, but in this
PR the issue is simply got around by using `daemon` instead of `nobody`.
Closes https://github.com/systemd/systemd/issues/8196.
Followed PEP8 and PEP3101 rules (#8079)
Imports re-ordered by Alphabetical Standarts for following PEP8
Old type string formattings (" example %s " % exampleVar ) re-writed as new type string
formattings ( " example {} ".format(exampleVar) ) for following PEP3101
On Debian/Ubuntu systems the default passwd/group files use a
slightly strange mapping. E.g. in passwd:
```
man❌6:12::/var/cache/man:/sbin/nologin
```
and in group:
```
disk❌6:
man❌12:
```
This is not supported in systemd-sysusers right now because
sysusers will not re-use an existing uid/gid in its normal
mode of operation. Unfortunately this reuse is needed to
replicate the default Debian/Ubuntu users/groups.
This commit enforces reuse when the "uid:gid" syntax is used
to fix this.
I also added a test that replicates the Debian base-passwd
passwd/group file to ensure things are ok.
This is a bit painful because a separate build of systemd is necessary. The
tests are guarded by tests!=false and slow-tests==true. Running them is not
slow, but compilation certainly is. If this proves unwieldy, we can add a
separate option controlling those builds later.
The build for each sanitizer has its own directory, and we build all fuzzer
tests there, and then pull them out one-by-one by linking into the target
position as necessary. It would be nicer to just build the desired fuzzer, but
we need to build the whole nested build as one unit.
[I also tried making systemd and nested meson subproject. This would work
nicely, but meson does not allow that because the nested target names are the
same as the outer project names. If that is ever fixed, that would be the way
to go.]
v2:
- make sure things still work if memory sanitizer is not available
v3:
- switch to syntax which works with meson 0.42.1 found in Ubuntu
This test tests the systemd-sysuser binary via the --root=$TESTDIR
option and ensures that for the given inputs the expected passwd
and group files will be generated.
Let's be more restrictive when validating PID files and MAINPID=
messages: don't accept PIDs that make no sense, and if the configuration
source is not trusted, don't accept out-of-cgroup PIDs. A configuratin
source is considered trusted when the PID file is owned by root, or the
message was received from root.
This should lock things down a bit, in case service authors write out
PID files from unprivileged code or use NotifyAccess=all with
unprivileged code. Note that doing so was always problematic, just now
it's a bit less problematic.
When we open the PID file we'll now use the CHASE_SAFE chase_symlinks()
logic, to ensure that we won't follow an unpriviled-owned symlink to a
privileged-owned file thinking this was a valid privileged PID file,
even though it really isn't.
Fixes: #6632
same motivation as in #5816:
- distributions have scripts to rewrite shebangs on installation and
they know what locations to rely on.
- For tests/compilation we should rather rely on the user to have setup
there PATH correctly.
We need to specify a full path to the "ip" binary and busybox "ip" has a
slightly different output than the normal ip, and won't show "DOWN".
hence instead ensure that at lest not "UP" is in there.
With Type=notify services, EXTEND_TIMEOUT_USEC= messages will delay any startup/
runtime/shutdown timeouts.
A service that hasn't timed out, i.e, start time < TimeStartSec,
runtime < RuntimeMaxSec and stop time < TimeoutStopSec, may by sending
EXTEND_TIMEOUT_USEC=, allow the service to continue beyond the limit for
the execution phase (i.e TimeStartSec, RunTimeMaxSec and TimeoutStopSec).
EXTEND_TIMEOUT_USEC= must continue to be sent (in the same way as
WATCHDOG=1) within the time interval specified to continue to reprevent
the timeout from occuring.
Watchdog timeouts are also extended if a EXTEND_TIMEOUT_USEC is greater
than the remaining time on the watchdog counter.
Fixes#5868.
Since the new option `--network-namespace-path=` of systemd-nspawn
cannot be used together with other network-related options, we need
to add more smoke tests for checking these conditions of options.
Ignore mkosi.builddir. In the future we can also add other patterns
if necessary.
run-intergration-tests.sh is updated to use the new script, and modified
to work from arbitrary directory.
Follow-up for #7494.
We currently look for "nobody" and "nfsnobody" when testing groups, both
of which do not exist on Ubuntu, our main testing environment. Let's
extend the tests slightly to also use "nogroup" if it exists.
Apparently there are a myriad of netcat implementations around, and they
all behave slightly differently. The one I have on my Fedora 27
installation will cause a failure when invoked as "nc -U" on an AF_UNIX
socket whose connections are immediately disconnected, thus causing the
test to fail.
Let's avoid all ambiguities in this regard, and drop usage of netcat
altoegther. Instead let's use a FIFO in the file system, which we can
connect to with only shell commands, and is hence much simpler and
more reliable to test with.
The actual test is supposed to validate that PID 1 doesn't hang when
activation of a socket-activated service fails, hence which transport
mechanism is used ultimately doesn't matter, as long as we activate the
service, and we do here...
These tests check the stderr. So, if the systemd.log_level=debug
is set in the kernel command line, then these tests fail.
This set log_level to info in hwdb-test.sh and meson-check-help.sh,
the kernel command line not to change the output of the target
programs.
Fixes#7362.
