If udevd receives an exit signal, it releases its reference on the udev
monitor in manager_exit(). If at this time a worker is hanging, and if
the event timeout for this worker expires before udevd exits, udevd
crashes in on_sigchld()->udev_monitor_send_device(), because the monitor
has already been freed.
Fix this by testing the validity of manager->monitor in on_sigchld().
If udevd receives an exit signal, it releases its reference on the udev
monitor in manager_exit(). If at this time a worker is hanging, and if
the event timeout for this worker expires before udevd exits, udevd
crashes in on_sigchld()->udev_monitor_send_device(), because the monitor
has already been freed.
Fix this by releasing the main process's monitor ref later, in
manager_free().
On some systems with lots of devices, device probing for certain drivers can
take a very long time. If systemd-udevd detects a timeout and kills the worker
running modprobe using SIGKILL, some devices will not be probed, or end up in
unusable state. The --event-timeout option can be used to modify the maximum
time spent in an uevent handler. But if systemd-udevd exits, it uses a
different timeout, hard-coded to 30s, and exits when this timeout expires,
causing all workers to be KILLed by systemd afterwards. In practice, this may
lead to workers being killed after significantly less time than specified with
the event-timeout. This is particularly significant during initrd processing:
systemd-udevd will be stopped by systemd when initrd-switch-root.target is
about to be isolated, which usually happens quickly after finding and mounting
the root FS.
If systemd-udevd is started by PID 1 (i.e. basically always), systemd will
kill both udevd and the workers after expiry of TimeoutStopSec. This is
actually better than the built-in udevd timeout, because it's more transparent
and configurable for users. This way users can avoid the mentioned boot problem
by simply increasing StopTimeoutSec= in systemd-udevd.service.
If udevd is not started by systemd (standalone), this is still an
improvement. udevd will kill hanging workers when the event timeout is
reached, which is configurable via the udev.event_timeout= kernel
command line parameter. Before this patch, udevd would simply exit with
workers still running, which would then become zombie processes.
With the timeout removed, the sd_event_now() assertion in manager_exit() can be
dropped.
We'd log to the "console", losing structured logs during configuration file parsing.
Let's be nice to journalctl users, and log to the journal immediately.
# udevadm control --property=HELLO=WORLD
Received udev control message (ENV), unsetting 'HELLO'
# udevadm control --property=HELLO=
Received udev control message (ENV), setting 'HELLO='
Oh no, it's busted. Let's try removing this one little negation real quick
to see if it helps...
# udevadm control --property=HELLO=WORLD
Received udev control message (ENV), setting 'HELLO=WORLD'
# udevadm control --property=HELLO=
Received udev control message (ENV), unsetting 'HELLO'
Feels much better now.
This does the following:
- rename enum udev_builtin_cmd -> UdevBuiltinCmd
- rename struct udev_builtin -> UdevBuiltin
- move type definitions to udev-rules.h
- move prototypes of functions defined in udev-rules.c to udev-rules.h
- drop to use strbuf
- propagate critical errors in applying rules,
- drop limitation for number of tokens per line.
Follow-up for faae64fa3d, which increased the
default number of udev workers per cpu regardless of how big the system is.
It's not really clear from the commit message if the new number of workers
improved the overall time for the boot process or only reduced the number of
times the max number of children limit was reached (and in this case
5406c36844 commit might have been more appropriate in the first place).
But systems with ~1000 CPUs are not rare these days and the worker numbers get
quite large with CPU factor of 8. Spawning more than 2000 workers can't be
healthy on any system, no matter how big.
Indeed the main mistake is the belief that udev is CPU-intensive, and thus the
number of allowed workers has to increase with the number of CPUs. It is not,
at probably has never been. It's I/O bound, and sometimes, bound by resources
such as locks.
This is an argument to:
- scale only weakly with the number of CPUs, and the rationale to switch back
to a scale factor C=2 but with a higher offset number which should affect
systems with a small number of CPUs only. With this patch applied the offset
is increased from O=8 to O=16.
- put an absolute maximum limit to make sure no more than 2048 workers are
spawned no matter how big the system is.
This still provides more workers for the laptop cases (where the number of CPUs
is limited), while avoiding sky-rocketing numbers for big systems.
Note that on most desktop systems, the memory limit will kick in. The following
table collects numbers about children-max. For each scenario, the first column
is the "cpu_limit" limit, and the second number is the minimum amount of memory
for the "cpu_limit" limit to become relevant (with less RAM, memory will limit
the number of children thus "mem_limit" will become the active limit).
| > v240 | < v240 | this patch |
CPUs | C = 8, O = 8 | C = 2, O = 8 | C = 2, O = 16 |
-------------------------------------------------------
1 | 16 2 | 10 1.3 | 18 2 |
2 | 24 3 | 12 1.5 | 20 2 |
4 | 40 5 | 16 2 | 24 3 |
8 | 72 9 | 24 3 | 32 4 |
16 | 136 17 | 40 5 | 48 5 |
64 | 520 65 | 136 17 | 144 18 |
1024 | 8200 1025 | 2056 263 | 2048 256 |
2048 |16392 2049 | 4104 513 | 2048 256 |
This patch is mainly based on Martin Wilck's analyze and comments.
When booting with "udev.log-priority=debug" for example, the output might be
spammed with messages like this:
systemd-udevd[23545]: maximum number (248) of children reached
systemd-udevd[23545]: maximum number (248) of children reached
systemd-udevd[23545]: maximum number (248) of children reached
systemd-udevd[23545]: maximum number (248) of children reached
systemd-udevd[23545]: maximum number (248) of children reached
systemd-udevd[23545]: maximum number (248) of children reached
systemd-udevd[23545]: maximum number (248) of children reached
While the message itself is useful, printing it per batch of events should be
enough.
Originally commented as "devices names might have changed/swapped in the meantime",
but may not. For safety, let's block the following events with same
devpath.
This may fix#6514.
Now, not a few udevd debug logs come from sd-device or sd-hwdb.
Only setting LOG_REALM_UDEV may not sufficient to debug.
We have already similar code in main() and udevadm.
c4b69e990f effectively moved the initalization of socket.
Before that commit:
run → listen_fds → udev_ctrl_new → udev_ctrl_new_from_fd → socket()
After:
run → main_loop → manager_new → udev_ctrl_new_from_fd → socket()
The problem is that main_loop was called after daemonization. Move manager_new
out of main_loop and before daemonization.
Fixes#11314 (hopefully ;)).
v2: Yu Watanabe
sd_event is initialized in main_loop().
$ build/systemctl --version
systemd 239-3555-g6178cbb5b5
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN +PCRE2 default-hierarchy=hybrid
$ git tag v240 -m 'v240'
$ ninja -C build
ninja: Entering directory `build'
[76/76] Linking target fuzz-unit-file.
$ build/systemctl --version
systemd 240
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN +PCRE2 default-hierarchy=hybrid
This is very useful during development, because a precise version string is
embedded in the build product and displayed during boot, so we don't have to
guess answers for questions like "did I just boot the latest version or the one
from before?".
This change creates an overhead for "noop" builds. On my laptop, 'ninja -C
build' that does nothing goes from 0.1 to 0.5 s. It would be nice to avoid
this, but I think that <1 s is still acceptable.
Fixes#7183.
PACKAGE_VERSION is renamed to GIT_VERSION, to make it obvious that this is the
more dynamically changing version string.
Why save to a file? It would be easy to generate the version tag using
run_command(), but we want to go through a file so that stuff gets rebuilt when
this file changes. If we just defined an variable in meson, ninja wouldn't know
it needs to rebuild things.
From the results of CIs in #11076, changing buffer size may cause
issue #10754. So, let's prohibit to change the size if it is already
bound.
This also reverts commit 986ab0d2dc.
Before c4b69e990f, if the socket fd is
passed from pid1, `udev_monitor_set_receive_buffer_size()` (now it is
a wrapper of `sd_device_monitor_set_receive_buffer_size()`) was not
called. Let's preserve the original logic.
