Currently, if an event source callback returns an error, we'll disable
the event source and continue. This adds a per-event source flag that if
turned on goes further: the event loop is also exited, propagating the
error code.
This is inspired by some patterns repeatedly seen in #15206.
The idea is that event sources that server the "primary" function of a
program are marked like this, so that if they fail the failure is
instantly propagated and terminates the program.
This tries to address the "bind"/"unbind" uevent kernel API breakage, by
changing the semantics of device tags.
Previously, tags would be applied on uevents (and the database entries
they result in) only depending on the immediate context. This means that
if one uevent causes the tag to be set and the next to be unset, this
would immediately effect what apps would see and the database entries
would contain each time. This is problematic however, as tags are a
filtering concept, and if tags vanish then clients won't hence notice
when a device stops being relevant to them since not only the tags
disappear but immediately also the uevents for it are filtered including
the one necessary for the app to notice that the device lost its tag and
hence relevance.
With this change tags become "sticky". If a tag is applied is once
applied to a device it will stay in place forever, until the device is
removed. Tags can never be removed again. This means that an app
watching a specific set of devices by filtering for a tag is guaranteed
to not only see the events where the tag is set but also all follow-up
events where the tags might be removed again.
This change of behaviour is unfortunate, but is required due to the
kernel introducing new "bind" and "unbind" uevents that generally have
the effect that tags and properties disappear and apps hence don't
notice when a device looses relevance to it. "bind"/"unbind" events were
introduced in kernel 4.12, and are now used in more and more subsystems.
The introduction broke userspace widely, and this commit is an attempt
to provide a way for apps to deal with it.
While tags are now "sticky" a new automatic device property
CURRENT_TAGS is introduced (matching the existing TAGS property) that
always reflects the precise set of tags applied on the most recent
events. Thus, when subscribing to devices through tags, all devices that
ever had the tag put on them will be be seen, and by CURRENT_TAGS it may
be checked whether the device right at the moment matches the tag
requirements.
See: #7587#7018#8221
We frequently want to set a timer relative to the current time. Let's
add an explicit API for this. This not only saves us a few lines of code
everywhere and simplifies things, but also allows us to do correct
overflow checking.
SD_JOURNAL_FOREACH_DATA() and SD_JOURNAL_FOREACH_UNIQUE() would immediately
terminate when a field couldn't be accessed. This can happen for example when a
field is compressed with an unavailable compression format. But it's likely
that this is the wrong thing to do: the caller for example might want to
iterate over the fields but isn't interested in all of them. coredumpctl is
like this: it uses SD_JOURNAL_FOREACH_DATA() but only uses a subset of the
fields.
Add two new functions sd_journal_enumerate_good_data() and
sd_journal_enumerate_good_unique() that retry sd_journal_enumerate_data() and
sd_journal_enumerate_unique() if the return value is something that applies to
a single field: ENOBUS, E2BIG, EOPNOTSUPP.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1856037.
An alternative would be to make the macros themselves smarter instead of adding
new symbols, and do the looping internally in the macro. I don't like that
approach for two reasons. First, it would embed the logic in the macro, so
recompilation would be required if we decide to update the logic. With the
current version of the patch, recompilation is required to use the new symbols,
but after that, library upgrades are enough. So the current approach is safer
in case further updates are needed. Second, our headers use primitive C, and it
is hard to do the macros without using newer features.
Those are fairly trivial to reimplement, but any non-trivial user of sd-bus
is likely to need them. So let's expose them to save everyone the trouble.
I'm keeping the internal functions and making the public ones thin wrappers,
because for the internal uses we don't need the additional asserts, and also we
can't expose _pure_ annotation easily, and dropping it would likely make the
compiled code a bit less efficient.
Consumers of the sd-bus convenience API can't make convenience
helpers of their own without va_list variants.
This commit is a mechanical change splitting out the existing function
bodies into bare va_list variants having a 'v' suffixed to the names.
The original functions now simply create the va_list before forwarding
the call on to the va_list variant, and the va_list variants dispense
with those steps.
When authorizing via PolicyKit we want to process incoming method calls
twice: once to process and figure out that we need PK authentication,
and a second time after we aquired PK authentication to actually execute
the operation. With this new call sd_bus_enqueue_for_read() we have a
way to put an incoming message back into the read queue for this
purpose.
This might have other uses too, for example debugging.
This allows marking messages that contain "sensitive" data with a flag.
If it's set then the messages are erased from memory when the message is
freed.
Similar, a flag may be set on vtable entries: incoming/outgoing message
matching the entry will then automatically be flagged this way.
This is supposed to be an easy method to mark messages containing
potentially sensitive data (such as passwords) for proper destruction.
(Note that this of course is only is as safe as the broker in between is
doing something similar. But let's at least not be the ones at fault
here.)
This adds support for watching for process exits via Linux new pidfd
concept. This makes watching processes and killing them race-free if
properly used, fixing a long-standing UNIX misdesign.
This patch adds implicit and explicit pidfd support to sd-event: if a
process shall be watched and is specified by PID we will now internally
create a pidfd for it and use that, if available. Alternatively a new
constructor for child process event sources is added that takes pidfds
as input.
Besides mere watching of child processes via pidfd two additional
features are added:
→ sd_event_source_send_child_signal() allows sending a signal to the
process being watched in the safest way possible (wrapping
the new pidfd_send_signal() syscall).
→ sd_event_source_set_child_process_own() allows marking a process
watched for destruction as soon as the event source is freed. This is
currently implemented in userspace, but hopefully will become a kernel
feature eventually.
Altogether this means an sd_event_source object is now a safe and stable
concept for referencing processes in race-free way, with automatic
fallback to pre-pidfd kernels.
Note that this patch adds support for this only to sd-event, not to PID
1. That's because PID 1 needs to use waitid(P_ALL) for reaping any
process that might get reparented to it. This currently semantically
conflicts with pidfd use for watching processes since we P_ALL is
undirected and thus might reap process earlier than the pidfd notifies
process end, which is hard to handle. The kernel will likely gain a
concept for excluding specific pidfds from P_ALL watching, as soon as
that is around we can start making use of this in PID 1 too.
In 856ad2a86b sd_bus_add_object_vtable() and
sd_bus_add_fallback_vtable() were changed to take an updated sd_bus_vtable[]
array with additional 'features' and 'names' fields in the union.
The commit tried to check whether the old or the new table format is used, by
looking at the vtable[0].x.start.element_size field, on the assumption that the
added fields caused the structure size to grow. Unfortunately, this assumption
was false, and on arm32 (at least), the structure size is unchanged.
In libsystemd we use symbol versioning and a major.minor.patch semantic
versioning of the library name (major equals the number in the so-name). When
systemd-242 was released, the minor number was (correctly) bumped, but this is
not enough, because no new symbols were added or symbol versions changed. This
means that programs compiled with the new systemd headers and library could be
successfully linked to older versions of the library. For example rpm only
looks at the so-name and the list of versioned symbols, completely ignoring the
major.minor numbers in the library name. But the older library does not
understand the new vtable format, and would return -EINVAL after failing the
size check (on those architectures where the structure size did change, i.e.
all 64 bit architectures).
To force new libsystemd (with the functions that take the updated
sd_bus_vtable[] format) to be used, let's pull in a dummy symbol from the table
definition. This is a bit wasteful, because a dummy pointer has to be stored,
but the effect is negligible. In particular, the pointer doesn't even change
the size of the structure because if fits in an unused area in the union.
The number stored in the new unsigned integer is not checked anywhere. If the
symbol exists, we already know we have the new version of the library, so an
additional check would not tell us anything.
An alternative would be to make sd_bus_add_{object,fallback}_vtable() versioned
symbols, using .symver linker annotations. We would provide
sd_bus_add_{object,fallback}_vtable@LIBSYSTEMD_221 (for backwards
compatibility) and e.g. sd_bus_add_{object,fallback}_vtable@@LIBSYSTEMD_242
(the default) with the new implementation. This would work too, but is more
work. We would have to version at least those two functions. And it turns out
that the .symver linker instructions have to located in the same compilation
unit as the function being annotated. We first compile libsystemd.a, and then
link it into libsystemd.so and various other targets, including
libsystemd-shared.so, and the nss modules. If the .symver annotations were
placed next to the function definitions (in bus-object.c), they would influence
all targets that link libsystemd.a, and cause problems, because those functions
should not be exported there. To export them only in libsystemd.so, compilation
would have to be rearranged, so that the functions exported in libsystemd.so
would not be present in libsystemd.a, but a separate compilation unit containg
them and the .symver annotations would be linked solely into libsystemd.so.
This is certainly possible, but more work than the approach in this patch.
856ad2a86b has one more issue: it relies on the
undefined fields in sd_bus_vtable[] array to be zeros. But the structure
contains a union, and fields of the union do not have to be zero-initalized by
the compiler. This means that potentially, we could have garbarge values there,
for example when reading the old vtable format definition from the new function
implementation. In practice this should not be an issue at all, because vtable
definitions are static data and are placed in the ro-data section, which is
fully initalized, so we know that those undefined areas will be zero. Things
would be different if somebody defined the vtable array on the heap or on the
stack. Let's just document that they should zero-intialize the unused areas
in this case.
The symbol checking code had to be updated because otherwise gcc warns about a
cast from unsigned to a pointer.
It's similar to sd_bus_flush_close_unref() but doesn't do the flushing.
This is useful since this will still discnnect the connection properly
but not synchronously wait for the peer to take our messages.
Primary usecase is within _cleanup_() expressions where synchronously
waiting on the peer is not OK.
This adds a new pair of API calls sd_bus_set_close_on_exit() and
sd_bus_get_close_on_exit(). They control whether an sd_bus object
attached to a an sd-event loop shall automatically be flushed/closed
when the event loop goes down. Usually that's a good thing, except for
very few cases where the bus connection is longer living than the event
loop it is attached on. Specifically, this is the case for nspawn, where
we run the event loop only while the container is up, but afterwards
still want to be able to use the bus connection.
This adds sd_bus_{get,set}_method_call_timeout().
If the timeout is not set or set to 0, then the timeout value is
parsed from $SYSTEMD_BUS_TIMEOUT= environment variable. If the
environment variable is not set, then built-in timeout is used.
The D-Bus library supplies a va_list variant of
`sd_bus_message_append()` called `sd_bus_message_appendv()`,
but failed to provide a va_list variant of its opposite,
`sd_bus_message_read()`. This commit publicizes a previously static
function as `sd_bus_message_readv()`.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
This adds a function sd_bus_slot_set_destroy_callback() to set a function
which can free userdata or perform other cleanups.
sd_bus_slot_get_destory_callback() queries the callback, and is included
for completeness.
Without something like this, for floating asynchronous callbacks, which might
be called or not, depending on the sequence of events, it's hard to perform
resource cleanup. The alternative would be to always perform the cleanup from
the caller too, but that requires more coordination and keeping of some shared
state. It's nicer to keep the cleanup contained between the callback and the
function that requests the callback.
This adds a new call sd_event_add_inotify() which allows watching for
inotify events on specified paths.
sd-event will try to minimize the number of inotify fds allocated, and
will try to add file watches to the same inotify fd objects as far as
that's possible. Doing this kind of inotify object should optimize
behaviour in programs that watch a limited set of mostly independent
files as in most cases a single inotify object will suffice for watching
all files.
Traditionally, this kind of coalescing logic (i.e. that multiple event
sources are implemented on top of a single inotify object) was very hard
to do, as the inotify API had serious limitations: it only allowed
adding watches by path, and would implicitly merge watches installed on
the same node via different path, without letting the caller know about
whether such merging took place or not.
With the advent of O_PATH this issue can be dealt with to some point:
instead of adding a path to watch to an inotify object with
inotify_add_watch() right away, we can open the path with O_PATH first,
call fstat() on the fd, and check the .st_dev/.st_ino fields of that
against a list of watches we already have in place. If we find one we
know that the inotify_add_watch() will update the watch mask of the
existing watch, otherwise it will create a new watch. To make this
race-free we use inotify_add_watch() on the /proc/self/fd/ path of the
O_PATH fd, instead of the original path, so that we do the checking and
watch updating with guaranteed the same inode.
This approach let's us deal safely with inodes that may appear under
various different paths (due to symlinks, hardlinks, bind mounts, fs
namespaces). However it's not a perfect solution: currently the kernel
has no API for changing the watch mask of an existing watch -- unless
you have a path or fd to the original inode. This means we can "merge"
the watches of the same inode of multiple event sources correctly, but
we cannot "unmerge" it again correctly in many cases, as access to the
original inode might have been lost, due to renames, mount/unmount, or
deletions. We could in theory always keep open an O_PATH fd of the inode
to watch so that we can change the mask anytime we want, but this is
highly problematics, as it would consume too many fds (and in fact the
scarcity of fds is the reason why watch descriptors are a separate
concepts from fds) and would keep the backing mounts busy (wds do not
keep mounts busy, fds do). The current implemented approach to all this:
filter in userspace and accept that the watch mask on some inode might
be higher than necessary due to earlier installed event sources that
might have ceased to exist. This approach while ugly shouldn't be too
bad for most cases as the same inodes are probably wacthed for the same
masks in most implementations.
In order to implement priorities correctly a seperate inotify object is
allocated for each priority that is used. This way we get separate
per-priority event queues, of which we never dequeue more than a few
events at a time.
Fixes: #3982
This new call allows explicit control of the "floating" state of a bus
slot object. This is useful for creating a bus slot object first,
retaining a reference to it, using it for making changes to the slot
object (for example, set a description) and then handing it over to
sd-bus for lifecycle management.
It's also useful to fix#8551.
sd_bus_open/sd_bus_open_system/sd_bus_open_user are convenient, but
don't allow the description to be set. After they return, the bus is
is already started, and sd_bus_set_description() fails with -EBUSY.
It would be possible to allow sd_bus_set_description() to update the
description "live", but messages are already emitted from sd_bus_open
functions, so it's better to allow the description to be set in
sd_bus_open/sd_bus_open_system/sd_bus_open_user.
Fixes message like:
Bus n/a: changing state UNSET → OPENING
It is often the case that a file descriptor and its corresponding IO
sd_event_source share a life span. When this is the case, developers will
have to unref the event source and close the file descriptor. Instead, we
can just have the event source take ownership of the file descriptor and
close it when the event source is freed. This is especially useful when
combined with cleanup attributes and sd_event_source_unrefp().
This patch adds two new public functions:
sd_event_source_get_io_fd_own()
sd_event_source_set_io_fd_own()
This is useful on direct connections to generate messages with valid
sender fields.
This is particularly useful for services that are accessible both
through direct connections and the broker, as it allows clients to
install matches on the sender service name, and they work the same in
both cases.
With this new API sd-bus can synthesize a local "Connected" signal when
the connection is fully established. It mirrors the local "Disconnected"
signal that is already generated when the connection is terminated. This
is useful to be notified when connection setup is done, in order to
start method calls then, in particular when using "slow" connection
methods (for example slow TCP, or most importantly the "watch_bind"
inotify logic).
Note that one could also use hook into the initial NameAcquired signal
received from the bus broker, but that scheme works only if we actually
connect to a bus. The benefit of "Connected" OTOH is that it works with
any kind of connection.
Ideally, we'd just generate this message unconditionally, but in order
not to break clients that do not expect this message it is opt-in.
This new call is much light sd_bus_is_open(), but returns true only if
the connection is fully set up, i.e. after we finished with the
authentication and Hello() phase. This API is useful for clients in
particular when using the "watch_bind" feature, as that way it can be
determined in advance whether it makes sense to sync on some operation.
These are convenience helpers that hide the match string logic (which we
probably should never have exposed), and instead just takes regular C
arguments.
We usually enqueue a number of these calls on each service
initialization. Let's do this asynchronously, and thus remove
synchronization points. This improves both performance behaviour and
reduces the chances to deadlock.
They do the same thing as their synchronous counterparts, but only
enqueue the operation, thus removing synchronization points during
service initialization.
If the callback function is passed as NULL we'll fallback to generic
implementations of the reply handlers, that terminate the connection if
the requested name cannot be acquired, under the assumption that not
being able to acquire the name is a technical problem.
This adds a "watch-bind" feature to sd-bus connections. If set and the
AF_UNIX socket we are connecting to doesn't exist yet, we'll establish
an inotify watch instead, and wait for the socket to appear. In other
words, a missing AF_UNIX just makes connecting slower.
This is useful for daemons such as networkd or resolved that shall be
able to run during early-boot, before dbus-daemon is up, and want to
connect to dbus-daemon as soon as it becomes ready.
Lennart Poettering