This is similar to the grandparent commit 'fix calculation of offsets table',
except that now the change is for array elements. Same story as before: we need
to make sure that the offsets increase enough taking alignment into account.
While at it, rename 'p' to 'previous' to match similar code in other places.
The offsets specify the ends of variable length data. We would trust the
incoming data, putting the offsets specified in our message
into the offsets tables after doing some superficial verification.
But when actually reading the data we apply alignment, so we would take
the previous offset, align it, making it bigger then current offset, and
then we'd try to read data of negative length.
In the attached example, the message specifies the following offsets:
[1, 4]
but the alignment of those items is
[1, 8]
so we'd calculate the second item as starting at 8 and ending at 4.
The alternative would be to treat gvariant and !gvariant messages differently.
But this is a problem because we check signatures is variuos places before we
have an actual message, for example in sd_bus_add_object_vtable(). It seems
better to treat things consistent (i.e. follow the lowest common denominator)
and disallow empty structures everywhere.
We didn't free one of the fields in two of the places.
$ valgrind --show-leak-kinds=all --leak-check=full \
build/fuzz-bus-message \
test/fuzz/fuzz-bus-message/leak-c09c0e2256d43bc5e2d02748c8d8760e7bc25d20
...
==14457== HEAP SUMMARY:
==14457== in use at exit: 3 bytes in 1 blocks
==14457== total heap usage: 509 allocs, 508 frees, 51,016 bytes allocated
==14457==
==14457== 3 bytes in 1 blocks are definitely lost in loss record 1 of 1
==14457== at 0x4C2EBAB: malloc (vg_replace_malloc.c:299)
==14457== by 0x53AFE79: strndup (in /usr/lib64/libc-2.27.so)
==14457== by 0x4F52EB8: free_and_strndup (string-util.c:1039)
==14457== by 0x4F8E1AB: sd_bus_message_peek_type (bus-message.c:4193)
==14457== by 0x4F76CB5: bus_message_dump (bus-dump.c:144)
==14457== by 0x108F12: LLVMFuzzerTestOneInput (fuzz-bus-message.c:24)
==14457== by 0x1090F7: main (fuzz-main.c:34)
==14457==
==14457== LEAK SUMMARY:
==14457== definitely lost: 3 bytes in 1 blocks
v2: fix error in free_and_strndup()
When the orignal and copied message were the same, but shorter than specified
length l, memory read past the end of the buffer would be performed. A test
case is included: a string that had an embedded NUL ("q\0") is used to replace
"q".
v3: Fix one more bug in free_and_strndup and add tests.
v4: Some style fixed based on review, one more use of free_and_replace, and
make the tests more comprehensive.
--machine has been missing for a while in systemd-stdio-bridge
this syntax can be switched to be more standard.
v2: Support the old syntax too.
timedatectl -H server1.myhostingcompany.com:5555/container1
Closes: #8071
We would verify destination e.g. in sd_bus_message_new_call, but allow setting
any value later on with sd_bus_message_set_destination. I assume this check was
omitted not on purpose.
machined exposes the pseudo-container ".host" as a reference to the host
system, and this means "machinectl login .host" and "machinectl shell
.host" get your a login/shell on the host. systemd-run currently doesn't
allow that. Let's fix that, and make sd-bus understand ".host" as an
alias for connecting to the host system.
This makes bus_slot_disconnect() unref the slot object from bus when
`unref == true` and it is floating, as the function removes the
reference from the relevant bus object.
This reverts 20d4ee2cbc, as it
introduces #9604.
Fixes#9604.
The switch to memory_startswith() changed the logic to only look for a space or
NUL byte after the matched word, but matching the full size should also be
acceptable.
This changed the behavior of parsing of "AUTH\r\n", where m will be set to 4,
since even though the word will match, the check for it being followed by ' '
or NUL will make line_begins() return false.
Tested:
- Using netcat to connect to the private socket directly:
$ echo -ne '\0AUTH\r\n' | sudo nc -U /run/systemd/private
REJECTED EXTERNAL ANONYMOUS
- Running the Ignition blackbox test:
$ sudo sh -c 'PATH=$PWD/bin/amd64:$PATH ./tests.test'
PASS
Fixes: d27b725abf
This adds sd_bus_{get,set}_method_call_timeout().
If the timeout is not set or set to 0, then the timeout value is
parsed from $SYSTEMD_BUS_TIMEOUT= environment variable. If the
environment variable is not set, then built-in timeout is used.
When a slot is disconnected, then slot->match_callback.install_slot
is also disconnected. So, bus_slot_disconnect() removes the install_slot
from the list of slots in bus, although it is a floating object.
This makes install_slot unreffed from bus when it is disconnected.
Fixes#9505 and #9510.
The D-Bus library supplies a va_list variant of
`sd_bus_message_append()` called `sd_bus_message_appendv()`,
but failed to provide a va_list variant of its opposite,
`sd_bus_message_read()`. This commit publicizes a previously static
function as `sd_bus_message_readv()`.
This makes OBJECT_PATH_FOREACH_PREFIX consistent with PATH_FOREACH_PREFIX
and also fixes 7 alerts reported by LGTM at
ac0a087003/files/src/libsystemd/sd-bus/bus-objects.c?sort=name&dir=ASC&mode=heatmap&showExcluded=true#V1383
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
Let's always write "1 << 0", "1 << 1" and so on, except where we need
more than 31 flag bits, where we write "UINT64(1) << 0", and so on to force
64bit values.
Since bb28e68477 parsing failures of
certain unit file settings will result in load failures of units. This
introduces a new load state "bad-setting" that is entered in precisely
this case.
With this addition error messages on bad settings should be a lot more
explicit, as we don't have to show some generic "errno" error in that
case, but can explicitly say that a bad setting is at fault.
Internally this unit load state is entered as soon as any configuration
loader call returns ENOEXEC. Hence: config parser calls should return
ENOEXEC now for such essential unit file settings. Turns out, they
generally already do.
Fixes: #9107
This adds a function sd_bus_slot_set_destroy_callback() to set a function
which can free userdata or perform other cleanups.
sd_bus_slot_get_destory_callback() queries the callback, and is included
for completeness.
Without something like this, for floating asynchronous callbacks, which might
be called or not, depending on the sequence of events, it's hard to perform
resource cleanup. The alternative would be to always perform the cleanup from
the caller too, but that requires more coordination and keeping of some shared
state. It's nicer to keep the cleanup contained between the callback and the
function that requests the callback.
When we allocate an asynchronous match object we will allocate an
asynchronous bus call object to install the match server side.
Previously the call slot would be created as regular slot, i.e.
non-floating which meant installing the match even if it was itself
floating would result in a non-floating slot to be created internally,
which ultimately would mean the sd_bus object would be referenced by it,
and thus never be freed.
Let's fix that by making the match method callback floating in any case
as we have no interest in leaving the bus allocated beyond the match
slot.
Fixes: #8551
This new call allows explicit control of the "floating" state of a bus
slot object. This is useful for creating a bus slot object first,
retaining a reference to it, using it for making changes to the slot
object (for example, set a description) and then handing it over to
sd-bus for lifecycle management.
It's also useful to fix#8551.
This adds a small service "systemd-portabled" and a matching client
"portablectl", which implement the "portable service" concept.
The daemon implements the actual operations, is PolicyKit-enabled and is
activated on demand with exit-on-idle.
Both the daemon and the client are an optional build artifact, enabled
by default rhough.
We were inconsitently using them in some cases, but in majority not.
Using assignment in assert_se is very common, not an exception like in
'if', so let's drop the extra parens everywhere.
Previously we were a bit sloppy with the index and size types of arrays,
we'd regularly use unsigned. While I don't think this ever resulted in
real issues I think we should be more careful there and follow a
stricter regime: unless there's a strong reason not to use size_t for
array sizes and indexes, size_t it should be. Any allocations we do
ultimately will use size_t anyway, and converting forth and back between
unsigned and size_t will always be a source of problems.
Note that on 32bit machines "unsigned" and "size_t" are equivalent, and
on 64bit machines our arrays shouldn't grow that large anyway, and if
they do we have a problem, however that kind of overly large allocation
we have protections for usually, but for overflows we do not have that
so much, hence let's add it.
So yeah, it's a story of the current code being already "good enough",
but I think some extra type hygiene is better.
This patch tries to be comprehensive, but it probably isn't and I missed
a few cases. But I guess we can cover that later as we notice it. Among
smaller fixes, this changes:
1. strv_length()' return type becomes size_t
2. the unit file changes array size becomes size_t
3. DNS answer and query array sizes become size_t
Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=76745
Double newlines (i.e. one empty lines) are great to structure code. But
let's avoid triple newlines (i.e. two empty lines), quadruple newlines,
quintuple newlines, …, that's just spurious whitespace.
It's an easy way to drop 121 lines of code, and keeps the coding style
of our sources a bit tigther.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
sd_bus_open/sd_bus_open_system/sd_bus_open_user are convenient, but
don't allow the description to be set. After they return, the bus is
is already started, and sd_bus_set_description() fails with -EBUSY.
It would be possible to allow sd_bus_set_description() to update the
description "live", but messages are already emitted from sd_bus_open
functions, so it's better to allow the description to be set in
sd_bus_open/sd_bus_open_system/sd_bus_open_user.
Fixes message like:
Bus n/a: changing state UNSET → OPENING
This macro will read a pointer of any type, return it, and set the
pointer to NULL. This is useful as an explicit concept of passing
ownership of a memory area between pointers.
This takes inspiration from Rust:
https://doc.rust-lang.org/std/option/enum.Option.html#method.take
and was suggested by Alan Jenkins (@sourcejedi).
It drops ~160 lines of code from our codebase, which makes me like it.
Also, I think it clarifies passing of ownership, and thus helps
readability a bit (at least for the initiated who know the new macro)
Some versions of asan report the following false positive
when strict_string_checks=1 is passed:
=================================================================
==3297==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f64e4090286 bp 0x7ffe46acd9a0 sp 0x7ffe46acd118 T0)
==3297==The signal is caused by a READ memory access.
==3297==Hint: address points to the zero page.
#0 0x7f64e4090285 in __strlen_sse2 (/lib64/libc.so.6+0xaa285)
#1 0x7f64e5a51e46 (/lib64/libasan.so.4+0x41e46)
#2 0x7f64e4e5e3a0 (/lib64/libglib-2.0.so.0+0x383a0)
#3 0x7f64e4e5e536 in g_dgettext (/lib64/libglib-2.0.so.0+0x38536)
#4 0x7f64e48fac5f (/lib64/libgio-2.0.so.0+0xc1c5f)
#5 0x7f64e4c03978 in g_type_class_ref (/lib64/libgobject-2.0.so.0+0x30978)
#6 0x7f64e4be9567 in g_object_new_with_properties (/lib64/libgobject-2.0.so.0+0x16567)
#7 0x7f64e4be9fd0 in g_object_new (/lib64/libgobject-2.0.so.0+0x16fd0)
#8 0x7f64e48fd43e in g_dbus_message_new_from_blob (/lib64/libgio-2.0.so.0+0xc443e)
#9 0x564a6aa0de52 in main ../src/libsystemd/sd-bus/test-bus-marshal.c:228
#10 0x7f64e4007009 in __libc_start_main (/lib64/libc.so.6+0x21009)
#11 0x564a6aa0a569 in _start (/home/vagrant/systemd/build/test-bus-marshal+0x5569)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib64/libc.so.6+0xaa285) in __strlen_sse2
==3297==ABORTING
It's an external library and errors in external libraries are generally not very
useful for looking for internal bugs.
It would be better not to change the code and use standard suppression
techinques decribed at
https://clang.llvm.org/docs/AddressSanitizer.html#suppressing-reports-in-external-libraries,
but, unfortunaley, none of them seems to be able to suppress fatal errors in asan intself.
There isn't much difference, but in general we prefer to use the standard
functions. glibc provides reallocarray since version 2.26.
I moved explicit_bzero is configure test to the bottom, so that the two stdlib
functions are at the bottom.
This adds some paranoia code that moves some of the fds we allocate for
longer periods of times to fds > 2 if they are allocated below this
boundary. This is a paranoid safety thing, in order to avoid that
external code might end up erroneously use our fds under the assumption
they were valid stdin/stdout/stderr. Think: some app closes
stdin/stdout/stderr and then invokes 'fprintf(stderr, …' which causes
writes on our fds.
This both adds the helper to do the moving as well as ports over a
number of users to this new logic. Since we don't want to litter all our
code with invocations of this I tried to strictly focus on fds we keep
open for long periods of times only and only in code that is frequently
loaded into foreign programs (under the assumptions that in our own
codebase we are smart enough to always keep stdin/stdout/stderr
allocated to avoid this pitfall). Specifically this means all code used
by NSS and our sd-xyz API:
1. our logging APIs
2. sd-event
3. sd-bus
4. sd-resolve
5. sd-netlink
This changed was inspired by this:
https://github.com/systemd/systemd/issues/8075#issuecomment-363689755
This shows that apparently IRL there are programs that do close
stdin/stdout/stderr, and we should accomodate for that.
Note that this won't fix any bugs, this just makes sure that buggy
programs are less likely to interfere with out own code.
we still invoke ssh unnecessarily when there in incompatible or erreneous input
The fallow-up to finish that would make the code a bit more verbose,
as it would require repeating this bit:
```
r = bus_connect_transport(arg_transport, arg_host, false, &bus);
if (r < 0) {
log_error_errno(r, "Failed to create bus connection: %m");
goto finish;
}
sd_bus_set_allow_interactive_authorization(bus, arg_ask_password);
```
in every verb, after parsing.
v2: add waitpid() to avoid a zombie process, switch to SIGTERM from SIGKILL
v3: refactor, wait in bus_start_address()
Currently, sd-bus supports the ability to have thread-local default busses.
However, this is less useful than it can be since all functions which
require an sd_bus* as input require the caller to pass it. This patch adds
a new macro which allows the developer to pass a constant SD_BUS_DEFAULT,
SD_BUS_DEFAULT_USER or SD_BUS_DEFAULT_SYSTEM instead. This reduces work for
the caller.
For example:
r = sd_bus_default(&bus);
r = sd_bus_call_method(bus, ...);
sd_bus_unref(bus);
Becomes:
r = sd_bus_call_method(SD_BUS_DEFAULT, ...);
If the specified thread-local default bus does not exist, the function
calls will return -ENOPKG. No bus will ever be implicitly created.
log.h really should only include the bare minimum of other headers, as
it is really pulled into pretty much everything else and already in
itself one of the most basic pieces of code we have.
Let's hence drop inclusion of:
1. sd-id128.h because it's entirely unneeded in current log.h
2. errno.h, dito.
3. sys/signalfd.h which we can replace by a simple struct forward
declaration
4. process-util.h which was needed for getpid_cached() which we now hide
in a funciton log_emergency_level() instead, which nicely abstracts
the details away.
5. sys/socket.h which was needed for struct iovec, but a simple struct
forward declaration suffices for that too.
Ultimately this actually makes our source tree larger (since users of
the functionality above must now include it themselves, log.h won't do
that for them), but I think it helps to untangle our web of includes a
tiny bit.
(Background: I'd like to isolate the generic bits of src/basic/ enough
so that we can do a git submodule import into casync for it)
This is unused since kdbus is gone, hence remove this too. This permits
us to get rid of sd_bus_send_internal() and just implement sd_bus_send()
directly.
This is useful on direct connections to generate messages with valid
sender fields.
This is particularly useful for services that are accessible both
through direct connections and the broker, as it allows clients to
install matches on the sender service name, and they work the same in
both cases.
This way sd_bus_call_method_async() (which is just a wrapper around
sd_bus_call_async()) can be used to put method calls together that
expect no reply.
With this new API sd-bus can synthesize a local "Connected" signal when
the connection is fully established. It mirrors the local "Disconnected"
signal that is already generated when the connection is terminated. This
is useful to be notified when connection setup is done, in order to
start method calls then, in particular when using "slow" connection
methods (for example slow TCP, or most importantly the "watch_bind"
inotify logic).
Note that one could also use hook into the initial NameAcquired signal
received from the bus broker, but that scheme works only if we actually
connect to a bus. The benefit of "Connected" OTOH is that it works with
any kind of connection.
Ideally, we'd just generate this message unconditionally, but in order
not to break clients that do not expect this message it is opt-in.
This new call is much light sd_bus_is_open(), but returns true only if
the connection is fully set up, i.e. after we finished with the
authentication and Hello() phase. This API is useful for clients in
particular when using the "watch_bind" feature, as that way it can be
determined in advance whether it makes sense to sync on some operation.
Let's directly reference /run instead, so that we can work without /var
being around, or with /var/run being incorrectly set up.
Note that we keep the old socket path in place when referencing the
system bus of containers, as they might be foreign operating systems,
that still don't have adopted /run, and where it makes sense to use the
standardized name instead. On local systems, we insist on /run being set
up properly however, hence this limitation does not apply.
Also, get rid of the UNIX_SYSTEM_BUS_ADDRESS and
UNIX_USER_BUS_ADDRESS_FMT defines. They had a purpose when we still did
kdbus, as we then had to support two different backends. But since
that's gone, we don't need this indirection anymore, hence settle on a
one define only.
Let's remove a number of synchronization points from our service
startups: let's drop synchronous match installation, and let's opt for
asynchronous instead.
Also, let's use sd_bus_match_signal() instead of sd_bus_add_match()
where we can.
These are convenience helpers that hide the match string logic (which we
probably should never have exposed), and instead just takes regular C
arguments.
We usually enqueue a number of these calls on each service
initialization. Let's do this asynchronously, and thus remove
synchronization points. This improves both performance behaviour and
reduces the chances to deadlock.
We currently wait for the RemoveMatch() reply, but then ignore what it
actually says. Let's optimize this a bit, and not even ask for an answer
back: just enqueue the RemoveMatch() operation, and do not request not
wait for any answer.
They do the same thing as their synchronous counterparts, but only
enqueue the operation, thus removing synchronization points during
service initialization.
If the callback function is passed as NULL we'll fallback to generic
implementations of the reply handlers, that terminate the connection if
the requested name cannot be acquired, under the assumption that not
being able to acquire the name is a technical problem.
When kdbus was still around we always had two implementations of the
various control calls: one for dbus1 and one for kdbus. Let'sget rid of
this, simplify things, and just merge the wrappers that used to
multiplex this with the implementations.
No change in behaviour, just some merging of functions
Currently, reply callback timeouts are started the instant the method
calls are enqueued, which can be very early on. For example, the Hello()
method call is enqueued right when sd_bus_start() is called, i.e. before
the socket connection and everything is established.
With this change we instead start the method timeout the moment we
actually leave the authentication phase of the connection. This way, the
timeout the kernel applies on socket connecting, and we apply on the
authentication phase no longer runs in parallel to the Hello() method
call, but all three run serially one after the other, which is
definitely a cleaner approach.
Moreover, this makes the "watch bind" feature a lot more useful, as it
allows enqueuing method calls while we are still waiting for inotify
events, without them timeouting until the connection is actually
established, i.e. when the method call actually has a chance of being
actually run.
This is a change of behaviour of course, but I think the new behaviour
is much better than the old one, since we don't race timeouts against
each other anymore...
This adds a "watch-bind" feature to sd-bus connections. If set and the
AF_UNIX socket we are connecting to doesn't exist yet, we'll establish
an inotify watch instead, and wait for the socket to appear. In other
words, a missing AF_UNIX just makes connecting slower.
This is useful for daemons such as networkd or resolved that shall be
able to run during early-boot, before dbus-daemon is up, and want to
connect to dbus-daemon as soon as it becomes ready.
If we can't process the bus for some reason we shouldn't just disable
the event source, but log something and give up on the connection. Hence
do that, and disconnect.
Currently, when sd-bus is used to issue a method call, and we get a
reply and the specified reply handler fails, we log this locally at
debug priority and proceed. The idea is that a bad server-side reply
should not be fatal for the program, except when the developer
explicitly terminates the event loop.
The reply to the initial Hello() method call we issue when joining a bus
should not be handled like that however. Instead, propagate the error
immediately, as anything that is wrong with the Hello() reply should be
considered a fatal connection problem.
Also, drop UID/GID validity checks from getpeercred() as the kernel will
never pass us invalid UID/GID on userns, but the overflow UID/GID
instead. Add a comment about this.
Using wait_for_terminate_and_check() instead of wait_for_terminate()
let's us simplify, shorten and unify the return value checking and
logging of waitid(). Hence, let's use it all over the place.
This adds a new safe_fork() wrapper around fork() and makes use of it
everywhere. The new wrapper does a couple of things we previously did
manually and separately in a safer, more correct and automatic way:
1. Optionally resets signal handlers/mask in the child
2. Sets a name on all processes we fork off right after forking off (and
the patch assigns useful names for all processes we fork off now,
following a systematic naming scheme: always enclosed in () – in order
to indicate that these are not proper, exec()ed processes, but only
forked off children, and if the process is long-running with only our
own code, without execve()'ing something else, it gets am "sd-" prefix.)
3. Optionally closes all file descriptors in the child
4. Optionally sets a PR_SET_DEATHSIG to SIGTERM in the child, in a safe
way so that the parent dying before this happens being handled
safely.
5. Optionally reopens the logs
6. Optionally connects stdin/stdout/stderr to /dev/null
7. Debug logs about the forked off processes.
Quoting Lennart Poettering in
https://github.com/systemd/systemd/pull/6464#issuecomment-319029293:
> If the kernel allows us to query that data we should also be Ok with passing
> it on to our own caller, regardless if selinux is technically on or off...
The advantage is that this allows gcc to be smarter and reduce linkage:
(before)$ ldd build/libnss_systemd.so.2
linux-vdso.so.1 (0x00007ffeb46ff000)
librt.so.1 => /lib64/librt.so.1 (0x00007f2f60da6000)
libcap.so.2 => /lib64/libcap.so.2 (0x00007f2f60ba1000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f2f60978000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2f60759000)
libc.so.6 => /lib64/libc.so.6 (0x00007f2f60374000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2f61294000)
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f2f600f0000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f2f5feec000)
(after )$ ldd build/libnss_systemd.so.2
linux-vdso.so.1 (0x00007ffe5f543000)
librt.so.1 => /lib64/librt.so.1 (0x00007f427dcaa000)
libcap.so.2 => /lib64/libcap.so.2 (0x00007f427daa5000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f427d886000)
libc.so.6 => /lib64/libc.so.6 (0x00007f427d4a1000)
/lib64/ld-linux-x86-64.so.2 (0x00007f427e196000)
Note that this only works in conjuction with the previous commit: either
of the two commits alone does not have the desired effect on linkage.
Replaces #6464.
Let's replace usage of fputc_unlocked() and friends by __fsetlocking(f,
FSETLOCKING_BYCALLER). This turns off locking for the entire FILE*,
instead of doing individual per-call decision whether to use normal
calls or _unlocked() calls.
This has various benefits:
1. It's easier to read and easier not to forget
2. It's more comprehensive, as fprintf() and friends are covered too
(as these functions have no _unlocked() counterpart)
3. Philosophically, it's a bit more correct, because it's more a
property of the file handle really whether we ever pass it on to another
thread, not of the operations we then apply to it.
This patch reworks all pieces of codes that so far used fxyz_unlocked()
calls to use __fsetlocking() instead. It also reworks all places that
use open_memstream(), i.e. use stdio FILE* for string manipulations.
Note that this in some way a revert of 4b61c87511.
The advantage is that is the name is mispellt, cpp will warn us.
$ git grep -Ee "conf.set\('(HAVE|ENABLE)_" -l|xargs sed -r -i "s/conf.set\('(HAVE|ENABLE)_/conf.set10('\1_/"
$ git grep -Ee '#ifn?def (HAVE|ENABLE)' -l|xargs sed -r -i 's/#ifdef (HAVE|ENABLE)/#if \1/; s/#ifndef (HAVE|ENABLE)/#if ! \1/;'
$ git grep -Ee 'if.*defined\(HAVE' -l|xargs sed -i -r 's/defined\((HAVE_[A-Z0-9_]*)\)/\1/g'
$ git grep -Ee 'if.*defined\(ENABLE' -l|xargs sed -i -r 's/defined\((ENABLE_[A-Z0-9_]*)\)/\1/g'
+ manual changes to meson.build
squash! build-sys: use #if Y instead of #ifdef Y everywhere
v2:
- fix incorrect setting of HAVE_LIBIDN2
THe match cookie was used by kdbus to identify matches we install
uniquely. But given that kdbus is gone, the cookie serves no process
anymore, let's kill it.
When debug logging is enabled we show brief information about every bus
message we send or receieve. Pretty much all information is shown,
except for the error name if a message is an error (interestingly we do
print the error text however). Fix that, and add the error name as well.
This makes it possible to run more dbus tests in a build
environment/chroot where no system bus is available.
To run the dbus test one then can use dbus-run-session.
It's confusing to use a single void* to store data with two different
types, i.e. a userdata value which is safe to pass to ->find(), and a
userdata value which identifies the found object.
Name the latter `found_u`. This naming treats (!c->find) as a degenerate
case. (I.e. at that point, we know the object has already been found :).
Before this commit, if you run `loginctl user-status` from
debug-shell.service (and you have no login sessions for root), you always
see this output:
0
Linger: no
because Properties.GetAll is returning success but without any properties,
when the only find() callback had returned 0 to mean "no object found".
After:
Could not get properties: Unknown object:
'/org/freedesktop/login1/user/self'
BTW I have a fix for more user-friendly messages from logind in this case.
It is pending in my local branch for #6829 "fix `loginctl enable-linger`".
As it turns out the authentication phase times out too often than is
good, mostly due to PRNG pools not being populated during boot. Hence,
let's increase the authentication timeout from 25s to 90s, to cover for
that.
(Note that we leave the D-Bus method call timeout at 25s, matching the
reference implementation's value. And if the auth phase managed to
complete then the pools should be populated enough and mehtod calls
shouldn't take needlessly long anymore).
Fixes: #6418
This prevents `systemctl` from runnning /bin/touch when the following
command is used:
```
systemctl -H '-oProxyCommand=/bin/touch i-shouldnt-be-here' show-environment
```
If a message is too large to fit into the output buffer, it will be
transmitted to the kernel in several chunks. However, the FDs must
only ever be transmitted once or they will bereceived by the remote
end repeatedly.
The D-Bus specification disallows several sets of FDs attached to
one message, however, the reference implementation of D-Bus will
not reject such a message, rather it will reassign the duplicate
FDs to subsequent FD-carrying messages.
This attaches the FD array only to the first byte of the message.
Fixes:
```
$ env -i valgrind --leak-check=full ./build/test-bus-chat
...
==7763== 1,888 (1,824 direct, 64 indirect) bytes in 1 blocks are
definitely lost in loss record 2 of 2
==7763== at 0x4C2FA50: calloc (vg_replace_malloc.c:711)
==7763== by 0x4F8FF9A: sd_bus_new (sd-bus.c:175)
==7763== by 0x4F938BF: sd_bus_open_user (sd-bus.c:1138)
==7763== by 0x109ACD: server_init (test-bus-chat.c:70)
==7763== by 0x10BCF8: main (test-bus-chat.c:526)
==7763==
```
Closes#6481
Let's add a proper validation function, since validation isn't entirely
trivial. Make use of it where applicable. Also make use of
AUDIT_SESSION_INVALID where we need a marker for an invalid audit
session.
Some kdbus_flag and memfd related parts are left behind, because they
are entangled with the "legacy" dbus support.
test-bus-benchmark is switched to "manual". It was already broken before
(in the non-kdbus mode) but apparently nobody noticed. Hopefully it can
be fixed later.
As a follow-up for db3f45e2d2 let's do the
same for all other cases where we create a FILE* with local scope and
know that no other threads hence can have access to it.
For most cases this shouldn't change much really, but this should speed
dbus introspection and calender time formatting up a bit.
This moves pretty much all uses of getpid() over to getpid_raw(). I
didn't specifically check whether the optimization is worth it for each
replacement, but in order to keep things simple and systematic I
switched over everything at once.
Newer D-Bus versions implement the GetConnectionCredentials() driver
call to get all connection creds in one go. Make use of that to reduce
the number of bus calls we do.
When only a single credential field is queried we will still use the old
calls, which we'll also use if the new call isn't implemented.
The bus driver service is always implemented by the owner of the bus,
hence let's shortcut the credential operation and use our cached data.
This makes sure things simply work, given that dbus itself doesn't
support GetConnectionSELinuxSecurityContext() on the bus driver name
itself.
Fixes: #6120
Previously we'd propagate errors returned by user callbacks configured
in vtables back to the users only for method handlers and property
get/set handlers. This does the same for child enumeration and when we
check whether a fallback unit exists.
Without this the failure will be treated as a non-recoverable connection
error and result in connection termination.
Fixes: #6059
The code is mostly correct, but gcc is trying to outsmart us, and emits a
warning for a "llu vs lu" mismatch, even though they are the same size (on alpha):
src/libsystemd/sd-bus/bus-control.c: In function ‘kernel_get_list’:
src/libsystemd/sd-bus/bus-control.c:267:42: error: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘__u64 {aka long unsigned int}’ [-Werror=format=]
if (asprintf(&n, ":1.%llu", name->id) < 0) {
^
src/libsystemd/sd-bus/bus-control.c: In function ‘bus_get_name_creds_kdbus’:
src/libsystemd/sd-bus/bus-control.c:714:47: error: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘__u64 {aka long unsigned int}’ [-Werror=format=]
if (asprintf(&c->unique_name, ":1.%llu", conn_info->id) < 0) {
^
This is hard to work around properly, because kdbus.h uses __u64 which is
defined-differently-despite-being-the-same-size then uint64_t. Thus the simple
solution of using %PRIu64 fails on amd64:
src/libsystemd/sd-bus/bus-control.c:714:47: error: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘__u64 {aka long long unsigned int}’ [-Werror=format=]
if (asprintf(&c->unique_name, ":1.%"PRIu64, conn_info->id) < 0) {
^~~~~~
Let's just avoid the whole issue for now by silencing the warning.
After the next release, we should just get rid of the kdbus code.
Fixes#5561.
Without cc9daff228, this results in:
src/libsystemd/sd-bus/test-bus-vtable-cc.cc:56:1: sorry, unimplemented: non-trivial designated initializers not supported
};
^
This test is mostly a compilation test that checks that various defines in
sd-bus-vtable.h are valid C++. The code is executed, but the results are not
checked (apart from sd-bus functions not returning an error). test-bus-objects
contains pretty extensive tests for this functionality.
The C++ version is only added to meson, since it's simpler there.
Because of the .cc extension, meson will compile the executable with c++.
This test is necessary to properly check the macros in sd-bus-vtable.h. Just
running the headers through g++ is not enough, because the macros are not
exercised.
Follow-up for #5941.
This reverts commit 6355e75610.
The previously mentioned commit inadvertently broke a lot of SELinux related
functionality for both unprivileged users and systemd instances running as
MANAGER_USER. In particular, setting the correct SELinux context after a User=
directive is used would fail to work since we attempt to set the security
context after changing UID. Additionally, it causes activated socket units to
be mislabeled for systemd --user processes since setsockcreatecon() would never
be called.
Reverting this fixes the issues with labeling outlined above, and reinstates
SELinux access checks on unprivileged user services.
We already report builtin interfaces with InterfacesAdded and InterfacesRemoved. However,
we never reported them in GetManagedObjects(). This might end up confusing callers that
want to use those interfaces (or simply rely on the interface count to be coherent).
Report the builtins for all objects that are queried.
This is our own header, we should include use the local-include syntax
("" not <>), to make it clear we are including the one from the build tree.
All other includes of files from src/systemd/ use this scheme.
busctl is not part of libsystemd, and should not be stored under libsystemd.
In particular this is confusing because busctl is linked with libshared, but
stuff in libsystemd is not supposed to depend on libshared.
cg_[all_]unified() test whether a specific controller or all controllers are on
the unified hierarchy. While what's being asked is a simple binary question,
the callers must assume that the functions may fail any time, which
unnecessarily complicates their usages. This complication is unnecessary.
Internally, the test result is cached anyway and there are only a few places
where the test actually needs to be performed.
This patch simplifies cg_[all_]unified().
* cg_[all_]unified() are updated to return bool. If the result can't be
decided, assertion failure is triggered. Error handlings from their callers
are dropped.
* cg_unified_flush() is updated to calculate the new result synchrnously and
return whether it succeeded or not. Places which need to flush the test
result are updated to test for failure. This ensures that all the following
cg_[all_]unified() tests succeed.
* Places which expected possible cg_[all_]unified() failures are updated to
call and test cg_unified_flush() before calling cg_[all_]unified(). This
includes functions used while setting up mounts during boot and
manager_setup_cgroup().
76ec966f0e changed the code from ESHUTDOWN to ERFKILL, but missed one
spot in bus-common-errors.c. Fix that.
The code in transaction.c was checking for ERFKILL, but I'm not sure if this
mismatch had any effect, i.e. if there were any code paths in which the wrong
code actually made difference.
Also add comments when ESHUTDOWN is used in the journal code, so it's easy to
distinguish those cases when grepping. Standarize on the same capitalization.
(There's also a bunch of uses in sd-bus.c, but that's clearly different.)
According to the D-Bus spec (v0.29),
| The direction element on <arg> may be omitted, in which case it
| defaults to "in" for method calls and "out" for signals. Signals only
| allow "out" so while direction may be specified, it's pointless.
Therefore we still should accept a 'direction' attribute, even if it's
useless in reality.
Closes: #4616
We don't have plural in the name of any other -util files and this
inconsistency trips me up every time I try to type this file name
from memory. "formats-util" is even hard to pronounce.
This adds a new invocation ID concept to the service manager. The invocation ID
identifies each runtime cycle of a unit uniquely. A new randomized 128bit ID is
generated each time a unit moves from and inactive to an activating or active
state.
The primary usecase for this concept is to connect the runtime data PID 1
maintains about a service with the offline data the journal stores about it.
Previously we'd use the unit name plus start/stop times, which however is
highly racy since the journal will generally process log data after the service
already ended.
The "invocation ID" kinda matches the "boot ID" concept of the Linux kernel,
except that it applies to an individual unit instead of the whole system.
The invocation ID is passed to the activated processes as environment variable.
It is additionally stored as extended attribute on the cgroup of the unit. The
latter is used by journald to automatically retrieve it for each log logged
message and attach it to the log entry. The environment variable is very easily
accessible, even for unprivileged services. OTOH the extended attribute is only
accessible to privileged processes (this is because cgroupfs only supports the
"trusted." xattr namespace, not "user."). The environment variable may be
altered by services, the extended attribute may not be, hence is the better
choice for the journal.
Note that reading the invocation ID off the extended attribute from journald is
racy, similar to the way reading the unit name for a logging process is.
This patch adds APIs to read the invocation ID to sd-id128:
sd_id128_get_invocation() may be used in a similar fashion to
sd_id128_get_boot().
PID1's own logging is updated to always include the invocation ID when it logs
information about a unit.
A new bus call GetUnitByInvocationID() is added that allows retrieving a bus
path to a unit by its invocation ID. The bus path is built using the invocation
ID, thus providing a path for referring to a unit that is valid only for the
current runtime cycleof it.
Outlook for the future: should the kernel eventually allow passing of cgroup
information along AF_UNIX/SOCK_DGRAM messages via a unique cgroup id, then we
can alter the invocation ID to be generated as hash from that rather than
entirely randomly. This way we can derive the invocation race-freely from the
messages.
Let's bump it further, as this the current limit turns out to be problematic
IRL. Let's bump it to more than twice what we know of is needed.
Fixes: #4068
Old libdbus has a feature that the process is terminated whenever the the bus
connection receives a disconnect. This is pretty useful on desktop apps (where
a disconnect indicates session termination), as well as on command line apps
(where we really shouldn't stay hanging in most cases if dbus daemon goes
down).
Add a similar feature to sd-bus, but make it opt-in rather than opt-out, like
it is on libdbus. Also, if the bus is attached to an event loop just exit the
event loop rather than the the whole process.
If the server side kicks us from the bus, from our view no names are on the bus
anymore, hence let's make sure to dispatch all tracking objects immediately.
In order to add a name to a bus tracking object we need to do some bus
operations: we need to check if the name already exists and add match for it.
Both are synchronous bus calls. While processing those we need to make sure
that the tracking object is not dispatched yet, as it might still be empty, but
is not going to be empty for very long.
hence, block dispatching by removing the object from the dispatch queue while
adding it, and readding it on error.
This adds two (privileged) bus calls Ref() and Unref() to the Unit interface.
The two calls may be used by clients to pin a unit into memory, so that various
runtime properties aren't flushed out by the automatic GC. This is necessary
to permit clients to race-freely acquire runtime results (such as process exit
status/code or accumulated CPU time) on successful service termination.
Ref() and Unref() are fully recursive, hence act like the usual reference
counting concept in C. Taking a reference is a privileged operation, as this
allows pinning units into memory which consumes resources.
Transient units may also gain a reference at the time of creation, via the new
AddRef property (that is only defined for transient units at the time of
creation).
This adds an optional "recursive" counting mode to sd_bus_track. If enabled
adding the same name multiple times to an sd_bus_track object is counted
individually, so that it also has to be removed the same number of times before
it is gone again from the tracking object.
This functionality is useful for implementing local ref counted objects that
peers make take references on.
A following patch will update cgroup handling so that the systemd controller
(/sys/fs/cgroup/systemd) can use the unified hierarchy even if the kernel
resource controllers are on the legacy hierarchies. This would require
distinguishing whether all controllers are on cgroup v2 or only the systemd
controller is. In preparation, this patch renames cg_unified() to
cg_all_unified().
This patch doesn't cause any functional changes.
This adds a new boolean setting DynamicUser= to service files. If set, a new
user will be allocated dynamically when the unit is started, and released when
it is stopped. The user ID is allocated from the range 61184..65519. The user
will not be added to /etc/passwd (but an NSS module to be added later should
make it show up in getent passwd).
For now, care should be taken that the service writes no files to disk, since
this might result in files owned by UIDs that might get assigned dynamically to
a different service later on. Later patches will tighten sandboxing in order to
ensure that this cannot happen, except for a few selected directories.
A simple way to test this is:
systemd-run -p DynamicUser=1 /bin/sleep 99999
If "systemctl -H" is used, let's make sure we first terminate the bus
connection, and only then close the pager. If done in this order ssh will get
an EOF on stdin (as we speak D-Bus through ssh's stdin/stdout), and then
terminate. This makes sure the standard error we were invoked on is released by
ssh, and only that makes sure we don't deadlock on the pager which waits for
all clients closing its input pipe.
(Similar fixes for the various other xyzctl tools that support both pagers and
-H)
Fixes: #3543
This is now the recommended way to do monitoring by upstream D-Bus.
It's also allowed in the default policy, whereas eavesdrop is not
anymore, which effectively broke busctl on many systems.
That function doesn't draw anything on it's own, just returns a string, which
sometimes is more than one character. Also remove "DRAW_" prefix from character
names, TREE_* and ARROW and BLACK_CIRCLE are unambigous on their own, don't
draw anything, and are always used as an argument to special_glyph().
Rename "DASH" to "MDASH", as there's more than one type of dash.
The macro determines the right length of a AF_UNIX "struct sockaddr_un" to pass to
connect() or bind(). It automatically figures out if the socket refers to an
abstract namespace socket, or a socket in the file system, and properly handles
the full length of the path field.
This macro is not only safer, but also simpler to use, than the usual
offsetof() + strlen() logic.
If the SD_BUS_CREDS_SUPPLEMENTARY_GIDS value is requested, the pid is
queried to find out the supplementary gids value from /proc/pid/status.
Otherwise sd_bus_creds_get_supplementary_gids() won't work unless some
other value in mask triggered fetching the pid information.
We don't allow using config symlinks to enable units, but the error message we
printed was awful. Fix that, and generate a more readable error.
Fixes#3010.
Many subsystems define own pager_open_if_enabled() function which
checks '--no-pager' command line argument and open pager depends
on its value. All implementations of pager_open_if_enabled() are
the same. Let's merger this function with pager_open() from the
shared/pager.c and remove pager_open_if_enabled() from all subsytems
to prevent code duplication.
Throughout the tree there's spurious use of spaces separating ++ and --
operators from their respective operands. Make ++ and -- operator
consistent with the majority of existing uses; discard the spaces.
As kdbus won't land in the anticipated way, the bus-proxy is not needed in
its current form. It can be resurrected at any time thanks to the history,
but for now, let's remove it from the sources. If we'll have a similar tool
in the future, it will look quite differently anyway.
Note that stdio-bridge is still available. It was restored from a version
prior to f252ff17, and refactored to make use of the current APIs.
ISO/IEC 9899:1999 §7.21.1/2 says:
Where an argument declared as size_t n specifies the length of the array
for a function, n can have the value zero on a call to that
function. Unless explicitly stated otherwise in the description of a
particular function in this subclause, pointer arguments on such a call
shall still have valid values, as described in 7.1.4.
In base64_append_width memcpy was called as memcpy(x, NULL, 0). GCC 4.9
started making use of this and assumes This worked fine under -O0, but
does something strange under -O3.
This patch fixes a bug in base64_append_width(), fixes a possible bug in
journal_file_append_entry_internal(), and makes use of the new function
to simplify the code in other places.
This commit changes the mapping of the BUS_ERROR_UNIT_MASKED error to ESHUTDOWN. This error is used whenever the
transaction engine is asked to operate on a masked unit. ESHUTDOWN is what is used for the similar case when the unit
file enable/disable logic hits a masked unit file, hence is a natural candidate to be used here too.
Background: before this patch both "job type not applicable" and "unit masked" where mapped to EBADR, which
transaction_add_job_and_dependencies() then checked for. It actually wanted to check exclusively for the former error
condition, not the latter but due to the same mapping this failed to work.
This patch semi-undoes an accidental change made in caffa4ef70, however restores the
error number to ESHUTDOWN instead of the original ENOSYS (for the reasons indicated above).
To make this easier to grok for the future, I added comments to explaining which error conditions are checked for.
Fixes: #2315
This is useful for alternative network management solutions (such as NetworkManager) to push DNS configuration data
into resolved.
The calls will fail should networkd already have taken possesion of a link, so that the bus API is only available if
we don't get the data from networkd.
Setting of dst_id was based on interplay of two booleans,
making the logic hard to follow (for humans and compilers alike).
gcc was confused and emmitted a warning about an uninitialized
variable. Rework the code to make it obvious that dst_id is
set properly.
Go over the entries in the map and check that they make sense.
Tests are added. In the future we might want to do additional
checks, e.g. verifying that the error names are in the expected
format.
errno_from_name used an unusual return convention where 0 meant
"not found". This tripped up config_parse_syscall_errno(),
which would treat that as success. Return -EINVAL instead,
and adjust bus_error_name_to_errno() for the new convention.
Also remove a goto which was used as a simple if and clean
up surroudning code a bit.
If we already degraded the feature level below DO don't bother with sending requests for DS, DNSKEY, RRSIG, NSEC, NSEC3
or NSEC3PARAM RRs. After all, we cannot do DNSSEC validation then anyway, and we better not press a legacy server like
this with such modern concepts.
This also has the benefit that when we try to validate a response we received using DNSSEC, and we detect a limited
server support level while doing so, all further auxiliary DNSSEC queries will fail right-away.
Fixes:
$ make valgrind-tests TESTS=test-bus-cleanup
==6363== 9 bytes in 1 blocks are possibly lost in loss record 1 of 28
==6363== at 0x4C2BBCF: malloc (vg_replace_malloc.c:299)
==6363== by 0x197D12: hexmem (hexdecoct.c:79)
==6363== by 0x183083: bus_socket_start_auth_client (bus-socket.c:639)
==6363== by 0x1832A0: bus_socket_start_auth (bus-socket.c:678)
==6363== by 0x183438: bus_socket_connect (bus-socket.c:705)
==6363== by 0x14B0F2: bus_start_address (sd-bus.c:1053)
==6363== by 0x14B592: sd_bus_start (sd-bus.c:1134)
==6363== by 0x14B95E: sd_bus_open_system (sd-bus.c:1235)
==6363== by 0x1127E2: test_bus_open (test-bus-cleanup.c:42)
==6363== by 0x112AAE: main (test-bus-cleanup.c:87)
==6363==
...
$ ./libtool --mode=execute valgrind ./test-bus-cleanup
==6584== LEAK SUMMARY:
...
==6584== possibly lost: 10,566 bytes in 27 blocks
