This adds a minimal Varlink (https://varlink.org/) implementation to our
tree. Given that we already have a JSON logic it's an easy thing to add.
Why bother?
We currently have major problems with IPC before dbus-daemon is up, and
in all components that dbus-daemon itself makes use of (such as various
NSS modules to resolve users as well as the journal which dbus-daemon
logs to). Because of that we so far ended up creating various (usually
crappy) work-arounds either coming up with secondary IPC systems or
sharing data statelessly in /run or similar. Let's clean this up, and
instead use a clean, well-defined, broker-less IPC for cases like that.
This is a minimal implementation of Varlink, i.e. the most basic logic
only. Stuff that's missing is left out on purpose: there's no
introspection/validation and there's no name service. It might make
sense to add that later, but for now let's only do the minimum buy-in we
can get away with. In particular as I'd assume that at least initially
we only use this IPC for our internal communication avoiding
introspection and the name service should be fine.
Specifically, I'd expect that we add IPC interfaces to the following
concepts with this scheme:
1. nss-resolve (so that hostname lookups with resolved work before
resolved is up)
2. journald (so that IPC calls to journald don't have to go through
dbus-daemon thus creating a cyclic dependency between journald and
dbus-daemon)
3. nss-systemd (so that dynamic user lookups via PID 1 work sanely even
inside of dbus-daemon, because otherwise we'd want to use dbus to run
dbus which causes deadlocks)
4. networkd (to make sure one can talk to it in the initrd already,
long before dbus is around)
And there might be other cases similar to this.
Use a trivial header file to share mnt_free_tablep and mnt_free_iterp.
It would be nicer put this in mount-util.h, but libmount.h is not in the
default include path, and the build system would have to be adjusted to pass
pkg-config include path in various places, and it's just not worth the trouble.
A separate header file works nicely.
Apparently, people do use nscd, hence play somewhat nice with it, and
let's explicitly flush nscd caches whenever we register a new
user/group.
This patch only adds the actual refresh request invocation. Later
commits then issue this call at appropriate moments.
Note that the nscd protocol is not officially documented though very
simple. This code is written very defensively so that incompatibilities
don't affect us much.
Given that glibc really has a duty to maintain compat between
differently compiled programs and their system nscd they can't break API
and thus it should be safe for us to implement an alternative,
minimalistic client.
Ideally this kind of explicit, global cache flushing would not be necessary.
However nscd currently has no cache coherency protocol, hence we can't
really implement this better. The only concept it knows is a TTL for
positive hosts lookups. Hoewver for negative lookups or any of the other
tables nothing is available.
This splits out a bunch of functions from fileio.c that have to do with
temporary files. Simply to make the header files a bit shorter, and to
group things more nicely.
No code changes, just some rearranging of source files.
This is really basic stuff and in a follow-up commit will use it all
across the codebase, including in process-util.[ch] which is in
src/basic/. Hence let's move it back to src/basic/ itself.
This doesn't have much effect on the final build, because we link libbasic.a
into libsystemd-shared.so, so in the end, all the object built from basic/
end up in libsystemd-shared. And when the static library is linked into binaries,
any objects that are included in it but are not used are trimmed. Hence, the
size of output artifacts doesn't change:
$ du -sb /var/tmp/inst*
54181861 /var/tmp/inst1 (old)
54207441 /var/tmp/inst1s (old split-usr)
54182477 /var/tmp/inst2 (new)
54208041 /var/tmp/inst2s (new split-usr)
(The negligible change in size is because libsystemd-shared.so is bigger
by a few hundred bytes. I guess it's because symbols are named differently
or something like that.)
The effect is on the build process, in particular partial builds. This change
effectively moves the requirements on some build steps toward the leaves of the
dependency tree. Two effects:
- when building items that do not depend on libsystemd-shared, we
build less stuff for libbasic.a (which wouldn't be used anyway,
so it's a net win).
- when building items that do depend on libshared, we reduce libbasic.a as a
synchronization point, possibly allowing better parallelism.
Method:
1. copy list of .h files from src/basic/meson.build to /tmp/basic
2. $ for i in $(grep '.h$' /tmp/basic); do echo $i; git --no-pager grep "include \"$i\"" src/basic/ 'src/lib*' 'src/nss-*' 'src/journal/sd-journal.c' |grep -v "${i%.h}.c";echo ;done | less
Let's be more careful with what we serialize: let's ensure we never
serialize strings that are longer than LONG_LINE_MAX, so that we know we
can read them back with read_line(…, LONG_LINE_MAX, …) safely.
In order to implement this all serialization functions are move to
serialize.[ch], and internally will do line size checks. We'd rather
skip a serialization line (with a loud warning) than write an overly
long line out. Of course, this is just a second level protection, after
all the data we serialize shouldn't be this long in the first place.
While we are at it also clean up logging: while serializing make sure to
always log about errors immediately. Also, (void)ify all calls we don't
expect errors in (or catch errors as part of the general
fflush_and_check() at the end.
Mempool use is enabled or disabled based on the mempool_use_allowed symbol that
is linked in.
Should fix assert crashes in external programs caused by #9792.
Replaces #10286.
v2:
- use two different source files instead of a gcc constructor
Meson does not care either way, so let's use the simpler syntax. And files()
already gives a list, so nesting this in a list wouldn't be necessary even
if meson did not flatten everything.
Unfortunately this needs libshared to link to libkmod. Before it was linked
into systemd-udevd, udevadm, and systemd each seperately. On most systems this
doesn't make much difference, because at least systemd would be installed, but
it might not be in small chroots. It is a small library, so I hope this is not
a big issue.
This means that when those targets are built, all the sources are built again,
instead of reusing the work done to create libbasic.a and other convenience static
libraries. It would be nice to not do this, but there seems to be no support in
our toolchain for joining multiple static libraries into one. When linking
a static library, any -l arguments are simply ignored by ar/gcc-ar, and .a
libraries given as positional arguments are copied verbatim into the archive
so they objects in them cannot be accessed.
https://stackoverflow.com/questions/2157629/linking-static-libraries-to-other-static-libraries
suggests either unzipping all the archives and putting them back togather,
or using a linker script. Unzipping and zipping back together seems ugly.
The other option is not very nice. The linker script language does not
allow "+" to appear in the filenames, and filenames that meson generates
use that, so files would have to be renamed before a linker script was used.
And we would have to generate the linker script on the fly. Either way, this
doesn't seem attractive. Since those static libraries are a niche use case,
it seems reasonable to just go with the easiest and safest solution and
recompile all the source files. Thanks to ccache, this is probably almost as
cheap as actually reusing the convenience .a libraries.
test-libsystemd-sym.c and test-libudev-sym.c compile fine with the generated
static libs, so it seems that they indeed provide all the symbols they should.
pager.[ch] doesn't use any APIs from src/libsystemd/ or src/shared/
hence there's no reason for it to be in src/shared/, let's move it to
src/basic/ instead.
This enables us to use pager.[ch] APIs from other code in src/basic/,
for example pager_have() and suchlike.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
Instead of compiling those files twice, once for libsystemd and once for
libshared, compile once as a static archive and then link into both.
This reduce the meson target for man=no compile to 1291.
We were including gcrypt-util.[ch] by hand in the few places where it
was used. Create a convenience library to avoid compiling the same
files multiple times.
v2:
- use a separate static library instead of mergin into libbasic
gcrypt_util_sources had to be moved because otherwise they appeared twice
in libshared.so halfproducts, causing an error.
-fvisibility=default is added to libbasic, libshared_static so that the symbols
appear properly in the exported symbol list in libshared.
The advantage is that files are not compiled twice. When configured with -Dman=false,
the ninja target list is reduced from 1588 to 1347 targets. The difference in compilation
time is small (<10%). I think this is because of -O0 and ccache and multiple cores, and
in different settings the compilation time could be reduced. The main advantage is that
errors and warnings are not reported twice.
We already use the "_static" suffix for libshared_static ("shared" is the name
of the library, "static" is the format) and other libs, so let's rename for
consistency.
Also change libsystemd_static_sources to libsystemd_sources, since the same
list is used for both and shorter is better.
So far I avoided adding license headers to meson files, but they are pretty
big and important and should carry license headers like everything else.
I added my own copyright, even though other people modified those files too.
But this is mostly symbolic, so I hope that's OK.
The advantage is that is the name is mispellt, cpp will warn us.
$ git grep -Ee "conf.set\('(HAVE|ENABLE)_" -l|xargs sed -r -i "s/conf.set\('(HAVE|ENABLE)_/conf.set10('\1_/"
$ git grep -Ee '#ifn?def (HAVE|ENABLE)' -l|xargs sed -r -i 's/#ifdef (HAVE|ENABLE)/#if \1/; s/#ifndef (HAVE|ENABLE)/#if ! \1/;'
$ git grep -Ee 'if.*defined\(HAVE' -l|xargs sed -i -r 's/defined\((HAVE_[A-Z0-9_]*)\)/\1/g'
$ git grep -Ee 'if.*defined\(ENABLE' -l|xargs sed -i -r 's/defined\((ENABLE_[A-Z0-9_]*)\)/\1/g'
+ manual changes to meson.build
squash! build-sys: use #if Y instead of #ifdef Y everywhere
v2:
- fix incorrect setting of HAVE_LIBIDN2
This helps prevent symbol collisions with other programs and libraries. In particular,
because PAM modules are loaded into the process that is creating the session, and
systemd creates PAM sessions, the potential for collisions is high.
Disambiguate all systemd calls by tagging a 'version' SD_SHARED.
Fixes#6624
