This adds a unified "copy_flags" parameter to all copy_xyz() function
calls, replacing the various boolean flags so far used. This should make
many invocations more readable as it is clear what behaviour is
precisely requested. This also prepares ground for adding support for
more modes later on.
Given that other file systems (notably: xfs) support reflinks these days, let's
extend the file system snapshotting logic to fall back to plan copies or
reflinks when full btrfs subvolume snapshots are not available.
This essentially makes "systemd-nspawn --ephemeral" and "systemd-nspawn
--template=" available on non-btrfs subvolumes. Of course, both operations will
still be slower on non-btrfs than on btrfs (simply because reflinking each file
individually in a directory tree is still slower than doing this in one step
for a whole subvolume), but it's probably good enough for many cases, and we
should provide the users with the tools, they have to figure out what's good
for them.
Note that "machinectl clone" already had a fallback like this in place, this
patch generalizes this, and adds similar support to our other cases.
The macro determines the right length of a AF_UNIX "struct sockaddr_un" to pass to
connect() or bind(). It automatically figures out if the socket refers to an
abstract namespace socket, or a socket in the file system, and properly handles
the full length of the path field.
This macro is not only safer, but also simpler to use, than the usual
offsetof() + strlen() logic.
Let's make sigkill_wait() take a normal pid_t, and add sigkill_waitp() that
takes a pointer (which is useful for usage in _cleanup_), following the usual
logic we have for this.
gcc is confused by the common idiom of
return errno ? -errno : -ESOMETHING
and thinks a positive value may be returned. Replace this condition
with errno > 0 to help gcc and avoid many spurious warnings. I filed
a gcc rfe a long time ago, but it hard to say if it will ever be
implemented [1].
Both conventions were used in the codebase, this change makes things
more consistent. This is a follow up to bcb161b023.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61846
Change the capability bounding set parser and logic so that the bounding
set is kept as a positive set internally. This means that the set
reflects those capabilities that we want to keep instead of drop.
The current code is not compatible with current dkr protocols anyway,
and dkr has a different focus ("microservices") than nspawn anyway
("whole machine containers"), hence drop support for it, we cannot
reasonably keep this up to date, and it creates the impression we'd
actually care for the microservices usecase.
GLIB has recently started to officially support the gcc cleanup
attribute in its public API, hence let's do the same for our APIs.
With this patch we'll define an xyz_unrefp() call for each public
xyz_unref() call, to make it easy to use inside a
__attribute__((cleanup())) expression. Then, all code is ported over to
make use of this.
The new calls are also documented in the man pages, with examples how to
use them (well, I only added docs where the _unref() call itself already
had docs, and the examples, only cover sd_bus_unrefp() and
sd_event_unrefp()).
This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we
tend to call our destructors these days.
Note that this defines no public macro that wraps gcc's attribute and
makes it easier to use. While I think it's our duty in the library to
make our stuff easy to use, I figure it's not our duty to make gcc's own
features easy to use on its own. Most likely, client code which wants to
make use of this should define its own:
#define _cleanup_(function) __attribute__((cleanup(function)))
Or similar, to make the gcc feature easier to use.
Making this logic public has the benefit that we can remove three header
files whose only purpose was to define these functions internally.
See #2008.
Rather than passing a pointer to return the result, return it directly
from the function calls.
Also, return the result in native endianess, and let the callers care
about the conversion. For hash tables and bloom filters, we don't care,
but in order to keep MAC addresses and DHCP client IDs stable, we
explicitly convert to LE.
Change the "out" parameter from uint8_t[8] to uint64_t. On architectures which
enforce pointer alignment this fixes crashes when we previously cast an
unaligned array to uint64_t*, and on others this should at least improve
performance as the compiler now aligns these properly.
This also simplifies the code in most cases by getting rid of typecasts. The
only place which we can't change is struct duid's en.id, as that is _packed_
and public API, so we can't enforce alignment of the "id" field and have to
use memcpy instead.
The files are named too generically, so that they might conflict with
the upstream project headers. Hence, let's add a "-util" suffix, to
clarify that this are just our utility headers and not any official
upstream headers.
There are more than enough calls doing string manipulations to deserve
its own files, hence do something about it.
This patch also sorts the #include blocks of all files that needed to be
updated, according to the sorting suggestions from CODING_STYLE. Since
pretty much every file needs our string manipulation functions this
effectively means that most files have sorted #include blocks now.
Also touches a few unrelated include files.
With this change we understand more than just leaf quota groups for
btrfs file systems. Specifically:
- When we create a subvolume we can now optionally add the new subvolume
to all qgroups its parent subvolume was member of too. Alternatively
it is also possible to insert an intermediary quota group between the
parent's qgroups and the subvolume's leaf qgroup, which is useful for
a concept of "subtree" qgroups, that contain a subvolume and all its
children.
- The remove logic for subvolumes has been updated to optionally remove
any leaf qgroups or "subtree" qgroups, following the logic above.
- The snapshot logic for subvolumes has been updated to replicate the
original qgroup setup of the source, if it follows the "subtree"
design described above. It will not cover qgroup setups that introduce
arbitrary qgroups, especially those orthogonal to the subvolume
hierarchy.
This also tries to be more graceful when setting up /var/lib/machines as
btrfs. For example, if mkfs.btrfs is missing we don't even try to set it
up as loopback device.
Fixes#1559Fixes#1129
This also allows us to drop build.h from a ton of files, hence do so.
Since we touched the #includes of those files, let's order them properly
according to CODING_STYLE.
Also, make it slightly more powerful, by accepting a flags argument, and
make it safe for handling if more than one cmsg attribute happens to be
attached.
off_t is a really weird type as it is usually 64bit these days (at least
in sane programs), but could theoretically be 32bit. We don't support
off_t as 32bit builds though, but still constantly deal with safely
converting from off_t to other types and back for no point.
Hence, never use the type anymore. Always use uint64_t instead. This has
various benefits, including that we can expose these values directly as
D-Bus properties, and also that the values parse the same in all cases.
Extra details for an action can be supplied when calling polkit's
CheckAuthorization method. Details are a list of key/value string pairs.
Custom policy can use these details when making authorization decisions.
As it turns out machine_name_is_valid() does the exact same thing as
hostname_is_valid() these days, as it just invoked that and checked the
name length was < 64. However, hostname_is_valid() checks the length
against HOST_NAME_MAX anyway (which is 64 on Linux), hence any
additional check is redundant.
We hence replace machine_name_is_valid() by a macro that simply maps it
to hostname_is_valid() but sets the allow_trailing_dot parameter to
false. We also move this this call to hostname-util.h, to the same place
as the hostname_is_valid() declaration.
Sparse archives are good for files like utmp/wtmp and journals, but it
is still expenseive in having to read the files twice to archive them.
Additionally there is a supportability issue as GNU sparse archives are
not supported by many archive libraries and tooling.
Tests are modified to check behaviour with relax and without relax.
New tests are added for hostname_cleanup().
Tests are moved a new file (test-hostname-util) because there's
now a bunch of them.
New parameter is not used anywhere, except in tests, so there should
be no observable change.
free() cannot be used with const pointers. However, our _cleanup_free_
handler features cast logic that hides that qualifier, so we don't get a
warning.
sd_bus_flush_close_unref() is a call that simply combines sd_bus_flush()
(which writes all unwritten messages out) + sd_bus_close() (which
terminates the connection, releasing all unread messages) +
sd_bus_unref() (which frees the connection).
The combination of this call is used pretty frequently in systemd tools
right before exiting, and should also be relevant for most external
clients, and is hence useful to cover in a call of its own.
Previously the combination of the three calls was already done in the
_cleanup_bus_close_unref_ macro, but this was only available internally.
Also see #327
When pulling by image digest the identifiers that
were produced by parsing image digest were later
overwritten by code parsing image tag.
This resulted in invalid identifiers being used
when contacting the remote endpoint, resulting in 404.
Reported here:
http://lists.freedesktop.org/archives/systemd-devel/2015-June/033039.html
This ports a lot of manual code over to sigprocmask_many() and friends.
Also, we now consistly check for sigprocmask() failures with
assert_se(), since the call cannot realistically fail unless there's a
programming error.
Also encloses a few sd_event_add_signal() calls with (void) when we
ignore the return values for it knowingly.
This adds a "char *extra" parameter to tempfn_xxxxxx(), tempfn_random(),
tempfn_ranomd_child(). If non-NULL this string is included in the middle
of the newly created file name. This is useful for being able to
distuingish the kind of temporary file when we see one.
This also adds tests for the three call.
For now, we don't make use of this at all, but port all users over.
Also, when the child is potentially long-running make sure to set a
death signal.
Also, ignore the result of the reset operations explicitly by casting
them to (void).
If NULL is specified for the bus it is now automatically derived from
the passed in message.
This commit also changes a number of invocations of sd_bus_send() to
make use of this.
This should simplify the prototype a bit. The bus parameter is redundant
in most cases, and in the few where it matters it can be derived from
the message via sd_bus_message_get_bus().
Change cunescape() to return a normal error code, so that we can
distuingish OOM errors from parse errors.
This also adds a flags parameter to control whether "relaxed" or normal
parsing shall be done. If set no parse failures are generated, and the
only reason why cunescape() can fail is OOM.
like:
src/shared/install.c: In function ‘unit_file_lookup_state’:
src/shared/install.c:1861:16: warning: ‘r’ may be used uninitialized in
this function [-Wmaybe-uninitialized]
return r < 0 ? r : state;
^
src/shared/install.c:1796:13: note: ‘r’ was declared here
int r;
^
renameat2() exists since Linux 3.15 but btrfs support for the flag
RENAME_NOREPLACE was added later.
This patch implements a fallback when renameat2() returns EINVAL.
EINVAL is the error returned when the filesystem does not support one of
the flags.
This also adds "machinectl import-raw" and "machinectl import-tar" to
wrap these new bus calls.
THe commands basically do for local files that "machinectl pull-raw" and
friends do for remote files.
If /var/lib/machines is mounted as btrfs loopback file system in
/var/lib/machines.raw with this change we automatically grow the file
system as it fills up. After each 10M we write to it during imports, we
check the free disk space, and if the fill level grows beyond 66% we
increase the size of the file system to 3x the fill level (thus lowering
it to 33%).
gpg2 insists on created a trust db even if we tun off all trust db
support. Hence create a temporary home where the trust db is placed, and
remove it after use.
When manipulating container and VM images we need efficient and atomic
directory snapshots and file copies, as well as disk quota. btrfs
provides this, legacy file systems do not. Hence, implicitly create a
loopback file system in /var/lib/machines.raw and mount it to
/var/lib/machines, if that directory is not on btrfs anyway.
This is done implicitly and transparently the first time the user
invokes "machinectl import-xyz".
This allows us to take benefit of btrfs features for container
management without actually having the rest of the system use btrfs.
The loopback is sized 500M initially. Patches to grow it dynamically are
to follow.
The call iterates through cmsg list and closes all fds passed via
SCM_RIGHTS.
This patch also ensures the call is used wherever appropriate, where we
might get spurious fds sent and we should better close them, then leave
them lying around.
Also, allow clients to alter their own objects without any further
priviliges. i.e. this allows clients to kill and lock their own sessions
without involving PK.
We will be woken up on rtnl or dbus activity, so let's just quit if some time has passed and that is the only thing that can happen.
Note that we will always stay around if we expect network activity (e.g. DHCP is enabled), as we are not restarted on that.
After all it is now much more like strjoin() than strappend(). At the
same time, add support for NULL sentinels, even if they are normally not
necessary.
With this change the pull protocol implementation processes will pass
progress data to importd which then passes this information on via the
bus. We use sd_notify() as generic transport for this communication,
making importd listen to them, while matching the incoming messages to
the right transfer.
The old "systemd-import" binary is now an internal tool. We still use it
as asynchronous backend for systemd-importd. Since the import tool might
require some IO and CPU resources (due to qcow2 explosion, and
decompression), and because we might want to run it with more minimal
priviliges we still keep it around as the worker binary to execute as
child process of importd.
machinectl now has verbs for pulling down images, cancelling them and
listing them.
With this change the import tool will now unpack qcow2 images into
normal raw disk images, suitable for usage with nspawn.
This allows has the benefit of also allowing importing Ubuntu Cloud
images for usage with nspawn.
Given that this is also the place to store raw disk images which are
very much bootable with qemu/kvm it sounds like a misnomer to call the
directory "container". Hence, let's change this sooner rather than
later, and use the generic name, in particular since we otherwise try to
use the generic "machine" preferably over the more specific "container"
or "vm".
After all, nspawn can now dissect MBR partition levels, too, hence
".gpt" appears a misnomer. Moreover, the the .raw suffix for these files
is already pretty popular (the Fedora disk images use it for example),
hence sounds like an OK scheme to adopt.
Given the write patterns on disk images, we better should turn COW off
for them. In particular as the file systems used inside the disk images
should do their own data integrity checks anyway and we don't need
multiple layers of it.
I figure "pull-dck" is not a good name, given that one could certainly
read the verb in a way that might be funny for 16year-olds. ;-)
Also, don't hardcode the index URL to use, make it runtime and configure
time configurable instead.
This adds a simply but powerful tool for downloading container images
from the most popular container solution used today. Use it like
this:
# systemd-import pull-dck mattdm/fedora
# systemd-nspawn -M fedora
This will donwload the layers for "mattdm/fedora", and make them
available locally as /var/lib/container/fedora.
The tool is pretty complete, as long as it's only about pulling down
images, or updating them. Pushing or searching is not supported yet.
