picnoir
/
Systemd
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge pull request #10357 from poettering/import-fs 

machinectl import-fs command and other fixes
This commit is contained in:
Zbigniew Jędrzejewski-Szmek 2018-11-29 16:38:46 +01:00 committed by GitHub
parent f781c8fe78 fe1bff158e
commit 401faa3533
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
40 changed files with 1264 additions and 693 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
TODO
View File

@ -41,6 +41,9 @@ Features:
* bootctl,sd-boot: actually honour the "architecture" key
* when a socket unit is spawned with an AF_UNIX path in /var/run, complain and
patch it to use /run instead
* consider splitting out all temporary file creation APIs (we have so many in
fileio.h and elsewhere!) into a new util file of its own.
@ -235,8 +238,7 @@ Features:
the runtime dir as we maintain for the fdstore: i.e. keep it around as long
as the unit is running or has a job queued.
* support projid-based quota in machinectl for containers, and then drop
implicit btrfs loopback magic in machined
* support projid-based quota in machinectl for containers
* Add NetworkNamespacePath= to specify a path to a network namespace
@ -881,9 +883,6 @@ Features:
- "machinectl commit" that takes a writable snapshot of a tree, invokes a
shell in it, and marks it read-only after use
* importd:
- generate a nice warning if mkfs.btrfs is missing
* cryptsetup:
- cryptsetup-generator: allow specification of passwords in crypttab itself
- support rd.luks.allow-discards= kernel cmdline params in cryptsetup generator

56
man/machinectl.xml
View File

@ -70,11 +70,12 @@
top-level directories <filename>/usr</filename>,
<filename>/etc</filename>, and so on.</para></listitem>
<listitem><para>btrfs subvolumes containing OS trees, similar to
normal directory trees.</para></listitem>
<listitem><para>btrfs subvolumes containing OS trees, similar to regular directory trees.</para></listitem>
<listitem><para>Binary "raw" disk images containing MBR or GPT
partition tables and Linux file system partitions.</para></listitem>
<listitem><para>Binary "raw" disk image files containing MBR or GPT partition tables and Linux file
systems.</para></listitem>
<listitem><para>Similarly, block devices containing MBR or GPT partition tables and file systems.</para></listitem>
<listitem><para>The file system tree of the host OS itself.</para></listitem>
</itemizedlist>
@ -649,22 +650,7 @@
units. If the size limit shall be disabled, specify
<literal>-</literal> as size.</para>
<para>Note that per-container size limits are only supported
on btrfs file systems. Also note that, if
<command>set-limit</command> is invoked without an image
parameter, and <filename>/var/lib/machines</filename> is
empty, and the directory is not located on btrfs, a btrfs
loopback file is implicitly created as
<filename>/var/lib/machines.raw</filename> with the given
size, and mounted to
<filename>/var/lib/machines</filename>. The size of the
loopback may later be readjusted with
<command>set-limit</command>, as well. If such a
loopback-mounted <filename>/var/lib/machines</filename>
directory is used, <command>set-limit</command> without an image
name alters both the quota setting within the file system as
well as the loopback file and file system size
itself.</para></listitem>
<para>Note that per-container size limits are only supported on btrfs file systems.</para></listitem>
</varlistentry>
<varlistentry>
@ -802,18 +788,23 @@
image is read from standard input, in which case the second
argument is mandatory.</para>
<para>Both <command>pull-tar</command> and <command>pull-raw</command>
will resize <filename>/var/lib/machines.raw</filename> and the
filesystem therein as necessary. Optionally, the
<option>--read-only</option> switch may be used to create a
read-only container or VM image. No cryptographic validation
is done when importing the images.</para>
<para>Optionally, the <option>--read-only</option> switch may be used to create a read-only container or VM
image. No cryptographic validation is done when importing the images.</para>
<para>Much like image downloads, ongoing imports may be listed
with <command>list-transfers</command> and aborted with
<command>cancel-transfer</command>.</para></listitem>
</varlistentry>
<varlistentry>
<term><command>import-fs</command> <replaceable>DIRECTORY</replaceable> [<replaceable>NAME</replaceable>]</term>
<listitem><para>Imports a container image stored in a local directory into
<filename>/var/lib/machines/</filename>, operates similar to <command>import-tar</command> or
<command>import-raw</command>, but the first argument is the source directory. If supported, this command will
create btrfs snapshot or subvolume for the new image.</para></listitem>
</varlistentry>
<varlistentry>
<term><command>export-tar</command> <replaceable>NAME</replaceable> [<replaceable>FILE</replaceable>]</term>
<term><command>export-raw</command> <replaceable>NAME</replaceable> [<replaceable>FILE</replaceable>]</term>
@ -910,18 +901,7 @@
<filename>/var/lib/machines/</filename> to make them available for
control with <command>machinectl</command>.</para>
<para>Note that some image operations are only supported,
efficient or atomic on btrfs file systems. Due to this, if the
<command>pull-tar</command>, <command>pull-raw</command>,
<command>import-tar</command>, <command>import-raw</command> and
<command>set-limit</command> commands notice that
<filename>/var/lib/machines</filename> is empty and not located on
btrfs, they will implicitly set up a loopback file
<filename>/var/lib/machines.raw</filename> containing a btrfs file
system that is mounted to
<filename>/var/lib/machines</filename>. The size of this loopback
file may be controlled dynamically with
<command>set-limit</command>.</para>
<para>Note that some image operations are only supported, efficient or atomic on btrfs file systems.</para>
<para>Disk images are understood by
<citerefentry><refentrytitle>systemd-nspawn</refentrytitle><manvolnum>1</manvolnum></citerefentry>

12
meson.build
View File

@ -227,6 +227,7 @@ conf.set_quoted('ROOTLIBEXECDIR', rootlibexecdir)
conf.set_quoted('BOOTLIBDIR', bootlibdir)
conf.set_quoted('SYSTEMD_PULL_PATH', join_paths(rootlibexecdir, 'systemd-pull'))
conf.set_quoted('SYSTEMD_IMPORT_PATH', join_paths(rootlibexecdir, 'systemd-import'))
conf.set_quoted('SYSTEMD_IMPORT_FS_PATH', join_paths(rootlibexecdir, 'systemd-import-fs'))
conf.set_quoted('SYSTEMD_EXPORT_PATH', join_paths(rootlibexecdir, 'systemd-export'))
conf.set_quoted('VENDOR_KEYRING_PATH', join_paths(rootlibexecdir, 'import-pubring.gpg'))
conf.set_quoted('USER_KEYRING_PATH', join_paths(pkgsysconfdir, 'import-pubring.gpg'))
@ -2137,6 +2138,14 @@ if conf.get('ENABLE_IMPORTD') == 1
install : true,
install_dir : rootlibexecdir)
systemd_import_fs = executable('systemd-import-fs',
systemd_import_fs_sources,
include_directories : includes,
link_with : [libshared],
install_rpath : rootlibexecdir,
install : true,
install_dir : rootlibexecdir)
systemd_export = executable('systemd-export',
systemd_export_sources,
include_directories : includes,
@ -2148,7 +2157,8 @@ if conf.get('ENABLE_IMPORTD') == 1
install_rpath : rootlibexecdir,
install : true,
install_dir : rootlibexecdir)
public_programs += [systemd_pull, systemd_import, systemd_export]
public_programs += [systemd_pull, systemd_import, systemd_import_fs, systemd_export]
endif
if conf.get('ENABLE_REMOTE') == 1 and conf.get('HAVE_LIBCURL') == 1

119
src/basic/btrfs-util.c
View File

@ -870,96 +870,6 @@ int btrfs_subvol_set_subtree_quota_limit(const char *path, uint64_t subvol_id, u
return btrfs_subvol_set_subtree_quota_limit_fd(fd, subvol_id, referenced_max);
}
int btrfs_resize_loopback_fd(int fd, uint64_t new_size, bool grow_only) {
struct btrfs_ioctl_vol_args args = {};
char p[SYS_BLOCK_PATH_MAX("/loop/backing_file")], q[DEV_NUM_PATH_MAX];
_cleanup_free_ char *backing = NULL;
_cleanup_close_ int loop_fd = -1, backing_fd = -1;
struct stat st;
dev_t dev = 0;
int r;
/* In contrast to btrfs quota ioctls ftruncate() cannot make sense of "infinity" or file sizes > 2^31 */
if (!FILE_SIZE_VALID(new_size))
return -EINVAL;
/* btrfs cannot handle file systems < 16M, hence use this as minimum */
if (new_size < 16*1024*1024)
new_size = 16*1024*1024;
r = btrfs_get_block_device_fd(fd, &dev);
if (r < 0)
return r;
if (r == 0)
return -ENODEV;
xsprintf_sys_block_path(p, "/loop/backing_file", dev);
r = read_one_line_file(p, &backing);
if (r == -ENOENT)
return -ENODEV;
if (r < 0)
return r;
if (isempty(backing) || !path_is_absolute(backing))
return -ENODEV;
backing_fd = open(backing, O_RDWR|O_CLOEXEC|O_NOCTTY);
if (backing_fd < 0)
return -errno;
if (fstat(backing_fd, &st) < 0)
return -errno;
if (!S_ISREG(st.st_mode))
return -ENODEV;
if (new_size == (uint64_t) st.st_size)
return 0;
if (grow_only && new_size < (uint64_t) st.st_size)
return -EINVAL;
xsprintf_dev_num_path(q, "block", dev);
loop_fd = open(q, O_RDWR|O_CLOEXEC|O_NOCTTY);
if (loop_fd < 0)
return -errno;
if (snprintf(args.name, sizeof(args.name), "%" PRIu64, new_size) >= (int) sizeof(args.name))
return -EINVAL;
if (new_size < (uint64_t) st.st_size) {
/* Decrease size: first decrease btrfs size, then shorten loopback */
if (ioctl(fd, BTRFS_IOC_RESIZE, &args) < 0)
return -errno;
}
if (ftruncate(backing_fd, new_size) < 0)
return -errno;
if (ioctl(loop_fd, LOOP_SET_CAPACITY, 0) < 0)
return -errno;
if (new_size > (uint64_t) st.st_size) {
/* Increase size: first enlarge loopback, then increase btrfs size */
if (ioctl(fd, BTRFS_IOC_RESIZE, &args) < 0)
return -errno;
}
/* Make sure the free disk space is correctly updated for both file systems */
(void) fsync(fd);
(void) fsync(backing_fd);
return 1;
}
int btrfs_resize_loopback(const char *p, uint64_t new_size, bool grow_only) {
_cleanup_close_ int fd = -1;
fd = open(p, O_RDONLY|O_NOCTTY|O_CLOEXEC);
if (fd < 0)
return -errno;
return btrfs_resize_loopback_fd(fd, new_size, grow_only);
}
int btrfs_qgroupid_make(uint64_t level, uint64_t id, uint64_t *ret) {
assert(ret);
@ -1503,7 +1413,12 @@ static int copy_subtree_quota_limits(int fd, uint64_t old_subvol, uint64_t new_s
return changed;
}
static int subvol_snapshot_children(int old_fd, int new_fd, const char *subvolume, uint64_t old_subvol_id, BtrfsSnapshotFlags flags) {
static int subvol_snapshot_children(
int old_fd,
int new_fd,
const char *subvolume,
uint64_t old_subvol_id,
BtrfsSnapshotFlags flags) {
struct btrfs_ioctl_search_args args = {
.key.tree_id = BTRFS_ROOT_TREE_OBJECTID,
@ -1683,7 +1598,14 @@ static int subvol_snapshot_children(int old_fd, int new_fd, const char *subvolum
return 0;
}
int btrfs_subvol_snapshot_fd(int old_fd, const char *new_path, BtrfsSnapshotFlags flags) {
int btrfs_subvol_snapshot_fd_full(
int old_fd,
const char *new_path,
BtrfsSnapshotFlags flags,
copy_progress_path_t progress_path,
copy_progress_bytes_t progress_bytes,
void *userdata) {
_cleanup_close_ int new_fd = -1;
const char *subvolume;
int r;
@ -1711,7 +1633,7 @@ int btrfs_subvol_snapshot_fd(int old_fd, const char *new_path, BtrfsSnapshotFlag
} else if (r < 0)
return r;
r = copy_directory_fd(old_fd, new_path, COPY_MERGE|COPY_REFLINK);
r = copy_directory_fd_full(old_fd, new_path, COPY_MERGE|COPY_REFLINK, progress_path, progress_bytes, userdata);
if (r < 0)
goto fallback_fail;
@ -1748,7 +1670,14 @@ int btrfs_subvol_snapshot_fd(int old_fd, const char *new_path, BtrfsSnapshotFlag
return subvol_snapshot_children(old_fd, new_fd, subvolume, 0, flags);
}
int btrfs_subvol_snapshot(const char *old_path, const char *new_path, BtrfsSnapshotFlags flags) {
int btrfs_subvol_snapshot_full(
const char *old_path,
const char *new_path,
BtrfsSnapshotFlags flags,
copy_progress_path_t progress_path,
copy_progress_bytes_t progress_bytes,
void *userdata) {
_cleanup_close_ int old_fd = -1;
assert(old_path);
@ -1758,7 +1687,7 @@ int btrfs_subvol_snapshot(const char *old_path, const char *new_path, BtrfsSnaps
if (old_fd < 0)
return -errno;
return btrfs_subvol_snapshot_fd(old_fd, new_path, flags);
return btrfs_subvol_snapshot_fd_full(old_fd, new_path, flags, progress_path, progress_bytes, userdata);
}
int btrfs_qgroup_find_parents(int fd, uint64_t qgroupid, uint64_t **ret) {

15
src/basic/btrfs-util.h
View File

@ -7,6 +7,7 @@
#include "sd-id128.h"
#include "copy.h"
#include "time-util.h"
typedef struct BtrfsSubvolInfo {
@ -61,14 +62,18 @@ int btrfs_quota_scan_start(int fd);
int btrfs_quota_scan_wait(int fd);
int btrfs_quota_scan_ongoing(int fd);
int btrfs_resize_loopback_fd(int fd, uint64_t size, bool grow_only);
int btrfs_resize_loopback(const char *path, uint64_t size, bool grow_only);
int btrfs_subvol_make(const char *path);
int btrfs_subvol_make_fd(int fd, const char *subvolume);
int btrfs_subvol_snapshot_fd(int old_fd, const char *new_path, BtrfsSnapshotFlags flags);
int btrfs_subvol_snapshot(const char *old_path, const char *new_path, BtrfsSnapshotFlags flags);
int btrfs_subvol_snapshot_fd_full(int old_fd, const char *new_path, BtrfsSnapshotFlags flags, copy_progress_path_t progress_path, copy_progress_bytes_t progress_bytes, void *userdata);
static inline int btrfs_subvol_snapshot_fd(int old_fd, const char *new_path, BtrfsSnapshotFlags flags) {
return btrfs_subvol_snapshot_fd_full(old_fd, new_path, flags, NULL, NULL, NULL);
}
int btrfs_subvol_snapshot_full(const char *old_path, const char *new_path, BtrfsSnapshotFlags flags, copy_progress_path_t progress_path, copy_progress_bytes_t progress_bytes, void *userdata);
static inline int btrfs_subvol_snapshot(const char *old_path, const char *new_path, BtrfsSnapshotFlags flags) {
return btrfs_subvol_snapshot_full(old_path, new_path, flags, NULL, NULL, NULL);
}
int btrfs_subvol_remove(const char *path, BtrfsRemoveFlags flags);
int btrfs_subvol_remove_fd(int fd, const char *subvolume, BtrfsRemoveFlags flags);

118
src/basic/copy.c
View File

@ -90,7 +90,9 @@ int copy_bytes_full(
uint64_t max_bytes,
CopyFlags copy_flags,
void **ret_remains,
size_t *ret_remains_size) {
size_t *ret_remains_size,
copy_progress_bytes_t progress,
void *userdata) {
bool try_cfr = true, try_sendfile = true, try_splice = true;
int r, nonblock_pipe = -1;
@ -161,8 +163,6 @@ int copy_bytes_full(
return 1; /* we copied only some number of bytes, which worked, but this means we didn't hit EOF, return 1 */
}
}
log_debug_errno(r, "Reflinking didn't work, falling back to non-reflink copying: %m");
}
}
}
@ -308,10 +308,17 @@ int copy_bytes_full(
}
next:
if (progress) {
r = progress(n, userdata);
if (r < 0)
return r;
}
if (max_bytes != (uint64_t) -1) {
assert(max_bytes >= (uint64_t) n);
max_bytes -= n;
}
/* sendfile accepts at most SSIZE_MAX-offset bytes to copy,
* so reduce our maximum by the amount we already copied,
* but don't go below our copy buffer size, unless we are
@ -363,7 +370,9 @@ static int fd_copy_regular(
const char *to,
uid_t override_uid,
gid_t override_gid,
CopyFlags copy_flags) {
CopyFlags copy_flags,
copy_progress_bytes_t progress,
void *userdata) {
_cleanup_close_ int fdf = -1, fdt = -1;
struct timespec ts[2];
@ -381,7 +390,7 @@ static int fd_copy_regular(
if (fdt < 0)
return -errno;
r = copy_bytes(fdf, fdt, (uint64_t) -1, copy_flags);
r = copy_bytes_full(fdf, fdt, (uint64_t) -1, copy_flags, NULL, NULL, progress, userdata);
if (r < 0) {
(void) unlinkat(dt, to, 0);
return r;
@ -483,7 +492,11 @@ static int fd_copy_directory(
unsigned depth_left,
uid_t override_uid,
gid_t override_gid,
CopyFlags copy_flags) {
CopyFlags copy_flags,
const char *display_path,
copy_progress_path_t progress_path,
copy_progress_bytes_t progress_bytes,
void *userdata) {
_cleanup_close_ int fdf = -1, fdt = -1;
_cleanup_closedir_ DIR *d = NULL;
@ -524,6 +537,8 @@ static int fd_copy_directory(
r = 0;
FOREACH_DIRENT_ALL(de, d, return -errno) {
const char *child_display_path = NULL;
_cleanup_free_ char *dp = NULL;
struct stat buf;
int q;
@ -535,6 +550,17 @@ static int fd_copy_directory(
continue;
}
if (progress_path) {
if (display_path)
child_display_path = dp = strjoin(display_path, "/", de->d_name);
else
child_display_path = de->d_name;
r = progress_path(child_display_path, &buf, userdata);
if (r < 0)
return r;
}
if (S_ISDIR(buf.st_mode)) {
/*
* Don't descend into directories on other file systems, if this is requested. We do a simple
@ -566,9 +592,9 @@ static int fd_copy_directory(
continue;
}
q = fd_copy_directory(dirfd(d), de->d_name, &buf, fdt, de->d_name, original_device, depth_left-1, override_uid, override_gid, copy_flags);
q = fd_copy_directory(dirfd(d), de->d_name, &buf, fdt, de->d_name, original_device, depth_left-1, override_uid, override_gid, copy_flags, child_display_path, progress_path, progress_bytes, userdata);
} else if (S_ISREG(buf.st_mode))
q = fd_copy_regular(dirfd(d), de->d_name, &buf, fdt, de->d_name, override_uid, override_gid, copy_flags);
q = fd_copy_regular(dirfd(d), de->d_name, &buf, fdt, de->d_name, override_uid, override_gid, copy_flags, progress_bytes, userdata);
else if (S_ISLNK(buf.st_mode))
q = fd_copy_symlink(dirfd(d), de->d_name, &buf, fdt, de->d_name, override_uid, override_gid, copy_flags);
else if (S_ISFIFO(buf.st_mode))
@ -606,7 +632,18 @@ static int fd_copy_directory(
return r;
}
int copy_tree_at(int fdf, const char *from, int fdt, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags) {
int copy_tree_at_full(
int fdf,
const char *from,
int fdt,
const char *to,
uid_t override_uid,
gid_t override_gid,
CopyFlags copy_flags,
copy_progress_path_t progress_path,
copy_progress_bytes_t progress_bytes,
void *userdata) {
struct stat st;
assert(from);
@ -616,9 +653,9 @@ int copy_tree_at(int fdf, const char *from, int fdt, const char *to, uid_t overr
return -errno;
if (S_ISREG(st.st_mode))
return fd_copy_regular(fdf, from, &st, fdt, to, override_uid, override_gid, copy_flags);
return fd_copy_regular(fdf, from, &st, fdt, to, override_uid, override_gid, copy_flags, progress_bytes, userdata);
else if (S_ISDIR(st.st_mode))
return fd_copy_directory(fdf, from, &st, fdt, to, st.st_dev, COPY_DEPTH_MAX, override_uid, override_gid, copy_flags);
return fd_copy_directory(fdf, from, &st, fdt, to, st.st_dev, COPY_DEPTH_MAX, override_uid, override_gid, copy_flags, NULL, progress_path, progress_bytes, userdata);
else if (S_ISLNK(st.st_mode))
return fd_copy_symlink(fdf, from, &st, fdt, to, override_uid, override_gid, copy_flags);
else if (S_ISFIFO(st.st_mode))
@ -629,11 +666,14 @@ int copy_tree_at(int fdf, const char *from, int fdt, const char *to, uid_t overr
return -EOPNOTSUPP;
}
int copy_tree(const char *from, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags) {
return copy_tree_at(AT_FDCWD, from, AT_FDCWD, to, override_uid, override_gid, copy_flags);
}
int copy_directory_fd_full(
int dirfd,
const char *to,
CopyFlags copy_flags,
copy_progress_path_t progress_path,
copy_progress_bytes_t progress_bytes,
void *userdata) {
int copy_directory_fd(int dirfd, const char *to, CopyFlags copy_flags) {
struct stat st;
assert(dirfd >= 0);
@ -645,10 +685,17 @@ int copy_directory_fd(int dirfd, const char *to, CopyFlags copy_flags) {
if (!S_ISDIR(st.st_mode))
return -ENOTDIR;
return fd_copy_directory(dirfd, NULL, &st, AT_FDCWD, to, st.st_dev, COPY_DEPTH_MAX, UID_INVALID, GID_INVALID, copy_flags);
return fd_copy_directory(dirfd, NULL, &st, AT_FDCWD, to, st.st_dev, COPY_DEPTH_MAX, UID_INVALID, GID_INVALID, copy_flags, NULL, progress_path, progress_bytes, userdata);
}
int copy_directory(const char *from, const char *to, CopyFlags copy_flags) {
int copy_directory_full(
const char *from,
const char *to,
CopyFlags copy_flags,
copy_progress_path_t progress_path,
copy_progress_bytes_t progress_bytes,
void *userdata) {
struct stat st;
assert(from);
@ -660,10 +707,16 @@ int copy_directory(const char *from, const char *to, CopyFlags copy_flags) {
if (!S_ISDIR(st.st_mode))
return -ENOTDIR;
return fd_copy_directory(AT_FDCWD, from, &st, AT_FDCWD, to, st.st_dev, COPY_DEPTH_MAX, UID_INVALID, GID_INVALID, copy_flags);
return fd_copy_directory(AT_FDCWD, from, &st, AT_FDCWD, to, st.st_dev, COPY_DEPTH_MAX, UID_INVALID, GID_INVALID, copy_flags, NULL, progress_path, progress_bytes, userdata);
}
int copy_file_fd(const char *from, int fdt, CopyFlags copy_flags) {
int copy_file_fd_full(
const char *from,
int fdt,
CopyFlags copy_flags,
copy_progress_bytes_t progress_bytes,
void *userdata) {
_cleanup_close_ int fdf = -1;
int r;
@ -674,7 +727,7 @@ int copy_file_fd(const char *from, int fdt, CopyFlags copy_flags) {
if (fdf < 0)
return -errno;
r = copy_bytes(fdf, fdt, (uint64_t) -1, copy_flags);
r = copy_bytes_full(fdf, fdt, (uint64_t) -1, copy_flags, NULL, NULL, progress_bytes, userdata);
(void) copy_times(fdf, fdt);
(void) copy_xattr(fdf, fdt);
@ -682,7 +735,16 @@ int copy_file_fd(const char *from, int fdt, CopyFlags copy_flags) {
return r;
}
int copy_file(const char *from, const char *to, int flags, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags) {
int copy_file_full(
const char *from,
const char *to,
int flags,
mode_t mode,
unsigned chattr_flags,
CopyFlags copy_flags,
copy_progress_bytes_t progress_bytes,
void *userdata) {
int fdt = -1, r;
assert(from);
@ -697,7 +759,7 @@ int copy_file(const char *from, const char *to, int flags, mode_t mode, unsigned
if (chattr_flags != 0)
(void) chattr_fd(fdt, chattr_flags, (unsigned) -1, NULL);
r = copy_file_fd(from, fdt, copy_flags);
r = copy_file_fd_full(from, fdt, copy_flags, progress_bytes, userdata);
if (r < 0) {
close(fdt);
(void) unlink(to);
@ -712,7 +774,15 @@ int copy_file(const char *from, const char *to, int flags, mode_t mode, unsigned
return 0;
}
int copy_file_atomic(const char *from, const char *to, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags) {
int copy_file_atomic_full(
const char *from,
const char *to,
mode_t mode,
unsigned chattr_flags,
CopyFlags copy_flags,
copy_progress_bytes_t progress_bytes,
void *userdata) {
_cleanup_(unlink_and_freep) char *t = NULL;
_cleanup_close_ int fdt = -1;
int r;
@ -745,7 +815,7 @@ int copy_file_atomic(const char *from, const char *to, mode_t mode, unsigned cha
if (chattr_flags != 0)
(void) chattr_fd(fdt, chattr_flags, (unsigned) -1, NULL);
r = copy_file_fd(from, fdt, copy_flags);
r = copy_file_fd_full(from, fdt, copy_flags, progress_bytes, userdata);
if (r < 0)
return r;

52
src/basic/copy.h
View File

@ -1,9 +1,11 @@
/* SPDX-License-Identifier: LGPL-2.1+ */
#pragma once
#include <fcntl.h>
#include <inttypes.h>
#include <stdbool.h>
#include <stdint.h>
#include <sys/stat.h>
#include <sys/types.h>
typedef enum CopyFlags {
@ -13,16 +15,46 @@ typedef enum CopyFlags {
COPY_SAME_MOUNT = 1 << 3, /* Don't descend recursively into other file systems, across mount point boundaries */
} CopyFlags;
int copy_file_fd(const char *from, int to, CopyFlags copy_flags);
int copy_file(const char *from, const char *to, int open_flags, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags);
int copy_file_atomic(const char *from, const char *to, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags);
int copy_tree(const char *from, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags);
int copy_tree_at(int fdf, const char *from, int fdt, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags);
int copy_directory_fd(int dirfd, const char *to, CopyFlags copy_flags);
int copy_directory(const char *from, const char *to, CopyFlags copy_flags);
int copy_bytes_full(int fdf, int fdt, uint64_t max_bytes, CopyFlags copy_flags, void **ret_remains, size_t *ret_remains_size);
static inline int copy_bytes(int fdf, int fdt, uint64_t max_bytes, CopyFlags copy_flags) {
return copy_bytes_full(fdf, fdt, max_bytes, copy_flags, NULL, NULL);
typedef int (*copy_progress_bytes_t)(uint64_t n_bytes, void *userdata);
typedef int (*copy_progress_path_t)(const char *path, const struct stat *st, void *userdata);
int copy_file_fd_full(const char *from, int to, CopyFlags copy_flags, copy_progress_bytes_t progress, void *userdata);
static inline int copy_file_fd(const char *from, int to, CopyFlags copy_flags) {
return copy_file_fd_full(from, to, copy_flags, NULL, NULL);
}
int copy_file_full(const char *from, const char *to, int open_flags, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags, copy_progress_bytes_t progress, void *userdata);
static inline int copy_file(const char *from, const char *to, int open_flags, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags) {
return copy_file_full(from, to, open_flags, mode, chattr_flags, copy_flags, NULL, NULL);
}
int copy_file_atomic_full(const char *from, const char *to, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags, copy_progress_bytes_t progress, void *userdata);
static inline int copy_file_atomic(const char *from, const char *to, mode_t mode, unsigned chattr_flags, CopyFlags copy_flags) {
return copy_file_atomic_full(from, to, mode, chattr_flags, copy_flags, NULL, NULL);
}
int copy_tree_at_full(int fdf, const char *from, int fdt, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags, copy_progress_path_t progress_path, copy_progress_bytes_t progress_bytes, void *userdata);
static inline int copy_tree_at(int fdf, const char *from, int fdt, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags) {
return copy_tree_at_full(fdf, from, fdt, to, override_uid, override_gid, copy_flags, NULL, NULL, NULL);
}
static inline int copy_tree(const char *from, const char *to, uid_t override_uid, gid_t override_gid, CopyFlags copy_flags) {
return copy_tree_at_full(AT_FDCWD, from, AT_FDCWD, to, override_uid, override_gid, copy_flags, NULL, NULL, NULL);
}
int copy_directory_fd_full(int dirfd, const char *to, CopyFlags copy_flags, copy_progress_path_t progress_path, copy_progress_bytes_t progress_bytes, void *userdata);
static inline int copy_directory_fd(int dirfd, const char *to, CopyFlags copy_flags) {
return copy_directory_fd_full(dirfd, to, copy_flags, NULL, NULL, NULL);
}
int copy_directory_full(const char *from, const char *to, CopyFlags copy_flags, copy_progress_path_t progress_path, copy_progress_bytes_t progress_bytes, void *userdata);
static inline int copy_directory(const char *from, const char *to, CopyFlags copy_flags) {
return copy_directory_full(from, to, copy_flags, NULL, NULL, NULL);
}
int copy_bytes_full(int fdf, int fdt, uint64_t max_bytes, CopyFlags copy_flags, void **ret_remains, size_t *ret_remains_size, copy_progress_bytes_t progress, void *userdata);
static inline int copy_bytes(int fdf, int fdt, uint64_t max_bytes, CopyFlags copy_flags) {
return copy_bytes_full(fdf, fdt, max_bytes, copy_flags, NULL, NULL, NULL, NULL);
}
int copy_times(int fdf, int fdt);
int copy_xattr(int fdf, int fdt);

2
src/basic/fd-util.c
View File

@ -648,7 +648,7 @@ int fd_duplicate_data_fd(int fd) {
if ((size_t) isz >= DATA_FD_MEMORY_LIMIT) {
r = copy_bytes_full(fd, pipefds[1], DATA_FD_MEMORY_LIMIT, 0, &remains, &remains_size);
r = copy_bytes_full(fd, pipefds[1], DATA_FD_MEMORY_LIMIT, 0, &remains, &remains_size, NULL, NULL);
if (r < 0 && r != -EAGAIN)
return r; /* If we get EAGAIN it could be because of the source or because of
* the destination fd, we can't know, as sendfile() and friends won't

34
src/basic/path-util.c
View File

@ -750,6 +750,9 @@ const char *last_path_component(const char *path) {
unsigned l, k;
if (!path)
return NULL;
l = k = strlen(path);
if (l == 0) /* special case — an empty string */
return path;
@ -766,6 +769,37 @@ const char *last_path_component(const char *path) {
return path + k;
}
int path_extract_filename(const char *p, char **ret) {
_cleanup_free_ char *a = NULL;
const char *c, *e = NULL, *q;
/* Extracts the filename part (i.e. right-most component) from a path, i.e. string that passes
* filename_is_valid(). A wrapper around last_path_component(), but eats up trailing slashes. */
if (!p)
return -EINVAL;
c = last_path_component(p);
for (q = c; *q != 0; q++)
if (*q != '/')
e = q + 1;
if (!e) /* no valid character? */
return -EINVAL;
a = strndup(c, e - c);
if (!a)
return -ENOMEM;
if (!filename_is_valid(a))
return -EINVAL;
*ret = TAKE_PTR(a);
return 0;
}
bool filename_is_valid(const char *p) {
const char *e;

1
src/basic/path-util.h
View File

@ -133,6 +133,7 @@ int parse_path_argument_and_warn(const char *path, bool suppress_root, char **ar
char* dirname_malloc(const char *path);
const char *last_path_component(const char *path);
int path_extract_filename(const char *p, char **ret);
bool filename_is_valid(const char *p) _pure_;
bool path_is_valid(const char *p) _pure_;

8
src/basic/rm-rf.h
View File

@ -22,3 +22,11 @@ static inline void rm_rf_physical_and_free(char *p) {
free(p);
}
DEFINE_TRIVIAL_CLEANUP_FUNC(char*, rm_rf_physical_and_free);
/* Similar as above, but also has magic btrfs subvolume powers */
static inline void rm_rf_subvolume_and_free(char *p) {
PROTECT_ERRNO;
(void) rm_rf(p, REMOVE_ROOT|REMOVE_PHYSICAL|REMOVE_SUBVOLUME);
free(p);
}
DEFINE_TRIVIAL_CLEANUP_FUNC(char*, rm_rf_subvolume_and_free);

23
src/basic/stat-util.c
View File

@ -296,3 +296,26 @@ int fd_verify_regular(int fd) {
return stat_verify_regular(&st);
}
int stat_verify_directory(const struct stat *st) {
assert(st);
if (S_ISLNK(st->st_mode))
return -ELOOP;
if (!S_ISDIR(st->st_mode))
return -ENOTDIR;
return 0;
}
int fd_verify_directory(int fd) {
struct stat st;
assert(fd >= 0);
if (fstat(fd, &st) < 0)
return -errno;
return stat_verify_directory(&st);
}

3
src/basic/stat-util.h
View File

@ -59,3 +59,6 @@ int path_is_temporary_fs(const char *path);
int stat_verify_regular(const struct stat *st);
int fd_verify_regular(int fd);
int stat_verify_directory(const struct stat *st);
int fd_verify_directory(int fd);

24
src/import/export-tar.c
View File

@ -141,6 +141,26 @@ static void tar_export_report_progress(TarExport *e) {
e->last_percent = percent;
}
static int tar_export_finish(TarExport *e) {
int r;
assert(e);
assert(e->tar_fd >= 0);
if (e->tar_pid > 0) {
r = wait_for_terminate_and_check("tar", e->tar_pid, WAIT_LOG);
e->tar_pid = 0;
if (r < 0)
return r;
if (r != EXIT_SUCCESS)
return -EPROTO;
}
e->tar_fd = safe_close(e->tar_fd);
return 0;
}
static int tar_export_process(TarExport *e) {
ssize_t l;
int r;
@ -156,7 +176,7 @@ static int tar_export_process(TarExport *e) {
e->tried_splice = true;
} else if (l == 0) {
r = 0;
r = tar_export_finish(e);
goto finish;
} else {
e->written_uncompressed += l;
@ -172,7 +192,7 @@ static int tar_export_process(TarExport *e) {
uint8_t input[COPY_BUFFER_SIZE];
if (e->eof) {
r = 0;
r = tar_export_finish(e);
goto finish;
}

4
src/import/export.c
View File

@ -95,7 +95,7 @@ static int export_tar(int argc, char *argv[], void *userdata) {
fd = STDOUT_FILENO;
(void) readlink_malloc("/proc/self/fd/1", &pretty);
(void) fd_get_path(fd, &pretty);
log_info("Exporting '%s', saving to '%s' with compression '%s'.", local, strna(pretty), import_compress_type_to_string(arg_compress));
}
@ -172,7 +172,7 @@ static int export_raw(int argc, char *argv[], void *userdata) {
fd = STDOUT_FILENO;
(void) readlink_malloc("/proc/self/fd/1", &pretty);
(void) fd_get_path(fd, &pretty);
log_info("Exporting '%s', saving to '%s' with compression '%s'.", local, strna(pretty), import_compress_type_to_string(arg_compress));
}

109
src/import/import-common.c
View File

@ -5,10 +5,15 @@
#include <sys/stat.h>
#include <unistd.h>
#include "alloc-util.h"
#include "btrfs-util.h"
#include "capability-util.h"
#include "dirent-util.h"
#include "fd-util.h"
#include "fileio.h"
#include "fs-util.h"
#include "import-common.h"
#include "os-util.h"
#include "process-util.h"
#include "signal-util.h"
#include "util.h"
@ -147,3 +152,107 @@ int import_fork_tar_c(const char *path, pid_t *ret) {
return TAKE_FD(pipefd[0]);
}
int import_mangle_os_tree(const char *path) {
_cleanup_closedir_ DIR *d = NULL, *cd = NULL;
_cleanup_free_ char *child = NULL, *t = NULL;
const char *joined;
struct dirent *de;
int r;
assert(path);
/* Some tarballs contain a single top-level directory that contains the actual OS directory tree. Try to
* recognize this, and move the tree one level up. */
r = path_is_os_tree(path);
if (r < 0)
return log_error_errno(r, "Failed to determine whether '%s' is an OS tree: %m", path);
if (r > 0) {
log_debug("Directory tree '%s' is a valid OS tree.", path);
return 0;
}
log_debug("Directory tree '%s' is not recognizable as OS tree, checking whether to rearrange it.", path);
d = opendir(path);
if (!d)
return log_error_errno(r, "Failed to open directory '%s': %m", path);
errno = 0;
de = readdir_no_dot(d);
if (!de) {
if (errno != 0)
return log_error_errno(errno, "Failed to iterate through directory '%s': %m", path);
log_debug("Directory '%s' is empty, leaving it as it is.", path);
return 0;
}
child = strdup(de->d_name);
if (!child)
return log_oom();
errno = 0;
de = readdir_no_dot(d);
if (de) {
if (errno != 0)
return log_error_errno(errno, "Failed to iterate through directory '%s': %m", path);
log_debug("Directory '%s' does not look like a directory tree, and has multiple children, leaving as it is.", path);
return 0;
}
joined = strjoina(path, "/", child);
r = path_is_os_tree(joined);
if (r == -ENOTDIR) {
log_debug("Directory '%s' does not look like a directory tree, and contains a single regular file only, leaving as it is.", path);
return 0;
}
if (r < 0)
return log_error_errno(r, "Failed to determine whether '%s' is an OS tree: %m", joined);
if (r == 0) {
log_debug("Neither '%s' nor '%s' is a valid OS tree, leaving them as they are.", path, joined);
return 0;
}
/* Nice, we have checked now:
*
* 1. The top-level directory does not qualify as OS tree
* 1. The top-level directory only contains one item
* 2. That item is a directory
* 3. And that directory qualifies as OS tree
*
* Let's now rearrange things, moving everything in the inner directory one level up */
cd = xopendirat(dirfd(d), child, O_NOFOLLOW);
if (!cd)
return log_error_errno(errno, "Can't open directory '%s': %m", joined);
log_info("Rearranging '%s', moving OS tree one directory up.", joined);
/* Let's rename the child to an unguessable name so that we can be sure all files contained in it can be
* safely moved up and won't collide with the name. */
r = tempfn_random(child, NULL, &t);
if (r < 0)
return log_oom();
r = rename_noreplace(dirfd(d), child, dirfd(d), t);
if (r < 0)
return log_error_errno(r, "Unable to rename '%s' to '%s/%s': %m", joined, path, t);
FOREACH_DIRENT_ALL(de, cd, return log_error_errno(errno, "Failed to iterate through directory '%s': %m", joined)) {
if (dot_or_dot_dot(de->d_name))
continue;
r = rename_noreplace(dirfd(cd), de->d_name, dirfd(d), de->d_name);
if (r < 0)
return log_error_errno(r, "Unable to move '%s/%s/%s' to '%s/%s': %m", path, t, de->d_name, path, de->d_name);
}
if (unlinkat(dirfd(d), t, AT_REMOVEDIR) < 0)
return log_error_errno(errno, "Failed to remove temporary directory '%s/%s': %m", path, t);
log_info("Successfully rearranged OS tree.");
return 0;
}

4
src/import/import-common.h
View File

@ -1,8 +1,12 @@
/* SPDX-License-Identifier: LGPL-2.1+ */
#pragma once
#include <sys/types.h>
int import_make_read_only_fd(int fd);
int import_make_read_only(const char *path);
int import_fork_tar_c(const char *path, pid_t *ret);
int import_fork_tar_x(const char *path, pid_t *ret);
int import_mangle_os_tree(const char *path);

327
src/import/import-fs.c Normal file
View File

@ -0,0 +1,327 @@
/* SPDX-License-Identifier: LGPL-2.1+ */
#include <getopt.h>
#include "alloc-util.h"
#include "btrfs-util.h"
#include "fd-util.h"
#include "fileio.h"
#include "fs-util.h"
#include "hostname-util.h"
#include "import-common.h"
#include "import-util.h"
#include "machine-image.h"
#include "mkdir.h"
#include "ratelimit.h"
#include "rm-rf.h"
#include "string-util.h"
#include "verbs.h"
#include "parse-util.h"
static bool arg_force = false;
static bool arg_read_only = false;
static const char *arg_image_root = "/var/lib/machines";
typedef struct ProgressInfo {
RateLimit limit;
char *path;
uint64_t size;
bool started;
bool logged_incomplete;
} ProgressInfo;
static volatile sig_atomic_t cancelled = false;
static void sigterm_sigint(int sig) {
cancelled = true;
}
static void progress_info_free(ProgressInfo *p) {
free(p->path);
}
static void progress_show(ProgressInfo *p) {
assert(p);
/* Show progress only every now and then. */
if (!ratelimit_below(&p->limit))
return;
/* Suppress the first message, start with the second one */
if (!p->started) {
p->started = true;
return;
}
/* Mention the list is incomplete before showing first output. */
if (!p->logged_incomplete) {
log_notice("(Note, file list shown below is incomplete, and is intended as sporadic progress report only.)");
p->logged_incomplete = true;
}
if (p->size == 0)
log_info("Copying tree, currently at '%s'...", p->path);
else {
char buffer[FORMAT_BYTES_MAX];
log_info("Copying tree, currently at '%s' (@%s)...", p->path, format_bytes(buffer, sizeof(buffer), p->size));
}
}
static int progress_path(const char *path, const struct stat *st, void *userdata) {
ProgressInfo *p = userdata;
int r;
assert(p);
if (cancelled)
return -EOWNERDEAD;
r = free_and_strdup(&p->path, path);
if (r < 0)
return r;
p->size = 0;
progress_show(p);
return 0;
}
static int progress_bytes(uint64_t nbytes, void *userdata) {
ProgressInfo *p = userdata;
assert(p);
assert(p->size != UINT64_MAX);
if (cancelled)
return -EOWNERDEAD;
p->size += nbytes;
progress_show(p);
return 0;
}
static int import_fs(int argc, char *argv[], void *userdata) {
_cleanup_(rm_rf_subvolume_and_freep) char *temp_path = NULL;
_cleanup_(progress_info_free) ProgressInfo progress = {};
const char *path = NULL, *local = NULL, *final_path;
_cleanup_close_ int open_fd = -1;
struct sigaction old_sigint_sa, old_sigterm_sa;
static const struct sigaction sa = {
.sa_handler = sigterm_sigint,
.sa_flags = SA_RESTART,
};
int r, fd;
if (argc >= 2)
path = argv[1];
if (isempty(path) || streq(path, "-"))
path = NULL;
if (argc >= 3)
local = argv[2];
else if (path)
local = basename(path);
if (isempty(local) || streq(local, "-"))
local = NULL;
if (local) {
if (!machine_name_is_valid(local)) {
log_error("Local image name '%s' is not valid.", local);
return -EINVAL;
}
if (!arg_force) {
r = image_find(IMAGE_MACHINE, local, NULL);
if (r < 0) {
if (r != -ENOENT)
return log_error_errno(r, "Failed to check whether image '%s' exists: %m", local);
} else {
log_error("Image '%s' already exists.", local);
return -EEXIST;
}
}
} else
local = "imported";
if (path) {
open_fd = open(path, O_DIRECTORY|O_RDONLY|O_CLOEXEC);
if (open_fd < 0)
return log_error_errno(errno, "Failed to open directory to import: %m");
fd = open_fd;
log_info("Importing '%s', saving as '%s'.", path, local);
} else {
_cleanup_free_ char *pretty = NULL;
fd = STDIN_FILENO;
(void) fd_get_path(fd, &pretty);
log_info("Importing '%s', saving as '%s'.", strempty(pretty), local);
}
final_path = strjoina(arg_image_root, "/", local);
r = tempfn_random(final_path, NULL, &temp_path);
if (r < 0)
return log_oom();
(void) mkdir_parents_label(temp_path, 0700);
RATELIMIT_INIT(progress.limit, 200*USEC_PER_MSEC, 1);
/* Hook into SIGINT/SIGTERM, so that we can cancel things then */
assert(sigaction(SIGINT, &sa, &old_sigint_sa) >= 0);
assert(sigaction(SIGTERM, &sa, &old_sigterm_sa) >= 0);
r = btrfs_subvol_snapshot_fd_full(
fd,
temp_path,
BTRFS_SNAPSHOT_FALLBACK_COPY|BTRFS_SNAPSHOT_RECURSIVE|BTRFS_SNAPSHOT_FALLBACK_DIRECTORY|BTRFS_SNAPSHOT_QUOTA,
progress_path,
progress_bytes,
&progress);
if (r == -EOWNERDEAD) { /* SIGINT + SIGTERM cause this, see signal handler above */
log_error("Copy cancelled.");
goto finish;
}
if (r < 0) {
log_error_errno(r, "Failed to copy directory: %m");
goto finish;
}
r = import_mangle_os_tree(temp_path);
if (r < 0)
goto finish;
(void) import_assign_pool_quota_and_warn(temp_path);
if (arg_read_only) {
r = import_make_read_only(temp_path);
if (r < 0) {
log_error_errno(r, "Failed to make directory read-only: %m");
goto finish;
}
}
if (arg_force)
(void) rm_rf(final_path, REMOVE_ROOT|REMOVE_PHYSICAL|REMOVE_SUBVOLUME);
r = rename_noreplace(AT_FDCWD, temp_path, AT_FDCWD, final_path);
if (r < 0) {
log_error_errno(r, "Failed to move image into place: %m");
goto finish;
}
temp_path = mfree(temp_path);
log_info("Exiting.");
finish:
/* Put old signal handlers into place */
assert(sigaction(SIGINT, &old_sigint_sa, NULL) >= 0);
assert(sigaction(SIGTERM, &old_sigterm_sa, NULL) >= 0);
return 0;
}
static int help(int argc, char *argv[], void *userdata) {
printf("%s [OPTIONS...] {COMMAND} ...\n\n"
"Import container images from a file system.\n\n"
" -h --help Show this help\n"
" --version Show package version\n"
" --force Force creation of image\n"
" --image-root=PATH Image root directory\n"
" --read-only Create a read-only image\n\n"
"Commands:\n"
" run DIRECTORY [NAME] Import a directory\n",
program_invocation_short_name);
return 0;
}
static int parse_argv(int argc, char *argv[]) {
enum {
ARG_VERSION = 0x100,
ARG_FORCE,
ARG_IMAGE_ROOT,
ARG_READ_ONLY,
};
static const struct option options[] = {
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, ARG_VERSION },
{ "force", no_argument, NULL, ARG_FORCE },
{ "image-root", required_argument, NULL, ARG_IMAGE_ROOT },
{ "read-only", no_argument, NULL, ARG_READ_ONLY },
{}
};
int c;
assert(argc >= 0);
assert(argv);
while ((c = getopt_long(argc, argv, "h", options, NULL)) >= 0)
switch (c) {
case 'h':
return help(0, NULL, NULL);
case ARG_VERSION:
return version();
case ARG_FORCE:
arg_force = true;
break;
case ARG_IMAGE_ROOT:
arg_image_root = optarg;
break;
case ARG_READ_ONLY:
arg_read_only = true;
break;
case '?':
return -EINVAL;
default:
assert_not_reached("Unhandled option");
}
return 1;
}
static int import_fs_main(int argc, char *argv[]) {
static const Verb verbs[] = {
{ "help", VERB_ANY, VERB_ANY, 0, help },
{ "run", 2, 3, 0, import_fs },
{}
};
return dispatch_verb(argc, argv, verbs, NULL);
}
int main(int argc, char *argv[]) {
int r;
setlocale(LC_ALL, "");
log_parse_environment();
log_open();
r = parse_argv(argc, argv);
if (r <= 0)
goto finish;
r = import_fs_main(argc, argv);
finish:
return r < 0 ? EXIT_FAILURE : EXIT_SUCCESS;
}

13
src/import/import-raw.c
View File

@ -37,7 +37,6 @@ struct RawImport {
char *local;
bool force_local;
bool read_only;
bool grow_machine_directory;
char *temp_path;
char *final_path;
@ -47,8 +46,6 @@ struct RawImport {
ImportCompress compress;
uint64_t written_since_last_grow;
sd_event_source *input_event_source;
uint8_t buffer[16*1024];
@ -95,7 +92,6 @@ int raw_import_new(
_cleanup_(raw_import_unrefp) RawImport *i = NULL;
_cleanup_free_ char *root = NULL;
bool grow;
int r;
assert(ret);
@ -104,8 +100,6 @@ int raw_import_new(
if (!root)
return -ENOMEM;
grow = path_startswith(root, "/var/lib/machines");
i = new(RawImport, 1);
if (!i)
return -ENOMEM;
@ -117,7 +111,6 @@ int raw_import_new(
.userdata = userdata,
.last_percent = (unsigned) -1,
.image_root = TAKE_PTR(root),
.grow_machine_directory = grow,
};
RATELIMIT_INIT(i->progress_rate_limit, 100 * USEC_PER_MSEC, 1);
@ -307,11 +300,6 @@ static int raw_import_write(const void *p, size_t sz, void *userdata) {
RawImport *i = userdata;
ssize_t n;
if (i->grow_machine_directory && i->written_since_last_grow >= GROW_INTERVAL_BYTES) {
i->written_since_last_grow = 0;
grow_machine_directory();
}
n = sparse_write(i->output_fd, p, sz, 64);
if (n < 0)
return (int) n;
@ -319,7 +307,6 @@ static int raw_import_write(const void *p, size_t sz, void *userdata) {
return -EIO;
i->written_uncompressed += sz;
i->written_since_last_grow += sz;
return 0;
}

19
src/import/import-tar.c
View File

@ -37,7 +37,6 @@ struct TarImport {
char *local;
bool force_local;
bool read_only;
bool grow_machine_directory;
char *temp_path;
char *final_path;
@ -47,8 +46,6 @@ struct TarImport {
ImportCompress compress;
uint64_t written_since_last_grow;
sd_event_source *input_event_source;
uint8_t buffer[16*1024];
@ -102,7 +99,6 @@ int tar_import_new(
_cleanup_(tar_import_unrefp) TarImport *i = NULL;
_cleanup_free_ char *root = NULL;
bool grow;
int r;
assert(ret);
@ -111,8 +107,6 @@ int tar_import_new(
if (!root)
return -ENOMEM;
grow = path_startswith(root, "/var/lib/machines");
i = new(TarImport, 1);
if (!i)
return -ENOMEM;
@ -124,7 +118,6 @@ int tar_import_new(
.userdata = userdata,
.last_percent = (unsigned) -1,
.image_root = TAKE_PTR(root),
.grow_machine_directory = grow,
};
RATELIMIT_INIT(i->progress_rate_limit, 100 * USEC_PER_MSEC, 1);
@ -182,8 +175,14 @@ static int tar_import_finish(TarImport *i) {
i->tar_pid = 0;
if (r < 0)
return r;
if (r != EXIT_SUCCESS)
return -EPROTO;
}
r = import_mangle_os_tree(i->temp_path);
if (r < 0)
return r;
if (i->read_only) {
r = import_make_read_only(i->temp_path);
if (r < 0)
@ -241,17 +240,11 @@ static int tar_import_write(const void *p, size_t sz, void *userdata) {
TarImport *i = userdata;
int r;
if (i->grow_machine_directory && i->written_since_last_grow >= GROW_INTERVAL_BYTES) {
i->written_since_last_grow = 0;
grow_machine_directory();
}
r = loop_write(i->tar_fd, p, sz, false);
if (r < 0)
return r;
i->written_uncompressed += sz;
i->written_since_last_grow += sz;
return 0;
}

4
src/import/import.c
View File

@ -96,7 +96,7 @@ static int import_tar(int argc, char *argv[], void *userdata) {
fd = STDIN_FILENO;
(void) readlink_malloc("/proc/self/fd/0", &pretty);
(void) fd_get_path(fd, &pretty);
log_info("Importing '%s', saving as '%s'.", strna(pretty), local);
}
@ -192,7 +192,7 @@ static int import_raw(int argc, char *argv[], void *userdata) {
fd = STDIN_FILENO;
(void) readlink_malloc("/proc/self/fd/0", &pretty);
(void) fd_get_path(fd, &pretty);
log_info("Importing '%s', saving as '%s'.", strempty(pretty), local);
}

183
src/import/importd.c
View File

@ -10,6 +10,7 @@
#include "bus-util.h"
#include "def.h"
#include "fd-util.h"
#include "float.h"
#include "hostname-util.h"
#include "import-util.h"
#include "machine-pool.h"
@ -21,6 +22,7 @@
#include "process-util.h"
#include "signal-util.h"
#include "socket-util.h"
#include "stat-util.h"
#include "string-table.h"
#include "strv.h"
#include "syslog-util.h"
@ -34,6 +36,7 @@ typedef struct Manager Manager;
typedef enum TransferType {
TRANSFER_IMPORT_TAR,
TRANSFER_IMPORT_RAW,
TRANSFER_IMPORT_FS,
TRANSFER_EXPORT_TAR,
TRANSFER_EXPORT_RAW,
TRANSFER_PULL_TAR,
@ -94,6 +97,7 @@ struct Manager {
static const char* const transfer_type_table[_TRANSFER_TYPE_MAX] = {
[TRANSFER_IMPORT_TAR] = "import-tar",
[TRANSFER_IMPORT_RAW] = "import-raw",
[TRANSFER_IMPORT_FS] = "import-fs",
[TRANSFER_EXPORT_TAR] = "export-tar",
[TRANSFER_EXPORT_RAW] = "export-raw",
[TRANSFER_PULL_TAR] = "pull-tar",
@ -156,6 +160,7 @@ static int transfer_new(Manager *m, Transfer **ret) {
.stdin_fd = -1,
.stdout_fd = -1,
.verify = _IMPORT_VERIFY_INVALID,
.progress_percent= (unsigned) -1,
};
id = m->current_transfer_id + 1;
@ -177,6 +182,15 @@ static int transfer_new(Manager *m, Transfer **ret) {
return 0;
}
static double transfer_percent_as_double(Transfer *t) {
assert(t);
if (t->progress_percent == (unsigned) -1)
return -DBL_MAX;
return (double) t->progress_percent / 100.0;
}
static void transfer_send_log_line(Transfer *t, const char *line) {
int r, priority = LOG_INFO;
@ -196,7 +210,7 @@ static void transfer_send_log_line(Transfer *t, const char *line) {
priority,
line);
if (r < 0)
log_error_errno(r, "Cannot emit message: %m");
log_warning_errno(r, "Cannot emit log message signal, ignoring: %m");
}
static void transfer_send_logs(Transfer *t, bool flush) {
@ -301,17 +315,16 @@ static int transfer_on_pid(sd_event_source *s, const siginfo_t *si, void *userda
if (si->si_code == CLD_EXITED) {
if (si->si_status != 0)
log_error("Import process failed with exit code %i.", si->si_status);
log_error("Transfer process failed with exit code %i.", si->si_status);
else {
log_debug("Import process succeeded.");
log_debug("Transfer process succeeded.");
success = true;
}
} else if (IN_SET(si->si_code, CLD_KILLED, CLD_DUMPED))
log_error("Import process terminated by signal %s.", signal_to_string(si->si_status));
log_error("Transfer process terminated by signal %s.", signal_to_string(si->si_status));
else
log_error("Import process failed due to unknown reason.");
log_error("Transfer process failed due to unknown reason.");
t->pid = 0;
@ -326,14 +339,12 @@ static int transfer_on_log(sd_event_source *s, int fd, uint32_t revents, void *u
assert(t);
l = read(fd, t->log_message + t->log_message_size, sizeof(t->log_message) - t->log_message_size);
if (l < 0)
log_error_errno(errno, "Failed to read log message: %m");
if (l <= 0) {
/* EOF/read error. We just close the pipe here, and
* close the watch, waiting for the SIGCHLD to arrive,
* before we do anything else. */
if (l < 0)
log_error_errno(errno, "Failed to read log message: %m");
t->log_event_source = sd_event_source_unref(t->log_event_source);
return 0;
}
@ -360,7 +371,7 @@ static int transfer_start(Transfer *t) {
return r;
if (r == 0) {
const char *cmd[] = {
NULL, /* systemd-import, systemd-export or systemd-pull */
NULL, /* systemd-import, systemd-import-fs, systemd-export or systemd-pull */
NULL, /* tar, raw */
NULL, /* --verify= */
NULL, /* verify argument */
@ -393,17 +404,52 @@ static int transfer_start(Transfer *t) {
_exit(EXIT_FAILURE);
}
if (IN_SET(t->type, TRANSFER_IMPORT_TAR, TRANSFER_IMPORT_RAW))
cmd[k++] = SYSTEMD_IMPORT_PATH;
else if (IN_SET(t->type, TRANSFER_EXPORT_TAR, TRANSFER_EXPORT_RAW))
cmd[k++] = SYSTEMD_EXPORT_PATH;
else
cmd[k++] = SYSTEMD_PULL_PATH;
switch (t->type) {
if (IN_SET(t->type, TRANSFER_IMPORT_TAR, TRANSFER_EXPORT_TAR, TRANSFER_PULL_TAR))
case TRANSFER_IMPORT_TAR:
case TRANSFER_IMPORT_RAW:
cmd[k++] = SYSTEMD_IMPORT_PATH;
break;
case TRANSFER_IMPORT_FS:
cmd[k++] = SYSTEMD_IMPORT_FS_PATH;
break;
case TRANSFER_EXPORT_TAR:
case TRANSFER_EXPORT_RAW:
cmd[k++] = SYSTEMD_EXPORT_PATH;
break;
case TRANSFER_PULL_TAR:
case TRANSFER_PULL_RAW:
cmd[k++] = SYSTEMD_PULL_PATH;
break;
default:
assert_not_reached("Unexpected transfer type");
}
switch (t->type) {
case TRANSFER_IMPORT_TAR:
case TRANSFER_EXPORT_TAR:
case TRANSFER_PULL_TAR:
cmd[k++] = "tar";
else
break;
case TRANSFER_IMPORT_RAW:
case TRANSFER_EXPORT_RAW:
case TRANSFER_PULL_RAW:
cmd[k++] = "raw";
break;
case TRANSFER_IMPORT_FS:
cmd[k++] = "run";
break;
default:
break;
}
if (t->verify != _IMPORT_VERIFY_INVALID) {
cmd[k++] = "--verify";
@ -513,7 +559,6 @@ static int manager_on_notify(sd_event_source *s, int fd, uint32_t revents, void
struct ucred *ucred = NULL;
Manager *m = userdata;
struct cmsghdr *cmsg;
unsigned percent;
char *p, *e;
Transfer *t;
Iterator i;
@ -569,15 +614,15 @@ static int manager_on_notify(sd_event_source *s, int fd, uint32_t revents, void
e = strchrnul(p, '\n');
*e = 0;
r = safe_atou(p, &percent);
if (r < 0 || percent > 100) {
r = parse_percent(p);
if (r < 0) {
log_warning("Got invalid percent value, ignoring.");
return 0;
}
t->progress_percent = percent;
t->progress_percent = (unsigned) r;
log_debug("Got percentage from client: %u%%", percent);
log_debug("Got percentage from client: %u%%", t->progress_percent);
return 0;
}
@ -636,12 +681,9 @@ static Transfer *manager_find(Manager *m, TransferType type, const char *remote)
assert(type >= 0);
assert(type < _TRANSFER_TYPE_MAX);
HASHMAP_FOREACH(t, m->transfers, i) {
if (t->type == type &&
streq_ptr(t->remote, remote))
HASHMAP_FOREACH(t, m->transfers, i)
if (t->type == type && streq_ptr(t->remote, remote))
return t;
}
return NULL;
}
@ -675,10 +717,14 @@ static int method_import_tar_or_raw(sd_bus_message *msg, void *userdata, sd_bus_
if (r < 0)
return r;
r = fd_verify_regular(fd);
if (r < 0)
return r;
if (!machine_name_is_valid(local))
return sd_bus_error_setf(error, SD_BUS_ERROR_INVALID_ARGS, "Local name %s is invalid", local);
r = setup_machine_directory((uint64_t) -1, error);
r = setup_machine_directory(error);
if (r < 0)
return r;
@ -711,6 +757,72 @@ static int method_import_tar_or_raw(sd_bus_message *msg, void *userdata, sd_bus_
return sd_bus_reply_method_return(msg, "uo", id, object);
}
static int method_import_fs(sd_bus_message *msg, void *userdata, sd_bus_error *error) {
_cleanup_(transfer_unrefp) Transfer *t = NULL;
int fd, force, read_only, r;
const char *local, *object;
Manager *m = userdata;
uint32_t id;
assert(msg);
assert(m);
r = bus_verify_polkit_async(
msg,
CAP_SYS_ADMIN,
"org.freedesktop.import1.import",
NULL,
false,
UID_INVALID,
&m->polkit_registry,
error);
if (r < 0)
return r;
if (r == 0)
return 1; /* Will call us back */
r = sd_bus_message_read(msg, "hsbb", &fd, &local, &force, &read_only);
if (r < 0)
return r;
r = fd_verify_directory(fd);
if (r < 0)
return r;
if (!machine_name_is_valid(local))
return sd_bus_error_setf(error, SD_BUS_ERROR_INVALID_ARGS, "Local name %s is invalid", local);
r = setup_machine_directory(error);
if (r < 0)
return r;
r = transfer_new(m, &t);
if (r < 0)
return r;
t->type = TRANSFER_IMPORT_FS;
t->force_local = force;
t->read_only = read_only;
t->local = strdup(local);
if (!t->local)
return -ENOMEM;
t->stdin_fd = fcntl(fd, F_DUPFD_CLOEXEC, 3);
if (t->stdin_fd < 0)
return -errno;
r = transfer_start(t);
if (r < 0)
return r;
object = t->object_path;
id = t->id;
t = NULL;
return sd_bus_reply_method_return(msg, "uo", id, object);
}
static int method_export_tar_or_raw(sd_bus_message *msg, void *userdata, sd_bus_error *error) {
_cleanup_(transfer_unrefp) Transfer *t = NULL;
int fd, r;
@ -743,6 +855,10 @@ static int method_export_tar_or_raw(sd_bus_message *msg, void *userdata, sd_bus_
if (!machine_name_is_valid(local))
return sd_bus_error_setf(error, SD_BUS_ERROR_INVALID_ARGS, "Local name %s is invalid", local);
r = fd_verify_regular(fd);
if (r < 0)
return r;
type = streq_ptr(sd_bus_message_get_member(msg), "ExportTar") ? TRANSFER_EXPORT_TAR : TRANSFER_EXPORT_RAW;
r = transfer_new(m, &t);
@ -821,7 +937,7 @@ static int method_pull_tar_or_raw(sd_bus_message *msg, void *userdata, sd_bus_er
if (v < 0)
return sd_bus_error_setf(error, SD_BUS_ERROR_INVALID_ARGS, "Unknown verification mode %s", verify);
r = setup_machine_directory((uint64_t) -1, error);
r = setup_machine_directory(error);
if (r < 0)
return r;
@ -886,7 +1002,7 @@ static int method_list_transfers(sd_bus_message *msg, void *userdata, sd_bus_err
transfer_type_to_string(t->type),
t->remote,
t->local,
(double) t->progress_percent / 100.0,
transfer_percent_as_double(t),
t->object_path);
if (r < 0)
return r;
@ -982,7 +1098,7 @@ static int property_get_progress(
assert(reply);
assert(t);
return sd_bus_message_append(reply, "d", (double) t->progress_percent / 100.0);
return sd_bus_message_append(reply, "d", transfer_percent_as_double(t));
}
static BUS_DEFINE_PROPERTY_GET_ENUM(property_get_type, transfer_type, TransferType);
@ -1005,6 +1121,7 @@ static const sd_bus_vtable manager_vtable[] = {
SD_BUS_VTABLE_START(0),
SD_BUS_METHOD("ImportTar", "hsbb", "uo", method_import_tar_or_raw, SD_BUS_VTABLE_UNPRIVILEGED),
SD_BUS_METHOD("ImportRaw", "hsbb", "uo", method_import_tar_or_raw, SD_BUS_VTABLE_UNPRIVILEGED),
SD_BUS_METHOD("ImportFileSystem", "hsbb", "uo", method_import_fs, SD_BUS_VTABLE_UNPRIVILEGED),
SD_BUS_METHOD("ExportTar", "shs", "uo", method_export_tar_or_raw, SD_BUS_VTABLE_UNPRIVILEGED),
SD_BUS_METHOD("ExportRaw", "shs", "uo", method_export_tar_or_raw, SD_BUS_VTABLE_UNPRIVILEGED),
SD_BUS_METHOD("PullTar", "sssb", "uo", method_pull_tar_or_raw, SD_BUS_VTABLE_UNPRIVILEGED),

6
src/import/meson.build
View File

@ -38,6 +38,12 @@ systemd_import_sources = files('''
qcow2-util.h
'''.split())
systemd_import_fs_sources = files('''
import-fs.c
import-common.c
import-common.h
'''.split())
systemd_export_sources = files('''
export.c
export-tar.c

20
src/import/org.freedesktop.import1.conf
View File

@ -46,6 +46,26 @@
send_interface="org.freedesktop.import1.Manager"
send_member="CancelTransfer"/>
<allow send_destination="org.freedesktop.import1"
send_interface="org.freedesktop.import1.Manager"
send_member="ImportTar"/>
<allow send_destination="org.freedesktop.import1"
send_interface="org.freedesktop.import1.Manager"
send_member="ImportRaw"/>
<allow send_destination="org.freedesktop.import1"
send_interface="org.freedesktop.import1.Manager"
send_member="ImportFileSystem"/>
<allow send_destination="org.freedesktop.import1"
send_interface="org.freedesktop.import1.Manager"
send_member="ExportTar"/>
<allow send_destination="org.freedesktop.import1"
send_interface="org.freedesktop.import1.Manager"
send_member="ExportRaw"/>
<allow send_destination="org.freedesktop.import1"
send_interface="org.freedesktop.import1.Manager"
send_member="PullTar"/>

10
src/import/pull-job.c
View File

@ -74,7 +74,6 @@ static int pull_job_restart(PullJob *j) {
j->payload_allocated = 0;
j->written_compressed = 0;
j->written_uncompressed = 0;
j->written_since_last_grow = 0;
r = pull_job_begin(j);
if (r < 0)
@ -224,11 +223,6 @@ static int pull_job_write_uncompressed(const void *p, size_t sz, void *userdata)
if (j->disk_fd >= 0) {
if (j->grow_machine_directory && j->written_since_last_grow >= GROW_INTERVAL_BYTES) {
j->written_since_last_grow = 0;
grow_machine_directory();
}
if (j->allow_sparse)
n = sparse_write(j->disk_fd, p, sz, 64);
else {
@ -250,7 +244,6 @@ static int pull_job_write_uncompressed(const void *p, size_t sz, void *userdata)
}
j->written_uncompressed += sz;
j->written_since_last_grow += sz;
return 0;
}
@ -577,9 +570,6 @@ int pull_job_begin(PullJob *j) {
if (j->state != PULL_JOB_INIT)
return -EBUSY;
if (j->grow_machine_directory)
grow_machine_directory();
r = curl_glue_make(&j->curl, j->url, j);
if (r < 0)
return r;

3
src/import/pull-job.h
View File

@ -80,9 +80,6 @@ struct PullJob {
char *checksum;
bool grow_machine_directory;
uint64_t written_since_last_grow;
VerificationStyle style;
};

6
src/import/pull-raw.c
View File

@ -56,7 +56,6 @@ struct RawPull {
char *local;
bool force_local;
bool grow_machine_directory;
bool settings;
bool roothash;
@ -119,7 +118,6 @@ int raw_pull_new(
_cleanup_(sd_event_unrefp) sd_event *e = NULL;
_cleanup_(raw_pull_unrefp) RawPull *i = NULL;
_cleanup_free_ char *root = NULL;
bool grow;
int r;
assert(ret);
@ -128,8 +126,6 @@ int raw_pull_new(
if (!root)
return -ENOMEM;
grow = path_startswith(root, "/var/lib/machines");
if (event)
e = sd_event_ref(event);
else {
@ -150,7 +146,6 @@ int raw_pull_new(
.on_finished = on_finished,
.userdata = userdata,
.image_root = TAKE_PTR(root),
.grow_machine_directory = grow,
.event = TAKE_PTR(e),
.glue = TAKE_PTR(g),
};
@ -689,7 +684,6 @@ int raw_pull_start(
i->raw_job->on_open_disk = raw_pull_job_on_open_disk_raw;
i->raw_job->on_progress = raw_pull_job_on_progress;
i->raw_job->calc_checksum = verify != IMPORT_VERIFY_NO;
i->raw_job->grow_machine_directory = i->grow_machine_directory;
r = pull_find_old_etags(url, i->image_root, DT_REG, ".raw-", ".raw", &i->raw_job->old_etags);
if (r < 0)

6
src/import/pull-tar.c
View File

@ -52,7 +52,6 @@ struct TarPull {
char *local;
bool force_local;
bool grow_machine_directory;
bool settings;
pid_t tar_pid;
@ -112,7 +111,6 @@ int tar_pull_new(
_cleanup_(sd_event_unrefp) sd_event *e = NULL;
_cleanup_(tar_pull_unrefp) TarPull *i = NULL;
_cleanup_free_ char *root = NULL;
bool grow;
int r;
assert(ret);
@ -121,8 +119,6 @@ int tar_pull_new(
if (!root)
return -ENOMEM;
grow = path_startswith(root, "/var/lib/machines");
if (event)
e = sd_event_ref(event);
else {
@ -143,7 +139,6 @@ int tar_pull_new(
.on_finished = on_finished,
.userdata = userdata,
.image_root = TAKE_PTR(root),
.grow_machine_directory = grow,
.event = TAKE_PTR(e),
.glue = TAKE_PTR(g),
};
@ -512,7 +507,6 @@ int tar_pull_start(
i->tar_job->on_open_disk = tar_pull_job_on_open_disk_tar;
i->tar_job->on_progress = tar_pull_job_on_progress;
i->tar_job->calc_checksum = verify != IMPORT_VERIFY_NO;
i->tar_job->grow_machine_directory = i->grow_machine_directory;
r = pull_find_old_etags(url, i->image_root, DT_DIR, ".tar-", NULL, &i->tar_job->old_etags);
if (r < 0)

135
src/machine/machinectl.c
View File

@ -5,6 +5,7 @@
#include <fcntl.h>
#include <getopt.h>
#include <locale.h>
#include <math.h>
#include <net/if.h>
#include <netinet/in.h>
#include <string.h>
@ -1999,28 +2000,38 @@ static int transfer_image_common(sd_bus *bus, sd_bus_message *m) {
return -r;
}
static const char *nullify_dash(const char *p) {
if (isempty(p))
return NULL;
if (streq(p, "-"))
return NULL;
return p;
}
static int import_tar(int argc, char *argv[], void *userdata) {
_cleanup_(sd_bus_message_unrefp) sd_bus_message *m = NULL;
_cleanup_free_ char *ll = NULL;
_cleanup_close_ int fd = -1;
_cleanup_free_ char *ll = NULL, *fn = NULL;
const char *local = NULL, *path = NULL;
_cleanup_close_ int fd = -1;
sd_bus *bus = userdata;
int r;
assert(bus);
if (argc >= 2)
path = argv[1];
if (isempty(path) || streq(path, "-"))
path = NULL;
path = nullify_dash(argv[1]);
if (argc >= 3)
local = argv[2];
else if (path)
local = basename(path);
if (isempty(local) || streq(local, "-"))
local = NULL;
local = nullify_dash(argv[2]);
else if (path) {
r = path_extract_filename(path, &fn);
if (r < 0)
return log_error_errno(r, "Cannot extract container name from filename: %m");
local = fn;
}
if (!local) {
log_error("Need either path or local name.");
return -EINVAL;
@ -2068,26 +2079,26 @@ static int import_tar(int argc, char *argv[], void *userdata) {
static int import_raw(int argc, char *argv[], void *userdata) {
_cleanup_(sd_bus_message_unrefp) sd_bus_message *m = NULL;
_cleanup_free_ char *ll = NULL;
_cleanup_close_ int fd = -1;
_cleanup_free_ char *ll = NULL, *fn = NULL;
const char *local = NULL, *path = NULL;
_cleanup_close_ int fd = -1;
sd_bus *bus = userdata;
int r;
assert(bus);
if (argc >= 2)
path = argv[1];
if (isempty(path) || streq(path, "-"))
path = NULL;
path = nullify_dash(argv[1]);
if (argc >= 3)
local = argv[2];
else if (path)
local = basename(path);
if (isempty(local) || streq(local, "-"))
local = NULL;
local = nullify_dash(argv[2]);
else if (path) {
r = path_extract_filename(path, &fn);
if (r < 0)
return log_error_errno(r, "Cannot extract container name from filename: %m");
local = fn;
}
if (!local) {
log_error("Need either path or local name.");
return -EINVAL;
@ -2133,6 +2144,67 @@ static int import_raw(int argc, char *argv[], void *userdata) {
return transfer_image_common(bus, m);
}
static int import_fs(int argc, char *argv[], void *userdata) {
_cleanup_(sd_bus_message_unrefp) sd_bus_message *m = NULL;
const char *local = NULL, *path = NULL;
_cleanup_free_ char *fn = NULL;
_cleanup_close_ int fd = -1;
sd_bus *bus = userdata;
int r;
assert(bus);
if (argc >= 2)
path = nullify_dash(argv[1]);
if (argc >= 3)
local = nullify_dash(argv[2]);
else if (path) {
r = path_extract_filename(path, &fn);
if (r < 0)
return log_error_errno(r, "Cannot extract container name from filename: %m");
local = fn;
}
if (!local) {
log_error("Need either path or local name.");
return -EINVAL;
}
if (!machine_name_is_valid(local)) {
log_error("Local name %s is not a suitable machine name.", local);
return -EINVAL;
}
if (path) {
fd = open(path, O_DIRECTORY|O_RDONLY|O_CLOEXEC);
if (fd < 0)
return log_error_errno(errno, "Failed to open directory '%s': %m", path);
}
r = sd_bus_message_new_method_call(
bus,
&m,
"org.freedesktop.import1",
"/org/freedesktop/import1",
"org.freedesktop.import1.Manager",
"ImportFileSystem");
if (r < 0)
return bus_log_create_error(r);
r = sd_bus_message_append(
m,
"hsbb",
fd >= 0 ? fd : STDIN_FILENO,
local,
arg_force,
arg_read_only);
if (r < 0)
return bus_log_create_error(r);
return transfer_image_common(bus, m);
}
static void determine_compression_from_filename(const char *p) {
if (arg_format)
return;
@ -2464,12 +2536,21 @@ static int list_transfers(int argc, char *argv[], void *userdata) {
(int) max_remote, "REMOTE");
for (j = 0; j < n_transfers; j++)
printf("%*" PRIu32 " %*u%% %-*s %-*s %-*s\n",
(int) MAX(2U, DECIMAL_STR_WIDTH(max_id)), transfers[j].id,
(int) 6, (unsigned) (transfers[j].progress * 100),
(int) max_type, transfers[j].type,
(int) max_local, transfers[j].local,
(int) max_remote, transfers[j].remote);
if (transfers[j].progress < 0)
printf("%*" PRIu32 " %*s %-*s %-*s %-*s\n",
(int) MAX(2U, DECIMAL_STR_WIDTH(max_id)), transfers[j].id,
(int) 7, "n/a",
(int) max_type, transfers[j].type,
(int) max_local, transfers[j].local,
(int) max_remote, transfers[j].remote);
else
printf("%*" PRIu32 " %*u%% %-*s %-*s %-*s\n",
(int) MAX(2U, DECIMAL_STR_WIDTH(max_id)), transfers[j].id,
(int) 6, (unsigned) (transfers[j].progress * 100),
(int) max_type, transfers[j].type,
(int) max_local, transfers[j].local,
(int) max_remote, transfers[j].remote);
if (arg_legend) {
if (n_transfers > 0)
@ -2687,6 +2768,7 @@ static int help(int argc, char *argv[], void *userdata) {
" pull-raw URL [NAME] Download a RAW container or VM image\n"
" import-tar FILE [NAME] Import a local TAR container image\n"
" import-raw FILE [NAME] Import a local RAW container or VM image\n"
" import-fs DIRECTORY [NAME] Import a local directory container image\n"
" export-tar NAME [FILE] Export a TAR container image locally\n"
" export-raw NAME [FILE] Export a RAW container or VM image locally\n"
" list-transfers Show list of downloads in progress\n"
@ -3008,6 +3090,7 @@ static int machinectl_main(int argc, char *argv[], sd_bus *bus) {
{ "disable", 2, VERB_ANY, 0, enable_machine },
{ "import-tar", 2, 3, 0, import_tar },
{ "import-raw", 2, 3, 0, import_raw },
{ "import-fs", 2, 3, 0, import_fs },
{ "export-tar", 2, 3, 0, export_tar },
{ "export-raw", 2, 3, 0, export_raw },
{ "pull-tar", 2, 3, 0, pull_tar },

31
src/machine/machined-dbus.c
View File

@ -41,15 +41,10 @@ static int property_get_pool_usage(
_cleanup_close_ int fd = -1;
uint64_t usage = (uint64_t) -1;
struct stat st;
assert(bus);
assert(reply);
/* We try to read the quota info from /var/lib/machines, as
* well as the usage of the loopback file
* /var/lib/machines.raw, and pick the larger value. */
fd = open("/var/lib/machines", O_RDONLY|O_CLOEXEC|O_DIRECTORY);
if (fd >= 0) {
BtrfsQuotaInfo q;
@ -58,11 +53,6 @@ static int property_get_pool_usage(
usage = q.referenced;
}
if (stat("/var/lib/machines.raw", &st) >= 0) {
if (usage == (uint64_t) -1 || st.st_blocks * 512ULL > usage)
usage = st.st_blocks * 512ULL;
}
return sd_bus_message_append(reply, "t", usage);
}
@ -77,15 +67,10 @@ static int property_get_pool_limit(
_cleanup_close_ int fd = -1;
uint64_t size = (uint64_t) -1;
struct stat st;
assert(bus);
assert(reply);
/* We try to read the quota limit from /var/lib/machines, as
* well as the size of the loopback file
* /var/lib/machines.raw, and pick the smaller value. */
fd = open("/var/lib/machines", O_RDONLY|O_CLOEXEC|O_DIRECTORY);
if (fd >= 0) {
BtrfsQuotaInfo q;
@ -94,11 +79,6 @@ static int property_get_pool_limit(
size = q.referenced_max;
}
if (stat("/var/lib/machines.raw", &st) >= 0) {
if (size == (uint64_t) -1 || (uint64_t) st.st_size < size)
size = st.st_size;
}
return sd_bus_message_append(reply, "t", size);
}
@ -877,19 +857,10 @@ static int method_set_pool_limit(sd_bus_message *message, void *userdata, sd_bus
return 1; /* Will call us back */
/* Set up the machine directory if necessary */
r = setup_machine_directory(limit, error);
r = setup_machine_directory(error);
if (r < 0)
return r;
/* Resize the backing loopback device, if there is one, except if we asked to drop any limit */
if (limit != (uint64_t) -1) {
r = btrfs_resize_loopback("/var/lib/machines", limit, false);
if (r == -ENOTTY)
return sd_bus_error_setf(error, SD_BUS_ERROR_NOT_SUPPORTED, "Quota is only supported on btrfs.");
if (r < 0 && r != -ENODEV) /* ignore ENODEV, as that's what is returned if the file system is not on loopback */
return sd_bus_error_set_errnof(error, r, "Failed to adjust loopback limit: %m");
}
(void) btrfs_qgroup_set_limit("/var/lib/machines", 0, limit);
r = btrfs_subvol_set_subtree_quota_limit("/var/lib/machines", 0, limit);

2
src/shared/import-util.c
View File

@ -160,7 +160,7 @@ int import_assign_pool_quota_and_warn(const char *path) {
if (r < 0)
return log_error_errno(r, "Failed to set up default quota hierarchy for %s: %m", path);
if (r > 0)
log_info("Set up default quota hierarchy for %s.", path);
log_debug("Set up default quota hierarchy for %s.", path);
return 0;
}

2
src/shared/machine-image.h
View File

@ -4,6 +4,8 @@
#include <stdbool.h>
#include <stdint.h>
#include "sd-id128.h"
#include "hashmap.h"
#include "lockfile-util.h"
#include "macro.h"

369
src/shared/machine-pool.c
View File

@ -1,46 +1,13 @@
/* SPDX-License-Identifier: LGPL-2.1+ */
#include <errno.h>
#include <fcntl.h>
#include <linux/loop.h>
#include <signal.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/file.h>
#include <sys/ioctl.h>
#include <sys/mount.h>
#include <sys/prctl.h>
#include <sys/stat.h>
#include <sys/statfs.h>
#include <sys/statvfs.h>
#include <unistd.h>
#include "sd-bus-protocol.h"
#include "sd-bus.h"
#include "alloc-util.h"
#include "btrfs-util.h"
#include "fd-util.h"
#include "fileio.h"
#include "fs-util.h"
#include "label.h"
#include "lockfile-util.h"
#include "log.h"
#include "machine-pool.h"
#include "macro.h"
#include "missing.h"
#include "mkdir.h"
#include "mount-util.h"
#include "parse-util.h"
#include "path-util.h"
#include "process-util.h"
#include "signal-util.h"
#include "stat-util.h"
#include "string-util.h"
#define VAR_LIB_MACHINES_SIZE_START (1024UL*1024UL*500UL)
#define VAR_LIB_MACHINES_FREE_MIN (1024UL*1024UL*750UL)
static int check_btrfs(void) {
struct statfs sfs;
@ -56,344 +23,24 @@ static int check_btrfs(void) {
return F_TYPE_EQUAL(sfs.f_type, BTRFS_SUPER_MAGIC);
}
static int setup_machine_raw(uint64_t size, sd_bus_error *error) {
_cleanup_free_ char *tmp = NULL;
_cleanup_close_ int fd = -1;
struct statvfs ss;
pid_t pid = 0;
int setup_machine_directory(sd_bus_error *error) {
int r;
/* We want to be able to make use of btrfs-specific file
* system features, in particular subvolumes, reflinks and
* quota. Hence, if we detect that /var/lib/machines.raw is
* not located on btrfs, let's create a loopback file, place a
* btrfs file system into it, and mount it to
* /var/lib/machines. */
fd = open("/var/lib/machines.raw", O_RDWR|O_CLOEXEC|O_NONBLOCK|O_NOCTTY);
if (fd >= 0)
return TAKE_FD(fd);
if (errno != ENOENT)
return sd_bus_error_set_errnof(error, errno, "Failed to open /var/lib/machines.raw: %m");
r = tempfn_xxxxxx("/var/lib/machines.raw", NULL, &tmp);
if (r < 0)
return r;
(void) mkdir_p_label("/var/lib", 0755);
fd = open(tmp, O_RDWR|O_CREAT|O_EXCL|O_NOCTTY|O_CLOEXEC, 0600);
if (fd < 0)
return sd_bus_error_set_errnof(error, errno, "Failed to create /var/lib/machines.raw: %m");
if (fstatvfs(fd, &ss) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to determine free space on /var/lib/machines.raw: %m");
goto fail;
}
if (ss.f_bsize * ss.f_bavail < VAR_LIB_MACHINES_FREE_MIN) {
r = sd_bus_error_setf(error, SD_BUS_ERROR_FAILED, "Not enough free disk space to set up /var/lib/machines.");
goto fail;
}
if (ftruncate(fd, size) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to enlarge /var/lib/machines.raw: %m");
goto fail;
}
r = safe_fork("(mkfs)", FORK_RESET_SIGNALS|FORK_DEATHSIG, &pid);
if (r < 0) {
sd_bus_error_set_errnof(error, r, "Failed to fork mkfs.btrfs: %m");
goto fail;
}
if (r == 0) {
/* Child */
fd = safe_close(fd);
execlp("mkfs.btrfs", "-Lvar-lib-machines", tmp, NULL);
if (errno == ENOENT)
_exit(99);
_exit(EXIT_FAILURE);
}
r = wait_for_terminate_and_check("mkfs", pid, 0);
pid = 0;
if (r < 0) {
sd_bus_error_set_errnof(error, r, "Failed to wait for mkfs.btrfs: %m");
goto fail;
}
if (r == 99) {
r = sd_bus_error_set_errnof(error, ENOENT, "Cannot set up /var/lib/machines, mkfs.btrfs is missing");
goto fail;
}
if (r != EXIT_SUCCESS) {
r = sd_bus_error_setf(error, SD_BUS_ERROR_FAILED, "mkfs.btrfs failed with error code %i", r);
goto fail;
}
r = rename_noreplace(AT_FDCWD, tmp, AT_FDCWD, "/var/lib/machines.raw");
if (r < 0) {
sd_bus_error_set_errnof(error, r, "Failed to move /var/lib/machines.raw into place: %m");
goto fail;
}
return TAKE_FD(fd);
fail:
unlink_noerrno(tmp);
if (pid > 1)
kill_and_sigcont(pid, SIGKILL);
return r;
}
int setup_machine_directory(uint64_t size, sd_bus_error *error) {
_cleanup_(release_lock_file) LockFile lock_file = LOCK_FILE_INIT;
struct loop_info64 info = {
.lo_flags = LO_FLAGS_AUTOCLEAR,
};
_cleanup_close_ int fd = -1, control = -1, loop = -1;
_cleanup_free_ char* loopdev = NULL;
char tmpdir[] = "/tmp/machine-pool.XXXXXX", *mntdir = NULL;
bool tmpdir_made = false, mntdir_made = false, mntdir_mounted = false;
char buf[FORMAT_BYTES_MAX];
int r, nr = -1;
/* btrfs cannot handle file systems < 16M, hence use this as minimum */
if (size == (uint64_t) -1)
size = VAR_LIB_MACHINES_SIZE_START;
else if (size < 16*1024*1024)
size = 16*1024*1024;
/* Make sure we only set the directory up once at a time */
r = make_lock_file("/run/systemd/machines.lock", LOCK_EX, &lock_file);
if (r < 0)
return r;
r = check_btrfs();
if (r < 0)
return sd_bus_error_set_errnof(error, r, "Failed to determine whether /var/lib/machines is located on btrfs: %m");
if (r > 0) {
(void) btrfs_subvol_make_label("/var/lib/machines");
r = btrfs_quota_enable("/var/lib/machines", true);
if (r < 0)
log_warning_errno(r, "Failed to enable quota for /var/lib/machines, ignoring: %m");
r = btrfs_subvol_auto_qgroup("/var/lib/machines", 0, true);
if (r < 0)
log_warning_errno(r, "Failed to set up default quota hierarchy for /var/lib/machines, ignoring: %m");
return 1;
}
if (path_is_mount_point("/var/lib/machines", NULL, AT_SYMLINK_FOLLOW) > 0) {
log_debug("/var/lib/machines is already a mount point, not creating loopback file for it.");
return 0;
}
r = dir_is_populated("/var/lib/machines");
if (r < 0 && r != -ENOENT)
return r;
if (r > 0) {
log_debug("/var/log/machines is already populated, not creating loopback file for it.");
return 0;
}
r = mkfs_exists("btrfs");
if (r == 0)
return sd_bus_error_set_errnof(error, ENOENT, "Cannot set up /var/lib/machines, mkfs.btrfs is missing");
if (r < 0)
return r;
fd = setup_machine_raw(size, error);
if (fd < 0)
return fd;
control = open("/dev/loop-control", O_RDWR|O_CLOEXEC|O_NOCTTY|O_NONBLOCK);
if (control < 0)
return sd_bus_error_set_errnof(error, errno, "Failed to open /dev/loop-control: %m");
nr = ioctl(control, LOOP_CTL_GET_FREE);
if (nr < 0)
return sd_bus_error_set_errnof(error, errno, "Failed to allocate loop device: %m");
if (asprintf(&loopdev, "/dev/loop%i", nr) < 0) {
r = -ENOMEM;
goto fail;
}
loop = open(loopdev, O_CLOEXEC|O_RDWR|O_NOCTTY|O_NONBLOCK);
if (loop < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to open loopback device: %m");
goto fail;
}
if (ioctl(loop, LOOP_SET_FD, fd) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to bind loopback device: %m");
goto fail;
}
if (ioctl(loop, LOOP_SET_STATUS64, &info) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to enable auto-clear for loopback device: %m");
goto fail;
}
/* We need to make sure the new /var/lib/machines directory
* has an access mode of 0700 at the time it is first made
* available. mkfs will create it with 0755 however. Hence,
* let's mount the directory into an inaccessible directory
* below /tmp first, fix the access mode, and move it to the
* public place then. */
if (!mkdtemp(tmpdir)) {
r = sd_bus_error_set_errnof(error, errno, "Failed to create temporary mount parent directory: %m");
goto fail;
}
tmpdir_made = true;
mntdir = strjoina(tmpdir, "/mnt");
if (mkdir(mntdir, 0700) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to create temporary mount directory: %m");
goto fail;
}
mntdir_made = true;
if (mount(loopdev, mntdir, "btrfs", 0, NULL) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to mount loopback device: %m");
goto fail;
}
mntdir_mounted = true;
r = btrfs_quota_enable(mntdir, true);
if (r < 0)
log_warning_errno(r, "Failed to enable quota, ignoring: %m");
r = btrfs_subvol_auto_qgroup(mntdir, 0, true);
if (r < 0)
log_warning_errno(r, "Failed to set up default quota hierarchy, ignoring: %m");
if (chmod(mntdir, 0700) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to fix owner: %m");
goto fail;
}
(void) mkdir_p_label("/var/lib/machines", 0700);
if (mount(mntdir, "/var/lib/machines", NULL, MS_BIND, NULL) < 0) {
r = sd_bus_error_set_errnof(error, errno, "Failed to mount directory into right place: %m");
goto fail;
}
(void) syncfs(fd);
log_info("Set up /var/lib/machines as btrfs loopback file system of size %s mounted on /var/lib/machines.raw.", format_bytes(buf, sizeof(buf), size));
(void) umount2(mntdir, MNT_DETACH);
(void) rmdir(mntdir);
(void) rmdir(tmpdir);
return 1;
fail:
if (mntdir_mounted)
(void) umount2(mntdir, MNT_DETACH);
if (mntdir_made)
(void) rmdir(mntdir);
if (tmpdir_made)
(void) rmdir(tmpdir);
if (loop >= 0) {
(void) ioctl(loop, LOOP_CLR_FD);
loop = safe_close(loop);
}
(void) ioctl(control, LOOP_CTL_REMOVE, nr);
return r;
}
static int sync_path(const char *p) {
_cleanup_close_ int fd = -1;
fd = open(p, O_RDONLY|O_CLOEXEC|O_NOCTTY);
if (fd < 0)
return -errno;
if (syncfs(fd) < 0)
return -errno;
return 0;
}
int grow_machine_directory(void) {
char buf[FORMAT_BYTES_MAX];
struct statvfs a, b;
uint64_t old_size, new_size, max_add;
int r;
/* Ensure the disk space data is accurate */
sync_path("/var/lib/machines");
sync_path("/var/lib/machines.raw");
if (statvfs("/var/lib/machines.raw", &a) < 0)
return -errno;
if (statvfs("/var/lib/machines", &b) < 0)
return -errno;
/* Don't grow if not enough disk space is available on the host */
if (((uint64_t) a.f_bavail * (uint64_t) a.f_bsize) <= VAR_LIB_MACHINES_FREE_MIN)
return 0;
/* Don't grow if at least 1/3th of the fs is still free */
if (b.f_bavail > b.f_blocks / 3)
return 0;
/* Calculate how much we are willing to add at most */
max_add = ((uint64_t) a.f_bavail * (uint64_t) a.f_bsize) - VAR_LIB_MACHINES_FREE_MIN;
/* Calculate the old size */
old_size = (uint64_t) b.f_blocks * (uint64_t) b.f_bsize;
/* Calculate the new size as three times the size of what is used right now */
new_size = ((uint64_t) b.f_blocks - (uint64_t) b.f_bavail) * (uint64_t) b.f_bsize * 3;
/* Always, grow at least to the start size */
if (new_size < VAR_LIB_MACHINES_SIZE_START)
new_size = VAR_LIB_MACHINES_SIZE_START;
/* If the new size is smaller than the old size, don't grow */
if (new_size < old_size)
return 0;
/* Ensure we never add more than the maximum */
if (new_size > old_size + max_add)
new_size = old_size + max_add;
r = btrfs_resize_loopback("/var/lib/machines", new_size, true);
if (r < 0)
return log_debug_errno(r, "Failed to resize loopback: %m");
if (r == 0)
return 0;
/* Also bump the quota, of both the subvolume leaf qgroup, as
* well as of any subtree quota group by the same id but a
* higher level, if it exists. */
r = btrfs_qgroup_set_limit("/var/lib/machines", 0, new_size);
if (r < 0)
log_debug_errno(r, "Failed to set btrfs limit: %m");
(void) btrfs_subvol_make_label("/var/lib/machines");
r = btrfs_subvol_set_subtree_quota_limit("/var/lib/machines", 0, new_size);
r = btrfs_quota_enable("/var/lib/machines", true);
if (r < 0)
log_debug_errno(r, "Failed to set btrfs subtree limit: %m");
log_warning_errno(r, "Failed to enable quota for /var/lib/machines, ignoring: %m");
r = btrfs_subvol_auto_qgroup("/var/lib/machines", 0, true);
if (r < 0)
log_warning_errno(r, "Failed to set up default quota hierarchy for /var/lib/machines, ignoring: %m");
log_info("Grew /var/lib/machines btrfs loopback file system to %s.", format_bytes(buf, sizeof(buf), new_size));
return 1;
}

6
src/shared/machine-pool.h
View File

@ -5,8 +5,4 @@
#include "sd-bus.h"
/* Grow the /var/lib/machines directory after each 10MiB written */
#define GROW_INTERVAL_BYTES (UINT64_C(10) * UINT64_C(1024) * UINT64_C(1024))
int setup_machine_directory(uint64_t size, sd_bus_error *error);
int grow_machine_directory(void);
int setup_machine_directory(sd_bus_error *error);

2
src/shared/verbs.h
View File

@ -1,6 +1,8 @@
/* SPDX-License-Identifier: LGPL-2.1+ */
#pragma once
#include <stdbool.h>
#define VERB_ANY ((unsigned) -1)
typedef enum VerbFlags {

41
src/test/test-path-util.c
View File

@ -406,6 +406,7 @@ static void test_file_in_same_dir(void) {
}
static void test_last_path_component(void) {
assert_se(last_path_component(NULL) == NULL);
assert_se(streq(last_path_component("a/b/c"), "c"));
assert_se(streq(last_path_component("a/b/c/"), "c/"));
assert_se(streq(last_path_component("/"), "/"));
@ -424,6 +425,45 @@ static void test_last_path_component(void) {
assert_se(streq(last_path_component("/a/"), "a/"));
}
static void test_path_extract_filename_one(const char *input, const char *output, int ret) {
_cleanup_free_ char *k = NULL;
int r;
r = path_extract_filename(input, &k);
log_info("%s → %s/%s [expected: %s/%s]", strnull(input), strnull(k), strerror(-r), strnull(output), strerror(-ret));
assert_se(streq_ptr(k, output));
assert_se(r == ret);
}
static void test_path_extract_filename(void) {
test_path_extract_filename_one(NULL, NULL, -EINVAL);
test_path_extract_filename_one("a/b/c", "c", 0);
test_path_extract_filename_one("a/b/c/", "c", 0);
test_path_extract_filename_one("/", NULL, -EINVAL);
test_path_extract_filename_one("//", NULL, -EINVAL);
test_path_extract_filename_one("///", NULL, -EINVAL);
test_path_extract_filename_one(".", NULL, -EINVAL);
test_path_extract_filename_one("./.", NULL, -EINVAL);
test_path_extract_filename_one("././", NULL, -EINVAL);
test_path_extract_filename_one("././/", NULL, -EINVAL);
test_path_extract_filename_one("/foo/a", "a", 0);
test_path_extract_filename_one("/foo/a/", "a", 0);
test_path_extract_filename_one("", NULL, -EINVAL);
test_path_extract_filename_one("a", "a", 0);
test_path_extract_filename_one("a/", "a", 0);
test_path_extract_filename_one("/a", "a", 0);
test_path_extract_filename_one("/a/", "a", 0);
test_path_extract_filename_one("/////////////a/////////////", "a", 0);
test_path_extract_filename_one("xx/.", NULL, -EINVAL);
test_path_extract_filename_one("xx/..", NULL, -EINVAL);
test_path_extract_filename_one("..", NULL, -EINVAL);
test_path_extract_filename_one("/..", NULL, -EINVAL);
test_path_extract_filename_one("../", NULL, -EINVAL);
test_path_extract_filename_one(".", NULL, -EINVAL);
test_path_extract_filename_one("/.", NULL, -EINVAL);
test_path_extract_filename_one("./", NULL, -EINVAL);
}
static void test_filename_is_valid(void) {
char foo[FILENAME_MAX+2];
int i;
@ -542,6 +582,7 @@ int main(int argc, char **argv) {
test_prefix_root();
test_file_in_same_dir();
test_last_path_component();
test_path_extract_filename();
test_filename_is_valid();
test_hidden_or_backup_file();
test_skip_dev_prefix();

1
test/TEST-25-IMPORT/Makefile Symbolic link
View File

@ -0,0 +1 @@
../TEST-01-BASIC/Makefile

43
test/TEST-25-IMPORT/test.sh Executable file
View File

@ -0,0 +1,43 @@
#!/bin/bash
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
# ex: ts=8 sw=4 sts=4 et filetype=sh
set -e
TEST_DESCRIPTION="test importd"
. $TEST_BASE_DIR/test-functions
test_setup() {
create_empty_image
mkdir -p $TESTDIR/root
mount ${LOOPDEV}p1 $TESTDIR/root
(
LOG_LEVEL=5
eval $(udevadm info --export --query=env --name=${LOOPDEV}p2)
setup_basic_environment
dracut_install dd gunzip mv tar diff
# setup the testsuite service
cat >$initdir/etc/systemd/system/testsuite.service <<EOF
[Unit]
Description=Testsuite service
[Service]
ExecStart=/testsuite.sh
Type=oneshot
StandardOutput=tty
StandardError=tty
NotifyAccess=all
EOF
cp testsuite.sh $initdir/
setup_testsuite
) || return 1
setup_nspawn_root
ddebug "umount $TESTDIR/root"
umount $TESTDIR/root
}
do_test "$@"

128
test/TEST-25-IMPORT/testsuite.sh Executable file
View File

@ -0,0 +1,128 @@
#!/bin/bash
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
# ex: ts=8 sw=4 sts=4 et filetype=sh
set -ex
set -o pipefail
export SYSTEMD_PAGER=cat
dd if=/dev/urandom of=/var/tmp/testimage.raw bs=$((1024*1024+7)) count=5
# Test import
machinectl import-raw /var/tmp/testimage.raw
machinectl image-status testimage
test -f /var/lib/machines/testimage.raw
cmp /var/tmp/testimage.raw /var/lib/machines/testimage.raw
# Test export
machinectl export-raw testimage /var/tmp/testimage2.raw
cmp /var/tmp/testimage.raw /var/tmp/testimage2.raw
rm /var/tmp/testimage2.raw
# Test compressed export (gzip)
machinectl export-raw testimage /var/tmp/testimage2.raw.gz
gunzip /var/tmp/testimage2.raw.gz
cmp /var/tmp/testimage.raw /var/tmp/testimage2.raw
rm /var/tmp/testimage2.raw
# Test clone
machinectl clone testimage testimage3
test -f /var/lib/machines/testimage3.raw
machinectl image-status testimage3
test -f /var/lib/machines/testimage.raw
machinectl image-status testimage
cmp /var/tmp/testimage.raw /var/lib/machines/testimage.raw
cmp /var/tmp/testimage.raw /var/lib/machines/testimage3.raw
# Test removal
machinectl remove testimage
! test -f /var/lib/machines/testimage.raw
! machinectl image-status testimage
# Test export of clone
machinectl export-raw testimage3 /var/tmp/testimage3.raw
cmp /var/tmp/testimage.raw /var/tmp/testimage3.raw
rm /var/tmp/testimage3.raw
# Test rename
machinectl rename testimage3 testimage4
test -f /var/lib/machines/testimage4.raw
machinectl image-status testimage4
! test -f /var/lib/machines/testimage3.raw
! machinectl image-status testimage3
cmp /var/tmp/testimage.raw /var/lib/machines/testimage4.raw
# Test export of rename
machinectl export-raw testimage4 /var/tmp/testimage4.raw
cmp /var/tmp/testimage.raw /var/tmp/testimage4.raw
rm /var/tmp/testimage4.raw
# Test removal
machinectl remove testimage4
! test -f /var/lib/machines/testimage4.raw
! machinectl image-status testimage4
# → And now, let's test directory trees ← #
# Set up a directory we can import
mkdir /var/tmp/scratch
mv /var/tmp/testimage.raw /var/tmp/scratch/
touch /var/tmp/scratch/anotherfile
mkdir /var/tmp/scratch/adirectory
echo "piep" > /var/tmp/scratch/adirectory/athirdfile
# Test import-fs
machinectl import-fs /var/tmp/scratch/
test -d /var/lib/machines/scratch
machinectl image-status scratch
# Test export-tar
machinectl export-tar scratch /var/tmp/scratch.tar.gz
test -f /var/tmp/scratch.tar.gz
mkdir /var/tmp/extract
(cd /var/tmp/extract ; tar xzf /var/tmp/scratch.tar.gz)
diff -r /var/tmp/scratch/ /var/tmp/extract/
rm -rf /var/tmp/extract
# Test import-tar
machinectl import-tar /var/tmp/scratch.tar.gz scratch2
test -d /var/lib/machines/scratch2
machinectl image-status scratch2
diff -r /var/tmp/scratch/ /var/lib/machines/scratch2
# Test removal
machinectl remove scratch
! test -f /var/lib/machines/scratch
! machinectl image-status scratch
# Test clone
machinectl clone scratch2 scratch3
test -d /var/lib/machines/scratch2
machinectl image-status scratch2
test -d /var/lib/machines/scratch3
machinectl image-status scratch3
diff -r /var/tmp/scratch/ /var/lib/machines/scratch3
# Test removal
machinectl remove scratch2
! test -f /var/lib/machines/scratch2
! machinectl image-status scratch2
# Test rename
machinectl rename scratch3 scratch4
test -d /var/lib/machines/scratch4
machinectl image-status scratch4
! test -f /var/lib/machines/scratch3
! machinectl image-status scratch3
diff -r /var/tmp/scratch/ /var/lib/machines/scratch4
# Test removal
machinectl remove scratch4
! test -f /var/lib/machines/scratch4
! machinectl image-status scratch4
rm -rf /var/tmp/scratch
echo OK > /testok
exit 0

7
units/var-lib-machines.mount
View File

@ -7,8 +7,13 @@
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
# This unit is required for pre-240 versions of systemd that automatically set
# up /var/lib/machines.raw as loopback-mounted btrfs file system. Later
# versions don't do that anymore, but let's keep minimal compatibility by
# mounting the image still, if it exists.
[Unit]
Description=Virtual Machine and Container Storage
Description=Virtual Machine and Container Storage (Compatibility)
ConditionPathExists=/var/lib/machines.raw
[Mount]