Commit Graph

271 Commits

Author SHA1 Message Date
Lennart Poettering 754061ce71 importd: enable btrfs quota in /var/lib/machines, if necessary 2015-02-24 18:46:49 +01:00
Lennart Poettering 113b3fc1a8 importd: create a loopback btrfs file system for /var/lib/machines, if necessary
When manipulating container and VM images we need efficient and atomic
directory snapshots and file copies, as well as disk quota. btrfs
provides this, legacy file systems do not. Hence, implicitly create a
loopback file system in /var/lib/machines.raw and mount it to
/var/lib/machines, if that directory is not on btrfs anyway.

This is done implicitly and transparently the first time the user
invokes "machinectl import-xyz".

This allows us to take benefit of btrfs features for container
management without actually having the rest of the system use btrfs.

The loopback is sized 500M initially. Patches to grow it dynamically are
to follow.
2015-02-24 17:27:53 +01:00
Lennart Poettering ce06fdfb3d import: print nice warning if we need btrfs but /var/lib/machines is not btrfs 2015-02-24 17:27:53 +01:00
Lennart Poettering 1c8da04446 shared: introduce cmsg_close_all() call
The call iterates through cmsg list and closes all fds passed via
SCM_RIGHTS.

This patch also ensures the call is used wherever appropriate, where we
might get spurious fds sent and we should better close them, then leave
them lying around.
2015-02-18 19:42:24 +01:00
Lennart Poettering c529695e7a logind: open up most bus calls for unpriviliged processes, using PolicyKit
Also, allow clients to alter their own objects without any further
priviliges. i.e. this allows clients to kill and lock their own sessions
without involving PK.
2015-02-18 12:55:25 +01:00
Thomas Hindoe Paaboel Andersen b89c454b37 import: remove unused variable 2015-02-17 20:06:13 +01:00
Tom Gundersen a97dcc12e4 networkd: exit on idle
We will be woken up on rtnl or dbus activity, so let's just quit if some time has passed and that is the only thing that can happen.

Note that we will always stay around if we expect network activity (e.g. DHCP is enabled), as we are not restarted on that.
2015-02-05 12:04:19 +01:00
Lennart Poettering 63c372cb9d util: rework strappenda(), and rename it strjoina()
After all it is now much more like strjoin() than strappend(). At the
same time, add support for NULL sentinels, even if they are normally not
necessary.
2015-02-03 02:05:59 +01:00
Torstein Husebø cc98b3025e treewide: fix multiple typos 2015-01-26 10:39:47 -05:00
Zbigniew Jędrzejewski-Szmek c9403dced7 #pragma once here and there 2015-01-23 09:30:44 -05:00
Lennart Poettering 7079cfeffb importd: when listing transfers, show progress percentage
With this change the pull protocol implementation processes will pass
progress data to importd which then passes this information on via the
bus. We use sd_notify() as generic transport for this communication,
making importd listen to them, while matching the incoming messages to
the right transfer.
2015-01-23 01:17:55 +01:00
Lennart Poettering a92ccc5ba2 importd: fix bus policy 2015-01-23 01:17:55 +01:00
Lennart Poettering e026c242af import: we need CAP_DAC_OVERRIDE for untarring systems after all 2015-01-23 01:17:55 +01:00
Lennart Poettering 7315edd36a import: lock tar into its own private network namespace
That way it cannot get access to the network
2015-01-22 18:19:58 +01:00
Lennart Poettering 2c140ded48 import: drop all capabilities when invoking tar 2015-01-22 18:12:31 +01:00
Lennart Poettering b9a5f8588f import: only define the _to_string() enum mapping function, thus making gcc shut up 2015-01-22 17:50:50 +01:00
Lennart Poettering aa9bd4992f import: now that the worker binary is called "systemd-pull" we can shorten the verbs
Atfer all "systemd-pull pull-tar" is unnecessarily redundant, over
"systemd-pull tar"...
2015-01-22 17:50:50 +01:00
Lennart Poettering ff828763c4 importd: try to minimize confusion by renaming "systemd-import" binary to "systemd-pull"
This way "systemd-importd" is the daemon that uses "systemd-pull" as
backend worker.
2015-01-22 17:50:50 +01:00
Lennart Poettering 09d46cfd41 importd: minor log improvements 2015-01-22 17:50:50 +01:00
Lennart Poettering 1c49d1ba85 import: make the user verficiation keyring override the vendor keyring, instead of extending it
This way the user has the ability to remove keys from the
vendor-supplied keyring if he intends so.
2015-01-22 17:50:50 +01:00
Lennart Poettering 7f444afa1b import: rename --verify=sum to --verify=checksum
This is how we call it internally, and also a bit more descriptive.
2015-01-22 15:14:36 +01:00
Lennart Poettering 3d7415f43f import: introduce new mini-daemon systemd-importd, and make machinectl a client to it
The old "systemd-import" binary is now an internal tool. We still use it
as asynchronous backend for systemd-importd. Since the import tool might
require some IO and CPU resources (due to qcow2 explosion, and
decompression), and because we might want to run it with more minimal
priviliges we still keep it around as the worker binary to execute as
child process of importd.

machinectl now has verbs for pulling down images, cancelling them and
listing them.
2015-01-22 04:02:07 +01:00
Lennart Poettering f4c135bf2f impot: minor cleanups 2015-01-22 04:02:07 +01:00
Lennart Poettering ff2670ad11 import: simplify dkr importer, by making use of generic import-job logic, used by the raw and tar importers
This gets us progress output as well xz/bzip2 support.
2015-01-21 20:05:31 +01:00
Lennart Poettering 8b71fce8c2 import: minor cleanups for the tar and raw importers 2015-01-21 20:05:31 +01:00
Lennart Poettering 8af3cf74df import: support downloading bzip2-encoded images
This way, we can import CoreOS images unmodified.
2015-01-21 13:44:29 +01:00
Lennart Poettering 0100b6e1bf import: also add verification support to tar importer 2015-01-21 04:03:54 +01:00
Lennart Poettering 98c3800184 import: make verification code generic, in preparation for using it pull-tar 2015-01-21 04:03:54 +01:00
Lennart Poettering 5a3b1abd0e import: improve logging 2015-01-21 04:03:54 +01:00
Lennart Poettering 90bc083bda import: show download speed while downloading 2015-01-21 04:03:54 +01:00
Lennart Poettering 3576d6315f import: add image verification using gpg
This also adds an initial keyring for the verification, that contains
Ubuntu's and Fedora's key. We should probably add more entries sooner or
later.
2015-01-21 04:03:54 +01:00
Lennart Poettering c660bb0942 import: add a couple of additional suffixes to remove from raw images 2015-01-20 20:40:44 +01:00
Lennart Poettering 8f6950587a import: make image verification optional 2015-01-20 20:40:44 +01:00
Lennart Poettering 85dbc41dc6 import: add a simple scheme for validating the SHA256 sums of downloaded raw files 2015-01-20 15:06:58 +01:00
Lennart Poettering 88a1aadc48 import: be less aggressive when allocating memory for downloaded payload 2015-01-20 15:06:58 +01:00
Lennart Poettering 68c913fd75 import: improve logging a bit 2015-01-20 15:06:58 +01:00
Lennart Poettering 0d6e763b48 import: port pull-raw to helper tools implemented for pull-tar
This allows us to reuse a lot more code, and simplify pull-raw
drastically.
2015-01-20 15:06:58 +01:00
Lennart Poettering 56ebfaf1ca import: add support for pulling raw tar balls as containers
Ubuntu provides their cloud images optionally as tarball, hence also
support downloading those.
2015-01-20 15:06:58 +01:00
Lennart Poettering a2e0337875 util: make http url validity checks more generic, and move them to util.c 2015-01-20 15:06:58 +01:00
Lennart Poettering ec5cb56ee1 import: clarify when we are unpacking the qcow2 device 2015-01-19 20:24:10 +01:00
Lennart Poettering 0716faad4a import: make sure don't leak the LZMA context 2015-01-19 20:24:10 +01:00
Lennart Poettering 1c7dd82563 qcow2: when dissecting qcow2, use btrfs clone ioctls for reflinking blocks to target 2015-01-19 20:24:09 +01:00
Lennart Poettering 1e20b41187 import: when downloading raw files, show simple progress reports 2015-01-19 20:24:09 +01:00
Lennart Poettering 2f64ba0e6e import: simplify the code a bit 2015-01-19 20:24:09 +01:00
Lennart Poettering ff6a74609b import-raw: when downloading raw images, generate sparse files if we can 2015-01-19 20:24:09 +01:00
Lennart Poettering 47bc4fd86d import-raw: set NOCOW flag on all raw images we create 2015-01-19 20:24:09 +01:00
Zbigniew Jędrzejewski-Szmek a2341f6836 Move DEFINE_TRIVIAL_CLEANUP_FUNC to macro.h
This remove the need for various header files to include the
(relatively heavyweight) util.h.
2015-01-18 19:06:48 -05:00
Cristian Rodríguez 0193ad26ba util: replace RUN_WITH_LOCALE with extended locale functions
There were two callers, one can use strtod_l() and the other strptime_l().

(David: fix up commit-msg and coding-style)
2015-01-18 22:08:44 +01:00
Lennart Poettering edce2aed3a import: support importing qcow2 images
With this change the import tool will now unpack qcow2 images into
normal raw disk images, suitable for usage with nspawn.

This allows has the benefit of also allowing importing Ubuntu Cloud
images for usage with nspawn.
2015-01-16 20:09:33 +01:00
Lennart Poettering 49bb233bb7 import: support downloading .xz compressed images
That way we can download fedora cloud raw images as-is and decompress
them on-the-fly.
2015-01-16 20:09:33 +01:00
Lennart Poettering 5f129649b9 nspawn,machined: change default container image location from /var/lib/container to /var/lib/machines
Given that this is also the place to store raw disk images which are
very much bootable with qemu/kvm it sounds like a misnomer to call the
directory "container". Hence, let's change this sooner rather than
later, and use the generic name, in particular since we otherwise try to
use the generic "machine" preferably over the more specific "container"
or "vm".
2015-01-15 01:47:21 +01:00
Lennart Poettering aceac2f0b6 import: rename "gpt" disk image type to "raw"
After all, nspawn can now dissect MBR partition levels, too, hence
".gpt" appears a misnomer. Moreover, the the .raw suffix for these files
is already pretty popular (the Fedora disk images use it for example),
hence sounds like an OK scheme to adopt.
2015-01-15 01:47:21 +01:00
Lennart Poettering 6da023a048 import: make sure we don't mangle file ownerships with the local passwd database when untarring 2015-01-13 13:55:15 +01:00
Lennart Poettering dfd1520d3a import: also set NOCOW for gpt disk images
Given the write patterns on disk images, we better should turn COW off
for them. In particular as the file systems used inside the disk images
should do their own data integrity checks anyway and we don't need
multiple layers of it.
2015-01-08 01:25:40 +01:00
David Herrmann c5285fbfce import: fix mem-leak in CurlGlue
Make sure to actually free the underlying object in CurlGlue unref.
2014-12-31 16:01:37 +01:00
Lennart Poettering 087682d103 import: make image root directory configurable, instead of hardcoding /var/lib/container 2014-12-26 19:33:15 +01:00
Lennart Poettering f0be89eee9 import: properly remove pre-existing images if --force is used 2014-12-26 19:33:15 +01:00
Lennart Poettering 8620a9a323 import: beef up gpt importer to optionally make writable copy of read-only vendor image 2014-12-26 19:21:58 +01:00
Lennart Poettering e9d7333468 import: minor improvements to dkr importer 2014-12-26 19:21:58 +01:00
Lennart Poettering 10f9c75519 machined: beef up machined image listing with creation/modification times of subvolumes
We make use of the btrfs subvol crtime for this, and for gpt images of a
manually managed xattr, if we can.
2014-12-25 03:19:19 +01:00
Lennart Poettering 5fa89b2cb3 import: prefer usec_t over time_t 2014-12-25 03:14:09 +01:00
Lennart Poettering 901992209e import: add a new "pull-gpt" verb for downloading GPT disk images from the internet 2014-12-24 16:53:05 +01:00
Lennart Poettering 0c7bf33a98 import: three minor fixes 2014-12-24 16:53:05 +01:00
Lennart Poettering a36544cd45 import: print friendly error messages on errors 2014-12-24 16:53:05 +01:00
Lennart Poettering 14ed8b9273 import: remember when we were finished importing 2014-12-24 16:53:05 +01:00
Lennart Poettering ea1ae8c38e import: make the dkr import URL a part of the import object, not the import name object 2014-12-24 16:53:05 +01:00
Lennart Poettering 6eeeb84c96 import: fix compiler warning 2014-12-23 19:15:27 +01:00
Lennart Poettering 91f4347ef7 import: rename 'poll-dck' to 'pull-dkr'
I figure "pull-dck" is not a good name, given that one could certainly
read the verb in a way that might be funny for 16year-olds. ;-)

Also, don't hardcode the index URL to use, make it runtime and configure
time configurable instead.
2014-12-23 03:25:36 +01:00
Lennart Poettering 7eeeb28e45 import: Verb[] array can be static, too 2014-12-19 19:19:29 +01:00
Tom Gundersen eac8e8c6de import: dck - fix curl error handling 2014-12-19 03:00:56 +01:00
Lennart Poettering 72648326ea import: add new minimal tool "systemd-import" for pulling down foreign containers and install them locally
This adds a simply but powerful tool for downloading container images
from the most popular container solution used today. Use it like
this:

       # systemd-import pull-dck mattdm/fedora
       # systemd-nspawn -M fedora

This will donwload the layers for "mattdm/fedora", and make them
available locally as /var/lib/container/fedora.

The tool is pretty complete, as long as it's only about pulling down
images, or updating them. Pushing or searching is not supported yet.
2014-12-19 02:08:14 +01:00