This adds a new invocation ID concept to the service manager. The invocation ID
identifies each runtime cycle of a unit uniquely. A new randomized 128bit ID is
generated each time a unit moves from and inactive to an activating or active
state.
The primary usecase for this concept is to connect the runtime data PID 1
maintains about a service with the offline data the journal stores about it.
Previously we'd use the unit name plus start/stop times, which however is
highly racy since the journal will generally process log data after the service
already ended.
The "invocation ID" kinda matches the "boot ID" concept of the Linux kernel,
except that it applies to an individual unit instead of the whole system.
The invocation ID is passed to the activated processes as environment variable.
It is additionally stored as extended attribute on the cgroup of the unit. The
latter is used by journald to automatically retrieve it for each log logged
message and attach it to the log entry. The environment variable is very easily
accessible, even for unprivileged services. OTOH the extended attribute is only
accessible to privileged processes (this is because cgroupfs only supports the
"trusted." xattr namespace, not "user."). The environment variable may be
altered by services, the extended attribute may not be, hence is the better
choice for the journal.
Note that reading the invocation ID off the extended attribute from journald is
racy, similar to the way reading the unit name for a logging process is.
This patch adds APIs to read the invocation ID to sd-id128:
sd_id128_get_invocation() may be used in a similar fashion to
sd_id128_get_boot().
PID1's own logging is updated to always include the invocation ID when it logs
information about a unit.
A new bus call GetUnitByInvocationID() is added that allows retrieving a bus
path to a unit by its invocation ID. The bus path is built using the invocation
ID, thus providing a path for referring to a unit that is valid only for the
current runtime cycleof it.
Outlook for the future: should the kernel eventually allow passing of cgroup
information along AF_UNIX/SOCK_DGRAM messages via a unique cgroup id, then we
can alter the invocation ID to be generated as hash from that rather than
entirely randomly. This way we can derive the invocation race-freely from the
messages.
This adds two (privileged) bus calls Ref() and Unref() to the Unit interface.
The two calls may be used by clients to pin a unit into memory, so that various
runtime properties aren't flushed out by the automatic GC. This is necessary
to permit clients to race-freely acquire runtime results (such as process exit
status/code or accumulated CPU time) on successful service termination.
Ref() and Unref() are fully recursive, hence act like the usual reference
counting concept in C. Taking a reference is a privileged operation, as this
allows pinning units into memory which consumes resources.
Transient units may also gain a reference at the time of creation, via the new
AddRef property (that is only defined for transient units at the time of
creation).
This adds a new boolean setting DynamicUser= to service files. If set, a new
user will be allocated dynamically when the unit is started, and released when
it is stopped. The user ID is allocated from the range 61184..65519. The user
will not be added to /etc/passwd (but an NSS module to be added later should
make it show up in getent passwd).
For now, care should be taken that the service writes no files to disk, since
this might result in files owned by UIDs that might get assigned dynamically to
a different service later on. Later patches will tighten sandboxing in order to
ensure that this cannot happen, except for a few selected directories.
A simple way to test this is:
systemd-run -p DynamicUser=1 /bin/sleep 99999
The unit load queue can be processed in the middle of setting the
unit's properties, so its load_state would no longer be UNIT_STUB
for the check in bus_unit_set_properties(), which would cause it to
incorrectly return an error.
This new method returns information by unit names. Instead of ListUnitsByPatterns
this method returns information of inactive and even unexisting units.
Moved dbus unit reply logic into a separate shared function.
Resolves https://github.com/coreos/fleet/pull/1418
We generally follow the rule that for time settings we suffix the setting name
with "Sec" to indicate the default unit if none is specified. The only
exception was the rate limiting interval settings. Fix this, and keep the old
names for compatibility.
Do the same for journald's RateLimitInterval= setting
This adds a new GetProcesses() bus call to the Unit object which returns an
array consisting of all PIDs, their process names, as well as their full cgroup
paths. This is then used by "systemctl status" to show the per-unit process
tree.
This has the benefit that the client-side no longer needs to access the
cgroupfs directly to show the process tree of a unit. Instead, it now uses this
new API, which means it also works if -H or -M are used correctly, as the
information from the specific host is used, and not the one from the local
system.
Fixes: #2945
Fixes#2191:
$ systemctl --root=/ enable sddm
Created symlink /etc/systemd/system/display-manager.service, pointing to /usr/lib/systemd/system/sddm.service.
$ sudo build/systemctl --root=/ enable gdm
Failed to enable unit, file /etc/systemd/system/display-manager.service already exists and is a symlink to /usr/lib/systemd/system/sddm.service.
$ sudo build/systemctl --root= enable sddm
$ sudo build/systemctl --root= enable gdm
Failed to enable unit: File /etc/systemd/system/display-manager.service already exists and is a symlink to /usr/lib/systemd/system/sddm.service.
(I tried a few different approaches to pass the error information back to the
caller. Adding a new parameter to hold the error results in a gigantic patch
and a lot of hassle to pass the args arounds. Adding this information to the
changes array is straightforward and can be more easily extended in the
future.)
In case local installation is performed, the full set of errors can be reported
and we do that. When running over dbus, only the first error is reported.
With any masked unit that would that would be enabled by presets, we'd get:
test@rawhide $ sudo systemctl preset-all
Failed to execute operation: Unit file is masked.
test@rawhide $ sudo systemctl --root=/ preset-all
Operation failed: Cannot send after transport endpoint shutdown
Simply ignore those units:
test@rawhide $ sudo systemctl preset-all
Unit xxx.service is masked, ignoring.
If the error code ever leaks (we print the strerror error instead of providing
our own), the message for ESHUTDOWN is "Cannot send after transport endpoint
shutdown", which can be misleading. In particular it suggest that some
mishandling of the dbus connection occured. Let's change that to ERFKILL which
has the advantage that a) it sounds implausible as actual error, b) has the
connotation of disabling something manually.
We don't allow using config symlinks to enable units, but the error message we
printed was awful. Fix that, and generate a more readable error.
Fixes#3010.
This allows dropping all user configuration and reverting back to the vendor
default of a unit file. It basically undoes what "systemctl edit", "systemctl
set-property" and "systemctl mask" do.
Now, that the search path logic knows the unit path for transient units we also
can introduce an explicit unit file state "transient" that clarifies to the
user what kind of unit file he is encountering.
Previously, we had two enums ManagerRunningAs and UnitFileScope, that were
mostly identical and converted from one to the other all the time. The latter
had one more value UNIT_FILE_GLOBAL however.
Let's simplify things, and remove ManagerRunningAs and replace it by
UnitFileScope everywhere, thus making the translation unnecessary. Introduce
two new macros MANAGER_IS_SYSTEM() and MANAGER_IS_USER() to simplify checking
if we are running in one or the user context.
A long time ago – when generators where first introduced – the directories for
them were randomly created via mkdtemp(). This was changed later so that they
use fixed name directories now. Let's make use of this, and add the genrator
dirs to the LookupPaths structure and into the unit file search path maintained
in it. This has the benefit that the generator dirs are now normal part of the
search path for all tools, and thus are shown in "systemctl list-unit-files"
too.
Add path argument to clock_is_localtime() and default to "/etc/adjtime" if it's
NULL. This makes the function testable.
Add test-clock: initial test cases for some scenarios, using a temporary file.
This also checks the behaviour with a NULL (i. e. the system's /etc/adjtime)
file.
How to reproduce
$ systemctl set-default multi-user # https://github.com/systemd/systemd/issues/2298
$ systemctl preset-all
Failed to execute operation: Too many levels of symbolic links
$ systemctl poweroff
Fixes:
==1==
==1== HEAP SUMMARY:
==1== in use at exit: 65,645 bytes in 7 blocks
==1== total heap usage: 40,539 allocs, 40,532 frees, 30,147,547 bytes allocated
==1==
==1== 109 (24 direct, 85 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 7
==1== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1== by 0x4C2DE2F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1== by 0x23DA71: unit_file_changes_add (install.c:233)
==1== by 0x23E45D: remove_marked_symlinks_fd (install.c:453)
==1== by 0x23E267: remove_marked_symlinks_fd (install.c:405)
==1== by 0x23E641: remove_marked_symlinks (install.c:494)
==1== by 0x243A91: execute_preset (install.c:2190)
==1== by 0x244343: unit_file_preset_all (install.c:2351)
==1== by 0x18AAA2: method_preset_all_unit_files (dbus-manager.c:1846)
==1== by 0x1D8157: method_callbacks_run (bus-objects.c:420)
==1== by 0x1DA9E9: object_find_and_run (bus-objects.c:1257)
==1== by 0x1DB02B: bus_process_object (bus-objects.c:1373)
==1==
==1== LEAK SUMMARY:
==1== definitely lost: 24 bytes in 1 blocks
==1== indirectly lost: 85 bytes in 1 blocks
==1== possibly lost: 0 bytes in 0 blocks
==1== still reachable: 65,536 bytes in 5 blocks
==1== suppressed: 0 bytes in 0 blocks
==1== Reachable blocks (those to which a pointer was found) are not shown.
==1== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==1==
==1== For counts of detected and suppressed errors, rerun with: -v
==1== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Fixes:
==1== by 0x23E44C: remove_marked_symlinks_fd (install.c:453)
==1== by 0x23E256: remove_marked_symlinks_fd (install.c:405)
==1== by 0x23E630: remove_marked_symlinks (install.c:494)
==1== by 0x2427A0: unit_file_disable (install.c:1876)
==1== by 0x18A633: method_disable_unit_files_generic (dbus-manager.c:1760)
==1== by 0x18A6CA: method_disable_unit_files (dbus-manager.c:1768)
==1== by 0x1D8146: method_callbacks_run (bus-objects.c:420)
==1== by 0x1DA9D8: object_find_and_run (bus-objects.c:1257)
==1== by 0x1DB01A: bus_process_object (bus-objects.c:1373)
==1==
==1== 228 (48 direct, 180 indirect) bytes in 2 blocks are definitely lost in loss record 8 of 14
==1== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1== by 0x4C2DE2F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1== by 0x23DA60: unit_file_changes_add (install.c:233)
==1== by 0x23DDB2: create_symlink (install.c:298)
==1== by 0x240C5C: install_info_symlink_wants (install.c:1328)
==1== by 0x240FC8: install_info_apply (install.c:1384)
==1== by 0x241211: install_context_apply (install.c:1439)
==1== by 0x242563: unit_file_enable (install.c:1830)
==1== by 0x18A06E: method_enable_unit_files_generic (dbus-manager.c:1650)
==1== by 0x18A141: method_enable_unit_files (dbus-manager.c:1660)
==1== by 0x1D8146: method_callbacks_run (bus-objects.c:420)
==1== by 0x1DA9D8: object_find_and_run (bus-objects.c:1257)
==1==
==1== 467 (144 direct, 323 indirect) bytes in 3 blocks are definitely lost in loss record 9 of 14
==1== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1== by 0x23DA60: unit_file_changes_add (install.c:233)
==1== by 0x23DE97: create_symlink (install.c:320)
==1== by 0x242CFC: unit_file_set_default (install.c:1951)
==1== by 0x18A881: method_set_default_target (dbus-manager.c:1802)
==1== by 0x1D8146: method_callbacks_run (bus-objects.c:420)
==1== by 0x1DA9D8: object_find_and_run (bus-objects.c:1257)
==1== by 0x1DB01A: bus_process_object (bus-objects.c:1373)
==1== by 0x259143: process_message (sd-bus.c:2567)
==1== by 0x259326: process_running (sd-bus.c:2609)
==1== by 0x259BDC: bus_process_internal (sd-bus.c:2798)
==1== by 0x259CAD: sd_bus_process (sd-bus.c:2817)
==1==
==1== LEAK SUMMARY:
==1== definitely lost: 216 bytes in 6 blocks
==1== indirectly lost: 560 bytes in 14 blocks
==1== possibly lost: 0 bytes in 0 blocks
==1== still reachable: 65,536 bytes in 5 blocks
==1== suppressed: 0 bytes in 0 blocks
==1== Reachable blocks (those to which a pointer was found) are not shown.
==1== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==1==
Output the same message when a request to change the log level is
received over dbus and through a signal. From the user point of view
those two operations are very similar and it's easy to think that the
dbus operation didn't work when the expected message is not emitted.
Also "downgrade" the message level to info, since this is a normal
user initiated action.
GLIB has recently started to officially support the gcc cleanup
attribute in its public API, hence let's do the same for our APIs.
With this patch we'll define an xyz_unrefp() call for each public
xyz_unref() call, to make it easy to use inside a
__attribute__((cleanup())) expression. Then, all code is ported over to
make use of this.
The new calls are also documented in the man pages, with examples how to
use them (well, I only added docs where the _unref() call itself already
had docs, and the examples, only cover sd_bus_unrefp() and
sd_event_unrefp()).
This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we
tend to call our destructors these days.
Note that this defines no public macro that wraps gcc's attribute and
makes it easier to use. While I think it's our duty in the library to
make our stuff easy to use, I figure it's not our duty to make gcc's own
features easy to use on its own. Most likely, client code which wants to
make use of this should define its own:
#define _cleanup_(function) __attribute__((cleanup(function)))
Or similar, to make the gcc feature easier to use.
Making this logic public has the benefit that we can remove three header
files whose only purpose was to define these functions internally.
See #2008.
'set-property' has been primarly designed to change some properties of
*active* units.
However it can easily work on inactive units as well. In that case
changes are only saved in a drop-in for futur uses and changes will be
effective when unit will be started.
Actually it already works on inactive units but that was not
documented and not fully supported. Indeed the inactive units had to
be known by the manager otherwise it was reported as not loaded:
$ systemctl status my-test.service
* my-test.service - My Testing Unit
Loaded: loaded (/etc/systemd/system/my-test.service; static; vendor preset: disabled)
Drop-In: /etc/systemd/system/my-test.service.d
Active: inactive (dead)
$ systemctl set-property my-test.service MemoryLimit=1000000
Failed to set unit properties on my-test.service: Unit my-test.service is not loaded.
[ Note: that the unit load state reported by the 'status' command
might be confusing since it claimed the unit as loaded but
'set-property' reported the contrary. ]
One can possibily workaround this by making the unit a dependency of
another active unit so the manager will keep it around:
$ systemctl add-wants multi-user.target my-test.service
Created symlink from /etc/systemd/system/multi-user.target.wants/my-test.service to /etc/systemd/system/my-test.service.
$ systemctl set-property my-test.service MemoryLimit=1000000
$ systemctl status my-test.service
* my-test.service - My Testing Unit
Loaded: loaded (/etc/systemd/system/my-test.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/my-test.service.d
`-50-MemoryLimit.conf
Active: inactive (dead)
This patch simply forces 'SetUnitProperties()' to load the unit if
it's not already the case.
It also documents the fact that 'set-property' can be used on inactive
units.
manager_load_unit() will dispatch the load queue anyway, but let's make
sure we also dispatch it immediately, after truning a unit into a
transient one and loading the properties from the message. That way the
know about the validity of the unit before we begin processing the next
auxiliary unit.
Lets introduce unit_is_pristine() that verifies whether a unit is
suitable to become a transient unit, by checking that it is no
referenced yet and has no data on disk assigned.
Let's move the validation checks into the loop that sets up the main and
auxiliary transient units, so that we can generate pretty error messages
for all units a transient unit transaction generates, not just for the
main unit.
Some distributions use alias unit files via symlinks in /usr to cover
for legacy service names. With this change we'll allow "systemctl
enable" on such aliases.
Previously, our rule was that symlinks are user configuration that
"systemctl enable" + "systemctl disable" creates and removes, while unit
files is where the instructions to do so are store. As a result of the
rule we'd never read install information through symlinks, since that
would mix enablement state with installation instructions.
Now, the new rule is that only symlinks inside of /etc are
configuration. Unit files, and symlinks in /usr are now valid for
installation instructions.
This patch is quite a rework of the whole install logic, and makes the
following addional changes:
- Adds a complete test "test-instal-root" that tests the install logic
pretty comprehensively.
- Never uses canonicalize_file_name(), because that's incompatible with
operation relative to a specific root directory.
- unit_file_get_state() is reworked to return a proper error, and
returns the state in a call-by-ref parameter. This cleans up confusion
between the enum type and errno-like errors.
- The new logic puts a limit on how long to follow unit file symlinks:
it will do so only for 64 steps at max.
- The InstallContext object's fields are renamed to will_process and
has_processed (will_install and has_installed) since they are also
used for deinstallation and all kinds of other operations.
- The root directory is always verified before use.
- install.c is reordered to place the exported functions together.
- Stricter rules are followed when traversing symlinks: the unit suffix
must say identical, and it's not allowed to link between regular units
and templated units.
- Various modernizations
- The "invalid" unit file state has been renamed to "bad", in order to
avoid confusion between UNIT_FILE_INVALID and
_UNIT_FILE_STATE_INVALID. Given that the state should normally not be
seen and is not documented this should not be a problematic change.
The new name is now documented however.
Fixes#1375, #1718, #1706
Instead, let the caller do that. Fix this by moving masked unit messages
into the caller, by returning a clear error code (ESHUTDOWN) by which
this may be detected.
Snapshots were never useful or used for anything. Many systemd
developers that I spoke to at systemd.conf2015, didn't even know they
existed, so it is fairly safe to assume that this type can be deleted
without harm.
The fundamental problem with snapshots is that the state of the system
is dynamic, devices come and go, users log in and out, timers fire...
and restoring all units to some state from the past would "undo"
those changes, which isn't really possible.
Tested by creating a snapshot, running the new binary, and checking
that the transition did not cause errors, and the snapshot is gone,
and snapshots cannot be created anymore.
New systemctl says:
Unknown operation snapshot.
Old systemctl says:
Failed to create snapshot: Support for snapshots has been removed.
IgnoreOnSnaphost settings are warned about and ignored:
Support for option IgnoreOnSnapshot= has been removed and it is ignored
http://lists.freedesktop.org/archives/systemd-devel/2015-November/034872.html
There are more than enough calls doing string manipulations to deserve
its own files, hence do something about it.
This patch also sorts the #include blocks of all files that needed to be
updated, according to the sorting suggestions from CODING_STYLE. Since
pretty much every file needs our string manipulation functions this
effectively means that most files have sorted #include blocks now.
Also touches a few unrelated include files.
We already stop boot if /etc/mtab is not a symlink right now, and most
likely we'll stop referecing it at all in the future, either way there's
no point in keeping it around as taint flag.
Lennart Poettering