Merge pull request #7314 from poettering/gc-mode

add new CollectMode= unit file setting for tweaking the unit garbage collection logic
This commit is contained in:
Lennart Poettering 2017-11-16 16:32:36 +01:00 committed by GitHub
commit 83123ab8fd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 186 additions and 28 deletions

9
TODO
View File

@ -38,11 +38,6 @@ Features:
* maybe use SOURCE_DATE_EPOCH (i.e. the env var the reproducible builds folks
introduced) as the RTC epoch, instead of the mtime of NEWS.
* Introduce GCMode= as unit file property or so, for tweaking the GC
logic. Specifically, there should be a way to tell systemd to collect
individual units even on failure. Then, make systemd-run --wait use this, so
that failed transient units in that case don't stick around.
* add a way to lock down cgroup migration: a boolean, which when set for a unit
makes sure the processes in it can never migrate out of it
@ -87,6 +82,10 @@ Features:
friendly log messages for the returned errors, so that we don't have to
duplicate that in nspawn, systemd-dissect and PID 1.
* add "systemctl wait" or so, which does what "systemd-run --wait" does, but
for all units. It should be both a way to pin units into memory as well as a
wait to retrieve their exit data.
* maybe set a new set of env vars for services, based on RuntimeDirectory=,
StateDirectory=, LogsDirectory=, CacheDirectory= and ConfigurationDirectory=
automatically. For example, there could be $RUNTIME_DIRECTORY,

View File

@ -324,6 +324,21 @@
<option>--no-block</option>, <option>--scope</option> or the various timer options.</para></listitem>
</varlistentry>
<varlistentry>
<term><option>-G</option></term>
<term><option>--collect</option></term>
<listitem><para>Unload the transient unit after it completed, even if it failed. Normally, without this option,
all units that ran and failed are kept in memory until the user explicitly resets their failure state with
<command>systemctl reset-failed</command> or an equivalent command. On the other hand, units that ran
successfully are unloaded immediately. If this option is turned on the "garbage collection" of units is more
aggressive, and unloads units regardless if they exited successfully or failed. This option is a shortcut for
<command>--property=CollectMode=inactive-or-failed</command>, see the explanation for
<varname>CollectMode=</varname> in
<citerefentry><refentrytitle>systemd.unit</refentrytitle><manvolnum>5</manvolnum></citerefentry> for further
information.</para></listitem>
</varlistentry>
<xi:include href="user-system-options.xml" xpointer="user" />
<xi:include href="user-system-options.xml" xpointer="system" />
<xi:include href="user-system-options.xml" xpointer="host" />

View File

@ -213,7 +213,6 @@
socket-based activation which make dependencies implicit,
resulting in a both simpler and more flexible system.</para>
<para>Optionally, units may be instantiated from a
template file at runtime. This allows creation of
multiple units from a single configuration file. If
@ -417,6 +416,45 @@
</para>
</refsect1>
<refsect1>
<title>Unit Garbage Collection</title>
<para>The system and service manager loads a unit's configuration automatically when a unit is referenced for the
first time. It will automatically unload the unit configuration and state again when the unit is not needed anymore
("garbage collection"). A unit may be referenced through a number of different mechanisms:</para>
<orderedlist>
<listitem><para>Another loaded unit references it with a dependency such as <varname>After=</varname>,
<varname>Wants=</varname>, …</para></listitem>
<listitem><para>The unit is currently starting, running, reloading or stopping.</para></listitem>
<listitem><para>The unit is currently in the <constant>failed</constant> state. (But see below.)</para></listitem>
<listitem><para>A job for the unit is pending.</para></listitem>
<listitem><para>The unit is pinned by an active IPC client program.</para></listitem>
<listitem><para>The unit is a special "perpetual" unit that is always active and loaded. Examples for perpetual
units are the root mount unit <filename>-.mount</filename> or the scope unit <filename>init.scope</filename> that
the service manager itself lives in.</para></listitem>
<listitem><para>The unit has running processes associated with it.</para></listitem>
</orderedlist>
<para>The garbage collection logic may be altered with the <varname>CollectMode=</varname> option, which allows
configuration whether automatic unloading of units that are in <constant>failed</constant> state is permissible,
see below.</para>
<para>Note that when a unit's configuration and state is unloaded, all execution results, such as exit codes, exit
signals, resource consumption and other statistics are lost, except for what is stored in the log subsystem.</para>
<para>Use <command>systemctl daemon-reload</command> or an equivalent command to reload unit configuration while
the unit is already loaded. In this case all configuration settings are flushed out and replaced with the new
configuration (which however might not be in effect immediately), however all runtime state is
saved/restored.</para>
</refsect1>
<refsect1>
<title>[Unit] Section Options</title>
@ -747,6 +785,23 @@
ones.</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>CollectMode=</varname></term>
<listitem><para>Tweaks the "garbage collection" algorithm for this unit. Takes one of <option>inactive</option>
or <option>inactive-or-failed</option>. If set to <option>inactive</option> the unit will be unloaded if it is
in the <constant>inactive</constant> state and is not referenced by clients, jobs or other units — however it
is not unloaded if it is in the <constant>failed</constant> state. In <option>failed</option> mode, failed
units are not unloaded until the user invoked <command>systemctl reset-failed</command> on them to reset the
<constant>failed</constant> state, or an equivalent command. This behaviour is altered if this option is set to
<option>inactive-or-failed</option>: in this case the unit is unloaded even if the unit is in a
<constant>failed</constant> state, and thus an explicitly resetting of the <constant>failed</constant> state is
not necessary. Note that if this mode is used unit results (such as exit codes, exit signals, consumed
resources, …) are flushed out immediately after the unit completed, except for what is stored in the logging
subsystem. Defaults to <option>inactive</option>.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><varname>JobTimeoutSec=</varname></term>
<term><varname>JobRunningTimeoutSec=</varname></term>

View File

@ -37,6 +37,7 @@
#include "strv.h"
#include "user-util.h"
static BUS_DEFINE_PROPERTY_GET_ENUM(property_get_collect_mode, collect_mode, CollectMode);
static BUS_DEFINE_PROPERTY_GET_ENUM(property_get_load_state, unit_load_state, UnitLoadState);
static BUS_DEFINE_PROPERTY_GET_ENUM(property_get_job_mode, job_mode, JobMode);
static BUS_DEFINE_PROPERTY_GET_ENUM(property_get_emergency_action, emergency_action, EmergencyAction);
@ -798,6 +799,7 @@ const sd_bus_vtable bus_unit_vtable[] = {
SD_BUS_PROPERTY("StartLimitAction", "s", property_get_emergency_action, offsetof(Unit, start_limit_action), SD_BUS_VTABLE_PROPERTY_CONST),
SD_BUS_PROPERTY("RebootArgument", "s", NULL, offsetof(Unit, reboot_arg), SD_BUS_VTABLE_PROPERTY_CONST),
SD_BUS_PROPERTY("InvocationID", "ay", bus_property_get_id128, offsetof(Unit, invocation_id), 0),
SD_BUS_PROPERTY("CollectMode", "s", property_get_collect_mode, offsetof(Unit, collect_mode), 0),
SD_BUS_METHOD("Start", "s", "o", method_start, SD_BUS_VTABLE_UNPRIVILEGED),
SD_BUS_METHOD("Stop", "s", "o", method_stop, SD_BUS_VTABLE_UNPRIVILEGED),
@ -1354,6 +1356,25 @@ static int bus_unit_set_transient_property(
return 1;
} else if (streq(name, "CollectMode")) {
const char *s;
CollectMode m;
r = sd_bus_message_read(message, "s", &s);
if (r < 0)
return r;
m = collect_mode_from_string(s);
if (m < 0)
return sd_bus_error_setf(error, SD_BUS_ERROR_INVALID_ARGS, "Unknown garbage collection mode: %s", s);
if (mode != UNIT_CHECK) {
u->collect_mode = m;
unit_write_drop_in_format(u, mode, name, "[Unit]\nCollectMode=%s", collect_mode_to_string(m));
}
return 1;
} else if (streq(name, "Slice")) {
Unit *slice;
const char *s;

View File

@ -264,6 +264,7 @@ Unit.AssertACPower, config_parse_unit_condition_string, CONDITION_A
Unit.AssertUser, config_parse_unit_condition_string, CONDITION_USER, offsetof(Unit, asserts)
Unit.AssertGroup, config_parse_unit_condition_string, CONDITION_GROUP, offsetof(Unit, asserts)
Unit.AssertNull, config_parse_unit_condition_null, 0, offsetof(Unit, asserts)
Unit.CollectMode, config_parse_collect_mode, 0, offsetof(Unit, collect_mode)
m4_dnl
Service.PIDFile, config_parse_unit_path_printf, 0, offsetof(Service, pid_file)
Service.ExecStartPre, config_parse_exec, SERVICE_EXEC_START_PRE, offsetof(Service, exec_command)

View File

@ -103,6 +103,8 @@ int config_parse_warn_compat(
return 0;
}
DEFINE_CONFIG_PARSE_ENUM(config_parse_collect_mode, collect_mode, CollectMode, "Failed to parse garbage collection mode");
int config_parse_unit_deps(
const char *unit,
const char *filename,

View File

@ -122,6 +122,7 @@ int config_parse_exec_keyring_mode(const char *unit, const char *filename, unsig
int config_parse_job_timeout_sec(const char *unit, const char *filename, unsigned line, const char *section, unsigned section_line, const char *lvalue, int ltype, const char *rvalue, void *data, void *userdata);
int config_parse_job_running_timeout_sec(const char *unit, const char *filename, unsigned line, const char *section, unsigned section_line, const char *lvalue, int ltype, const char *rvalue, void *data, void *userdata);
int config_parse_log_extra_fields(const char *unit, const char *filename, unsigned line, const char *section, unsigned section_line, const char *lvalue, int ltype, const char *rvalue, void *data, void *userdata);
int config_parse_collect_mode(const char *unit, const char *filename, unsigned line, const char *section, unsigned section_line, const char *lvalue, int ltype, const char *rvalue, void *data, void *userdata);
/* gperf prototypes */
const struct ConfigPerfItem* load_fragment_gperf_lookup(const char *key, GPERF_LEN_TYPE length);

View File

@ -294,6 +294,9 @@ static void service_fd_store_unlink(ServiceFDStore *fs) {
static void service_release_fd_store(Service *s) {
assert(s);
if (s->n_keep_fd_store > 0)
return;
log_unit_debug(UNIT(s), "Releasing all stored fds");
while (s->fd_store)
service_fd_store_unlink(s->fd_store);
@ -301,7 +304,7 @@ static void service_release_fd_store(Service *s) {
assert(s->n_fd_store == 0);
}
static void service_release_resources(Unit *u, bool inactive) {
static void service_release_resources(Unit *u) {
Service *s = SERVICE(u);
assert(s);
@ -315,8 +318,7 @@ static void service_release_resources(Unit *u, bool inactive) {
s->stdout_fd = safe_close(s->stdout_fd);
s->stderr_fd = safe_close(s->stderr_fd);
if (inactive)
service_release_fd_store(s);
service_release_fd_store(s);
}
static void service_done(Unit *u) {
@ -360,7 +362,7 @@ static void service_done(Unit *u) {
s->timer_event_source = sd_event_source_unref(s->timer_event_source);
service_release_resources(u, true);
service_release_resources(u);
}
static int on_fd_store_io(sd_event_source *e, int fd, uint32_t revents, void *userdata) {
@ -1524,6 +1526,10 @@ static void service_enter_dead(Service *s, ServiceResult f, bool allow_restart)
if (s->result != SERVICE_SUCCESS)
log_unit_warning(UNIT(s), "Failed with result '%s'.", service_result_to_string(s->result));
/* Make sure service_release_resources() doesn't destroy our FD store, while we are changing through
* SERVICE_FAILED/SERVICE_DEAD before entering into SERVICE_AUTO_RESTART. */
s->n_keep_fd_store ++;
service_set_state(s, s->result != SERVICE_SUCCESS ? SERVICE_FAILED : SERVICE_DEAD);
if (s->result != SERVICE_SUCCESS)
@ -1532,8 +1538,10 @@ static void service_enter_dead(Service *s, ServiceResult f, bool allow_restart)
if (allow_restart && service_shall_restart(s)) {
r = service_arm_timer(s, usec_add(now(CLOCK_MONOTONIC), s->restart_usec));
if (r < 0)
if (r < 0) {
s->n_keep_fd_store--;
goto fail;
}
service_set_state(s, SERVICE_AUTO_RESTART);
} else
@ -1541,6 +1549,11 @@ static void service_enter_dead(Service *s, ServiceResult f, bool allow_restart)
* user can still introspect the counter. Do so on the next start. */
s->flush_n_restarts = true;
/* The new state is in effect, let's decrease the fd store ref counter again. Let's also readd us to the GC
* queue, so that the fd store is possibly gc'ed again */
s->n_keep_fd_store--;
unit_add_to_gc_queue(UNIT(s));
/* The next restart might not be a manual stop, hence reset the flag indicating manual stops */
s->forbid_restart = false;

View File

@ -186,6 +186,7 @@ struct Service {
ServiceFDStore *fd_store;
unsigned n_fd_store;
unsigned n_fd_store_max;
unsigned n_keep_fd_store;
char *usb_function_descriptors;
char *usb_function_strings;

View File

@ -56,6 +56,7 @@
#include "special.h"
#include "stat-util.h"
#include "stdio-util.h"
#include "string-table.h"
#include "string-util.h"
#include "strv.h"
#include "umask-util.h"
@ -75,7 +76,7 @@ const UnitVTable * const unit_vtable[_UNIT_TYPE_MAX] = {
[UNIT_TIMER] = &timer_vtable,
[UNIT_PATH] = &path_vtable,
[UNIT_SLICE] = &slice_vtable,
[UNIT_SCOPE] = &scope_vtable
[UNIT_SCOPE] = &scope_vtable,
};
static void maybe_warn_about_dependency(Unit *u, const char *other, UnitDependency dependency);
@ -331,9 +332,12 @@ int unit_set_description(Unit *u, const char *description) {
bool unit_check_gc(Unit *u) {
UnitActiveState state;
bool inactive;
assert(u);
/* Checks whether the unit is ready to be unloaded for garbage collection. Returns true, when the unit shall
* stay around, false if there's no reason to keep it loaded. */
if (u->job)
return true;
@ -341,18 +345,11 @@ bool unit_check_gc(Unit *u) {
return true;
state = unit_active_state(u);
inactive = state == UNIT_INACTIVE;
/* If the unit is inactive and failed and no job is queued for
* it, then release its runtime resources */
/* If the unit is inactive and failed and no job is queued for it, then release its runtime resources */
if (UNIT_IS_INACTIVE_OR_FAILED(state) &&
UNIT_VTABLE(u)->release_resources)
UNIT_VTABLE(u)->release_resources(u, inactive);
/* But we keep the unit object around for longer when it is
* referenced or configured to not be gc'ed */
if (!inactive)
return true;
UNIT_VTABLE(u)->release_resources(u);
if (u->perpetual)
return true;
@ -363,6 +360,25 @@ bool unit_check_gc(Unit *u) {
if (sd_bus_track_count(u->bus_track) > 0)
return true;
/* But we keep the unit object around for longer when it is referenced or configured to not be gc'ed */
switch (u->collect_mode) {
case COLLECT_INACTIVE:
if (state != UNIT_INACTIVE)
return true;
break;
case COLLECT_INACTIVE_OR_FAILED:
if (!IN_SET(state, UNIT_INACTIVE, UNIT_FAILED))
return true;
break;
default:
assert_not_reached("Unknown garbage collection mode");
}
if (UNIT_VTABLE(u)->check_gc)
if (UNIT_VTABLE(u)->check_gc(u))
return true;
@ -1087,6 +1103,7 @@ void unit_dump(Unit *u, FILE *f, const char *prefix) {
"%s\tNeed Daemon Reload: %s\n"
"%s\tTransient: %s\n"
"%s\tPerpetual: %s\n"
"%s\tGarbage Collection Mode: %s\n"
"%s\tSlice: %s\n"
"%s\tCGroup: %s\n"
"%s\tCGroup realized: %s\n",
@ -1104,6 +1121,7 @@ void unit_dump(Unit *u, FILE *f, const char *prefix) {
prefix, yes_no(unit_need_daemon_reload(u)),
prefix, yes_no(u->transient),
prefix, yes_no(u->perpetual),
prefix, collect_mode_to_string(u->collect_mode),
prefix, strna(unit_slice_name(u)),
prefix, strna(u->cgroup_path),
prefix, yes_no(u->cgroup_realized));
@ -5125,3 +5143,10 @@ void unit_unlink_state_files(Unit *u) {
u->exported_log_extra_fields = false;
}
}
static const char* const collect_mode_table[_COLLECT_MODE_MAX] = {
[COLLECT_INACTIVE] = "inactive",
[COLLECT_INACTIVE_OR_FAILED] = "inactive-or-failed",
};
DEFINE_STRING_TABLE_LOOKUP(collect_mode, CollectMode);

View File

@ -45,6 +45,13 @@ typedef enum KillOperation {
_KILL_OPERATION_INVALID = -1
} KillOperation;
typedef enum CollectMode {
COLLECT_INACTIVE,
COLLECT_INACTIVE_OR_FAILED,
_COLLECT_MODE_MAX,
_COLLECT_MODE_INVALID = -1,
} CollectMode;
static inline bool UNIT_IS_ACTIVE_OR_RELOADING(UnitActiveState t) {
return IN_SET(t, UNIT_ACTIVE, UNIT_RELOADING);
}
@ -282,6 +289,9 @@ struct Unit {
/* How to start OnFailure units */
JobMode on_failure_job_mode;
/* Tweaking the GC logic */
CollectMode collect_mode;
/* The current invocation ID */
sd_id128_t invocation_id;
char invocation_id_string[SD_ID128_STRING_MAX]; /* useful when logging */
@ -453,9 +463,8 @@ struct UnitVTable {
* way */
bool (*check_gc)(Unit *u);
/* When the unit is not running and no job for it queued we
* shall release its runtime resources */
void (*release_resources)(Unit *u, bool inactive);
/* When the unit is not running and no job for it queued we shall release its runtime resources */
void (*release_resources)(Unit *u);
/* Invoked on every child that died */
void (*sigchld_event)(Unit *u, pid_t pid, int code, int status);
@ -774,3 +783,6 @@ void unit_unlink_state_files(Unit *u);
#define LOG_UNIT_MESSAGE(unit, fmt, ...) "MESSAGE=%s: " fmt, (unit)->id, ##__VA_ARGS__
#define LOG_UNIT_ID(unit) (unit)->manager->unit_log_format_string, (unit)->id
#define LOG_UNIT_INVOCATION_ID(unit) (unit)->manager->invocation_log_format_string, (unit)->invocation_id_string
const char* collect_mode_to_string(CollectMode m) _const_;
CollectMode collect_mode_from_string(const char *s) _pure_;

View File

@ -75,6 +75,7 @@ static usec_t arg_on_unit_inactive = 0;
static const char *arg_on_calendar = NULL;
static char **arg_timer_property = NULL;
static bool arg_quiet = false;
static bool arg_aggressive_gc = false;
static void help(void) {
printf("%s [OPTIONS...] {COMMAND} [ARGS...]\n\n"
@ -102,7 +103,8 @@ static void help(void) {
" -t --pty Run service on pseudo TTY as STDIN/STDOUT/\n"
" STDERR\n"
" -P --pipe Pass STDIN/STDOUT/STDERR directly to service\n"
" -q --quiet Suppress information messages during runtime\n\n"
" -q --quiet Suppress information messages during runtime\n"
" -G --collect Unload unit after it ran, even when failed\n\n"
"Timer options:\n"
" --on-active=SECONDS Run after SECONDS delay\n"
" --on-boot=SECONDS Run SECONDS after machine was booted up\n"
@ -178,6 +180,7 @@ static int parse_argv(int argc, char *argv[]) {
{ "timer-property", required_argument, NULL, ARG_TIMER_PROPERTY },
{ "no-block", no_argument, NULL, ARG_NO_BLOCK },
{ "no-ask-password", no_argument, NULL, ARG_NO_ASK_PASSWORD },
{ "collect", no_argument, NULL, 'G' },
{},
};
@ -186,7 +189,7 @@ static int parse_argv(int argc, char *argv[]) {
assert(argc >= 0);
assert(argv);
while ((c = getopt_long(argc, argv, "+hrH:M:E:p:tPq", options, NULL)) >= 0)
while ((c = getopt_long(argc, argv, "+hrH:M:E:p:tPqG", options, NULL)) >= 0)
switch (c) {
@ -372,6 +375,10 @@ static int parse_argv(int argc, char *argv[]) {
arg_wait = true;
break;
case 'G':
arg_aggressive_gc = true;
break;
case '?':
return -EINVAL;
@ -461,6 +468,12 @@ static int transient_unit_set_properties(sd_bus_message *m, char **properties) {
if (r < 0)
return r;
if (arg_aggressive_gc) {
r = sd_bus_message_append(m, "(sv)", "CollectMode", "s", "inactive-or-failed");
if (r < 0)
return r;
}
r = bus_append_unit_property_assignment_many(m, properties);
if (r < 0)
return r;

View File

@ -371,7 +371,7 @@ int bus_append_unit_property_assignment(sd_bus_message *m, const char *assignmen
"RootDirectory", "SyslogIdentifier", "ProtectSystem",
"ProtectHome", "SELinuxContext", "Restart", "RootImage",
"NotifyAccess", "RuntimeDirectoryPreserve", "Personality",
"KeyringMode"))
"KeyringMode", "CollectMode"))
r = sd_bus_message_append(m, "v", "s", eq);
else if (STR_IN_SET(field, "AppArmorProfile", "SmackProcessLabel")) {