Let's avoid the overly abbreviated "cgroups" terminology. Let's instead write:
"Linux Control Groups (cgroups)" is the long form wherever the term is
introduced in prose. Use "control groups" in the short form wherever the term
is used within brief explanations.
Follow-up to: #4381
There are overlapping control group resource settings for the unified and
legacy hierarchies. To help transition, the settings are translated back and
forth. When both versions of a given setting are present, the one matching the
cgroup hierarchy type in use is used. Unfortunately, this is more confusing to
use and document than necessary because there is no clear static precedence.
Update the translation logic so that the settings for the unified hierarchy are
always preferred. systemd.resource-control man page is updated to reflect the
change and reorganized so that the deprecated settings are at the end in its
own section.
Unfortunately, due to the disagreements in the kernel development community,
CPU controller cgroup v2 support has not been merged and enabling it requires
applying two small out-of-tree kernel patches. The situation is explained in
the following documentation.
https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/tree/Documentation/cgroup-v2-cpu.txt?h=cgroup-v2-cpu
While it isn't clear what will happen with CPU controller cgroup v2 support,
there are critical features which are possible only on cgroup v2 such as
buffered write control making cgroup v2 essential for a lot of workloads. This
commit implements systemd CPU controller support on the unified hierarchy so
that users who choose to deploy CPU controller cgroup v2 support can easily
take advantage of it.
On the unified hierarchy, "cpu.weight" knob replaces "cpu.shares" and "cpu.max"
replaces "cpu.cfs_period_us" and "cpu.cfs_quota_us". [Startup]CPUWeight config
options are added with the usual compat translation. CPU quota settings remain
unchanged and apply to both legacy and unified hierarchies.
v2: - Error in man page corrected.
- CPU config application in cgroup_context_apply() refactored.
- CPU accounting now works on unified hierarchy.
This adds support for a TasksMax=40% syntax for specifying values relative to
the system's configured maximum number of processes. This is useful in order to
neatly subdivide the available room for tasks within containers.
If a percentage is used, it is taken relative to the installed RAM size. This
should make it easier to write generic unit files that adapt to the local system.
Recently added cgroup unified hierarchy support uses "max" in configurations
for no upper limit. While consistent with what the kernel uses for no upper
limit, it is inconsistent with what systemd uses for other controllers such as
memory or pids. There's no point in introducing another term. Update cgroup
unified hierarchy support so that "infinity" is the only term that systemd
uses for no upper limit.
On the unified hierarchy, memory controller implements three control knobs -
low, high and max which enables more useable and versatile control over memory
usage. This patch implements support for the three control knobs.
* MemoryLow, MemoryHigh and MemoryMax are added for memory.low, memory.high and
memory.max, respectively.
* As all absolute limits on the unified hierarchy use "max" for no limit, make
memory limit parse functions accept "max" in addition to "infinity" and
document "max" for the new knobs.
* Implement compatibility translation between MemoryMax and MemoryLimit.
v2:
- Fixed missing else's in config_parse_memory_limit().
- Fixed missing newline when writing out drop-ins.
- Coding style updates to use "val > 0" instead of "val".
- Minor updates to documentation.
Due to the substantial interface changes in cgroup unified hierarchy, new IO
settings are introduced. Currently, IO settings apply only to unified
hierarchy and BlockIO to legacy. While the transition is necessary, it's
painful for users to have to provide configs for both. This patch implements
translation from one config set to another for configs which make sense.
* The translation takes place during application of the configs. Users won't
see IO or BlockIO settings appearing without being explicitly created.
* The translation takes place only if there is no config for the matching
cgroup hierarchy type at all.
While this doesn't provide comprehensive compatibility, it should considerably
ease transition to the new IO settings which are a superset of BlockIO
settings.
v2:
- Update test-cgroup-mask.c so that it accounts for the fact that
CGROUP_MASK_IO and CGROUP_MASK_BLKIO move together. Also, test/parent.slice
now sets IOWeight instead of BlockIOWeight.
cgroup IO controller supports maximum limits for both bandwidth and IOPS but
systemd resource control currently only supports bandwidth limits. This patch
adds support for IOReadIOPSMax and IOWriteIOPSMax when unified cgroup hierarchy
is in use.
It isn't difficult to also add BlockIOReadIOPS and BlockIOWriteIOPS for legacy
hierarchies but IO control on legacy hierarchies is half-broken anyway, so
let's leave it alone for now.
With this change for each setting we say which hierarachy it applies to briefly
in the first sentence of the description, plus in longer form in an extra
pargraph at the end, with a recommendation for the counterpart of the option in
the other hierarchy.
Also adds markup and the "=" suffix to all mentioned settings.
On the unified hierarchy, blkio controller is renamed to io and the interface
is changed significantly.
* blkio.weight and blkio.weight_device are consolidated into io.weight which
uses the standardized weight range [1, 10000] with 100 as the default value.
* blkio.throttle.{read|write}_{bps|iops}_device are consolidated into io.max.
Expansion of throttling features is being worked on to support
work-conserving absolute limits (io.low and io.high).
* All stats are consolidated into io.stats.
This patchset adds support for the new interface. As the interface has been
revamped and new features are expected to be added, it seems best to treat it
as a separate controller rather than trying to expand the blkio settings
although we might add automatic translation if only blkio settings are
specified.
* io.weight handling is mostly identical to blkio.weight[_device] handling
except that the weight range is different.
* Both read and write bandwidth settings are consolidated into
CGroupIODeviceLimit which describes all limits applicable to the device.
This makes it less painful to add new limits.
* "max" can be used to specify the maximum limit which is equivalent to no
config for max limits and treated as such. If a given CGroupIODeviceLimit
doesn't contain any non-default configs, the config struct is discarded once
the no limit config is applied to cgroup.
* lookup_blkio_device() is renamed to lookup_block_device().
Signed-off-by: Tejun Heo <htejun@fb.com>
Support for net_cls.class_id through the NetClass= configuration directive
has been added in v227 in preparation for a per-unit packet filter mechanism.
However, it turns out the kernel people have decided to deprecate the net_cls
and net_prio controllers in v2. Tejun provides a comprehensive justification
for this in his commit, which has landed during the merge window for kernel
v4.5:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=bd1060a1d671
As we're aiming for full support for the v2 cgroup hierarchy, we can no
longer support this feature. Userspace tool such as nftables are moving over
to setting rules that are specific to the full cgroup path of a task, which
obsoletes these controllers anyway.
This commit removes support for tweaking details in the net_cls controller,
but keeps the NetClass= directive around for legacy compatibility reasons.
For all units ensure there's an "Automatic Dependencies" section in the
man page, and explain which dependencies are automatically added in all
cases, and which ones are added on top if DefaultDependencies=yes is
set.
This is also done for systemd.exec(5), systemd.resource-control(5) and
systemd.unit(5) as these pages describe common behaviour of various unit
types.
Let's stop using the "unsigned long" type for weights/shares, and let's
just use uint64_t for this, as that's what we expose on the bus.
Unify parsers, and always validate the range for these fields.
Correct the default blockio weight to 500, since that's what the kernel
actually uses.
When parsing the weight/shares settings from unit files accept the empty
string as a way to reset the weight/shares value. When getting it via
the bus, uniformly map (uint64_t) -1 to unset.
Open up StartupCPUShares= and StartupBlockIOWeight= to transient units.
This adds support for the new "pids" cgroup controller of 4.3 kernels.
It allows accounting the number of tasks in a cgroup and enforcing
limits on it.
This adds two new setting TasksAccounting= and TasksMax= to each unit,
as well as a gloabl option DefaultTasksAccounting=.
This also updated "cgtop" to optionally make use of the new
kernel-provided accounting.
systemctl has been updated to show the number of tasks for each service
if it is available.
This patch also adds correct support for undoing memory limits for units
using a MemoryLimit=infinity syntax. We do the same for TasksMax= now
and hence keep things in sync here.
This did not really work out as we had hoped. Trying to do this upstream
introduced several problems that probably makes it better suited as a
downstream patch after all. At any rate, it is not releaseable in the
current state, so we at least need to revert this before the release.
* by adjusting the path to binaries, but not do the same thing to the
search path we end up with inconsistent man-pages. Adjusting the search
path too would be quite messy, and it is not at all obvious that this is
worth the effort, but at any rate it would have to be done before we
could ship this.
* this means that distributed man-pages does not make sense as they depend
on config options, and for better or worse we are still distributing
man pages, so that is something that definitely needs sorting out before
we could ship with this patch.
* we have long held that split-usr is only minimally supported in order
to boot, and something we hope will eventually go away. So before we start
adding even more magic/effort in order to make this work nicely, we should
probably question if it makes sense at all.
In particular, use /lib/systemd instead of /usr/lib/systemd in distributions
like Debian which still have not adopted a /usr merge setup.
Use XML entities from man/custom-entities.ent to replace configured paths while
doing XSLT processing of the original XML files. There was precedent of some
files (such as systemd.generator.xml) which were already using this approach.
This addresses most of the (manual) fixes from this patch:
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Fix-paths-in-man-pages.patch?h=experimental-220
The idea of using generic XML entities was presented here:
http://lists.freedesktop.org/archives/systemd-devel/2015-May/032240.html
This patch solves almost all the issues, with the exception of:
- Path to /bin/mount and /bin/umount.
- Generic statements about preference of /lib over /etc.
These will be handled separately by follow up patches.
Tested:
- With default configure settings, ran "make install" to two separate
directories and compared the output to confirm they matched exactly.
- Used a set of configure flags including $CONFFLAGS from Debian:
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/rules
Installed the tree and confirmed the paths use /lib/systemd instead of
/usr/lib/systemd and that no other unexpected differences exist.
- Confirmed that `make distcheck` still passes.
In the long run we really should figure out if we want to stick with 8ch
or 2ch indenting, and not continue with half-and-half. For now, just
make emacs aware of the files that use 2ch indenting.
For priviliged units this resource control property ensures that the
processes have all controllers systemd manages enabled.
For unpriviliged services (those with User= set) this ensures that
access rights to the service cgroup is granted to the user in question,
to create further subgroups. Note that this only applies to the
name=systemd hierarchy though, as access to other controllers is not
safe for unpriviliged processes.
Delegate=yes should be set for container scopes where a systemd instance
inside the container shall manage the hierarchies below its own cgroup
and have access to all controllers.
Delegate=yes should also be set for user@.service, so that systemd
--user can run, controlling its own cgroup tree.
This commit changes machined, systemd-nspawn@.service and user@.service
to set this boolean, in order to ensure that container management will
just work, and the user systemd instance can run fine.
Only accept cpu quota values in percentages, get rid of period
definition.
It's not clear whether the CFS period controllable per-cgroup even has a
future in the kernel, hence let's simplify all this, hardcode the period
to 100ms and only accept percentage based quota values.
Similar to CPUShares= and BlockIOWeight= respectively. However only
assign the specified weight during startup. Each control group
attribute is re-assigned as weight by CPUShares=weight and
BlockIOWeight=weight after startup. If not CPUShares= or
BlockIOWeight= be specified, then the attribute is re-assigned to each
default attribute value. (default cpu.shares=1024, blkio.weight=1000)
If only CPUShares=weight or BlockIOWeight=weight be specified, then
that implies StartupCPUShares=weight and StartupBlockIOWeight=weight.
According to Wikipedia it is customary to specify hardware metrics and
transfer speeds to the basis 1000 (SI decimal), while software metrics
and physical volatile memory (RAM) sizes to the basis 1024 (IEC binary).
So far we specified everything in IEC, let's fix that and be more
true to what's otherwise customary. Since we don't want to parse "Mi"
instead of "M" we document each time what the context used is.
This is a recurring submission and includes corrections to various
issue spotted. I guess I can just skip over reporting ubiquitous comma
placement fixes…
Since cgroups are mostly now an implementation detail of systemd lets
deemphasize it a bit in the man pages. This renames systemd.cgroup(5) to
systemd.resource-control(5) and uses the term "resource control" rather
than "cgroup" where appropriate.
This leaves the word "cgroup" in at a couple of places though, like for
example systemd-cgtop and systemd-cgls where cgroup stuff is at the core
of what is happening.
AsciiWolf