Commit graph

32034 commits

Author SHA1 Message Date
Lennart Poettering 69cd51de5d meson: bump so revision and systemd version in preparation for v237 2018-01-25 14:18:14 +01:00
Lennart Poettering a1bb4de545 hwdb: run "update-hwdb" in preparation for v237 2018-01-25 14:16:25 +01:00
Lennart Poettering aa79f93268 update TODO 2018-01-25 14:14:35 +01:00
Lennart Poettering 95894b91b4 NEWS: start putting together an entry for v237 2018-01-25 13:21:00 +01:00
Michael Vogt e2c2060f7b sysusers: make ADD_GROUP always create a group
Do not merge group creation with user creation because with the
new uid:gid syntax this can result in confusing (and unwanted)
behavior.
2018-01-25 12:51:06 +01:00
Michael Vogt 1e589ed264 test: add TEST-21-SYSUSERS test
This test tests the systemd-sysuser binary via the --root=$TESTDIR
option and ensures that for the given inputs the expected passwd
and group files will be generated.
2018-01-25 12:51:00 +01:00
Michael Vogt 5bc9c980d0 sysuser: use OrderedHashmap
This means we have more predicable behavior for "u foo uid:gid" lines
and also makes the generated files appear in the same order as the
inputs. So e.g.
```
u      root         0 -     /root
u      daemon       1 -     /usr/sbin
u      games        5:60 -     /usr/games
```
will generate
```
root0:0::/root:/bin/sh
daemon1:1::/usr/sbin:/sbin/nologin
games5:60::/usr/games:/sbin/nologin
```
2018-01-25 12:50:37 +01:00
Michael Vogt 4cb41413c8 sysusers: allow uid:gid in sysusers.conf files
This PR allows to write sysuser.conf lines like:
```
u games 5:60 -
```
This will create an a "games" user with uid 5 and games group with
gid 60. This is arguable ugly, however it is required to represent
certain configurations like the default passwd file on Debian and
Ubuntu.

When the ":" syntax is used and there is a group with the given
gid already then no new group is created. This allows writing the
following:
```
g unrelated 60
u games 5:60 -
```
which will create a "games" user with the uid 5 and the primary
gid 60. No group games is created here (might be useful for [1]).

[1] https://pagure.io/packaging-committee/issue/442
2018-01-25 12:50:37 +01:00
Zbigniew Jędrzejewski-Szmek 5eb83fa645
Merge pull request #7991 from poettering/n-on-console
a comprehensive fix for the n_on_console miscounting issue
2018-01-25 13:48:08 +03:00
Lennart Poettering 7a466dc453
Merge pull request #7983 from poettering/tmpfiles-eexist
make "f" tmpfiles.d behaviour work like documentation suggests + coccinelle fixes
2018-01-25 11:34:15 +01:00
Zbigniew Jędrzejewski-Szmek f26f5b60d0 Merge pull request #7915 from poettering/pids-max-tweak 2018-01-25 10:24:35 +01:00
Lennart Poettering c7c5b69095
Merge pull request #7990 from poettering/log-deadlock
fix for logging deadlock
2018-01-25 10:06:19 +01:00
Lennart Poettering adefcf2821 core: rework how we count the n_on_console counter
Let's add a per-unit boolean that tells us whether our unit is currently
counted or not. This way it's unlikely we get out of sync again and
things are generally more robust.

This also allows us to remove the counting logic specific to service
units (which was in fact mostly a copy from the generic implementation),
in favour of fully generic code.

Replaces: #7824
2018-01-24 20:14:51 +01:00
Lennart Poettering bb2c768545 core: add a new unit_needs_console() call
This call determines whether a specific unit currently needs access to
the console. It's a fancy wrapper around
exec_context_may_touch_console() ultimately, however for service units
we'll explicitly exclude the SERVICE_EXITED state from when we report
true.
2018-01-24 19:54:26 +01:00
Lennart Poettering 46fb617bf9 manager: minor manager_get_show_status() simplification
Since the the whole function ultimately is just a fancy getter for the
show_status field, let's actually return it as last step literally
without an extra needless "if".
2018-01-24 19:52:29 +01:00
Lennart Poettering 5a69973ff2 manager: add some explanatory comments to manager_dispatch_idle_pipe_fd() 2018-01-24 19:52:14 +01:00
Lennart Poettering 6c347d5024 log: remove LOG_TARGET_SAFE pseudo log target
This removes LOG_TARGET_SAFE. It's made redundant by the new
"prohibit-ipc" logging flag, as it used to have a similar effect: avoid
logging to the journal/syslog, i.e. any local services in order to avoid
deadlocks when we lock from PID 1 or its utility processes (such as
generators).

All previous users of LOG_TARGET_SAFE are switched over to the new
setting. This makes things a bit safer for all, as not even the
SYSTEMD_LOG_TARGET env var can be used to accidentally log to the
journal anymore in these programs.
2018-01-24 18:22:56 +01:00
Lennart Poettering 7a3be263d2 log: add brief comment for log_set_open_when_needed() and log_set_always_reopen_console()
These two deserve some explanation...
2018-01-24 18:22:56 +01:00
Lennart Poettering d075092f14 pid1: make use of new "prohibit_ipc" logging flag in PID 1
Let's set it initially, and then toggle it only when we know its safe.
2018-01-24 18:22:56 +01:00
Lennart Poettering adf47c919a log: add new "prohibit_ipc" flag to logging system
If set, we'll avoid logging to any IPC log targets, i.e. syslog or the
journal, but allow stderr, kmsg, console logging.

This is useful as PID 1 wants to turn this off explicitly as long as the
journal is not up.

Previously we'd open/close the log stream to these services whenever
needed but this is incompatible with the "open_when_needed" logic
introduced in #6915, which might open the log streams whenever it likes,
including possibly inside of the child process we fork off that'll
become journald later on. Hence, let's make this all explicit, and
instead of managing when we open/close log streams add a boolean that
clearly prohibits the IPC targets when needed, so that opening can be
done at any time, but will honour this.

See: #7985
2018-01-24 18:22:56 +01:00
Lennart Poettering 6fdb8de42f log: make log_set_upgrade_syslog_to_journal() take effect immediately
This doesn't matter much, and we don't rely on it, but I think it's much
nicer if we log_set_target() and log_set_upgrade_syslog_to_journal() can
be called in either order and have the same effect.
2018-01-24 18:22:56 +01:00
Nathaniel McCallum ab93297cd0 Add fd close support to sd_event_source
It is often the case that a file descriptor and its corresponding IO
sd_event_source share a life span. When this is the case, developers will
have to unref the event source and close the file descriptor. Instead, we
can just have the event source take ownership of the file descriptor and
close it when the event source is freed. This is especially useful when
combined with cleanup attributes and sd_event_source_unrefp().

This patch adds two new public functions:

    sd_event_source_get_io_fd_own()
    sd_event_source_set_io_fd_own()
2018-01-24 17:57:27 +01:00
Nathaniel McCallum 2b97a9573f Include time.h in sd-event.h
The time-related functions in sd-event.h take as inputs constants (CLOCK_*)
defined in time.h. By including time.h in sd-event.h, we free the developer
from having to do this manually.
2018-01-24 16:34:57 +01:00
Lennart Poettering 7d94e8a4b3
Merge pull request #7988 from ssahani/follow-7712
Networkd: cleanup  code a bit
2018-01-24 16:15:26 +01:00
Susant Sahani 6964cf4558 networkd: routing policy rules rename Manager object m to manager. 2018-01-24 20:27:53 +05:30
Susant Sahani d6ba0f2c4a networkd: remove unused variable manager -> m 2018-01-24 20:27:27 +05:30
Ewout van Mansom 21112e594c hwdb: Workaround Purism Librem 13 V2 change 43 to backslash (#7984)
hwdb: use symbolic name rather than numeric key value for Purism Librem 13 entry

Fixes: #7978
2018-01-24 13:58:43 +01:00
Susant Sahani 420d205867 networkd: sd_ndisc reset mac address when MAC address change detected.
When there is a change in mac address we are not currently not changing
the MAC address and resulting  v6 connectivity is gone.

When kernel reports a change in mac address change the MAC of ndisc
client too.

Closes # 7806
2018-01-24 12:05:38 +01:00
Lennart Poettering 7b7e6b8df5
Merge pull request #7860 from poettering/watch-pids-rework
rework how the manager watches PIDs
2018-01-24 11:18:04 +01:00
Lennart Poettering db4a47e9fe coccinelle: O_NDELAY → O_NONBLOCK
Apparently O_NONBLOCK is the modern name used in most documentation and
for most cases in our sources. Let's hence replace the old alias
O_NDELAY and stick to O_NONBLOCK everywhere.
2018-01-24 11:09:29 +01:00
Lennart Poettering 81b1dc271f coccinelle: let's use STRLEN() rather strlen() at one more place 2018-01-24 11:07:47 +01:00
Lennart Poettering 49e87292dc tmpfiles: make "f" lines behaviour match what the documentation says
CHANGE OF BEHAVIOUR — with this commit "f" line's behaviour is altered
to match what the documentation says: if an "argument" string is
specified it is written to the file only when the file didn't exist
before. Previously, it would be appended to the file each time
systemd-tmpfiles was invoked — which is not a particularly useful
behaviour as the tool is not idempotent then and the indicated files
grow without bounds each time the tool is invoked.

I did some spelunking whether this change in behaviour would break
things, but afaics nothing relies on the previous O_APPEND behaviour of
this line type, hence I think it's relatively safe to make "f" lines
work the way the docs say, rather than adding a new modifier for it or
so.

Triggered by:

https://lists.freedesktop.org/archives/systemd-devel/2018-January/040171.html
2018-01-24 10:54:10 +01:00
Lennart Poettering 1302e7aa94
Merge pull request #7979 from yuwata/unused-variables
Small cleanups
2018-01-24 09:40:04 +01:00
Michael Biebl 546e635a4e resolve: fix build without gcrypt
Follow-up for 73a4cd17c3.

Fixes #7977.
2018-01-24 12:59:22 +09:00
Yu Watanabe a219c2698e systemctl: remove unused variable
Follow-up for 98f6093687.
2018-01-24 12:54:15 +09:00
Yu Watanabe e61d3d8aae core/device: remove unused variable
Follow-up for bf70ff2cff.
2018-01-24 12:52:23 +09:00
Lennart Poettering dad105c678
Merge pull request #7964 from poettering/tmpfiles-more-fixes
a number of tmpfiles fixes
2018-01-23 21:37:55 +01:00
Lennart Poettering 1a58b91079 update TODO 2018-01-23 21:29:31 +01:00
Lennart Poettering 9acac21249 service: simplify condition
The left side of the || expression is conditionalized on SERVICE_START,
but SERVICE_START is blanket listed on the right side anyway, hence we
can drop the left side entirely without any change in behaviour.

Moreover, if main_pid is initialized, it should be watched, hence this
is even the safe and right thing to do.
2018-01-23 21:29:31 +01:00
Lennart Poettering eabd3e56a6 service: don't bother with watching PIDs during deserialization
service_coldplug() takes care of that anyway, hence drop the
unit_watch_pid() invocation entirely during serialization, it's
redundant.
2018-01-23 21:29:31 +01:00
Lennart Poettering 62a769136d core: rework how we track which PIDs to watch for a unit
Previously, we'd maintain two hashmaps keyed by PIDs, pointing to Unit
interested in SIGCHLD events for them. This scheme allowed a specific
PID to be watched by exactly 0, 1 or 2 units.

With this rework this is replaced by a single hashmap which is primarily
keyed by the PID and points to a Unit interested in it. However, it
optionally also keyed by the negated PID, in which case it points to a
NULL terminated array of additional Unit objects also interested. This
scheme means arbitrary numbers of Units may now watch the same PID.

Runtime and memory behaviour should not be impact by this change, as for
the common case (i.e. each PID only watched by a single unit) behaviour
stays the same, but for the uncommon case (a PID watched by more than
one unit) we only pay with a single additional memory allocation for the
array.

Why this all? Primarily, because allowing exactly two units to watch a
specific PID is not sufficient for some niche cases, as processes can
belong to more than one unit these days:

1. sd_notify() with MAINPID= can be used to attach a process from a
   different cgroup to multiple units.

2. Similar, the PIDFile= setting in unit files can be used for similar
   setups,

3. By creating a scope unit a main process of a service may join a
   different unit, too.

4. On cgroupsv1 we frequently end up watching all processes remaining in
   a scope, and if a process opens lots of scopes one after the other it
   might thus end up being watch by many of them.

This patch hence removes the 2-unit-per-PID limit. It also makes a
couple of other changes, some of them quite relevant:

- manager_get_unit_by_pid() (and the bus call wrapping it) when there's
  ambiguity will prefer returning the Unit the process belongs to based on
  cgroup membership, and only check the watch-pids hashmap if that
  fails. This change in logic is probably more in line with what people
  expect and makes things more stable as each process can belong to
  exactly one cgroup only.

- Every SIGCHLD event is now dispatched to all units interested in its
  PID. Previously, there was some magic conditionalization: the SIGCHLD
  would only be dispatched to the unit if it was only interested in a
  single PID only, or the PID belonged to the control or main PID or we
  didn't dispatch a signle SIGCHLD to the unit in the current event loop
  iteration yet. These rules were quite arbitrary and also redundant as
  the the per-unit handlers would filter the PIDs anyway a second time.
  With this change we'll hence relax the rules: all we do now is
  dispatch every SIGCHLD event exactly once to each unit interested in
  it, and it's up to the unit to then use or ignore this. We use a
  generation counter in the unit to ensure that we only invoke the unit
  handler once for each event, protecting us from confusion if a unit is
  both associated with a specific PID through cgroup membership and
  through the "watch_pids" logic. It also protects us from being
  confused if the "watch_pids" hashmap is altered while we are
  dispatching to it (which is a very likely case).

- sd_notify() message dispatching has been reworked to be very similar
  to SIGCHLD handling now. A generation counter is used for dispatching
  as well.

This also adds a new test that validates that "watch_pid" registration
and unregstration works correctly.
2018-01-23 21:29:31 +01:00
Lennart Poettering ae91745500
Merge pull request #7968 from poettering/manager-mini-tweak
some manager fixes
2018-01-23 21:23:19 +01:00
Lennart Poettering 11aef522c1 core: unify call we use to synthesize cgroup empty events when we stopped watching any unit PIDs
This code is very similar in scope and service units, let's unify it in
one function. This changes little for service units, but for scope units
makes sure we go through the cgroup queue, which is something we should
do anyway.
2018-01-23 21:22:50 +01:00
Lennart Poettering 5cdabc8d7b service: don't send out dbus change notifications spuriously on SIGCHLD
Let's send them out only if the main or control processe exited and we
recorded a new exit status that is worth reporting. But if any other
service process died this is nothing to report since we don't expose any
properties about that anyway.
2018-01-23 21:22:50 +01:00
Lennart Poettering 2ca9d97943 core: fix manager_get_unit_by_pid() special casing of manager PID
Previously, we'd hard map PID 1 to the manager scope unit. That's wrong
however when we are run in --user mode, as the PID 1 is outside of the
subtree we manage and the manager PID might be very differently. Correct
that by checking for getpid() rather than hardcoding 1.
2018-01-23 21:22:50 +01:00
Lennart Poettering eb5eb254dd test: test-process-util additions
Add some extra paranoia tests that PTR_TO_PID() and PID_TO_PTR() deals
correctly with negative PID.
2018-01-23 21:22:50 +01:00
Lennart Poettering bc2fcf7f7a process-util: replace PTR_TO_PID() and PID_TO_PTR macro by inline functions
This way we gain some typesafety at no cost.
2018-01-23 21:22:50 +01:00
Lennart Poettering 60644c3dea cgroup: fix handling of TasksAccounting= property 2018-01-23 21:22:50 +01:00
Lennart Poettering 2e276b1d9b UIDS-GIDS.md: explicitly mention one more user of the overflowuid
File systems with only 16bit UID support (i.e. old ext2) also use the
overflowuid to map users they can't map. Briefly mention this.
2018-01-23 21:20:09 +01:00
Lennart Poettering 45a080944d hwdb: whitespace fix to make "ninja test" work again
Fixes: #7975
2018-01-23 21:19:49 +01:00