When dealing with a large number of template instances, for example
when launching daemons per VRF, it is hard for operators to correlate
log lines to arguments.
Add a new with-unit mode which, if available, prefixes unit and user
unit names when displaying its log messages instead of the syslog
identifier. It will also use the full timestamp with timezones, like
the short-full mode.
Double newlines (i.e. one empty lines) are great to structure code. But
let's avoid triple newlines (i.e. two empty lines), quadruple newlines,
quintuple newlines, …, that's just spurious whitespace.
It's an easy way to drop 121 lines of code, and keeps the coding style
of our sources a bit tigther.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
Red is used for highligting, the same as grep does. Except when the line is
highlighted red already, because it has high priority, in which case plain ansi
highlight is used for the matched substring.
Coloring is implemented for short and cat outputs, and not for other types.
I guess we could also add it for verbose output in the future.
Incorporating the fix from d00f1d57 into other output formats of journalctl.
If journal files are corrupted, e.g. not cleanly closed, some journal
entries can not be read by output options other than 'short' (default).
If such entries has been identified, they will now just be skipped.
Using wait_for_terminate_and_check() instead of wait_for_terminate()
let's us simplify, shorten and unify the return value checking and
logging of waitid(). Hence, let's use it all over the place.
This adds a new safe_fork() wrapper around fork() and makes use of it
everywhere. The new wrapper does a couple of things we previously did
manually and separately in a safer, more correct and automatic way:
1. Optionally resets signal handlers/mask in the child
2. Sets a name on all processes we fork off right after forking off (and
the patch assigns useful names for all processes we fork off now,
following a systematic naming scheme: always enclosed in () – in order
to indicate that these are not proper, exec()ed processes, but only
forked off children, and if the process is long-running with only our
own code, without execve()'ing something else, it gets am "sd-" prefix.)
3. Optionally closes all file descriptors in the child
4. Optionally sets a PR_SET_DEATHSIG to SIGTERM in the child, in a safe
way so that the parent dying before this happens being handled
safely.
5. Optionally reopens the logs
6. Optionally connects stdin/stdout/stderr to /dev/null
7. Debug logs about the forked off processes.
This makes things a bit easier to read I think, and also makes sure we
always use the _unlikely_ wrapper around it, which so far we used
sometimes and other times we didn't. Let's clean that up.
This option allows restricting the shown fields in the output modes that
would normally show all fields. It allows clients that are only
interested in a subset of the fields to access those more efficiently.
Also, it makes the resulting size of the output more predictable.
It has no effect on the various `short` output modes, because those
already only show a subset of the fields.
By vectorizing parse_field() the chain of parse_field() calls in
output_short() can be replaced with a single call receiving a description
of the desired fields and their targets.
While at it, eliminate the repeated strlen() calls performed on constant
field names by making parse_field() receive the field length, and storing
it in the ParseFieldVec at compile time.
Also sort the output_short() fields so the short ones are tried first, for
a minor efficiency gain.
In addition to making the code less repetitive, gcc in my tests now inlines
the parse_fieldv() call in output_short().
These are similar to memdup() and newdup(), but reserve one extra NUL
byte at the end of the new allocation and initialize it. It's useful
when copying out data from fixed size character arrays where NUL
termination can't be assumed.
e268b81e moved an fflush() from output_json() to the generic
output_journal(), when it probably should have deleted all fflush()
calls from logs-show.c altogether.
The caller supplies the FILE * to these functions, and should be in
charge of flushing as needed. The current implementation essentially
defeats any buffering stdio was bringing to the table, resulting in
extraneous tiny write() calls in commands like `journalctl -b`.
This commit removes the fflush() call from output_journal(), and adds
them to journalctl before waiting for more entries and at completion.
This way in the hot path when journalctl loops on entries stdio can
combine multiple entries into bulkier write() calls.
usec_t is always 64bit, which means it can cover quite a number of
years. However, 4 digit year display and glibc limitations around time_t
limit what we can actually parse and format. Let's make this explicit,
so that we never end up formatting dates we can#t parse and vice versa.
Note that this is really just about formatting/parsing. Internal
calculations with times outside of the formattable range are not
affected.
We don't have plural in the name of any other -util files and this
inconsistency trips me up every time I try to type this file name
from memory. "formats-util" is even hard to pronounce.
This new output mode formats all timestamps using the usual format_timestamp()
call we use pretty much everywhere else. Timestamps formatted this way are some
ways more useful than traditional syslog timestamps as they include weekday,
month and timezone information, while not being much longer. They are also not
locale-dependent. The primary advantage however is that they may be passed
directly to journalctl's --since= and --until= switches as soon as #3869 is
merged.
While we are at it, let's also add "short-unix" to shell completion.
With this change, binary record data is formatted as string if --all is
specified when using json output. This is inline with the effect of --all on
the other available output modes.
Fixes: #3416
If journal files are not cleanly closed it might happen that intermediaery
journal entries cannot be read. Handle this nicely, skip over the unreadable
entries, and log a debug message about it; after all we generally follow the
logic that we try to make the best of corrupted files.
Throughout the tree there's spurious use of spaces separating ++ and --
operators from their respective operands. Make ++ and -- operator
consistent with the majority of existing uses; discard the spaces.
parse_field() checks if the field has the expected format, and returns
0 if it doesn't. In that case, value and size are not
set. Nevertheless, we would try to continue, and hit an assert in
safe_atou64. This case shouldn't happen, unless sd_j_get_data is borked,
so cleanly assert that we got the expected field.
Also, oom is the only way that parse_field can fail, which we log
already. Instead of outputting a debug statement and carrying on,
treat oom as fatal.
GLIB has recently started to officially support the gcc cleanup
attribute in its public API, hence let's do the same for our APIs.
With this patch we'll define an xyz_unrefp() call for each public
xyz_unref() call, to make it easy to use inside a
__attribute__((cleanup())) expression. Then, all code is ported over to
make use of this.
The new calls are also documented in the man pages, with examples how to
use them (well, I only added docs where the _unref() call itself already
had docs, and the examples, only cover sd_bus_unrefp() and
sd_event_unrefp()).
This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we
tend to call our destructors these days.
Note that this defines no public macro that wraps gcc's attribute and
makes it easier to use. While I think it's our duty in the library to
make our stuff easy to use, I figure it's not our duty to make gcc's own
features easy to use on its own. Most likely, client code which wants to
make use of this should define its own:
#define _cleanup_(function) __attribute__((cleanup(function)))
Or similar, to make the gcc feature easier to use.
Making this logic public has the benefit that we can remove three header
files whose only purpose was to define these functions internally.
See #2008.
There are more than enough calls doing string manipulations to deserve
its own files, hence do something about it.
This patch also sorts the #include blocks of all files that needed to be
updated, according to the sorting suggestions from CODING_STYLE. Since
pretty much every file needs our string manipulation functions this
effectively means that most files have sorted #include blocks now.
Also touches a few unrelated include files.
Let's underline the header line of the table shown by cgtop, how it is
customary for tables. In order to do this, let's introduce new ANSI
underline macros, and clean up the existing ones as side effect.
As it turns out machine_name_is_valid() does the exact same thing as
hostname_is_valid() these days, as it just invoked that and checked the
name length was < 64. However, hostname_is_valid() checks the length
against HOST_NAME_MAX anyway (which is 64 on Linux), hence any
additional check is redundant.
We hence replace machine_name_is_valid() by a macro that simply maps it
to hostname_is_valid() but sets the allow_trailing_dot parameter to
false. We also move this this call to hostname-util.h, to the same place
as the hostname_is_valid() declaration.
To be able to use `systemd-run` or `machinectl login` on a container
that is in a private user namespace, the sub-process must have entered
the user namespace before connecting to the container's D-Bus, otherwise
the UID and GID in the peer credentials are garbage.
So we extend namespace_open and namespace_enter to support UID namespaces,
and we enter the UID namespace in bus_container_connect_{socket,kernel}.
namespace_open will degrade to a no-op if user namespaces are not enabled
in the kernel.
Special handling is required for the setns call in namespace_enter with
a user namespace, since transitioning to your own namespace is forbidden,
as it would result in re-entering your user namespace as root.
Arguably it may be valid to check this at the call site, rather than
inside namespace_enter, but it is less code to do it inside, and if the
intention of calling namespace_enter is to *be* in the target namespace,
rather than to transition to the target namespace, it is a reasonable
approach.
The check for whether the user namespace is the same must happen before
entering namespaces, as we may not be able to access /proc during the
intermediate transition stage.
We can't instead attempt to enter the user namespace and then ignore
the failure from it being the same namespace, since the error code is
not distinct, and we can't compare namespaces while mid-transition.
All functions should either log the errors they run into, or only return
them in which case the caller should log them.
Make sure this rule is followed, so that each error is logged precisely
once, and neither never, nor more than once.
like:
src/shared/install.c: In function ‘unit_file_lookup_state’:
src/shared/install.c:1861:16: warning: ‘r’ may be used uninitialized in
this function [-Wmaybe-uninitialized]
return r < 0 ? r : state;
^
src/shared/install.c:1796:13: note: ‘r’ was declared here
int r;
^
Usually when using loop_read(), we want to read the full buffer.
Add a helper that mirrors loop_write(), and returns 0 when full buffer
was read, and an error otherwise.
Use -ENODATA for the short read, to distinguish it from a read error.
This patch removes includes that are not used. The removals were found with
include-what-you-use which checks if any of the symbols from a header is
in use.
include-what-you-use automatically does this and it makes finding
unnecessary harder to spot. The only content of poll.h is a include
of sys/poll.h so should be harmless.
After all it is now much more like strjoin() than strappend(). At the
same time, add support for NULL sentinels, even if they are normally not
necessary.
Using:
find . -name '*.[ch]' | while read f; do perl -i.mmm -e \
'local $/;
local $_=<>;
s/(if\s*\([^\n]+\))\s*{\n(\s*)(log_[a-z_]*_errno\(\s*([->a-zA-Z_]+)\s*,[^;]+);\s*return\s+\g4;\s+}/\1\n\2return \3;/msg;
print;'
$f
done
And a couple of manual whitespace fixups.
As a followup to 086891e5c1 "log: add an "error" parameter to all
low-level logging calls and intrdouce log_error_errno() as log calls
that take error numbers", use sed to convert the simple cases to use
the new macros:
find . -name '*.[ch]' | xargs sed -r -i -e \
's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/'
Multi-line log_*() invocations are not covered.
And we also should add log_unit_*_errno().
It is redundant to store 'hash' and 'compare' function pointers in
struct Hashmap separately. The functions always comprise a pair.
Store a single pointer to struct hash_ops instead.
systemd keeps hundreds of hashmaps, so this saves a little bit of
memory.
safe_close_pair() is more like safe_close(), except that it handles
pairs of fds, and doesn't make and misleading allusion, as it works
similarly well for socketpairs() as for pipe()s...
safe_close() automatically becomes a NOP when a negative fd is passed,
and returns -1 unconditionally. This makes it easy to write lines like
this:
fd = safe_close(fd);
Which will close an fd if it is open, and reset the fd variable
correctly.
By making use of this new scheme we can drop a > 200 lines of code that
was required to test for non-negative fds or to reset the closed fd
variable afterwards.
If a message had zero length, journalctl would print no newline, and
two output lines would be concatenated. Fix. The problem was
introduced in commit 31f7bf1994 ("logs-show: print multiline
messages"). Affected short and verbose output modes.
Before fix:
Feb 09 21:16:17 glyph dhclient[1323]: Feb 09 21:16:17 glyph NetworkManager[788]: <info> (enp4s2): DHCPv4 state changed nbi -> preinit
after:
Feb 09 21:16:17 glyph dhclient[1323]:
Feb 09 21:16:17 glyph NetworkManager[788]: <info> (enp4s2): DHCPv4 state changed nbi -> preinit
This adds the new library call sd_journal_open_container() and a new
"-M" switch to journalctl. Particular care is taken that journalctl's
"-b" switch resolves to the current boot ID of the container, not the
host.
Condition that is checked is taken from upower:
active(anon) < free swap * 0.98
This is really stupid, because the kernel knows the situation better,
e.g. there could be two swap files, and then hibernation would be
impossible despite passing this check, or the kernel could start
supporting compressed swap and/or compressed hibernation images, and
then this this check would be too stringent. Nevertheless, until
we have something better, this should at least return a true negative
if there's no swap.
Logging of capabilities in the journal is changed to not strip leading
zeros. I consider this more readable anyway.
http://cgit.freedesktop.org/upower/tree/src/up-daemon.c#n613https://bugzilla.redhat.com/show_bug.cgi?id=1007059
This makes verbose behave like short mode, i.e. try to show
the source timestamp, and fall back to journald timestamp
only if unavailable or unparsable. I think verbose should
be like short, only showing more fields, and showing different
timestamps would be confusing.
So far, we would show up to 128 bytes from a message, simply
cutting of the rest. With multiline messages, it is quite common
for a message to be longer than that, and this model doesn't really
work anymore.
A new limit is added: up to 3 lines will be shown, unless --full is
used (c.f. first line below). The limit for bytes is extended to 300
bytes. An ellipsis will always be used, if some form of truncation
occurs. If the tail of the message is cut off, either because of
length or line limit, dots will be shown at the end of the last
line. If this last line is short, the dots will be simply appended. If
the last line is too long for that, it will be ellipsized with dots at
the very end.
Note that the limits are in bytes, not characters, and we suck at
outputting unicode strings (c.f. last three lines below).
Aug 11 10:46:21 fedora python[67]: test message
line
line...
Aug 11 10:50:47 fedora python[76]: test message word word word word word word word word word word word wor...
Aug 11 10:55:11 fedora python[83]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
Aug 11 11:03:21 fedora python[90]: ąąąąąąąąąąąąąąąąąąąąąąąąąąąąąą...
Aug 11 11:03:53 fedora python[97]: aąąąąąąąąąąąąąąąąąąąąąąąąąąąąąą...
Aug 11 11:25:45 fedora python[121]: aąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąą�...
When looking at verbose output, additional "work" is required to
pick out the interesting MESSAGE= lines from all the fields.
Also, show long fields in full in verbose output mode when
OUTPUT_FULL_WIDTH is specified.
