HUGE_SIZE was defined inconsistently.
> In file included from ../src/basic/alloc-util.h:9,
> from ../src/journal/test-compress.c:9:
> ../src/journal/test-compress.c: In function ‘main’:
> ../src/journal/test-compress.c:280:33: error: ‘HUGE_SIZE’ undeclared (first use in this function)
> 280 | assert_se(huge = malloc(HUGE_SIZE));
Don't assume that 4MB can be allocated from stack since there could be smaller
DefaultLimitSTACK= in force, so let's use malloc(). NUL terminate the huge
strings by hand, also ensure termination in test_lz4_decompress_partial() and
optimize the memset() for the string.
Some items in /proc and /etc may not be accessible to poor unprivileged users
due to e.g. SELinux, BOFH or both, so check for EACCES and EPERM.
/var/tmp may be a symlink to /tmp and then path_compare() will always fail, so
let's stick to /tmp like elsewhere.
/tmp may be mounted with noexec option and then trying to execute scripts from
there would fail.
Detect and warn if seccomp is already in use, which could make seccomp test
fail if the syscalls are already blocked.
Unset $TMPDIR so it will not break specifier tests where %T is assumed to be
/tmp and %V /var/tmp.
Our journal code is generally supposed to be written in a fashion that
the underlying file can be deallocated any time, i.e. our mmap of it
suddenly becomes all zeroes. The idea is that we catch that when parsing
everything. For that to work safely we need to make sure that when doing
arithmetics or comparisons on values read from the map we don't run into
TTOCTTOU issues when determining validity. Hence we need to copy out the
values before use and operate on the copies. This requires some special
care since the C compiler could suppress our copies as optimization.
Hence use the new READ_NOW() macro to force a copy by using memcpy(),
and use it whenever we start doing an arithmetic operation on it, or
validity checking of multiple steps.
Fixes: #14943
Mappings canbe replaced by all zeroes under our feet if vacuuming
decides to unallocate some file. Hence let's not check for this kind of
stuff in an assert.
(Typically, we should genreate runtime errors in this case, in
particular EBADMSG, which the callers generally look for. But in this
case this is just an extra precaution check anyway, so let's just remove
it.)
Let's be extra careful whenever we return from recvmsg() and see
MSG_CTRUNC set. This generally means we ran into a programming error, as
we didn't size the control buffer large enough. It's an error condition
we should at least log about, or propagate up. Hence do that.
This is particularly important when receiving fds, since for those the
control data can be of any size. In particular on stream sockets that's
nasty, because if we miss an fd because of control data truncation we
cannot recover, we might not even realize that we are one off.
(Also, when failing early, if there's any chance the socket might be
AF_UNIX let's close all received fds, all the time. We got this right
most of the time, but there were a few cases missing. God, UNIX is hard
to use)
It's not that I think that "hostname" is vastly superior to "host name". Quite
the opposite — the difference is small, and in some context the two-word version
does fit better. But in the tree, there are ~200 occurrences of the first, and
>1600 of the other, and consistent spelling is more important than any particular
spelling choice.
Let's make it optional whether auditing is enabled at journald start-up
or not.
Note that this only controls whether audit is enabled/disabled in the
kernel. Either way we'll still collect the audit data if it is
generated, i.e. if some other tool enables it, we'll collect it.
Fixes: #959
journal_file_fstat() returns an error if we call it on already unlinked
journal file and hence we never reach remove_file_real() which is the
entire point.
I must have made some mistake while testing the fix that got me thinking
the issue is gone while opposite was true.
Fixes#14695
This regression was introduced in #14913.
The current_file variable can be NULL, as, for example, with the
following commands:
* journalctl --list-boots
* journalctl -b -1 --no-pager
Since current_file is only checked for pointer equality with f, removing
the assertion is safe here.
When having a service which intentionally outputs multiple equal lines,
all these messages might be inserted with the same timestamp.
journalctl has a mechanism to avoid duplicate lines, which might be in
different journal files.
This patch allows duplicate lines, if they are from the same file.
It fully initializes the address structure, so no need for pre-initialization,
and also returns the length of the address, so no need to recalculate using
SOCKADDR_UN_LEN().
socklen_t is unsigned, so let's not use an int for it. (It doesn't matter, but
seems cleaner and more portable to not assume anything about the type.)
.msg_namelen was set to a bogus value before we actually stored the path in the
the structure. sockaddr_un_set_path() returns the length, so just use that.
Fixes#14799.
If we have exit on idle, then operations such as "journalctl
--namespace=foo --rotate" should work even if the journal daemon is
currently not running.
(Note that we don't do activation by varlink for the main instance of
journald, I am not sure the deadlocks it might introduce are worth it)
If we do, we operate on a separate set of logs and runtime objects
The namespace is configured via argv[1].
Fixes: #12123Fixes: #10230#9519
(These latter two issues ask for slightly different stuff, but the
usecases generally can be solved by running separate instances of
journald now, hence also declaring that as "Fixes:")
Let's just use the path that is already stored in JournalStorage,
instead of generating our own. While we are at it, split out the loop
into its own function.
Write a user unit's invocation ID to /run/user/<uid>/systemd/units/ similar
to how a system unit's invocation ID is written to /run/systemd/units/.
This lets the journal read and add a user unit's invocation ID to the
_SYSTEMD_INVOCATION_ID field of logs instead of the user manager's
invocation ID.
Fixes#12474
This cleans up and unifies the outut of --help texts a bit:
1. Highlight the human friendly description string, not the command
line via ANSI sequences. Previously both this description string and
the brief command line summary was marked with the same ANSI
highlight sequence, but given we auto-page to less and less does not
honour multi-line highlights only the command line summary was
affectively highlighted. Rationale: for highlighting the description
instead of the command line: the command line summary is relatively
boring, and mostly the same for out tools, the description on the
other hand is pregnant, important and captions the whole thing and
hence deserves highlighting.
2. Always suffix "Options" with ":" in the help text
3. Rename "Flags" → "Options" in one case
4. Move commands to the top in a few cases
5. add coloring to many more help pages
6. Unify on COMMAND instead of {COMMAND} in the command line summary.
Some tools did it one way, others the other way. I am not sure what
precisely {} is supposed to mean, that uppercasing doesn't, hence
let's simplify and stick to the {}-less syntax
And minor other tweaks.
For some unrelated stuff I wanted the machine ID in UUID format, and it
was annoying doing that manually. So let's add a switch for this, so
that this works:
systemd-id128 machine-id -u
journald assumes that getsockopt(SO_PEERCRED) correctly identifies the
process on the remote end of the socket. However, this is incorrect
according to man 7 socket:
The returned credentials are those that were in effect at the
time of the call to connect(2) or socketpair(2).
This becomes a problem when a new process inherits the stdout stream
from a parent. First, log messages from the child process will
be attributed to the parent. Second, the struct ucred used by journald
becomes invalid as soon as the parent exits. Further sendmsg calls then
fail with ENOENT. Logs for the child process then vanish from the journal.
Fix this by using recvmsg on the stdout stream, and refreshing the cached
struct ucred if SCM_CREDENTIALS indicate a new process.
Fixes#13708
Right now the `systemd-journal-remote` service does not constrain its
resource usage (I just run out of space on my 100GB partition, for
example). This patch does not change that, but it at least makes it
possible to run something like:
journalctl --directory /var/log/journal/remote --rotate --vacuum-size=90G
fixes#2376
Co-authored-by: Mike Auty <ikelos@gentoo.org>
chase_symlinks() would return negative on error, and either a non-negative status
or a non-negative fd when CHASE_OPEN was given. This made the interface quite
complicated, because dependning on the flags used, we would get two different
"types" of return object. Coverity was always confused by this, and flagged
every use of chase_symlinks() without CHASE_OPEN as a resource leak (because it
would this that an fd is returned). This patch uses a saparate output parameter,
so there is no confusion.
(I think it is OK to have functions which return either an error or an fd. It's
only returning *either* an fd or a non-fd that is confusing.)
"ratelimit" is a real word, so we don't need to use the other form anywhere.
We had both forms in various places, let's standarize on the shorter and more
correct one.
The use of an unordered hashmap means that the output of
'journalctl --update-catalog' differs between runs despite there being no
changes in the input files.
By changing all instances of Hashmap to OrderedHashmap we fix this, and now
the catalog is reproducible.
Motivation: https://reproducible-builds.org
Signed-off-by: Daniel Edgecumbe <git@esotericnonsense.com>
journalctl --unit= already did this, and allows you to tail all the logs
for a certain slice easily. It seemed only natural to make --user-unit
behave in a similar way.
The _SYSTEMD_USER_SLICE field was not documented as being added by
journald, so I have added that to the documentation too.
Furthermore, I have documented the existing behaviour of --unit= and the
new behaviour of --user-unit=
The behaviour was actually not documented before, so I am also OK with
removing the match for the --unit= command instead. The user would then
have to manually provide _SYSTEMD_SLICE= filter to journalctl in both
cases. Both options work for me.
Ubunut autopkgtest fails with:
405/501 test-journal-flush FAIL 0.74 s (killed by signal 6 SIGABRT)
--- command ---
SYSTEMD_KBD_MODEL_MAP='/tmp/autopkgtest.BgjJJv/build.yAM/systemd/src/locale/kbd-model-map' SYSTEMD_LANGUAGE_FALLBACK_MAP='/tmp/autopkgtest.BgjJJv/build.yAM/systemd/src/locale/language-fallback-map' PATH='/tmp/autopkgtest.BgjJJv/build.yAM/systemd/build-deb:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games' /tmp/autopkgtest.BgjJJv/build.yAM/systemd/build-deb/test-journal-flush
--- stderr ---
Assertion 'r >= 0' failed at src/journal/test-journal-flush.c:48, function main(). Aborting.
-------
It's hard to say what is going on here without any error messages whatsoever.
The test goes into deep details of journal file handling, so it needs to also
do logging on its own.
https://bugzilla.redhat.com/show_bug.cgi?id=1715699
> /dev/mapper/live-rw 6.4G 5.7G 648M 91% /
> systemd-journald[905]: Fixed min_use=1.0M max_use=648.7M max_size=81.0M min_size=512.0K keep_free=973.1M n_max_files=100
When journald is started, we pick keep_free as 15% of the disk size. When the
fs is almost filled, we will only keep one journal file around and rotate very
often (because min_size is very small).
Let's set min use to something reasonable, so that we get more useful logs that
will cover at least the full boot.
Some cases considered in the PR:
> /dev/mapper/live-rw 6.4G 5.7G 648M 91% /
keep_free→MIN(327,100)→100 MB.
min_use→16MB.
effective range: 16 MB – 548 MB
> /dev/mapper/fedora_krowka-root 78G 69G 5.7G 93% /
keep_free → MIN(4GB, 100MB)→100MB
min_use→16MB
effective range: 16 MB – 5.6 GB
(but then there's the max_use limit, which cuts the range down)
> 4TB, 4GB free
keep_free → MIN(209715, 100) → 100 MB
min_use→16MB
effective range: 16 MB – 4.9 GB
(also effectively limited by max_use)
Also replace unneeded width suffixes with spaces, I think this is more
readable, and drop DEFAULT_ prefixes in cases where this setting is
simply a bound, and cannot be overridden by user config, hence is not
a default.
Not everybody has those dirs in the filesystem (and they don't need to).
When creating an installation package using $DESTDIR, it is easy enough to
remove or ignore those directories, but if installing into a real root, it
is ugly to create and remove them. Let's add an option so people can skip
it if they want.
Inspired by #12930.
The functions to retrieve and print process cmdlines were based on the
assumption that they contain printable ASCII, and everything else
should be filtered out. That assumption doesn't hold in today's world,
where people are free to use unicode everywhere.
This replaces the custom cmdline reading code with a more generic approach
using utf8_escape_non_printable_full().
For kernel threads, truncation is done on the parenthesized name, so we'll
get "[worker]", "[worker…]", …, "[w…]", "[…", "…" as we reduce the number of
available columns.
This implementation is most likely slower for very long cmdlines, but I don't
think this is very important. The common case is to have short commandlines,
and should print those properly. Absurdly long cmdlines are the exception,
which needs to be handled correctly and safely, but speed is not too important.
Fixes#12532.
v2:
- use size_t for the number of columns. This change propagates into various
other functions that call get_process_cmdline(), increasing the size of the
patch, but the changes are rather trivial.
When journalctl is compiled with PCRE2 support, let's return a non-zero
exit code when --grep is used and no match for given pattern is found.
This should allow users to use journalctl --grep in scripts instead of
piping journalctl into grep
Fixes#8152
The latter is identical to the former, but becomes a NOP if
/var/log/journal is on the same mount as /, and thus during shutdown
unmounting /var is not necessary and hence we can keep logging until the
very end.
When emitting the calendarspec warning we want to see some color.
Follow-up for 04220fda5c.
Exceptions:
- systemctl, because it has a lot hand-crafted coloring
- tmpfiles, sysusers, stdio-bridge, etc, because they are also used in
services and I'm not sure if this wouldn't mess up something.
We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed.
4 sp was the most common, in particular the majority of scripts under test/
used that. Let's standarize on 4 sp, because many commandlines are long and
there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp
also seems to be the default indentation, so this will make it less likely
that people will mess up if they don't load the editor config. (I think people
often use vi, and vi has no support to load project-wide configuration
automatically. We distribute a .vimrc file, but it is not loaded by default,
and even the instructions in it seem to discourage its use for security
reasons.)
Also remove the few vim config lines that were left. We should either have them
on all files, or none.
Also remove some strange stuff like '#!/bin/env bash', yikes.
The journal files might not be tiny hence let's write them to /var/tmp/
instead of /tmp. Also, let's turn on NOCOW on the files, as these tests
might apparently be slow on btrfs.
Fixes: #12210
systemd-journal-remote always wrote the boot-id of the device it was running on
to the header of its journal files. When the source had a different boot-id
(because it was generated on a different boot, or a different device), the
boot-ids in the file were inconsistent. The _BOOT_ID field was that of the
source, but the journal file header and each entry object header were that of
the device systemd-journal-remote ran on. This breaks journalctl --list-boots
on any of these files.
Set the boot-id in the header to be that of the source. This also fixes the
entry object headers.
Let's be helpful to static analyzers which care about whether we
knowingly ignore return values. We do in these cases, since they are
usually part of error paths.
Let's unify the two similar code paths to watch /run/systemd/journal.
The code in manager.c is similar, but it uses mkdir_p_label(), and unifying
that would be too much trouble, so let's just adjust the error messages to
be the same.
CID #1400224.
