Commit graph

2099 commits

Author SHA1 Message Date
Yu Watanabe 4bbccb02ea tree-wide: introduce strerror_safe() 2019-07-05 02:43:56 +09:00
Zbigniew Jędrzejewski-Szmek 62c6bbbc09 tree-wide: use PROJECT_FILE instead of __FILE__
This replaces the internal uses of __FILE__ with the new macro.
2019-07-04 10:36:00 +02:00
Zbigniew Jędrzejewski-Szmek 6ed5ef9819 meson: create /var/log/journal/{,remote/} conditionally
Not everybody has those dirs in the filesystem (and they don't need to).
When creating an installation package using $DESTDIR, it is easy enough to
remove or ignore those directories, but if installing into a real root, it
is ugly to create and remove them. Let's add an option so people can skip
it if they want.

Inspired by #12930.
2019-07-04 10:16:48 +02:00
Yu Watanabe 270384b2d4 tree-wide: replace strjoina() with prefix_roota() 2019-06-25 01:31:26 +09:00
Lennart Poettering cee97d5768
Merge pull request #12836 from yuwata/tree-wide-replace-strjoin
tree-wide: replace strjoin() with path_join()
2019-06-22 20:02:46 +02:00
Yu Watanabe 657ee2d82b tree-wide: replace strjoin() with path_join() 2019-06-21 03:26:16 +09:00
Lennart Poettering 64ef83139c
Merge pull request #12837 from yuwata/tree-wide-lgtm-fixes
tree-wide: fix issues found by lgtm
2019-06-20 12:35:34 +02:00
Yu Watanabe 6d946490ba tree-wide: drop alloca() in loop 2019-06-20 06:29:19 +09:00
Yu Watanabe aa89266900 util: introduce format_bytes_full()
And move it into format-util.c.
2019-06-19 23:15:19 +09:00
Lennart Poettering d9adc8a863 journald: use memdup_suffix0() when copying string from potentially binary data
Fixes: #12484
2019-06-18 20:53:46 +03:00
Zbigniew Jędrzejewski-Szmek 587694bcd5 journal: also disable memory tricks when hashing under msan
Might help with #11738.
2019-06-12 17:27:19 +02:00
Yu Watanabe 0bd3c2102f journalctl: make 'journalctl --flush' or friends not fail with varlink timeout
Closes #12570.
2019-06-04 23:27:26 +09:00
Yu Watanabe f2083c718b journalctl: mention --smart-relinquish-var in log message 2019-06-04 17:04:35 +09:00
Yu Watanabe 17087340c3 journalctl: fix error cause in log message
If varlink_call() returns negative errno, then `error` is null.
2019-06-04 17:03:36 +09:00
Zbigniew Jędrzejewski-Szmek 98d4a4dfb3 sd-journal: voidify call
To appease coverity, CID#1400674.
2019-05-30 23:05:31 +02:00
Yu Watanabe 627df1dc42 journal: use cleanup attribute at one more place 2019-05-28 18:07:18 +09:00
Yu Watanabe c377a6f3ad journal: do not trigger assertion when journal_file_close() get NULL
We generally expect destructors to not complain if a NULL argument is passed.

Closes #12400.
2019-05-28 18:07:18 +09:00
Lennart Poettering 05332e243c
Merge pull request #12590 from keszybz/unicode-cmdlines
Use unicode for cmdline printing
2019-05-24 10:41:30 +02:00
Lennart Poettering 5087825ea7 journald: output a proper error message when the journal is used on fs that doesn't do mmap() properly
Prompted by:

https://lists.freedesktop.org/archives/systemd-devel/2019-May/042708.html
2019-05-22 18:56:24 +02:00
Zbigniew Jędrzejewski-Szmek 09c1dceef1 basic/process-util: convert bool arg to flags
In preparation for the next commit…
2019-05-22 10:15:49 +02:00
Zbigniew Jędrzejewski-Szmek bc28751ed2 Rework cmdline printing to use unicode
The functions to retrieve and print process cmdlines were based on the
assumption that they contain printable ASCII, and everything else
should be filtered out. That assumption doesn't hold in today's world,
where people are free to use unicode everywhere.

This replaces the custom cmdline reading code with a more generic approach
using utf8_escape_non_printable_full().
For kernel threads, truncation is done on the parenthesized name, so we'll
get "[worker]", "[worker…]", …, "[w…]", "[…", "…" as we reduce the number of
available columns.

This implementation is most likely slower for very long cmdlines, but I don't
think this is very important. The common case is to have short commandlines,
and should print those properly. Absurdly long cmdlines are the exception,
which needs to be handled correctly and safely, but speed is not too important.

Fixes #12532.

v2:
- use size_t for the number of columns. This change propagates into various
  other functions that call get_process_cmdline(), increasing the size of the
  patch, but the changes are rather trivial.
2019-05-22 10:08:17 +02:00
Zbigniew Jędrzejewski-Szmek 21c491e106 Introduce sc_arg_max() helper
Just a cast and an assert.
2019-05-21 10:57:23 +02:00
Zbigniew Jędrzejewski-Szmek e6042f682f
Merge pull request #12535 from poettering/color-grey
redefine how we select the ANSI color grey
2019-05-16 21:15:06 +02:00
Frantisek Sumsal 6cda6774bd journalctl: return a non-zero EC when --grep returns no matches
When journalctl is compiled with PCRE2 support, let's return a non-zero
exit code when --grep is used and no match for given pattern is found.
This should allow users to use journalctl --grep in scripts instead of
piping journalctl into grep

Fixes #8152
2019-05-16 18:54:01 +02:00
Lennart Poettering af1060e1ab journalctl: don't output ANSI sequences if disabled 2019-05-10 15:00:55 -04:00
Zbigniew Jędrzejewski-Szmek 77740b5993 journalctl: improve error messages
Follow-up for #12230.
2019-05-10 16:55:37 +02:00
Zbigniew Jędrzejewski-Szmek 1d3fe304fd Use sd_event_source_disable_unref() 2019-05-10 16:55:37 +02:00
Lennart Poettering c0dfcb318c journalctl: add new --relinquish and --smart-relinquish options
The latter is identical to the former, but becomes a NOP if
/var/log/journal is on the same mount as /, and thus during shutdown
unmounting /var is not necessary and hence we can keep logging until the
very end.
2019-05-09 14:26:42 -04:00
Lennart Poettering b4e26d1d8e journald: add API to move logging from /var to /run again
We now have this nice little Varlink API, let's beef it up a bit.
2019-05-09 14:26:42 -04:00
Lennart Poettering 4f413af2a0 journalctl: port --flush/--sync/--rotate to use varlink method calls 2019-05-09 14:26:40 -04:00
Lennart Poettering 1ec23479e1 journald: also offer flush/rotate/sync as varlink method calls
This makes the operations previously available via asynchronous signals
also available as regular varlink method calls, i.e. with sane
completion.
2019-05-09 14:14:20 -04:00
Lennart Poettering e95be7def2
Merge pull request #12411 from keszybz/pr/12394
run: when emitting the calendarspec warning, use red
2019-05-08 10:11:32 -04:00
Zbigniew Jędrzejewski-Szmek 1a04395959 Enable log colors for most of tools in /usr/bin
When emitting the calendarspec warning we want to see some color.
Follow-up for 04220fda5c.

Exceptions:
- systemctl, because it has a lot hand-crafted coloring
- tmpfiles, sysusers, stdio-bridge, etc, because they are also used in
  services and I'm not sure if this wouldn't mess up something.
2019-05-08 09:50:21 +02:00
Ben Boeckel 5238e95759 codespell: fix spelling errors 2019-04-29 16:47:18 +02:00
Lennart Poettering e3d36a8da3 journald: modernize config_parse_compress() a bit 2019-04-12 14:23:12 +02:00
Lennart Poettering 337fabf749 journald: rebreak a few comments 2019-04-12 14:23:12 +02:00
Lennart Poettering e3d78cb160 journald: no need to check ptr for non-NULL before _unref(), as function does that anyway 2019-04-12 14:23:12 +02:00
Lennart Poettering e4d9c98512 journald: use structure initialization 2019-04-12 14:23:08 +02:00
Lennart Poettering 3661dc349e
Merge pull request #12217 from keszybz/unlocked-operations
Refactor how we do unlocked file operations
2019-04-12 13:51:53 +02:00
Zbigniew Jędrzejewski-Szmek 2fe21124a6 Add open_memstream_unlocked() wrapper 2019-04-12 11:44:57 +02:00
Zbigniew Jędrzejewski-Szmek cc5549ca12 scripts: use 4 space indentation
We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed.
4 sp was the most common, in particular the majority of scripts under test/
used that. Let's standarize on 4 sp, because many commandlines are long and
there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp
also seems to be the default indentation, so this will make it less likely
that people will mess up if they don't load the editor config. (I think people
often use vi, and vi has no support to load project-wide configuration
automatically. We distribute a .vimrc file, but it is not loaded by default,
and even the instructions in it seem to discourage its use for security
reasons.)

Also remove the few vim config lines that were left. We should either have them
on all files, or none.

Also remove some strange stuff like '#!/bin/env bash', yikes.
2019-04-12 08:30:31 +02:00
Lennart Poettering 4ff9bc2ea6 tree-wide: port users over to use new ERRNO_IS_ACCEPT_AGAIN() call 2019-04-10 22:11:18 +02:00
Lennart Poettering 949082ac21 test-journal: move tests to /var/tmp/ and set FS_NOCOW_FL
The journal files might not be tiny hence let's write them to /var/tmp/
instead of /tmp. Also, let's turn on NOCOW on the files, as these tests
might apparently be slow on btrfs.

Fixes: #12210
2019-04-04 12:18:13 +02:00
Frantisek Sumsal 1a862e2151 journal: LGTM doesn't recognize suppressions in /* */ 2019-04-02 12:47:14 +02:00
Chris Morin 924426a703 journal-remote: use source's boot-id
systemd-journal-remote always wrote the boot-id of the device it was running on
to the header of its journal files. When the source had a different boot-id
(because it was generated on a different boot, or a different device), the
boot-ids in the file were inconsistent. The _BOOT_ID field was that of the
source, but the journal file header and each entry object header were that of
the device systemd-journal-remote ran on. This breaks journalctl --list-boots
on any of these files.

Set the boot-id in the header to be that of the source. This also fixes the
entry object headers.
2019-04-02 10:32:21 +02:00
Zbigniew Jędrzejewski-Szmek f2dc22b447 headers: add missing includes
Fixes #12125.
2019-03-28 19:59:56 +01:00
Lennart Poettering 6990fb6bc6 tree-wide: (void)ify a few unlink() and rmdir()
Let's be helpful to static analyzers which care about whether we
knowingly ignore return values. We do in these cases, since they are
usually part of error paths.
2019-03-27 18:09:56 +01:00
Zbigniew Jędrzejewski-Szmek 330b8fb379 journalctl: voidify mkdir_p() call and unify two similar code paths
Let's unify the two similar code paths to watch /run/systemd/journal.
The code in manager.c is similar, but it uses mkdir_p_label(), and unifying
that would be too much trouble, so let's just adjust the error messages to
be the same.

CID #1400224.
2019-03-27 13:01:44 +01:00
Zbigniew Jędrzejewski-Szmek 6e5dcce4b1 Voidify more mkdir_p calls 2019-03-27 11:53:12 +01:00
Zbigniew Jędrzejewski-Szmek ca78ad1de9 headers: remove unneeded includes from util.h
This means we need to include many more headers in various files that simply
included util.h before, but it seems cleaner to do it this way.
2019-03-27 11:53:12 +01:00
Chris Morin 08f9e80b3f journal-file: handle SIGBUS on offlining thread
The thread launched in journal_file_set_offline() accesses a memory
mapped file, so it needs to handle SIGBUS. Leave SIGBUS unblocked on the
offlining thread so that it uses the same handler as the main thread.

The result of triggering SIGBUS in a thread where it's blocked is
undefined in Linux. The tested implementations were observed to cause
the default handler to run, taking down the whole journald process.

We can leave SIGBUS unblocked in multiple threads since it's handler is
thread-safe. If SIGBUS is sent to the journald process asynchronously
(i.e. with kill, sigqueue, or raise), either thread handling it will
result in the same behavior: it will install the default handler and
reraise the signal, killing the process.

Fixes: #12042
2019-03-20 13:02:04 +01:00
dana 4890482531 journalctl: support -b all to negate effect of -b
Also fix an issue where -b without argument didn't always behave as -b0
2019-03-19 10:48:26 +01:00
Lennart Poettering d8b4d14df4 util: split out nulstr related stuff to nulstr-util.[ch] 2019-03-14 13:25:52 +01:00
Lennart Poettering 2b2fec7db0 util: split out errno related stuff 2019-03-14 13:25:51 +01:00
Lennart Poettering 760877e90c util: split out sorting related calls to new sort-util.[ch] 2019-03-13 12:16:43 +01:00
Lennart Poettering 0a9707187b util: split out memcmp()/memset() related calls into memory-util.[ch]
Just some source rearranging.
2019-03-13 12:16:43 +01:00
Lennart Poettering 0c0e87fcd5 catalog: _cleanup_close_ is your friend 2019-03-07 15:10:06 +01:00
Lennart Poettering 3d477d6101 catalog: (void)ify all things 2019-03-07 15:10:06 +01:00
Lennart Poettering 5432d8a187 catalog: use structured initialization more 2019-03-07 15:10:06 +01:00
Lennart Poettering 3f1e2714a4 catalog: let's use mkdir_parents() and use same mode as tmpfiles.d for this dir
let's use mkdir_parents() (because its shorter), and 0755 as access
mode, so that things have the access mode tmpfiles.d also suggests.

Prompted by: #11903
2019-03-07 15:09:21 +01:00
Lennart Poettering 0640d48afb catalog: be a bit less aggressive with line breaking. After all 109ch is what we going for 2019-03-07 14:46:35 +01:00
Lennart Poettering bbe804325e catalog: don't make catalog_entry_lang() clobber output params on failure
A minor fix to bring this in-line with our coding style
2019-03-07 14:46:35 +01:00
Jörg Sommer d9e15cbd18 journalctl: New option --cursor-file
The option cursor-file takes a filename as argument. If the file exists and
contains a valid cursor, this is used to start the output after this position.
At the end, the last cursor gets written to the file.

This allows for an easy implementation of a timer that regularly looks in the
journal for some messages.

    journalctl --cursor-file err-cursor -b -p err
    journalctl --cursor-file audit-cursor -t audit --grep DENIED

Or you might want to walk the journal in steps of 10 messages:

    journalctl --cursor-file ./curs -n10 --since=today -t systemd
2019-02-27 18:43:31 +01:00
Lennart Poettering eb7e351496
Merge pull request #11578 from keszybz/gcc-9-fixes
Packed struct alignment workarounds for gcc-9
2019-02-07 15:25:05 +01:00
Zbigniew Jędrzejewski-Szmek 62353f69cc journal: drop _packed_ attribute in a few places
The justification is the safe as for the grandparent commit.
2019-02-04 20:32:44 +01:00
Zbigniew Jędrzejewski-Szmek 00e1adf8b1 journal: avoid buffer overread when locale name is too long
We could potentially create an unterminated string and then call normal string
operations on it. Let's be more careful: first remove the suffix we ignore anyway,
then find if the string is of acceptable length, and possibly ignore it if it
is too long. The code rejects lengths above 31 bytes. Language names that are
actually used are much shorter, so this doesn't matter much.
2019-01-27 18:34:15 +01:00
Lennart Poettering 6e9417f5b4 tree-wide: use newa() instead of alloca() wherever we can
Typesafety is nice. And this way we can take benefit of the new size
assert() the previous commit added.
2019-01-26 16:17:04 +01:00
Zbigniew Jędrzejewski-Szmek 91714a7f42 journald: periodically drop cache for all dead PIDs
In normal use, this allow us to drop dead entries from the cache and reduces
the cache size so that we don't evict entries unnecessarily. The time limit is
there mostly to serve as a guard against malicious logging from many different
PIDs.
2019-01-26 12:43:27 +01:00
Zbigniew Jędrzejewski-Szmek b12a480829 journal: limit the number of entries in the cache based on available memory
This is far from perfect, but should give mostly reasonable values. My
assumption is that if somebody has a few hundred MB of memory, they are
unlikely to have thousands of processes logging. A hundred would already be a
lot. So let's scale the cache size propritionally to the total memory size,
with clamping on both ends.

The formula gives 64 cache entries for each GB of RAM.
2019-01-26 12:42:52 +01:00
Topi Miettinen a1e92eee3e Remove 'inline' attributes from static functions in .c files (#11426)
Let the compiler perform inlining (see #11397).
2019-01-15 08:12:28 +01:00
Topi Miettinen 7ae3561a5a Delete duplicate lines
Found by inspecting results of running this small program:

int main(int argc, const char **argv) {
	for (int i = 1; i < argc; i++) {
		FILE *f;
		char line[1024], prev[1024], *r;
		int lineno;

		prev[0] = '\0';
		lineno = 1;
		f = fopen(argv[i], "r");
		if (!f)
			exit(1);
		do {
			r = fgets(line, sizeof(line), f);
			if (!r)
				break;
			if (strcmp(line, prev) == 0)
				printf("%s:%d: error: dup %s", argv[i], lineno, line);
			lineno++;
			strcpy(prev, line);
		} while (!feof(f));
		fclose(f);
	}
}
2019-01-12 16:02:26 +01:00
Zbigniew Jędrzejewski-Szmek 6670c9de19 journald: lower the maximum entry size limit to ½ for non-sealed fds
We immediately read the whole contents into memory, making thigs much more
expensive. Sealed fds should be used instead since they are more efficient
on our side.
2019-01-09 23:41:53 +01:00
Zbigniew Jędrzejewski-Szmek 964ef920ea journald: when processing a native message, bail more quickly on overbig messages
We'd first parse all or most of the message, and only then consider if it
is not too large. Also, when encountering a single field over the limit,
we'd still process the preceding part of the message. Let's be stricter,
and check size limits early, and let's refuse the whole message if it fails
any of the size limits.
2019-01-09 23:41:53 +01:00
Zbigniew Jędrzejewski-Szmek 052c57f132 journald: set a limit on the number of fields (1k)
We allocate a iovec entry for each field, so with many short entries,
our memory usage and processing time can be large, even with a relatively
small message size. Let's refuse overly long entries.

CVE-2018-16865
https://bugzilla.redhat.com/show_bug.cgi?id=1653861

What from I can see, the problem is not from an alloca, despite what the CVE
description says, but from the attack multiplication that comes from creating
many very small iovecs: (void* + size_t) for each three bytes of input message.
2019-01-09 23:41:53 +01:00
Zbigniew Jędrzejewski-Szmek 084eeb865c journald: do not store the iovec entry for process commandline on stack
This fixes a crash where we would read the commandline, whose length is under
control of the sending program, and then crash when trying to create a stack
allocation for it.

CVE-2018-16864
https://bugzilla.redhat.com/show_bug.cgi?id=1653855

The message actually doesn't get written to disk, because
journal_file_append_entry() returns -E2BIG.
2019-01-09 23:41:53 +01:00
Zbigniew Jędrzejewski-Szmek bc2762a309 journald: remove unnecessary {} 2019-01-09 23:41:53 +01:00
Michael Sloan a08c3e8f25 cat: add --stderr-priority option 2019-01-08 10:37:20 +01:00
Frantisek Sumsal 4a2c3dc318
Merge pull request #11252 from evverx/use-asan-wrapper-on-travis-ci
travis: run PID1, journald and everything else under ASan+UBsan
2019-01-06 18:48:38 +01:00
Chris Down 4e1dfa45e9 cgroup: s/cgroups? ?v?([0-9])/cgroup v\1/gI
Nitpicky, but we've used a lot of random spacings and names in the past,
but we're trying to be completely consistent on "cgroup vN" now.

Generated by `fd -0 | xargs -0 -n1 sed -ri --follow-symlinks 's/cgroups?  ?v?([0-9])/cgroup v\1/gI'`.

I manually ignored places where it's not appropriate to replace (eg.
"cgroup2" fstype and in src/shared/linux).
2019-01-03 11:32:40 +09:00
Evgeny Vereshchagin ef30f7cac1 journal: rely on _cleanup_free_ to free a temporary string used in client_context_read_cgroup
Closes https://github.com/systemd/systemd/issues/11253.
2018-12-29 19:14:35 +01:00
Zbigniew Jędrzejewski-Szmek cd2a429ed7 tree-wide: use assert_se() for signal operations with constants
Continuation of a3ebe5eb620e49f0d24082876cafc7579261e64f:
in other places we sometimes use assert_se(), and sometimes normal error
handling. sigfillset and sigaddset can only fail if mask is NULL (which cannot
happen if we are passing in a reference), or if the signal number is invalid
(which really shouldn't happen when we are using a constant like SIGCHLD. If
SIGCHLD is invalid, we have a bigger problem). So let's simplify things and
always use assert_se() in those cases.

In sigset_add_many() we could conceivably pass an invalid signal, so let's keep
normal error handling here. The caller can do assert_se() around the
sigprocmask_many() call if appropriate.

'>= 0' is used for consistency with the rest of the codebase.
2018-12-21 19:49:28 +01:00
Michal Sekletar 672773b63a journald: correctly attribute log messages also with cgroupsv1
With cgroupsv1 a zombie process is migrated to root cgroup in all
hierarchies. This was changed for unified hierarchy and /proc/PID/cgroup
reports cgroup to which process belonged before it exited.

Be more suspicious about cgroup path reported by the kernel and use
unit_id provided by the log client if the kernel reports that process is
running in the root cgroup.

Users tend to care the most about 'log->unit_id' mapping so systemctl
status can correctly report last log lines. Also we wouldn't be able to
infer anything useful from "/" path anyway.

See: 2e91fa7f6d
2018-12-17 15:16:11 +01:00
Yu Watanabe 18bf324ab5 journal: use missing_audit.h to generate audit_type-list.txt 2018-12-07 11:48:37 +09:00
Yu Watanabe f2a3de0116 tree-wide: add whitespace between type and variable name 2018-12-04 09:29:54 +01:00
Lennart Poettering 63e688cc3b
Merge pull request #11031 from poettering/gcc-attr-cleanup
various gcc attribute clean-ups
2018-12-03 21:59:00 +01:00
Lennart Poettering 76b31bbb24
Merge pull request #10920 from yuwata/hashmap-destructor
hashmap: make hashmap_free() call destructors of key or value
2018-12-03 17:59:44 +01:00
Lennart Poettering 0df54921bc tree-wide: use new macro HAS_FEATURE_ADDRESS_SANITIZER everywhere 2018-12-03 17:38:50 +01:00
Lennart Poettering 686d13b9f2 util-lib: split out env file parsing code into env-file.c
It's quite complex, let's split this out.

No code changes, just some file rearranging.
2018-12-02 13:22:29 +01:00
Lennart Poettering e4de72876e util-lib: split out all temporary file related calls into tmpfiles-util.c
This splits out a bunch of functions from fileio.c that have to do with
temporary files. Simply to make the header files a bit shorter, and to
group things more nicely.

No code changes, just some rearranging of source files.
2018-12-02 13:22:29 +01:00
Yu Watanabe 7a08d314f2 tree-wide: make hash_ops typesafe 2018-12-02 07:53:27 +01:00
Zbigniew Jędrzejewski-Szmek 652ef29887 path-util: allow NULLs in arguments to path_join()
This removes the need to remember to put strempty() in places, thus reducing
the likelihood of a stupid mistake.
2018-11-30 22:21:17 +01:00
Zbigniew Jędrzejewski-Szmek 62a85ee0a9 tree-wide: rename path_join_many() to path_join()
$ git grep -e path_join_many -l|xargs sed -r -i 's/path_join_many/path_join/g'

The two test functions are merged into one.
2018-11-30 10:59:47 +01:00
Zbigniew Jędrzejewski-Szmek 30016f21b3 tree-wide: replace path_join with path_join_many 2018-11-30 10:40:38 +01:00
Evgeny Vereshchagin c90c39ff7b catalog: reject entries where the language is too short early
Closes https://oss-fuzz.com/testcase-detail/5674475278827520
2018-11-29 13:41:40 +09:00
Lennart Poettering 5cfa2c3dc0 tree-wide: use IOVEC_MAKE() at many places 2018-11-27 10:12:27 +01:00
Lennart Poettering 30acbadc6a journald: don't try to rotate user journals in /var/log/journal/ if we are still in log-to-runtime-journal mode
Fixes: #10879
2018-11-27 11:09:15 +09:00
Lennart Poettering da9fc98ded tree-wide: port more code over to PATH_STARTSWITH_SET() 2018-11-26 14:08:46 +01:00
Lennart Poettering 0cbd293e12 tree-wide: port over more cases to STR_IN_SET() 2018-11-26 14:08:46 +01:00
Zbigniew Jędrzejewski-Szmek baaa35ad70 coccinelle: make use of SYNTHETIC_ERRNO
Ideally, coccinelle would strip unnecessary braces too. But I do not see any
option in coccinelle for this, so instead, I edited the patch text using
search&replace to remove the braces. Unfortunately this is not fully automatic,
in particular it didn't deal well with if-else-if-else blocks and ifdefs, so
there is an increased likelikehood be some bugs in such spots.

I also removed part of the patch that coccinelle generated for udev, where we
returns -1 for failure. This should be fixed independently.
2018-11-22 10:54:38 +01:00
Zbigniew Jędrzejewski-Szmek 294bf0c34a Split out pretty-print.c and move pager.c and main-func.h to shared/
This is high-level functionality, and fits better in shared/ (which is for
our executables), than in basic/ (which is also for libraries).
2018-11-20 18:40:02 +01:00
Zbigniew Jędrzejewski-Szmek a03d43593c journal: fix sort order of header includes 2018-11-20 07:27:37 +01:00
Yu Watanabe 278939be7a
Merge pull request #10847 from evverx/journald-stream-fuzzer
tests: add a fuzzer for journald streams
2018-11-20 13:37:27 +09:00
Evgeny Vereshchagin 9541f5ff5c tests: add a fuzzer for journald streams 2018-11-20 03:03:32 +01:00
Lennart Poettering 5e332028f2 util-lib: move main() definition macros to its own header file
This way, we can extend the macro a bit with stuff pulled in from other
headers without this affecting everything which pulls in macro.h, which
is one of our most basic headers.

This is just refactoring, no change in behaviour, in prepartion for
later changes.
2018-11-19 21:14:34 +01:00
Lennart Poettering 2a44bf5099
Merge pull request #10811 from keszybz/define-main-through-macro
Define main through macro
2018-11-19 15:28:17 +01:00
Evgeny Vereshchagin 1dab14aba7 journald: check whether sscanf has changed the value corresponding to %n
It's possible for sscanf to receive strings containing all three fields
and not matching the template at the same time. When this happens the
value of k doesn't change, which basically means that process_audit_string
tries to access memory randomly. Sometimes it works and sometimes it doesn't :-)

See also https://bugzilla.redhat.com/show_bug.cgi?id=1059314.
2018-11-17 11:25:19 +01:00
Zbigniew Jędrzejewski-Szmek 634a0ad3e5 cat: define main through macro 2018-11-17 09:13:35 +01:00
Evgeny Vereshchagin 090a20cfaf tests: add a fuzzer for process_audit_string 2018-11-16 23:14:31 +01:00
Lennart Poettering 567e5d5549
Merge pull request #10791 from keszybz/sd-event-get-enabled
Some minor tweaks for sd_event_source_get_enabled
2018-11-16 12:46:30 +01:00
Zbigniew Jędrzejewski-Szmek b6cdfbe5c4 journal: simplify use of sd_event_source_get_enabled() 2018-11-16 09:03:41 +01:00
Zbigniew Jędrzejewski-Szmek ca5d90d4d9 journal-file: get rid of a helper variable
It doesn't really save much in code length. Having the event source named
explicitly makes it easier to understand the code at a glance.
2018-11-16 09:03:41 +01:00
Evgeny Vereshchagin 080d112caa journal: fix an off-by-one error in dev_kmsg_record 2018-11-16 08:45:16 +01:00
Evgeny Vereshchagin 8857fb9beb tests: add a fuzzer for dev_kmsg_record 2018-11-16 08:44:35 +01:00
Zbigniew Jędrzejewski-Szmek cd5a29ce98
Merge pull request #10742 from poettering/c-utf8
default to C.UTF-8 locale, and many improvements to env var file parsing/kernel cmdline parsing
2018-11-15 12:47:17 +01:00
Lennart Poettering 13df9c398d fileio: automatically add NULL sentinel to parse_env_file()
Let's modernize things a bit.
2018-11-14 17:01:55 +01:00
Lennart Poettering aa8fbc74e3 fileio: drop "newline" parameter for env file parsers
Now that we don't (mis-)use the env file parser to parse kernel command
lines there's no need anymore to override the used newline character
set. Let's hence drop the argument and just "\n\r" always. This nicely
simplifies our code.
2018-11-14 17:01:54 +01:00
Zbigniew Jędrzejewski-Szmek fde32028a4 Move LONG_LINE_MAX definition to fileio.h
All users of the macro (except for one, in serialize.c), use the macro in
connection with read_line(), so they must include fileio.h.  Let's not play
libc games and require multiple header file to be included for the most common
use of a function.

The removal of def.h includes is not exact. I mostly went over the commits that
switch over to use read_line() and add def.h at the same time and reverted the
addition of def.h in those files.
2018-11-14 16:25:32 +01:00
Zbigniew Jędrzejewski-Szmek 0221d68a13 basic/pager: convert the pager options to a flags argument
Pretty much everything uses just the first argument, and this doesn't make this
common pattern more complicated, but makes it simpler to pass multiple options.
2018-11-14 16:25:11 +01:00
Zbigniew Jędrzejewski-Szmek 8e143a1232 journalctl: do not treat EINTR as an error when waiting for events
Fixup for 2a1e0f2228. Fixes #10724.

Reproducer: start 'journalctl -f' in a terminal window, change window size.
2018-11-11 12:40:27 +01:00
Zbigniew Jędrzejewski-Szmek 2f5ed71556 Drop copyright for Marius Vollmer
Ack in #9320.
2018-10-31 09:34:38 +01:00
Lennart Poettering 8cb17a64c4
Merge pull request #10563 from keszybz/lz4-quickfix-quickfix
journal: adapt for new improved LZ4_decompress_safe_partial()
2018-10-30 14:51:55 +01:00
Zbigniew Jędrzejewski-Szmek e41ef6fd00 journal: adapt for new improved LZ4_decompress_safe_partial()
With lz4 1.8.3, this function can now decompress partial results into a smaller
buffer. The release news don't say anything interesting, but the test case that
was previously failing now works OK.

Fixes #10259.

A test is added. It shows that with *older* lz4, a partial decompression can
occur with the returned size smaller then the requested number of bytes _and_
smaller then the size of the compressed data:

(lz4-libs-1.8.2-1.fc29.x86_64)
Compressed 4194304 → 16464
Decompressed → 4194304
Decompressed partial 12/4194304 → 4194304
Decompressed partial 1/1 → -2 (bad)
Decompressed partial 2/2 → -2 (bad)
Decompressed partial 3/3 → -2 (bad)
Decompressed partial 4/4 → -2 (bad)
Decompressed partial 5/5 → -2 (bad)
Decompressed partial 6/6 → 6 (good)
Decompressed partial 7/7 → 6 (good)
Decompressed partial 8/8 → 6 (good)
Decompressed partial 9/9 → 6 (good)
Decompressed partial 10/10 → 6 (good)
Decompressed partial 11/11 → 6 (good)
Decompressed partial 12/12 → 6 (good)
Decompressed partial 13/13 → 6 (good)
Decompressed partial 14/14 → 6 (good)
Decompressed partial 15/15 → 6 (good)
Decompressed partial 16/16 → 6 (good)
Decompressed partial 17/17 → 6 (good)
Decompressed partial 18/18 → -16459 (bad)

(lz4-libs-1.8.3-1.fc29.x86_64)
Compressed 4194304 → 16464
Decompressed → 4194304
Decompressed partial 12/4194304 → 12
Decompressed partial 1/1 → 1 (good)
Decompressed partial 2/2 → 2 (good)
Decompressed partial 3/3 → 3 (good)
Decompressed partial 4/4 → 4 (good)
...

If we got such a short "successful" decompression in decompress_startswith() as
implemented before this patch, we could be confused and return a false negative
result. But it turns out that this only occurs with small output buffer
sizes. We use greedy_realloc() to manager the buffer, so it is always at least
64 bytes. I couldn't hit a case where decompress_startswith() would actually
return a bogus result. But since the lack of proof is not conclusive, the code
for *older* lz4 is changed too, just to be safe. We cannot rule out that on a
different architecture or with some unlucky compressed string we could hit this
corner case.

The fallback code is guarded by a version check. The check uses a function not
the compile-time define, because there was no soversion bump in lz4 or new
symbols, and we could be compiled against a newer lz4 and linked at runtime
with an older one. (This happens routinely e.g. when somebody upgrades a subset
of distro packages.)
2018-10-30 11:04:51 +01:00
Zbigniew Jędrzejewski-Szmek ba17efce44 test-compress: add test for short decompress_startswith calls
I thought this might fail with lz4 < 1.8.3, but it seems that because of
greedy_realloc, we always use a buffer that is large enough, and it always
passes.
2018-10-30 09:28:51 +01:00
Zbigniew Jędrzejewski-Szmek e0a1d4b049 Drop support for lz4 < 1.3.0
lz4-r130 was released on May 29th, 2015. Let's drop the work-around for older
versions. In particular, we won't test any new code against those ancient
releases, so we shouldn't pretend they are supported.
2018-10-29 21:54:42 +01:00
Evgeny Vereshchagin e0f768c356 journal: drop an unused variable clang is complaining about
../../src/systemd/src/journal/journal-file.c:3592:30: warning: unused variable 'p' [-Wunused-variable]
        _cleanup_free_ char *p = NULL;
                             ^
1 warning generated.

This is a follow-up to 6812765891.
2018-10-29 15:21:58 +00:00
Lennart Poettering 2a1e0f2228 journalctl: in --follow mode watch stdout for POLLHUP/POLLERR and exit
Fixes: #9374
2018-10-26 12:28:38 +02:00
Lennart Poettering 971b52c485 journal-file: structured initialization is your friend 2018-10-25 21:44:48 +02:00
Lennart Poettering a33687b792 journald: when we are asked to rotate all files, let's also look at closed files
Before this when asked for rotation we'd only rotate files we have open
anyway. However there might be a number of other files on disk that are
active (i.e. not archived yet) but not open. Let's take care of those
too, so that rotation is always comprehensive, and the user gets the
guarantee that afterthe rotation all stored data is in archived files.

Fixes: #1017
2018-10-25 21:44:48 +02:00
Lennart Poettering e5b2d45c1d journalctl: fix error number in error message 2018-10-25 21:44:48 +02:00
Lennart Poettering 8df64fd01d journalctl: add ability to vacuum and rotate in one step
journalctl --vacuum-*= only vacuums archived files. To archive all
active files the rotate operation is used. Let's add a new switch that
combines both, so that the user a single command to first move all
running journal files into archival and then vacuum them.

See: #1017
2018-10-25 21:44:48 +02:00
Lennart Poettering d03077759e journald: log about an OOM condition 2018-10-25 21:44:48 +02:00
Lennart Poettering e859154447 journald: debug log when we cannot read the machine ID 2018-10-25 21:44:48 +02:00
Lennart Poettering 6812765891 journal-file: refactor journal_file_open_reliably()
Let's split out the part that actually renames the file in case we can't
open it into a new function journal_file_dispose().

This way we can reuse the function in other cases where we want to open
a file but can't.
2018-10-25 21:43:09 +02:00
Lennart Poettering 7a4d21ad20 journal-file: refactor journal_file_rotate()
Let's split the function in three: the part where we archive the old
file into journal_file_archive(), and the part where we initiate the
deferred closing into journal_file_initiate_close().
journal_file_rotate() then simply becomes a wrapper around these two
calls, and the opening of the new journal file.

This useful so that we can archive journal files without having to open
new ones, i.e. to do only the archival part of the rotation, without the
rotation part.
2018-10-25 21:43:09 +02:00
Lennart Poettering f760d8a858 journal: refactor out loop that processes deferred closes into its own function 2018-10-25 21:43:09 +02:00
Lennart Poettering 180e7f4e62 journal: fix some type confusion in journal_directory_vacuum()
Let's store array sizes and indexes in size_t. And let's count numbers
of files in uint64_t (simply because that is the type of the input
parameter for this of the function)
2018-10-25 21:43:09 +02:00
Lennart Poettering ab41da087d journald: use structured initialization 2018-10-25 21:42:05 +02:00
Lennart Poettering 6761e0a2fd journald: use usec_sub_unsigned() where we can 2018-10-25 21:42:05 +02:00
Evgeny Vereshchagin c8758e726c journald: use the "driver" transport for logging a refused connection
Otherwise, these messages can end up in `/dev/null` when journald is run
in containers as was shown in https://github.com/systemd/systemd/pull/10444#issuecomment-430927793.
2018-10-23 19:36:52 +02:00
Evgeny Vereshchagin 9e8b1ec08e sd-journal: escape binary data in match_make_string()
Fixes: #10383
2018-10-23 19:33:01 +02:00
Lennart Poettering 490c5a37cb tree-wide: some automatic coccinelle fixes (#10463)
Nothing fancy, just coccinelle doing its work.
2018-10-20 00:07:46 +09:00
Yu Watanabe 218b4609d0
Merge pull request #10447 from poettering/fgets-excorcism
let's get rid of fgets()
2018-10-19 08:36:01 +09:00
Lennart Poettering 2ff48e981e tree-wide: introduce setsockopt_int() helper and make use of it everywhere
As suggested by @heftig:

6d5e65f645 (commitcomment-30938667)
2018-10-18 19:50:29 +02:00
Lennart Poettering df2b06d1c2 catalog: fgets() excorcism 2018-10-18 13:39:29 +02:00
Anita Zhang 90fc172e19 core: implement per unit journal rate limiting
Add LogRateLimitIntervalSec= and LogRateLimitBurst= options for
services. If provided, these values get passed to the journald
client context, and those values are used in the rate limiting
function in the journal over the the journald.conf values.

Part of #10230
2018-10-18 09:56:20 +02:00
Lennart Poettering 8aeb1d3176
Merge pull request #10244 from poettering/nofile-bump
bump RLIMIT_NOFILE
2018-10-17 17:59:36 +02:00
Yu Watanabe 90c88092e6 tree-wide: use CMP() macro where applicable
Follow-up for 6dd91b3682.
2018-10-16 19:55:38 +02:00
Yu Watanabe 8da2f9e807
Merge pull request #10412 from poettering/sockaddr-sun-path
various fixes related to struct sockaddr_un handling
2018-10-17 01:25:58 +09:00
Lennart Poettering 6dd91b3682 tree-wide: CMP()ify all the things
Let's employ coccinelle to fix everything up automatically for us.
2018-10-16 17:45:53 +02:00
Lennart Poettering 1abaf4887d tree-wide: uniformly bump RLIMIT_NOFILE in all our tools that access the journal
This makes use of rlimit_nofile_bump() in all tools that access the
journal. In some cases this replaces older code to achieve this, and
others we add it in where it was missing.
2018-10-16 16:33:55 +02:00
Lennart Poettering a53f90ca24 siphash24: add helper for calculating the hash value for a string
Let's shorten some code.
2018-10-16 16:27:15 +02:00
Lennart Poettering 6d5e65f645 tree-wide: add a single version of "static const int one = 1"
All over the place we define local variables for the various sockopts
that take a bool-like "int" value. Sometimes they are const, sometimes
static, sometimes both, sometimes neither.

Let's clean this up, introduce a common const variable "const_int_one"
(as well as one matching "const_int_zero") and use it everywhere, all
acorss the codebase.
2018-10-15 19:40:51 +02:00
Lennart Poettering 15a3e96f92 tree-wide: port various users over to sockaddr_un_set_path()
CID 1396140
CID 1396141
2018-10-15 19:40:51 +02:00
Lennart Poettering 155b687669 tree-wide: use sockaddr_un_unlink() whereever appropriate
Let's port everything over.
2018-10-15 19:35:00 +02:00
Lennart Poettering 48440643f7 journal: voidify fd_nonblock()
CID #1396098
CID #1396096
CID #1396091
CID #1396086
2018-10-13 12:37:13 +02:00
Lennart Poettering a6ee956610
Merge pull request #10356 from dtardon/covscan
assorted coverity/clang fixes
2018-10-12 18:43:04 +02:00
David Tardon c52368509f journal-file: avoid calling ftruncate with invalid fd
This can happen if journal_file_close is called from the failure
handling code of journal_file_open before f->fd was established.
2018-10-12 14:51:35 +02:00
Lennart Poettering 8e04444385 journalctl: port JSON output mode to new JSON API
Also, while we are at it, beef it up, by adding json-seq support (i.e.
https://tools.ietf.org/html/rfc7464). This is particularly useful in
conjunction with jq's --seq switch.
2018-10-11 17:25:27 +02:00
Yu Watanabe 9c57a73b13 tree-wide: use CMP() macros where applicable 2018-10-09 14:45:55 +02:00
Lennart Poettering db9a42545a chattr: optionally, return the old flags when updating them 2018-10-08 21:40:44 +02:00
Zbigniew Jędrzejewski-Szmek e80ef269a1
Merge pull request #10255 from poettering/hide-new-id128
drop references to "journalctl --new-id128"
2018-10-02 17:54:36 +02:00
Lennart Poettering 29088d374e
Merge pull request #9898 from keszybz/id128
Add a new tool 'systemd-id128'
2018-10-02 17:43:37 +02:00
Lennart Poettering 19a073db9b journalctl: drop --new-id128 from help and man texts
Let's remove redundancy and not advertise "journalctl --new-id128"
anymore, now that we have "systemd-id128 new" in a proper tool.

This allows us to reduce the overly large journalctl command set a bit.

Note that this just removes the --help and man text, the call remains
available for compat reasons.
2018-10-02 16:42:30 +02:00
Zbigniew Jędrzejewski-Szmek 0d1d512f7f systemd-id128: a new tool to print machine/boot/invocation/app-specific ids
The raison d'etre for this program is printing machine-app-specific IDs. We
provide a library function for that, but not a convenient API. We can hardly
ask people to quickly hack their own C programs or call libsystemd through CFFI
in python or another scripting language if they just want to print an ID.

Verb 'new' was already available as 'journalctl --new-id128', but this makes
it more discoverable.

v2:
- rename binary to systemd-id128
- make --app-specific= into a switch that applies to boot-id and machine-id
2018-10-02 15:15:10 +02:00
Zbigniew Jędrzejewski-Szmek ff7dad484c journalctl: move generate_new_id128() to shared 2018-10-02 15:13:17 +02:00
Zbigniew Jędrzejewski-Szmek 0a587335d2 journal-verify: add comment and silence LGTM warning 2018-10-02 12:54:00 +02:00
Zbigniew Jędrzejewski-Szmek 6f48815918 test-journal-syslog: initialize variable
The *priority argument to syslog_parse_priority() needs to be initialized
if the last argument (with_facility) is false.

CID #1394690.
2018-09-25 11:34:52 +02:00
Yu Watanabe 93bab28895 tree-wide: use typesafe_qsort() 2018-09-19 08:02:52 +09:00
Yu Watanabe 0ac4f1a513
Merge pull request #10070 from keszybz/test-reporting
Test reporting improvements
2018-09-15 15:55:15 +09:00
Yu Watanabe 3332004e28 test-catalog: drop unused variable
Follow-up for 49cdae63d1.
2018-09-15 01:59:14 +09:00
Zbigniew Jędrzejewski-Szmek 6d7c403324 tests: use a helper function to parse environment and open logging
The advantages are that we save a few lines, and that we can override
logging using environment variables in more test executables.
2018-09-14 09:29:57 +02:00
Zbigniew Jędrzejewski-Szmek 317bb217d3 tests: add helper to unify skipping a test and exiting 2018-09-14 09:29:57 +02:00
Yu Watanabe 964bc0ad60 test: log when skipping tests in more cases
Follow-up for the previous commit.
2018-09-13 12:07:34 +02:00
Zbigniew Jędrzejewski-Szmek 0cf29baac0 tests: centralize check for slow tests 2018-09-13 12:07:27 +02:00
Filipe Brandenburger 49cdae63d1 test: use ${builddir}/systemd-runtest.env for $SYSTEMD_CATALOG_DIR
This makes it so that tests no longer need to know the absolute paths to the
source and build dirs, instead using the systemd-runtest.env file to get these
paths when running from the build tree.

Confirmed that test-catalog works on `ninja test`, when called standalone and
also when the environment file is not present, in which case it will use the
installed location under /usr/lib/systemd/catalog.

The location can now also be overridden for this test by setting the
$SYSTEMD_CATALOG_DIR environment variable.
2018-09-12 09:49:03 -07:00
Yu Watanabe d9b6baa699 test: make test-catalog relocatable
Fixes #10045.
2018-09-11 09:19:36 +09:00
Evgeny Vereshchagin 21acb27b71 journald: make server_process_native_message compatible with fuzz_journald_processing_function 2018-09-03 07:03:52 +00:00
Renaud Métrich fd790d6f09 journald: fixed assertion failure when system journal rotation fails (#9893) 2018-09-03 12:42:39 +09:00
Yu Watanabe 7a3c343cb8 tree-wide: use '#pragma once' for header guard
Follow-up for a2b635eb39 (#9959).
2018-08-30 06:10:43 +03:00
Yu Watanabe 8301aa0bf1 tree-wide: use DEFINE_TRIVIAL_REF_UNREF_FUNC() macro or friends where applicable 2018-08-27 14:01:46 +09:00
Yu Watanabe cf4b2f9906 tree-wide: use unsigned for refcount 2018-08-27 13:48:04 +09:00
Yu Watanabe 8bebb09ce7 journal: replace udev_device by sd_device 2018-08-23 04:57:39 +09:00
Lennart Poettering 37ec0fdd34 tree-wide: add clickable man page link to all --help texts
This is a bit like the info link in most of GNU's --help texts, but we
don't do info but man pages, and we make them properly clickable on
terminal supporting that, because awesome.

I think it's generally advisable to link up our (brief) --help texts and
our (more comprehensive) man pages a bit, so this should be an easy and
straight-forward way to do it.
2018-08-20 11:33:04 +02:00
Evgeny Vereshchagin 937b117137 journald: take leading spaces into account in syslog_parse_identifier
This is a kind of follow-up to e88baee88f which should finally fix
the issue which that commit was supposed to fix.
2018-08-10 13:35:06 +00:00
Evgeny Vereshchagin 30eddcd51b journald: free the allocated memory before returning from dev_kmsg_record
This fixes a minor memory leak.
2018-08-10 13:35:06 +00:00
Evgeny Vereshchagin 1e0c5fc2a7 journald: make it clear that dev_kmsg_record modifies the string passed to it
The function replaces a couple commas, a semicolon and the final newline with
zero bytes in the string passed to it. The 'const' seems to have been added
by accident during a bulk edit (more specifically 3b3154df7e).
2018-08-10 13:34:24 +00:00
Yu Watanabe 8595102d3d journal: do not remove multiple spaces after identifier in syslog message
Single space is used as separator.
C.f. discussions in #156.

Fixes #9839 introduced by a6aadf4ae0.
2018-08-10 10:31:29 +03:00
Lennart Poettering 9e888a9c5b
Merge pull request #9827 from yuwata/fix-9795-9820
journal: fixes issues reported by ASan
2018-08-08 14:07:40 +02:00
Yu Watanabe a5ee33b951 syslog: fix segfault in syslog_parse_priority() 2018-08-08 18:27:15 +09:00
Yu Watanabe b9b8874f8f journal: fix size of buffer 2018-08-08 17:17:23 +09:00
Yu Watanabe a6aadf4ae0 journal: fix syslog_parse_identifier()
Fixes #9829.
2018-08-08 17:10:56 +09:00
Yu Watanabe 75f8d86eea journal: do not use newa() or strjoina() for message
Fixes another issue reported in #9795.
2018-08-08 17:10:21 +09:00
Yu Watanabe 7289c769fb journal: do not hide options in help message
Even if built without gcrypt, show the relevant options in help message.
Otherwise, the help message diverges from the man page or suggestions
by the shell completion.
2018-08-08 15:29:18 +09:00
Yu Watanabe 57019d5f75 journal: do not pass a negative value to memcpy()
The message may contains only whitespaces.

Fixes #9795.
2018-08-08 12:40:47 +09:00
Filipe Brandenburger a0edd02e43 tree-wide: Convert compare_func's to use CMP() macro wherever possible.
Looked for definitions of functions using the *_compare_func() suffix.

Tested:
- Unit tests passed (ninja -C build/ test)
- Installed this build and booted with it.
2018-08-06 19:26:35 -07:00
Yu Watanabe f330408d62 tree-wide: drop empty lines in comments 2018-07-23 08:44:24 +02:00
Lennart Poettering 8d568e8d3e
Merge pull request #9346 from keszybz/journald-exact2
Store a copy of the input message if any stripping or truncation occurs
2018-07-18 21:00:43 +02:00
Zbigniew Jędrzejewski-Szmek 7c30c3c44f journal: store the original timestamp as SYSLOG_TIMESTAMP=
This is useful if someone wants to recreate the original syslog datagram. We
already include timestamp information as _SOURCE_REALTIME_TIMESTAMP=, and in
normal use that timestamp, converted back to the form used by syslog
(Mth dd HH:MM:SS) would usually give the value. But there are various
circumstances where this might not be true. Most obviously, if the datagram is
sent a bit later after being prepared, the time is rounded to the nearest
second, and it might be off. This is especially bad around New Year when the
syslog timestamp wraps around. Then the same timezone and locale need to be
used to recreate the original timestamp. In the end doing this reliably is
complicated, and it seems much easier to just unconditionally include the
original timestamp.

If the original timestamp cannot be located, we store the full log line.
This way, it should be always possible to recreate the original input.

Example:
MESSAGE=x
SYSLOG_TIMESTAMP=Sep 15 15:07:58
SYSLOG_RAW
^]^@^@^@^@^@^@^@<13>Sep 15 15:07:58 HOST: x^@y
_PID=3318
_SOURCE_REALTIME_TIMESTAMP=1530743976393553

Fixes #2398.
2018-07-05 00:40:35 +02:00
Zbigniew Jędrzejewski-Szmek df8701a3f2 journal: store the original syslog input as SYSLOG_RAW=
This allows the original stream to be recreated and/or verified. The new field
is written if any stripping was done or if the input message contained embeded
NULs.

$ printf '<13>Sep 15 15:07:58 HOST: x\0y' | nc -w1 -u -U /run/systemd/journal/dev-log

$ journalctl -o json-pretty ...
{
  ...
  "MESSAGE" : "x",
  "SYSLOG_RAW" : [ 60, 49, 51, 62, 83, 101, 112, 32, 49, 53, 32, 49, 53, 58, 48, 55, 58, 53, 56, 32, 72, 79, 83, 84, 58, 32, 120, 0, 121 ]
}

$ journalctl -o export ... | cat -v
...
MESSAGE=x
SYSLOG_RAW
^]^@^@^@^@^@^@^@<13>Sep 15 15:07:58 HOST: x^@y

This mostly fixes #4863.
2018-07-04 18:18:04 +02:00
Zbigniew Jędrzejewski-Szmek c3950a9bbe journal: optimize the common case where whitespace stripping is not needed
If we can just use the original data, let's do that.
2018-07-04 18:02:50 +02:00
Zbigniew Jędrzejewski-Szmek 90d75140c8 journald: drop copyright headers for Sebastian Thorarensen
For #9320.
2018-07-03 01:32:25 +02:00
Lennart Poettering ca285cdb40
Merge pull request #9422 from yuwata/conf-files
cleanups for default conf files
2018-06-28 10:17:54 +02:00
Yu Watanabe e0f691e1fd tree-wide: use localtime_r() instead of localtime()
Follow-up for e46acb7950.
2018-06-26 14:26:39 +09:00
Yu Watanabe 2dcacc7363 journal: add missing option in journald.conf 2018-06-26 04:05:42 +09:00
Yu Watanabe 29bfb683a8 journal: drop redundant condition 2018-06-25 13:36:49 +09:00
Kevin Kuehler 321ed3643a journalctl: handle inotify exhaustion nicer
Instead of failing, log that inotify wds
are exhausted and fallback to -n switch.

Fixes #1296.
2018-06-21 14:46:53 +02:00
Zbigniew Jędrzejewski-Szmek d9b02e1697 tree-wide: drop copyright headers from frequent contributors
Fixes #9320.

for p in Shapovalov Chevalier Rozhkov Sievers Mack Herrmann Schmidt Rudenberg Sahani Landden Andersen Watanabe; do
  git grep -e 'Copyright.*'$p -l|xargs perl -i -0pe 's|/([*][*])?[*]\s+([*#]\s+)?Copyright[^\n]*'$p'[^\n]*\s*[*]([*][*])?/\n*|\n|gms; s|\s+([*#]\s+)?Copyright[^\n]*'$p'[^\n]*\n*|\n|gms'
done
2018-06-20 11:58:53 +02:00
Jérémy Rosen 14c1abccf0 add _AUDIT_TYPE_NAME field to audit records in the journal 2018-06-19 10:48:38 +02:00
Chris Lamb 3fe910794b Correct a number of trivial typos. 2018-06-18 22:44:44 +02:00
Jérémy Rosen ed563b609c do not filter out deprecated USER audit messages 2018-06-18 14:48:09 +02:00
Zbigniew Jędrzejewski-Szmek 0cd41d4dff Drop my copyright headers
perl -i -0pe 's/\s*Copyright © .... Zbigniew Jędrzejewski.*?\n/\n/gms' man/*xml
git grep -e 'Copyright.*Jędrzejewski' -l | xargs perl -i -0pe 's/(#\n)?# +Copyright © [0-9, -]+ Zbigniew Jędrzejewski.*?\n//gms'
git grep -e 'Copyright.*Jędrzejewski' -l | xargs perl -i -0pe 's/\s*\/\*\*\*\s+Copyright © [0-9, -]+ Zbigniew Jędrzejewski[^\n]*?\s*\*\*\*\/\s*/\n\n/gms'
git grep -e 'Copyright.*Jędrzejewski' -l | xargs perl -i -0pe 's/\s+Copyright © [0-9, -]+ Zbigniew Jędrzejewski[^\n]*//gms'
2018-06-14 13:03:20 +02:00
Lennart Poettering 96b2fb93c5 tree-wide: beautify remaining copyright statements
Let's unify an beautify our remaining copyright statements, with a
unicode ©. This means our copyright statements are now always formatted
the same way. Yay.
2018-06-14 10:20:21 +02:00
Lennart Poettering 0c69794138 tree-wide: remove Lennart's copyright lines
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
2018-06-14 10:20:20 +02:00
Lennart Poettering 810adae9e9 tree-wide: use proper unicode © instead of (C) where we can
Let's use a proper unicode copyright symbol where we can, it's prettier.

This important patch is very important.
2018-06-14 10:20:20 +02:00
Lennart Poettering 818bf54632 tree-wide: drop 'This file is part of systemd' blurb
This part of the copyright blurb stems from the GPL use recommendations:

https://www.gnu.org/licenses/gpl-howto.en.html

The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.

hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
2018-06-14 10:20:20 +02:00
Zbigniew Jędrzejewski-Szmek 5d904a6aaa tree-wide: drop !! casts to booleans
They are not needed, because anything that is non-zero is converted
to true.

C11:
> 6.3.1.2: When any scalar value is converted to _Bool, the result is 0 if the
> value compares equal to 0; otherwise, the result is 1.

https://stackoverflow.com/questions/31551888/casting-int-to-bool-in-c-c
2018-06-13 10:52:40 +02:00
Michal Sekletar bb3ff70a86 journal: forward messages from /dev/log unmodified to syslog.socket 2018-06-11 21:26:22 +02:00
Filipe Brandenburger fffafb2b5e udev: fix usage of udev_device_new_from_stat_rdev() in journalctl
The refactor in #9200 inadvertently dropped the variable assignment to
traverse the device and its hierarchy in add_matches_for_device().

This was uncovered by Coverity (CID #1393310).

Fix that by restoring the assignment.

Tested: `journalctl /dev/sda` now filters journalctl output again.
2018-06-08 00:45:22 +02:00
Lennart Poettering b8b846d7b4 tree-wide: fix a number of log calls that use %m but have no errno set
This is mostly fall-out from d1a1f0aaf0,
however some cases are older bugs.

There might be more issues lurking, this was a simple grep for "%m"
across the tree, with all lines removed that mention "errno" at all.
2018-06-07 15:29:17 +02:00
Lennart Poettering e5ca27b772 udev: add helper udev_device_new_from_stat_rdev()
This is a simple wrapper around udev_device_new_from_devnum(), and uses
the data from a struct stat's .st_rdev field to derive the udev_device
object.
2018-06-07 13:36:19 +02:00
Lennart Poettering 0be9b12be2
Merge pull request #9147 from keszybz/runtime-enablement
Runtime enablement
2018-06-04 11:58:21 +02:00
Zbigniew Jędrzejewski-Szmek 846e541830 journal: small simplification 2018-05-31 20:42:04 +02:00
Zbigniew Jędrzejewski-Szmek d180c34998 journal: allow boot_id to be passed to journal_append_entry()
In this commit, this is done only in testing code, i.e. there is
no functional change apart from tests.
2018-05-31 14:30:23 +02:00
Zbigniew Jędrzejewski-Szmek 5a271b08b3 journal: remove unused args from journal_file_copy_entry() 2018-05-31 14:30:23 +02:00
Zbigniew Jędrzejewski-Szmek c627395366 journal: refuse an entry with invalid timestamp fields
The journal verification functions would reject such an entry. It would probably
still display fine (because we prefer _SOURCE_REALTIME_TIMESTAMP= if present), but
it seems wrong to create an entry that would not pass verification.
2018-05-31 14:30:23 +02:00
Zbigniew Jędrzejewski-Szmek 9b972c9aab journal: rename output_journal to show_journal_entry
We have show_journal, and output_journal, and it's not immediately clear
how they related. Rename the first to show that it just prints one entry.
2018-05-31 13:04:18 +02:00
Zbigniew Jędrzejewski-Szmek fd4885df94 journal: allow writing journal files even if machine-id is missing
The code to open journal files seems like the wrong place to enforce this. We
already check during boot and refuse to boot if machine-id is missing, no need
to enforce this here. In particular, it seems better to write logs from
journald even if they are not completely functional rather than refuse to
operate at all, and systemd-journal-remote also writes journal files and may
even be run on a system without systemd at all.

The docker image that oss-fuzz uses has an empty /etc/machine-id. Obviously
this is an error in the docker, but docker is fact of life, and it seems better
for systemd-journal-remote to work in such an incomplete environment.
2018-05-31 13:04:18 +02:00
Zbigniew Jędrzejewski-Szmek 5d889c1365 journal: rewrap function args 2018-05-31 13:04:18 +02:00
Vito Caputo 83bf6b6741 journal-file: avoid joining offline thread
In journal_file_set_online() the offline thread doesn't need to be
joined if it's been canceled before actually reaching the phase of
writing the offline state.
2018-05-29 17:01:23 +02:00
Luca Boccassi 49805b3d81 journalctl: add with-unit mode
When dealing with a large number of template instances, for example
when launching daemons per VRF, it is hard for operators to correlate
log lines to arguments.
Add a new with-unit mode which, if available, prefixes unit and user
unit names when displaying its log messages instead of the syslog
identifier. It will also use the full timestamp with timezones, like
the short-full mode.
2018-05-25 14:45:34 +02:00
Lennart Poettering 1a5a177eaf fileio: accept FILE* in addition to path in parse_env_file()
Most our other parsing functions do this, let's do this here too,
internally we accept that anyway. Also, the closely related
load_env_file() and load_env_file_pairs() also do this, so let's be
systematic.
2018-05-24 17:01:57 +02:00
Lennart Poettering 8904ab86b0
Merge pull request #9062 from poettering/parse-conf-macro
add new CONFIG_PARSER_PROTOTYPE() macro
2018-05-22 16:14:49 +02:00
Lennart Poettering a210692525 tree-wide: port over all code to the new CONFIG_PARSER_PROTOTYPE() macro
This makes most header files easier to look at. Also Emacs gets really
slow when browsing through large sections of overly long prototypes,
which is much improved by this macro.

We should probably not do something similar with too many other cases,
as macros like this might help readability for some, but make it worse
for others. But I think given the complexity of this specific prototype
and how often we use it, it's worth doing.
2018-05-22 13:18:44 +02:00
Lennart Poettering 5c828e66b5 tree-wide: port various bits of the tree over to the new DUMP_STRING_TABLE() macro 2018-05-22 13:14:18 +02:00
Zbigniew Jędrzejewski-Szmek 627d2bac24 fs-util,test: add helper to remove tempfiles
This simplifies the use of tempfiles in tests and fixes "leaked"
temporary files in test-fileio, test-catalog, test-conf-parser.

Not the whole tree is converted.
2018-05-18 20:11:31 +02:00
Zbigniew Jędrzejewski-Szmek d18cb3937b Turn VALGRIND variable into a meson configuration switch
Configuration through environment variable is inconvenient with meson, because
they cannot be convieniently changed and/or are not preserved during
reconfiguration (https://github.com/mesonbuild/meson/issues/1503).
This adds -Dvalgrind=true/false, which has the advantage that it can be set
at any time with meson configure -Dvalgrind=... and ninja will rebuild targets
as necessary. Additional minor advantages are better consistency with the
options for hashmap debugging, and typo avoidance with '#if' instead of '#ifdef'.
2018-05-17 09:54:36 -07:00
Zbigniew Jędrzejewski-Szmek 717fb9bc24
Merge pull request #8950 from dtardon/cleanup
use automatic cleanup more
2018-05-10 17:23:40 +02:00
David Tardon 17c9aff8ce journal: use automatic cleanup more 2018-05-10 13:42:36 +02:00
Zbigniew Jędrzejewski-Szmek f201daec89 Introduce _cleanup_(strbuf_cleanupp) and use it to fix null deref on error
catalog_update() would call strbuf_cleanup(NULL) on allocation error.
CID #1390928.
2018-05-10 01:36:50 +02:00
Lennart Poettering 702bd55cfb journal-send: drop redundant parenthesis
As suggested in @keszybz' comments on #8840
2018-04-27 17:32:01 +02:00
Lennart Poettering cf409d15fa tree-wide: use newa() rather than alloca() where we can 2018-04-27 14:29:06 +02:00
Lennart Poettering da6053d0a7 tree-wide: be more careful with the type of array sizes
Previously we were a bit sloppy with the index and size types of arrays,
we'd regularly use unsigned. While I don't think this ever resulted in
real issues I think we should be more careful there and follow a
stricter regime: unless there's a strong reason not to use size_t for
array sizes and indexes, size_t it should be. Any allocations we do
ultimately will use size_t anyway, and converting forth and back between
unsigned and size_t will always be a source of problems.

Note that on 32bit machines "unsigned" and "size_t" are equivalent, and
on 64bit machines our arrays shouldn't grow that large anyway, and if
they do we have a problem, however that kind of overly large allocation
we have protections for usually, but for overflows we do not have that
so much, hence let's add it.

So yeah, it's a story of the current code being already "good enough",
but I think some extra type hygiene is better.

This patch tries to be comprehensive, but it probably isn't and I missed
a few cases. But I guess we can cover that later as we notice it. Among
smaller fixes, this changes:

1. strv_length()' return type becomes size_t

2. the unit file changes array size becomes size_t

3. DNS answer and query array sizes become size_t

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=76745
2018-04-27 14:29:06 +02:00
Lennart Poettering 8e766630f0 tree-wide: drop redundant _cleanup_ macros (#8810)
This drops a good number of type-specific _cleanup_ macros, and patches
all users to just use the generic ones.

In most recent code we abstained from defining type-specific macros, and
this basically removes all those added already, with the exception of
the really low-level ones.

Having explicit macros for this is not too useful, as the expression
without the extra macro is generally just 2ch wider. We should generally
emphesize generic code, unless there are really good reasons for
specific code, hence let's follow this in this case too.

Note that _cleanup_free_ and similar really low-level, libc'ish, Linux
API'ish macros continue to be defined, only the really high-level OO
ones are dropped. From now on this should really be the rule: for really
low-level stuff, such as memory allocation, fd handling and so one, go
ahead and define explicit per-type macros, but for high-level, specific
program code, just use the generic _cleanup_() macro directly, in order
to keep things simple and as readable as possible for the uninitiated.

Note that before this patch some of the APIs (notable libudev ones) were
already used with the high-level macros at some places and with the
generic _cleanup_ macro at others. With this patch we hence unify on the
latter.
2018-04-25 12:31:45 +02:00
Lennart Poettering 5d13a15b1d tree-wide: drop spurious newlines (#8764)
Double newlines (i.e. one empty lines) are great to structure code. But
let's avoid triple newlines (i.e. two empty lines), quadruple newlines,
quintuple newlines, …, that's just spurious whitespace.

It's an easy way to drop 121 lines of code, and keeps the coding style
of our sources a bit tigther.
2018-04-19 12:13:23 +02:00
Zbigniew Jędrzejewski-Szmek 11a1589223 tree-wide: drop license boilerplate
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.

I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
2018-04-06 18:58:55 +02:00
David Tardon 3de8ff5a69 journald: bump rate limits (#8660)
Apparently, it is quite common to hit a problem, where systemd-journald
would drop messages because service is logging too fast.
2018-04-05 13:06:59 +02:00
Yu Watanabe 1cc6c93a95 tree-wide: use TAKE_PTR() and TAKE_FD() macros 2018-04-05 14:26:26 +09:00
Lennart Poettering cf852ff63b journald: make use of fd_get_path() where appropriate 2018-03-28 22:04:58 +02:00
Evgeny Vereshchagin 1700f4f42c
Merge pull request #8576 from keszybz/oss-fuzz
oss-fuzz adjustments and other cleanups
2018-03-27 20:30:02 +03:00
Lennart Poettering 96d4d0244b journal-file: we can't use a chain cache entry if we don't know where it starts (#8542)
It might happen that we try to bisect through a chain of offset arrays in the
journal whose last element was just allocated but no item yet written
to. In that case that array will be all NUL, but it might still end up
in our array chain cache. If it does, we cannot use it for bisection,
since for bisection we need to know the value of the first entry in that
array, but if it's uninitialized it does not have a first value. Hence,
as a simple fix, in this unlikely case, simply ignore the chain cache.

This is supposed to fix the issue pointed out in #8432, but in a more
permissive way, as this case isn't strictly a badly formatted journal
but actually a valid state (though one within a very short time window),
and we should make the best of it, and handle it gracefully.

Background: in each journal file entries are linked up in large arrays
of offsets. In each array the entries are strictly ordered by the
offsets of the entries, which permits search by bisection. These arrays
are allocated with a fixed size and then filled up as entries are added
to the journal file. If an array is fully filled up, a new array
(double in size as the old one) is appended to the journal file, and
linked up. This means, the journal file will contain a series of chained
up arrays, each time doubling in size, and strictly ordered. When
looking for an entry we maintain a "chain cache", which allows us to
bypass traversing the chain in full if we look for entries close to each
other in a short time. With the fix above we make sure we don't
erroneously use a chain cache item that doesn't carry enough information
for this bisection to work.

Original issue identified (with patch) by @Kxuan.

Replaces: #8432
2018-03-27 09:36:49 +02:00