Commit Graph

190 Commits

Author SHA1 Message Date
Lennart Poettering 4850d39ab7 journald: add a couple of static asserts checking logging constants
Whenever we include a log level or facility in a journal string field, make sure the compiler checks for us that that's
actually the right thing to do.
2016-01-26 14:43:24 +01:00
Lennart Poettering 1d35b2d6e2 Merge pull request #2424 from keszybz/journald-disk-usage
Journald disk usage
2016-01-26 14:20:45 +01:00
Zbigniew Jędrzejewski-Szmek 32917e3388 journald: restore oom safety
v2:
- use xsprintf
2016-01-25 10:53:51 -05:00
Zbigniew Jędrzejewski-Szmek 9d5a981398 Merge pull request #2318 from vcaputo/coalesce-ftruncates-redux
journal: coalesce ftruncate()s in 250ms windows
2016-01-23 22:09:51 -05:00
Zbigniew Jędrzejewski-Szmek 282c5c4e42 journald: use structured message + catalog entry for disk usage
The format of the journald disk usage log entry was changed back and
forth a few times. It is annoying to have a very verbose message, but
if it is short it is hard to understand. But we have a tool for this,
the catalogue.

$ journalctl -x -u systemd-journald
Jan 23 18:48:50 rawhide systemd-journald[891]: Runtime journal (/run/log/journal/) is 8.0M, max 196.2M, 188.2M free.
-- Subject: Disk space used by the journal
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Runtime journal (/run/log/journal/) is currently using 8.0M.
-- Maximum allowed usage is set to 196.2M.
-- Leaving at least 294.3M free (of currently available 1.9G of disk space).
-- Enforced usage limit is thus 196.2M, of which 188.2M are still available.
--
-- The limits controlling how much disk space is used by the journal may
-- be configured with SystemMaxUse=, SystemKeepFree=, SystemMaxFileSize=,
-- RuntimeMaxUse=, RuntimeKeepFree=, RuntimeMaxFileSize= settings in
-- /etc/systemd/journald.conf. See journald.conf(5) for details.
Jan 23 18:48:50 rawhide systemd-journald[891]: System journal (/var/log/journal/) is 480.1M, max 1.6G, 1.2G free.
-- Subject: Disk space used by the journal
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- System journal (/var/log/journal/) is currently using 480.1M.
-- Maximum allowed usage is set to 1.6G.
-- Leaving at least 2.5G free (of currently available 5.8G of disk space).
-- Enforced usage limit is thus 1.6G, of which 1.2G are still available.
--
-- The limits controlling how much disk space is used by the journal may
-- be configured with SystemMaxUse=, SystemKeepFree=, SystemMaxFileSize=,
-- RuntimeMaxUse=, RuntimeKeepFree=, RuntimeMaxFileSize= settings in
-- /etc/systemd/journald.conf. See journald.conf(5) for details.
2016-01-23 19:49:00 -05:00
Zbigniew Jędrzejewski-Szmek 8a03c9ef74 journald: allow additional payload in server_driver_message
The code to format the iovec is shared with log.c. All call sites to
server_driver_message are changed to include the additional "MESSAGE="
part, but the new functionality is not used and change in functionality
is not expected.

iovec is preallocated, so the maximum number of messages is limited.
In server_driver_message N_IOVEC_PAYLOAD_FIELDS is currently set to 1.

New code is not oom safe, it will fail if memory cannot be allocated.
This will be fixed in subsequent commit.
2016-01-23 19:49:00 -05:00
Zbigniew Jędrzejewski-Szmek ff82c36c79 journald: do not free uninitialized pointer in error path 2016-01-18 15:21:28 -05:00
Vito Caputo 7a24f3bf2f journal: coalesce ftruncate()s in 250ms windows
Prior to this change every journal append causes an ftruncate() for the
sake of inotify propagation of the mmap-based writes.

With this change the notification is deferred up to ~250ms, coalescing
any repeated journal writes during the deferred period into a single
ftruncate().  The ftruncate() call isn't free and doing it on every
append adds unnecessary overhead and latency in the journald event loop.

Introduces journal_file_enable_post_change_timer() which manages a
timer on the provided sd-event instance for scheduling coalesced
ftruncates.  The ftruncate() behavior is unchanged unless
journal_file_enable_post_change_timer() is called on the JournalFile.

While not a tremendous improvement, profiling systemd-journald event loop
latencies using instrumentation as introduced by 34b8751 it was observed that
coalescing the ftruncates was low-hanging fruit worth pursuing.

Note orders 12 and 13 shifting left into order 11 and order 6 dipping into
order 5:

Unmodified:
     log2(us)   1 2 3  4 5  6   7   8  9   10 11   12   13 14 15 16 17 18 19
                -----------------------------------------------------------
[10685.414572]  0 0 0  0 38 602 61  2  290 60 1643 2554 13 1  4  1  0  0  1
[10690.415114]  0 0 0  0 0  646 54  7  309 44 2073 2148 17 1  3  0  0  0  1
[10695.415509]  0 0 0  0 1  650 73  3  324 37 2071 2270 9  0  0  1  0  1  0
[10700.416297]  0 0 0  0 0  659 50  4  318 38 2111 2152 6  0  1  0  0  1  1
[10705.417136]  0 0 0  0 2  660 48  4  320 38 2129 2146 12 1  1  0  0  1  1
[10710.489114]  0 0 0  0 0  673 38  3  321 37 1925 2339 7  0  0  0  0  1  1
[10715.489613]  0 0 0  0 3  656 64  8  317 48 2365 2007 7  0  0  0  0  0  1

Coalesced:
     log2(us)   1 2 3  4 5  6   7   8  9   10 11   12   13 14 15 16 17 18 19
                -----------------------------------------------------------
[ 6169.161360]  0 0 0  1 24 786 54  11 389 24 4192 771  6  4  0  0  1  0  1
[ 6174.161705]  0 0 0  1 18 800 35  6  380 27 3977 893  3  1  0  0  1  0  1
[ 6179.162741]  0 0 0  1 28 768 51  4  391 16 3998 831  5  3  0  0  0  0  2
[ 6184.162856]  0 0 0  0 19 770 60  2  376 26 3795 1004 9  5  1  0  1  0  1
[ 6189.163279]  0 0 0  0 28 761 49  7  372 27 3729 1056 3  2  0  0  1  0  1
[ 6194.164255]  0 0 0  0 25 785 49  7  394 19 3996 908  6  3  2  0  0  0  1
[ 6199.164658]  0 0 0  0 29 797 35  5  389 18 3995 898  3  4  1  1  1  0  1

The remaining high-order delays are a result of the synchronous fsyncs in
systemd-journald, beyond the scope of this commit.
2016-01-14 16:36:07 -08:00
David Herrmann de418eb91c Merge pull request #2053 from poettering/selinux-fix
Two unrelated fixes
2015-11-30 19:30:03 +01:00
Zbigniew Jędrzejewski-Szmek 5c3bde3fa8 journal: move the gist of server_fix_perms to acl-util.[hc]
Most of the function is moved to acl-util.c to make it possible to
add tests in subsequent commit.

Setting of the mode in server_fix_perms is removed:
- we either just created the file ourselves, and the permission be better right,
- or the file was already there, and we should not modify the permissions.

server_fix_perms is renamed to server_fix_acls to better reflect new
meaning, and made static because it is only used in one file.
2015-11-27 23:32:32 -05:00
Lennart Poettering 6355e75610 selinux: split up mac_selinux_have() from mac_selinux_use()
Let's distuingish the cases where our code takes an active role in
selinux management, or just passively reports whatever selinux
properties are set.

mac_selinux_have() now checks whether selinux is around for the passive
stuff, and mac_selinux_use() for the active stuff. The latter checks the
former, plus also checks UID == 0, under the assumption that only when
we run priviliged selinux management really makes sense.

Fixes: #1941
2015-11-27 20:28:13 +01:00
Daniel Mack a57246551a Merge pull request #1926 from phomes/include-order-libudev
tree-wide: group include of libudev.h with sd-*
2015-11-17 09:36:25 +01:00
Thomas Hindoe Paaboel Andersen b4bbcaa9c4 tree-wide: group include of libudev.h with sd-* 2015-11-17 07:06:08 +01:00
Lennart Poettering 65c1d46b09 journald: trivial simplification 2015-11-17 00:52:10 +01:00
Lennart Poettering 4a0b58c4a3 tree-wide: use right cast macros for UIDs, GIDs and PIDs 2015-11-17 00:52:10 +01:00
Thomas Hindoe Paaboel Andersen cf0fbc49e6 tree-wide: sort includes
Sort the includes accoding to the new coding style.
2015-11-16 22:09:36 +01:00
Lennart Poettering 4de2402b60 journald: fix accuracy of watchdog timer event
Adding 3/4th of the watchdog frequency as accuracy on top of 1/2 of the
watchdog frequency means we might end up at 5/4th of the frequency which
means we might miss the message from time to time.

Maybe fixes #1804
2015-11-12 12:33:10 +01:00
Lennart Poettering 33d52ab92f journald: rework --sync/--rotate logic to use CLOCK_MONOTONIC timestamp files
Previously, we'd rely on the mtime timestamps of the touch files to see
if our sync/rotation requests were already suppressed. This means we
rely on CLOCK_REALTIME timestamps. With this patch we instead store the
CLOCK_MONOTONIC timestamp *in* the touch files, and avoid relying on
mtime.

This should make things more reliable when the clock or underlying mtime
granularity is not very good.

This also adds warning messages if writing any of the flag files fails.
2015-11-12 11:17:01 +01:00
Lennart Poettering dbd6e31cf9 journalctl: make --rotate synchronous, too
Of course, ideally we'd just use normal synchronous bus calls, but this
is out of the question as long as we rely on dbus-daemon (which logs to
journald, and thus cannot use to avoid cyclic sync loops). Hence,
instead, reuse the wait logic already implemented for --sync, and use a
signal in one direction, and a mtime watch file for the reply.
2015-11-11 14:29:10 +01:00
Lennart Poettering 94b6551662 journalctl: add new --sync switch for syncing the journal to disk
With this new "--sync" switch we add a synchronous way to sync
everything queued to disk, and return only after that's complete. This
command gives the guarantee that anything queued before has hit the disk
before the command returns.

While we are at it, also improve the man pages and help text for
journalctl a bit.
2015-11-11 13:39:18 +01:00
Lennart Poettering 17afc8f27b journald: be less picky when receiving epoll events
The event might be flagged with stuff we don't expect, hence don't
be needlessly picky, just rely on the kernel passing us sensible events.
2015-11-10 21:04:39 +01:00
Lennart Poettering b374689c02 journald: dispatch SIGTERM/SIGINT with a low priority
Let's make sure to process all queued log data before exiting, so that
we don't unnecessary lose messages when shutting down.

https://github.com/systemd/systemd/pull/1812#issuecomment-155149871
2015-11-10 17:36:52 +01:00
Lennart Poettering 75eb615480 defs: rework CONF_DIRS_NULSTR() macro
The macro is generically useful for putting together search paths, hence
let's make it truly generic, by dropping the implicit ".d" appending it
does, and leave that to the caller. Also rename it from
CONF_DIRS_NULSTR() to CONF_PATHS_NULSTR(), since it's not strictly about
dirs that way, but any kind of file system path.

Also, mark CONF_DIR_SPLIT_USR() as internal macro by renaming it to
_CONF_PATHS_SPLIT_USR() so that the leading underscore indicates that
it's internal.
2015-11-10 17:31:31 +01:00
Michal Schmidt 709f6e46a3 treewide: use the negative error codes returned by our functions
Our functions return negative error codes.
Do not rely on errno being set after calling our own functions.
2015-11-05 13:44:06 +01:00
Lennart Poettering 119e9655dc journal: restore watchdog support 2015-11-03 17:45:12 +01:00
Lennart Poettering e22aa3d328 journald: never block when sending messages on NOTIFY_SOCKET socket
Otherwise we might run into deadlocks, when journald blocks on the
notify socket on PID 1, and PID 1 blocks on IPC to dbus-daemon and
dbus-daemon blocks on logging to journald. Break this cycle by making
sure that journald never ever blocks on PID 1.

Note that this change disables support for event loop watchdog support,
as these messages are sent in blocking style by sd-event. That should
not be a big loss though, as people reported frequent problems with the
watchdog hitting journald on excessively slow IO.

Fixes: #1505.
2015-11-01 22:12:29 +01:00
Lennart Poettering b5efdb8af4 util-lib: split out allocation calls into alloc-util.[ch] 2015-10-27 13:45:53 +01:00
Lennart Poettering afc5dbf37f io-util.h: move iovec stuff from macro.h to io-util.h 2015-10-27 13:25:57 +01:00
Lennart Poettering 4e731273ed util-lib: move /proc/cmdline parsing code to proc-cmdline.[ch] 2015-10-27 13:25:57 +01:00
Lennart Poettering 430f0182b7 src/basic: rename audit.[ch] → audit-util.[ch] and capability.[ch] → capability-util.[ch]
The files are named too generically, so that they might conflict with
the upstream project headers. Hence, let's add a "-util" suffix, to
clarify that this are just our utility headers and not any official
upstream headers.
2015-10-27 13:25:57 +01:00
Lennart Poettering 8b43440b7e util-lib: move string table stuff into its own string-table.[ch] 2015-10-27 13:25:56 +01:00
Lennart Poettering f4f15635ec util-lib: move a number of fs operations into fs-util.[ch] 2015-10-27 13:25:56 +01:00
Lennart Poettering a09561746f util-lib: introduce dirent-util.[ch] for directory entry calls
Also, move a couple of more path-related functions to path-util.c.
2015-10-27 13:25:56 +01:00
Lennart Poettering 6bedfcbb29 util-lib: split string parsing related calls from util.[ch] into parse-util.[ch] 2015-10-27 13:25:55 +01:00
Tom Gundersen 0dec689b09 journal: add missing include 2015-10-26 13:12:30 +01:00
Lennart Poettering 24a99732e2 Merge pull request #1681 from ssahani/journal
journald-server: port to extract_first_word
2015-10-26 12:25:34 +01:00
Susant Sahani d581d9d91f journald-server: port to extract_first_word 2015-10-26 15:32:08 +05:30
Lennart Poettering 3ffd4af220 util-lib: split out fd-related operations into fd-util.[ch]
There are more than enough to deserve their own .c file, hence move them
over.
2015-10-25 13:19:18 +01:00
Lennart Poettering 07630cea1f util-lib: split our string related calls from util.[ch] into its own file string-util.[ch]
There are more than enough calls doing string manipulations to deserve
its own files, hence do something about it.

This patch also sorts the #include blocks of all files that needed to be
updated, according to the sorting suggestions from CODING_STYLE. Since
pretty much every file needs our string manipulation functions this
effectively means that most files have sorted #include blocks now.

Also touches a few unrelated include files.
2015-10-24 23:05:02 +02:00
Lennart Poettering 09eba4d46b journald: make sure r is always initialized 2015-10-19 23:07:18 +02:00
Lennart Poettering ac5b0c13d8 tree-wide: add more void casts for various syscall invocations 2015-10-19 23:07:18 +02:00
Lennart Poettering 18438f262c Merge pull request #1542 from keszybz/journal-audit-optional
Make journald audit socket maskable
2015-10-13 17:23:33 +02:00
Zbigniew Jędrzejewski-Szmek 37b7affefd Document journald sockets paths a bit more
Adding them to the documentation makes it easier to find
the right man page for people who are trying to understand
where some socket in the filesystem is coming from.
2015-10-12 13:58:54 -04:00
Zbigniew Jędrzejewski-Szmek 7d18d348da journald: make audit socket optional
If we were given some sockets through socket activation, and audit
socket is not among them, do not try to open it. This way, if the
socket unit is disabled, we will not receive audit events.

https://bugzilla.redhat.com/show_bug.cgi?id=1227379
2015-10-12 13:12:08 -04:00
Lennart Poettering 8580d1f73d journal: rework vacuuming logic
Implement a maximum limit on number of journal files to keep around.
Enforcing a limit is useful on this since our performance when viewing
pays a heavy penalty for each journal file to interleve. This setting is
turned on now by default, and set to 100.

Also, actully implement what 348ced9097
promised: use whatever we find on disk at startup as lower bound on how
much disk space we can use. That commit introduced some provisions to
implement this, but actually never did.

This also adds "journalctl --vacuum-files=" to vacuum files on disk by
their number explicitly.
2015-10-02 23:21:59 +02:00
Lennart Poettering 0fb398316c journal: use automatic clenup for ACL types 2015-10-02 23:20:51 +02:00
Lennart Poettering 65089b8240 journal: improve some messages
Indicate that we are ignoring errors, when we ignore them, and log that
at LOG_WARNING level.

Use the right error code for the log message.
2015-10-02 23:19:00 +02:00
Lennart Poettering 84267e4043 journal: prefer stack allocation 2015-10-02 22:37:53 +02:00
Lennart Poettering 804ae586d4 journal: make journal_file_close() return NULL
The way it is customary everywhere else in our sources.
2015-10-02 22:36:33 +02:00
Lennart Poettering 8f4cc91472 Merge pull request #1430 from evverx/driver-syslog
journald: add syslog fields for driver messages
2015-10-01 16:29:13 +02:00