SD_JOURNAL_FOREACH_DATA() and SD_JOURNAL_FOREACH_UNIQUE() would immediately
terminate when a field couldn't be accessed. This can happen for example when a
field is compressed with an unavailable compression format. But it's likely
that this is the wrong thing to do: the caller for example might want to
iterate over the fields but isn't interested in all of them. coredumpctl is
like this: it uses SD_JOURNAL_FOREACH_DATA() but only uses a subset of the
fields.
Add two new functions sd_journal_enumerate_good_data() and
sd_journal_enumerate_good_unique() that retry sd_journal_enumerate_data() and
sd_journal_enumerate_unique() if the return value is something that applies to
a single field: ENOBUS, E2BIG, EOPNOTSUPP.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1856037.
An alternative would be to make the macros themselves smarter instead of adding
new symbols, and do the looping internally in the macro. I don't like that
approach for two reasons. First, it would embed the logic in the macro, so
recompilation would be required if we decide to update the logic. With the
current version of the patch, recompilation is required to use the new symbols,
but after that, library upgrades are enough. So the current approach is safer
in case further updates are needed. Second, our headers use primitive C, and it
is hard to do the macros without using newer features.
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
With this change a new flag SD_JOURNAL_OS_ROOT is introduced. If specified
while opening the journal with the per-directory calls (specifically:
sd_journal_open_directory() and sd_journal_open_directory_fd()) the passed
directory is assumed to be the root directory of an OS tree, and the journal
files are searched for in /var/log/journal, /run/log/journal relative to it.
This is useful to allow usage of sd-journal on file descriptors returned by the
OpenRootDirectory() call of machined.
Also, expose this via the "journalctl --file=-" syntax for STDIN. This feature
remains undocumented though, as it is probably not too useful in real-life as
this still requires fds that support mmaping and seeking, i.e. does not work
for pipes, for which reading from STDIN is most commonly used.
This adds two new calls to get the list of all journal fields names currently in use.
This is the low-level support to implement the feature requested in #2176 in a more optimized way.
Also introduce sd_journal_has_runtime_files() and
sd_journal_has_persistent_files() to the public API. These functions
can be used to easily find out if the open journal files are runtime
and/or persistent.
GLIB has recently started to officially support the gcc cleanup
attribute in its public API, hence let's do the same for our APIs.
With this patch we'll define an xyz_unrefp() call for each public
xyz_unref() call, to make it easy to use inside a
__attribute__((cleanup())) expression. Then, all code is ported over to
make use of this.
The new calls are also documented in the man pages, with examples how to
use them (well, I only added docs where the _unref() call itself already
had docs, and the examples, only cover sd_bus_unrefp() and
sd_event_unrefp()).
This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we
tend to call our destructors these days.
Note that this defines no public macro that wraps gcc's attribute and
makes it easier to use. While I think it's our duty in the library to
make our stuff easy to use, I figure it's not our duty to make gcc's own
features easy to use on its own. Most likely, client code which wants to
make use of this should define its own:
#define _cleanup_(function) __attribute__((cleanup(function)))
Or similar, to make the gcc feature easier to use.
Making this logic public has the benefit that we can remove three header
files whose only purpose was to define these functions internally.
See #2008.
Previously the returned object of constructor functions where sometimes
returned as last, sometimes as first and sometimes as second parameter.
Let's clean this up a bit. Here are the new rules:
1. The object the new object is derived from is put first, if there is any
2. The object we are creating will be returned in the next arguments
3. This is followed by any additional arguments
Rationale:
For functions that operate on an object we always put that object first.
Constructors should probably not be too different in this regard. Also,
if the additional parameters might want to use varargs which suggests to
put them last.
Note that this new scheme only applies to constructor functions, not to
all other functions. We do give a lot of freedom for those.
Note that this commit only changes the order of the new functions we
added, for old ones we accept the wrong order and leave it like that.
This adds the new library call sd_journal_open_container() and a new
"-M" switch to journalctl. Particular care is taken that journalctl's
"-b" switch resolves to the current boot ID of the container, not the
host.
So far we tried to use epoll directly wherever we needed an event loop.
However, that has various shortcomings, such as the inability to handle
larger amounts of timers (since each timerfd costs one fd, which is a
very limited resource, usually bounded to 1024), and inability to do
priorisation between multiple queued events.
Let's add a minimal event loop API around epoll that is suitable for
implementation of our own daemons and maybe one day can become public
API for those who desire it.
This loop is part of libsystemd-bus, but may be used independently of
it.
This allows the caller to explicitly specify which journal files
should be opened. The same functionality could be achieved before
by creating a directory and playing around with symlinks. It
is useful to debug stuff and explore the journal, and has been
requested before.
Waiting is supported, the journal will notice modifications on
the files supplied when opening the journal, but will not add
any new files.
This is the just the library part.
SD_JOURNAL_CURRENT_USER flags is added to sd_j_open(), to open
files from current user.
SD_JOURNAL_SYSTEM_ONLY is renamed to SD_JOURNAL_SYSTEM,
and changed to mean to (also) open system files. This way various
flags can be combined, which gives them nicer semantics, especially
if other ones are added later.
Backwards compatibility is kept, because SD_JOURNAL_SYSTEM_ONLY
is equivalent to SD_JOURNAL_SYSTEM if used alone, and before there
we no other flags.
gcc (and other compilers) sometimes generate spurious warnings, and
thus users of public headers must be able to disable warnings.
Printf format attributes can be disabled by setting
#define _sd_printf_attr_
before including the header file.
Also, add similar logic for sentinel attribute:
#define _sd_sentinel_attr_
before including the header file disables the attribute.
When using "-p" and "-b" in combination with "-u", the output is not
what you would expect. The reason is the sd_journal_add_disjunction()
call in add_matches_for_unit() and add_matches_for_user_unit(), which
adds two ORs without taking the other conditions to every OR.
Adding another level on top with AND and sd_journal_add_conjunction()
solves the problem.
Output before:
$ journalctl -o short-monotonic -ab -p 0 -u sshd.service
-- Reboot --
[ 3.216305] lenovo systemd[1]: Starting OpenSSH server daemon...
-- Reboot --
[ 3.168666] lenovo systemd[1]: Starting OpenSSH server daemon...
[ 3.169639] lenovo systemd[1]: Started OpenSSH server daemon.
[36285.635389] lenovo systemd[1]: Stopped OpenSSH server daemon.
-- Reboot --
[ 10.838657] lenovo systemd[1]: Starting OpenSSH server daemon...
[ 10.913698] lenovo systemd[1]: Started OpenSSH server daemon.
[ 6881.035183] lenovo systemd[1]: Stopped OpenSSH server daemon.
-- Reboot --
[ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon...
[ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon.
[ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22.
[ 6.681379] lenovo sshd[397]: Server listening on :: port 22.
As we see, the output is from _every_ boot and priority 0 is not taken
into account.
Output after patch:
$ journalctl -o short-monotonic -ab -p 0 -u sshd.service
-- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:58:21 CET. --
Increasing the priority:
$ journalctl -o short-monotonic -ab -p 6 -u sshd.service
-- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:59:12 CET. --
[ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon...
[ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon.
[ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22.
[ 6.681379] lenovo sshd[397]: Server listening on :: port 22.
This function should be used when filling in "struct pollfd"'s .events
field for watching the journal. It will always return POLLIN for now,
but we should keep our options open to change this later on.
This mimics libsystemd-bus' sd_bus_get_events() call with the same
purpose.
This introduces a new data threshold setting for sd_journal objects
which controls the maximum size of objects to decompress. This is
relieves the library from having to decompress full data objects even
if a client program is only interested in the initial part of them.
This speeds up "systemd-coredumpctl" drastically when invoked without
parameters.
The message catalog can be used to attach short help texts to log lines,
keyed by their MESSAGE_ID= fields. This is useful to help the
administrator understand the context and cause of a message, find
possible solutions and find further related documentation.
Since this is keyed off MESSAGE_ID= this will only work for native
journal messages.
The message catalog supports i18n, and is useful to augment english
language system messages with explanations in the local language.
This commit only includes short explanatory messages for a few example
message IDs, we'll add more complete documentation for the relevant
systemd messages later on.
Network file systems generally do not offer inotify() that would work
across the network. We hence cannot rely on inotify() exclusiely in
those case. Provide an API to determine these cases, and suggest doing
manual regular rechecks.
Note that this is not complete yet, as we need to rescan journal dirs on
network file systems explicitly to find new/removed files
The new 'unique' API allows listing all unique field values that a field
specified by a field name can take in all entries of the journal. This
allows answering queries such as "What units logged to the journal?",
"What hosts have logged into the journal?", "Which boot IDs have logged
into the journal?".
Ultimately this allows implementation of tools similar to lastlog based
on journal data.
Note that listing these field values will not work for journal files
created with older journald, as the field values are not indexed in
older files.
we now can take multiple matches, and they will apply as AND if they
apply to different fields and OR if they apply to the same fields. Also,
terms of this kind can be combined with an overreaching OR.
There's now sd_journal_new_directory() for watching specific journal
directories. This is exposed in journalctl -D.
sd_journal_wait() and sd_journal_process() now return whether changes in
the journal are invalidating or just appending.
We now create inotify kernel watches only when we actually need them
We finally got the OK from all contributors with non-trivial commits to
relicense systemd from GPL2+ to LGPL2.1+.
Some udev bits continue to be GPL2+ for now, but we are looking into
relicensing them too, to allow free copy/paste of all code within
systemd.
The bits that used to be MIT continue to be MIT.
The big benefit of the relicensing is that closed source code may now
link against libsystemd-login.so and friends.
