https://hamberg.no/erlend/posts/2013-02-18-static-array-indices.html
This only works with clang, unfortunately gcc doesn't seem to implement the check
(tested with gcc-8.2.1-5.fc29.x86_64).
Simulated error:
[2/3] Compiling C object 'systemd-nspawn@exe/src_nspawn_nspawn.c.o'.
../src/nspawn/nspawn.c:3179:45: warning: array argument is too small; contains 15 elements, callee requires at least 16 [-Warray-bounds]
candidate = (uid_t) siphash24(arg_machine, strlen(arg_machine), hash_key);
^ ~~~~~~~~
../src/basic/siphash24.h:24:64: note: callee declares array parameter as static here
uint64_t siphash24(const void *in, size_t inlen, const uint8_t k[static 16]);
^~~~~~~~~~~~
v2: fix error in free_and_strndup()
When the orignal and copied message were the same, but shorter than specified
length l, memory read past the end of the buffer would be performed. A test
case is included: a string that had an embedded NUL ("q\0") is used to replace
"q".
v3: Fix one more bug in free_and_strndup and add tests.
v4: Some style fixed based on review, one more use of free_and_replace, and
make the tests more comprehensive.
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
oss-fuzz flags this as:
==1==WARNING: MemorySanitizer: use-of-uninitialized-value
0. 0x7fce77519ca5 in ascii_is_valid systemd/src/basic/utf8.c:252:9
1. 0x7fce774d203c in ellipsize_mem systemd/src/basic/string-util.c:544:13
2. 0x7fce7730a299 in print_multiline systemd/src/shared/logs-show.c:244:37
3. 0x7fce772ffdf3 in output_short systemd/src/shared/logs-show.c:495:25
4. 0x7fce772f5a27 in show_journal_entry systemd/src/shared/logs-show.c:1077:15
5. 0x7fce772f66ad in show_journal systemd/src/shared/logs-show.c:1164:29
6. 0x4a2fa0 in LLVMFuzzerTestOneInput systemd/src/fuzz/fuzz-journal-remote.c:64:21
...
I didn't reproduce the issue, but this looks like an obvious error: the length
is specified, so we shouldn't use the string with any functions for normal
C-strings.
First, ellipsize() and ellipsize_mem() should not read past the input
buffer. Those functions take an explicit length for the input data, so they
should not assume that the buffer is terminated by a nul.
Second, ellipsization was off in various cases where wide on multi-byte
characters were used.
We had some basic test for ellipsize(), but apparently it wasn't enough to
catch more serious cases.
Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8686.
For short buffer sizes cellescape() was a bit wasteful, as it might
suffice to to drop a single character to find enough place for the full
four byte ellipsis, if that one character was a four character escape.
With this rework we'll guarantee to drop the minimum number of
characters from the end to fit in the ellipsis.
If the buffers we write to are large this doesn't matter much. However,
if they are short (as they are when talking about the process comm
field) then it starts to matter that we put as much information as we
can in the space we get.
It's not supposed to be the most efficient, but instead fast and simple to use.
I kept the logic in ellipsize_mem() to use unicode ellipsis even in non-unicode
locales. I'm not quite convinced things should be this way, especially that with
this patch it'd actually be simpler to always use "…" in unicode locale and "..."
otherwise, but Lennart wanted it this way for some reason.
With the recent terminal_urlify() APIs we'll now sometimes generate
clickable link CSO sequences. Hence we should also be able to remove
them again from strings. This beefs up the logic to do so.
Follow-up for: 23b27b39d2
This primarily changes to things:
1. Ellipsation to 0, 1 or 2 characters is now supported. Previously we'd
hit an assert if the new lengths was < 3, this is now permitted. The
result strings won't show too much info still of course, but the code
becomes a bit more generic and robust to use.
2. If a UTF-8 mode is disabled and the input string is pure ASCII, then
"..." is used for ellipsation, otherwise (as before) "…". This means
on a pure-ASCII system we should remain pure-ASCII, matching
behaviour otherwise exposed with special_glyph() and friends. Note
that we'll use "…" for ellipsiation as soon as either the locale
settings indicate an UTF-8 mode or the input string already contains
non-ASCII unicode characters.
Testing for these special cases is improved.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
Red is used for highligting, the same as grep does. Except when the line is
highlighted red already, because it has high priority, in which case plain ansi
highlight is used for the matched substring.
Coloring is implemented for short and cat outputs, and not for other types.
I guess we could also add it for verbose output in the future.
Let's replace usage of fputc_unlocked() and friends by __fsetlocking(f,
FSETLOCKING_BYCALLER). This turns off locking for the entire FILE*,
instead of doing individual per-call decision whether to use normal
calls or _unlocked() calls.
This has various benefits:
1. It's easier to read and easier not to forget
2. It's more comprehensive, as fprintf() and friends are covered too
(as these functions have no _unlocked() counterpart)
3. Philosophically, it's a bit more correct, because it's more a
property of the file handle really whether we ever pass it on to another
thread, not of the operations we then apply to it.
This patch reworks all pieces of codes that so far used fxyz_unlocked()
calls to use __fsetlocking() instead. It also reworks all places that
use open_memstream(), i.e. use stdio FILE* for string manipulations.
Note that this in some way a revert of 4b61c87511.
This adds a new flavour of strextend(), called
strextend_with_separator(), which takes an optional separator string. If
specified, the separator is inserted between each appended string, as
well as before the first one, but only if the original string was
non-empty.
This new call is particularly useful when appending new options to mount
option strings and suchlike, which need to be comma-separated, and
initially start out from an empty string.
Let's say that (size_t) -1 (i.e. SIZE_T_MAX) is equivalent to
"unbounded" ellipsation, i.e. ellipsation as NOP. In which case the
relevant functions become little more than strdup()/strndup().
This is useful to simplify caller code in case we want to turn off
ellipsation in certain code paths with minimal caller-side handling for
this.
This is a legacy of autotools, where one detection routine used a different
prefix then the others.
$ git grep -e HAVE_DECL_ -l|xargs sed -i s/HAVE_DECL_/HAVE_/g
Trivial performance boost by explicitly bypassing the implicit
locking of stdio.
This significantly affects common cases of `journalctl` usage:
Before:
# time ./journalctl -b -1 > /dev/null
real 0m26.628s
user 0m26.495s
sys 0m0.125s
# time ./journalctl -b -1 > /dev/null
real 0m27.069s
user 0m26.936s
sys 0m0.134s
# time ./journalctl -b -1 > /dev/null
real 0m26.727s
user 0m26.607s
sys 0m0.119s
After:
# time ./journalctl -b -1 > /dev/null
real 0m23.394s
user 0m23.244s
sys 0m0.142s
# time ./journalctl -b -1 > /dev/null
real 0m23.283s
user 0m23.160s
sys 0m0.121s
# time ./journalctl -b -1 > /dev/null
real 0m23.274s
user 0m23.125s
sys 0m0.144s
Fixes https://github.com/systemd/systemd/issues/6341
explicit_bzero was added in glibc 2.25. Make use of it.
explicit_bzero is hardcoded to zero the memory, so string erase now
truncates the string, instead of overwriting it with 'x'. This causes
a visible difference only in the journalctl case.
We were already unconditionally using the unicode character when the
input string was not pure ASCII, leading to different behaviour in
depending on the input string.
systemd[1]: Starting printit.service.
python3[19962]: foooooooooooooooooooooooooooooooooooo…oooo
python3[19964]: fooąęoooooooooooooooooooooooooooooooo…oooo
python3[19966]: fooąęoooooooooooooooooooooooooooooooo…ąęąę
python3[19968]: fooąęoooooooooooooooooąęąęąęąęąęąęąęą…ąęąę
systemd[1]: Started printit.service.
"#pragma GCC optimize" is merely a convenience to decorate multiple
functions with attribute optimize. And the manual has this to say about
this attribute:
This attribute should be used for debugging purposes only. It
is not suitable in production code.
Some versions of GCC also seem to have a problem with this pragma in
combination with LTO, resulting in ICEs.
So use a different approach (indirect the memset call via a volatile
function pointer) as implemented in openssl's crypto/mem_clr.c.
Closes: #3811
We already have tolower() calls there, hence let's unify this at one place.
Also, update the code to only use ASCII operations, so that we don't end up
being locale dependant.
Throughout the tree there's spurious use of spaces separating ++ and --
operators from their respective operands. Make ++ and -- operator
consistent with the majority of existing uses; discard the spaces.
rework C11 utf8.[ch] to use char32_t instead of uint32_t when referring
to unicode chars, to make things more expressive.
[
@zonque:
* rebased to current master
* use AC_CHECK_DECLS to detect availibility of char{16,32}_t
* make utf8_encoded_to_unichar() return int
]
In contrast to ascii_strcasecmp_nn() it takes two character buffers with their individual length. It will then compare
the buffers up the smaller size of the two buffers, and finally the length themselves.
Zbigniew Jędrzejewski-Szmek