This uses GREEDY_ALLOC_ROUND_UP() to grow the allocation size
exponentially. This should speed allocation loops up a bit, given
that we often call strextend() repeatedly in a loop on the same
buffer.
The current overflow checking is broken in the corner case of the strings'
combined length being exactly SIZE_MAX: After the loop, l would be SIZE_MAX,
but we're not testing whether the l+1 expression overflows.
Fix it by simply pre-accounting for the final '\0': initialize l to 1 instead
of 0.
The loops over (x, then all varargs, until a NULL is found) can be written much
simpler with an ordinary for loop. Just initialize the loop variable to x, test
that, and in the increment part, fetch the next va_arg(). That removes a level
of indentation, and avoids doing a separate strlen()/stpcpy() call for x.
While touching this code anyway, change (size_t)-1 to the more readable
SIZE_MAX.
I had to move STRV_MAKE to macro.h. There is a circular dependency between
extract-word.h, strv.h, and string-util.h that makes it hard to define the
inline function otherwise.
split() and FOREACH_WORD really should die, and everything be moved to
extract_first_word() and friends, but let's at least make sure that for
the remaining code using it we can't deadlock by not progressing in the
word iteration.
Fixes: #15305
When showing logs from a container, we would fail to show various lines:
Oct 29 09:50:51 krowka systemd-nspawn[61376]: Detected architecture x86-64.
Oct 29 09:50:51 krowka systemd-nspawn[61376]: [1B blob data]
Oct 29 09:50:51 krowka systemd-nspawn[61376]: Welcome to Fedora 32 (Rawhide)!
Oct 29 09:50:51 krowka systemd-nspawn[61376]: [1B blob data]
Those are only harmless \r characters that trail the line. We already replace
tabs and strip various ansi characters that we deem inconsequential, so let's
also strip trailing carriage returns. Non-trailing ones are different, because
they change what would be displayed.
Let's drop the 'static' logic when a parameter can be NULL.
I think asan/ubsan are right here, judging by the C99 spec language:
"A declaration of a parameter as ‘‘array of type’’ shall be adjusted to
‘‘qualified pointer to type’’, where the type qualifiers (if any) are
those specified within the [ and ] of the array type derivation. If the
keyword static also appears within the [ and ] of the array type
derivation, then for each call to the function, the value of the
corresponding actual argument shall provide access to the first element
of an array with at least as many elements as specified by the size
expression."
If we specify NULL, then we certainly don't pvode access to any valid
array.
Fixes: #13039
https://hamberg.no/erlend/posts/2013-02-18-static-array-indices.html
This only works with clang, unfortunately gcc doesn't seem to implement the check
(tested with gcc-8.2.1-5.fc29.x86_64).
Simulated error:
[2/3] Compiling C object 'systemd-nspawn@exe/src_nspawn_nspawn.c.o'.
../src/nspawn/nspawn.c:3179:45: warning: array argument is too small; contains 15 elements, callee requires at least 16 [-Warray-bounds]
candidate = (uid_t) siphash24(arg_machine, strlen(arg_machine), hash_key);
^ ~~~~~~~~
../src/basic/siphash24.h:24:64: note: callee declares array parameter as static here
uint64_t siphash24(const void *in, size_t inlen, const uint8_t k[static 16]);
^~~~~~~~~~~~
v2: fix error in free_and_strndup()
When the orignal and copied message were the same, but shorter than specified
length l, memory read past the end of the buffer would be performed. A test
case is included: a string that had an embedded NUL ("q\0") is used to replace
"q".
v3: Fix one more bug in free_and_strndup and add tests.
v4: Some style fixed based on review, one more use of free_and_replace, and
make the tests more comprehensive.
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
oss-fuzz flags this as:
==1==WARNING: MemorySanitizer: use-of-uninitialized-value
0. 0x7fce77519ca5 in ascii_is_valid systemd/src/basic/utf8.c:252:9
1. 0x7fce774d203c in ellipsize_mem systemd/src/basic/string-util.c:544:13
2. 0x7fce7730a299 in print_multiline systemd/src/shared/logs-show.c:244:37
3. 0x7fce772ffdf3 in output_short systemd/src/shared/logs-show.c:495:25
4. 0x7fce772f5a27 in show_journal_entry systemd/src/shared/logs-show.c:1077:15
5. 0x7fce772f66ad in show_journal systemd/src/shared/logs-show.c:1164:29
6. 0x4a2fa0 in LLVMFuzzerTestOneInput systemd/src/fuzz/fuzz-journal-remote.c:64:21
...
I didn't reproduce the issue, but this looks like an obvious error: the length
is specified, so we shouldn't use the string with any functions for normal
C-strings.
First, ellipsize() and ellipsize_mem() should not read past the input
buffer. Those functions take an explicit length for the input data, so they
should not assume that the buffer is terminated by a nul.
Second, ellipsization was off in various cases where wide on multi-byte
characters were used.
We had some basic test for ellipsize(), but apparently it wasn't enough to
catch more serious cases.
Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8686.
For short buffer sizes cellescape() was a bit wasteful, as it might
suffice to to drop a single character to find enough place for the full
four byte ellipsis, if that one character was a four character escape.
With this rework we'll guarantee to drop the minimum number of
characters from the end to fit in the ellipsis.
If the buffers we write to are large this doesn't matter much. However,
if they are short (as they are when talking about the process comm
field) then it starts to matter that we put as much information as we
can in the space we get.
It's not supposed to be the most efficient, but instead fast and simple to use.
I kept the logic in ellipsize_mem() to use unicode ellipsis even in non-unicode
locales. I'm not quite convinced things should be this way, especially that with
this patch it'd actually be simpler to always use "…" in unicode locale and "..."
otherwise, but Lennart wanted it this way for some reason.
With the recent terminal_urlify() APIs we'll now sometimes generate
clickable link CSO sequences. Hence we should also be able to remove
them again from strings. This beefs up the logic to do so.
Follow-up for: 23b27b39d2
This primarily changes to things:
1. Ellipsation to 0, 1 or 2 characters is now supported. Previously we'd
hit an assert if the new lengths was < 3, this is now permitted. The
result strings won't show too much info still of course, but the code
becomes a bit more generic and robust to use.
2. If a UTF-8 mode is disabled and the input string is pure ASCII, then
"..." is used for ellipsation, otherwise (as before) "…". This means
on a pure-ASCII system we should remain pure-ASCII, matching
behaviour otherwise exposed with special_glyph() and friends. Note
that we'll use "…" for ellipsiation as soon as either the locale
settings indicate an UTF-8 mode or the input string already contains
non-ASCII unicode characters.
Testing for these special cases is improved.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
Red is used for highligting, the same as grep does. Except when the line is
highlighted red already, because it has high priority, in which case plain ansi
highlight is used for the matched substring.
Coloring is implemented for short and cat outputs, and not for other types.
I guess we could also add it for verbose output in the future.
Let's replace usage of fputc_unlocked() and friends by __fsetlocking(f,
FSETLOCKING_BYCALLER). This turns off locking for the entire FILE*,
instead of doing individual per-call decision whether to use normal
calls or _unlocked() calls.
This has various benefits:
1. It's easier to read and easier not to forget
2. It's more comprehensive, as fprintf() and friends are covered too
(as these functions have no _unlocked() counterpart)
3. Philosophically, it's a bit more correct, because it's more a
property of the file handle really whether we ever pass it on to another
thread, not of the operations we then apply to it.
This patch reworks all pieces of codes that so far used fxyz_unlocked()
calls to use __fsetlocking() instead. It also reworks all places that
use open_memstream(), i.e. use stdio FILE* for string manipulations.
Note that this in some way a revert of 4b61c87511.
This adds a new flavour of strextend(), called
strextend_with_separator(), which takes an optional separator string. If
specified, the separator is inserted between each appended string, as
well as before the first one, but only if the original string was
non-empty.
This new call is particularly useful when appending new options to mount
option strings and suchlike, which need to be comma-separated, and
initially start out from an empty string.
Let's say that (size_t) -1 (i.e. SIZE_T_MAX) is equivalent to
"unbounded" ellipsation, i.e. ellipsation as NOP. In which case the
relevant functions become little more than strdup()/strndup().
This is useful to simplify caller code in case we want to turn off
ellipsation in certain code paths with minimal caller-side handling for
this.
Lennart Poettering