Commit graph

1715 commits

Author SHA1 Message Date
Yu Watanabe cd4f53c5b5 conf-parse: use path_simplify_and_warn() in config_parse_path() 2018-06-03 23:57:30 +09:00
Yu Watanabe 4805426279 path-util: make path_make_relative() support path including dots 2018-06-03 23:54:32 +09:00
Yu Watanabe 858d36c1ec path-util: introduce path_simplify()
The function is similar to path_kill_slashes() but also removes
initial './', trailing '/.', and '/./' in the path.
When the second argument of path_simplify() is false, then it
behaves as the same as path_kill_slashes(). Hence, this also
replaces path_kill_slashes() with path_simplify().
2018-06-03 23:39:26 +09:00
Zbigniew Jędrzejewski-Szmek 9924aef690 test-ellipsize: add tests for ellipsize_mem, fix bugs
First, ellipsize() and ellipsize_mem() should not read past the input
buffer. Those functions take an explicit length for the input data, so they
should not assume that the buffer is terminated by a nul.

Second, ellipsization was off in various cases where wide on multi-byte
characters were used.

We had some basic test for ellipsize(), but apparently it wasn't enough to
catch more serious cases.

Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8686.
2018-06-02 21:53:25 +02:00
Zbigniew Jędrzejewski-Szmek 4dae3ef6c0
Merge pull request #9156 from shr-project/jansa/gcc8
time-util: fix build with gcc8 -Werror=format-truncation=
2018-06-02 16:18:59 +02:00
Zbigniew Jędrzejewski-Szmek f1880a4b02 test-time-util: print names of test functions
This makes it easier to find the right spot in the long output.
2018-06-02 10:41:26 +02:00
Mike Gilbert 96f64eb574 basic: set errno in raw_clone() on sparc
sparc sets the carry bit when a syscall fails. Use this information to
set errno and return -1 as appropriate.

The added test case calls raw_clone() with flags known to be invalid
according to the clone(2) manpage.
2018-06-02 09:48:19 +02:00
Lennart Poettering ce26882553 process-util: also filter non-printable characters in get_process_com()
We already do that in get_process_cmdline(), which is very similar in
behaviour otherwise. Hence, let's be safe and also filter them in
get_process_comm(). Let's try to retain as much information as we can
though and escape rather than suppress unprintable characters. Let's not
increase comm names beyond the kernel limit on such names however.

Also see discussion about this here:

https://marc.info/?l=linux-api&m=152649570404881&w=2
2018-06-01 21:53:13 +02:00
Lennart Poettering 61f6e27671 string-util: tweak cellescape() a bit
For short buffer sizes cellescape() was a bit wasteful, as it might
suffice to to drop a single character to find enough place for the full
four byte ellipsis, if that one character was a four character escape.
With this rework we'll guarantee to drop the minimum number of
characters from the end to fit in the ellipsis.

If the buffers we write to are large this doesn't matter much. However,
if they are short (as they are when talking about the process comm
field) then it starts to matter that we put as much information as we
can in the space we get.
2018-06-01 21:49:16 +02:00
Lennart Poettering 92f14395cd missing: define kernel internal limit TASK_COMM_LEN in userspace too
We already use it at two places, and we are about to add one too.
Arbitrary literally hardcoded limits suck.
2018-06-01 21:49:16 +02:00
Lennart Poettering 89544ae658
Merge pull request #9014 from keszybz/fuzz-journal-remote
A fuzzer for journal-remote
2018-05-31 15:33:44 +02:00
Zbigniew Jędrzejewski-Szmek 4d9685be5f Use const char* for timestamp strings which we don't plan to modify
Makes the intent a bit clearer.
2018-05-31 14:30:23 +02:00
Zbigniew Jędrzejewski-Szmek d3d280242c Always allow timestamps to be printed
If the timestamp is above 9999-12-30, (or 2038-something-something on 32 bit),
use XXXX-XX-XX XX:XX:XX as the replacement.

The problem with refusing to print timestamps is that our code accepts such
timestamps, so we can't really just refuse to process them afterwards. Also, it
makes journal files non-portable, because suddently we might completely refuse
to print entries which are totally OK on a different machine.
2018-05-31 14:30:23 +02:00
Zbigniew Jędrzejewski-Szmek 8409f68858 basic/string-util: add a convenience function to cescape mostly-ascii fields
It's not supposed to be the most efficient, but instead fast and simple to use.

I kept the logic in ellipsize_mem() to use unicode ellipsis even in non-unicode
locales. I'm not quite convinced things should be this way, especially that with
this patch it'd actually be simpler to always use "…" in unicode locale and "..."
otherwise, but Lennart wanted it this way for some reason.
2018-05-31 14:27:07 +02:00
Lennart Poettering 2f0726c08f coccinelle says we should use EOPNOTSUPP rather than ENOTSUP 2018-05-31 12:06:54 +02:00
Lennart Poettering 9b8ff18319 string-util: add new memory_startswith() helper
We have code like this at various placer, let's make things shorter and
more readable with a helper for it.
2018-05-30 13:07:40 +02:00
Zbigniew Jędrzejewski-Szmek 018eaf7445 test-tables,test-network-tables: add a few more tests 2018-05-28 10:40:00 +02:00
Zbigniew Jędrzejewski-Szmek 279eadd933 test-tables*: sort 2018-05-28 10:32:52 +02:00
Lennart Poettering 3e36211be0 conf-files: beef up conf-files.[ch] a bit
This adds fozr new flags:

- If CONF_FILES_DIRECTORY is specified conf_file_list() and friends
  will look for directories only.

- Similar CONF_FILES_REGULAR means we'll look only for regular files.

- If CONF_FILES_BASENAME is specified the resulting list will contain
  only the basenames of all discovered files or directories, not the
  full paths.

- If CONF_FILES_FILTER_MASKED is specified the resulting list will have
  masked entries removed (i.e. those symlinked to /dev/null and
  suchlike)

These four flags are useful for discovering portable service profile
information.

While we are at it, also improve a couple of other things:

- More debug logging

- use path_hash_ops instead of string_hash_ops when putting together the
  path lists
2018-05-24 17:01:57 +02:00
Lennart Poettering d58ad743f9 os-util: add helpers for finding /etc/os-release
Place this new helpers in a new source file os-util.[ch], and move the
existing and related call path_is_os_tree() to it as well.
2018-05-24 17:01:57 +02:00
Lennart Poettering 1a5a177eaf fileio: accept FILE* in addition to path in parse_env_file()
Most our other parsing functions do this, let's do this here too,
internally we accept that anyway. Also, the closely related
load_env_file() and load_env_file_pairs() also do this, so let's be
systematic.
2018-05-24 17:01:57 +02:00
Lennart Poettering 4960ce43ff fd-util: add new helper call fd_duplicate_data_fd()
This call creates an fd from another fd containing the same data.
Specifically, repeated read() on the returned fd should return the same
data as the original fd. This call is useful when we want to copy data
out of disk images and suchlike, and want to be pass fds with the data
around without having to keep the disk image continously mounted.

The implementation tries to be somewhat smart and tries to prefer
memfds/pipes over files in /tmp or /var/tmp based on the size of the
data, but has appropropriate fallbacks in place.
2018-05-24 17:01:57 +02:00
Lennart Poettering cdc0f9be92
Merge pull request #8817 from yuwata/cleanup-nsflags
core: allow to specify RestrictNamespaces= multiple times
2018-05-24 16:49:13 +02:00
Zbigniew Jędrzejewski-Szmek 994282d2de test-sizeof: show that a small 64 field is not enough to force the enum to be 64 bits
On both 32 and 64 bits, the result is:
enum Enum → 32 bits, unsigned
enum BigEnum → 32 bits, unsigned
enum BigEnum2 → 64 bits, unsigned
big_enum2_pos → 4
big_enum2_neg → 8

The last two lines show that even if the enum is 64 bit, and the field of an
enum is defined with UINT64_C(), the field can still be smaller.
2018-05-22 10:09:07 +02:00
Lennart Poettering 6f8fa29465
Merge pull request #8981 from keszybz/ratelimit-and-dbus
Ratelimit renaming and dbus error message fix
2018-05-18 21:38:30 +02:00
Lennart Poettering c385b10a13
Merge pull request #8993 from keszybz/sd-resolve-coverity-and-related-fixes
sd-resolve coverity and related fixes
2018-05-18 20:30:12 +02:00
Zbigniew Jędrzejewski-Szmek 627d2bac24 fs-util,test: add helper to remove tempfiles
This simplifies the use of tempfiles in tests and fixes "leaked"
temporary files in test-fileio, test-catalog, test-conf-parser.

Not the whole tree is converted.
2018-05-18 20:11:31 +02:00
Lennart Poettering 4f424df760 core: move config_parse_limit() to the generic conf-parser.[ch]
That way we can use it in nspawn.

Also, while we are at it, let's rename the call config_parse_rlimit(),
i.e. insert the "r", to clarify what kind of limit this is about.
2018-05-17 20:36:52 +02:00
Lennart Poettering 6550c24c7f rlimit-util: rework rlimit_{from|to}_string() to work without "Limit" prefix
let's make the call more generic, so that we can also easily use it for
parsing "RLIMIT_xyz" style constants.
2018-05-17 20:36:52 +02:00
Felipe Sateler 57b7a260c2 core: undo the dependency inversion between unit.h and all unit types 2018-05-15 14:24:34 -04:00
Zbigniew Jędrzejewski-Szmek 9421cab6db test-copy: remove unnecessary initialization
Initializing just some of the values doesn't make sense. We should handle them
all in the same way. In those tests there are no jumps, all steps are covered
with assert_se(), so we know everything will be set if we get to the end of the
function. And _not_ initializing those variables has the advantage that it
allows valgrind to catch potential initalization errors in the function being
tested.
2018-05-14 17:22:22 +02:00
Zbigniew Jędrzejewski-Szmek dcd6361ea1 tree-wide: do not wrap assert_se in extra parentheses
We were inconsitently using them in some cases, but in majority not.
Using assignment in assert_se is very common, not an exception like in
'if', so let's drop the extra parens everywhere.
2018-05-14 17:22:22 +02:00
Yu Watanabe 3ecdd18f87 test: use _cleanup_hashmap_free_ instread of _cleanup_(hashmap_freep) 2018-05-14 14:14:17 +09:00
Zbigniew Jędrzejewski-Szmek 7994ac1d85 Rename ratelimit_test to ratelimit_below
When I see "test", I have to think three times what the return value
means. With "below" this is immediately clear. ratelimit_below(&limit)
sounds almost like English and is imho immediately obvious.

(I also considered ratelimit_ok, but this strongly implies that being under the
limit is somehow better. Most of the times this is true, but then we use the
ratelimit to detect triple-c-a-d, and "ok" doesn't fit so well there.)

C.f. a1bcaa07.
2018-05-13 22:08:30 +02:00
Yu Watanabe b4152e9be1 test: add tests for unit_name_from_dbus_path() 2018-05-13 18:44:25 +02:00
Zbigniew Jędrzejewski-Szmek 88136bbc98
Merge pull request #8962 from floppym/issue8905
basic: timezone_is_valid: check for magic bytes "TZif"
2018-05-13 18:43:11 +02:00
Mike Gilbert 089fb8653f basic: add log_level argument to timezone_is_valid 2018-05-12 15:20:13 -04:00
Zbigniew Jędrzejewski-Szmek 2407ed7b63
Merge pull request #8924 from yuwata/fix-3682
resolve: allow whitespaces in the digest
2018-05-11 11:48:05 +02:00
Lennart Poettering cb91deaf77 terminal-util: add a function that shows a pretty separator line
Follow-up for #8824
2018-05-11 08:15:33 +02:00
Yu Watanabe 2d2c21afb2 test: do not call alloca() inside the list of arguments 2018-05-11 15:01:35 +09:00
Lennart Poettering e4915c2797
Merge pull request #8953 from yuwata/bus-macro
core: simplify dbus properties
2018-05-10 22:51:17 -07:00
Yu Watanabe 1c5873096e test-execute: remove state directories before running tests
Fixes #8679.
2018-05-10 22:50:51 -07:00
Yu Watanabe 4937b81a7b hexdecoct: ignore whitespace within the input hexadecimal text of unhexmem() 2018-05-11 14:36:22 +09:00
Joost Heitbrink b6887d7ae4 test: add testcase for environment file variables with backslashes
systemd silently strips out backslashes in variables from environment
files. Add a testcase that explicitly tests for this behaviour.
2018-05-11 13:15:16 +09:00
Lennart Poettering 81321f51cf
Merge pull request #8824 from keszybz/analyze-show-config
systemd-analyze show-config
2018-05-10 11:14:23 -07:00
Yu Watanabe 945403e6ed path-util: introduce empty_to_root() and use it many places 2018-05-11 01:47:33 +09:00
Yu Watanabe 3d924e7731
Merge pull request #8943 from keszybz/coverity-fixes
Coverity fixes
2018-05-10 23:22:38 +09:00
Filipe Brandenburger 4f29e0db12 conf-parser: accept trailing backslash at the end of the file (#8941)
This makes it behave the same whether there is a blank line or not at
the end of the file.  This is also consistent with the behavior of the
shell on a shell script that ends on a trailing backslash at the last
line.

Added tests to test_config_parse(), which only pass if the corresponding
change to config_parse() is included.
2018-05-09 18:10:07 -07:00
Zbigniew Jędrzejewski-Szmek f201daec89 Introduce _cleanup_(strbuf_cleanupp) and use it to fix null deref on error
catalog_update() would call strbuf_cleanup(NULL) on allocation error.
CID #1390928.
2018-05-10 01:36:50 +02:00
Zbigniew Jędrzejewski-Szmek db983479af shared/sleep-config: fix memleak of strv, add test
CID #1390921, #1390951.
2018-05-10 01:36:50 +02:00
Yu Watanabe 1dcf96c29f test: add tests for merging RestrictNamespaces= 2018-05-05 11:07:37 +09:00
Yu Watanabe 86c2a9f1c2 nsflsgs: drop namespace_flag_{from,to}_string()
This also drops namespace_flag_to_string_many_with_check(), and
renames namespace_flag_{from,to}_string_many() to
namespace_flags_{from,to}_string().
2018-05-05 11:07:37 +09:00
Evgeny Vereshchagin 1ab0a250c2
Merge pull request #8865 from yuwata/fix-signal
util: fix integer overflow
2018-05-03 16:35:38 +03:00
Lennart Poettering ec2b24f079 test: list more up-to-date urls in test-web-util
This is based on @jsynacek's patch from #8837, but adds the new URL in
two flavours instead of replacing the old, also making @keszybz happy.

Replaces: #8837
2018-05-03 10:55:16 +02:00
Yu Watanabe 50fb3437cd test: add tests for signal_from_string() 2018-05-03 16:52:55 +09:00
Lennart Poettering c1c80f6c37
Merge pull request #8866 from yuwata/fix-8842
core: disable namespace sandboxing for '+' prefixed lines
2018-05-02 16:15:26 +02:00
Yu Watanabe 76283e5fd4 set: drop unused set_make() function (#8879)
The function causes compiler error when built with '-Ddebug=hashmap',
and is not used anymore. Let's drop it.
2018-05-02 10:54:52 +02:00
Yu Watanabe cfa24ca0e6 test: add tests for PrivateDevices= with '+' prefix 2018-05-01 13:44:24 +09:00
Lennart Poettering cf409d15fa tree-wide: use newa() rather than alloca() where we can 2018-04-27 14:29:06 +02:00
Lennart Poettering da6053d0a7 tree-wide: be more careful with the type of array sizes
Previously we were a bit sloppy with the index and size types of arrays,
we'd regularly use unsigned. While I don't think this ever resulted in
real issues I think we should be more careful there and follow a
stricter regime: unless there's a strong reason not to use size_t for
array sizes and indexes, size_t it should be. Any allocations we do
ultimately will use size_t anyway, and converting forth and back between
unsigned and size_t will always be a source of problems.

Note that on 32bit machines "unsigned" and "size_t" are equivalent, and
on 64bit machines our arrays shouldn't grow that large anyway, and if
they do we have a problem, however that kind of overly large allocation
we have protections for usually, but for overflows we do not have that
so much, hence let's add it.

So yeah, it's a story of the current code being already "good enough",
but I think some extra type hygiene is better.

This patch tries to be comprehensive, but it probably isn't and I missed
a few cases. But I guess we can cover that later as we notice it. Among
smaller fixes, this changes:

1. strv_length()' return type becomes size_t

2. the unit file changes array size becomes size_t

3. DNS answer and query array sizes become size_t

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=76745
2018-04-27 14:29:06 +02:00
Zbigniew Jędrzejewski-Szmek 854a42fb2e analyze: add 'cat-config' verb
This is used as 'systemd-analyze show-config systemd/logind.conf', which
will dump
   /etc/systemd/system/user@.service
   /etc/systemd/system/user@.service.d/*.conf
   /run/systemd/system/user@.service.d/*.conf
   /usr/local/lib/systemd/system/user@.service.d/*.conf
   /usr/lib/systemd/system/user@.service.d/*.conf

The idea is to make it easy to dump the configuration using the same locations
and order that systemd programs use themselves (including masking, in the right
order, etc.). This is the generic variant that works with any configuration
scheme that follows the same general rules:

$ systemd-analyze cat-config systemd/system.conf
$ systemd-analyze cat-config systemd/user.conf
$ systemd-analyze cat-config systemd/logind.conf
$ systemd-analyze cat-config systemd/sleep.conf
$ systemd-analyze cat-config systemd/journald.conf
$ systemd-analyze cat-config systemd/journal-remote.conf
$ systemd-analyze cat-config systemd/journal-upload.conf
$ systemd-analyze cat-config systemd/coredump.conf
$ systemd-analyze cat-config systemd/resolved.conf
$ systemd-analyze cat-config systemd/timesyncd.conf
$ systemd-analyze cat-config udev/udev.conf
2018-04-27 10:06:24 +02:00
Lennart Poettering 385f3a0d8d
Merge pull request #7599 from keszybz/slice-templates
Make user@.service independent of logind
2018-04-26 21:39:05 +02:00
Zbigniew Jędrzejewski-Szmek 81f5e51368 Move function to cat file & dropins into basic/
This fixes a buglet where the second and later drop-in would not be seperated
properly by a newline.
2018-04-26 13:52:46 +02:00
Lennart Poettering f91c6093ef parse-util: add explicit parsers for MTU values
We use MTUs all over the place, let's add a unified, strict parser for
it, that takes MTU ranges into account.

We already have parse_ifindex() close-by, hence this appears to be a
natural addition, in particular as the range checking is not entirely
trivial to do, as it depends on the protocol used.
2018-04-26 13:51:44 +02:00
Lennart Poettering 8e766630f0 tree-wide: drop redundant _cleanup_ macros (#8810)
This drops a good number of type-specific _cleanup_ macros, and patches
all users to just use the generic ones.

In most recent code we abstained from defining type-specific macros, and
this basically removes all those added already, with the exception of
the really low-level ones.

Having explicit macros for this is not too useful, as the expression
without the extra macro is generally just 2ch wider. We should generally
emphesize generic code, unless there are really good reasons for
specific code, hence let's follow this in this case too.

Note that _cleanup_free_ and similar really low-level, libc'ish, Linux
API'ish macros continue to be defined, only the really high-level OO
ones are dropped. From now on this should really be the rule: for really
low-level stuff, such as memory allocation, fd handling and so one, go
ahead and define explicit per-type macros, but for high-level, specific
program code, just use the generic _cleanup_() macro directly, in order
to keep things simple and as readable as possible for the uninitiated.

Note that before this patch some of the APIs (notable libudev ones) were
already used with the high-level macros at some places and with the
generic _cleanup_ macro at others. With this patch we hence unify on the
latter.
2018-04-25 12:31:45 +02:00
Lennart Poettering 695a944c64 string-util: teach strip_tab_ansi() to deal with CSO sequences
With the recent terminal_urlify() APIs we'll now sometimes generate
clickable link CSO sequences. Hence we should also be able to remove
them again from strings. This beefs up the logic to do so.

Follow-up for: 23b27b39d2
2018-04-24 15:58:31 +02:00
Zbigniew Jędrzejewski-Szmek 250e9fadbc Add %j/%J unit specifiers
Those are quite similar to %i/%I, but refer to the last dash-separated
component of the name prefix.

The new functionality of dash-dropins could largely supersede the template
functionality, so it would be tempting to overload %i/%I. But that would
not be backwards compatible. So let's add the two new letters instead.
2018-04-24 10:05:04 +02:00
Zbigniew Jędrzejewski-Szmek 0682ed5cf4 tests: add some tests for unit_name_is_valid() and related functions
I was surprised to see that foo@bar@bar.service is a valid unit
name. Apparently it is according to current code and docs.
2018-04-24 09:59:03 +02:00
Yu Watanabe c9825701e6 test: drop unused variables
Follow-up for 49eb36596b.
2018-04-24 15:43:41 +09:00
Lennart Poettering 23b27b39d2 terminal: add internal API to format URLs for display in capable terminals
Newer terminals (in particular gnome-terminal) understand special escape
sequence for formatting clickable links. Let's support that to make our
tool output more clickable where that's appropriate.

For details see this:

https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

The one big issue is that 'less' currently doesn't grok this, and
doesn't ignore sequence like regular terminal implementations do if they
don't support it. Hence for now, let's disable URL output if a pager is
used. We should revisit that though as soon as less added support for it
and enough time passed for it to enter various distributions.
2018-04-19 18:04:26 +02:00
Lennart Poettering 7a9a0c05d4
Merge pull request #8765 from poettering/test-fixes
some short fixes for the tests
2018-04-19 16:18:46 +02:00
Lennart Poettering 5d13a15b1d tree-wide: drop spurious newlines (#8764)
Double newlines (i.e. one empty lines) are great to structure code. But
let's avoid triple newlines (i.e. two empty lines), quadruple newlines,
quintuple newlines, …, that's just spurious whitespace.

It's an easy way to drop 121 lines of code, and keeps the coding style
of our sources a bit tigther.
2018-04-19 12:13:23 +02:00
Lennart Poettering 63d6135fea tests: compare with correct shell
We synthesize the passwd record for UID 0, hence we need to compare with
our synthesized data and not with the data stored in /etc/passwd
2018-04-19 11:41:28 +02:00
Lennart Poettering 7aab22308e
Merge pull request #8708 from poettering/namespace-repeat
pid1 namespacing fixes
2018-04-18 18:46:44 +02:00
Lennart Poettering 613bddf7d1
Merge pull request #8709 from poettering/format-table
generic table formatter
2018-04-18 16:20:13 +02:00
Lennart Poettering 57ea45e11a util-lib: introduce new empty_or_root() helper (#8746)
We check the same condition at various places. Let's add a trivial,
common helper for this, and use it everywhere.

It's not going to make things much faster or much shorter, but I think a
lot more readable
2018-04-18 14:20:49 +02:00
Lennart Poettering 49eb36596b fs-util: add new CHASE_STEP flag to chase_symlinks()
If the flag is set only a single step of the normalization is executed,
and the resulting path is returned.

This allows callers to normalize piecemeal, taking into account every
single intermediary path of the normalization.
2018-04-18 14:15:48 +02:00
Zbigniew Jędrzejewski-Szmek e206fcc164 test-locale-util: show special glyphs
This is mostly useful as a sanity check.
2018-04-18 12:51:15 +02:00
Zbigniew Jędrzejewski-Szmek 7c6c2e07fc test-utf8: add a smoke test for utf8_console_width() 2018-04-18 12:51:15 +02:00
Lennart Poettering 1960e73611 basic: add minimalistic table formatter
We have plenty of code in our codebase that outputs tables to the
console, and all is homegrown and awful. Let's replace it with a generic
implementation that can do automatically what the old implementations
did manually.

Features:

1. Ellipsation (for fields overly long) and alignment (for
   fields overly short)

2. Sorting of rows

3. automatically copies formatting from the same cell in the row above

4. Heavy use of varargs to make putting together tables easy

5. can expand and compress tables, with weights

6. Has a minimal understanding of unicode wide characters in order to
   match unicode strings to character cell terminals.

7. Columns can be reordered and individually turned off.

8. pretty printing for various data types

And more.
2018-04-18 12:51:15 +02:00
Lennart Poettering c30a49b2d0 string-util: tweak ellipsation a bit
This primarily changes to things:

1. Ellipsation to 0, 1 or 2 characters is now supported. Previously we'd
   hit an assert if the new lengths was < 3, this is now permitted. The
   result strings won't show too much info still of course, but the code
   becomes a bit more generic and robust to use.

2. If a UTF-8 mode is disabled and the input string is pure ASCII, then
   "..." is used for ellipsation, otherwise (as before) "…". This means
   on a pure-ASCII system we should remain pure-ASCII, matching
   behaviour otherwise exposed with special_glyph() and friends. Note
   that we'll use "…" for ellipsiation as soon as either the locale
   settings indicate an UTF-8 mode or the input string already contains
   non-ASCII unicode characters.

Testing for these special cases is improved.
2018-04-18 12:35:45 +02:00
Lennart Poettering 2cb36f7c1e
Merge pull request #8575 from keszybz/non-absolute-paths
Do not require absolute paths in ExecStart and friends
2018-04-17 15:54:10 +02:00
Zbigniew Jędrzejewski-Szmek 7e4a49b42b test-execute: make find invocation a bit more efficent, increase timeout
We go through the whole file system, so this test can take arbitrary time.  But
this test is still quite useful, so let's at least try to make it more efficent
by not descending at all into the directories we would filter out later on
anyway.

Also increase the timeout, in case the previous step doesn't help enough.
2018-04-16 16:09:46 +02:00
Zbigniew Jędrzejewski-Szmek 5008da1ec1 systemd: do not require absolute paths in ExecStart
Absolute paths make everything simple and quick, but sometimes this requirement
can be annoying. A good example is calling 'test', which will be located in
/usr/bin/ or /bin depending on the distro. The need the provide the full path
makes it harder a portable unit file in such cases.

This patch uses a fixed search path (DEFAULT_PATH which was already used as the
default value of $PATH), and if a non-absolute file name is found, it is
immediately resolved to a full path using this search path when the unit is
loaded. After that, everything behaves as if an absolute path was specified. In
particular, the executable must exist when the unit is loaded.
2018-04-16 16:09:46 +02:00
Zbigniew Jędrzejewski-Szmek ba412430a9 tests: use manager_load_startable_unit_or_warn() to load units
Doing manager_load_unit() followed by UNIT_VTABLE(unit)->start(unit) would
result in an assertion failure in ->start() if the unit failed to load
properly. Something like this is okey-ish is tests, since the test units are
not expected to fail to load, but the reason for failure is clearer if we
fail immediately.
2018-04-16 16:08:52 +02:00
Lennart Poettering 7ce49e656b test: add test for prefix unit loading 2018-04-13 11:34:48 +02:00
Lennart Poettering 72406c2f7d unit-name: add new unit_name_build_from_type() helper
The new helper is much like unit_name_build() but expects a UnitType
value instead of a suffix.
2018-04-13 11:34:48 +02:00
Lennart Poettering 8460289f35 path-util: document a few other special cases for last_path_component() 2018-04-12 11:02:47 +02:00
Lennart Poettering b667d50d34
Merge pull request #8700 from keszybz/hibernation
Various improvements related to hibernation
2018-04-11 10:26:27 +02:00
Zbigniew Jędrzejewski-Szmek b71c9758d1 shared/sleep-config: return a custom message when not enough swap for hibernation
$ sudo swapoff -av
swapoff /dev/vda4
$ sudo systemctl hibernate
Failed to hibernate system via logind: Not enough swap space for hibernation

Fixes #6729.
2018-04-11 09:26:14 +02:00
Lennart Poettering 4d09e1c8ba
Merge pull request #8676 from keszybz/drop-license-boilerplate
Drop license boilerplate
2018-04-10 14:53:31 +02:00
Zbigniew Jędrzejewski-Szmek aad69cce2d
Merge pull request #8406 from dell/hibernate-disk-offset
Add support for setting a disk offset when hibernating
2018-04-10 09:53:17 +02:00
Mario Limonciello 17c40b3a8f sleep: Add support for setting a disk offset when hibernating
The Linux kernel is adding support for configuring the offset
into a disk.  This allows swapfiles to be more usable as users
will no longer need to set the offset on their kernel command
line.

Use this API in systemd when hibernating as well.

Signed-off-by: Mario Limonciello <mario.limonciello@dell.com>
2018-04-09 13:17:56 -05:00
Zbigniew Jędrzejewski-Szmek 11a1589223 tree-wide: drop license boilerplate
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.

I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
2018-04-06 18:58:55 +02:00
Zbigniew Jędrzejewski-Szmek c0f9017c11
Merge pull request #8605 from poettering/drop-in-name-fix
Fix validation of unit file drop-in naming in install.c.
2018-04-04 11:52:14 +02:00
Filipe Brandenburger edb3ca0d61 test-path: Set umask explicitly (#8616)
Running `test-path` under an umask such as 027 fails with:

  Assertion '(s.st_mode & S_IRWXO) == 0004' failed at ../src/test/test-path.c:247, function test_path_makedirectory_directorymode(). Aborting.

Looking at directory /tmp/test-path_makedirectory, it was indeed created
with mode 0740, applying the umask to the requested 0744.

Set an explicit umask for this test, to ensure reproducible results.
2018-04-04 11:17:19 +02:00
Lennart Poettering f2324783ce fd-util: introduce fd_reopen() helper for reopening an fd
We have the same code for this in place at various locations, let's
unify that. Also, let's repurpose test-fs-util.c as a test for this new
helper cal..
2018-03-29 15:33:12 +02:00
Mario Limonciello e68c79db91 Rename suspend-to-hibernate to suspend-then-hibernate
Per some discussion with Gnome folks, they would prefer this name
as it's more descriptive of what's happening.
2018-03-28 15:11:10 -05:00
Zbigniew Jędrzejewski-Szmek 435ce146ac basic/strbuf: include empty strings in count
Not that it matters much, but it seems cleaner to also count those
inputs, even if they do not consume extra storage space.

The test is extended to include an empty input and counts in the test are
adjusted to include it.
2018-03-26 15:28:03 +02:00
Lennart Poettering 959071cac2
Merge pull request #8552 from keszybz/test-improvements
Test and diagnostics improvements
2018-03-23 15:26:54 +01:00
Zbigniew Jędrzejewski-Szmek 37c1d5e97d tree-wide: warn when a directory path already exists but has bad mode/owner/type
When we are attempting to create directory somewhere in the bowels of /var/lib
and get an error that it already exists, it can be quite hard to diagnose what
is wrong (especially for a user who is not aware that the directory must have
the specified owner, and permissions not looser than what was requested). Let's
print a warning in most cases. A warning is appropriate, because such state is
usually a sign of borked installation and needs to be resolved by the adminstrator.

$ build/test-fs-util

Path "/tmp/test-readlink_and_make_absolute" already exists and is not a directory, refusing.
   (or)
Directory "/tmp/test-readlink_and_make_absolute" already exists, but has mode 0775 that is too permissive (0755 was requested), refusing.
   (or)
Directory "/tmp/test-readlink_and_make_absolute" already exists, but is owned by 1001:1000 (1000:1000 was requested), refusing.

Assertion 'mkdir_safe(tempdir, 0755, getuid(), getgid(), MKDIR_WARN_MODE) >= 0' failed at ../src/test/test-fs-util.c:320, function test_readlink_and_make_absolute(). Aborting.

No functional change except for the new log lines.
2018-03-23 10:26:38 +01:00
Zbigniew Jędrzejewski-Szmek d50b5839b0 basic/mkdir: convert bool flag to enum
In preparation for subsequent changes...
2018-03-22 15:57:56 +01:00
Zbigniew Jędrzejewski-Szmek 37cbc1d579 When mangling names, optionally emit a warning (#8400)
The warning is not emitted for absolute paths like /dev/sda or /home, which are
converted to .device and .mount unit names without any fuss.

Most of the time it's unlikely that users use invalid unit names on purpose,
so let's warn them. Warnings are silenced when --quiet is used.

$ build/systemctl show -p Id hello@foo-bar/baz
Invalid unit name "hello@foo-bar/baz" was escaped as "hello@foo-bar-baz" (maybe you should use systemd-escape?)
Id=hello@foo-bar-baz.service

$ build/systemd-run --user --slice foo-bar/baz --unit foo-bar/foo true
Invalid unit name "foo-bar/foo" was escaped as "foo-bar-foo" (maybe you should use systemd-escape?)
Invalid unit name "foo-bar/baz" was escaped as "foo-bar-baz" (maybe you should use systemd-escape?)
Running as unit: foo-bar-foo.service

Fixes #8302.
2018-03-21 15:26:47 +01:00
Zbigniew Jędrzejewski-Szmek 55c36ec0c1
Merge pull request #8508 from poettering/more-cocci
two new coccinelle rules files and their results
2018-03-21 12:50:49 +01:00
Alex Gartrell 1b7cf0e587 journal: make the compression threshold tunable
Allow a user to set a number of bytes as Compress to use as the compression
threshold.
2018-03-20 14:54:07 -07:00
Lennart Poettering 43dc7aa2ba coccinelle: always use fcntl(fd, FD_DUPFD, 3) instead of dup(fd)
Let's avoid fds 0…2 for safety reasons.
2018-03-20 22:31:14 +01:00
Lennart Poettering 8c637fe242
Merge pull request #8452 from keszybz/use-libmount-more
Use libmount in systemd-shutdown, add tests
2018-03-20 09:53:34 +01:00
Yu Watanabe bcabcde5d2
Merge pull request #8408 from keszybz/ln-relative
bugs.fd.o bug archelogy
2018-03-19 18:32:30 +09:00
Evgeny Vereshchagin e4711004d6
Merge pull request #8461 from keszybz/oss-fuzz-fixes
Oss fuzz fixes
2018-03-19 00:06:44 +03:00
Zbigniew Jędrzejewski-Szmek 5685efde88 test-socket-util: add test for ListenNetlink with spaces 2018-03-18 21:05:43 +01:00
Zbigniew Jędrzejewski-Szmek 10062bbc35 unit-file: do not allow bogus IOSchedulingClass values
We have only three bits of space, i.e. 8 possible classes. Immediately reject
anything outside of that range. Add the fuzzer test case and an additional
unit test.

oss-fuzz #6908.
2018-03-17 09:01:53 +01:00
Filipe Brandenburger 9b796f3523 test-execute: Introduce tests for environment values containing newlines
Also fix one case where the presence of a newline was used to generate
an invalid environment assignment.

Tested: with mkosi, which builds the local tree and run ninja tests.
2018-03-16 16:31:17 -07:00
Filipe Brandenburger b4346b9a77 basic/env-util: Allow newlines in values of environment variables
They are allowed by the shell and the EnvironmentFile parsing passes
them through, so we should just accept them, same as we accept tabs.
2018-03-16 13:45:03 -07:00
Zbigniew Jędrzejewski-Szmek 71ae04c400 core/umount: use libmount to enumerate /proc/swaps
example.swaps with "(deleted)" does not cause bogus entries in the list now,
but a memleak in libmount instead. The memleaks is not very important since
this code is run just once.
Reported as https://github.com/karelzak/util-linux/issues/596.

$ build/test-umount
...
/* test_swap_list("/proc/swaps") */
path=/var/tmp/swap o= f=0x0 try-ro=no dev=0:0
path=/dev/dm-2 o= f=0x0 try-ro=no dev=0:0
/* test_swap_list("/home/zbyszek/src/systemd/test/test-umount/example.swaps") */
path=/some/swapfile o= f=0x0 try-ro=no dev=0:0
path=/dev/dm-2 o= f=0x0 try-ro=no dev=0:0
==26912==
==26912== HEAP SUMMARY:
==26912==     in use at exit: 16 bytes in 1 blocks
==26912==   total heap usage: 1,546 allocs, 1,545 frees, 149,008 bytes allocated
==26912==
==26912== 16 bytes in 1 blocks are definitely lost in loss record 1 of 1
==26912==    at 0x4C31C15: realloc (vg_replace_malloc.c:785)
==26912==    by 0x55C5D8C: _IO_vfscanf (in /usr/lib64/libc-2.26.so)
==26912==    by 0x55D8AEC: vsscanf (in /usr/lib64/libc-2.26.so)
==26912==    by 0x55D25C3: sscanf (in /usr/lib64/libc-2.26.so)
==26912==    by 0x53236D0: mnt_table_parse_stream (in /usr/lib64/libmount.so.1.1.0)
==26912==    by 0x53249B6: mnt_table_parse_file (in /usr/lib64/libmount.so.1.1.0)
==26912==    by 0x10D157: swap_list_get (umount.c:194)
==26912==    by 0x10B06E: test_swap_list (test-umount.c:34)
==26912==    by 0x10B24B: main (test-umount.c:56)
==26912==
==26912== LEAK SUMMARY:
==26912==    definitely lost: 16 bytes in 1 blocks
==26912==    indirectly lost: 0 bytes in 0 blocks
==26912==      possibly lost: 0 bytes in 0 blocks
==26912==    still reachable: 0 bytes in 0 blocks
==26912==         suppressed: 0 bytes in 0 blocks
2018-03-16 10:12:50 +01:00
Zbigniew Jędrzejewski-Szmek 1fd8edb53a test-umount: add a simple test for swap_list_get()
The implementation seems buggy:
/* test_swap_list("/home/zbyszek/src/systemd/test/test-umount/example.swaps") */
path=0 o= f=0x0 try-ro=no dev=0:0
path=/some/swapfile2 o= f=0x0 try-ro=no dev=0:0
path=/some/swapfile o= f=0x0 try-ro=no dev=0:0
path=/dev/dm-2 o= f=0x0 try-ro=no dev=0:0
2018-03-16 10:12:50 +01:00
Zbigniew Jędrzejewski-Szmek a6dcd22976 core/umount: use _cleanup_ 2018-03-16 10:12:50 +01:00
Zbigniew Jędrzejewski-Szmek 6fa392bf91 tests: add a simple test for the mountinfo parsing logic 2018-03-16 10:12:50 +01:00
Zbigniew Jędrzejewski-Szmek 5ee45c6d63 test-calendarspec: add the test case from oss-fuzz 6886
Before the fix 55a30fd4e8 in this would crash in calendarspec_from_time_t().
2018-03-15 00:15:15 +01:00
Zbigniew Jędrzejewski-Szmek 8d89e51f3c fuzz: commit test case for oss-fuzz issue 6884
This seems to be a false positive in msan:
https://github.com/google/sanitizers/issues/767.

I don't see anything wrong with the code either, and valgrind does not see the
issue. Anyway, let's add the test case.

We don't have msan hooked up yet, but hopefully we'll in the future.

oss-fuzz #6884.
2018-03-14 15:20:02 +01:00
Zbigniew Jędrzejewski-Szmek e8112e67e4 Make MANAGER_TEST_RUN_MINIMAL just allocate data structures
When running tests like test-unit-name, there is not point in setting
up the cgroup and signals and interacting with the environment. Similarly
when running fuzz testing of the parser.

Add new MANAGER_TEST_RUN_BASIC which takes the role of MANAGER_TEST_RUN_MINIMAL,
and redefine MANAGER_TEST_RUN_MINIMAL to just create the basic data structures.
2018-03-11 16:33:59 +01:00
Zbigniew Jędrzejewski-Szmek dc409696cf Introduce _cleanup_(unit_freep) 2018-03-11 16:33:58 +01:00
Zbigniew Jędrzejewski-Szmek c70cac548a Introduce _cleanup_(manager_freep) 2018-03-11 16:33:57 +01:00
Evegeny Vereshchagin c58fd466a3 tests: make / private after creating a mount namespace
so that the test never affects the root namespace.
2018-03-09 14:39:26 +00:00
Evegeny Vereshchagin 112cc3b5b2 tests: skip the rest of test_mnt_id after getting any error
This mainly gets around a kernel bug making it possible to
have non-existent paths in /proc/self/mountinfo, but it should also
prevent flaky failures that can happen if something changes immediately
after or during reading /proc/self/mountinfo.

Closes https://github.com/systemd/systemd/issues/8286.
2018-03-09 14:39:11 +00:00
Zbigniew Jędrzejewski-Szmek 8750ac0238 pid1: make use of high rt signals on hppa with newer kernels
Back in 4dffec1459 we stopped using SIGRTMIN+26
and higher on hppa because they were not available. Then they became available
in linux 3.18:

  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1f25df2eff5b25f52c139d3ff31bc883eee9a0ab

Instead of hard-coding the list based on architecture, let's use a runtime
check like signal(7) says.

(A note about implementation: RTSIG_IF_AVAILABLE is defined to take the full
signal and not just an offset from SIGRTMIN so that it's still possible to
grep for SIGRTMIN\+.)

Add a simple "test" to print the signal values.

Fixes https://bugs.freedesktop.org/show_bug.cgi?id=84931.
2018-03-09 10:35:33 +01:00
Mario Limonciello c58493c00a Introduce suspend-to-hibernate (#8274)
Suspend to Hibernate is a new sleep method that invokes suspend
for a predefined period of time before automatically waking up
and hibernating the system.

It's similar to HybridSleep however there isn't a performance
impact on every suspend cycle.

It's intended to use with systems that may have a higher power
drain in their supported suspend states to prevent battery and
data loss over an extended suspend cycle.

Signed-off-by: Mario Limonciello <mario.limonciello@dell.com>
2018-03-08 14:17:33 +01:00
Zbigniew Jędrzejewski-Szmek 18ce247c4c test-cgroup-util: bail out when running under mock (#8365)
The builds were failing in Fedora koji, where something strange is mounted
on /sys/fs/cgroup.

Also closes https://github.com/systemd/systemd/issues/8383.
2018-03-08 00:41:25 +03:00
Evegeny Vereshchagin 9a92e25545 tests: close a leftover file descriptor in test-fileio
This should make it a bit easier to search for real file descriptor leaks.

```
$ valgrind --leak-check=full --track-fds=yes ./build/test-fileio
...
==29457==
==29457== FILE DESCRIPTORS: 4 open at exit.
==29457== Open file descriptor 3: /tmp/test-systemd_writing_tmpfile.lyV5Rc
==29457==    at 0x4B9AD9E: open (open.c:43)
==29457==    by 0x4B19B24: __gen_tempname (tempname.c:261)
==29457==    by 0x4BA5CC3: mkostemp64 (mkostemp64.c:32)
==29457==    by 0x48F739B: mkostemp_safe (fileio.c:1206)
==29457==    by 0x10D968: test_writing_tmpfile (test-fileio.c:620)
==29457==    by 0x10E930: main (test-fileio.c:767)
==29457==
```
2018-03-05 22:17:23 +00:00
Evegeny Vereshchagin 362973902c tests: stop using freopen in test-fileio
This helps get around a bug confusing `glibc` and making the test bail
out with the following error under `asan` on `x86`:

Fatal error: glibc detected an invalid stdio handle
Aborted (core dumped)

The bug has been reported in https://github.com/google/sanitizers/issues/778,
but it is unlikely to be fixed anytime soon.
2018-03-05 21:57:14 +00:00
Zbigniew Jędrzejewski-Szmek de5fbba9e2
Merge pull request #8341 from yuwata/test-execute-ambient
test-execute: check capabilities before runnig tests
2018-03-05 17:45:28 +01:00
Zbigniew Jędrzejewski-Szmek b719b66b0a
Merge pull request #8354 from keszybz/new-NEWS
NEWS for v238
2018-03-05 17:16:04 +01:00
Zbigniew Jędrzejewski-Szmek 46dcfbbdb6 basic/glob-util: do not use names with "_" prefix
Names starting with _ or __ are reserved by the standard, better to
avoid them.
2018-03-05 09:04:39 +01:00
Yu Watanabe e5ba1d324d test-execute: use CAP_CHOWN instead of CAP_NET_ADMIN
CAP_NET_ADMIN is somtrimes dropped by container runtime.
This changes to use CAP_CHOWN instead of CAP_NET_ADMIN, as it is
less likely to be dropped.
2018-03-05 00:02:22 +09:00
Yu Watanabe b7856f9218 test-execute: check capabilities before running tests
Fixes #8193.
2018-03-05 00:00:35 +09:00
Yu Watanabe 5cd33ccc2c test-execute: change log level from error to notice
Skipping some tests due to the missing e.g., capsh binary
or kernel support, are not error.
This changes the log level for such messages.
2018-03-04 16:53:50 +09:00
Yu Watanabe b6dc25eebc test-execute: rename tests for AmbientCapabilities=
The unit files for test-execute are named like
`exec-(setting-name-in-lower-character)-(optional-text).service`.
However, test units for AmbientCapabilities= are not following this.
So, let's rename them for the consistency.
This does not change anything in the functionality of the test.
2018-03-04 16:37:58 +09:00
Lennart Poettering 6cc7e918ff
Merge pull request #8314 from poettering/rearrange-stdio
refactor how we rearrange fds for stdin/stdout/stderr
2018-03-02 15:42:03 +01:00
Zbigniew Jędrzejewski-Szmek 48449f41b4
Merge pull request #8303 from yuwata/fix-8276
test: use synthesize_nobody() in test-execute
2018-03-02 12:53:11 +01:00
Lennart Poettering aa11e28bf2 fd-util: add new call rearrange_stdio()
Quite often we need to set up a number of fds as stdin/stdout/stderr of
a process we are about to start. Add a generic implementation for a
routine doing that that takes care to do so properly:

1. Can handle the case where stdin/stdout/stderr where previously
   closed, and the fds to set as stdin/stdout/stderr hence likely in the
   0..2 range.  handling this properly is nasty, since we need to first
   move the fds out of this range in order to later move them back in, to
   make things fully robust.

2. Can optionally open /dev/null in case for one or more of the fds, in
   a smart way, sharing the open file if possible between multiple of
   the fds.

3. Guarantees that O_CLOEXEC is not set on the three fds, even if the fds
   already were in the 0..2 range and hence possibly weren't moved.
2018-03-02 11:42:10 +01:00
Yu Watanabe 69b0740785 test-execute: add tests with user/group daemon
The nobody user/group may not synthesized by systemd.
To run tests the functionalities in such situation, this adds tests
by user/group by daemon, as it is expected to exists all environments.
2018-03-02 16:42:53 +09:00
Douglas Christman 63547675a4 test-libudev: make "-m" equivalent to "--monitor"
"-m" is specified as a short form of "--monitor" in the option struct,
but not included in getopt_long's optstring.  Update the optstring
to be consistent with the option struct.
2018-03-01 21:50:38 +08:00
Lennart Poettering 902c8502ad
Merge pull request #8149 from poettering/fake-root-cgroup
Properly synthesize CPU+memory accounting data for the root cgroup
2018-03-01 11:10:24 +01:00
Yu Watanabe d2f95a223f test-execute: add a test for the case that NOBODY_GROUP_NAME is nogroup 2018-03-01 18:31:26 +09:00
Yu Watanabe 57c2efa0c3 test-execute: check nobody user and group are configured correctly
Several tests request nobody user or group. If they are badly
configured, then tests may fail.

This makes test-execute check nobody user and group are configured
correctly before running such tests.

Fixes #8276.
2018-03-01 18:29:28 +09:00
Zbigniew Jędrzejewski-Szmek 9177fa9f2b basic/cgroup-util: simplify cg_get_keyed_attribute(), add test
I didn't like the nested loop where we'd count what we have acquired already,
since we should always know that.
2018-03-01 09:34:33 +01:00
Evgeny Vereshchagin b8eded40be
Merge pull request #8282 from poettering/khash-enokey
deal with borked ENOKEY on centos kernel's AF_ALG support
2018-02-27 12:34:41 +03:00
Lennart Poettering 09b9348e82 khash: try to detect broken AF_ALG support in centos kernels
Fixes: #8278
2018-02-26 15:52:16 +01:00
Zbigniew Jędrzejewski-Szmek e4de62591b basic/xattr-util: do not cast ssize_t to int
gcc warns about unitialized memory access because it notices that ssize_t which
is < 0 could be cast to positive int value. We know that this can't really
happen because only -1 can be returned, but OTOH, in principle a large
*positive* value cannot be cast properly. This is unlikely too, since xattrs
cannot be too large, but it seems cleaner to just use a size_t to return the
value and avoid the cast altoghter. This makes the code simpler and gcc is
happy too.

The following warning goes away:
[113/1502] Compiling C object 'src/basic/basic@sta/xattr-util.c.o'.
In file included from ../src/basic/alloc-util.h:28:0,
                 from ../src/basic/xattr-util.c:30:
../src/basic/xattr-util.c: In function ‘fd_getcrtime_at’:
../src/basic/macro.h:207:60: warning: ‘b’ may be used uninitialized in this function [-Wmaybe-uninitialized]
                 UNIQ_T(A,aq) < UNIQ_T(B,bq) ? UNIQ_T(A,aq) : UNIQ_T(B,bq); \
                                                            ^
../src/basic/xattr-util.c:155:19: note: ‘b’ was declared here
         usec_t a, b;
                   ^
2018-02-26 15:49:18 +01:00
Yu Watanabe ac1f08b92b test: use setup_fake_runtime_dir() in test-execute 2018-02-26 12:50:03 +09:00
Zbigniew Jędrzejewski-Szmek f8e6f4aaed basic/log: make sure header is printed correctly, add test
If log_do_header() was called with overly long parameters, it'd generate
improper output. Essentially, it'd be truncated at random point, in particular
missing a newline at the end, so it'd run with the next field, usually MESSAGE=.

log_do_header is called with parameters from compiled code (file name, lien
nubmer, etc), so in practice this was unlikely to ever be a problem, but it is
possible. In particular, if systemd was compiled from sources in some deeply
nested directory (which happens for example in mock and other build roots), the
filename could be very long.

As a safety measure, let's truncate all parameters to 256 bytes. So we have
5 fields which are 256 bytes (plus the field name prefix), and a few other
fields with fixed width. This must always fit in the 2048 byte buffer.
I don't think there's much gain in calculating the required length precisely,
since it's a lot of fields and a few bytes allocated on the stack don't matter.
2018-02-23 11:13:42 +01:00
Patrick Uiterwijk 5547c12503 Fix format-truncation compile failure by typecasting USB IDs (#8250)
This patch adds safe_atoux16 for parsing an unsigned hexadecimal 16bit int, and
uses that for parsing USB device and vendor IDs.

This fixes a compile error with gcc-8 because while we know that USB IDs are 2 bytes,
the compiler does not know that.

../src/udev/udev-builtin-hwdb.c:80:38: error: '%04X' directive output may be
truncated writing between 4 and 8 bytes into a region of size between 2 and 6
[-Werror=format-truncation=]

Signed-off-by: Adam Williamson <awilliam@redhat.com>
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2018-02-22 19:41:30 +01:00
Lennart Poettering 2ae7ee58fa bpf: beef up bpf detection, check if BPF_F_ALLOW_MULTI is supported
This improves the BPF/cgroup detection logic, and looks whether
BPF_ALLOW_MULTI is supported. This flag allows execution of multiple
BPF filters in a recursive fashion for a whole cgroup tree. It enables
us to properly report IP accounting for slice units, as well as
delegation of BPF support to units without breaking our own IP
accounting.
2018-02-21 16:43:36 +01:00
Evgeny Vereshchagin 7b13a721f5
Merge pull request #8235 from keszybz/skip-nobody-test
Skip tests for nobody if necessary
2018-02-21 12:19:02 +03:00
Zbigniew Jędrzejewski-Szmek 7559b2da10 test-user-util: skip most tests for nobody if synthentization is off
When synthetisation is turned off, there's just too many ways those tests can
go wrong. We are not interested in verifying that the db on disk is correct,
let's just skip all checks.

In the first version of this patch, I recorded if we detected a mismatch during
configuration and only skipped tests in that case, but actually it is possible
to change the host configuration between our configuration phase and running
of the tests. It's just more robust to skip always. (This is particularly true
if tests are installed.)
2018-02-21 09:57:35 +01:00
Alan Jenkins 59e00b2a16
Merge pull request #7908 from yuwata/rfe-7895
core: add TemporaryFileSystem= setting and 'tmpfs' option to ProtectHome=
2018-02-21 08:57:11 +00:00
Yu Watanabe 4cac89bd7c test: add tests for TemporaryFileSystem= 2018-02-21 09:18:14 +09:00
Yu Watanabe 2abd4e388a core: add new setting TemporaryFileSystem=
This introduces a new setting TemporaryFileSystem=. This is useful
to hide files not relevant to the processes invoked by unit, while
necessary files or directories can be still accessed by combining
with Bind{,ReadOnly}Paths=.
2018-02-21 09:17:52 +09:00
Yu Watanabe f27b437b4c test: add tests for mount_option_mangle() 2018-02-21 09:06:51 +09:00
Zbigniew Jędrzejewski-Szmek 52c6e6a8a0 test-user-util: print function delimiters
This makes it easier to see what is going on. Crashes may happen in a
nested test_{uid,gid}_to_name_one() function, and the default backtrace
doesn't show the actual string being tested.
2018-02-20 22:10:45 +01:00
Lennart Poettering 4c2e1b399f xattr-util: use crtime/btime if statx() is available for implementation of fd_setcrtime() and friends
The Linux kernel exposes the birth time now for files through statx()
hence make use of it where available. We keep the xattr logic in place
for this however, since only a subset of file systems on Linux currently
expose the birth time. NFS and tmpfs for example do not support it. OTOH
there are other file systems that do support the birth time but might
not support xattrs (smb…), hence make the best of the two, in particular
in order to deal with journal files copied between file system types and
to maintain compatibility with older file systems that are updated to
newer version of the file system.
2018-02-20 15:41:49 +01:00
Lennart Poettering 11b29a96e9 fs-util: move fsync_directory_of_file() into generic code
This function used by the journal code is pretty useful generically,
let's move it to fs-util.c to make it useful for other code too.
2018-02-20 15:39:31 +01:00
Lennart Poettering 30663b6c25
Merge pull request #8199 from keszybz/small-things
Sundry small cleanups
2018-02-19 16:55:10 +01:00
Zbigniew Jędrzejewski-Szmek e2cbc803f0 tests: add a simple test for utf8_n_codepoints() 2018-02-19 15:18:54 +01:00
Zbigniew Jędrzejewski-Szmek 56c8d7444a pid1: do not initialize join_controllers by default
We're moving towards unified cgroup hierarchy where this is not necessary.
This makes main.c a bit simpler.
2018-02-19 15:18:54 +01:00
Zbigniew Jędrzejewski-Szmek 9ecdba8cb7 Move config_parse_join_controllers to shared, add test
config_parse_join_controllers would free the destination argument on failure,
which is contrary to our normal style, where failed parsing has no effect.
Moving it to shared also allows a test to be added.
2018-02-19 15:02:13 +01:00
Zbigniew Jędrzejewski-Szmek 6fb05b0799 test-process-util: one more (void) 2018-02-19 15:02:13 +01:00
Lennart Poettering a94ab7acfd
Merge pull request #8175 from keszybz/gc-cleanup
Garbage collection cleanup
2018-02-15 17:47:37 +01:00
Zbigniew Jędrzejewski-Szmek 730f40eb57 Merge pull request #8184 from poettering/color-ask-pw
Trivial merge conflict resolved locally.
2018-02-15 17:14:59 +01:00
Lennart Poettering daa557208d tty-ask-password-agent: don't open terminal multiple times
We already have the terminal open, hence pass the fd we got to
ask_password_tty(), so that it doesn't have to reopen it a second time.

This is mostly an optimization, but it has the nice benefit of making us
independent from RLIMIT_NOFILE issues and so on, as we don't need to
allocate another fd needlessly.
2018-02-14 00:11:16 +01:00
Zbigniew Jędrzejewski-Szmek bff8f950ab test-strv: add a simple test for strv_free_free() 2018-02-13 08:57:31 +01:00
Lennart Poettering 43767d9d5e fs-util: add new unlinkat_deallocate() helper
This new helper not only removes a file from a directory but also
ensures its space on disk is deallocated, by either punching a hole over
the full file or truncating the file afterwards if the file's link
counter is 0. This is useful in "vacuuming" algorithms to ensure that
client's can't keep the disk space the vacuuming is supposed to recover
pinned simply by keeping an fd open to it.
2018-02-12 11:27:11 +01:00
Lennart Poettering 46e16b347f hash-func: add generic hash_ops implementation for hashing paths
This is similar to string_hash_ops but operates one file system paths
specifically. It will ensure that "/foo//bar" and "///foo/bar" are
considered to be the same path for hashmap purposes.

This makes use of the existing path_compare() API, and adds a matching
hashing function for it.

Note that relative and absolute paths will hash to different values,
however whether the path is suffixed with a slash or not is not
detected. This matches the existing path_compare() behaviour, and
follows the logic that on Linux there can't be two different objects at
path /foo/bar and /foo/bar/ either.
2018-02-12 11:07:55 +01:00
Yu Watanabe b8e2400586
Merge pull request #8143 from yuwata/drop-unused-func
Drop several unused functions
2018-02-10 12:47:12 +09:00
Yu Watanabe 92db49a0c2 socket-util: drop getnameinfo_pretty() 2018-02-10 11:10:07 +09:00
Lennart Poettering 7fe2903c23 fd-util: move certain fds above fd #2 (#8129)
This adds some paranoia code that moves some of the fds we allocate for
longer periods of times to fds > 2 if they are allocated below this
boundary. This is a paranoid safety thing, in order to avoid that
external code might end up erroneously use our fds under the assumption
they were valid stdin/stdout/stderr. Think: some app closes
stdin/stdout/stderr and then invokes 'fprintf(stderr, …' which causes
writes on our fds.

This both adds the helper to do the moving as well as ports over a
number of users to this new logic. Since we don't want to litter all our
code with invocations of this I tried to strictly focus on fds we keep
open for long periods of times only and only in code that is frequently
loaded into foreign programs (under the assumptions that in our own
codebase we are smart enough to always keep stdin/stdout/stderr
allocated to avoid this pitfall). Specifically this means all code used
by NSS and our sd-xyz API:

1. our logging APIs
2. sd-event
3. sd-bus
4. sd-resolve
5. sd-netlink

This changed was inspired by this:

https://github.com/systemd/systemd/issues/8075#issuecomment-363689755

This shows that apparently IRL there are programs that do close
stdin/stdout/stderr, and we should accomodate for that.

Note that this won't fix any bugs, this just makes sure that buggy
programs are less likely to interfere with out own code.
2018-02-09 17:53:28 +01:00
Lennart Poettering a04fcf17ba procfs-util: add APIs to get consumed CPU time and used memory from /proc
This is preparation for emulating the "usage_usec" keyed attribute of
the "cpu.stat" property of the root cgroup from data in /proc. Similar,
for emulating the "memory.current" attribute.
2018-02-09 17:32:26 +01:00
Zbigniew Jędrzejewski-Szmek 7e684baf90 shared/path-lookup: rearrange paths in --global mode to match --user mode
It's not good if the paths are in different order. With --user, we expect
more paths, but it must be a strict superset, and the order for the ones
that appear in both sets must be the same.

$  diff -u <(build/systemd-analyze --global unit-paths) <(build/systemd-analyze --user unit-paths)|colordiff
--- /proc/self/fd/14	2018-02-08 14:11:45.425353107 +0100
+++ /proc/self/fd/15	2018-02-08 14:11:45.426353116 +0100
@@ -1,6 +1,17 @@
+/home/zbyszek/.config/systemd/system.control
+/run/user/1000/systemd/system.control
+/run/user/1000/systemd/transient
+/run/user/1000/systemd/generator.early
+/home/zbyszek/.config/systemd/user
 /etc/systemd/user
+/run/user/1000/systemd/user
 /run/systemd/user
+/run/user/1000/systemd/generator
+/home/zbyszek/.local/share/systemd/user
+/home/zbyszek/.local/share/flatpak/exports/share/systemd/user
+/var/lib/flatpak/exports/share/systemd/user
 /usr/local/share/systemd/user
 /usr/share/systemd/user
 /usr/local/lib/systemd/user
 /usr/lib/systemd/user
+/run/user/1000/systemd/generator.late

A test is added so that we don't regress on this.
2018-02-09 12:27:34 +01:00
Yu Watanabe 842e456e79 fs-util: drop readlink_and_canonicalize() 2018-02-09 16:49:54 +09:00
Lennart Poettering 960c7c2791
Merge pull request #8101 from keszybz/two-gcc-workarounds
Two gcc workarounds
2018-02-05 18:59:52 +01:00
Zbigniew Jędrzejewski-Szmek 1fe101747c basic/glob-util: add closedir wrapper to silence gcc
../src/test/test-glob-util.c: In function 'test_glob_no_dot':
../src/test/test-glob-util.c:61:32: warning: cast between incompatible function types from 'int (*)(DIR *)' {aka 'int (*)(struct __dirstream *)'} to 'void (*)(void *)' [-Wcast-function-type]
                 .gl_closedir = (void (*)(void *)) closedir,
                                ^
2018-02-05 15:20:46 +01:00
Zbigniew Jędrzejewski-Szmek bd181f27d4 test: add a simple smoke test for string_hashsum()
This is enough to show memory leakages pointed out by Stef Bon <stefbon@gmail.com>.
2018-02-05 09:48:38 +01:00
Zbigniew Jędrzejewski-Szmek 6e888894fc basic/strv: add function to insert items at position 2018-02-02 10:35:30 +01:00
Lennart Poettering 52dca0de99
Merge pull request #7042 from vcaputo/iteratedcache
RFC: Optionally cache hashmap iterated results
2018-02-01 18:08:50 +01:00
Yu Watanabe e7b2ea7c97 strv: drop strv_join_quoted() (#8057)
The function `strv_join_quoted()` is now not used, and has a bug
in the buffer size calculation when the strings needs to escaped,
as reported in #8056.
So, let's remove the function.

Closes #8056.
2018-02-01 10:11:02 +01:00
Lennart Poettering 7755083256
Merge pull request #7881 from keszybz/pcre
Add new --grep option to journalctl
2018-01-28 15:29:10 +01:00
Zbigniew Jędrzejewski-Szmek b4766d5f15 journalctl: add highlighting for matched substring
Red is used for highligting, the same as grep does. Except when the line is
highlighted red already, because it has high priority, in which case plain ansi
highlight is used for the matched substring.

Coloring is implemented for short and cat outputs, and not for other types.
I guess we could also add it for verbose output in the future.
2018-01-28 14:50:01 +01:00
Zbigniew Jędrzejewski-Szmek 61c5f8a1f0 journalctl: make matching optionally case sensitive
Case sensitive or case insensitive matching can be requested using
--case-sensitive[=yes|no].

Unless specified, matching is case sensitive if the pattern contains any
uppercase letters, and case insensitive otherwise. This matches what
forward-search does in emacs, and recently also --ignore-case in less.  This
works surprisingly well, because usually when one is wants to do case-sensitive
matching, the pattern is usually camel-cased. In the less frequent case when
case-sensitive matching is required with an all-lowercase pattern,
--case-sensitive can be used to override the automatic logic.
2018-01-28 14:50:01 +01:00
Vito Caputo 647c7b7440 test-hashmap: test IteratedCache
Add some rudimentary testing of the new IteratedCache
2018-01-27 13:11:58 -08:00
Zbigniew Jędrzejewski-Szmek f26f5b60d0 Merge pull request #7915 from poettering/pids-max-tweak 2018-01-25 10:24:35 +01:00
Lennart Poettering 62a769136d core: rework how we track which PIDs to watch for a unit
Previously, we'd maintain two hashmaps keyed by PIDs, pointing to Unit
interested in SIGCHLD events for them. This scheme allowed a specific
PID to be watched by exactly 0, 1 or 2 units.

With this rework this is replaced by a single hashmap which is primarily
keyed by the PID and points to a Unit interested in it. However, it
optionally also keyed by the negated PID, in which case it points to a
NULL terminated array of additional Unit objects also interested. This
scheme means arbitrary numbers of Units may now watch the same PID.

Runtime and memory behaviour should not be impact by this change, as for
the common case (i.e. each PID only watched by a single unit) behaviour
stays the same, but for the uncommon case (a PID watched by more than
one unit) we only pay with a single additional memory allocation for the
array.

Why this all? Primarily, because allowing exactly two units to watch a
specific PID is not sufficient for some niche cases, as processes can
belong to more than one unit these days:

1. sd_notify() with MAINPID= can be used to attach a process from a
   different cgroup to multiple units.

2. Similar, the PIDFile= setting in unit files can be used for similar
   setups,

3. By creating a scope unit a main process of a service may join a
   different unit, too.

4. On cgroupsv1 we frequently end up watching all processes remaining in
   a scope, and if a process opens lots of scopes one after the other it
   might thus end up being watch by many of them.

This patch hence removes the 2-unit-per-PID limit. It also makes a
couple of other changes, some of them quite relevant:

- manager_get_unit_by_pid() (and the bus call wrapping it) when there's
  ambiguity will prefer returning the Unit the process belongs to based on
  cgroup membership, and only check the watch-pids hashmap if that
  fails. This change in logic is probably more in line with what people
  expect and makes things more stable as each process can belong to
  exactly one cgroup only.

- Every SIGCHLD event is now dispatched to all units interested in its
  PID. Previously, there was some magic conditionalization: the SIGCHLD
  would only be dispatched to the unit if it was only interested in a
  single PID only, or the PID belonged to the control or main PID or we
  didn't dispatch a signle SIGCHLD to the unit in the current event loop
  iteration yet. These rules were quite arbitrary and also redundant as
  the the per-unit handlers would filter the PIDs anyway a second time.
  With this change we'll hence relax the rules: all we do now is
  dispatch every SIGCHLD event exactly once to each unit interested in
  it, and it's up to the unit to then use or ignore this. We use a
  generation counter in the unit to ensure that we only invoke the unit
  handler once for each event, protecting us from confusion if a unit is
  both associated with a specific PID through cgroup membership and
  through the "watch_pids" logic. It also protects us from being
  confused if the "watch_pids" hashmap is altered while we are
  dispatching to it (which is a very likely case).

- sd_notify() message dispatching has been reworked to be very similar
  to SIGCHLD handling now. A generation counter is used for dispatching
  as well.

This also adds a new test that validates that "watch_pid" registration
and unregstration works correctly.
2018-01-23 21:29:31 +01:00
Lennart Poettering eb5eb254dd test: test-process-util additions
Add some extra paranoia tests that PTR_TO_PID() and PID_TO_PTR() deals
correctly with negative PID.
2018-01-23 21:22:50 +01:00
Lennart Poettering 9aef9a672d util-lib: add new procfs-util.[ch] API for dealing with tasks limits
As it turns out the limit on concurrent tasks on Linux nasty to
determine, hence let's appropriate helpers for this.
2018-01-22 16:26:55 +01:00
Lennart Poettering d72495759b tree-wide: port all code to use safe_getcwd() 2018-01-17 11:17:38 +01:00
Zbigniew Jędrzejewski-Szmek e0b6d3cabe
Merge pull request #7816 from poettering/chase-pid
Make MAINPID= and PIDFile= handling more restrictive (and other stuff)
2018-01-15 14:14:34 +04:00
Lennart Poettering 1ed34d75d4 fs-util: add new chase_symlinks() flag CHASE_OPEN
The new flag returns the O_PATH fd of the final component, which may be
converted into a proper fd by open()ing it again through the
/proc/self/fd/xyz path.

Together with O_SAFE this provides us with a somewhat safe way to open()
files in directories potentially owned by unprivileged code, where we
want to refuse operation if any symlink tricks are played pointing to
privileged files.
2018-01-11 15:12:16 +01:00
Lennart Poettering f14f1806e3 fs-util: add new CHASE_SAFE flag to chase_symlinks()
When the flag is specified we won't transition to a privilege-owned
file or directory from an unprivileged-owned one. This is useful when
privileged code wants to load data from a file unprivileged users have
write access to, and validates the ownership, but want's to make sure
that no symlink games are played to read a root-owned system file
belonging to a different context.
2018-01-11 15:12:16 +01:00
Lennart Poettering dccca82b1a log: minimize includes in log.h
log.h really should only include the bare minimum of other headers, as
it is really pulled into pretty much everything else and already in
itself one of the most basic pieces of code we have.

Let's hence drop inclusion of:

1. sd-id128.h because it's entirely unneeded in current log.h
2. errno.h, dito.
3. sys/signalfd.h which we can replace by a simple struct forward
   declaration
4. process-util.h which was needed for getpid_cached() which we now hide
   in a funciton log_emergency_level() instead, which nicely abstracts
   the details away.
5. sys/socket.h which was needed for struct iovec, but a simple struct
   forward declaration suffices for that too.

Ultimately this actually makes our source tree larger (since users of
the functionality above must now include it themselves, log.h won't do
that for them), but I think it helps to untangle our web of includes a
tiny bit.

(Background: I'd like to isolate the generic bits of src/basic/ enough
so that we can do a git submodule import into casync for it)
2018-01-11 14:44:31 +01:00
Yu Watanabe 78e2f089fd test: free after use
CID 1384238.
2018-01-10 20:52:51 +09:00
Lennart Poettering 2767027bb7 test: fix condition test if there are no controllers
As an optimization cg_mask_to_string() returns NULL if there are no
controllers available. We need to handle that.
2018-01-05 13:58:32 +01:00
Lennart Poettering 8a5cd31e5f sd-bus: optionally, use inotify to wait for bus sockets to appear
This adds a "watch-bind" feature to sd-bus connections. If set and the
AF_UNIX socket we are connecting to doesn't exist yet, we'll establish
an inotify watch instead, and wait for the socket to appear. In other
words, a missing AF_UNIX just makes connecting slower.

This is useful for daemons such as networkd or resolved that shall be
able to run during early-boot, before dbus-daemon is up, and want to
connect to dbus-daemon as soon as it becomes ready.
2018-01-05 13:55:08 +01:00