Commit graph

227 commits

Author SHA1 Message Date
Alex Gartrell 57850536d5 journal: provide compress_threshold_bytes parameter
Previously the compression threshold was hardcoded to 512, which meant that
smaller values wouldn't be compressed. This left some storage savings on the
table, so instead, we make that number tunable.
2018-03-20 11:48:52 -07:00
Zbigniew Jędrzejewski-Szmek f83f8c70bd meson: use triple-quote delimition in one more place 2018-03-09 14:15:39 +01:00
Lennart Poettering 2b33ab0957 tree-wide: port various places over to use new rearrange_stdio() 2018-03-02 11:42:10 +01:00
Lennart Poettering 3ccf323dfd journal-upload: make use of safe_close() where appropriate 2018-02-26 18:45:28 +01:00
Lennart Poettering 7755083256
Merge pull request #7881 from keszybz/pcre
Add new --grep option to journalctl
2018-01-28 15:29:10 +01:00
Zbigniew Jędrzejewski-Szmek b4766d5f15 journalctl: add highlighting for matched substring
Red is used for highligting, the same as grep does. Except when the line is
highlighted red already, because it has high priority, in which case plain ansi
highlight is used for the matched substring.

Coloring is implemented for short and cat outputs, and not for other types.
I guess we could also add it for verbose output in the future.
2018-01-28 14:50:01 +01:00
Lennart Poettering dccca82b1a log: minimize includes in log.h
log.h really should only include the bare minimum of other headers, as
it is really pulled into pretty much everything else and already in
itself one of the most basic pieces of code we have.

Let's hence drop inclusion of:

1. sd-id128.h because it's entirely unneeded in current log.h
2. errno.h, dito.
3. sys/signalfd.h which we can replace by a simple struct forward
   declaration
4. process-util.h which was needed for getpid_cached() which we now hide
   in a funciton log_emergency_level() instead, which nicely abstracts
   the details away.
5. sys/socket.h which was needed for struct iovec, but a simple struct
   forward declaration suffices for that too.

Ultimately this actually makes our source tree larger (since users of
the functionality above must now include it themselves, log.h won't do
that for them), but I think it helps to untangle our web of includes a
tiny bit.

(Background: I'd like to isolate the generic bits of src/basic/ enough
so that we can do a git submodule import into casync for it)
2018-01-11 14:44:31 +01:00
Lennart Poettering b6e1fff13d process-util: add another fork_safe() flag for enabling LOG_ERR/LOG_WARN logging 2018-01-04 13:27:26 +01:00
Lennart Poettering 4c253ed1ca tree-wide: introduce new safe_fork() helper and port everything over
This adds a new safe_fork() wrapper around fork() and makes use of it
everywhere. The new wrapper does a couple of things we previously did
manually and separately in a safer, more correct and automatic way:

1. Optionally resets signal handlers/mask in the child

2. Sets a name on all processes we fork off right after forking off (and
   the patch assigns useful names for all processes we fork off now,
   following a systematic naming scheme: always enclosed in () – in order
   to indicate that these are not proper, exec()ed processes, but only
   forked off children, and if the process is long-running with only our
   own code, without execve()'ing something else, it gets am "sd-" prefix.)

3. Optionally closes all file descriptors in the child

4. Optionally sets a PR_SET_DEATHSIG to SIGTERM in the child, in a safe
   way so that the parent dying before this happens being handled
   safely.

5. Optionally reopens the logs

6. Optionally connects stdin/stdout/stderr to /dev/null

7. Debug logs about the forked off processes.
2017-12-25 11:48:21 +01:00
Lennart Poettering f1d34068ef tree-wide: add DEBUG_LOGGING macro that checks whether debug logging is on (#7645)
This makes things a bit easier to read I think, and also makes sure we
always use the _unlikely_ wrapper around it, which so far we used
sometimes and other times we didn't. Let's clean that up.
2017-12-15 11:09:00 +01:00
Lennart Poettering fbd0b64f44
tree-wide: make use of new STRLEN() macro everywhere (#7639)
Let's employ coccinelle to do this for us.

Follow-up for #7625.
2017-12-14 19:02:29 +01:00
Marcus Folkesson 672a008b48 journal-upload: remove duplication of --help and --version in help text
Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
2017-12-03 18:40:13 +01:00
Zbigniew Jędrzejewski-Szmek 1599f593d6 journal-remote: add helper function and use hashmap_free_with_destructor 2017-11-28 21:34:46 +01:00
Shawn Landden 4831981d89 tree-wide: adjust fall through comments so that gcc is happy
Distcc removes comments, making the comment silencing
not work.

I know there was a decision against a macro in commit
ec251fe7d5
2017-11-20 13:06:25 -08:00
Zbigniew Jędrzejewski-Szmek 3a726fcd08 Add license headers and SPDX identifiers to meson.build files
So far I avoided adding license headers to meson files, but they are pretty
big and important and should carry license headers like everything else.
I added my own copyright, even though other people modified those files too.
But this is mostly symbolic, so I hope that's OK.
2017-11-19 19:08:15 +01:00
Zbigniew Jędrzejewski-Szmek 53e1b68390 Add SPDX license identifiers to source files under the LGPL
This follows what the kernel is doing, c.f.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5fd54ace4721fc5ce2bb5aef6318fcf17f421460.
2017-11-19 19:08:15 +01:00
Lennart Poettering bcde742e78 conf-parser: turn three bool function params into a flags fields
This makes things more readable and fixes some issues with incorrect
flag propagation between the various flavours of config_parse().
2017-11-13 10:24:03 +01:00
martingh 723afa4f96 Reverting erroneous changes made of the course of adding a timeout to journal-remote event handling (#7219) 2017-11-01 10:15:38 +01:00
Yu Watanabe 2e293b4a14 journal-remote: set description to timer_event (#7234)
Follow-up for 5e38eb931f.
2017-11-01 10:13:00 +01:00
Lars Karlitski cc25a67e2a journalctl: add --output-fields= (#7181)
This option allows restricting the shown fields in the output modes that
would normally show all fields. It allows clients that are only
interested in a subset of the fields to access those more efficiently.
Also, it makes the resulting size of the output more predictable.

It has no effect on the various `short` output modes, because those
already only show a subset of the fields.
2017-10-27 12:10:47 +09:00
martingh 5e38eb931f Adding timer_event to wakeup as suggested by MHD_get_timeout (#7152) 2017-10-26 10:54:30 +02:00
Zbigniew Jędrzejewski-Szmek 349cc4a507 build-sys: use #if Y instead of #ifdef Y everywhere
The advantage is that is the name is mispellt, cpp will warn us.

$ git grep -Ee "conf.set\('(HAVE|ENABLE)_" -l|xargs sed -r -i "s/conf.set\('(HAVE|ENABLE)_/conf.set10('\1_/"
$ git grep -Ee '#ifn?def (HAVE|ENABLE)' -l|xargs sed -r -i 's/#ifdef (HAVE|ENABLE)/#if \1/; s/#ifndef (HAVE|ENABLE)/#if ! \1/;'
$ git grep -Ee 'if.*defined\(HAVE' -l|xargs sed -i -r 's/defined\((HAVE_[A-Z0-9_]*)\)/\1/g'
$ git grep -Ee 'if.*defined\(ENABLE' -l|xargs sed -i -r 's/defined\((ENABLE_[A-Z0-9_]*)\)/\1/g'
+ manual changes to meson.build

squash! build-sys: use #if Y instead of #ifdef Y everywhere

v2:
- fix incorrect setting of HAVE_LIBIDN2
2017-10-04 12:09:29 +02:00
Yu Watanabe cb30d3ba66 journal-remote: show error if SplitMode=host with 'active' source is specified 2017-08-30 16:56:41 +09:00
Yu Watanabe 6b1b9f75c8 journal-remote: show error message if output file name does not end with .journal
`journalctl -o export | systemd-journal-remote -o /tmp/dir -`
gives the following error messages.
```
Failed to open output journal /tmp/dir: Invalid argument
Failed to get writer for source stdin: Invalid argument
Failed to create source for fd:0 (stdin): Invalid argument
```
And these are hard to understand what is the problem.
This commit makes journal-remote check whether the output file name
ends with .journal suffix or not, and if not, output error message.
2017-08-27 16:34:57 +09:00
Yu Watanabe e2e8683e6b microhttpd-util: add comment 2017-08-07 10:35:05 +09:00
Yu Watanabe a77e00a5dc journal-remote: remove MHD_USE_PEDANTIC_CHECKS from the default flags
Follow-up for 0105858734.
2017-08-06 23:26:46 +09:00
Yu Watanabe 0105858734 journal-remote: use MHD_OPTION_STRICT_FOR_CLIENT if MHD_USE_PEDANTIC_CHECKS is deprecated
The option MHD_OPTION_STRICT_FOR_CLIENT is provided since libmicrohttpd-0.9.54, and
MHD_USE_PEDANTIC_CHECKS will be deprecated in future.
This makes support both option.
2017-07-28 16:22:14 +09:00
Yu Watanabe 315629a83f journal-gateway: use MHD_USE_POLL_INTERNAL_THREAD instead of MHD_USE_POLL
The option MHD_USE_THREAD_PER_CONNECTION requires MHD_USE_POLL_INTERNAL_THREAD
since libmicrohttpd-0.9.53.
If MHD_USE_POLL is used instead of MHD_USE_POLL_INTERNAL_THREAD, then
the library outputs the following warning:
```
Warning: MHD_USE_THREAD_PER_CONNECTION must be used only with
MHD_USE_INTERNAL_POLLING_THREAD. Flag MHD_USE_INTERNAL_POLLING_THREAD was added.
Consider setting MHD_USE_INTERNAL_POLLING_THREAD explicitly.
```
The option MHD_USE_POLL_INTERNAL_THREAD is defined as
`MHD_USE_POLL_INTERNAL_THREAD = MHD_USE_POLL | MHD_USE_INTERNAL_POLLING_THREAD,`
So, let's use MHD_USE_POLL_INTERNAL_THREAD instead of MHD_USE_POLL.
2017-07-28 13:37:10 +09:00
Yu Watanabe c831aa7554 journal-remote,gateway: use MHD_USE_TLS instead of MHD_USE_SSL
The option is renamed in libmicrohttpd-0.9.52.
2017-07-28 13:37:10 +09:00
Lennart Poettering df0ff12775 tree-wide: make use of getpid_cached() wherever we can
This moves pretty much all uses of getpid() over to getpid_raw(). I
didn't specifically check whether the optimization is worth it for each
replacement, but in order to keep things simple and systematic I
switched over everything at once.
2017-07-20 20:27:24 +02:00
Zbigniew Jędrzejewski-Szmek e5f752082e build-sys: drop gitignore patterns for in-tree builds
... and other autotools-generated files.
2017-07-18 10:05:06 -04:00
Zbigniew Jędrzejewski-Szmek 72cdb3e783 build-sys: drop automake support
v2:
- also mention m4
2017-07-18 10:04:44 -04:00
Lennart Poettering 54d8ef14d8 time-util: rename usec_sub() to usec_sub_signed() and add usec_sub_unsigned()
Quite often we just want to subtract two normal usec_t values, hence
provide an implementation for that.
2017-06-22 20:51:06 +02:00
Zbigniew Jędrzejewski-Szmek 2ddb70d26f journal-remote: fix memleak of the name of the remote source 2017-05-19 11:40:56 -04:00
Michael Biebl 8251ee109c Merge pull request #5842 from keszybz/meson-status-and-conditionals
Meson status and conditional simplification
2017-05-03 16:45:31 +02:00
Zbigniew Jędrzejewski-Szmek 2c201c2140 meson: use booleans for conf.set and drop unecessary conditionals
Using conf.set() with a boolean argument does the right thing:
either #ifdef or #undef. This means that conf.set can be used unconditionally.

Previously I used '1' as the placeholder value, and that needs to be changed to
'true' for consistency (under meson 1 cannot be used in boolean context). All
checks need to be adjusted.
2017-05-02 16:29:11 -04:00
Jörg Thalheim 3e67e5c992 more portable python shebangs (#5816)
This is useful on systems like NixOS, where python3 is not in
/usr/bin/python3 as well as for people using alternative ways to
install python such as virtualenv/pyenv.
2017-04-30 20:26:56 -04:00
userwithuid 1c9f131444 meson: do not install files from disabled features (#5811)
Mirror conditions from Makefile.am.
2017-04-27 13:47:04 -04:00
Zbigniew Jędrzejewski-Szmek 37efbbd821 meson: reindent all files with 8 spaces
The indentation for emacs'es meson-mode is added .dir-locals.

All files are reindented automatically, using the lasest meson-mode from git.
Indentation should now be fairly consistent.
2017-04-23 21:47:29 -04:00
Zbigniew Jędrzejewski-Szmek f6e49d4e52 meson: add journal-install and hwdb-update hooks
v2:
- ignore errors in chown/chmod/setfacl
- obey -Dadm-group=false, -Dwheel-group=false
- fix reversed condition for systemd-hwdb update hook
2017-04-23 21:47:29 -04:00
Michael Biebl e17e5ba9bf meson: use join_paths consistently
With -Dsplit-usr=true, we set rootprefix to /. This leads to //lib/systemd or
//lib/udev for various dir variables. Using join_paths() avoids this.
2017-04-23 21:47:28 -04:00
Zbigniew Jędrzejewski-Szmek a3e34ac80d meson: create /var/log/journal/remote
C.f. https://github.com/mesonbuild/meson/issues/1604.
2017-04-23 21:47:27 -04:00
Zbigniew Jędrzejewski-Szmek 5c23128dab meson: build systemd using meson
It's crucial that we can build systemd using VS2010!

... er, wait, no, that's not the official reason. We need to shed old systems
by requring python 3! Oh, no, it's something else. Maybe we need to throw out
345 years of knowlege accumulated in autotools? Whatever, this new thing is
cool and shiny, let's use it.

This is not complete, I'm throwing it out here for your amusement and critique.

- rules for sd-boot are missing. Those might be quite complicated.

- rules for tests are missing too. Those are probably quite simple and
  repetitive, but there's lots of them.

- it's likely that I didn't get all the conditions right, I only tested "full"
  compilation where most deps are provided and nothing is disabled.

- busname.target and all .busname units are skipped on purpose.

  Otherwise, installation into $DESTDIR has the same list of files and the
  autoconf install, except for .la files.

It'd be great if people had a careful look at all the library linking options.
I added stuff until things compiled, and in the end there's much less linking
then in the old system. But it seems that there's still a lot of unnecessary
deps.

meson has a `shared_module` statement, which sounds like something appropriate
for our nss and pam modules. Unfortunately, I couldn't get it to work. For the
nss modules, we need an .so version of '2', but `shared_module` disallows the
version argument. For the pam module, it also didn't work, I forgot the reason.

The handling of .m4 and .in and .m4.in files is rather awkward. It's likely
that this could be simplified. If make support is ever dropped, I think it'd
make sense to switch to a different templating system so that two different
languages and not required, which would make everything simpler yet.

v2:
- use get_pkgconfig_variable
- use sh not bash
- use add_project_arguments

v3:
- drop required:true and fix progs/prog typo

v4:
- use find_library('bz2')
- add TTY_GID definition
- define __SANE_USERSPACE_TYPES__
- use join_paths(prefix, ...) is used on all paths to make them all absolute

v5:
- replace all declare_dependency's with []
- add more conf.get guards around optional components

v6:
- drop -pipe, -Wall which are the default in meson
- use compiler.has_function() and compiler.has_header_symbol instead of the
  hand-rolled checks.
- fix duplication in 'liblibsystemd' library name
- use the right .sym file for pam_systemd
- rename 'compiler' to 'cc': shorter, and more idiomatic.

v7:
- use ENABLE_ENVIRONMENT_D not HAVE_ENVIRONMENT_D
- rename prefix to prefixdir, rootprefix to rootprefixdir
  ("prefix" is too common of a name and too easy to overwrite by mistake)
- wrap more stuff with conf.get('ENABLE...') == 1
- use rootprefix=='/' and rootbindir as install_dir, to fix paths under
  split-usr==true.

v8:
- use .split() also for src/coredump. Now everything is consistent ;)
- add rootlibdir option and use it on the libraries that require it

v9:
- indentation

v10:
- fix check for qrencode and libaudit

v11:
- unify handling of executable paths, provide options for all progs

  This makes the meson build behave slightly differently than the
  autoconf-based one, because we always first try to find the executable in the
  filesystem, and fall back to the default. I think different handling of
  loadkeys, setfont, and telinit was just a historical accident.

  In addition to checking in $PATH, also check /usr/sbin/, /sbin for programs.
  In Fedora $PATH includes /usr/sbin, (and /sbin is is a symlink to /usr/sbin),
  but in Debian, those directories are not included in the path.

  C.f. https://github.com/mesonbuild/meson/issues/1576.

- call all the options 'xxx-path' for clarity.
- sort man/rules/meson.build properly so it's stable
2017-04-23 21:47:26 -04:00
Djalal Harouni 74e941c022 Merge pull request #5774 from keszybz/printf-annotations
Printf annotation improvements
2017-04-23 01:03:42 +02:00
Martin Pitt 56744c037d Merge pull request #5756 from keszybz/make-cleanups
Various meson-independent cleanups from the meson patchset
2017-04-21 21:36:56 +02:00
Zbigniew Jędrzejewski-Szmek c8304ba902 µhttpd-util: use #pragma to silence warning about nonliteral pattern
This is safe, because we're taking a pattern which was already marked with
_printf_ and appending a literal string.
2017-04-21 13:36:48 -04:00
Zbigniew Jędrzejewski-Szmek 58eb279840 microhttpd-util: silence warnings about deprecated options
C.f. 21b6ff3684.
2017-04-21 12:05:55 -04:00
Yu Watanabe da4128543f tree-wide: fix wrong indent (#5757)
Fixes wrong indent introduced by the commit 43688c49d1.
2017-04-19 08:48:29 +02:00
Zbigniew Jędrzejewski-Szmek b18453eda6 Move export format parsing from src/journal-remote/ to src/basic/
No functional change.
2017-02-14 23:56:48 -05:00
Evgeny Vereshchagin b5267219dd Merge pull request #5166 from keszybz/gcc7
Fixes for gcc 7 and new µhttpd & glibc warnings
2017-02-01 12:02:50 +03:00