picnoir
/
Systemd
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Systemd/src/basic/set.h

155 lines
5.4 KiB
C
Raw Normal View History

license: LGPL-2.1+ -> LGPL-2.1-or-later
2020-11-09 05:23:58 +01:00
/* SPDX-License-Identifier: LGPL-2.1-or-later */
use #pragma once instead of foo*foo #define guards #pragma once has been "un-deprecated" in gcc since 3.3, and is widely supported in other compilers. I've been using and maintaining (rebasing) this patch for a while now, as it annoyed me to see #ifndef fooblahfoo, etc all over the place, almost arrogant about the annoyance of having to define all these names to perform a commen but neccicary functionality, when a completely superior alternative exists. I havn't sent it till now, cause its kindof a style change, and it is bad voodoo to mess with style that has been established by more established editors. So feel free to lambast me as a crazy bafoon. v2 - preserve externally used headers
2012-07-18 19:07:51 +02:00
#pragma once
initial commit
2009-11-18 00:42:52 +01:00
set: add new set_put_strsplit() call It's like set_put_strdup(), but splits up a string via an extract_first_word() loop.
2016-06-15 22:36:58 +02:00
#include "extract-word.h"
initial commit
2009-11-18 00:42:52 +01:00
#include "hashmap.h"
Move DEFINE_TRIVIAL_CLEANUP_FUNC to macro.h This remove the need for various header files to include the (relatively heavyweight) util.h.
2015-01-18 05:20:00 +01:00
#include "macro.h"
initial commit
2009-11-18 00:42:52 +01:00
shared/unit-file: make sure the old hashmaps and sets are freed upon replacement Possibly fixes #15220. (There might be another leak. I'm still investigating.) The leak would occur when the path cache was rebuilt. So in normal circumstances it wouldn't be too bad, since usually the path cache is not rebuilt too often. But the case in #15220, where new unit files are created in a loop and started, the leak occurs once for each unit file: $ for i in {1..300}; do cp ~/.config/systemd/user/test0001.service ~/.config/systemd/user/test$(printf %04d $i).service; systemctl --user start test$(printf %04d $i).service;done
2020-05-28 14:58:35 +02:00
#define set_free_and_replace(a, b) \
({ \
set_free(a); \
(a) = (b); \
(b) = NULL; \
0; \
})
basic/hashmap,set: move pointer symbol adjactent to the returned value I think this is nicer in general, and here in particular we have a lot of code like: static inline IteratedCache* hashmap_iterated_cache_new(Hashmap *h) { return (IteratedCache*) _hashmap_iterated_cache_new(HASHMAP_BASE(h)); } and it's visually appealing to use the same whitespace in the function signature and the cast in the body of the function.
2020-09-01 13:22:14 +02:00
Set* _set_new(const struct hash_ops *hash_ops HASHMAP_DEBUG_PARAMS);
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
#define set_new(ops) _set_new(ops HASHMAP_DEBUG_SRC_ARGS)
initial commit
2009-11-18 00:42:52 +01:00
basic/hashmap,set: move pointer symbol adjactent to the returned value I think this is nicer in general, and here in particular we have a lot of code like: static inline IteratedCache* hashmap_iterated_cache_new(Hashmap *h) { return (IteratedCache*) _hashmap_iterated_cache_new(HASHMAP_BASE(h)); } and it's visually appealing to use the same whitespace in the function signature and the cast in the body of the function.
2020-09-01 13:22:14 +02:00
static inline Set* set_free(Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return (Set*) _hashmap_free(HASHMAP_BASE(s), NULL, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
initial commit
2009-11-18 00:42:52 +01:00
basic/hashmap,set: move pointer symbol adjactent to the returned value I think this is nicer in general, and here in particular we have a lot of code like: static inline IteratedCache* hashmap_iterated_cache_new(Hashmap *h) { return (IteratedCache*) _hashmap_iterated_cache_new(HASHMAP_BASE(h)); } and it's visually appealing to use the same whitespace in the function signature and the cast in the body of the function.
2020-09-01 13:22:14 +02:00
static inline Set* set_free_free(Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return (Set*) _hashmap_free(HASHMAP_BASE(s), free, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
/* no set_free_free_free */
basic/hashmap,set: propagate allocation location info in _copy() Also use double space before the tracking args at the end. Without the comma this looks ugly, but it's a bit better with the double space. At least it doesn't look like a variable with a type.
2020-06-05 14:55:41 +02:00
#define set_copy(s) ((Set*) _hashmap_copy(HASHMAP_BASE(h) HASHMAP_DEBUG_SRC_ARGS))
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
int _set_ensure_allocated(Set **s, const struct hash_ops *hash_ops HASHMAP_DEBUG_PARAMS);
#define set_ensure_allocated(h, ops) _set_ensure_allocated(h, ops HASHMAP_DEBUG_SRC_ARGS)
initial commit
2009-11-18 00:42:52 +01:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
int set_put(Set *s, const void *key);
/* no set_update */
/* no set_replace */
util: constify the second argument of set_get()
2020-09-04 16:17:49 +02:00
static inline void *set_get(const Set *s, const void *key) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_get(HASHMAP_BASE((Set *) s), key);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
/* no set_get2 */
basic/set: constify operations which don't modify Set No functional change, but it's nicer to the reader.
2019-07-08 17:31:30 +02:00
static inline bool set_contains(const Set *s, const void *key) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_contains(HASHMAP_BASE((Set *) s), key);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
shared: set - make argument to set_remove() const
2015-04-01 13:46:59 +02:00
static inline void *set_remove(Set *s, const void *key) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_remove(HASHMAP_BASE(s), key);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
/* no set_remove2 */
/* no set_remove_value */
int set_remove_and_put(Set *s, const void *old_key, const void *new_key);
/* no set_remove_and_replace */
implement hashmap_copy() and hashmap_merge()
2010-01-18 23:49:49 +01:00
int set_merge(Set *s, Set *other);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
static inline int set_reserve(Set *h, unsigned entries_add) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_reserve(HASHMAP_BASE(h), entries_add);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
static inline int set_move(Set *s, Set *other) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_move(HASHMAP_BASE(s), HASHMAP_BASE(other));
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
static inline int set_move_one(Set *s, Set *other, const void *key) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_move_one(HASHMAP_BASE(s), HASHMAP_BASE(other), key);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
basic/set: constify operations which don't modify Set No functional change, but it's nicer to the reader.
2019-07-08 17:31:30 +02:00
static inline unsigned set_size(const Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_size(HASHMAP_BASE((Set *) s));
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
basic/set: constify operations which don't modify Set No functional change, but it's nicer to the reader.
2019-07-08 17:31:30 +02:00
static inline bool set_isempty(const Set *s) {
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
return set_size(s) == 0;
}
basic/set: constify operations which don't modify Set No functional change, but it's nicer to the reader.
2019-07-08 17:31:30 +02:00
static inline unsigned set_buckets(const Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_buckets(HASHMAP_BASE((Set *) s));
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
initial commit
2009-11-18 00:42:52 +01:00
basic/hashmap,set: inline trivial set_iterate() wrapper The compiler would do this to, esp. with LTO, but we can short-circuit the whole process and make everything a bit simpler by avoiding the separate definition. (It would be nice to do the same for _set_new(), _set_ensure_allocated() and other similar functions which are one-line trivial wrappers too. Unfortunately that would require enum HashmapType to be made public, which we don't want to do.)
2020-09-01 13:18:56 +02:00
static inline bool set_iterate(const Set *s, Iterator *i, void **value) {
return _hashmap_iterate(HASHMAP_BASE((Set*) s), i, value, NULL);
}
initial commit
2009-11-18 00:42:52 +01:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
static inline void set_clear(Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
_hashmap_clear(HASHMAP_BASE(s), NULL, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
static inline void set_clear_free(Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
_hashmap_clear(HASHMAP_BASE(s), free, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
/* no set_clear_free_free */
static inline void *set_steal_first(Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_first_key_and_value(HASHMAP_BASE(s), true, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
Add set/hashmap helpers for non-trivial freeing and use where straighforward A macro is needed because otherwise we couldn't ensure type safety. Some simple tests are included. No functional change intended.
2017-11-28 12:35:49 +01:00
#define set_clear_with_destructor(_s, _f) \
({ \
void *_item; \
while ((_item = set_steal_first(_s))) \
_f(_item); \
})
#define set_free_with_destructor(_s, _f) \
({ \
set_clear_with_destructor(_s, _f); \
set_free(_s); \
})
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
/* no set_steal_first_key */
/* no set_first_key */
load-fragment: a few modernizations
2012-07-03 16:09:36 +02:00
basic/set: const-ify set_first() set_steal_first() is the version that modifies Set so this one should be const.
2019-10-05 03:14:19 +02:00
static inline void *set_first(const Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_first_key_and_value(HASHMAP_BASE((Set *) s), false, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
initial commit
2009-11-18 00:42:52 +01:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
/* no set_next */
static inline char **set_get_strv(Set *s) {
basic/hashmap,set: change "internal_" to "_" as the prefix "internal" is a lot of characters. Let's take a leaf out of the Python's book and simply use _ to mean private. Much less verbose, but the meaning is just as clear, or even more.
2020-05-28 15:34:15 +02:00
return _hashmap_get_strv(HASHMAP_BASE(s));
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
basic/set: add set_ensure_put() It's such a common operation to allocate the set and put an item in it, that it deserves a helper. set_ensure_put() has the same return values as set_put(). Comes with tests!
2020-06-03 14:25:18 +02:00
int _set_ensure_put(Set **s, const struct hash_ops *hash_ops, const void *key HASHMAP_DEBUG_PARAMS);
#define set_ensure_put(s, hash_ops, key) _set_ensure_put(s, hash_ops, key HASHMAP_DEBUG_SRC_ARGS)
basic/set: add set_ensure_consume() This combines set_ensure_allocated() with set_consume(). The cool thing is that because we know the hash ops, we can correctly free the item if appropriate. Similarly to set_consume(), the goal is to simplify handling of the case where the item needs to be freed on error and if already present in the set.
2020-06-04 19:46:14 +02:00
int _set_ensure_consume(Set **s, const struct hash_ops *hash_ops, void *key HASHMAP_DEBUG_PARAMS);
#define set_ensure_consume(s, hash_ops, key) _set_ensure_consume(s, hash_ops, key HASHMAP_DEBUG_SRC_ARGS)
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
int set_consume(Set *s, void *value);
basic/set,hashmap: pass through allocation info in more cases
2020-06-04 19:58:18 +02:00
hashmap: introduce {hashmap,set}_put_strdup_full() They can take hash_ops.
2020-10-13 15:39:02 +02:00
int _set_put_strdup_full(Set **s, const struct hash_ops *hash_ops, const char *p HASHMAP_DEBUG_PARAMS);
#define set_put_strdup_full(s, hash_ops, p) _set_put_strdup_full(s, hash_ops, p HASHMAP_DEBUG_SRC_ARGS)
#define set_put_strdup(s, p) set_put_strdup_full(s, &string_hash_ops_free, p)
int _set_put_strdupv_full(Set **s, const struct hash_ops *hash_ops, char **l HASHMAP_DEBUG_PARAMS);
#define set_put_strdupv_full(s, hash_ops, l) _set_put_strdupv_full(s, hash_ops, l HASHMAP_DEBUG_SRC_ARGS)
#define set_put_strdupv(s, l) set_put_strdupv_full(s, &string_hash_ops_free, l)
basic/set,hashmap: pass through allocation info in more cases
2020-06-04 19:58:18 +02:00
set: add new set_put_strsplit() call It's like set_put_strdup(), but splits up a string via an extract_first_word() loop.
2016-06-15 22:36:58 +02:00
int set_put_strsplit(Set *s, const char *v, const char *separators, ExtractFlags flags);
set: introduce set_get_strv()
2012-10-19 04:52:14 +02:00
tree-wide: define iterator inside of the macro
2020-09-08 11:58:29 +02:00
#define _SET_FOREACH(e, s, i) \
for (Iterator i = ITERATOR_FIRST; set_iterate((s), &i, (void**)&(e)); )
#define SET_FOREACH(e, s) \
_SET_FOREACH(e, s, UNIQ_T(i, UNIQ))
initial commit
2009-11-18 00:42:52 +01:00
resolved: fix notification iteration logic when transactions are completed When a transaction is complete, and we notify its owners, make sure we deal correctly with the requesters removing themselves from the list of owners while we continue iterating. This was previously already dealt with with transactions that require other transactions for DNSSEC purposes, fix this for other possibly transaction owners too now. Since iterating through "Set" objects is not safe regarding removal of entries from it, rework the logic to use two Sets, and move each entry we notified from one set to the other set before we dispatch the notification. This move operation requires no additional memory, and enables us to ensure that we don't notify any object twice. Fixes: #2676
2016-02-22 20:39:45 +01:00
#define SET_FOREACH_MOVE(e, d, s) \
for (; ({ e = set_first(s); assert_se(!e || set_move_one(d, s, e) >= 0); e; }); )
util: allow trailing semicolons on define_trivial_cleanup_func lines Emacs C indenting really gets confused by these lines if they carry no trailing semicolon, hence let's make this nicer for good old emacs. The other macros which define functions already do this too, so let's copy the scheme here. Also, let's use an uppercase name for the macro. So far our rough rule was that macros that are totally not function-like (like this ones, which define a function) are uppercase. (Well, admittedly it is a rough rule only, for example function and variable decorators are all lower-case SINCE THE CONSTANT YELLING IN THE SOURCES WOULD SUCK, and also they at least got underscore prefixes.) Also, the macros that define functions that we already have are all uppercase, so let's do the same here...
2013-10-14 04:59:26 +02:00
DEFINE_TRIVIAL_CLEANUP_FUNC(Set*, set_free);
DEFINE_TRIVIAL_CLEANUP_FUNC(Set*, set_free_free);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
macro: rework how we define cleanup macros There's now a generic _cleanup_ macro with an argument. The macros for specific types are now defined using this macro, and in the header files where they belong. All cleanup handlers are now inline functions.
2013-04-16 05:25:57 +02:00
#define _cleanup_set_free_ _cleanup_(set_freep)
#define _cleanup_set_free_free_ _cleanup_(set_free_freep)
set: introduce set_strjoin()
2020-12-06 12:10:48 +01:00
int set_strjoin(Set *s, const char *separator, char **ret);