picnoir
/
Systemd
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Systemd/src/basic/set.h

131 lines
4.0 KiB
C
Raw Normal View History

Add SPDX license identifiers to source files under the LGPL This follows what the kernel is doing, c.f. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5fd54ace4721fc5ce2bb5aef6318fcf17f421460.
2017-11-18 17:09:20 +01:00
/* SPDX-License-Identifier: LGPL-2.1+ */
use #pragma once instead of foo*foo #define guards #pragma once has been "un-deprecated" in gcc since 3.3, and is widely supported in other compilers. I've been using and maintaining (rebasing) this patch for a while now, as it annoyed me to see #ifndef fooblahfoo, etc all over the place, almost arrogant about the annoyance of having to define all these names to perform a commen but neccicary functionality, when a completely superior alternative exists. I havn't sent it till now, cause its kindof a style change, and it is bad voodoo to mess with style that has been established by more established editors. So feel free to lambast me as a crazy bafoon. v2 - preserve externally used headers
2012-07-18 19:07:51 +02:00
#pragma once
initial commit
2009-11-18 00:42:52 +01:00
set: add new set_put_strsplit() call It's like set_put_strdup(), but splits up a string via an extract_first_word() loop.
2016-06-15 22:36:58 +02:00
#include "extract-word.h"
initial commit
2009-11-18 00:42:52 +01:00
#include "hashmap.h"
Move DEFINE_TRIVIAL_CLEANUP_FUNC to macro.h This remove the need for various header files to include the (relatively heavyweight) util.h.
2015-01-18 05:20:00 +01:00
#include "macro.h"
initial commit
2009-11-18 00:42:52 +01:00
basic/set: remove some spurious spaces
2016-08-05 02:36:10 +02:00
Set *internal_set_new(const struct hash_ops *hash_ops HASHMAP_DEBUG_PARAMS);
#define set_new(ops) internal_set_new(ops HASHMAP_DEBUG_SRC_ARGS)
initial commit
2009-11-18 00:42:52 +01:00
set: return NULL on destructors Like we do it pretty much everywhere else.
2015-09-01 17:13:53 +02:00
static inline Set *set_free(Set *s) {
hash-func: add destructors for key and value If they are set, then they are called in hashmap_clear() or hashmap_free().
2018-11-27 14:25:40 +01:00
return (Set*) internal_hashmap_free(HASHMAP_BASE(s), NULL, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
initial commit
2009-11-18 00:42:52 +01:00
set: return NULL on destructors Like we do it pretty much everywhere else.
2015-09-01 17:13:53 +02:00
static inline Set *set_free_free(Set *s) {
hash-func: add destructors for key and value If they are set, then they are called in hashmap_clear() or hashmap_free().
2018-11-27 14:25:40 +01:00
return (Set*) internal_hashmap_free(HASHMAP_BASE(s), free, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
/* no set_free_free_free */
static inline Set *set_copy(Set *s) {
return (Set*) internal_hashmap_copy(HASHMAP_BASE(s));
}
basic/set: remove some spurious spaces
2016-08-05 02:36:10 +02:00
int internal_set_ensure_allocated(Set **s, const struct hash_ops *hash_ops HASHMAP_DEBUG_PARAMS);
#define set_ensure_allocated(h, ops) internal_set_ensure_allocated(h, ops HASHMAP_DEBUG_SRC_ARGS)
initial commit
2009-11-18 00:42:52 +01:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
int set_put(Set *s, const void *key);
/* no set_update */
/* no set_replace */
static inline void *set_get(Set *s, void *key) {
return internal_hashmap_get(HASHMAP_BASE(s), key);
}
/* no set_get2 */
static inline bool set_contains(Set *s, const void *key) {
return internal_hashmap_contains(HASHMAP_BASE(s), key);
}
shared: set - make argument to set_remove() const
2015-04-01 13:46:59 +02:00
static inline void *set_remove(Set *s, const void *key) {
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
return internal_hashmap_remove(HASHMAP_BASE(s), key);
}
/* no set_remove2 */
/* no set_remove_value */
int set_remove_and_put(Set *s, const void *old_key, const void *new_key);
/* no set_remove_and_replace */
implement hashmap_copy() and hashmap_merge()
2010-01-18 23:49:49 +01:00
int set_merge(Set *s, Set *other);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
static inline int set_reserve(Set *h, unsigned entries_add) {
return internal_hashmap_reserve(HASHMAP_BASE(h), entries_add);
}
static inline int set_move(Set *s, Set *other) {
return internal_hashmap_move(HASHMAP_BASE(s), HASHMAP_BASE(other));
}
static inline int set_move_one(Set *s, Set *other, const void *key) {
return internal_hashmap_move_one(HASHMAP_BASE(s), HASHMAP_BASE(other), key);
}
static inline unsigned set_size(Set *s) {
return internal_hashmap_size(HASHMAP_BASE(s));
}
static inline bool set_isempty(Set *s) {
return set_size(s) == 0;
}
static inline unsigned set_buckets(Set *s) {
return internal_hashmap_buckets(HASHMAP_BASE(s));
}
initial commit
2009-11-18 00:42:52 +01:00
hashmap: fix iterators to not skip entries Currently, the HASHMAP iterators stop at the first NULL entry in a hashmap. This is non-obvious and breaks users like sd-device, which legitimately store NULL values in a hashmap. Fix all the iterators by taking a pointer to the value storage, instead of returning it. The iterators now return a boolean that tells whether the end of the list was reached. Current users of HASHMAP_FOREACH() are *NOT* changed to explicitly check for NULL. If it turns out, there were users that inserted NULL into hashmaps, but didn't properly check for it during iteration, then we really want to find those and fix them.
2015-06-14 16:51:35 +02:00
bool set_iterate(Set *s, Iterator *i, void **value);
initial commit
2009-11-18 00:42:52 +01:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
static inline void set_clear(Set *s) {
hash-func: add destructors for key and value If they are set, then they are called in hashmap_clear() or hashmap_free().
2018-11-27 14:25:40 +01:00
internal_hashmap_clear(HASHMAP_BASE(s), NULL, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
static inline void set_clear_free(Set *s) {
hash-func: add destructors for key and value If they are set, then they are called in hashmap_clear() or hashmap_free().
2018-11-27 14:25:40 +01:00
internal_hashmap_clear(HASHMAP_BASE(s), free, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
/* no set_clear_free_free */
static inline void *set_steal_first(Set *s) {
hashmap: introduce hashmap_first_key_and_value() and friends
2018-10-13 09:30:49 +02:00
return internal_hashmap_first_key_and_value(HASHMAP_BASE(s), true, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
Add set/hashmap helpers for non-trivial freeing and use where straighforward A macro is needed because otherwise we couldn't ensure type safety. Some simple tests are included. No functional change intended.
2017-11-28 12:35:49 +01:00
#define set_clear_with_destructor(_s, _f) \
({ \
void *_item; \
while ((_item = set_steal_first(_s))) \
_f(_item); \
})
#define set_free_with_destructor(_s, _f) \
({ \
set_clear_with_destructor(_s, _f); \
set_free(_s); \
})
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
/* no set_steal_first_key */
/* no set_first_key */
load-fragment: a few modernizations
2012-07-03 16:09:36 +02:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
static inline void *set_first(Set *s) {
hashmap: introduce hashmap_first_key_and_value() and friends
2018-10-13 09:30:49 +02:00
return internal_hashmap_first_key_and_value(HASHMAP_BASE(s), false, NULL);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
}
initial commit
2009-11-18 00:42:52 +01:00
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
/* no set_next */
static inline char **set_get_strv(Set *s) {
return internal_hashmap_get_strv(HASHMAP_BASE(s));
}
int set_consume(Set *s, void *value);
int set_put_strdup(Set *s, const char *p);
int set_put_strdupv(Set *s, char **l);
set: add new set_put_strsplit() call It's like set_put_strdup(), but splits up a string via an extract_first_word() loop.
2016-06-15 22:36:58 +02:00
int set_put_strsplit(Set *s, const char *v, const char *separators, ExtractFlags flags);
set: introduce set_get_strv()
2012-10-19 04:52:14 +02:00
first attempt at proper service/socket logic
2010-01-26 04:18:44 +01:00
#define SET_FOREACH(e, s, i) \
hashmap: fix iterators to not skip entries Currently, the HASHMAP iterators stop at the first NULL entry in a hashmap. This is non-obvious and breaks users like sd-device, which legitimately store NULL values in a hashmap. Fix all the iterators by taking a pointer to the value storage, instead of returning it. The iterators now return a boolean that tells whether the end of the list was reached. Current users of HASHMAP_FOREACH() are *NOT* changed to explicitly check for NULL. If it turns out, there were users that inserted NULL into hashmaps, but didn't properly check for it during iteration, then we really want to find those and fix them.
2015-06-14 16:51:35 +02:00
for ((i) = ITERATOR_FIRST; set_iterate((s), &(i), (void**)&(e)); )
initial commit
2009-11-18 00:42:52 +01:00
resolved: fix notification iteration logic when transactions are completed When a transaction is complete, and we notify its owners, make sure we deal correctly with the requesters removing themselves from the list of owners while we continue iterating. This was previously already dealt with with transactions that require other transactions for DNSSEC purposes, fix this for other possibly transaction owners too now. Since iterating through "Set" objects is not safe regarding removal of entries from it, rework the logic to use two Sets, and move each entry we notified from one set to the other set before we dispatch the notification. This move operation requires no additional memory, and enables us to ensure that we don't notify any object twice. Fixes: #2676
2016-02-22 20:39:45 +01:00
#define SET_FOREACH_MOVE(e, d, s) \
for (; ({ e = set_first(s); assert_se(!e || set_move_one(d, s, e) >= 0); e; }); )
util: allow trailing semicolons on define_trivial_cleanup_func lines Emacs C indenting really gets confused by these lines if they carry no trailing semicolon, hence let's make this nicer for good old emacs. The other macros which define functions already do this too, so let's copy the scheme here. Also, let's use an uppercase name for the macro. So far our rough rule was that macros that are totally not function-like (like this ones, which define a function) are uppercase. (Well, admittedly it is a rough rule only, for example function and variable decorators are all lower-case SINCE THE CONSTANT YELLING IN THE SOURCES WOULD SUCK, and also they at least got underscore prefixes.) Also, the macros that define functions that we already have are all uppercase, so let's do the same here...
2013-10-14 04:59:26 +02:00
DEFINE_TRIVIAL_CLEANUP_FUNC(Set*, set_free);
DEFINE_TRIVIAL_CLEANUP_FUNC(Set*, set_free_free);
hashmap: rewrite the implementation This is a rewrite of the hashmap implementation. Its advantage is lower memory usage. It uses open addressing (entries are stored in an array, as opposed to linked lists). Hash collisions are resolved with linear probing and Robin Hood displacement policy. See the references in hashmap.c. Some fun empirical findings about hashmap usage in systemd on my laptop: - 98 % of allocated hashmaps are Sets. - Sets contain 78 % of all entries, plain Hashmaps 17 %, and OrderedHashmaps 5 %. - 60 % of allocated hashmaps contain only 1 entry. - 90 % of allocated hashmaps contain 5 or fewer entries. - 75 % of all entries are in hashmaps that use trivial_hash_ops. Clearly it makes sense to: - store entries in distinct entry types. Especially for Sets - their entries are the most numerous and they require the least information to store an entry. - have a way to store small numbers of entries directly in the hashmap structs, and only allocate the usual entry arrays when the direct storage is full. The implementation has an optional debugging feature (enabled by defining the ENABLE_HASHMAP_DEBUG macro), where it: - tracks all allocated hashmaps in a linked list so that one can easily find them in gdb, - tracks which function/line allocated a given hashmap, and - checks for invalid mixing of hashmap iteration and modification. Since entries are not allocated one-by-one anymore, mempools are not used for entries. Originally I meant to drop mempools entirely, but it's still worth it to use them for the hashmap structs. My testing indicates that it makes loading of units about 5 % faster (a test with 10000 units where more than 200000 hashmaps are allocated - pure malloc: 449±4 ms, mempools: 427±7 ms). Here are some memory usage numbers, taken on my laptop with a more or less normal Fedora setup after booting with SELinux disabled (SELinux increases systemd's memory usage significantly): systemd (PID 1) Original New Change dirty memory (from pmap -x 1) [KiB] 2152 1264 -41 % total heap allocations (from gdb-heap) [KiB] 1623 756 -53 %
2014-10-15 01:27:16 +02:00
macro: rework how we define cleanup macros There's now a generic _cleanup_ macro with an argument. The macros for specific types are now defined using this macro, and in the header files where they belong. All cleanup handlers are now inline functions.
2013-04-16 05:25:57 +02:00
#define _cleanup_set_free_ _cleanup_(set_freep)
#define _cleanup_set_free_free_ _cleanup_(set_free_freep)