Merge pull request #4771 from keszybz/udev-property-ordering

Udev property ordering
This commit is contained in:
Martin Pitt 2016-12-12 16:03:52 +01:00 committed by GitHub
commit 142a1afbb9
8 changed files with 257 additions and 132 deletions

View file

@ -1720,7 +1720,8 @@ EXTRA_DIST += \
test/bus-policy/signals.conf \
test/bus-policy/check-own-rules.conf \
test/bus-policy/many-rules.conf \
test/bus-policy/test.conf
test/bus-policy/test.conf \
test/hwdb/10-bad.hwdb
EXTRA_DIST += \

View file

@ -50,28 +50,88 @@
system-supplied hwdb file with a local file if needed;
a symlink in <filename>/etc</filename> with the same name as a hwdb file in
<filename>/usr/lib</filename>, pointing to <filename>/dev/null</filename>,
disables the hwdb file entirely. hwdb files must have the extension
disables that hwdb file entirely. hwdb files must have the extension
<filename>.hwdb</filename>; other extensions are ignored.</para>
<para>The hwdb file contains data records consisting of matches and
associated key-value pairs. Every record in the hwdb starts with one or
more match strings, specifying a shell glob to compare the database
lookup string against. Multiple match lines are specified in additional
consecutive lines. Every match line is compared individually, and they are
combined by OR. Every match line must start at the first character of
the line.</para>
<para>Each hwdb file contains data records consisting of matches and associated
key-value pairs. Every record in the hwdb starts with one or more match strings,
specifying a shell glob to compare the lookup string against. Multiple match lines
are specified in consecutive lines. Every match line is compared individually, and
they are combined by OR. Every match line must start at the first character of the
line.</para>
<para>The match lines are followed by one or more key-value pair lines, which
are recognized by a leading space character. The key name and value are separated
by <literal>=</literal>. An empty line signifies the end
of a record. Lines beginning with <literal>#</literal> are ignored.</para>
<para>The match lines are followed by one or more key-value pair lines, which are
recognized by a leading space character. The key name and value are separated by
<literal>=</literal>. An empty line signifies the end of a record. Lines beginning
with <literal>#</literal> are ignored.</para>
<para>In case multiple records match a given lookup string, the key-value pairs
from all records are combined. If a key is specified multiple times, the value
from the record with the highest priority is used (each key can have only a single
value). The priority is higher when the record is in a file that sorts later
lexicographically, and in case of records in the same file, later records have
higher priority.</para>
<para>The content of all hwdb files is read by
<citerefentry><refentrytitle>systemd-hwdb</refentrytitle><manvolnum>8</manvolnum></citerefentry>
and compiled to a binary database located at <filename>/etc/udev/hwdb.bin</filename>,
or alternatively <filename>/usr/lib/udev/hwdb.bin</filename> if you want ship the compiled
database in an immutable image.
During runtime, only the binary database is used.</para>
or alternatively <filename>/usr/lib/udev/hwdb.bin</filename> if you want ship the
compiled database in an immutable image. During runtime, only the binary database
is used.</para>
</refsect1>
<refsect1>
<title>Examples</title>
<example>
<title>General syntax of hwdb files</title>
<programlisting># /usr/lib/udev/hwdb.d/example.hwdb
# Comments can be placed before any records. This is a good spot
# to describe what that file is used for, what kind of properties
# it defines, and the ordering convention.
# A record with three matches and one property
mouse:*:name:*Trackball*:
mouse:*:name:*trackball*:
mouse:*:name:*TrackBall*:
ID_INPUT_TRACKBALL=1
# A record with a single match and five properties
mouse:usb:v046dp4041:name:Logitech MX Master:
MOUSE_DPI=1000@166
MOUSE_WHEEL_CLICK_ANGLE=15
MOUSE_WHEEL_CLICK_ANGLE_HORIZONTAL=26
MOUSE_WHEEL_CLICK_COUNT=24
MOUSE_WHEEL_CLICK_COUNT_HORIZONTAL=14
</programlisting>
</example>
<example>
<title>Overriding of properties</title>
<programlisting># /usr/lib/udev/hwdb.d/60-keyboard.hwdb
evdev:atkbd:dmi:bvn*:bvr*:bd*:svnAcer*:pn*
KEYBOARD_KEY_a1=help
KEYBOARD_KEY_a2=setup
KEYBOARD_KEY_a3=battery
evdev:atkbd:dmi:bvn*:bvr*:bd*:svnAcer*:pn123*
KEYBOARD_KEY_a2=wlan
# /etc/udev/hwdb.d/70-keyboard.hwdb
# disable wlan key on all at keyboards
evdev:atkbd:*
KEYBOARD_KEY_a2=reserved</programlisting>
<para>If the hwdb consists of those two files, a keyboard with the lookup string
<literal>evdev:atkbd:dmi:bvnAcer:bdXXXXX:bd08/05/2010:svnAcer:pn123</literal>
will match all three records, and end up with the following properties:</para>
<programlisting>KEYBOARD_KEY_a1=help
KEYBOARD_KEY_a2=reserved
KEYBOARD_KEY_a3=battery</programlisting>
</example>
</refsect1>
<refsect1>

View file

@ -492,7 +492,7 @@ void *xbsearch_r(const void *key, const void *base, size_t nmemb, size_t size,
u = nmemb;
while (l < u) {
idx = (l + u) / 2;
p = (void *)(((const char *) base) + (idx * size));
p = (const char *) base + idx * size;
comparison = compar(key, p, arg);
if (comparison < 0)
u = idx;

View file

@ -39,7 +39,7 @@
#include "verbs.h"
/*
* Generic udev properties, key/value database based on modalias strings.
* Generic udev properties, key-value database based on modalias strings.
* Uses a Patricia/radix trie to index all matches for efficient lookup.
*/
@ -70,7 +70,7 @@ struct trie_node {
struct trie_child_entry *children;
uint8_t children_count;
/* sorted array of key/value pairs */
/* sorted array of key-value pairs */
struct trie_value_entry *values;
size_t values_count;
};
@ -81,12 +81,13 @@ struct trie_child_entry {
struct trie_node *child;
};
/* value array item with key/value pairs */
/* value array item with key-value pairs */
struct trie_value_entry {
size_t key_off;
size_t value_off;
size_t filename_off;
size_t line_number;
uint32_t line_number;
uint16_t file_priority;
};
static int trie_children_cmp(const void *v1, const void *v2) {
@ -160,10 +161,9 @@ static int trie_values_cmp(const void *v1, const void *v2, void *arg) {
static int trie_node_add_value(struct trie *trie, struct trie_node *node,
const char *key, const char *value,
const char *filename, size_t line_number) {
const char *filename, uint16_t file_priority, uint32_t line_number) {
ssize_t k, v, fn;
struct trie_value_entry *val;
int r;
k = strbuf_add_string(trie->strings, key, strlen(key));
if (k < 0)
@ -183,19 +183,12 @@ static int trie_node_add_value(struct trie *trie, struct trie_node *node,
val = xbsearch_r(&search, node->values, node->values_count, sizeof(struct trie_value_entry), trie_values_cmp, trie);
if (val) {
/*
* At this point we have 2 identical properties on the same match-string. We
* strictly order them by filename+line-number, since we know the dynamic
* runtime lookup does the same for multiple matching nodes.
/* At this point we have 2 identical properties on the same match-string.
* Since we process files in order, we just replace the previous value.
*/
r = strcmp(filename, trie->strings->buf + val->filename_off);
if (r < 0 ||
(r == 0 && line_number < val->line_number))
return 0;
/* replace existing earlier key with new value */
val->value_off = v;
val->filename_off = fn;
val->file_priority = file_priority;
val->line_number = line_number;
return 0;
}
@ -210,6 +203,7 @@ static int trie_node_add_value(struct trie *trie, struct trie_node *node,
node->values[node->values_count].key_off = k;
node->values[node->values_count].value_off = v;
node->values[node->values_count].filename_off = fn;
node->values[node->values_count].file_priority = file_priority;
node->values[node->values_count].line_number = line_number;
node->values_count++;
qsort_r(node->values, node->values_count, sizeof(struct trie_value_entry), trie_values_cmp, trie);
@ -218,9 +212,9 @@ static int trie_node_add_value(struct trie *trie, struct trie_node *node,
static int trie_insert(struct trie *trie, struct trie_node *node, const char *search,
const char *key, const char *value,
const char *filename, uint64_t line_number) {
const char *filename, uint16_t file_priority, uint32_t line_number) {
size_t i = 0;
int err = 0;
int r = 0;
for (;;) {
size_t p;
@ -261,9 +255,9 @@ static int trie_insert(struct trie *trie, struct trie_node *node, const char *se
node->children_count = 0;
node->values = NULL;
node->values_count = 0;
err = node_add_child(trie, node, new_child, c);
if (err < 0)
return err;
r = node_add_child(trie, node, new_child, c);
if (r < 0)
return r;
new_child = NULL; /* avoid cleanup */
break;
@ -272,7 +266,7 @@ static int trie_insert(struct trie *trie, struct trie_node *node, const char *se
c = search[i];
if (c == '\0')
return trie_node_add_value(trie, node, key, value, filename, line_number);
return trie_node_add_value(trie, node, key, value, filename, file_priority, line_number);
child = node_lookup(node, c);
if (!child) {
@ -290,13 +284,13 @@ static int trie_insert(struct trie *trie, struct trie_node *node, const char *se
}
child->prefix_off = off;
err = node_add_child(trie, node, child, c);
if (err < 0) {
r = node_add_child(trie, node, child, c);
if (r < 0) {
free(child);
return err;
return r;
}
return trie_node_add_value(trie, child, key, value, filename, line_number);
return trie_node_add_value(trie, child, key, value, filename, file_priority, line_number);
}
node = child;
@ -335,11 +329,11 @@ static int64_t trie_store_nodes(struct trie_f *trie, struct trie_node *node) {
.children_count = node->children_count,
.values_count = htole64(node->values_count),
};
struct trie_child_entry_f *children = NULL;
_cleanup_free_ struct trie_child_entry_f *children = NULL;
int64_t node_off;
if (node->children_count) {
children = new0(struct trie_child_entry_f, node->children_count);
children = new(struct trie_child_entry_f, node->children_count);
if (!children)
return -ENOMEM;
}
@ -349,12 +343,13 @@ static int64_t trie_store_nodes(struct trie_f *trie, struct trie_node *node) {
int64_t child_off;
child_off = trie_store_nodes(trie, node->children[i].child);
if (child_off < 0) {
free(children);
if (child_off < 0)
return child_off;
}
children[i].c = node->children[i].c;
children[i].child_off = htole64(child_off);
children[i] = (struct trie_child_entry_f) {
.c = node->children[i].c,
.child_off = htole64(child_off),
};
}
/* write node */
@ -366,7 +361,6 @@ static int64_t trie_store_nodes(struct trie_f *trie, struct trie_node *node) {
if (node->children_count) {
fwrite(children, sizeof(struct trie_child_entry_f), node->children_count, trie->f);
trie->children_count += node->children_count;
free(children);
}
/* append values array */
@ -375,12 +369,13 @@ static int64_t trie_store_nodes(struct trie_f *trie, struct trie_node *node) {
.key_off = htole64(trie->strings_off + node->values[i].key_off),
.value_off = htole64(trie->strings_off + node->values[i].value_off),
.filename_off = htole64(trie->strings_off + node->values[i].filename_off),
.line_number = htole64(node->values[i].line_number),
.line_number = htole32(node->values[i].line_number),
.file_priority = htole16(node->values[i].file_priority),
};
fwrite(&v, sizeof(struct trie_value_entry2_f), 1, trie->f);
trie->values_count++;
}
trie->values_count += node->values_count;
return node_off;
}
@ -401,24 +396,21 @@ static int trie_store(struct trie *trie, const char *filename) {
.child_entry_size = htole64(sizeof(struct trie_child_entry_f)),
.value_entry_size = htole64(sizeof(struct trie_value_entry2_f)),
};
int err;
int r;
/* calculate size of header, nodes, children entries, value entries */
t.strings_off = sizeof(struct trie_header_f);
trie_store_nodes_size(&t, trie->root);
err = fopen_temporary(filename , &t.f, &filename_tmp);
if (err < 0)
return err;
r = fopen_temporary(filename , &t.f, &filename_tmp);
if (r < 0)
return r;
fchmod(fileno(t.f), 0444);
/* write nodes */
err = fseeko(t.f, sizeof(struct trie_header_f), SEEK_SET);
if (err < 0) {
fclose(t.f);
unlink_noerrno(filename_tmp);
return -errno;
}
if (fseeko(t.f, sizeof(struct trie_header_f), SEEK_SET) < 0)
goto error;
root_off = trie_store_nodes(&t, trie->root);
h.nodes_root_off = htole64(root_off);
pos = ftello(t.f);
@ -431,21 +423,14 @@ static int trie_store(struct trie *trie, const char *filename) {
/* write header */
size = ftello(t.f);
h.file_size = htole64(size);
err = fseeko(t.f, 0, SEEK_SET);
if (err < 0) {
fclose(t.f);
if (fseeko(t.f, 0, SEEK_SET) < 0)
goto error;
fwrite(&h, sizeof(struct trie_header_f), 1, t.f);
if (fclose(t.f) < 0 || rename(filename_tmp, filename) < 0) {
unlink_noerrno(filename_tmp);
return -errno;
}
fwrite(&h, sizeof(struct trie_header_f), 1, t.f);
err = ferror(t.f);
if (err)
err = -errno;
fclose(t.f);
if (err < 0 || rename(filename_tmp, filename) < 0) {
unlink_noerrno(filename_tmp);
return err < 0 ? err : -errno;
}
log_debug("=== trie on-disk ===");
log_debug("size: %8"PRIi64" bytes", size);
@ -458,39 +443,46 @@ static int trie_store(struct trie *trie, const char *filename) {
t.values_count * sizeof(struct trie_value_entry2_f), t.values_count);
log_debug("string store: %8zu bytes", trie->strings->len);
log_debug("strings start: %8"PRIu64, t.strings_off);
return 0;
error:
r = -errno;
fclose(t.f);
unlink(filename_tmp);
return r;
}
static int insert_data(struct trie *trie, char **match_list, char *line,
const char *filename, size_t line_number) {
const char *filename, uint16_t file_priority, uint32_t line_number) {
char *value, **entry;
assert(line[0] == ' ');
value = strchr(line, '=');
if (!value) {
log_error("Error, key/value pair expected but got '%s' in '%s':", line, filename);
return -EINVAL;
}
if (!value)
return log_syntax(NULL, LOG_WARNING, filename, line_number, EINVAL,
"Key-value pair expected but got \"%s\", ignoring", line);
value[0] = '\0';
value++;
/* libudev requires properties to start with a space */
/* Replace multiple leading spaces by a single space */
while (isblank(line[0]) && isblank(line[1]))
line++;
if (line[0] == '\0' || value[0] == '\0') {
log_error("Error, empty key or value '%s' in '%s':", line, filename);
return -EINVAL;
}
if (isempty(line + 1) || isempty(value))
return log_syntax(NULL, LOG_WARNING, filename, line_number, EINVAL,
"Empty %s in \"%s=%s\", ignoring",
isempty(line + 1) ? "key" : "value",
line, value);
STRV_FOREACH(entry, match_list)
trie_insert(trie, trie->root, *entry, line, value, filename, line_number);
trie_insert(trie, trie->root, *entry, line, value, filename, file_priority, line_number);
return 0;
}
static int import_file(struct trie *trie, const char *filename) {
static int import_file(struct trie *trie, const char *filename, uint16_t file_priority) {
enum {
HW_NONE,
HW_MATCH,
@ -499,7 +491,7 @@ static int import_file(struct trie *trie, const char *filename) {
_cleanup_fclose_ FILE *f = NULL;
char line[LINE_MAX];
_cleanup_strv_free_ char **match_list = NULL;
size_t line_number = 0;
uint32_t line_number = 0;
char *match = NULL;
int r;
@ -534,7 +526,8 @@ static int import_file(struct trie *trie, const char *filename) {
break;
if (line[0] == ' ') {
log_error("Error, MATCH expected but got '%s' in '%s':", line, filename);
log_syntax(NULL, LOG_WARNING, filename, line_number, EINVAL,
"Match expected but got indented property \"%s\", ignoring line", line);
break;
}
@ -553,14 +546,16 @@ static int import_file(struct trie *trie, const char *filename) {
case HW_MATCH:
if (len == 0) {
log_error("Error, DATA expected but got empty line in '%s':", filename);
log_syntax(NULL, LOG_WARNING, filename, line_number, EINVAL,
"Property expected, ignoring record with no properties");
state = HW_NONE;
strv_clear(match_list);
break;
}
/* another match */
if (line[0] != ' ') {
/* another match */
match = strdup(line);
if (!match)
return -ENOMEM;
@ -574,29 +569,34 @@ static int import_file(struct trie *trie, const char *filename) {
/* first data */
state = HW_DATA;
insert_data(trie, match_list, line, filename, line_number);
insert_data(trie, match_list, line, filename, file_priority, line_number);
break;
case HW_DATA:
/* end of record */
if (len == 0) {
/* end of record */
state = HW_NONE;
strv_clear(match_list);
break;
}
if (line[0] != ' ') {
log_error("Error, DATA expected but got '%s' in '%s':", line, filename);
log_syntax(NULL, LOG_WARNING, filename, line_number, EINVAL,
"Property or empty line expected, got \"%s\", ignoring record", line);
state = HW_NONE;
strv_clear(match_list);
break;
}
insert_data(trie, match_list, line, filename, line_number);
insert_data(trie, match_list, line, filename, file_priority, line_number);
break;
};
}
if (state == HW_MATCH)
log_syntax(NULL, LOG_WARNING, filename, line_number, EINVAL,
"Property expected, ignoring record with no properties");
return 0;
}
@ -624,7 +624,9 @@ static int hwdb_query(int argc, char *argv[], void *userdata) {
static int hwdb_update(int argc, char *argv[], void *userdata) {
_cleanup_free_ char *hwdb_bin = NULL;
_cleanup_(trie_freep) struct trie *trie = NULL;
char **files, **f;
_cleanup_strv_free_ char **files = NULL;
char **f;
uint16_t file_priority = 1;
int r;
trie = new0(struct trie, 1);
@ -645,13 +647,12 @@ static int hwdb_update(int argc, char *argv[], void *userdata) {
r = conf_files_list_strv(&files, ".hwdb", arg_root, conf_file_dirs);
if (r < 0)
return log_error_errno(r, "failed to enumerate hwdb files: %m");
return log_error_errno(r, "Failed to enumerate hwdb files: %m");
STRV_FOREACH(f, files) {
log_debug("reading file '%s'", *f);
import_file(trie, *f);
log_debug("Reading file \"%s\"", *f);
import_file(trie, *f, file_priority++);
}
strv_free(files);
strbuf_complete(trie->strings);

View file

@ -76,5 +76,7 @@ struct trie_value_entry2_f {
le64_t key_off;
le64_t value_off;
le64_t filename_off;
le64_t line_number;
le32_t line_number;
le16_t file_priority;
le16_t padding;
} _packed_;

View file

@ -48,8 +48,6 @@ struct sd_hwdb {
const char *map;
};
char *modalias;
OrderedHashmap *properties;
Iterator properties_iterator;
bool properties_modified;
@ -164,10 +162,38 @@ static int hwdb_add_property(sd_hwdb *hwdb, const struct trie_value_entry_f *ent
entry2 = (const struct trie_value_entry2_f *)entry;
old = ordered_hashmap_get(hwdb->properties, key);
if (old) {
/* on duplicates, we order by filename and line-number */
r = strcmp(trie_string(hwdb, entry2->filename_off), trie_string(hwdb, old->filename_off));
if (r < 0 ||
(r == 0 && entry2->line_number < old->line_number))
/* On duplicates, we order by filename priority and line-number.
*
*
* v2 of the format had 64 bits for the line number.
* v3 reuses top 32 bits of line_number to store the priority.
* We check the top bits if they are zero we have v2 format.
* This means that v2 clients will print wrong line numbers with
* v3 data.
*
* For v3 data: we compare the priority (of the source file)
* and the line number.
*
* For v2 data: we rely on the fact that the filenames in the hwdb
* are added in the order of priority (higher later), because they
* are *processed* in the order of priority. So we compare the
* indices to determine which file had higher priority. Comparing
* the strings alphabetically would be useless, because those are
* full paths, and e.g. /usr/lib would sort after /etc, even
* though it has lower priority. This is not reliable because of
* suffix compression, but should work for the most common case of
* /usr/lib/udev/hwbd.d and /etc/udev/hwdb.d, and is better than
* not doing the comparison at all.
*/
bool lower;
if (entry2->file_priority == 0)
lower = entry2->filename_off < old->filename_off ||
(entry2->filename_off == old->filename_off && entry2->line_number < old->line_number);
else
lower = entry2->file_priority < old->file_priority ||
(entry2->file_priority == old->file_priority && entry2->line_number < old->line_number);
if (lower)
return 0;
}
}
@ -234,7 +260,7 @@ static int trie_search_f(sd_hwdb *hwdb, const char *search) {
uint8_t c;
for (; (c = trie_string(hwdb, node->prefix_off)[p]); p++) {
if (c == '*' || c == '?' || c == '[')
if (IN_SET(c, '*', '?', '['))
return trie_fnmatch_f(hwdb, node, p, &buf, search + i + p);
if (c != search[i + p])
return 0;
@ -365,7 +391,6 @@ _public_ sd_hwdb *sd_hwdb_unref(sd_hwdb *hwdb) {
if (hwdb->map)
munmap((void *)hwdb->map, hwdb->st.st_size);
safe_fclose(hwdb->f);
free(hwdb->modalias);
ordered_hashmap_free(hwdb->properties);
free(hwdb);
}
@ -399,32 +424,13 @@ bool hwdb_validate(sd_hwdb *hwdb) {
}
static int properties_prepare(sd_hwdb *hwdb, const char *modalias) {
_cleanup_free_ char *mod = NULL;
int r;
assert(hwdb);
assert(modalias);
if (streq_ptr(modalias, hwdb->modalias))
return 0;
mod = strdup(modalias);
if (!mod)
return -ENOMEM;
ordered_hashmap_clear(hwdb->properties);
hwdb->properties_modified = true;
r = trie_search_f(hwdb, modalias);
if (r < 0)
return r;
free(hwdb->modalias);
hwdb->modalias = mod;
mod = NULL;
return 0;
return trie_search_f(hwdb, modalias);
}
_public_ int sd_hwdb_get(sd_hwdb *hwdb, const char *modalias, const char *key, const char **_value) {

View file

@ -32,11 +32,40 @@ D=$(mktemp --directory)
trap "rm -rf '$D'" EXIT INT QUIT PIPE
mkdir -p "$D/etc/udev"
ln -s "$ROOTDIR/hwdb" "$D/etc/udev/hwdb.d"
err=$("$SYSTEMD_HWDB" update --root "$D" 2>&1 >/dev/null)
# Test "good" properties" — no warnings or errors allowed
err=$("$SYSTEMD_HWDB" update --root "$D" 2>&1 >/dev/null) && rc= || rc=$?
if [ -n "$err" ]; then
echo "$err"
exit 1
exit ${rc:-1}
fi
if [ -n "$rc" ]; then
echo "$SYSTEMD_HWDB returned $rc"
exit $rc
fi
if [ ! -e "$D/etc/udev/hwdb.bin" ]; then
echo "$D/etc/udev/hwdb.bin was not generated"
exit 1
fi
# Test "bad" properties" — warnings required, errors not allowed
rm -f "$D/etc/udev/hwdb.bin" "$D/etc/udev/hwdb.d"
ln -s "$ROOTDIR/test/hwdb" "$D/etc/udev/hwdb.d"
err=$("$SYSTEMD_HWDB" update --root "$D" 2>&1 >/dev/null) && rc= || rc=$?
if [ -n "$rc" ]; then
echo "$SYSTEMD_HWDB returned $rc"
exit $rc
fi
if [ -n "$err" ]; then
echo "Expected warnings"
echo "$err"
else
echo "$SYSTEMD_HWDB unexpectedly printed no warnings"
exit 1
fi
if [ ! -e "$D/etc/udev/hwdb.bin" ]; then
echo "$D/etc/udev/hwdb.bin was not generated"
exit 1

26
test/hwdb/10-bad.hwdb Normal file
View file

@ -0,0 +1,26 @@
BAD:1:no properties
BAD:2:no properties
BAD:2:no properties
BAD:3:no properties
BAD:3:no properties
BAD:3:no properties
GOOD:5:bad property
NO_VALUE
GOOD:6:bad property
=NO_NAME
NO_VALUE=
BAD:7:match at wrong place
X=Y
BAD:7:match at wrong place
BAD:8:match at wrong place
X=Y
BAD:8:match at wrong place
Z=z
BAD:8:match at EOF