Systemd/test/run-integration-tests.sh

82 lines
1.8 KiB
Bash
Raw Normal View History

#!/usr/bin/env bash
set -e
BUILD_DIR="$($(dirname "$0")/../tools/find-build-dir.sh)"
if [ $# -gt 0 ]; then
args="$@"
else
args="setup run clean-again"
fi
args_no_clean=$(sed -r 's/\bclean.*\b//g' <<<$args)
do_clean=$( [ "$args" = "$args_no_clean" ]; echo $? )
declare -A results
2020-03-31 11:44:09 +02:00
declare -A times
COUNT=0
FAILURES=0
cd "$(dirname "$0")"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 09:37:19 +01:00
# Let's always do the cleaning operation first, because it destroys the image
# cache.
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 09:37:19 +01:00
if [ $do_clean = 1 ]; then
for TEST in TEST-??-* ; do
( set -x ; make -C "$TEST" "BUILD_DIR=$BUILD_DIR" clean )
done
[ -n "$args_no_clean" ] || exit 0
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 09:37:19 +01:00
fi
pass_deny_list() {
for marker in $DENY_LIST_MARKERS $BLACKLIST_MARKERS; do
if [ -f "$1/$marker" ]; then
echo "========== DENY-LISTED: $1 ($marker) =========="
return 1
fi
done
return 0
}
for TEST in TEST-??-* ; do
COUNT=$(($COUNT+1))
pass_deny_list $TEST || continue
2020-03-31 11:44:09 +02:00
start=$(date +%s)
echo -e "\n--x-- Running $TEST --x--"
set +e
( set -x ; make -C "$TEST" "BUILD_DIR=$BUILD_DIR" $args_no_clean )
RESULT=$?
set -e
echo "--x-- Result of $TEST: $RESULT --x--"
results["$TEST"]="$RESULT"
2020-03-31 11:44:09 +02:00
times["$TEST"]=$(( $(date +%s) - $start ))
[ "$RESULT" -ne "0" ] && FAILURES=$(($FAILURES+1))
done
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 09:37:19 +01:00
if [ $FAILURES -eq 0 -a $do_clean = 1 ]; then
for TEST in ${!results[@]}; do
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 09:37:19 +01:00
( set -x ; make -C "$TEST" "BUILD_DIR=$BUILD_DIR" clean-again )
done
fi
echo ""
for TEST in ${!results[@]}; do
RESULT="${results[$TEST]}"
2020-03-31 11:44:09 +02:00
time="${times[$TEST]}"
string=$([ "$RESULT" = "0" ] && echo "SUCCESS" || echo "FAIL")
printf "%-35s %-8s (%3s s)\n" "${TEST}:" "${string}" "$time"
done | sort
if [ "$FAILURES" -eq 0 ] ; then
echo -e "\nALL $COUNT TESTS PASSED"
else
echo -e "\nTOTAL FAILURES: $FAILURES OF $COUNT"
fi
exit "$FAILURES"