Continuous Fuzzing Integration with Fuzzit

includes two travis ci steps: 1) Every pull-request/push all fuzzing targets will do a quick sanity run on the generated corpus and crashes (via Fuzzit) 2) On a daily basis the fuzzing targets will be compiled (from master) and will and their respectible fuzzing job on Fuzzit will be updated to the new binary.
2019-06-11 09:25:45 +03:00 · 2019-06-11 09:25:45 +03:00 · 53a42e6268
parent c111cd9834
commit 53a42e6268
4 changed files with 86 additions and 2 deletions
--- a/.travis.yml
+++ b/.travis.yml
@ -15,6 +15,9 @@ stages:
    - name: Coverity
      if: type = cron

+    - name: Fuzzit-Fuzzing
+      if: type = cron
+
 jobs:
    include:
        - stage: Build & test
@ -91,6 +94,22 @@ jobs:
          after_script:
              - $CI_MANAGERS/debian.sh CLEANUP

+        - stage: Fuzzit-Sanity
+          name:  Continuous Fuzzing Sanity via Fuzzit (sanity)
+          language: bash
+          script:
+            - set -e
+            - $CI_MANAGERS/fuzzit.sh sanity
+            - set +e
+
+        - stage: Fuzzit-Fuzzing
+          name: Continuous Fuzzing Sanity via Fuzzit (fuzzing daily)
+          language: bash
+          script:
+            - set -e
+            - $CI_MANAGERS/fuzzit.sh fuzzing
+            - set +e
+
        - stage: Coverity
          language: bash
          env:
--- a/README.md
+++ b/README.md
@ -4,6 +4,7 @@
 <a href="https://in.waw.pl/systemd-github-state/systemd-systemd-pull-requests.svg"><img align="right" src="https://in.waw.pl/systemd-github-state/systemd-systemd-pull-requests-small.svg" alt="Count of open pull requests over time"></a>
 [![Semaphore CI Build Status](https://semaphoreci.com/api/v1/projects/28a5a3ca-3c56-4078-8b5e-7ed6ef912e14/443470/shields_badge.svg)](https://semaphoreci.com/systemd/systemd)<br/>
 [![Coverity Scan Status](https://scan.coverity.com/projects/350/badge.svg)](https://scan.coverity.com/projects/350)<br/>
+[![Fuzzit Status](https://app.fuzzit.dev/badge?org_id=RxqRpGNXquIvqrmp4iJS&branch=master)](https://app.fuzzit.dev/admin/RxqRpGNXquIvqrmp4iJS/dashboard)<br/>
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/1369/badge)](https://bestpractices.coreinfrastructure.org/projects/1369)<br/>
 [![Travis CI Build Status](https://travis-ci.org/systemd/systemd.svg?branch=master)](https://travis-ci.org/systemd/systemd)<br/>
 [![Language Grade: C/C++](https://img.shields.io/lgtm/grade/cpp/g/systemd/systemd.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/systemd/systemd/context:cpp)<br/>
--- a/docs/HACKING.md
+++ b/docs/HACKING.md
@ -96,8 +96,8 @@ Happy hacking!
 ## Fuzzers

 systemd includes fuzzers in `src/fuzz/` that use libFuzzer and are automatically
-run by [OSS-Fuzz](https://github.com/google/oss-fuzz) with sanitizers. To add a
-fuzz target, create a new `src/fuzz/fuzz-foo.c` file with a `LLVMFuzzerTestOneInput`
+run by [OSS-Fuzz](https://github.com/google/oss-fuzz) and [Fuzzit](https://fuzzit.dev) with sanitizers.
+To add a fuzz target, create a new `src/fuzz/fuzz-foo.c` file with a `LLVMFuzzerTestOneInput`
 function and add it to the list in `src/fuzz/meson.build`.

 Whenever possible, a seed corpus and a dictionary should also be added with new
@ -116,6 +116,10 @@ python infra/helper.py build_fuzzers --sanitizer memory systemd ../systemd
 python infra/helper.py run_fuzzer systemd fuzz-foo
 ```

+When you add a new target you should also add the target on [Fuzzit](https://app.fuzzit.dev/admin/RxqRpGNXquIvqrmp4iJS/dashboard)
+ (Please ask someone with permissions). One the target is configured on Fuzzit you need to add it to
+ `travis-ci/managers/fuzzit.sh` so the new target will run sanity tests on every pull-request and periodic fuzzing jobs.
+
 If you find a bug that impacts the security of systemd, please follow the
 guidance in [CONTRIBUTING.md](CONTRIBUTING.md) on how to report a security vulnerability.

--- a/travis-ci/managers/fuzzit.sh
+++ b/travis-ci/managers/fuzzit.sh
@ -0,0 +1,60 @@
+#!/bin/bash
+
+set -e
+set -x
+set -u
+
+REPO_ROOT=${REPO_ROOT:-$(pwd)}
+
+sudo bash -c "echo 'deb-src http://archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse' >>/etc/apt/sources.list"
+sudo apt-get update -y
+sudo apt-get build-dep systemd -y
+sudo apt-get install -y ninja-build python3-pip python3-setuptools
+pip3 install meson
+
+cd $REPO_ROOT
+export PATH="$HOME/.local/bin/:$PATH"
+export SANITIZER=address,undefined
+tools/oss-fuzz.sh
+
+export FUZZING_TYPE=${1:-sanity}
+if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then
+    export FUZZIT_BRANCH="${TRAVIS_BRANCH}"
+else
+    export FUZZIT_BRANCH="PR-${TRAVIS_PULL_REQUEST}"
+fi
+
+# Because we want Fuzzit to run on every pull-request and Travis/Azure doesnt support encrypted keys
+# on pull-request we use a write-only key which is ok for now. maybe there will be a better solution in the future
+export FUZZIT_API_KEY=7c1bd82fe0927ffe1b4bf1e2e86cc812b28dfe08a7080a7bf498e98715884a163402ee37ba95d4b1637247deffcea43e
+export FUZZIT_ADDITIONAL_FILES="./out/src/shared/libsystemd-shared-242.so"
+export FUZZIT_ARGS="--type ${FUZZING_TYPE} --branch "${FUZZIT_BRANCH}" --revision ${TRAVIS_COMMIT}"
+wget -O fuzzit https://bin.fuzzit.dev/fuzzit-1.1
+chmod +x fuzzit
+
+./fuzzit auth ${FUZZIT_API_KEY}
+./fuzzit c job ${FUZZIT_ARGS} 2ODbhEjfRF2AZtrUotMh ./out/fuzz-bus-label ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} 62XnUyWTLAvIRh1vFkEw ./out/fuzz-journald-stream ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} 6AdGwIiI3l1Edu9V4fvF ./out/fuzz-env-file ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} 7ubB4DVu2EiYgPVtRUNV ./out/fuzz-calendarspec ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} --asan_options "quarantine_size_mb=10"  8D0NrVtSwTpl23a9k0vv ./out/fuzz-nspawn-oci ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} 8tbrzwxsaIPalIRBHtK8 ./out/fuzz-link-parser ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} 9T5He9cANxHTBLaBURpz ./out/fuzz-journald-kmsg ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} BRaEBuU7QVlSp1HOjlDb ./out/fuzz-udev-database ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} DcE70rAA2mhrxdyBRH90 ./out/fuzz-udev-rules ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} KH6VEpV0ZoWynASJHm8z ./out/fuzz-dhcp6-client ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} MZNs1JG5UQstaIvfHYgb ./out/fuzz-netdev-parser ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} P1MpkewCNQCYLdMFggnU ./out/fuzz-journald-audit ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} RmD47BxVRbAZlq07XW30 ./out/fuzz-unit-file ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} S0dGMaaGwkvsLc0IqIJ7 ./out/fuzz-catalog ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} bgRZAE9E5uXRbUX76tId ./out/fuzz-ndisc-rs ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} cXCm75EhdDf5t2sSBLRC ./out/fuzz-hostname-util ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} cbgsYEyX6776MHFotO9O ./out/fuzz-nspawn-settings ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} d8lokp0LCLYgQwI7vyx6 ./out/fuzz-journald-native-fd ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} eoc9rbm2jKqIEg6Kdonv ./out/fuzz-network-parser ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} ge3eTzephghWD3Stw2TE ./out/fuzz-journald-syslog ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} nPIt1SCDkGkSFDth5RlG ./out/fuzz-json ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} nU0lRNNkQrXirDMNOpR1 ./out/fuzz-varlink ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} qCWFcENjlfWJX0Q3cIOT ./out/fuzz-journald-native ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} udjVYJfH4N01vaHNF5Kv ./out/fuzz-lldp ${FUZZIT_ADDITIONAL_FILES}
+./fuzzit c job ${FUZZIT_ARGS} vbYVccyWoDdgqzrQeln8 ./out/fuzz-bus-message ${FUZZIT_ADDITIONAL_FILES}