False positives in GA labeler bot

Don't automatically match on man/* and **/meson.build, since we change those
files as part of larger changes that we usually don't mark that way

.github/labeler.yml

@@ -4,7 +4,6 @@ units:
   - units/**/*
 documentation:
   - NEWS
-  - man/*
   - docs/*
 network:
   - src/libsystemd-network/**/*
@@ -17,7 +16,6 @@ selinux:
 apparmor:
   - '**/*apparmor*'
 meson:
-  - '**/meson.build'
   - meson_option.txt
 busctl:
   - src/busctl/*