picnoir/nix-gh
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
nix-gh/tests/binary-cache.sh

138 lines
3.8 KiB
Bash
Raw Normal View History

Add a test for the binary cache substituter
2012-07-26 23:13:14 +02:00
source common.sh
clearStore
Add a test for repairing paths
2014-02-17 12:22:50 +01:00
clearCache
Add a test for the binary cache substituter
2012-07-26 23:13:14 +02:00
Test "nix-env -qas" with the binary cache substituter
2012-07-27 20:15:03 +02:00
# Create the binary cache.
Add a test for the binary cache substituter
2012-07-26 23:13:14 +02:00
outPath=$(nix-build dependencies.nix --no-out-link)
nix-push --dest $cacheDir $outPath
Allow a binary cache to declare that it doesn't support "nix-env -qas" Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-28 00:16:05 +02:00
# By default, a binary cache doesn't support "nix-env -qas", but does
# support installation.
Add a test for the binary cache substituter
2012-07-26 23:13:14 +02:00
clearStore
Test the NAR info cache
2016-05-30 14:53:57 +02:00
clearCacheCache
Add a test for the binary cache substituter
2012-07-26 23:13:14 +02:00
Test NAR info caching
2013-04-23 12:43:28 +02:00
export _NIX_CACHE_FILE_URLS=1
Fix the test
2012-07-30 23:09:13 +02:00
nix-env --option binary-caches "file://$cacheDir" -f dependencies.nix -qas \* | grep -- "---"
Allow a binary cache to declare that it doesn't support "nix-env -qas" Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-28 00:16:05 +02:00
Fix the test
2012-07-30 23:09:13 +02:00
nix-store --option binary-caches "file://$cacheDir" -r $outPath
Allow a binary cache to declare that it doesn't support "nix-env -qas" Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-28 00:16:05 +02:00
Test executables in NARs
2014-02-26 18:59:01 +01:00
[ -x $outPath/program ]
Allow a binary cache to declare that it doesn't support "nix-env -qas" Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-28 00:16:05 +02:00
# But with the right configuration, "nix-env -qas" should also work.
clearStore
Test the NAR info cache
2016-05-30 14:53:57 +02:00
clearCacheCache
Allow a binary cache to declare that it doesn't support "nix-env -qas" Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-28 00:16:05 +02:00
echo "WantMassQuery: 1" >> $cacheDir/nix-cache-info
Fix the test
2012-07-30 23:09:13 +02:00
nix-env --option binary-caches "file://$cacheDir" -f dependencies.nix -qas \* | grep -- "--S"
Test "nix-env -qas" with the binary cache substituter
2012-07-27 20:15:03 +02:00
Test the ‘--prebuilt-only’ flag
2012-12-03 21:02:06 +01:00
x=$(nix-env -f dependencies.nix -qas \* --prebuilt-only)
[ -z "$x" ]
Fix the test
2012-07-30 23:09:13 +02:00
nix-store --option binary-caches "file://$cacheDir" -r $outPath
Add a test for the binary cache substituter
2012-07-26 23:13:14 +02:00
nix-store --check-validity $outPath
nix-store -qR $outPath | grep input-2
Allow a binary cache to declare that it doesn't support "nix-env -qas" Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-28 00:16:05 +02:00
Add a test for incomplete closures in the binary cache Issue #77.
2013-01-02 11:45:23 +01:00
Test whether Nix correctly checks the hash of downloaded NARs
2014-01-08 17:56:30 +01:00
# Test whether Nix notices if the NAR doesn't match the hash in the NAR info.
clearStore
nar=$(ls $cacheDir/*.nar.xz | head -n1)
mv $nar $nar.good
mkdir -p $TEST_ROOT/empty
nix-store --dump $TEST_ROOT/empty | xz > $nar
nix-build --option binary-caches "file://$cacheDir" dependencies.nix -o $TEST_ROOT/result 2>&1 | tee $TEST_ROOT/log
LocalStore::addToStore: Verify hash of the imported path
2016-05-30 13:44:09 +02:00
grep -q "hash mismatch" $TEST_ROOT/log
Test whether Nix correctly checks the hash of downloaded NARs
2014-01-08 17:56:30 +01:00
mv $nar.good $nar
Support cryptographically signed binary caches NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2014-01-08 15:23:41 +01:00
# Test whether this unsigned cache is rejected if the user requires signed caches.
clearStore
Test the NAR info cache
2016-05-30 14:53:57 +02:00
clearCacheCache
Support cryptographically signed binary caches NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2014-01-08 15:23:41 +01:00
Fix signed-binary-caches test
2014-01-08 17:56:58 +01:00
if nix-store --option binary-caches "file://$cacheDir" --option signed-binary-caches '*' -r $outPath; then
echo "unsigned binary cache incorrectly accepted"
exit 1
fi
Support cryptographically signed binary caches NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2014-01-08 15:23:41 +01:00
Test whether --fallback works if NARS have disappeared from the binary cache
2013-04-23 12:44:01 +02:00
# Test whether fallback works if we have cached info but the
# corresponding NAR has disappeared.
clearStore
nix-build --option binary-caches "file://$cacheDir" dependencies.nix --dry-run # get info
mkdir $cacheDir/tmp
Add support for uncompressed NARs in binary caches Issue NixOS/hydra#102.
2013-07-01 21:02:36 +02:00
mv $cacheDir/*.nar* $cacheDir/tmp/
Test whether --fallback works if NARS have disappeared from the binary cache
2013-04-23 12:44:01 +02:00
NIX_DEBUG_SUBST=1 nix-build --option binary-caches "file://$cacheDir" dependencies.nix -o $TEST_ROOT/result --fallback
mv $cacheDir/tmp/* $cacheDir/
Add a test for incomplete closures in the binary cache Issue #77.
2013-01-02 11:45:23 +01:00
# Test whether building works if the binary cache contains an
# incomplete closure.
clearStore
rm $(grep -l "StorePath:.*dependencies-input-2" $cacheDir/*.narinfo)
If a substitute closure is incomplete, build dependencies, then retry the substituter Issue #77.
2013-01-02 12:38:28 +01:00
nix-build --option binary-caches "file://$cacheDir" dependencies.nix -o $TEST_ROOT/result 2>&1 | tee $TEST_ROOT/log
Re-implement binary cache signature checking This is now done in LocalStore::addToStore(), rather than in the binary cache substituter (which no longer exists).
2016-05-30 13:55:09 +02:00
grep -q "fetching path" $TEST_ROOT/log
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
Make libsodium an optional dependency
2015-02-10 11:54:06 +01:00
if [ -n "$HAVE_SODIUM" ]; then
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
# Create a signed binary cache.
clearCache
nix-store --generate-binary-cache-key: Write key to disk This ensures proper permissions for the secret key.
2015-02-18 11:19:44 +01:00
declare -a res=($(nix-store --generate-binary-cache-key test.nixos.org-1 $TEST_ROOT/sk1 $TEST_ROOT/pk1 ))
publicKey="$(cat $TEST_ROOT/pk1)"
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
nix-store --generate-binary-cache-key: Write key to disk This ensures proper permissions for the secret key.
2015-02-18 11:19:44 +01:00
res=($(nix-store --generate-binary-cache-key test.nixos.org-1 $TEST_ROOT/sk2 $TEST_ROOT/pk2))
badKey="$(cat $TEST_ROOT/pk2)"
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
nix-store --generate-binary-cache-key: Write key to disk This ensures proper permissions for the secret key.
2015-02-18 11:19:44 +01:00
res=($(nix-store --generate-binary-cache-key foo.nixos.org-1 $TEST_ROOT/sk3 $TEST_ROOT/pk3))
otherKey="$(cat $TEST_ROOT/pk3)"
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
nix-store --generate-binary-cache-key: Write key to disk This ensures proper permissions for the secret key.
2015-02-18 11:19:44 +01:00
nix-push --dest $cacheDir --key-file $TEST_ROOT/sk1 $outPath
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
# Downloading should fail if we don't provide a key.
clearStore
Test the NAR info cache
2016-05-30 14:53:57 +02:00
clearCacheCache
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
(! nix-store -r $outPath --option binary-caches "file://$cacheDir" --option signed-binary-caches '*' )
# And it should fail if we provide an incorrect key.
clearStore
Test the NAR info cache
2016-05-30 14:53:57 +02:00
clearCacheCache
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
(! nix-store -r $outPath --option binary-caches "file://$cacheDir" --option signed-binary-caches '*' --option binary-cache-public-keys "$badKey")
# It should succeed if we provide the correct key.
nix-store -r $outPath --option binary-caches "file://$cacheDir" --option signed-binary-caches '*' --option binary-cache-public-keys "$otherKey $publicKey"
# It should fail if we corrupt the .narinfo.
clearStore
for i in $cacheDir/*.narinfo; do
grep -v References $i > $i.tmp
mv $i.tmp $i
done
Test the NAR info cache
2016-05-30 14:53:57 +02:00
clearCacheCache
Use libsodium instead of OpenSSL for binary cache signing Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2015-02-04 16:43:32 +01:00
(! nix-store -r $outPath --option binary-caches "file://$cacheDir" --option signed-binary-caches '*' --option binary-cache-public-keys "$publicKey")
Make libsodium an optional dependency
2015-02-10 11:54:06 +01:00
fi # HAVE_LIBSODIUM
Reference in a new issue Copy permalink