Impure derivations are derivations that can produce a different result
every time they're built. Example:
stdenv.mkDerivation {
name = "impure";
__impure = true; # marks this derivation as impure
buildCommand = "date > $out";
};
Some important characteristics:
* Impure derivations are not "cached". Thus, running "nix-build" on
the example above multiple times will cause a rebuild every time. In
the future, we could implement some mechanism for reusing impure
builds across invocations.
* The outputs of impure derivations are moved to a content-addressed
location after the build (i.e., the resulting store path will
correspond to the hash of the contents of the path). This way,
multiple builds of the same impure derivation do not collide.
* Because of content-addressability, the output paths of an impure
derivation recorded in its .drv file are "virtual" placeholders for
the actual outputs which are not known in advance. This also means
that "nix-store -q bla.drv" gives a meaningless path.
* Pure derivations are not allowed to depend on impure
derivations. The only exception is fixed-output derivations. Because
the latter always produce a known output, they can depend on impure
shenanigans just fine. Also, repeatedly running "nix-build" on such
a fixed-output derivation will *not* cause a rebuild of the impure
dependency. After all, if the fixed output exists, its dependencies
are no longer relevant. Thus, fixed-output derivations form an
"impurity barrier" in the dependency graph.
* When sandboxing is enabled, impure derivations can access the
network in the same way as fixed-output derivations. In relaxed
sandboxing mode, they can access the local filesystem.
* Currently, the output of an impure derivation must have no
references. This is because the content-addressing scheme must be
extended to handle references, in particular self-references (as
described in the ASE-2005 paper.)
* Currently, impure derivations can only have a single output. No real
reason for this.
* "nix-build" on an impure derivation currently creates a result
symlink to the incorrect, virtual output.
A motivating example is the problem of using "fetchurl" on a
dynamically generated tarball whose contents are deterministic, but
where the tarball does not have a canonical form. Previously, this
required "fetchurl" to do the unpacking in the same
derivation. (That's what "fetchzip" does.) But now we can say:
tarball = stdenv.mkDerivation {
__impure = true;
name = "tarball";
buildInputs = [ curl ];
buildCommand =
"curl --fail -Lk c1f89c077e > $out";
};
unpacked = stdenv.mkDerivation {
name = "unpacked";
outputHashAlgo = "sha256";
outputHashMode = "recursive";
outputHash = "1jl8n1n36w63wffkm56slcfa7vj9fxkv4ax0fr0mcfah55qj5l8s";
buildCommand =
"mkdir $out; tar xvf ${tarball} -C $out";
};
I needed this because <nix/fetchurl.nix> does not support unpacking,
and adding untar/unzip functionality would be annoying (especially
since we can't just call "tar" or "unzip" in a sandbox).
https://github.com/NixOS/nix/issues/520
The nix-shell fix in 668fef2e4f revealed
that we had some --pure tests that incorrectly depended on PATH from
config.nix's mkDerivation being overwritten by the caller's PATH.
http://hydra.nixos.org/build/49242478
Need to remember that std::map::insert() and emplace() don't overwrite
existing entries...
This fixes a regression relative to 1.11 that in particular triggers
in nested nix-shells.
Before:
$ nativeBuildInputs=/foo nix-shell -p hello --run 'hello'
build input /foo does not exist
After:
$ nativeBuildInputs=/foo nix-shell -p hello --run 'hello'
Hello, world!
... and use this in Downloader::downloadCached(). This fixes
$ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello
error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
Disabled hardened build because it makes the linker fail with messages like
relocation R_X86_64_PC32 against undefined symbol `BZ2_bzWriteOpen' can not be used when making a shared object; recompile with -fPIC
See https://fedoraproject.org/wiki/Changes/Harden_All_Packages.
This allows <nix/fetchurl.nix> to fetch private Git/Mercurial
repositories, e.g.
import <nix/fetchurl.nix> {
url = https://edolstra@bitbucket.org/edolstra/my-private-repo/get/80a14018daed.tar.bz2;
sha256 = "1mgqzn7biqkq3hf2697b0jc4wabkqhmzq2srdymjfa6sb9zb6qs7";
}
where /etc/nix/netrc contains:
machine bitbucket.org
login edolstra
password blabla...
This works even when sandboxing is enabled.
To do: add unpacking support (i.e. fetchzip functionality).
Some sites (e.g. BitBucket) give a helpful 401 error when trying to
download a private archive if the User-Agent contains "curl", but give
a redirect to a login page otherwise (so for instance
"nix-prefetch-url" will succeed but produce useless output).
This adds support for s3:// URIs in all places where Nix allows URIs,
e.g. in builtins.fetchurl, builtins.fetchTarball, <nix/fetchurl.nix>
and NIX_PATH. It allows fetching resources from private S3 buckets,
using credentials obtained from the standard places (i.e. AWS_*
environment variables, ~/.aws/credentials and the EC2 metadata
server). This may not be super-useful in general, but since we already
depend on aws-sdk-cpp, it's a cheap feature to add.
Currently, 'nix-daemon --stdio' is always failing for me, due to the
splice call always failing with (on a 32-bit host):
splice(0, NULL, 3, NULL, 4294967295, SPLICE_F_MOVE) = -1 EINVAL (Invalid argument)
With a bit of ftracing (and luck) the problem seems to be that splice()
always fails with EINVAL if the len cast as ssize_t is negative:
http://lxr.free-electrons.com/source/fs/read_write.c?v=4.4#L384
So use SSIZE_MAX instead of SIZE_MAX.
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not
every compilation unit includes config.h, we currently compile half of
Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS
unset. This causes major havoc with the Settings class on e.g. 32-bit ARM,
where different compilation units disagree with the struct layout.
E.g.:
diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc
@@ -166,6 +166,8 @@ void Settings::update()
_get(useSubstitutes, "build-use-substitutes");
+ fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes);
_get(buildUsersGroup, "build-users-group");
diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc
+++ b/src/libstore/remote-store.cc
@@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn)
void RemoteStore::setOptions(Connection & conn)
{
+ fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes);
conn.to << wopSetOptions
Gave me:
at Settings::update(): &useSubstitutes = 0xb6e5c5cb
at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7
That was not a fun one to debug!
Eelco Dolstra