macOS: Remove flags

In particular, UF_IMMUTABLE (uchg) needs to be cleared to allow the path to be garbage-collected or optimised. See https://github.com/NixOS/nixpkgs/issues/25819. + the file from being garbage-collected. (cherry picked from commit b5bdfdef73)
macOS: Ugly hack to make the tests succeed
2017-06-19 14:32:38 +02:00 · 2017-06-19 14:28:04 +02:00 · 2017-06-14 07:52:36 -04:00 · 2017-06-14 11:43:51 +02:00 · 2017-06-14 11:43:33 +02:00 · 2017-06-13 18:18:20 -05:00
52 changed files with 793 additions and 255 deletions
--- a/configure.ac
+++ b/configure.ac
@ -200,6 +200,13 @@ AC_SUBST(HAVE_SODIUM, [$have_sodium])
 PKG_CHECK_MODULES([LIBLZMA], [liblzma], [CXXFLAGS="$LIBLZMA_CFLAGS $CXXFLAGS"])


+# Look for libseccomp, required for Linux sandboxing.
+if test "$sys_name" = linux; then
+  PKG_CHECK_MODULES([LIBSECCOMP], [libseccomp],
+                    [CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS"])
+fi
+
+
 # Whether to use the Boehm garbage collector.
 AC_ARG_ENABLE(gc, AC_HELP_STRING([--enable-gc],
  [enable garbage collection in the Nix expression evaluator (requires Boehm GC) [default=no]]),
--- a/doc/manual/command-ref/conf-file.xml
+++ b/doc/manual/command-ref/conf-file.xml
@ -268,7 +268,12 @@ flag, e.g. <literal>--option gc-keep-outputs false</literal>.</para>
    to mount a path in a different location in the sandbox; for
    instance, <literal>/bin=/nix-bin</literal> will mount the path
    <literal>/nix-bin</literal> as <literal>/bin</literal> inside the
-    sandbox.</para>
+    sandbox. If <replaceable>source</replaceable> is followed by
+    <literal>?</literal>, then it is not an error if
+    <replaceable>source</replaceable> does not exist; for example,
+    <literal>/dev/nvidiactl?</literal> specifies that
+    <filename>/dev/nvidiactl</filename> will only be mounted in the
+    sandbox if it exists in the host filesystem.</para>

    <para>Depending on how Nix was built, the default value for this option
    may be empty or provide <filename>/bin/sh</filename> as a
@ -452,6 +457,29 @@ flag, e.g. <literal>--option gc-keep-outputs false</literal>.</para>
  </varlistentry>


+  <varlistentry><term><literal>netrc-file</literal></term>
+
+    <listitem><para>If set to an absolute path to a <filename>netrc</filename>
+    file, Nix will use the HTTP authentication credentials in this file when
+    trying to download from a remote host through HTTP or HTTPS. Defaults to
+    <filename>$NIX_CONF_DIR/netrc</filename>.</para>
+
+    <para>The <filename>netrc</filename> file consists of a list of
+    accounts in the following format:
+
+<screen>
+machine <replaceable>my-machine</replaceable>
+login <replaceable>my-username</replaceable>
+password <replaceable>my-password</replaceable>
+</screen>
+
+    For the exact syntax, see <link
+    xlink:href="https://ec.haxx.se/usingcurl-netrc.html">the
+    <literal>curl</literal> documentation.</link></para></listitem>
+
+  </varlistentry>
+
+
  <varlistentry><term><literal>system</literal></term>

    <listitem><para>This option specifies the canonical Nix system
--- a/doc/manual/command-ref/nix-env.xml
+++ b/doc/manual/command-ref/nix-env.xml
@ -367,6 +367,10 @@ number of possible ways:
  linkend="rsec-nix-store-realise">realised</link> and
  installed.</para></listitem>

+  <listitem><para>By default all outputs are installed for each derivation.
+  That can be reduced by setting <literal>meta.outputsToInstall</literal>.
+  </para></listitem> <!-- TODO: link nixpkgs docs on the ability to override those. -->
+
 </itemizedlist>

 </para>
--- a/doc/manual/expressions/simple-building-testing.xml
+++ b/doc/manual/expressions/simple-building-testing.xml
@ -7,15 +7,14 @@
 <title>Building and Testing</title>

 <para>You can now try to build Hello.  Of course, you could do
-<literal>nix-env -f pkgs/top-level/all-packages.nix -i hello</literal>,
-but you may not want to install a possibly broken package just yet.
-The best way to test the package is by using the command <command
-linkend="sec-nix-build">nix-build</command>, which builds a Nix
-expression and creates a symlink named <filename>result</filename> in
-the current directory:
+<literal>nix-env -i hello</literal>, but you may not want to install a
+possibly broken package just yet.  The best way to test the package is by
+using the command <command linkend="sec-nix-build">nix-build</command>,
+which builds a Nix expression and creates a symlink named
+<filename>result</filename> in the current directory:

 <screen>
-$ nix-build pkgs/top-level/all-packages.nix -A hello
+$ nix-build -A hello
 building path `/nix/store/632d2b22514d...-hello-2.1.1'
 hello-2.1.1/
 hello-2.1.1/intl/
@ -29,8 +28,7 @@ $ ./result/bin/hello
 Hello, world!</screen>

 The <link linkend='opt-attr'><option>-A</option></link> option selects
-the <literal>hello</literal> attribute from
-<filename>all-packages.nix</filename>.  This is faster than using the
+the <literal>hello</literal> attribute.  This is faster than using the
 symbolic package name specified by the <literal>name</literal>
 attribute (which also happens to be <literal>hello</literal>) and is
 unambiguous (there can be multiple packages with the symbolic name
@ -69,7 +67,7 @@ block (or perform other derivations if available) until the build
 finishes:

 <screen>
-$ nix-build pkgs/top-level/all-packages.nix -A hello
+$ nix-build -A hello
 waiting for lock on `/nix/store/0h5b7hp8d4hqfrw8igvx97x1xawrjnac-hello-2.1.1x'</screen>

 So it is always safe to run multiple instances of Nix in parallel
--- a/doc/manual/local.mk
+++ b/doc/manual/local.mk
@ -10,7 +10,7 @@ XSLTPROC = $(xsltproc) --nonet $(xmlflags) \
  --stringparam generate.toc "book toc" \
  --param keep.relative.image.uris 0

-docbookxsl = http://docbook.sourceforge.net/release/xsl-ns/1.78.1
+docbookxsl = http://docbook.sourceforge.net/release/xsl-ns/current
 docbookrng = http://docbook.org/xml/5.0/rng/docbook.rng

 MANUAL_SRCS := $(call rwildcard, $(d), *.xml)
--- a/doc/manual/release-notes/release-notes.xml
+++ b/doc/manual/release-notes/release-notes.xml
@ -12,6 +12,7 @@
 </partintro>
 -->

+<xi:include href="rl-1.11.10.xml" />
 <xi:include href="rl-1.11.xml" />
 <xi:include href="rl-1.10.xml" />
 <xi:include href="rl-1.9.xml" />
--- a/doc/manual/release-notes/rl-1.11.10.xml
+++ b/doc/manual/release-notes/rl-1.11.10.xml
@ -0,0 +1,31 @@
+<section xmlns="http://docbook.org/ns/docbook"
+      xmlns:xlink="http://www.w3.org/1999/xlink"
+      xmlns:xi="http://www.w3.org/2001/XInclude"
+      version="5.0"
+      xml:id="ssec-relnotes-1.11.10">
+
+<title>Release 1.11.10 (2017-06-12)</title>
+
+<para>This release fixes a security bug in Nix’s “build user” build
+isolation mechanism. Previously, Nix builders had the ability to
+create setuid binaries owned by a <literal>nixbld</literal>
+user. Such a binary could then be used by an attacker to assume a
+<literal>nixbld</literal> identity and interfere with subsequent
+builds running under the same UID.</para>
+
+<para>To prevent this issue, Nix now disallows builders to create
+setuid and setgid binaries. On Linux, this is done using a seccomp BPF
+filter. Note that this imposes a small performance penalty (e.g. 1%
+when building GNU Hello). Using seccomp, we now also prevent the
+creation of extended attributes and POSIX ACLs since these cannot be
+represented in the NAR format and (in the case of POSIX ACLs) allow
+bypassing regular Nix store permissions. On macOS, the restriction is
+implemented using the existing sandbox mechanism, which now uses a
+minimal “allow all except the creation of setuid/setgid binaries”
+profile when regular sandboxing is disabled. On other platforms, the
+“build user” mechanism is now disabled.</para>
+
+<para>Thanks go to Linus Heckemann for discovering and reporting this
+bug.</para>
+
+</section>
--- a/misc/docker/Dockerfile
+++ b/misc/docker/Dockerfile
@ -13,11 +13,11 @@ RUN set -x \
 ONBUILD ENV \
    ENV=/etc/profile \
    PATH=/root/.nix-profile/bin:/root/.nix-profile/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
-    GIT_SSL_CAINFO=/root/.nix-profile/etc/ca-bundle.crt \
-    SSL_CERT_FILE=/root/.nix-profile/etc/ca-bundle.crt
+    GIT_SSL_CAINFO=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt \
+    NIX_SSL_CERT_FILE=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt

 ENV \
    ENV=/etc/profile \
    PATH=/root/.nix-profile/bin:/root/.nix-profile/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
-    GIT_SSL_CAINFO=/root/.nix-profile/etc/ca-bundle.crt \
-    SSL_CERT_FILE=/root/.nix-profile/etc/ca-bundle.crt
+    GIT_SSL_CAINFO=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt \
+    NIX_SSL_CERT_FILE=/root/.nix-profile/etc/ssl/certs/ca-bundle.crt
--- a/misc/launchd/org.nixos.nix-daemon.plist.in
+++ b/misc/launchd/org.nixos.nix-daemon.plist.in
@ -12,5 +12,10 @@
    <string>/var/log/nix-daemon.log</string>
    <key>StandardOutPath</key>
    <string>/dev/null</string>
+    <key>EnvironmentVariables</key>
+    <dict>
+      <key>NIX_SSL_CERT_FILE</key>
+      <string>/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt</string>
+    </dict>
  </dict>
 </plist>
--- a/nix.spec.in
+++ b/nix.spec.in
@ -24,9 +24,11 @@ Requires: perl-DBD-SQLite
 Requires: bzip2
 Requires: gzip
 Requires: xz
+Requires: libseccomp
 BuildRequires: bzip2-devel
 BuildRequires: sqlite-devel
 BuildRequires: libcurl-devel
+BuildRequires: libseccomp-devel

 # Hack to make that shitty RPM scanning hack shut up.
 Provides: perl(Nix::SSH)
--- a/perl/lib/Nix/Config.pm.in
+++ b/perl/lib/Nix/Config.pm.in
@ -12,6 +12,12 @@ $logDir = $ENV{"NIX_LOG_DIR"} || "@localstatedir@/log/nix";
 $confDir = $ENV{"NIX_CONF_DIR"} || "@sysconfdir@/nix";
 $storeDir = $ENV{"NIX_STORE_DIR"} || "@storedir@";

+$caBundle = $ENV{"NIX_SSL_CERT_FILE"} // $ENV{"SSL_CERT_FILE"} // $ENV{"CURL_CA_BUNDLE"} // $ENV{"OPENSSL_X509_CERT_FILE"};
+$caBundle = "/etc/ssl/certs/ca-bundle.crt" if !$caBundle && -f "/etc/ssl/certs/ca-bundle.crt";
+$caBundle = "/etc/ssl/certs/ca-certificates.crt" if !$caBundle && -f "/etc/ssl/certs/ca-certificates.crt";
+
+$curlCaFlag = defined $caBundle ? "--cacert $caBundle" : "";
+
 $bzip2 = "@bzip2@";
 $xz = "@xz@";
 $curl = "@curl@";
--- a/release.nix
+++ b/release.nix
@ -27,6 +27,7 @@ let
          [ curl bison flex perl libxml2 libxslt bzip2 xz
            dblatex (dblatex.tex or tetex) nukeReferences pkgconfig sqlite libsodium
            docbook5 docbook5_xsl
+            libseccomp
          ] ++ lib.optional (!lib.inNixShell) git;

        configureFlags = ''
@ -77,8 +78,7 @@ let

    build = pkgs.lib.genAttrs systems (system:

-      # FIXME: temporarily use a different branch for the Darwin build.
-      with import (if system == "x86_64-darwin" then <nixpkgs-darwin> else <nixpkgs>) { inherit system; };
+      with import <nixpkgs> { inherit system; };

      releaseTools.nixBuild {
        name = "nix";
@ -86,6 +86,7 @@ let

        buildInputs =
          [ curl perl bzip2 xz openssl pkgconfig sqlite boehmgc ]
+          ++ lib.optional stdenv.isLinux libseccomp
          ++ lib.optional stdenv.isLinux libsodium;

        configureFlags = ''
@ -113,7 +114,7 @@ let
    binaryTarball = pkgs.lib.genAttrs systems (system:

      # FIXME: temporarily use a different branch for the Darwin build.
-      with import (if system == "x86_64-darwin" then <nixpkgs-darwin> else <nixpkgs>) { inherit system; };
+      with import <nixpkgs> { inherit system; };

      let
        toplevel = builtins.getAttr system jobs.build;
@ -155,7 +156,7 @@ let
        src = tarball;

        buildInputs =
-          [ curl perl bzip2 openssl pkgconfig sqlite xz libsodium
+          [ curl perl bzip2 openssl pkgconfig sqlite xz libsodium libseccomp
            # These are for "make check" only:
            graphviz libxml2 libxslt
          ];
@ -180,8 +181,6 @@ let
      };


-    rpm_fedora18i386 = makeRPM_i686 (diskImageFuns: diskImageFuns.fedora18i386) [];
-    rpm_fedora18x86_64 = makeRPM_x86_64 (diskImageFunsFun: diskImageFunsFun.fedora18x86_64) [];
    rpm_fedora19i386 = makeRPM_i686 (diskImageFuns: diskImageFuns.fedora19i386) [];
    rpm_fedora19x86_64 = makeRPM_x86_64 (diskImageFunsFun: diskImageFunsFun.fedora19x86_64) [];
    rpm_fedora20i386 = makeRPM_i686 (diskImageFuns: diskImageFuns.fedora20i386) [];
@ -190,23 +189,19 @@ let
    rpm_fedora21x86_64 = makeRPM_x86_64 (diskImageFunsFun: diskImageFunsFun.fedora21x86_64) [ "libsodium-devel" ];


-    deb_debian7i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.debian7i386) [];
-    deb_debian7x86_64 = makeDeb_x86_64 (diskImageFunsFun: diskImageFunsFun.debian7x86_64) [];
-    deb_debian8i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.debian8i386) [ "libsodium-dev" ];
-    deb_debian8x86_64 = makeDeb_x86_64 (diskImageFunsFun: diskImageFunsFun.debian8x86_64) [ "libsodium-dev" ];
+    deb_debian8i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.debian8i386) [ "libsodium-dev" ] [ "libsodium13" ];
+    deb_debian8x86_64 = makeDeb_x86_64 (diskImageFunsFun: diskImageFunsFun.debian8x86_64) [ "libsodium-dev" ] [ "libsodium13" ];

-    deb_ubuntu1210i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1210i386) [];
-    deb_ubuntu1210x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1210x86_64) [];
-    deb_ubuntu1304i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1304i386) [];
-    deb_ubuntu1304x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1304x86_64) [];
-    deb_ubuntu1310i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1310i386) [];
-    deb_ubuntu1310x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1310x86_64) [];
-    deb_ubuntu1404i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1404i386) [];
-    deb_ubuntu1404x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1404x86_64) [];
-    deb_ubuntu1410i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1410i386) [];
-    deb_ubuntu1410x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1410x86_64) [];
-    deb_ubuntu1504i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1504i386) [ "libsodium-dev" ];
-    deb_ubuntu1504x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1504x86_64) [ "libsodium-dev" ];
+    deb_ubuntu1404i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1404i386) [] [];
+    deb_ubuntu1404x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1404x86_64) [] [];
+    deb_ubuntu1410i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1410i386) [] [];
+    deb_ubuntu1410x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1410x86_64) [] [];
+    deb_ubuntu1504i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1504i386) [ "libsodium-dev" ] [ "libsodium13" ];
+    deb_ubuntu1504x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1504x86_64) [ "libsodium-dev" ] [ "libsodium13" ];
+    deb_ubuntu1510i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1510i386) [ "libsodium-dev" ] [ "libsodium13"];
+    deb_ubuntu1510x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1510x86_64) [ "libsodium-dev" ] [ "libsodium13" ];
+    deb_ubuntu1604i386 = makeDeb_i686 (diskImageFuns: diskImageFuns.ubuntu1604i386) [ "libsodium-dev" ] [ "libsodium18" ];
+    deb_ubuntu1604x86_64 = makeDeb_x86_64 (diskImageFuns: diskImageFuns.ubuntu1604x86_64) [ "libsodium-dev" ] [ "libsodium18" ];


    # System tests.
@ -218,6 +213,11 @@ let
      nix = build.x86_64-linux; system = "x86_64-linux";
    });

+    tests.setuid = pkgs.lib.genAttrs (pkgs.lib.filter (pkgs.lib.hasSuffix "-linux") systems) (system:
+      import ./tests/setuid.nix rec {
+        nix = build.${system}; inherit system;
+      });
+
    tests.binaryTarball =
      with import <nixpkgs> { system = "x86_64-linux"; };
      vmTools.runInLinuxImage (runCommand "nix-binary-tarball-test"
@ -226,13 +226,15 @@ let
        ''
          useradd -m alice
          su - alice -c 'tar xf ${binaryTarball.x86_64-linux}/*.tar.*'
-          mount -t tmpfs none /nix # Provide a writable /nix.
+          mkdir /dest-nix
+          mount -o bind /dest-nix /nix # Provide a writable /nix.
          chown alice /nix
          su - alice -c '_NIX_INSTALLER_TEST=1 ./nix-*/install'
          su - alice -c 'nix-store --verify'
-          su - alice -c 'nix-store -qR ${build.x86_64-linux}'
+          su - alice -c 'PAGER= nix-store -qR ${build.x86_64-linux}'
          mkdir -p $out/nix-support
          touch $out/nix-support/hydra-build-products
+          umount /nix
        ''); # */

    tests.evalNixpkgs =
@ -272,8 +274,8 @@ let
          binaryTarball.x86_64-darwin
          #binaryTarball.x86_64-freebsd
          binaryTarball.x86_64-linux
-          deb_debian7i386
-          deb_debian7x86_64
+          deb_debian8i386
+          deb_debian8x86_64
          deb_ubuntu1404i386 # LTS
          deb_ubuntu1404x86_64 # LTS
          deb_ubuntu1504i386
@ -306,7 +308,7 @@ let
      src = jobs.tarball;
      diskImage = (diskImageFun vmTools.diskImageFuns)
        { extraPackages =
-            [ "perl-DBD-SQLite" "perl-devel" "sqlite" "sqlite-devel" "bzip2-devel" "emacs" "perl-WWW-Curl" "libcurl-devel" "openssl-devel" "xz-devel" ]
+            [ "perl-DBD-SQLite" "perl-devel" "sqlite" "sqlite-devel" "bzip2-devel" "emacs" "perl-WWW-Curl" "libcurl-devel" "openssl-devel" "xz-devel" "libseccomp-devel" ]
            ++ extraPackages; };
      memSize = 1024;
      meta.schedulingPriority = 50;
@ -318,7 +320,7 @@ let
  makeDeb_x86_64 = makeDeb "x86_64-linux";

  makeDeb =
-    system: diskImageFun: extraPackages:
+    system: diskImageFun: extraPackages: extraDebPackages:

    with import <nixpkgs> { inherit system; };

@ -327,14 +329,15 @@ let
      src = jobs.tarball;
      diskImage = (diskImageFun vmTools.diskImageFuns)
        { extraPackages =
-            [ "libdbd-sqlite3-perl" "libsqlite3-dev" "libbz2-dev" "libwww-curl-perl" "libcurl-dev" "libcurl3-nss" "libssl-dev" "liblzma-dev" ]
+            [ "libdbd-sqlite3-perl" "libsqlite3-dev" "libbz2-dev" "libwww-curl-perl" "libcurl-dev" "libcurl3-nss" "libssl-dev" "liblzma-dev" "libseccomp-dev" ]
            ++ extraPackages; };
      memSize = 1024;
      meta.schedulingPriority = 50;
+      postInstall = "make installcheck";
      configureFlags = "--sysconfdir=/etc";
      debRequires =
-        [ "curl" "libdbd-sqlite3-perl" "libsqlite3-0" "libbz2-1.0" "bzip2" "xz-utils" "libwww-curl-perl" "libssl1.0.0" "liblzma5" ]
-        ++ lib.optionals (lib.elem "libsodium-dev" extraPackages) [ "libsodium13" ] ;
+        [ "curl" "libdbd-sqlite3-perl" "libsqlite3-0" "libbz2-1.0" "bzip2" "xz-utils" "libwww-curl-perl" "libssl1.0.0" "liblzma5" "libseccomp2" ]
+        ++ extraDebPackages;
      debMaintainer = "Eelco Dolstra <eelco.dolstra@logicblox.com>";
      doInstallCheck = true;
    };
--- a/scripts/download-from-binary-cache.pl.in
+++ b/scripts/download-from-binary-cache.pl.in
@ -41,9 +41,6 @@ my $activeRequests = 0;
 my $curlIdCount = 1;
 my %requests;
 my %scheduled;
-my $caBundle = $ENV{"SSL_CERT_FILE"} // $ENV{"CURL_CA_BUNDLE"} // $ENV{"OPENSSL_X509_CERT_FILE"};
-$caBundle = "/etc/ssl/certs/ca-bundle.crt" if !$caBundle && -f "/etc/ssl/certs/ca-bundle.crt";
-$caBundle = "/etc/ssl/certs/ca-certificates.crt" if !$caBundle && -f "/etc/ssl/certs/ca-certificates.crt";

 my $userName = getpwuid($<) || $ENV{"USER"} or die "cannot figure out user name";

@ -65,6 +62,8 @@ my $curlConnectTimeout = int(
    $Nix::Config::config{"connect-timeout"} //
    $ENV{"NIX_CONNECT_TIMEOUT"} // 0);

+my $netrcFile = $Nix::Config::config{"netrc-file"} //
+    "$Nix::Config::confDir/netrc";

 sub addRequest {
    my ($storePath, $url, $head) = @_;
@ -79,7 +78,7 @@ sub addRequest {
    open (my $fh, ">", \$requests{$curlId}->{content});
    $curl->setopt(CURLOPT_WRITEDATA, $fh);
    $curl->setopt(CURLOPT_FOLLOWLOCATION, 1);
-    $curl->setopt(CURLOPT_CAINFO, $caBundle) if defined $caBundle;
+    $curl->setopt(CURLOPT_CAINFO, $Nix::Config::caBundle) if defined $Nix::Config::caBundle;

    unless (isTrue($Nix::Config::config{"verify-https-binary-caches"} // "1")) {
        $curl->setopt(CURLOPT_SSL_VERIFYPEER, 0);
@ -91,6 +90,8 @@ sub addRequest {
    $curl->setopt(CURLOPT_FAILONERROR, 1);
    $curl->setopt(CURLOPT_CONNECTTIMEOUT, $curlConnectTimeout);
    $curl->setopt(CURLOPT_TIMEOUT, 20 * 60);
+    $curl->setopt(CURLOPT_NETRC_FILE, $netrcFile);
+    $curl->setopt(CURLOPT_NETRC, CURL_NETRC_OPTIONAL);

    if ($activeRequests >= $maxParallelRequests) {
        $scheduled{$curlId} = 1;
@ -566,7 +567,7 @@ sub downloadBinary {
        die if $requireSignedBinaryCaches && !defined $info->{signedBy};
        print STDERR "\n*** Downloading ‘$url’ ", ($requireSignedBinaryCaches ? "(signed by ‘$info->{signedBy}’) " : ""), "to ‘$storePath’...\n";
        checkURL $url;
-        if (system("$Nix::Config::curl --fail --location --connect-timeout $curlConnectTimeout -A '$userAgent' '$url' $decompressor | $Nix::Config::binDir/nix-store --restore $destPath") != 0) {
+        if (system("$Nix::Config::curl --fail --location --netrc-file $netrcFile --netrc-optional --connect-timeout $curlConnectTimeout -A '$userAgent' $Nix::Config::curlCaFlag '$url' $decompressor | $Nix::Config::binDir/nix-store --restore $destPath") != 0) {
            warn "download of ‘$url’ failed" . ($! ? ": $!" : "") . "\n";
            next;
        }
--- a/scripts/download-using-manifests.pl.in
+++ b/scripts/download-using-manifests.pl.in
@ -17,7 +17,7 @@ my $logFile = "$Nix::Config::logDir/downloads";
 # estimating the expected download size.
 my $fast = 1;

-my $curl = "$Nix::Config::curl --fail --location";
+my $curl = "$Nix::Config::curl $Nix::Config::curlCaFlag --fail --location";


 # Open the manifest cache and update it if necessary.
--- a/scripts/install-nix-from-closure.sh
+++ b/scripts/install-nix-from-closure.sh
@ -12,6 +12,12 @@ if ! [ -e $self/.reginfo ]; then
    exit 1
 fi

+# macOS support for 10.10 or higher
+if [[ "$(uname -s)" = "Darwin" && $(($(sw_vers -productVersion | cut -d '.' -f 2))) -lt 10 ]]; then
+    echo "$0: macOS $(sw_vers -productVersion) is not supported, upgrade to 10.10 or higher"
+    exit 1
+fi
+
 if [ -z "$USER" ]; then
    echo "$0: \$USER is not set" >&2
    exit 1
@ -73,9 +79,9 @@ if ! $nix/bin/nix-env -i "$nix"; then
 fi

 # Install an SSL certificate bundle.
-if [ -z "$SSL_CERT_FILE" -o ! -f "$SSL_CERT_FILE" ]; then
+if [ -z "$NIX_SSL_CERT_FILE" -o ! -f "$NIX_SSL_CERT_FILE" ]; then
    $nix/bin/nix-env -i "$cacert"
-    export SSL_CERT_FILE="$HOME/.nix-profile/etc/ssl/certs/ca-bundle.crt"
+    export NIX_SSL_CERT_FILE="$HOME/.nix-profile/etc/ssl/certs/ca-bundle.crt"
 fi

 # Subscribe the user to the Nixpkgs channel and fetch it.
@ -92,7 +98,7 @@ p=$NIX_LINK/etc/profile.d/nix.sh
 added=
 for i in .bash_profile .bash_login .profile; do
    fn="$HOME/$i"
-    if [ -e "$fn" ]; then
+    if [ -w "$fn" ]; then
        if ! grep -q "$p" "$fn"; then
            echo "modifying $fn..." >&2
            echo "if [ -e $p ]; then . $p; fi # added by Nix installer" >> $fn
--- a/scripts/nix-build.in
+++ b/scripts/nix-build.in
@ -17,7 +17,8 @@ my $runEnv = $0 =~ /nix-shell$/;
 my $pure = 0;
 my $fromArgs = 0;
 my $packages = 0;
-my $interactive = 1;
+# Same condition as bash uses for interactive shells
+my $interactive = -t STDIN && -t STDERR;

 my @instArgs = ();
 my @buildArgs = ();
@ -57,7 +58,7 @@ if ($runEnv && defined $ARGV[0] && $ARGV[0] !~ /nix-shell/) {
            while (<SCRIPT>) {
                chomp;
                if (/^\#\!\s*nix-shell (.*)$/) {
-                    push @ARGV, shellwords(/ /, $1);
+                    push @ARGV, shellwords($1);
                }
            }
        }
@ -288,7 +289,6 @@ foreach my $expr (@exprs) {
        writeFile(
            $rcfile,
            "rm -rf '$tmpDir'; " .
-            'unset BASH_ENV; ' .
            '[ -n "$PS1" ] && [ -e ~/.bashrc ] && source ~/.bashrc; ' .
            ($pure ? '' : 'p=$PATH; ' ) .
            'dontAddDisableDepTrack=1; ' .
@ -302,7 +302,6 @@ foreach my $expr (@exprs) {
            'shopt -u nullglob; ' .
            'unset TZ; ' . (defined $ENV{'TZ'} ? "export TZ='${ENV{'TZ'}}'; " : '') .
            $envCommand);
-        $ENV{BASH_ENV} = $rcfile;
        my @args = ($ENV{NIX_BUILD_SHELL} // "bash");
        push @args, "--rcfile" if $interactive;
        push @args, $rcfile;
--- a/scripts/nix-channel.in
+++ b/scripts/nix-channel.in
@ -32,6 +32,9 @@ mkpath(dirname $profile, 0, 0755);

 my %channels;

+my $netrcFile = $Nix::Config::config{"netrc-file"} //
+    "$Nix::Config::confDir/netrc";
+

 # Reads the list of channels.
 sub readChannels {
@ -104,7 +107,7 @@ sub update {
        # definition from a consistent location if the redirect changes mid-download.
        my $tmpdir = tempdir( CLEANUP => 1 );
        my $filename;
-        ($url, $filename) = `cd $tmpdir && $Nix::Config::curl --silent --write-out '%{url_effective}\n%{filename_effective}' -L '$url' -O`;
+        ($url, $filename) = `cd $tmpdir && $Nix::Config::curl $Nix::Config::curlCaFlag --netrc-file $netrcFile --netrc-optional --silent --write-out '%{url_effective}\n%{filename_effective}' -L '$url' -O`;
        chomp $url;
        die "$0: unable to check ‘$url’\n" if $? != 0;

@ -131,7 +134,7 @@ sub update {
        my $extraAttrs = "";
        if ($ret != 0) {
            # Check if the channel advertises a binary cache.
-            my $binaryCacheURL = `$Nix::Config::curl --silent '$url'/binary-cache-url`;
+            my $binaryCacheURL = `$Nix::Config::curl $Nix::Config::curlCaFlag --netrc-file $netrcFile --netrc-optional  --silent '$url'/binary-cache-url`;
            my $getManifest = ($Nix::Config::config{"force-manifest"} // "false") eq "true";
            if ($? == 0 && $binaryCacheURL ne "") {
                $extraAttrs .= "binaryCacheURL = \"$binaryCacheURL\"; ";
@ -151,7 +154,7 @@ sub update {

            # Download the channel tarball.
            my $fullURL = "$url/nixexprs.tar.xz";
-            system("$Nix::Config::curl --fail --silent --head '$fullURL' > /dev/null") == 0 or
+            system("$Nix::Config::curl $Nix::Config::curlCaFlag --netrc-file $netrcFile --netrc-optional  --fail --silent --head '$fullURL' > /dev/null") == 0 or
                $fullURL = "$url/nixexprs.tar.bz2";
            print STDERR "downloading Nix expressions from ‘$fullURL’...\n";
            (my $hash, $path) = `PRINT_PATH=1 QUIET=1 $Nix::Config::binDir/nix-prefetch-url '$fullURL'`;
--- a/scripts/nix-profile.sh.in
+++ b/scripts/nix-profile.sh.in
@ -20,16 +20,18 @@ if [ -n "$HOME" ]; then
    # channel.
    export NIX_PATH=${NIX_PATH:+$NIX_PATH:}nixpkgs=$HOME/.nix-defexpr/channels/nixpkgs

-    # Set $SSL_CERT_FILE so that Nixpkgs applications like curl work.
+    # Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
    if [ -e /etc/ssl/certs/ca-certificates.crt ]; then # NixOS, Ubuntu, Debian, Gentoo, Arch
-        export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
+        export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
+    elif [ -e /etc/ssl/ca-bundle.pem ]; then # openSUSE Tumbleweed
+        export NIX_SSL_CERT_FILE=/etc/ssl/ca-bundle.pem
    elif [ -e /etc/ssl/certs/ca-bundle.crt ]; then # Old NixOS
-        export SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt
+        export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt
    elif [ -e /etc/pki/tls/certs/ca-bundle.crt ]; then # Fedora, CentOS
-        export SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt
+        export NIX_SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt
    elif [ -e "$NIX_LINK/etc/ssl/certs/ca-bundle.crt" ]; then # fall back to cacert in Nix profile
-        export SSL_CERT_FILE="$NIX_LINK/etc/ssl/certs/ca-bundle.crt"
+        export NIX_SSL_CERT_FILE="$NIX_LINK/etc/ssl/certs/ca-bundle.crt"
    elif [ -e "$NIX_LINK/etc/ca-bundle.crt" ]; then # old cacert in Nix profile
-        export SSL_CERT_FILE="$NIX_LINK/etc/ca-bundle.crt"
+        export NIX_SSL_CERT_FILE="$NIX_LINK/etc/ca-bundle.crt"
    fi
 fi
--- a/scripts/nix-pull.in
+++ b/scripts/nix-pull.in
@ -8,7 +8,8 @@ use Nix::Manifest;
 binmode STDERR, ":encoding(utf8)";

 my $manifestDir = $Nix::Config::manifestDir;
-
+my $netrcFile = $Nix::Config::config{"netrc-file"} //
+    "$Nix::Config::confDir/netrc";

 # Prevent access problems in shared-stored installations.
 umask 0022;
@ -51,7 +52,7 @@ sub processURL {
    my $origUrl = $ENV{'NIX_ORIG_URL'} || $url;

    # First see if a bzipped manifest is available.
-    if (system("$Nix::Config::curl --fail --silent --location --head '$url'.bz2 > /dev/null") == 0) {
+    if (system("$Nix::Config::curl $Nix::Config::curlCaFlag --netrc-file $netrcFile --netrc-optional --fail --silent --location --head '$url'.bz2 > /dev/null") == 0) {
        print "fetching list of Nix archives at ‘$url.bz2’...\n";
        $manifest = downloadFile "$url.bz2";
    }
--- a/src/libexpr/eval.cc
+++ b/src/libexpr/eval.cc
@ -273,7 +273,7 @@ EvalState::EvalState(const Strings & _searchPath)

    /* Initialise the Nix expression search path. */
    Strings paths = parseNixPath(getEnv("NIX_PATH", ""));
-    for (auto & i : _searchPath) addToSearchPath(i, true);
+    for (auto & i : _searchPath) addToSearchPath(i);
    for (auto & i : paths) addToSearchPath(i);
    addToSearchPath("nix=" + settings.nixDataDir + "/nix/corepkgs");

@ -296,11 +296,15 @@ Path EvalState::checkSourcePath(const Path & path_)
    if (!restricted) return path_;

    /* Resolve symlinks. */
+    debug(format("checking access to ‘%s’") % path_);
    Path path = canonPath(path_, true);

-    for (auto & i : searchPath)
-        if (path == i.second || isInDir(path, i.second))
+    for (auto & i : searchPath) {
+        auto r = resolveSearchPathElem(i);
+        if (!r.first) continue;
+        if (path == r.second || isInDir(path, r.second))
            return path;
+    }

    /* To support import-from-derivation, allow access to anything in
       the store. FIXME: only allow access to paths that have been
@ -946,11 +950,18 @@ void EvalState::callFunction(Value & fun, Value & arg, Value & v, const Pos & po
    if (fun.type == tAttrs) {
      auto found = fun.attrs->find(sFunctor);
      if (found != fun.attrs->end()) {
+        /* fun may be allocated on the stack of the calling function,
+         * but for functors we may keep a reference, so heap-allocate
+         * a copy and use that instead.
+         */
+        auto & fun2 = *allocValue();
+        fun2 = fun;
+        /* !!! Should we use the attr pos here? */
        forceValue(*found->value, pos);
-        Value * v2 = allocValue();
-        callFunction(*found->value, fun, *v2, pos);
-        forceValue(*v2, pos);
-        return callFunction(*v2, arg, v, pos);
+        Value v2;
+        callFunction(*found->value, fun2, v2, pos);
+        forceValue(v2, pos);
+        return callFunction(v2, arg, v, pos);
      }
    }

--- a/src/libexpr/eval.hh
+++ b/src/libexpr/eval.hh
@ -55,7 +55,8 @@ typedef std::map<Path, Path> SrcToStore;
 std::ostream & operator << (std::ostream & str, const Value & v);


-typedef list<std::pair<string, Path> > SearchPath;
+typedef std::pair<std::string, std::string> SearchPathElem;
+typedef std::list<SearchPathElem> SearchPath;


 /* Initialise the Boehm GC, if applicable. */
@ -95,12 +96,14 @@ private:

    SearchPath searchPath;

+    std::map<std::string, std::pair<bool, std::string>> searchPathResolved;
+
 public:

    EvalState(const Strings & _searchPath);
    ~EvalState();

-    void addToSearchPath(const string & s, bool warn = false);
+    void addToSearchPath(const string & s);

    Path checkSourcePath(const Path & path);

@ -122,6 +125,9 @@ public:
    Path findFile(const string & path);
    Path findFile(SearchPath & searchPath, const string & path, const Pos & pos = noPos);

+    /* If the specified search path element is a URI, download it. */
+    std::pair<bool, std::string> resolveSearchPathElem(const SearchPathElem & elem);
+
    /* Evaluate an expression to normal form, storing the result in
       value `v'. */
    void eval(Expr * e, Value & v);
--- a/src/libexpr/get-drvs.cc
+++ b/src/libexpr/get-drvs.cc
@ -30,7 +30,7 @@ string DrvInfo::queryOutPath()
 }


-DrvInfo::Outputs DrvInfo::queryOutputs()
+DrvInfo::Outputs DrvInfo::queryOutputs(bool onlyOutputsToInstall)
 {
    if (outputs.empty()) {
        /* Get the ‘outputs’ list. */
@ -55,7 +55,23 @@ DrvInfo::Outputs DrvInfo::queryOutputs()
        } else
            outputs["out"] = queryOutPath();
    }
-    return outputs;
+    if (!onlyOutputsToInstall || !attrs)
+        return outputs;
+
+    /* Check for `meta.outputsToInstall` and return `outputs` reduced to that. */
+    const Value * outTI = queryMeta("outputsToInstall");
+    if (!outTI) return outputs;
+    const auto errMsg = Error("this derivation has bad ‘meta.outputsToInstall’");
+        /* ^ this shows during `nix-env -i` right under the bad derivation */
+    if (!outTI->isList()) throw errMsg;
+    Outputs result;
+    for (auto i = outTI->listElems(); i != outTI->listElems() + outTI->listSize(); ++i) {
+        if ((*i)->type != tString) throw errMsg;
+        auto out = outputs.find((*i)->string.s);
+        if (out == outputs.end()) throw errMsg;
+        result.insert(*out);
+    }
+    return result;
 }


@ -177,8 +193,8 @@ typedef set<Bindings *> Done;


 /* Evaluate value `v'.  If it evaluates to a set of type `derivation',
-   then put information about it in `drvs' (unless it's already in
-   `doneExprs').  The result boolean indicates whether it makes sense
+   then put information about it in `drvs' (unless it's already in `done').
+   The result boolean indicates whether it makes sense
   for the caller to recursively search for derivations in `v'. */
 static bool getDerivation(EvalState & state, Value & v,
    const string & attrPath, DrvInfos & drvs, Done & done,
--- a/src/libexpr/get-drvs.hh
+++ b/src/libexpr/get-drvs.hh
@ -42,7 +42,8 @@ public:
    string queryDrvPath();
    string queryOutPath();
    string queryOutputName();
-    Outputs queryOutputs();
+    /** Return the list of outputs. The "outputs to install" are determined by `mesa.outputsToInstall`. */
+    Outputs queryOutputs(bool onlyOutputsToInstall = false);

    StringSet queryMetaNames();
    Value * queryMeta(const string & name);
--- a/src/libexpr/json-to-value.cc
+++ b/src/libexpr/json-to-value.cc
@ -14,7 +14,7 @@ static void skipWhitespace(const char * & s)

 #if HAVE_BOEHMGC
 typedef std::vector<Value *, gc_allocator<Value *> > ValueVector;
-typedef std::map<Symbol, Value *, std::less<Symbol>, gc_allocator<Value *> > ValueMap;
+typedef std::map<Symbol, Value *, std::less<Symbol>, gc_allocator<std::pair<const Symbol, Value *> > > ValueMap;
 #else
 typedef std::vector<Value *> ValueVector;
 typedef std::map<Symbol, Value *> ValueMap;
--- a/src/libexpr/lexer.l
+++ b/src/libexpr/lexer.l
@ -8,6 +8,7 @@

 %x STRING
 %x IND_STRING
+%x INSIDE_DOLLAR_CURLY


 %{
@ -93,6 +94,8 @@ URI         [a-zA-Z][a-zA-Z0-9\+\-\.]*\:[a-zA-Z0-9\%\/\?\:\@\&\=\+\$\,\-\_\.\!\~

 %%

+<INITIAL,INSIDE_DOLLAR_CURLY>{
+

 if          { return IF; }
 then        { return THEN; }
@ -124,29 +127,42 @@ or          { return OR_KW; }
              return INT;
            }

-\$\{        { PUSH_STATE(INITIAL); return DOLLAR_CURLY; }
-\{          { PUSH_STATE(INITIAL); return '{'; }
-\}          { POP_STATE(); return '}'; }
+\$\{        { PUSH_STATE(INSIDE_DOLLAR_CURLY); return DOLLAR_CURLY; }
+}

-\"          { PUSH_STATE(STRING); return '"'; }
+\}                           { return '}'; }
+<INSIDE_DOLLAR_CURLY>\}      { POP_STATE(); return '}'; }
+\{                           { return '{'; }
+<INSIDE_DOLLAR_CURLY>\{      { PUSH_STATE(INSIDE_DOLLAR_CURLY); return '{'; }
+
+<INITIAL,INSIDE_DOLLAR_CURLY>\" {
+                PUSH_STATE(STRING); return '"';
+              }
 <STRING>([^\$\"\\]|\$[^\{\"\\]|\\.|\$\\.)*\$/\" |
 <STRING>([^\$\"\\]|\$[^\{\"\\]|\\.|\$\\.)+ {
-              /* It is impossible to match strings ending with '$' with one
-                 regex because trailing contexts are only valid at the end
-                 of a rule. (A sane but undocumented limitation.) */
-              yylval->e = unescapeStr(data->symbols, yytext);
-              return STR;
-            }
-<STRING>\$\{  { PUSH_STATE(INITIAL); return DOLLAR_CURLY; }
-<STRING>\"  { POP_STATE(); return '"'; }
-<STRING>.   return yytext[0]; /* just in case: shouldn't be reached */
+                /* It is impossible to match strings ending with '$' with one
+                   regex because trailing contexts are only valid at the end
+                   of a rule. (A sane but undocumented limitation.) */
+                yylval->e = unescapeStr(data->symbols, yytext);
+                return STR;
+              }
+<STRING>\$\{  { PUSH_STATE(INSIDE_DOLLAR_CURLY); return DOLLAR_CURLY; }
+<STRING>\"    { POP_STATE(); return '"'; }
+<STRING>\$|\\|\$\\ {
+                /* This can only occur when we reach EOF, otherwise the above
+                   (...|\$[^\{\"\\]|\\.|\$\\.)+ would have triggered.
+                   This is technically invalid, but we leave the problem to the
+                   parser who fails with exact location. */
+                return STR;
+              }

-\'\'(\ *\n)?     { PUSH_STATE(IND_STRING); return IND_STRING_OPEN; }
+<INITIAL,INSIDE_DOLLAR_CURLY>\'\'(\ *\n)?     { PUSH_STATE(IND_STRING); return IND_STRING_OPEN; }
 <IND_STRING>([^\$\']|\$[^\{\']|\'[^\'\$])+ {
                   yylval->e = new ExprIndStr(yytext);
                   return IND_STR;
                 }
-<IND_STRING>\'\'\$ {
+<IND_STRING>\'\'\$ |
+<IND_STRING>\$   {
                   yylval->e = new ExprIndStr("$");
                   return IND_STR;
                 }
@ -158,13 +174,14 @@ or          { return OR_KW; }
                   yylval->e = unescapeStr(data->symbols, yytext + 2);
                   return IND_STR;
                 }
-<IND_STRING>\$\{ { PUSH_STATE(INITIAL); return DOLLAR_CURLY; }
+<IND_STRING>\$\{ { PUSH_STATE(INSIDE_DOLLAR_CURLY); return DOLLAR_CURLY; }
 <IND_STRING>\'\' { POP_STATE(); return IND_STRING_CLOSE; }
 <IND_STRING>\'   {
                   yylval->e = new ExprIndStr("'");
                   return IND_STR;
                 }
-<IND_STRING>.    return yytext[0]; /* just in case: shouldn't be reached */
+
+<INITIAL,INSIDE_DOLLAR_CURLY>{

 {PATH}      { yylval->path = strdup(yytext); return PATH; }
 {HPATH}     { yylval->path = strdup(yytext); return HPATH; }
@ -177,6 +194,7 @@ or          { return OR_KW; }

 .           return yytext[0];

+}

 %%

--- a/src/libexpr/parser.y
+++ b/src/libexpr/parser.y
@ -590,7 +590,7 @@ Expr * EvalState::parseExprFromString(const string & s, const Path & basePath)
 }


-void EvalState::addToSearchPath(const string & s, bool warn)
+void EvalState::addToSearchPath(const string & s)
 {
    size_t pos = s.find('=');
    string prefix;
@ -602,16 +602,7 @@ void EvalState::addToSearchPath(const string & s, bool warn)
        path = string(s, pos + 1);
    }

-    if (isUri(path))
-        path = downloadFileCached(path, true);
-
-    path = absPath(path);
-    if (pathExists(path)) {
-        debug(format("adding path ‘%1%’ to the search path") % path);
-        /* Resolve symlinks in the path to support restricted mode. */
-        searchPath.push_back(std::pair<string, Path>(prefix, canonPath(path, true)));
-    } else if (warn)
-        printMsg(lvlError, format("warning: Nix search path entry ‘%1%’ does not exist, ignoring") % path);
+    searchPath.emplace_back(prefix, path);
 }


@ -624,17 +615,19 @@ Path EvalState::findFile(const string & path)
 Path EvalState::findFile(SearchPath & searchPath, const string & path, const Pos & pos)
 {
    for (auto & i : searchPath) {
-        assert(!isUri(i.second));
-        Path res;
+        std::string suffix;
        if (i.first.empty())
-            res = i.second + "/" + path;
+            suffix = "/" + path;
        else {
-            if (path.compare(0, i.first.size(), i.first) != 0 ||
-                (path.size() > i.first.size() && path[i.first.size()] != '/'))
+            auto s = i.first.size();
+            if (path.compare(0, s, i.first) != 0 ||
+                (path.size() > s && path[s] != '/'))
                continue;
-            res = i.second +
-                (path.size() == i.first.size() ? "" : "/" + string(path, i.first.size()));
+            suffix = path.size() == s ? "" : "/" + string(path, s);
        }
+        auto r = resolveSearchPathElem(i);
+        if (!r.first) continue;
+        Path res = r.second + suffix;
        if (pathExists(res)) return canonPath(res);
    }
    format f = format(
@ -645,4 +638,35 @@ Path EvalState::findFile(SearchPath & searchPath, const string & path, const Pos
 }


+std::pair<bool, std::string> EvalState::resolveSearchPathElem(const SearchPathElem & elem)
+{
+    auto i = searchPathResolved.find(elem.second);
+    if (i != searchPathResolved.end()) return i->second;
+
+    std::pair<bool, std::string> res;
+
+    if (isUri(elem.second)) {
+        try {
+            res = { true, downloadFileCached(elem.second, true) };
+        } catch (DownloadError & e) {
+            printMsg(lvlError, format("warning: Nix search path entry ‘%1%’ cannot be downloaded, ignoring") % elem.second);
+            res = { false, "" };
+        }
+    } else {
+        auto path = absPath(elem.second);
+        if (pathExists(path))
+            res = { true, path };
+        else {
+            printMsg(lvlError, format("warning: Nix search path entry ‘%1%’ does not exist, ignoring") % elem.second);
+            res = { false, "" };
+        }
+    }
+
+    debug(format("resolved search path element ‘%s’ to ‘%s’") % elem.second % res.second);
+
+    searchPathResolved[elem.second] = res;
+    return res;
+}
+
+
 }
--- a/src/libexpr/primops.cc
+++ b/src/libexpr/primops.cc
@ -176,6 +176,45 @@ static void prim_importNative(EvalState & state, const Pos & pos, Value * * args
 }


+/* Execute a program and parse its output */
+static void prim_exec(EvalState & state, const Pos & pos, Value * * args, Value & v)
+{
+    state.forceList(*args[0], pos);
+    auto elems = args[0]->listElems();
+    auto count = args[0]->listSize();
+    if (count == 0) {
+        throw EvalError(format("at least one argument to 'exec' required, at %1%") % pos);
+    }
+    PathSet context;
+    auto program = state.coerceToString(pos, *elems[0], context, false, false);
+    Strings commandArgs;
+    for (unsigned int i = 1; i < args[0]->listSize(); ++i) {
+        commandArgs.emplace_back(state.coerceToString(pos, *elems[i], context, false, false));
+    }
+    try {
+        realiseContext(context);
+    } catch (InvalidPathError & e) {
+        throw EvalError(format("cannot execute ‘%1%’, since path ‘%2%’ is not valid, at %3%")
+            % program % e.path % pos);
+    }
+
+    auto output = runProgram(program, true, commandArgs);
+    Expr * parsed;
+    try {
+        parsed = state.parseExprFromString(output, pos.file);
+    } catch (Error & e) {
+        e.addPrefix(format("While parsing the output from ‘%1%’, at %2%\n") % program % pos);
+        throw;
+    }
+    try {
+        state.eval(parsed, v);
+    } catch (Error & e) {
+        e.addPrefix(format("While evaluating the output from ‘%1%’, at %2%\n") % program % pos);
+        throw;
+    }
+}
+
+
 /* Return a string representing the type of the expression. */
 static void prim_typeOf(EvalState & state, const Pos & pos, Value * * args, Value & v)
 {
@ -765,7 +804,6 @@ static void prim_findFile(EvalState & state, const Pos & pos, Value * * args, Va

    SearchPath searchPath;

-    PathSet context;
    for (unsigned int n = 0; n < args[0]->listSize(); ++n) {
        Value & v2(*args[0]->listElems()[n]);
        state.forceAttrs(v2, pos);
@ -778,21 +816,23 @@ static void prim_findFile(EvalState & state, const Pos & pos, Value * * args, Va
        i = v2.attrs->find(state.symbols.create("path"));
        if (i == v2.attrs->end())
            throw EvalError(format("attribute ‘path’ missing, at %1%") % pos);
-        string path = state.coerceToPath(pos, *i->value, context);

-        searchPath.push_back(std::pair<string, Path>(prefix, state.checkSourcePath(path)));
+        PathSet context;
+        string path = state.coerceToString(pos, *i->value, context, false, false);
+
+        try {
+            realiseContext(context);
+        } catch (InvalidPathError & e) {
+            throw EvalError(format("cannot find ‘%1%’, since path ‘%2%’ is not valid, at %3%")
+                % path % e.path % pos);
+        }
+
+        searchPath.emplace_back(prefix, path);
    }

    string path = state.forceStringNoCtx(*args[1], pos);

-    try {
-        realiseContext(context);
-    } catch (InvalidPathError & e) {
-        throw EvalError(format("cannot find ‘%1%’, since path ‘%2%’ is not valid, at %3%")
-            % path % e.path % pos);
-    }
-
-    mkPath(v, state.findFile(searchPath, path, pos).c_str());
+    mkPath(v, state.checkSourcePath(state.findFile(searchPath, path, pos)).c_str());
 }

 /* Read a directory (without . or ..) */
@ -1657,6 +1697,7 @@ void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
    if (state.restricted) throw Error(format("‘%1%’ is not allowed in restricted mode") % who);

    string url;
+    string name;

    state.forceValue(*args[0]);

@ -1665,9 +1706,11 @@ void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
        state.forceAttrs(*args[0], pos);

        for (auto & attr : *args[0]->attrs) {
-            string name(attr.name);
-            if (name == "url")
+            string n(attr.name);
+            if (n == "url")
                url = state.forceStringNoCtx(*attr.value, *attr.pos);
+            else if (n == "name")
+                name = state.forceStringNoCtx(*attr.value, *attr.pos);
            else
                throw EvalError(format("unsupported argument ‘%1%’ to ‘%2%’, at %3%") % attr.name % who % attr.pos);
        }
@ -1678,7 +1721,7 @@ void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
    } else
        url = state.forceStringNoCtx(*args[0], pos);

-    Path res = downloadFileCached(url, unpack);
+    Path res = downloadFileCached(url, unpack, name);
    mkString(v, res, PathSet({res}));
 }

@ -1746,8 +1789,10 @@ void EvalState::createBaseEnv()
    mkApp(v, *baseEnv.values[baseEnvDispl - 1], *v2);
    forceValue(v);
    addConstant("import", v);
-    if (settings.enableImportNative)
+    if (settings.enableNativeCode) {
        addPrimOp("__importNative", 2, prim_importNative);
+        addPrimOp("__exec", 1, prim_exec);
+    }
    addPrimOp("__typeOf", 1, prim_typeOf);
    addPrimOp("isNull", 1, prim_isNull);
    addPrimOp("__isFunction", 1, prim_isFunction);
--- a/src/libmain/shared.cc
+++ b/src/libmain/shared.cc
@ -162,6 +162,14 @@ void initNix()

    if (char *pack = getenv("_NIX_OPTIONS"))
        settings.unpack(pack);
+
+    /* On macOS, don't use the per-session TMPDIR (as set e.g. by
+       sshd). This breaks build users because they don't have access
+       to the TMPDIR, in particular in ‘nix-store --serve’. */
+#if __APPLE__
+    if (getuid() == 0 && hasPrefix(getEnv("TMPDIR"), "/var/folders/"))
+        unsetenv("TMPDIR");
+#endif
 }


--- a/src/libstore/build.cc
+++ b/src/libstore/build.cc
@ -9,6 +9,7 @@
 #include "archive.hh"
 #include "affinity.hh"
 #include "builtins.hh"
+#include "finally.hh"

 #include <algorithm>
 #include <iostream>
@ -54,6 +55,7 @@
 #include <sys/mount.h>
 #include <sys/syscall.h>
 #include <linux/fs.h>
+#include <seccomp.h>
 #define pivot_root(new_root, put_old) (syscall(SYS_pivot_root, new_root, put_old))
 #endif

@ -761,7 +763,14 @@ private:
    GoalState state;

    /* Stuff we need to pass to initChild(). */
-    typedef map<Path, Path> DirsInChroot; // maps target path to source path
+    struct ChrootPath {
+        Path source;
+        bool optional;
+        ChrootPath(Path source = "", bool optional = false)
+            : source(source), optional(optional)
+        { }
+    };
+    typedef map<Path, ChrootPath> DirsInChroot; // maps target path to source path
    DirsInChroot dirsInChroot;
    typedef map<string, string> Environment;
    Environment env;
@ -1063,8 +1072,10 @@ void DerivationGoal::outputsSubstituted()
 {
    trace("all outputs substituted (maybe)");

-    if (nrFailed > 0 && nrFailed > nrNoSubstituters + nrIncompleteClosure && !settings.tryFallback)
-        throw Error(format("some substitutes for the outputs of derivation ‘%1%’ failed (usually happens due to networking issues); try ‘--fallback’ to build derivation from source ") % drvPath);
+    if (nrFailed > 0 && nrFailed > nrNoSubstituters + nrIncompleteClosure && !settings.tryFallback) {
+        done(BuildResult::TransientFailure, (format("some substitutes for the outputs of derivation ‘%1%’ failed (usually happens due to networking issues); try ‘--fallback’ to build derivation from source ") % drvPath).str());
+        return;
+    }

    /*  If the substitutes form an incomplete closure, then we should
        build the dependencies of this derivation, but after that, we
@ -1884,7 +1895,13 @@ void DerivationGoal::startBuilder()
    /* If `build-users-group' is not empty, then we have to build as
       one of the members of that group. */
    if (settings.buildUsersGroup != "") {
+#if defined(__linux__) || defined(__APPLE__)
        buildUser.acquire();
+#else
+        /* Don't know how to block the creation of setuid/setgid
+           binaries on this platform. */
+        throw Error("build users are not supported on this platform for security reasons");
+#endif
        assert(buildUser.getUID() != 0);
        assert(buildUser.getGID() != 0);

@ -1922,20 +1939,30 @@ void DerivationGoal::startBuilder()

        dirsInChroot.clear();

-        for (auto & i : dirs) {
+        for (auto i : dirs) {
+            if (i.empty()) continue;
+            bool optional = false;
+            if (i[i.size() - 1] == '?') {
+                optional = true;
+                i.pop_back();
+            }
            size_t p = i.find('=');
            if (p == string::npos)
-                dirsInChroot[i] = i;
+                dirsInChroot[i] = {i, optional};
            else
-                dirsInChroot[string(i, 0, p)] = string(i, p + 1);
+                dirsInChroot[string(i, 0, p)] = {string(i, p + 1), optional};
        }
        dirsInChroot[tmpDirInSandbox] = tmpDir;

        /* Add the closure of store paths to the chroot. */
        PathSet closure;
        for (auto & i : dirsInChroot)
-            if (isInStore(i.second))
-                computeFSClosure(worker.store, toStorePath(i.second), closure);
+            try {
+                if (isInStore(i.second.source))
+                    computeFSClosure(worker.store, toStorePath(i.second.source), closure);
+            } catch (Error & e) {
+                throw Error(format("while processing ‘build-sandbox-paths’: %s") % e.what());
+            }
        for (auto & i : closure)
            dirsInChroot[i] = i;

@ -2094,7 +2121,7 @@ void DerivationGoal::startBuilder()
                }
    }

-    if (settings.preBuildHook != "") {
+    if (useChroot && settings.preBuildHook != "" && dynamic_cast<Derivation *>(drv.get())) {
        printMsg(lvlChatty, format("executing pre-build hook ‘%1%’")
            % settings.preBuildHook);
        auto args = useChroot ? Strings({drvPath, chrootRootDir}) :
@ -2232,6 +2259,51 @@ void DerivationGoal::startBuilder()
 }


+void setupSeccomp()
+{
+#if __linux__
+    scmp_filter_ctx ctx;
+
+    if (!(ctx = seccomp_init(SCMP_ACT_ALLOW)))
+        throw SysError("unable to initialize seccomp mode 2");
+
+    Finally cleanup([&]() {
+        seccomp_release(ctx);
+    });
+
+    if (settings.thisSystem == "x86_64-linux" &&
+        seccomp_arch_add(ctx, SCMP_ARCH_X86) != 0)
+        throw SysError("unable to add 32-bit seccomp architecture");
+
+    /* Prevent builders from creating setuid/setgid binaries. */
+    for (int perm : { S_ISUID, S_ISGID }) {
+        if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(chmod), 1,
+                SCMP_A1(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
+            throw SysError("unable to add seccomp rule");
+
+        if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(fchmod), 1,
+                SCMP_A1(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
+            throw SysError("unable to add seccomp rule");
+
+        if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(fchmodat), 1,
+                SCMP_A2(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
+            throw SysError("unable to add seccomp rule");
+    }
+
+    /* Prevent builders from creating EAs or ACLs. Not all filesystems
+       support these, and they're not allowed in the Nix store because
+       they're not representable in the NAR serialisation. */
+    if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(ENOTSUP), SCMP_SYS(setxattr), 0) != 0 ||
+        seccomp_rule_add(ctx, SCMP_ACT_ERRNO(ENOTSUP), SCMP_SYS(lsetxattr), 0) != 0 ||
+        seccomp_rule_add(ctx, SCMP_ACT_ERRNO(ENOTSUP), SCMP_SYS(fsetxattr), 0) != 0)
+        throw SysError("unable to add seccomp rule");
+
+    if (seccomp_load(ctx) != 0)
+        throw SysError("unable to load seccomp BPF program");
+#endif
+}
+
+
 void DerivationGoal::runChild()
 {
    /* Warning: in the child we should absolutely not make any SQLite
@ -2243,6 +2315,12 @@ void DerivationGoal::runChild()

        commonChildInit(builderOut);

+        try {
+            setupSeccomp();
+        } catch (...) {
+            if (buildUser.enabled()) throw;
+        }
+
 #if __linux__
        if (useChroot) {

@ -2326,12 +2404,16 @@ void DerivationGoal::runChild()
               environment. */
            for (auto & i : dirsInChroot) {
                struct stat st;
-                Path source = i.second;
+                Path source = i.second.source;
                Path target = chrootRootDir + i.first;
                if (source == "/proc") continue; // backwards compatibility
                debug(format("bind mounting ‘%1%’ to ‘%2%’") % source % target);
-                if (stat(source.c_str(), &st) == -1)
-                    throw SysError(format("getting attributes of path ‘%1%’") % source);
+                if (stat(source.c_str(), &st) == -1) {
+                    if (i.second.optional && errno == ENOENT)
+                        continue;
+                    else
+                        throw SysError(format("getting attributes of path ‘%1%’") % source);
+                }
                if (S_ISDIR(st.st_mode))
                    createDirs(target);
                else {
@ -2456,91 +2538,93 @@ void DerivationGoal::runChild()

        const char *builder = "invalid";

-        string sandboxProfile;
        if (isBuiltin(*drv)) {
            ;
+        }
 #if __APPLE__
-        } else if (useChroot) {
-            /* Lots and lots and lots of file functions freak out if they can't stat their full ancestry */
-            PathSet ancestry;
+        else if (getEnv("_NIX_TEST_NO_SANDBOX") == "") {
+            /* This has to appear before import statements. */
+            std::string sandboxProfile = "(version 1)\n";

-            /* We build the ancestry before adding all inputPaths to the store because we know they'll
-               all have the same parents (the store), and there might be lots of inputs. This isn't
-               particularly efficient... I doubt it'll be a bottleneck in practice */
-            for (auto & i : dirsInChroot) {
-                Path cur = i.first;
-                while (cur.compare("/") != 0) {
-                    cur = dirOf(cur);
-                    ancestry.insert(cur);
+            if (useChroot) {
+
+                /* Lots and lots and lots of file functions freak out if they can't stat their full ancestry */
+                PathSet ancestry;
+
+                /* We build the ancestry before adding all inputPaths to the store because we know they'll
+                   all have the same parents (the store), and there might be lots of inputs. This isn't
+                   particularly efficient... I doubt it'll be a bottleneck in practice */
+                for (auto & i : dirsInChroot) {
+                    Path cur = i.first;
+                    while (cur.compare("/") != 0) {
+                        cur = dirOf(cur);
+                        ancestry.insert(cur);
+                    }
                }
-            }

-            /* And we want the store in there regardless of how empty dirsInChroot. We include the innermost
-               path component this time, since it's typically /nix/store and we care about that. */
-            Path cur = settings.nixStore;
-            while (cur.compare("/") != 0) {
-                ancestry.insert(cur);
-                cur = dirOf(cur);
-            }
+                /* And we want the store in there regardless of how empty dirsInChroot. We include the innermost
+                   path component this time, since it's typically /nix/store and we care about that. */
+                Path cur = settings.nixStore;
+                while (cur.compare("/") != 0) {
+                    ancestry.insert(cur);
+                    cur = dirOf(cur);
+                }

-            /* Add all our input paths to the chroot */
-            for (auto & i : inputPaths)
-                dirsInChroot[i] = i;
+                /* Add all our input paths to the chroot */
+                for (auto & i : inputPaths)
+                    dirsInChroot[i] = i;

-            /* This has to appear before import statements */
-            sandboxProfile += "(version 1)\n";
+                /* Violations will go to the syslog if you set this. Unfortunately the destination does not appear to be configurable */
+                if (settings.get("darwin-log-sandbox-violations", false)) {
+                    sandboxProfile += "(deny default)\n";
+                } else {
+                    sandboxProfile += "(deny default (with no-log))\n";
+                }

-            /* Violations will go to the syslog if you set this. Unfortunately the destination does not appear to be configurable */
-            if (settings.get("darwin-log-sandbox-violations", false)) {
-                sandboxProfile += "(deny default)\n";
-            } else {
-                sandboxProfile += "(deny default (with no-log))\n";
-            }
+                /* Our rwx outputs */
+                sandboxProfile += "(allow file-read* file-write* process-exec\n";
+                for (auto & i : missingPaths) {
+                    sandboxProfile += (format("\t(subpath \"%1%\")\n") % i.c_str()).str();
+                }
+                sandboxProfile += ")\n";

-            /* The tmpDir in scope points at the temporary build directory for our derivation. Some packages try different mechanisms
-               to find temporary directories, so we want to open up a broader place for them to dump their files, if needed. */
-            Path globalTmpDir = canonPath(getEnv("TMPDIR", "/tmp"), true);
+                /* Our inputs (transitive dependencies and any impurities computed above)

-            /* They don't like trailing slashes on subpath directives */
-            if (globalTmpDir.back() == '/') globalTmpDir.pop_back();
+                   without file-write* allowed, access() incorrectly returns EPERM
+                 */
+                sandboxProfile += "(allow file-read* file-write* process-exec\n";
+                for (auto & i : dirsInChroot) {
+                    if (i.first != i.second.source)
+                        throw Error(format(
+                            "can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin")
+                            % i.first % i.second.source);

-            /* Our rwx outputs */
-            sandboxProfile += "(allow file-read* file-write* process-exec\n";
-            for (auto & i : missingPaths) {
-                sandboxProfile += (format("\t(subpath \"%1%\")\n") % i.c_str()).str();
-            }
-            sandboxProfile += ")\n";
+                    string path = i.first;
+                    struct stat st;
+                    if (lstat(path.c_str(), &st)) {
+                        if (i.second.optional && errno == ENOENT)
+                            continue;
+                        throw SysError(format("getting attributes of path ‘%1%’") % path);
+                    }
+                    if (S_ISDIR(st.st_mode))
+                        sandboxProfile += (format("\t(subpath \"%1%\")\n") % path).str();
+                    else
+                        sandboxProfile += (format("\t(literal \"%1%\")\n") % path).str();
+                }
+                sandboxProfile += ")\n";

-            /* Our inputs (transitive dependencies and any impurities computed above)
+                /* Allow file-read* on full directory hierarchy to self. Allows realpath() */
+                sandboxProfile += "(allow file-read*\n";
+                for (auto & i : ancestry) {
+                    sandboxProfile += (format("\t(literal \"%1%\")\n") % i.c_str()).str();
+                }
+                sandboxProfile += ")\n";

-               without file-write* allowed, access() incorrectly returns EPERM
-             */
-            sandboxProfile += "(allow file-read* file-write* process-exec\n";
-            for (auto & i : dirsInChroot) {
-                if (i.first != i.second)
-                    throw Error(format(
-                        "can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin")
-                        % i.first % i.second);
+                sandboxProfile += additionalSandboxProfile;
+            } else
+                sandboxProfile += "(allow default)\n";

-                string path = i.first;
-                struct stat st;
-                if (lstat(path.c_str(), &st))
-                    throw SysError(format("getting attributes of path ‘%1%’") % path);
-                if (S_ISDIR(st.st_mode))
-                    sandboxProfile += (format("\t(subpath \"%1%\")\n") % path).str();
-                else
-                    sandboxProfile += (format("\t(literal \"%1%\")\n") % path).str();
-            }
-            sandboxProfile += ")\n";
-
-            /* Allow file-read* on full directory hierarchy to self. Allows realpath() */
-            sandboxProfile += "(allow file-read*\n";
-            for (auto & i : ancestry) {
-                sandboxProfile += (format("\t(literal \"%1%\")\n") % i.c_str()).str();
-            }
-            sandboxProfile += ")\n";
-
-            sandboxProfile += additionalSandboxProfile;
+            sandboxProfile += "(deny file-write-setugid)\n";

            debug("Generated sandbox profile:");
            debug(sandboxProfile);
@ -2551,6 +2635,13 @@ void DerivationGoal::runChild()

            writeFile(sandboxFile, sandboxProfile);

+            /* The tmpDir in scope points at the temporary build directory for our derivation. Some packages try different mechanisms
+               to find temporary directories, so we want to open up a broader place for them to dump their files, if needed. */
+            Path globalTmpDir = canonPath(getEnv("TMPDIR", "/tmp"), true);
+
+            /* They don't like trailing slashes on subpath directives */
+            if (globalTmpDir.back() == '/') globalTmpDir.pop_back();
+
            builder = "/usr/bin/sandbox-exec";
            args.push_back("sandbox-exec");
            args.push_back("-f");
@ -2558,8 +2649,9 @@ void DerivationGoal::runChild()
            args.push_back("-D");
            args.push_back("_GLOBAL_TMP_DIR=" + globalTmpDir);
            args.push_back(drv->builder);
+        }
 #endif
-        } else {
+        else {
            builder = drv->builder.c_str();
            string builderBasename = baseNameOf(drv->builder);
            args.push_back(builderBasename);
--- a/src/libstore/download.cc
+++ b/src/libstore/download.cc
@ -114,6 +114,10 @@ struct Curl
        curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, progressCallback_);
        curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, (void *) &curl);
        curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);
+        /* If no file exist in the specified path, curl continues to work
+         * anyway as if netrc support was disabled. */
+        curl_easy_setopt(curl, CURLOPT_NETRC_FILE, settings.netrcFile.c_str());
+        curl_easy_setopt(curl, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);
    }

    ~Curl()
@ -129,7 +133,8 @@ struct Curl
        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

        if (options.verifyTLS)
-            curl_easy_setopt(curl, CURLOPT_CAINFO, getEnv("SSL_CERT_FILE", "/etc/ssl/certs/ca-certificates.crt").c_str());
+            curl_easy_setopt(curl, CURLOPT_CAINFO,
+                getEnv("NIX_SSL_CERT_FILE", getEnv("SSL_CERT_FILE", "/etc/ssl/certs/ca-certificates.crt")).c_str());
        else {
            curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0);
            curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0);
@ -188,7 +193,7 @@ DownloadResult downloadFile(string url, const DownloadOptions & options)
 }


-Path downloadFileCached(const string & url, bool unpack)
+Path downloadFileCached(const string & url, bool unpack, string name)
 {
    Path cacheDir = getEnv("XDG_CACHE_HOME", getEnv("HOME", "") + "/.cache") + "/nix/tarballs";
    createDirs(cacheDir);
@ -223,9 +228,10 @@ Path downloadFileCached(const string & url, bool unpack)
            storePath = "";
    }

-    string name;
-    auto p = url.rfind('/');
-    if (p != string::npos) name = string(url, p + 1);
+    if (name == "") {
+        auto p = url.rfind('/');
+        if (p != string::npos) name = string(url, p + 1);
+    }

    if (!skip) {

--- a/src/libstore/download.hh
+++ b/src/libstore/download.hh
@ -20,7 +20,7 @@ struct DownloadResult

 DownloadResult downloadFile(string url, const DownloadOptions & options);

-Path downloadFileCached(const string & url, bool unpack);
+Path downloadFileCached(const string & url, bool unpack, string name = "");

 MakeError(DownloadError, Error)

--- a/src/libstore/globals.cc
+++ b/src/libstore/globals.cc
@ -61,7 +61,8 @@ Settings::Settings()
    envKeepDerivations = false;
    lockCPU = getEnv("NIX_AFFINITY_HACK", "1") == "1";
    showTrace = false;
-    enableImportNative = false;
+    enableNativeCode = false;
+    netrcFile = (format("%1%/%2%") % nixConfDir % "netrc").str();
 }


@ -185,9 +186,12 @@ void Settings::update()
    _get(sshSubstituterHosts, "ssh-substituter-hosts");
    _get(useSshSubstituter, "use-ssh-substituter");
    _get(logServers, "log-servers");
-    _get(enableImportNative, "allow-unsafe-native-code-during-evaluation");
+    _get(enableNativeCode, "allow-unsafe-native-code-during-evaluation");
    _get(useCaseHack, "use-case-hack");
    _get(preBuildHook, "pre-build-hook");
+    _get(keepGoing, "keep-going");
+    _get(keepFailed, "keep-failed");
+    _get(netrcFile, "netrc-file");

    string subs = getEnv("NIX_SUBSTITUTERS", "default");
    if (subs == "default") {
--- a/src/libstore/globals.hh
+++ b/src/libstore/globals.hh
@ -203,13 +203,17 @@ struct Settings {
    /* A list of URL prefixes that can return Nix build logs. */
    Strings logServers;

-    /* Whether the importNative primop should be enabled */
-    bool enableImportNative;
+    /* Whether native-code enabling primops should be enabled */
+    bool enableNativeCode;

    /* The hook to run just before a build to set derivation-specific
       build settings */
    Path preBuildHook;

+    /* Path to the netrc file used to obtain usernames/passwords for
+       downloads. */
+    Path netrcFile;
+
 private:
    SettingsMap settings, overrides;

--- a/src/libstore/local-store.cc
+++ b/src/libstore/local-store.cc
@ -28,6 +28,7 @@
 #include <sys/statvfs.h>
 #include <sys/mount.h>
 #include <sys/ioctl.h>
+#include <sys/xattr.h>
 #endif

 #include <sqlite3.h>
@ -225,6 +226,7 @@ LocalStore::LocalStore(bool reserveSpace)
    schemaPath = settings.nixDBPath + "/schema";

    if (settings.readOnlyMode) {
+        curSchema = getSchema();
        openDB(false);
        return;
    }
@ -309,6 +311,7 @@ LocalStore::LocalStore(bool reserveSpace)
    } catch (SysError & e) {
        if (e.errNo != EACCES) throw;
        settings.readOnlyMode = true;
+        curSchema = getSchema();
        openDB(false);
        return;
    }
@ -320,8 +323,8 @@ LocalStore::LocalStore(bool reserveSpace)

    /* Check the current database schema and if necessary do an
       upgrade.  */
-    int curSchema = getSchema();
-    if (curSchema > nixSchemaVersion)
+    curSchema = getSchema();
+    if (curSchema >= 0x100)
        throw Error(format("current Nix store schema is version %1%, but I only support %2%")
            % curSchema % nixSchemaVersion);

@ -470,16 +473,18 @@ void LocalStore::openDB(bool create)
        "select path from Refs join ValidPaths on referrer = id where reference = (select id from ValidPaths where path = ?);");
    stmtInvalidatePath.create(db,
        "delete from ValidPaths where path = ?;");
-    stmtRegisterFailedPath.create(db,
-        "insert or ignore into FailedPaths (path, time) values (?, ?);");
-    stmtHasPathFailed.create(db,
-        "select time from FailedPaths where path = ?;");
-    stmtQueryFailedPaths.create(db,
-        "select path from FailedPaths;");
-    // If the path is a derivation, then clear its outputs.
-    stmtClearFailedPath.create(db,
-        "delete from FailedPaths where ?1 = '*' or path = ?1 "
-        "or path in (select d.path from DerivationOutputs d join ValidPaths v on d.drv = v.id where v.path = ?1);");
+    if (curSchema < 9) {
+        stmtRegisterFailedPath.create(db,
+            "insert or ignore into FailedPaths (path, time) values (?, ?);");
+        stmtHasPathFailed.create(db,
+            "select time from FailedPaths where path = ?;");
+        stmtQueryFailedPaths.create(db,
+            "select path from FailedPaths;");
+        // If the path is a derivation, then clear its outputs.
+        stmtClearFailedPath.create(db,
+            "delete from FailedPaths where ?1 = '*' or path = ?1 "
+            "or path in (select d.path from DerivationOutputs d join ValidPaths v on d.drv = v.id where v.path = ?1);");
+    }
    stmtAddDerivationOutput.create(db,
        "insert or replace into DerivationOutputs (drv, id, path) values (?, ?, ?);");
    stmtQueryValidDerivers.create(db,
@ -566,6 +571,16 @@ static void canonicalisePathMetaData_(const Path & path, uid_t fromUid, InodesSe
 {
    checkInterrupt();

+#if __APPLE__
+    /* Remove flags, in particular UF_IMMUTABLE which would prevent
+       the file from being garbage-collected. FIXME: Use
+       setattrlist() to remove other attributes as well. */
+    if (lchflags(path.c_str(), 0)) {
+        if (errno != ENOTSUP)
+            throw SysError(format("clearing flags of path ‘%1%’") % path);
+    }
+#endif
+
    struct stat st;
    if (lstat(path.c_str(), &st))
        throw SysError(format("getting attributes of path ‘%1%’") % path);
@ -574,6 +589,29 @@ static void canonicalisePathMetaData_(const Path & path, uid_t fromUid, InodesSe
    if (!(S_ISREG(st.st_mode) || S_ISDIR(st.st_mode) || S_ISLNK(st.st_mode)))
        throw Error(format("file ‘%1%’ has an unsupported type") % path);

+#if __linux__
+    /* Remove extended attributes / ACLs. */
+    ssize_t eaSize = llistxattr(path.c_str(), nullptr, 0);
+
+    if (eaSize < 0) {
+        if (errno != ENOTSUP)
+            throw SysError(format("querying extended attributes of ‘%s’") % path);
+    } else if (eaSize > 0) {
+        std::vector<char> eaBuf(eaSize);
+
+        if ((eaSize = llistxattr(path.c_str(), eaBuf.data(), eaBuf.size())) < 0)
+            throw SysError(format("querying extended attributes of ‘%s’") % path);
+
+        for (auto & eaName: tokenizeString<Strings>(std::string(eaBuf.data(), eaSize), std::string("\000", 1))) {
+            /* Ignore SELinux security labels since these cannot be
+               removed even by root. */
+            if (eaName == "security.selinux") continue;
+            if (lremovexattr(path.c_str(), eaName.c_str()) == -1)
+                throw SysError(format("removing extended attribute ‘%s’ from ‘%s’") % eaName % path);
+        }
+    }
+#endif
+
    /* Fail if the file is not owned by the build user.  This prevents
       us from messing up the ownership/permissions of files
       hard-linked into the output (e.g. "ln /etc/shadow $out/foo").
@ -742,6 +780,7 @@ void LocalStore::addReference(unsigned long long referrer, unsigned long long re

 void LocalStore::registerFailedPath(const Path & path)
 {
+    if (curSchema >= 9) return;
    retry_sqlite {
        SQLiteStmtUse use(stmtRegisterFailedPath);
        stmtRegisterFailedPath.bind(path);
@ -754,6 +793,7 @@ void LocalStore::registerFailedPath(const Path & path)

 bool LocalStore::hasPathFailed(const Path & path)
 {
+    if (curSchema >= 9) return false;
    retry_sqlite {
        SQLiteStmtUse use(stmtHasPathFailed);
        stmtHasPathFailed.bind(path);
@ -767,6 +807,8 @@ bool LocalStore::hasPathFailed(const Path & path)

 PathSet LocalStore::queryFailedPaths()
 {
+    if (curSchema >= 9) return {};
+
    retry_sqlite {
        SQLiteStmtUse use(stmtQueryFailedPaths);

@ -788,6 +830,8 @@ PathSet LocalStore::queryFailedPaths()

 void LocalStore::clearFailedPaths(const PathSet & paths)
 {
+    if (curSchema >= 9) return;
+
    retry_sqlite {
        SQLiteTxn txn(db);

--- a/src/libstore/local-store.hh
+++ b/src/libstore/local-store.hh
@ -88,6 +88,8 @@ private:

    Path linksDir;

+    int curSchema = 0;
+
 public:

    /* Initialise the local store, upgrading the schema if
--- a/src/libstore/local.mk
+++ b/src/libstore/local.mk
@ -14,6 +14,10 @@ ifeq ($(OS), SunOS)
 	libstore_LDFLAGS += -lsocket
 endif

+ifeq ($(OS), Linux)
+	libstore_LDFLAGS += -lseccomp
+endif
+
 libstore_CXXFLAGS = \
 -DNIX_PREFIX=\"$(prefix)\" \
 -DNIX_STORE_DIR=\"$(storedir)\" \
--- a/src/libutil/finally.hh
+++ b/src/libutil/finally.hh
@ -0,0 +1,14 @@
+#pragma once
+
+#include <functional>
+
+/* A trivial class to run a function at the end of a scope. */
+class Finally
+{
+private:
+    std::function<void()> fun;
+
+public:
+    Finally(std::function<void()> fun) : fun(fun) { }
+    ~Finally() { fun(); }
+};
--- a/src/libutil/hash.cc
+++ b/src/libutil/hash.cc
@ -205,7 +205,7 @@ static void start(HashType ht, Ctx & ctx)


 static void update(HashType ht, Ctx & ctx,
-    const unsigned char * bytes, unsigned int len)
+    const unsigned char * bytes, size_t len)
 {
    if (ht == htMD5) MD5_Update(&ctx.md5, bytes, len);
    else if (ht == htSHA1) SHA1_Update(&ctx.sha1, bytes, len);
--- a/src/libutil/util.cc
+++ b/src/libutil/util.cc
@ -327,10 +327,11 @@ static void _deletePath(const Path & path, unsigned long long & bytesFreed)
        bytesFreed += st.st_blocks * 512;

    if (S_ISDIR(st.st_mode)) {
-        /* Make the directory writable. */
-        if (!(st.st_mode & S_IWUSR)) {
-            if (chmod(path.c_str(), st.st_mode | S_IWUSR) == -1)
-                throw SysError(format("making ‘%1%’ writable") % path);
+        /* Make the directory accessible. */
+        const auto PERM_MASK = S_IRUSR | S_IWUSR | S_IXUSR;
+        if ((st.st_mode & PERM_MASK) != PERM_MASK) {
+            if (chmod(path.c_str(), st.st_mode | PERM_MASK) == -1)
+                throw SysError(format("chmod ‘%1%’") % path);
        }

        for (auto & i : readDirectory(path))
@ -1170,6 +1171,12 @@ bool statusOk(int status)
 }


+bool hasPrefix(const string & s, const string & suffix)
+{
+    return s.compare(0, suffix.size(), suffix) == 0;
+}
+
+
 bool hasSuffix(const string & s, const string & suffix)
 {
    return s.size() >= suffix.size() && string(s, s.size() - suffix.size()) == suffix;
--- a/src/libutil/util.hh
+++ b/src/libutil/util.hh
@ -365,6 +365,10 @@ template<class N> bool string2Int(const string & s, N & n)
 }


+/* Return true iff `s' starts with `prefix'. */
+bool hasPrefix(const string & s, const string & prefix);
+
+
 /* Return true iff `s' ends in `suffix'. */
 bool hasSuffix(const string & s, const string & suffix);

--- a/src/nix-env/user-env.cc
+++ b/src/nix-env/user-env.cc
@ -63,8 +63,8 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
        if (drvPath != "")
            mkString(*state.allocAttr(v, state.sDrvPath), i.queryDrvPath());

-        // Copy each output.
-        DrvInfo::Outputs outputs = i.queryOutputs();
+        // Copy each output meant for installation.
+        DrvInfo::Outputs outputs = i.queryOutputs(true);
        Value & vOutputs = *state.allocAttr(v, state.sOutputs);
        state.mkList(vOutputs, outputs.size());
        unsigned int m = 0;
--- a/tests/common.sh.in
+++ b/tests/common.sh.in
@ -16,6 +16,10 @@ export NIX_DB_DIR=$TEST_ROOT/db
 export NIX_CONF_DIR=$TEST_ROOT/etc
 export NIX_MANIFESTS_DIR=$TEST_ROOT/var/nix/manifests
 export _NIX_TEST_SHARED=$TEST_ROOT/shared
+if [[ -n $NIX_STORE ]]; then
+    export _NIX_TEST_NO_SANDBOX=1
+fi
+export _NIX_IN_TEST=$TEST_ROOT/shared
 export NIX_REMOTE=$NIX_REMOTE_

 export PATH=@bindir@:$PATH
--- a/tests/lang/eval-okay-ind-string.exp
+++ b/tests/lang/eval-okay-ind-string.exp
@ -1 +1 @@
-"This is an indented multi-line string\nliteral.  An amount of whitespace at\nthe start of each line matching the minimum\nindentation of all lines in the string\nliteral together will be removed.  Thus,\nin this case four spaces will be\nstripped from each line, even though\n  THIS LINE is indented six spaces.\n\nAlso, empty lines don't count in the\ndetermination of the indentation level (the\nprevious empty line has indentation 0, but\nit doesn't matter).\nIf the string starts with whitespace\n  followed by a newline, it's stripped, but\n  that's not the case here. Two spaces are\n  stripped because of the \"  \" at the start. \nThis line is indented\na bit further.\nAnti-quotations, like so, are\nalso allowed.\n  The \\ is not special here.\n' can be followed by any character except another ', e.g. 'x'.\nLikewise for $, e.g. $$ or $varName.\nBut ' followed by ' is special, as is $ followed by {.\nIf you want them, use anti-quotations: '', ${.\n   Tabs are not interpreted as whitespace (since we can't guess\n   what tab settings are intended), so don't use them.\n\tThis line starts with a space and a tab, so only one\n   space will be stripped from each line.\nAlso note that if the last line (just before the closing ' ')\nconsists only of whitespace, it's ignored.  But here there is\nsome non-whitespace stuff, so the line isn't removed. \nThis shows a hacky way to preserve an empty line after the start.\nBut there's no reason to do so: you could just repeat the empty\nline.\n  Similarly you can force an indentation level,\n  in this case to 2 spaces.  This works because the anti-quote\n  is significant (not whitespace).\nstart on network-interfaces\n\nstart script\n\n  rm -f /var/run/opengl-driver\n  ln -sf 123 /var/run/opengl-driver\n\n  rm -f /var/log/slim.log\n   \nend script\n\nenv SLIM_CFGFILE=abc\nenv SLIM_THEMESDIR=def\nenv FONTCONFIG_FILE=/etc/fonts/fonts.conf  \t\t\t\t# !!! cleanup\nenv XKB_BINDIR=foo/bin         \t\t\t\t# Needed for the Xkb extension.\nenv LD_LIBRARY_PATH=libX11/lib:libXext/lib:/usr/lib/          # related to xorg-sys-opengl - needed to load libglx for (AI)GLX support (for compiz)\n\nenv XORG_DRI_DRIVER_PATH=nvidiaDrivers/X11R6/lib/modules/drivers/ \n\nexec slim/bin/slim\nEscaping of ' followed by ': ''\nEscaping of $ followed by {: ${\nAnd finally to interpret \\n etc. as in a string: \n, \r, \t.\nfoo\n'bla'\nbar\n"
+"This is an indented multi-line string\nliteral.  An amount of whitespace at\nthe start of each line matching the minimum\nindentation of all lines in the string\nliteral together will be removed.  Thus,\nin this case four spaces will be\nstripped from each line, even though\n  THIS LINE is indented six spaces.\n\nAlso, empty lines don't count in the\ndetermination of the indentation level (the\nprevious empty line has indentation 0, but\nit doesn't matter).\nIf the string starts with whitespace\n  followed by a newline, it's stripped, but\n  that's not the case here. Two spaces are\n  stripped because of the \"  \" at the start. \nThis line is indented\na bit further.\nAnti-quotations, like so, are\nalso allowed.\n  The \\ is not special here.\n' can be followed by any character except another ', e.g. 'x'.\nLikewise for $, e.g. $$ or $varName.\nBut ' followed by ' is special, as is $ followed by {.\nIf you want them, use anti-quotations: '', ${.\n   Tabs are not interpreted as whitespace (since we can't guess\n   what tab settings are intended), so don't use them.\n\tThis line starts with a space and a tab, so only one\n   space will be stripped from each line.\nAlso note that if the last line (just before the closing ' ')\nconsists only of whitespace, it's ignored.  But here there is\nsome non-whitespace stuff, so the line isn't removed. \nThis shows a hacky way to preserve an empty line after the start.\nBut there's no reason to do so: you could just repeat the empty\nline.\n  Similarly you can force an indentation level,\n  in this case to 2 spaces.  This works because the anti-quote\n  is significant (not whitespace).\nstart on network-interfaces\n\nstart script\n\n  rm -f /var/run/opengl-driver\n  ln -sf 123 /var/run/opengl-driver\n\n  rm -f /var/log/slim.log\n   \nend script\n\nenv SLIM_CFGFILE=abc\nenv SLIM_THEMESDIR=def\nenv FONTCONFIG_FILE=/etc/fonts/fonts.conf  \t\t\t\t# !!! cleanup\nenv XKB_BINDIR=foo/bin         \t\t\t\t# Needed for the Xkb extension.\nenv LD_LIBRARY_PATH=libX11/lib:libXext/lib:/usr/lib/          # related to xorg-sys-opengl - needed to load libglx for (AI)GLX support (for compiz)\n\nenv XORG_DRI_DRIVER_PATH=nvidiaDrivers/X11R6/lib/modules/drivers/ \n\nexec slim/bin/slim\nEscaping of ' followed by ': ''\nEscaping of $ followed by {: ${\nAnd finally to interpret \\n etc. as in a string: \n, \r, \t.\nfoo\n'bla'\nbar\ncut -d $'\\t' -f 1\nending dollar $$\n"
--- a/tests/lang/eval-okay-ind-string.nix
+++ b/tests/lang/eval-okay-ind-string.nix
@ -117,4 +117,12 @@ let
    bar
  '';

-in s1 + s2 + s3 + s4 + s5 + s6 + s7 + s8 + s9 + s10 + s11 + s12 + s13 + s14 + s15
+  # Regression test: accept $'.
+  s16 = ''
+    cut -d $'\t' -f 1
+  '';
+
+  # Accept dollars at end of strings 
+  s17 = ''ending dollar $'' + ''$'' + "\n";
+
+in s1 + s2 + s3 + s4 + s5 + s6 + s7 + s8 + s9 + s10 + s11 + s12 + s13 + s14 + s15 + s16 + s17
--- a/tests/lexer.nix
+++ b/tests/lexer.nix
@ -0,0 +1,2 @@
+let const = a: "const"; in
+''${ const { x = "q"; }}''
--- a/tests/lexer.sh
+++ b/tests/lexer.sh
@ -0,0 +1,3 @@
+source common.sh
+
+nix-instantiate --eval ./lexer.nix
--- a/tests/local.mk
+++ b/tests/local.mk
@ -11,7 +11,7 @@ nix_tests = \
  binary-patching.sh timeout.sh secure-drv-outputs.sh nix-channel.sh \
  multiple-outputs.sh import-derivation.sh fetchurl.sh optimise-store.sh \
  binary-cache.sh nix-profile.sh repair.sh dump-db.sh case-hack.sh \
-  check-reqs.sh pass-as-file.sh tarball.sh
+  check-reqs.sh pass-as-file.sh tarball.sh lexer.sh
  # parallel.sh

 install-tests += $(foreach x, $(nix_tests), tests/$(x))
--- a/tests/nix-copy-closure.nix
+++ b/tests/nix-copy-closure.nix
@ -4,7 +4,7 @@

 with import <nixpkgs/nixos/lib/testing.nix> { inherit system; };

-makeTest (let pkgA = pkgs.aterm; pkgB = pkgs.wget; pkgC = pkgs.hello; in {
+makeTest (let pkgA = pkgs.cowsay; pkgB = pkgs.wget; pkgC = pkgs.hello; in {

  nodes =
    { client =
--- a/tests/remote-builds.nix
+++ b/tests/remote-builds.nix
@ -14,7 +14,7 @@ let
    { services.openssh.enable = true;
      virtualisation.writableStore = true;
      nix.package = nix;
-      nix.useChroot = true;
+      nix.useSandbox = true;
    };

  # Trivial Nix expression to build remotely.
--- a/tests/setuid.nix
+++ b/tests/setuid.nix
@ -0,0 +1,108 @@
+# Verify that Linux builds cannot create setuid or setgid binaries.
+
+{ system, nix }:
+
+with import <nixpkgs/nixos/lib/testing.nix> { inherit system; };
+
+makeTest {
+
+  machine =
+    { config, lib, pkgs, ... }:
+    { virtualisation.writableStore = true;
+      nix.package = nix;
+      nix.binaryCaches = [ ];
+      nix.nixPath = [ "nixpkgs=${lib.cleanSource pkgs.path}" ];
+      virtualisation.pathsInNixDB = [ pkgs.stdenv pkgs.pkgsi686Linux.stdenv ];
+    };
+
+  testScript = { nodes }:
+    ''
+      startAll;
+
+      # Copying to /tmp should succeed.
+      $machine->succeed('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" {} "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 555 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      # Creating a setuid binary should fail.
+      $machine->fail('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" {} "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        chmod 4755 /tmp/id
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 555 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      # Creating a setgid binary should fail.
+      $machine->fail('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" {} "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        chmod 2755 /tmp/id
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 555 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      # The checks should also work on 32-bit binaries.
+      $machine->fail('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> { system = "i686-linux"; }; runCommand "foo" {} "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        chmod 2755 /tmp/id
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 555 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      # The tests above use fchmodat(). Test chmod() as well.
+      $machine->succeed('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" { buildInputs = [ perl ]; } "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        perl -e \"chmod 0666, qw(/tmp/id) or die\"
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 666 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      $machine->fail('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" { buildInputs = [ perl ]; } "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        perl -e \"chmod 04755, qw(/tmp/id) or die\"
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 555 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      # And test fchmod().
+      $machine->succeed('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" { buildInputs = [ perl ]; } "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        perl -e \"my \\\$x; open \\\$x, qw(/tmp/id); chmod 01750, \\\$x or die\"
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 1750 ]]');
+
+      $machine->succeed("rm /tmp/id");
+
+      $machine->fail('nix-build --option build-use-sandbox false -E \'(with import <nixpkgs> {}; runCommand "foo" { buildInputs = [ perl ]; } "
+        mkdir -p $out
+        cp ${pkgs.coreutils}/bin/id /tmp/id
+        perl -e \"my \\\$x; open \\\$x, qw(/tmp/id); chmod 04777, \\\$x or die\"
+      ")\' ');
+
+      $machine->succeed('[[ $(stat -c %a /tmp/id) = 555 ]]');
+
+      $machine->succeed("rm /tmp/id");
+    '';
+
+}
--- a/tests/tarball.sh
+++ b/tests/tarball.sh
@ -20,4 +20,10 @@ nix-build file://$tarball

 nix-build '<foo>' -I foo=file://$tarball

-nix-build -E "import (fetchTarball file://$tarball)"
+nix-build -o $TMPDIR/result -E "import (fetchTarball file://$tarball)"
+
+nix-instantiate --eval -E '1 + 2' -I fnord=file://no-such-tarball.tar.xz
+nix-instantiate --eval -E 'with <fnord/xyzzy>; 1 + 2' -I fnord=file://no-such-tarball.tar.xz
+(! nix-instantiate --eval -E '<fnord/xyzzy> 1' -I fnord=file://no-such-tarball.tar.xz)
+
+nix-instantiate --eval -E '<fnord/config.nix>' -I fnord=file://no-such-tarball.tar.xz -I fnord=.
--- a/2
+++ b/2
@ -1 +1 @@
-1.11.1
+1.11.11
Author	SHA1	Message	Date
Eelco Dolstra	026f4f9ae8	macOS: Remove flags In particular, UF_IMMUTABLE (uchg) needs to be cleared to allow the path to be garbage-collected or optimised. See https://github.com/NixOS/nixpkgs/issues/25819. + the file from being garbage-collected. (cherry picked from commit `b5bdfdef73`)	2017-06-19 14:32:38 +02:00
Eelco Dolstra	11dd08f02e	macOS: Ugly hack to make the tests succeed Sandboxes cannot be nested, so if Nix's build runs inside a sandbox, it cannot use a sandbox itself. I don't see a clean way to detect whether we're in a sandbox, so use a test-specific hack. https://github.com/NixOS/nix/issues/1413 (cherry picked from commit `1888f7889b`)	2017-06-19 14:28:04 +02:00
Shea Levy	3e574c3691	Merge branch '1.11-maintenance' of git://github.com/cyraxjoe/nix into 1.11-maintenance	2017-06-14 07:52:36 -04:00
Eelco Dolstra	5ac7088726	Bump version	2017-06-14 11:43:51 +02:00
Eelco Dolstra	35ea3d62dc	canonicalisePathMetaData(): Ignore security.selinux attribute Untested, hopefully fixes #1406. (cherry picked from commit `88b291ffc4`)	2017-06-14 11:43:33 +02:00
Joel Rivera	7917494c45	Add support for the curl netrc file in nix-channel/nix-pull. Based on the implementation in download-from-binary-cache.pl.in.	2017-06-13 18:18:20 -05:00
Eelco Dolstra	1e4885e316	Grmbl	2017-06-12 18:52:26 +02:00
Eelco Dolstra	36f363b8f0	On macOS, don't use /var/folders for TMPDIR This broke "nix-store --serve". (cherry picked from commit `25230a17a9`)	2017-06-12 18:39:34 +02:00
Eelco Dolstra	c33854513a	Remove Ubuntu 13.10 build Seccomp is too old there.	2017-06-12 16:47:16 +02:00
Eelco Dolstra	0be5b949d3	Don't run pre-build-hook if we don't have a derivation This fixes a build failure on OS X when using Hydra or Nix 1.12's build-remote (since they don't copy the derivation to the build machine). (cherry picked from commit `7f5b750b40`)	2017-06-12 16:46:31 +02:00
Eelco Dolstra	c20641ce56	OS X -> macOS	2017-06-12 14:04:52 +02:00
Eelco Dolstra	0fb60e4e0f	Add 1.11.10 release notes	2017-06-12 13:56:38 +02:00
Eelco Dolstra	3414f3804c	Fix build	2017-06-12 13:55:59 +02:00
Eelco Dolstra	8e298e8ad9	Always use the Darwin sandbox Even with "build-use-sandbox = false", we now use sandboxing with a permissive profile that allows everything except the creation of setuid/setgid binaries. Based on `85e93d7b87`.	2017-06-06 20:35:55 +02:00
Eelco Dolstra	f534627929	Fix bad cherrypick	2017-06-06 19:52:40 +02:00
Eelco Dolstra	0ca9502264	Disable the build user mechanism on all platforms except Linux and OS X (cherry picked from commit `c8cc50d46e`)	2017-06-06 19:52:24 +02:00
Eelco Dolstra	bcc21744df	Bump version	2017-06-01 16:53:10 +02:00
Eelco Dolstra	833aae4509	Fix coverage job (cherry picked from commit `b4b1f4525f`)	2017-06-01 16:51:32 +02:00
Eelco Dolstra	aabe20bf78	RPM, Deb: Add dependency on libseccomp (cherry picked from commit `ab5834f7a1`)	2017-06-01 16:51:03 +02:00
Eelco Dolstra	c48697d617	Remove listxattr assertion It appears that sometimes, listxattr() returns a different value for the query case (i.e. when the buffer size is 0). (cherry picked from commit `52fec8dde8`)	2017-06-01 16:50:23 +02:00
Eelco Dolstra	4be5a65b39	Fix seccomp build failure on clang Fixes src/libstore/build.cc:2321:45: error: non-constant-expression cannot be narrowed from type 'int' to 'scmp_datum_t' (aka 'unsigned long') in initializer list [-Wc++11-narrowing] (cherry picked from commit `fe08d17934`)	2017-06-01 16:50:08 +02:00
Eelco Dolstra	634d117ede	Add a seccomp rule to disallow setxattr() (cherry picked from commit `2ac99a32da`)	2017-06-01 16:50:04 +02:00
Eelco Dolstra	66618dbad5	canonicalisePathMetaData(): Remove extended attributes / ACLs EAs/ACLs are not part of the NAR canonicalisation. Worse, setting an ACL allows a builder to create writable files in the Nix store. So get rid of them. Closes #185. (cherry picked from commit `d798349ede`)	2017-06-01 16:50:00 +02:00
Eelco Dolstra	1e0f1dab1e	Require seccomp only in multi-user setups (cherry picked from commit `ff6becafa8`)	2017-06-01 16:49:52 +02:00
Eelco Dolstra	17da82e04d	Add test for setuid seccomp filter (cherry picked from commit `1d9ab273ba`)	2017-06-01 16:49:47 +02:00
Eelco Dolstra	a2cf0f1018	Fix seccomp initialisation on i686-linux (cherry picked from commit `cf93397d3f`)	2017-06-01 16:49:42 +02:00
Eelco Dolstra	e296b8884e	Add a seccomp filter to prevent creating setuid/setgid binaries This prevents builders from setting the S_ISUID or S_ISGID bits, preventing users from using a nixbld* user to create a setuid/setgid binary to interfere with subsequent builds under the same nixbld* uid. This is based on aszlig's seccomp code (`47f587700d`). Reported by Linus Heckemann. (cherry picked from commit `6cc6c15a2d`)	2017-06-01 16:48:57 +02:00
Guillaume Maudoux	a8d13e66ee	lexer: remove catch-all rules hiding real errors With catch-all rules, we hide potential errors. It turns out that `a4744254` made one cath-all useless. Flex detected that is was impossible to reach. The other is more subtle, as it can only trigger on unfinished escapes in unfinished strings, which only occurs at EOF. (cherry picked from commit `a143014d73`)	2017-05-01 11:26:44 +02:00
Guillaume Maudoux	a10bd3355a	Fix lexer to support `$'` in multiline strings. (cherry picked from commit `a474425425`)	2017-05-01 11:26:41 +02:00
Eelco Dolstra	26f7e8305b	Fix hash computation when importing NARs greater than 4 GiB This caused "nix-store --import" to compute an incorrect hash on NARs that don't fit in an unsigned int. The import would succeed, but "nix-store --verify-path" or subsequent exports would detect an incorrect hash. A deeper issue is that the export/import format does not contain a hash, so we can't detect such issues early. Also, I learned that -Wall does not warn about this. (cherry picked from commit `41c4558afe`)	2017-05-01 11:26:27 +02:00
Eelco Dolstra	5d59ec86d4	Bump version to 1.11.9	2017-04-25 16:58:00 +02:00
Eelco Dolstra	1ab5cc0f5a	Merge pull request #1333 from copumpkin/fix-schema-version Ensure that curSchema is set before opening the DB	2017-04-14 20:58:06 +02:00
Dan Peebles	ad9e6037a4	Ensure that curSchema is set before opening the DB Without this, it's possible to get `curSchema = 0` which then causes us not to trigger the branch that maintains forward compatibility with the 1.12 schema. Fixes #1332	2017-04-14 14:44:28 -04:00
Shea Levy	eb6f3d5159	builtins.exec: Make the argument just a list (cherry picked from commit `3ecb09a40a`)	2017-03-31 11:58:51 -04:00
Shea Levy	967f231981	Add exec primop behind allow-unsafe-native-code-during-evaluation. Execute a given program with the (optional) given arguments as the user running the evaluation, parsing stdout as an expression to be evaluated. There are many use cases for nix that would benefit from being able to run arbitrary code during evaluation, including but not limited to: * Automatic git fetching to get a sha256 from a git revision * git rev-parse HEAD * Automatic extraction of information from build specifications from other tools, particularly language-specific package managers like cabal or npm * Secrets decryption (e.g. with nixops) * Private repository fetching Ideally, we would add this functionality in a more principled way to nix, but in the mean time 'builtins.exec' can be used to get these tasks done. The primop is only available when the 'allow-unsafe-native-code-during-evaluation' nix option is true. That flag also enables the 'importNative' primop, which is strictly more powerful but less convenient (since it requires compiling a plugin against the running version of nix). (cherry picked from commit `0bb8db257d`)	2017-03-30 10:08:38 -04:00
Daiderd Jordan	773313591f	use std::tuple for ValueMap allocator (cherry picked from commit `023217f07c`)	2017-03-24 23:31:47 +01:00
Eelco Dolstra	206b61b074	useChroot -> useSandbox (cherry picked from commit `8d7c6644c5`)	2017-03-21 13:50:47 +01:00
Eelco Dolstra	201ad43d1a	Bump version to 1.11.8	2017-03-21 11:34:29 +01:00
Vladimír Čunát	9504bcf03c	nix-env: respect meta.outputsToInstall Discussed on https://github.com/NixOS/nixpkgs/pull/12653#discussion_r51601849 (cherry picked from commit `03cbb9ad59`)	2017-03-21 11:33:21 +01:00
Eelco Dolstra	d8638ae6b4	Merge pull request #1259 from shlevy/1.11-netrc Backport netrc-file option to 1.11	2017-03-20 17:37:54 +01:00
Shea Levy	a437b8c5d2	download-from-binary-cache.pl: Respect netrc-file	2017-03-05 17:40:17 -05:00
Shea Levy	37bdb9d7f2	Backport netrc-file option to 1.11	2017-03-04 09:55:49 -05:00
Domen Kožar	13fe83dc8e	bail out if macOS 10.9 or lower is used during installer (cherry picked from commit `48d4a23aa0`) Signed-off-by: Domen Kožar <domen@dev.si> cc @edolstra	2017-02-23 15:12:35 +01:00
Eelco Dolstra	697b5755e4	Merge pull request #1247 from veprbl/ssl_fix2 use --cacert instead of --capath	2017-02-22 20:14:47 +01:00
Dmitry Kalinkin	06b46f646d	use --cacert instead of --capath This forces curl to use nix bundled crt instead of picking one up from system. Fixes: `142c77711` ('Propagate path of CA bundle to curl child processes')	2017-02-22 14:04:47 -05:00
Eelco Dolstra	2936a8d1ca	Bump version	2017-02-22 12:43:54 +01:00
Eelco Dolstra	142c777112	Propagate path of CA bundle to curl child processes	2017-02-22 12:43:13 +01:00
Eelco Dolstra	41230dd463	SSL_CERT_FILE -> NIX_SSL_CERT_FILE This prevents collisions with the "native" OpenSSL, in particular on OS X. Fixes #921. (cherry picked from commit `fb2dd32100`)	2017-02-22 12:30:45 +01:00
Eelco Dolstra	8ca944e009	Add forward compatibility with the Nix 1.12 database schema	2017-02-22 12:19:42 +01:00
Eelco Dolstra	1fa2c86db5	Bump version to 1.11.6	2017-01-13 11:02:55 +01:00
Eelco Dolstra	0085fc6d3e	Merge pull request #1178 from dezgeg/bash-env-breakage-stable nix-shell: Fix 'nix-shell --command' doing nothing without TTY (stable backport)	2017-01-09 16:47:14 +01:00
Tuomas Tynkkynen	a4ad1ffa25	nix-shell: Fix 'nix-shell --command' doing nothing without TTY Regression from `8bddc3d4` ("Fix early removal of rc-file for nix-shell"). The removal of BASH_ENV causes nothing to be executed by bash if it detects itself in a non-interactive context. Instead, just use the same condition used by bash to launch bash differently. According to bash sources, the condition (stdin and stder both must be TTYs) is specified by POSIX so this should be pretty safe to rely on. Fixes #1171 on 1.11.	2017-01-07 19:36:45 +02:00
Scott R. Parish	0c67498be9	fix "nix-build" examples The existing "nix-build" examples were failing: error: cannot auto-call a function that has an argument without a default value (‘system’) Thanks to @groxxda on irc for pointing out the fix! Updated to completely remove unneeded path argument, suggested by @joachifm Updated to remove other occurences of `all-packages.nix` from files as suggested by @domenkozar (cherry picked from commit `96d3534a9e`)	2017-01-03 16:38:48 +01:00
Eelco Dolstra	d39f51fa34	Bump version number	2017-01-02 15:18:50 +01:00
Dmitry Kalinkin	5bf9689e0c	override rx directory permissions in deletePath() This fixes instantiation of pythonPackages.pytest that produces a directory with less permissions during one of it's tests that leads to a nix error like: error: opening directory ‘/tmp/nix-build-python2.7-pytest-2.9.2.drv-0/pytest-of-user/pytest-0/testdir/test_cache_failure_warns0/.cache’: Permission denied (cherry picked from commit `f91748ba73`)	2016-12-19 14:28:57 +01:00
Shea Levy	612c77a399	Update darwin build for optional sandbox paths Fixes #1132 (cherry picked from commit `8bf378e999`)	2016-11-22 10:46:26 +01:00
Eelco Dolstra	2eb840eefa	Support optional sandbox paths For example, you can now set build-sandbox-paths = /dev/nvidiactl? to specify that /dev/nvidiactl should only be mounted in the sandbox if it exists in the host filesystem. This is useful e.g. for EC2 images that should support both CUDA and non-CUDA instances. (cherry picked from commit `18b7363a69`)	2016-11-22 10:46:26 +01:00
Shea Levy	fd9fc15c0c	Add nix.conf options for -k and -K Fixes #1084	2016-10-27 12:51:27 -04:00
Shea Levy	8bddc3d406	Fix early removal of rc-file for nix-shell BASH_ENV causes all non-interactive shells called via eg. /etc/bashrc to remove the rc-file before the main shell gets to run it. Completion scripts will often do this. Fixes #976. Adapted from and fixes #1034.	2016-09-08 09:10:50 -04:00
Eelco Dolstra	fb577a431f	Fix Debian 8 build http://hydra.nixos.org/build/36462150 (cherry picked from commit `88b79cd55c`)	2016-09-06 13:12:53 +02:00
Eelco Dolstra	8c353ea698	Fix incomplete cherry-pick in `0eb8bbb31e`	2016-09-06 13:12:04 +02:00
Eelco Dolstra	a566927003	Fix OOM in the installer test http://hydra.nixos.org/build/36462209 (cherry picked from commit `1b5b654fe2`)	2016-09-06 13:07:54 +02:00
Dan Peebles	f2495212b1	Kill the temporary darwin-specific channel The issues have been resolved upstream in the main nixpkgs channel now (cherry picked from commit `c89783b6a7`)	2016-09-06 13:06:51 +02:00
Eelco Dolstra	7ead75ca85	Drop all distros that are not down with C++11 (cherry picked from commit `7251a81bde`)	2016-09-06 13:06:40 +02:00
Eelco Dolstra	f668fdb026	Fix nix-copy-closure test on 16.03 (cherry picked from commit `b86555aa2b`)	2016-09-06 12:59:48 +02:00
aszlig	16e88f0b5a	Don't hardcode docbook XSL namespace URL Docbook XSL got updated to version 1.79.1 in NixOS/nixpkgs@fb893a8 and we're still referring to the hardcoded previous version. So instead of just updating this to 1.79.1 we're going to use "current" in the hope that this won't happen again. I have tested this by building the manual under Nix(OS) but I haven't tested this in a non-Nix environment, so I'm not sure whether this could have implications. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Cc: @edolstra (cherry picked from commit `72fb2ccfa1`)	2016-09-06 12:52:49 +02:00
Eelco Dolstra	2179dd3e5b	Bump version	2016-09-02 14:58:47 +02:00
Eelco Dolstra	efa2e451fb	Make the search path lazier with non-fatal errors Thus, -I / $NIX_PATH entries are now downloaded only when they are needed for evaluation. An error to download an entry is a non-fatal warning (just like non-existant paths). This does change the semantics of builtins.nixPath, which now returns the original, rather than resulting path. E.g., before we had [ { path = "/nix/store/hgm3yxf1lrrwa3z14zpqaj5p9vs0qklk-nixexprs.tar.xz"; prefix = "nixpkgs"; } ... ] but now [ { path = "https://nixos.org/channels/nixos-16.03/nixexprs.tar.xz"; prefix = "nixpkgs"; } ... ] Fixes #792. (cherry picked from commit `363f37d084`)	2016-09-02 11:55:14 +02:00
Shea Levy	dea9de79b7	callFunction: Copy functors to the heap Normally it's impossible to take a reference to the function passed to callFunction, so some callers (e.g. ExprApp::eval) allocate that value on the stack. For functors, a reference to the functor itself may be kept, so we need to have it on the heap. Fixes #1045 (cherry picked from commit `9fa21765e7`)	2016-08-29 07:40:03 -04:00
Shea Levy	2ac966a464	Bump version to 1.11.3	2016-08-22 10:20:41 -04:00
Eelco Dolstra	6a493a7c44	launchd: Set $SSL_CERT_FILE Otherwise in particular https://cache.nixos.org won't work in the daemon. (cherry picked from commit `eff80419c7`)	2016-08-19 13:11:03 +02:00
Shea Levy	7bb4d028a8	builtins.fetch{url,tarball}: Allow name attribute (cherry picked from commit `d52d391164`)	2016-08-15 07:42:51 -04:00
Shea Levy	66151dc154	Respect --keep-going when a substituter fails. Fixes #977 (cherry picked from commit `18b0808475`)	2016-07-23 13:22:52 -04:00
Domen Kožar	0eb8bbb31e	add Ubuntu 16.03 .deb builds (cherry picked from commit `bf386de9f2`) Signed-off-by: Domen Kožar <domen@dev.si>	2016-05-26 15:55:51 +01:00
Eelco Dolstra	437d3cdc7a	Bump version to 1.11.2	2016-02-10 12:10:03 +01:00
Alex Cruice	0322c92560	Check shell profile is writeable before modifying The `set -e` at the top of the script causes the installation to fail to complete if the shell profile is not writeable. Checking file existence only is not enough. (cherry picked from commit `ad0dc41899`)	2016-02-10 12:04:47 +01:00
Brian McKenna	1852f7dbf3	Fix broken call to shellwords nix-shell shebangs were broken by `9018deab` (cherry picked from commit `3baf8be1d1`)	2016-02-10 12:04:18 +01:00
Eelco Dolstra	08500066ea	Revert "Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751 """ This reverts commit `b669d3d2e8`. (cherry picked from commit `5d8b7eb3e1`)	2016-02-10 12:04:08 +01:00
 @ -1 +1 @@
 .11.1
 .11.11