Commit graph

258 commits

Author SHA1 Message Date
Eelco Dolstra a478e8a7bb Remove nix-setuid-helper
AFAIK, nobody uses it, it's not maintained, and it has no tests.
2013-11-14 11:57:37 +01:00
Eelco Dolstra 297b762513 Turn on -Wall 2013-08-19 11:41:15 +02:00
Eelco Dolstra a583a2bc59 Run the daemon worker on the same CPU as the client
On a system with multiple CPUs, running Nix operations through the
daemon is significantly slower than "direct" mode:

$ NIX_REMOTE= nix-instantiate '<nixos>' -A system
real    0m0.974s
user    0m0.875s
sys     0m0.088s

$ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system
real    0m2.118s
user    0m1.463s
sys     0m0.218s

The main reason seems to be that the client and the worker get moved
to a different CPU after every call to the worker.  This patch adds a
hack to lock them to the same CPU.  With this, the overhead of going
through the daemon is very small:

$ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system
real    0m1.074s
user    0m0.809s
sys     0m0.098s
2013-08-07 14:02:04 +02:00
Eelco Dolstra 263d668222 Set the default GCC optimisation level to -O3 2013-08-06 14:21:46 +02:00
Gergely Risko 25a00cae5b Add gzip support for channel unpacking 2013-07-12 11:29:37 +02:00
Shea Levy 2c9cf50746 makeStoreWritable: Use statvfs instead of /proc/self/mountinfo to find out if /nix/store is a read-only bind mount
/nix/store could be a read-only bind mount even if it is / in its own filesystem, so checking the 4th field in mountinfo is insufficient.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-03-25 19:00:16 +01:00
Eelco Dolstra 0a4e90395c Urgggh
http://hydra.nixos.org/build/3661100
2013-01-02 23:52:15 +01:00
Eelco Dolstra f794465ca8 Disable use of vfork()
vfork() is just too weird.  For instance, in this build:

  http://hydra.nixos.org/build/3330487

the value fromHook.writeSide becomes corrupted in the parent, even
though the child only reads from it.  At -O0 the problem goes away.
Probably the child is overriding some spilled temporary variable.

If I get bored I may implement using posix_spawn() instead.
2012-11-15 19:35:18 +01:00
Eelco Dolstra ea89df2b76 Use vfork() instead of fork() if available
Hopefully this reduces the chance of hitting ‘unable to fork: Cannot
allocate memory’ errors.  vfork() is used for everything except
starting builders.
2012-11-09 18:00:33 +01:00
Mats Erik Andersson ecedd9c50c Out-of-tree building of perl modules. 2012-10-10 10:28:33 -04:00
Eelco Dolstra e35d6f78dc Rename nix-worker to nix-daemon 2012-10-03 17:59:23 -04:00
Eelco Dolstra 7586095504 Remove bin2c 2012-10-03 16:59:28 -04:00
Eelco Dolstra b674665765 Test whether GNU tar understands --warning=no-timestamp
http://hydra.nixos.org/build/3031618
2012-09-14 10:18:27 -04:00
Eelco Dolstra 8207359227 Whitespace 2012-08-27 11:28:48 -04:00
Eelco Dolstra 46f852cda0 Use a GNU tar flag to shut up warnings about implausibly old timestamp 2012-08-01 16:42:51 -04:00
Eelco Dolstra 8c79100839 Merge branch 'master' into no-manifests 2012-07-26 15:14:33 -04:00
Eelco Dolstra e98c029717 Handle platforms that don't support linking to a symlink
E.g. Darwin doesn't allow this.
2012-07-23 18:42:18 -04:00
Eelco Dolstra 6852289c46 Use lutimes() if available to canonicalise the timestamp of symlinks
Also use utimes() instead of utime() if lutimes() is not available.
2012-07-23 16:52:25 -04:00
Eelco Dolstra ae72be1b8b Add WWW::Curl as a dependency 2012-07-09 13:11:37 -04:00
Eelco Dolstra 4911a10a4e Use XZ compression in binary caches
XZ compresses significantly better than bzip2.  Here are the
compression ratios and execution times (using 4 cores in parallel) on
my /var/run/current-system (3.1 GiB):

  bzip2: total compressed size 849.56 MiB, 30.8% [2m08]
  xz -6: total compressed size 641.84 MiB, 23.4% [6m53]
  xz -7: total compressed size 621.82 MiB, 22.6% [7m19]
  xz -8: total compressed size 599.33 MiB, 21.8% [7m18]
  xz -9: total compressed size 588.18 MiB, 21.4% [7m40]

Note that compression takes much longer.  More importantly, however,
decompression is much faster:

  bzip2: 1m47.274s
  xz -6: 0m55.446s
  xz -7: 0m54.119s
  xz -8: 0m52.388s
  xz -9: 0m51.842s

The only downside to using -9 is that decompression takes a fair
amount (~65 MB) of memory.
2012-06-29 15:24:52 -04:00
Eelco Dolstra 4f7bab7db1 Support building with the Perl XS bindings disabled
Since the Perl bindings require shared libraries, this is required on
platforms such as Cygwin where we do a static build.
2012-05-10 19:03:23 -04:00
Eelco Dolstra d03a295192 Require SQLite >= 3.6.19
Nix needs SQLite's foreign key constraint feature, which was
introduced in 3.6.19.  Without it, the database won't be cleaned up
correctly when paths are deleted.  See
e.g. http://hydra.nixos.org/build/2494142.
2012-05-04 17:21:43 -04:00
Eelco Dolstra 969a14599d If "pv" is available at compile time, hard-code its path 2012-04-13 14:41:33 +02:00
Eelco Dolstra 4b23900ff1 Fix the optional dependency on OpenSSL
http://hydra.nixos.org/build/2306540
2012-03-19 01:06:13 +01:00
Eelco Dolstra b006a31d52 Drop the externals directory
Nix now requires SQLite and bzip2 to be pre-installed.  SQLite is
detected using pkg-config.  We required DBD::SQLite anyway, so
depending on SQLite is not a big problem.

The --with-bzip2, --with-openssl and --with-sqlite flags are gone.
2012-03-18 23:54:57 +01:00
Eelco Dolstra bd013b6f98 On Linux, make the Nix store really read-only by using the immutable bit
I was bitten one time too many by Python modifying the Nix store by
creating *.pyc files when run as root.  On Linux, we can prevent this
by setting the immutable bit on files and directories (as in ‘chattr
+i’).  This isn't supported by all filesystems, so it's not an error
if setting the bit fails.  The immutable bit is cleared by the garbage
collector before deleting a path.  The only tricky aspect is in
optimiseStore(), since it's forbidden to create hard links to an
immutable file.  Thus optimiseStore() temporarily clears the immutable
bit before creating the link.
2012-02-15 01:31:56 +01:00
Eelco Dolstra f86fca9b70 * Figure out the extension of dynamic libraries; don't hard-code .so. 2012-01-11 15:41:42 +00:00
Eelco Dolstra a7366a764a * Add a -I flag to the Perl bindings to nix-build and some other
scripts.
* Include the version and architecture in the -I flag so that there is
  at least a chance that a Nix binary built for one Perl version will
  run on another version.
2012-01-05 20:33:46 +00:00
Eelco Dolstra 9d43a02382 * Let --disable-gc work. 2012-01-04 12:45:40 +00:00
Eelco Dolstra 39d45a6b09 * Add a test for nix-channel.
* Refactor the nix-channel unpacker a bit.
2012-01-03 01:51:38 +00:00
Eelco Dolstra 4de3e2a0db * Doh. 2011-11-21 13:22:34 +00:00
Eelco Dolstra 23e933b3b3 * Put back the "sys_name" variable which got removed somewhere. This
broke building on Cygwin and Solaris.
2011-11-21 12:23:48 +00:00
Eelco Dolstra a6abade8e8 * Escape the [ and ] characters in the sed call, otherwise autoconf
will eat them.
2011-11-21 12:18:26 +00:00
Eelco Dolstra 964399c079 * "sed" on FreeBSD doesn't know the "+" operator. 2011-11-20 19:23:50 +00:00
Eelco Dolstra b92f76374f * In the platform, canonicalise "amd64" to "x86_64". FreeBSD 8.2's
uname reports amd64.
* Drop the FreeBSD version number, e.g. "i686-freebsd" instead of
  "i686-freebsd8.2".
2011-11-18 17:25:40 +00:00
Ludovic Courtès 42164d6de4 configure: Change i*86 to i686 as has always been done. 2011-11-16 20:58:21 +00:00
Ludovic Courtès 45ec69cbdf configure: Rely on `AC_CANONICAL_HOST' to determine the Nix system name.
This should be more robust and also plays better with
cross-compilation---it uses the host name, instead of using the build
name.
2011-11-16 20:39:59 +00:00
Eelco Dolstra c0b706213d * Boldly make SQLite WAL mode the default again. Hopefully the
intermittent problems are gone by now.  WAL mode is preferrable
  because it does way fewer fsyncs.
2011-11-07 21:11:59 +00:00
Eelco Dolstra 6fcdbcac20 * Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm under
the Nix:: namespace.
2011-10-10 21:11:08 +00:00
Eelco Dolstra 73fe6871c4 * Include the Nix Perl bindings in Nix itself. This will allow the
bindings to be used in Nix's own Perl scripts.

  The only downside is that Perl XS and Automake/libtool don't really
  like each other, so building is a bit tricky.
2011-10-10 18:12:40 +00:00
Eelco Dolstra d9a5959139 * Show the default for --with-store-dir (Nix/211). 2011-07-13 15:57:44 +00:00
Eelco Dolstra 34f4b91820 2011-07-04 14:44:53 +00:00
Eelco Dolstra 9c99aa2620 2011-06-27 09:01:09 +00:00
Eelco Dolstra e4d8148585 2011-06-27 08:04:54 +00:00
Eelco Dolstra 4891b21f34 * Use SQLite 3.7.6.2. 2011-05-03 16:17:24 +00:00
Eelco Dolstra 3cbf680f5d * configure: detect whether DBD::SQLite is present. If necessary the
location to DBI and DBD::SQLite can be passed with --with-dbi and
  --with-dbd-sqlite.
2011-04-11 10:13:53 +00:00
Eelco Dolstra 538b7caab0 * Don't allocate a big initial GC address space on machines with
little RAM.  Even if the memory isn't actually used, it can cause
  problems with the overcommit heuristics in the kernel.  So use a VM
  space of 25% of RAM, up to 384 MB.
2011-02-10 14:31:04 +00:00
Ludovic Courtès f1462c208b Use $BDW_GC_LIBS instead of a custom variable. 2011-02-09 21:23:13 +00:00
Eelco Dolstra 1876ab764f * A better fix. $boehmgc isn't set anywhere, we should use the flags
returned by pkg-config.
2011-02-09 14:13:09 +00:00
Eelco Dolstra c0340eec5a 2011-02-08 12:30:28 +00:00
Eelco Dolstra c931a7aec5 * Do a short sleep after SQLITE_BUSY. 2010-12-17 17:23:15 +00:00
Eelco Dolstra 4d57776813 * Use SQLite 3.7.4. 2010-12-10 11:45:56 +00:00
Eelco Dolstra fb9368b5a0 * Sync with the trunk. 2010-11-16 12:49:47 +00:00
Eelco Dolstra 3d71c8013e * Use pkgconfig to locate the Boehm GC (as suggested by Ludo), if
--enable-gc is given.
2010-10-29 14:00:47 +00:00
Eelco Dolstra 64c3325b0b * Make building against the Boehm GC a configure option. 2010-10-22 13:39:15 +00:00
Eelco Dolstra 95f4f2cf61 * If std::tr1::unordered_set is unavailable, use std::set. 2010-10-04 16:16:19 +00:00
Eelco Dolstra e41ecbf730 2010-08-27 11:09:04 +00:00
Eelco Dolstra 5fb824e896 * Urgh, this was supposed to go in the trunk... 2010-08-12 13:36:56 +00:00
Eelco Dolstra 5c5ab2bc12 * Don't link against pthreads. This was added way back in r211
because Berkeley DB needed it on some platforms, but we don't use
  BDB anymore.

  On FreeBSD, if you link against pthreads, then the main thread gets
  a 2 MB stack which cannot be overriden (it ignores "ulimit -s"):

    http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg62445.html
  
  This is not enough for Nix.  For instance, the garbage collector can
  fail if there is a pathologically deep chain of references
  (http://hydra.nixos.org/build/556199).  2 MB is also not enough for
  many Nix expressions.
  
  Arguably the garbage collector shouldn't use recursion, because in
  NixOS unprivileged users can DOS the garbage collector by creating a
  sufficiently deeply nested chain of references.  But getting rid of
  recursion is a bit harder.
2010-08-12 13:34:34 +00:00
Eelco Dolstra d7875d1648 * Use SQLite 3.7.0.1. 2010-08-06 07:51:27 +00:00
Eelco Dolstra 587dc8aa00 * Sync with the trunk. 2010-08-04 17:48:29 +00:00
Eelco Dolstra fd9c77dfc7 * Use SQLite 3.7.0's write-ahead logging (WAL mode). This is a lot
faster than the old mode when fsyncs are enabled, because it only
  performs an fsync() when doing a checkpoint, rather than at every
  commit.  Some timings for doing a "nix-instantiate /etc/nixos/nixos
  -A system" after modifying the stdenv setup script:

    42.5s - SQLite 3.6.23 with truncate mode and fsync
     3.4s - SQLite 3.6.23 with truncate mode and no fsync
    32.1s - SQLite 3.7.0 with truncate mode and fsync
    16.8s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
            every 1000 pages
     8.3s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
            every 8192 pages
     1.7s - SQLite 3.7.0 with WAL mode and no fsync

  The default is now to use WAL mode with fsyncs.  Because WAL doesn't
  work on remote filesystems such as NFS (as it uses shared memory),
  truncate mode can be re-enabled by setting the "use-sqlite-wal"
  option to false.
2010-08-04 17:35:59 +00:00
Peter Simons a0d29040f7 Revert "configure.ac: make flex and bison required programs"
This reverts commit 22405. Apparently, these programs aren't necessarily
required when building from a release archive.
2010-06-24 22:22:24 +00:00
Peter Simons af09fe12dd Fix build of bsdiff-4.3 on machines that don't have <err.h>, such as Solaris. 2010-06-24 17:51:31 +00:00
Peter Simons d63375d529 configure.ac: Incredibly enough, tr(1) on Solaris doesn't understand A-Z syntax for ranges. 2010-06-24 17:51:24 +00:00
Peter Simons 4c21c016c5 configure.ac: make flex and bison required programs
The build fails if these tools aren't available.
2010-06-24 17:51:19 +00:00
Eelco Dolstra aa45027818 * Sync with the trunk. 2010-05-12 22:13:09 +00:00
Eelco Dolstra d66ea83a76 * Sync with the trunk. 2010-04-20 09:20:29 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Eelco Dolstra f71ea9c911 * Use SQLite 3.6.23. 2010-03-23 10:38:59 +00:00
Lluís Batlle i Rossell 13cce8ec45 Making 'bin2c' to be built with the compiler for the local system.
I copied the configure.ac code about CC_FOR_BUILD from libX11.
2010-03-11 20:56:25 +00:00
Eelco Dolstra 070057c1b9 * Sync with the trunk. 2010-03-11 10:52:52 +00:00
Eelco Dolstra 05fbf61f0e * nix-prefetch-url depends on sed (see
http://hydra.nixos.org/build/311170).
2010-03-05 18:26:47 +00:00
Eelco Dolstra d8c5745c41 * Support static builds. 2010-03-02 17:12:47 +00:00
Eelco Dolstra 5414b3b2db * Remove the --enable-static-nix flag. 2010-03-02 16:40:40 +00:00
Eelco Dolstra 594eaddd11 * When using the included sqlite/aterm libraries, build with
--enable-shared.
* In libutil/libstore/libexpr etc., link against sqlite and aterm.
* Some more header file hygiene.
2010-03-02 15:58:13 +00:00
Eelco Dolstra 84a4dd5ff0 * Don't use fdatasync since it doesn't work on Snow Leopard.
* Don't refer to config.h in util.hh, because config.h is not
  installed (http://hydra.nixos.org/build/303053).
2010-02-24 15:46:06 +00:00
Eelco Dolstra fae0427324 * ADDITIONAL_NETWORK_LIBS -> LIBS. 2010-02-24 12:25:48 +00:00
Eelco Dolstra 5954eadf67 * Remove the fdatasync check since it's no longer needed. 2010-02-24 12:16:50 +00:00
Eelco Dolstra b4e6d98fc3 * configure: flag --with-sqlite. 2010-02-23 22:12:46 +00:00
Eelco Dolstra c1a07f9445 * Convert the Nix database to SQLite. 2010-02-18 13:16:59 +00:00
Eelco Dolstra 2723d9b56e * If fdatasync() isn't available, use fsync(). 2010-02-02 11:57:49 +00:00
Eelco Dolstra 327a232c85 * Remove support for old (before Nix 0.12pre12020) databases. 2009-11-06 01:15:44 +00:00
Sander van der Burg 53a4981fa2 Added optional parameter which adds -lnsl -lsocket to make the Nix package manager work on OpenSolaris 2009-10-08 14:50:37 +00:00
Peter Simons 18f0ff003d configure.ac: use AC_SYS_LARGEFILE to determine how to enable 64-bit file size support
Defining -D_FILE_OFFSET_BITS=64 works on most platforms, but not on all (i.e.
Solaris). Also, the Autoconf macro offers the user a switch to disable the
functionality in case of problems.
2009-10-06 09:14:06 +00:00
Eelco Dolstra 51ad64cc07 * Use xmllint (>= 2.7.4) for RelaxNG validation instead of Jing. 2009-09-18 11:45:56 +00:00
Eelco Dolstra 6a5599fd01 * Typo. 2009-05-07 11:33:57 +00:00
Eelco Dolstra 9856efc7b0 * Support doing a fully static build. Statically linked Nix binaries
will be useful as a fallback on platforms for which there are no
  RPMs/Debs available.
2009-03-26 12:22:08 +00:00
Eelco Dolstra 77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
2009-03-22 17:36:43 +00:00
Eelco Dolstra be88248add * Make the version available to release.nix. 2009-03-09 15:05:08 +00:00
Eelco Dolstra c504d90c11 * Support i686-linux builds directly on x86_64-linux Nix
installations.  This is implemented using the personality() syscall,
  which causes uname to return "i686" in child processes.
2009-01-12 16:30:32 +00:00
Eelco Dolstra ac36c6cd44 * Some hackery to make "make check" succeed on Cygwin. 2008-12-12 15:36:18 +00:00
Eelco Dolstra 2b70a8e7c9 * Detect whether unshare() is available. 2008-12-12 13:41:36 +00:00
Eelco Dolstra 7c54f1603f * Do chroot builds in a private namespace. This means that all the
bind-mounts we do are only visible to the builder process and its
  children.  So accidentally doing "rm -rf" on the chroot directory
  won't wipe out /nix/store and other bind-mounted directories
  anymore.  Also, the bind-mounts in the private namespace disappear
  automatically when the builder exits.
2008-12-11 17:00:12 +00:00
Eelco Dolstra 5d4eb9dd07 * Moved the build farm job for building Nix to the Nix tree. 2008-11-26 01:13:29 +00:00
Eelco Dolstra 2ab09a55cf * Bump the version number. 2008-11-20 21:51:58 +00:00
Eelco Dolstra 285d26374a * Don't set the prefix to /nix by default, rather use the Autoconf
default of /usr/local.  However, localstatedir and storedir are set
  to /nix/var/nix and /nix/store respectively unless they're
  explicitly overriden.
2008-11-20 14:14:35 +00:00
Eelco Dolstra b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00
Michael Raskin b4bc8b7616 --proxy=proxy:3128 2008-05-07 14:18:28 +00:00
Eelco Dolstra 658816ddc9 * Make really sure that we use bash. The line
NEED_PROG(shell, bash)

  actually uses the content of $shell if set, which often points at
  /bin/sh.
2008-04-10 09:54:23 +00:00