Commit graph

812 commits

Author SHA1 Message Date
Robert Hensing a05bc9eb92 allowed-uris: Match whole schemes also when scheme is not followed by slashes 2023-12-11 12:18:04 +01:00
Robert Hensing 1fa958dda1 isAllowedURI: Format 2023-12-11 12:12:43 +01:00
Robert Hensing 6cbba914a7 isAllowedURI: Remove incorrect note 2023-12-11 12:12:43 +01:00
Robert Hensing 91ba7b2307 isAllowedURI: Extract function and test 2023-12-11 12:12:42 +01:00
Robert Hensing c8458bd731
Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating
2023-12-09 03:55:58 +01:00
Robert Hensing 9b7b7a7561 Revert "Print the value in error: cannot coerce messages (#9553)"
This reverts commit f0ac2a35d5.

The request from the sibling PR, which also applies here, was not addressed.
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735
2023-12-09 02:13:32 +01:00
Robert Hensing d4f6b1d38b
Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
John Ericson f9ee1bedcf Avoid std::strstream, fix the clang build
According https://en.cppreference.com/w/cpp/io/strstream, it has been
deprecated since C++98! The Clang + Linux build systems to not have it
at all, or at least be hiding it.

We can just use `std::stringstream` instead, I think.
2023-12-08 13:18:52 -05:00
Rebecca Turner f0ac2a35d5
Print the value in error: cannot coerce messages (#9553)
* Print the value in `error: cannot coerce` messages

This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-08 16:36:57 +00:00
Rebecca Turner 0b80935c22
Pass positions when evaluating
This includes position information in more places, making debugging
easier.

Before:

```
$ nix-instantiate --show-trace --eval tests/functional/lang/eval-fail-using-set-as-attr-name.nix
error:
       … while evaluating an attribute name

         at «none»:0: (source not available)

       error: value is a set while a string was expected
```

After:

```
error:
       … while evaluating an attribute name

         at /pwd/lang/eval-fail-using-set-as-attr-name.nix:5:10:

            4| in
            5|   attr.${key}
             |          ^
            6|

       error: value is a set while a string was expected
```
2023-12-07 10:27:21 -08:00
Eelco Dolstra 8cafc754d8 Move access control from FSInputAccessor to FilteringInputAccessor 2023-11-30 21:54:53 +01:00
Eelco Dolstra ea95327e72 Move restricted/pure-eval access control out of the evaluator and into the accessor 2023-11-30 16:16:17 +01:00
Eelco Dolstra cb7f25869d
Merge pull request #9430 from hercules-ci/remove-vlas
Fix stack overflow in `filter`
2023-11-30 15:31:44 +01:00
Robert Hensing 4e790efade Use boost::container::small_vector in place of VLAs 2023-11-21 20:52:27 +01:00
Eelco Dolstra a0162d5732 Improve SourceAccessor path display
Backported from lazy-trees. This allows SourceAccessors to show the
origin of the accessor. E.g. we now get

  copying '«git+https://github.com/blender/blender.git?ref=refs/heads/main&rev=4edc1389337dd3679ff66969c332d2aff52e1992»/' to the store

instead of

  copying '/' to the store
2023-11-20 18:54:36 +01:00
Eelco Dolstra 1d6abec993 Revert use of boost::container::small_vector in the evaluator
It caused random crashes (https://hydra.nixos.org/build/241514506,
https://hydra.nixos.org/build/241443330) because the heap allocation
done by small_vector in the not-small case is not scanned for GC
roots.
2023-11-20 12:35:35 +01:00
Robert Hensing 4a539ac3ea Fix buildNoGc
Fixes https://hydra.nixos.org/build/241067941/nixlog/1

    src/libexpr/eval.cc:1776:54: error: variable 'boost::container::small_vector<nix::Value*, 4> vArgs' has initializer but incomplete type
2023-11-17 17:38:08 +01:00
Robert Hensing 1b9813e4e6 primops: Name stack reservation limits 2023-11-16 15:38:21 +01:00
Robert Hensing 898c47384f primops: Err on the side of less stack usage
Try to stay away from stack overflows.

These small vectors use stack space. Most instances will not need
to allocate because in general most things are small, and large
things are worth heap allocating.

16 * 3 * word = 384 bytes is still quite a bit, but these functions
tend not to be part of deep recursions.
2023-11-16 15:38:21 +01:00
Robert Hensing 91114a6fa4 ExprCall::eval: Heap allocate at arity 5+ 2023-11-16 15:38:21 +01:00
Robert Hensing 0daccb1121 libexpr: Check primop arity earlier 2023-11-16 12:28:32 +01:00
Théophane Hufschmitt ba3cb4a049 Remove all the occurences of VLAs
There's generally no strict reason for using them, and they are somewhat
fishy, so let's avoid them.
2023-11-16 12:27:30 +01:00
John Ericson ac89bb064a Split up util.{hh,cc}
All OS and IO operations should be moved out, leaving only some misc
portable pure functions.

This is useful to avoid copious CPP when doing things like Windows and
Emscripten ports.

Newly exposed functions to break cycles:

 - `restoreSignals`
 - `updateWindowSize`
2023-11-05 12:20:02 -05:00
Eelco Dolstra bcf5c31950 Add future FIXME 2023-10-20 16:58:33 +02:00
Eelco Dolstra 173abec0bc coerceToPath(): Handle __toString, add tests 2023-10-20 13:06:44 +02:00
Eelco Dolstra 5be7705ddf Remove stuff we don't need yet 2023-10-19 19:20:21 +02:00
Eelco Dolstra f16af08e83 Fix macOS compilation 2023-10-19 14:20:50 +02:00
Eelco Dolstra 12214fef09 InputAccessor::fetchToStore(): Support arbitrary ingestion methods 2023-10-19 14:19:10 +02:00
Eelco Dolstra df73c6eb8c Introduce MemoryInputAccessor and use it for corepkgs
MemoryInputAccessor is an in-memory virtual filesystem that returns
files like <nix/fetchurl.nix>. This removes the need for special hacks
to handle those files.
2023-10-18 17:38:11 +02:00
Eelco Dolstra ea38605d11 Introduce FSInputAccessor and use it
Backported from the lazy-trees branch. Note that this doesn't yet use
the access control features of FSInputAccessor.
2023-10-18 17:37:32 +02:00
Jacek Galowicz 54b350d517 Drop some moves that would happen anyway but forbid NRVO where appicable 2023-10-16 21:48:35 +01:00
Jacek Galowicz abf7df2b37 Fix moves that accidentally copy anyway 2023-10-16 21:48:35 +01:00
Robert Hensing c32084a12c printStats -> maybePrintStats 2023-10-09 16:34:35 +02:00
Robert Hensing f95364a803 eval: Run a full GC before printing stats
This makes the numbers more deterministic, especially when it comes
to the final heap size.
2023-10-06 00:21:05 +02:00
Tom Bereknyei 399ef84420 refactor: use string accessors
Create context, string_view, and c_str, accessors throughout in order to
better support improvements to the underlying string representation.
2023-09-27 00:33:01 -04:00
Robert Hensing 3720e811fa libexpr: Add nrExprs to NIX_SHOW_STATS 2023-09-12 13:21:55 +02:00
John Ericson fe71faa920 Delete EvalState::addToSearchPath
This function is now trivial enough that it doesn't need to exist.

`EvalState` can still be initialized with a custom search path, but we
don't have a need to mutate the search path after it has been
constructed, and I don't see why we would need to in the future.

Fixes #8229
2023-08-18 14:04:33 -04:00
John Ericson 9121fed4b4 Fixing #7479
Types converted:

- `NixStringContextElem`
- `OutputsSpec`
- `ExtendedOutputsSpec`
- `DerivationOutput`
- `DerivationType`

Existing ones mostly conforming the pattern cleaned up:

- `ContentAddressMethod`
- `ContentAddressWithReferences`

The `DerivationGoal::derivationType` field had a bogus initialization,
now caught, so I made it `std::optional`. I think #8829 can make it
non-optional again because it will ensure we always have the derivation
when we construct a `DerivationGoal`.

See that issue (#7479) for details on the general goal.

`git grep 'Raw::Raw'` indicates the two types I didn't yet convert
`DerivedPath` and `BuiltPath` (and their `Single` variants) . This is
because @roberth and I (can't find issue right now...) plan on reworking
them somewhat, so I didn't want to churn them more just yet.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-08-18 11:44:00 -04:00
John Ericson e7c39ff00b Rework evaluator SingleDerivedPath infra
`EvalState::mkSingleDerivedPathString` previously contained its own
inverse (printing, rather than parsing) in order to validate what was
parsed. Now that is pulled out into its own separate function:
`EvalState::coerceToSingleDerivedPath`.

In additional that pulled out logic is deduplicated with
`EvalState::mkOutputString` via `EvalState::mkOutputStringRaw`, which is
itself deduplicated (and generalized) with
`DownstreamPlaceholder::mkOutputStringRaw`.

All these changes make the unit tests simpler.

(We would ideally write more unit tests for `mkSingleDerivedPathString`
`coerceToSingleDerivedPath` directly, but we cannot yet do that because
the IO in reading the store path won't work when the dummy store cannot
hold anything. Someday we'll have a proper in-memory store which will
work for this.)

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-14 08:44:50 -04:00
John Ericson a04720e68c Rename optOutputPath to optStaticOutputPath
This choice of variable name makes it more clear what is going on.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-14 08:44:48 -04:00
John Ericson 60b7121d2c Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).

To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.

`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!

Important note: some JSON formats have changed.

We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-08-10 00:08:32 -04:00
Théophane Hufschmitt 5df0f1755f
Merge pull request #8692 from obsidiansystems/add-another-xp-check
Feature gate `DownstreamPlaceholder::unknownCaOutput`
2023-08-07 13:11:44 +02:00
John Ericson 1570e80219 Move evaluator settings (type and global) to separate file/header 2023-07-31 10:14:15 -04:00
John Ericson caabc4f648 Feature gate DownstreamPlaceholder::unknownCaOutput
This is a part of CA derivations that we forgot to put behind the
experimental feature.

This was caught by @fricklerhandwerk in
https://github.com/NixOS/nix/pull/8369#discussion_r1258133719
2023-07-13 07:56:33 -04:00
John Ericson be518e73ae Clean up SearchPath
- Better types

- Own header / C++ file pair

- Test factored out methods

- Pass parsed thing around more than strings

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-07-09 23:22:22 -04:00
John Ericson 87dcd09047 Clean up resolveSearchPathElem
We should use `std::optional<std::string>` not `std::pair<bool,
std::string>` for an optional string.
2023-07-09 23:13:30 -04:00
Robert Hensing 7b39a388b3
Merge pull request #8566 from inclyc/nixd/value-print-depth
libexpr: extend `Value::print` to allow limited depth
2023-07-01 20:08:52 +02:00
John Ericson 22b278e011 Automatically document builtin constants
This is done in roughly the same way builtin functions are documented.

Also auto-link experimental features for primops, subsuming PR #8371.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-27 09:37:54 -04:00
John Ericson e8067daf09 Generialize showType 2023-06-27 09:11:42 -04:00
Yingchi Long 1400fde144 libexpr: extend Value::print to allow limited depth 2023-06-22 18:28:30 +08:00