Commit graph

16019 commits

Author SHA1 Message Date
John Ericson 4580bed3e4 LegacySSHStore::openConnection move more logic inside catch block
Broader error handling logic is more robust.
2024-01-22 12:43:11 -05:00
John Ericson ce2f714e6d Start factoring out the serve protocol for Hydra to share
Factor out `ServeProto::BasicClientConnection` for Hydra to share

- `queryValidPaths`: Hydra uses the lock argument differently than Nix,
  so we un-hard-code it.

- `buildDerivationRequest`: Just the request half, as Hydra does some
  things between requesting and responding.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:03 -05:00
John Ericson 50ce3832dc
Merge pull request #9512 from hercules-ci/release-notes-bdwgc-traceable-allocator-patch
prerequisites-source: Add bdwgc-traceable-allocator-patch
2024-01-22 11:58:43 -05:00
John Ericson b74ec56da9
Merge pull request #9829 from obsidiansystems/fix-if-if-else-ambiguity
If `if`...`if`...`else` ambiguity
2024-01-22 10:58:46 -05:00
John Ericson 316e50cc7c Fix if...if...else ambiguity
This can be parsed two ways. Add a pair of braces so it must be parsed
the intended way.
2024-01-22 10:33:40 -05:00
John Ericson 176dcd5c61
Merge pull request #9626 from jvns/locked-input
Make fetchTree locked input error message clearer
2024-01-22 09:31:59 -05:00
Eelco Dolstra 16ccca2e86
Merge pull request #9703 from fricklerhandwerk/upgrade-no-channel
don't show `nix-channel` in upgrade instructions
2024-01-22 14:46:47 +01:00
John Ericson b2066c3574
Merge pull request #9521 from fricklerhandwerk/conf-reword-max-jobs
reword description of the `max-jobs` setting
2024-01-22 08:29:12 -05:00
John Ericson a0b86b3160
Merge pull request #9760 from hercules-ci/doc-glossary-nix-expression
doc/glossary: Nix expression can be language expression
2024-01-22 08:26:01 -05:00
Théophane Hufschmitt c8d33bca8d
Merge pull request #9822 from obsidiansystems/algo-vs-hash-algo
Start standardizing hash algo flags
2024-01-22 11:08:24 +01:00
Théophane Hufschmitt f51409cf98
Merge pull request #9805 from yshui/prefetch-unpack
Add --unpack to nix store prefetch-file
2024-01-22 10:51:50 +01:00
John Ericson da1aae2d06
Merge pull request #9821 from obsidiansystems/fix-typo
Fix typo in upcomming release notes
2024-01-20 18:03:25 -05:00
John Ericson 202c5e2afc Start standardizing hash algo flags
Do this if we want to do `--hash-algo` everywhere, and not `--algo` for
hash commands.

The new `nix hash convert` is updated. Deprecated new CLI commands are
left as-is (`nix hash path` needs to be redone and is also left as-is).
2024-01-20 17:29:35 -05:00
John Ericson a93b204c27
Merge pull request #9348 from obsidiansystems/json-formats
Document JSON formats
2024-01-20 17:22:39 -05:00
John Ericson 65294fe5fe Fix typo in upcomming release notes
Thanks @cole-h for finding in https://github.com/NixOS/nix/pull/9815#discussion_r1460604130
2024-01-20 17:07:21 -05:00
John Ericson edf3ecc497 Document JSON formats
Good to document these formats separately from commands that happen to
use them.

Eventually I would like this and `builtins.derivation` to refer to a
store section on derivations that is authoritative, but that doesn't yet
exist, and will take some time to make. So I think we're just best off
merging this now as is.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-20 17:03:47 -05:00
John Ericson 9b896bf7e9
Merge pull request #9809 from obsidiansystems/nix-store-add-algo
Add missing `--hash-algo` flag to `nix store add`
2024-01-19 23:37:11 -05:00
John Ericson 356352c370 Add missing --hash-algo flag to nix store add 2024-01-19 23:11:18 -05:00
Robert Hensing 382fa51ff0
Merge pull request #9794 from hercules-ci/queryPathInfoFromClientCache
refactor: Extract `Store::queryPathInfoFromClientCache`
2024-01-19 17:33:52 +01:00
Robert Hensing d19627e8b4 refactor: Remove throw from queryPathInfoFromClientCache
Return a value instead of throwing.

Rather than the more trivial refactor of wrapping the return value in
another std::optional, we retain the meaning of the outer optional:
"we know at least something."

So we have changed:
return nullopt    -> return nullopt
throw InvalidPath -> return make_optional(nullptr)
return vpi        -> return make_optional(vpi)
2024-01-19 17:01:19 +01:00
Robert Hensing 8983ee8b2e refactor: Un-callback transform Store::queryPathInfoFromClientCache
This part of the code was not necessarily callback based.
Removing CPS is always nice; particularly if there's no loss of
functionality, like here.
2024-01-19 17:01:19 +01:00
Yuxuan Shui 75a6e6dd0e Add --unpack to nix store prefetch-file 2024-01-19 15:53:45 +00:00
Eelco Dolstra e6e160a075
Merge pull request #9632 from cole-h/nix-daemon-testing
Use `nix daemon` in the test suite
2024-01-19 16:21:51 +01:00
Théophane Hufschmitt 10165c7593
Merge pull request #9811 from DavHau/fix-git-auth
fetchTree/fetchGit: re-enable shallow fetching
2024-01-19 16:14:41 +01:00
John Ericson 57d6284eb7
Merge pull request #9812 from NixOS/remove-n-shorthand-for---mode
Remove a nonsensical shorthand flag in `nix store add`
2024-01-19 08:34:43 -05:00
DavHau bc00fa4647 fetchTree/fetchGit: re-enable shallow fetching
Add several tests for git fetching:
- shallow-cache-separation: can fetch the same repo shallowly and non-shallowly
- shallow-ignore-ref: ensure that ref gets ignored when shallow=true is set
- ssh-shallow: can fetch a git repo via ssh using shallow=1
2024-01-19 20:30:47 +07:00
Robert Hensing d762caff46
Merge pull request #9806 from hercules-ci/fix-git-auth
Fix git auth
2024-01-19 13:47:52 +01:00
Théophane Hufschmitt 28d7db249a
Remove a nonsensical shorthand flag in nix store add
`-n` was an alias for `--mode`, but that seems to just be a copy-paste error as it doesn't make sense.
`--mode` probably doesn't need a shorthand flag at all, so remove it.

Noticed in https://github.com/NixOS/nix/pull/9809#issuecomment-1899890555
2024-01-19 10:10:00 +01:00
Eelco Dolstra 3b20cca962
Merge pull request #9804 from edolstra/missing-nar-crash
Fix crash when NAR is missing from binary cache
2024-01-19 09:38:53 +01:00
Robert Hensing 4e8483d09d
Merge pull request #9792 from fricklerhandwerk/move-section
move section on make variables
2024-01-19 00:55:33 +01:00
Robert Hensing 14f470ec4e
doc/hacking.md: Hint short option make -e
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-19 00:32:30 +01:00
Robert Hensing 346d513d86 tests/nixos/fetch-git: Add http-auth test 2024-01-18 22:34:38 +01:00
Robert Hensing 8d422c2fef Revert libgit2 fetching
libgit2 is not capable of using git-credentials helpers yet.
This prevents private repositories from being used.

Based on code that was replaced in https://github.com/NixOS/nix/pull/9240
(Introduce libgit2); hence:

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-01-18 22:29:26 +01:00
Robert Hensing 76a50b3a69 doc: GitRepoImpl::path 2024-01-18 22:29:26 +01:00
Robert Hensing ed975e953c tests/nixos/fetch-git: Testsupport for private repos 2024-01-18 22:29:26 +01:00
Robert Hensing 0bd9e10aea
Merge pull request #9676 from DavHau/git-testsuite
initialize test suite for git fetchers
2024-01-18 21:51:48 +01:00
Robert Hensing 1254170405 tests/nixos/fetch-git: Make the store paths unique 2024-01-18 21:00:40 +01:00
Robert Hensing 94eba0ebbb tests/nixos/fetch-git: Memoize -> save
Memoization is for thunk-like behavior whereas this is executed eagerly.
2024-01-18 21:00:40 +01:00
Robert Hensing fd41979d78 tests/nixos/fetch-git: Factor out gitea repo module 2024-01-18 21:00:40 +01:00
Eelco Dolstra dca0a80240 copyStorePath(): Bail out early if the store path already exists
In rare cases (e.g. when using allowSubstitutes = false), it's
possible that we simultaneously have a DerivationGoal *and* a
SubstitutionGoal building the same path. So if a DerivationGoal
already built the path while the SubstitutionGoal was waiting for a
download slot, it saves us a superfluous download to exit early.
2024-01-18 17:19:15 +01:00
Eelco Dolstra a18d8d688a LocalStore::addToStore(): Ignore exceptions from parseDump()
In the "discard" case (i.e. when the store path already exists
locally), when we call parseDump() from a Finally and it throws an
exception (e.g. if the download of the NAR fails), Nix crashes:

   terminate called after throwing an instance of 'nix::SubstituteGone'
     what():  error: file 'nar/06br3254rx4gz4cvjzxlv028jrx80zg5i4jr62vjmn416dqihgr7.nar.xz' does not exist in binary cache 'http://localhost'
   Aborted (core dumped)
2024-01-18 17:19:15 +01:00
Eelco Dolstra ab786e22f1 Show what goal is waiting for a build slot 2024-01-18 17:19:15 +01:00
Théophane Hufschmitt e652322a61
Merge pull request #9799 from obsidiansystems/push-add-to-store-from-dump-unsupported-down
Push `addToStoreFromDump` `unsupported(...)` down `Store` class hierarchy
2024-01-18 16:54:44 +01:00
Eelco Dolstra 32706b14a7
Merge pull request #9798 from edolstra/remote-store-eof
Print a more helpful message if the daemon crashes
2024-01-18 15:47:59 +01:00
Eelco Dolstra 1fe8f54bd3 Use BackedStringView 2024-01-18 15:27:57 +01:00
Robert Hensing 78074bdea4 tests/nixos/fetch-git: Apply suggestions 2024-01-18 12:53:53 +01:00
Théophane Hufschmitt 28674247ec
Merge pull request #9787 from obsidiansystems/bind-proc-syserror
`bind`: give same treatment as `connect` in #8544, dedup
2024-01-18 09:34:15 +01:00
John Ericson 574db83504 Push addToStoreFromDump unsupported(...) down Store class hierarchy
Instead of having it be the default method in `Store` itself, have it be
the implementation in `DummyStore` and `LegacySSHStore`. Then just the
implementations which fail to provide the method pay the "penalty" of
dealing with the icky `unimplemented` function for non-compliance.

Picks up where #8217. Getting close to no `unsupported` in the `Store`
interface itself!

More progress on issue #5729.
2024-01-17 23:46:03 -05:00
John Ericson 3016e67c21 bind: give same treatment as connect in #8544, dedup
It is good to propagate the underlying error so whether or not we use a
process to deal with path length issues is not observable.

Also, as these wrapper functions got more and more complex, the code
duplication got worse and worse. The new `bindConnectProcHelper`
function deduplicates them.
2024-01-17 23:35:08 -05:00
Eelco Dolstra a3cf27ca47 Print a more helpful message if the daemon crashes
Instead of

   error: unexpected end-of-file

you now get

   error: Nix daemon disconnected unexpectedly (maybe it crashed?)
2024-01-17 22:54:02 +01:00