TOREMOVE: readme

CanonPath, SourcePath: Change operator + to /

README.md

@@ -1,34 +1,19 @@
-# Nix
+# Nix Profiled With Tracy
 
-[![Open Collective supporters](https://opencollective.com/nixos/tiers/supporter/badge.svg?label=Supporters&color=brightgreen)](https://opencollective.com/nixos)
-[![Test](https://github.com/NixOS/nix/workflows/Test/badge.svg)](https://github.com/NixOS/nix/actions)
+## Profiling Idea
 
-Nix is a powerful package manager for Linux and other Unix systems that makes package
-management reliable and reproducible. Please refer to the [Nix manual](https://nixos.org/nix/manual)
-for more details.
+TODO:  explain
 
-## Installation and first steps
+- Tracing thunk force operations.
+- Eval is single-threaded and recursive: easy to build stack frames
+  out of global eval state.
+- Profiling infos are streamed to tracy server in that prototype.
+- In previous iterations, we tried:
+    - keep the profile in memory and save it to memory in a previous iteration. See [andir's version](https://git.alternativebit.fr/picnoir/Nix/src/branch/pic/tracy-on-top-of-andi) for code/more infos.
+    - eBPF approach using USDTs (through bcc). We kept over-running the kernel/userspace ring buffer. See [here](https://git.alternativebit.fr/picnoir/Nix/src/branch/nin/usdt-rust) for more infos/code.
 
-Visit [nix.dev](https://nix.dev) for [installation instructions](https://nix.dev/tutorials/install-nix) and [beginner tutorials](https://nix.dev/tutorials/first-steps).
+## TODOs
 
-Full reference documentation can be found in the [Nix manual](https://nixos.org/nix/manual).
-
-## Building And Developing
-
-See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
- set up a development environment and build Nix from source.
-
-## Contributing
-
-Check the [contributing guide](./CONTRIBUTING.md) if you want to get involved with developing Nix.
-
-## Additional Resources
-
-- [Nix manual](https://nixos.org/nix/manual)
-- [Nix jobsets on hydra.nixos.org](https://hydra.nixos.org/project/nix)
-- [NixOS Discourse](https://discourse.nixos.org/)
-- [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org)
-
-## License
-
-Nix is released under the [LGPL v2.1](./COPYING).
+- Add config flag to enable tracing.
+- Remove tracy submodule approach, inject it through Nix.
+-

doc/manual/src/installation/upgrading.md

@@ -16,7 +16,7 @@ nix (Nix) 2.18.1
 > Writing to the [local store](@docroot@/store/types/local-store.md) with a newer version of Nix, for example by building derivations with [`nix-build`](@docroot@/command-ref/nix-build.md) or [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md), may change the database schema!
 > Reverting to an older version of Nix may therefore require purging the store database before it can be used.
 
-### Linux multi-user
+## Linux multi-user
 
 ```console
 $ sudo su

doc/manual/src/language/import-from-derivation.md

@@ -1,6 +1,8 @@
 # Import From Derivation
 
-The value of a Nix expression can depend on the contents of a [store object](@docroot@/glossary.md#gloss-store-object).
+The value of a Nix expression can depend on the contents of a [store object].
+
+[store object]: @docroot@/glossary.md#gloss-store-object
 
 Passing an expression `expr` that evaluates to a [store path](@docroot@/glossary.md#gloss-store-path) to any built-in function which reads from the filesystem constitutes Import From Derivation (IFD):
 

doc/manual/src/language/operators.md

@@ -84,7 +84,7 @@ The `+` operator is overloaded to also work on strings and paths.
 >
 > *string* `+` *string*
 
-Concatenate two [string]s and merge their string contexts.
+Concatenate two [strings][string] and merge their string contexts.
 
 [String concatenation]: #string-concatenation
 
@@ -94,7 +94,7 @@ Concatenate two [string]s and merge their string contexts.
 >
 > *path* `+` *path*
 
-Concatenate two [path]s.
+Concatenate two [paths][path].
 The result is a path.
 
 [Path concatenation]: #path-concatenation
@@ -150,9 +150,9 @@ If an attribute name is present in both, the attribute value from the latter is
 
 Comparison is
 
-- [arithmetic] for [number]s
-- lexicographic for [string]s and [path]s
-- item-wise lexicographic for [list]s:
+- [arithmetic] for [numbers][number]
+- lexicographic for [strings][string] and [paths][path]
+- item-wise lexicographic for [lists][list]:
   elements at the same index in both lists are compared according to their type and skipped if they are equal.
 
 All comparison operators are implemented in terms of `<`, and the following equivalencies hold:
@@ -163,12 +163,12 @@ All comparison operators are implemented in terms of `<`, and the following equi
 | *a* `>`  *b* |       *b* `<` *a*     |
 | *a* `>=` *b* | `! (` *a* `<` *b* `)` |
 
-[Comparison]: #comparison-operators
+[Comparison]: #comparison
 
 ## Equality
 
-- [Attribute sets][attribute set] and [list]s are compared recursively, and therefore are fully evaluated.
-- Comparison of [function]s always returns `false`.
+- [Attribute sets][attribute set] and [lists][list] are compared recursively, and therefore are fully evaluated.
+- Comparison of [functions][function] always returns `false`.
 - Numbers are type-compatible, see [arithmetic] operators.
 - Floating point numbers only differ up to a limited precision.
 

src/libexpr/eval.cc

@@ -2689,14 +2689,14 @@ SourcePath resolveExprPath(SourcePath path)
         // Basic cycle/depth limit to avoid infinite loops.
         if (++followCount >= maxFollow)
             throw Error("too many symbolic links encountered while traversing the path '%s'", path);
-        auto p = path.parent().resolveSymlinks() + path.baseName();
+        auto p = path.parent().resolveSymlinks() / path.baseName();
         if (p.lstat().type != InputAccessor::tSymlink) break;
         path = {path.accessor, CanonPath(p.readLink(), path.path.parent().value_or(CanonPath::root))};
     }
 
     /* If `path' refers to a directory, append `/default.nix'. */
     if (path.resolveSymlinks().lstat().type == InputAccessor::tDirectory)
-        return path + "default.nix";
+        return path / "default.nix";
 
     return path;
 }

src/libexpr/primops.cc

@@ -1816,7 +1816,7 @@ static void prim_readDir(EvalState & state, const PosIdx pos, Value * * args, Va
             // detailed node info quickly in this case we produce a thunk to
             // query the file type lazily.
             auto epath = state.allocValue();
-            epath->mkPath(path + name);
+            epath->mkPath(path / name);
             if (!readFileType)
                 readFileType = &state.getBuiltin("readFileType");
             attr.mkApp(readFileType, epath);

src/libexpr/print.cc

@@ -152,7 +152,7 @@ struct ImportantFirstAttrNameCmp
     }
 };
 
-typedef std::set<Value *> ValuesSeen;
+typedef std::set<const void *> ValuesSeen;
 
 class Printer
 {
@@ -255,14 +255,14 @@ private:
             output << "»";
             if (options.ansiColors)
                 output << ANSI_NORMAL;
-        } catch (BaseError & e) {
+        } catch (Error & e) {
             printError_(e);
         }
     }
 
     void printAttrs(Value & v, size_t depth)
     {
-        if (seen && !seen->insert(&v).second) {
+        if (seen && !seen->insert(v.attrs).second) {
             printRepeated();
             return;
         }
@@ -405,7 +405,7 @@ private:
             output << ANSI_NORMAL;
     }
 
-    void printError_(BaseError & e)
+    void printError_(Error & e)
     {
         if (options.ansiColors)
             output << ANSI_RED;
@@ -422,7 +422,7 @@ private:
         if (options.force) {
             try {
                 state.forceValue(v, v.determinePos(noPos));
-            } catch (BaseError & e) {
+            } catch (Error & e) {
                 printError_(e);
                 return;
             }

src/libfetchers/filtering-input-accessor.cc

@@ -5,26 +5,26 @@ namespace nix {
 std::string FilteringInputAccessor::readFile(const CanonPath & path)
 {
     checkAccess(path);
-    return next->readFile(prefix + path);
+    return next->readFile(prefix / path);
 }
 
 bool FilteringInputAccessor::pathExists(const CanonPath & path)
 {
-    return isAllowed(path) && next->pathExists(prefix + path);
+    return isAllowed(path) && next->pathExists(prefix / path);
 }
 
 std::optional<InputAccessor::Stat> FilteringInputAccessor::maybeLstat(const CanonPath & path)
 {
     checkAccess(path);
-    return next->maybeLstat(prefix + path);
+    return next->maybeLstat(prefix / path);
 }
 
 InputAccessor::DirEntries FilteringInputAccessor::readDirectory(const CanonPath & path)
 {
     checkAccess(path);
     DirEntries entries;
-    for (auto & entry : next->readDirectory(prefix + path)) {
-        if (isAllowed(path + entry.first))
+    for (auto & entry : next->readDirectory(prefix / path)) {
+        if (isAllowed(path / entry.first))
             entries.insert(std::move(entry));
     }
     return entries;
@@ -33,12 +33,12 @@ InputAccessor::DirEntries FilteringInputAccessor::readDirectory(const CanonPath
 std::string FilteringInputAccessor::readLink(const CanonPath & path)
 {
     checkAccess(path);
-    return next->readLink(prefix + path);
+    return next->readLink(prefix / path);
 }
 
 std::string FilteringInputAccessor::showPath(const CanonPath & path)
 {
-    return next->showPath(prefix + path);
+    return next->showPath(prefix / path);
 }
 
 void FilteringInputAccessor::checkAccess(const CanonPath & path)

src/libfetchers/fs-input-accessor.cc

@@ -48,7 +48,7 @@ struct FSInputAccessor : InputAccessor, PosixSourceAccessor
 
     CanonPath makeAbsPath(const CanonPath & path)
     {
-        return root + path;
+        return root / path;
     }
 
     std::optional<CanonPath> getPhysicalPath(const CanonPath & path) override

src/libfetchers/git-utils.cc

@@ -295,7 +295,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
             throw Error("getting working directory status: %s", git_error_last()->message);
 
         /* Get submodule info. */
-        auto modulesFile = path + ".gitmodules";
+        auto modulesFile = path / ".gitmodules";
         if (pathExists(modulesFile.abs()))
             info.submodules = parseSubmodules(modulesFile);
 

src/libfetchers/git.cc

@@ -319,7 +319,7 @@ struct GitInputScheme : InputScheme
         if (!repoInfo.isLocal)
             throw Error("cannot commit '%s' to Git repository '%s' because it's not a working tree", path, input.to_string());
 
-        writeFile((CanonPath(repoInfo.url) + path).abs(), contents);
+        writeFile((CanonPath(repoInfo.url) / path).abs(), contents);
 
         auto result = runProgram(RunOptions {
             .program = "git",
@@ -680,7 +680,7 @@ struct GitInputScheme : InputScheme
             std::map<CanonPath, nix::ref<InputAccessor>> mounts;
 
             for (auto & submodule : repoInfo.workdirInfo.submodules) {
-                auto submodulePath = CanonPath(repoInfo.url) + submodule.path;
+                auto submodulePath = CanonPath(repoInfo.url) / submodule.path;
                 fetchers::Attrs attrs;
                 attrs.insert_or_assign("type", "git");
                 attrs.insert_or_assign("url", submodulePath.abs());

src/libfetchers/mercurial.cc

@@ -141,7 +141,7 @@ struct MercurialInputScheme : InputScheme
         if (!isLocal)
             throw Error("cannot commit '%s' to Mercurial repository '%s' because it's not a working tree", path, input.to_string());
 
-        auto absPath = CanonPath(repoPath) + path;
+        auto absPath = CanonPath(repoPath) / path;
 
         writeFile(absPath.abs(), contents);
 

src/libfetchers/path.cc

@@ -84,7 +84,7 @@ struct PathInputScheme : InputScheme
         std::string_view contents,
         std::optional<std::string> commitMsg) const override
     {
-        writeFile((CanonPath(getAbsPath(input)) + path).abs(), contents);
+        writeFile((CanonPath(getAbsPath(input)) / path).abs(), contents);
     }
 
     CanonPath getAbsPath(const Input & input) const

src/libstore/binary-cache-store.cc

@@ -235,14 +235,14 @@ ref<const ValidPathInfo> BinaryCacheStore::addToStoreCommon(
             std::regex regex2("^[0-9a-f]{38}\\.debug$");
 
             for (auto & [s1, _type] : narAccessor->readDirectory(buildIdDir)) {
-                auto dir = buildIdDir + s1;
+                auto dir = buildIdDir / s1;
 
                 if (narAccessor->lstat(dir).type != SourceAccessor::tDirectory
                     || !std::regex_match(s1, regex1))
                     continue;
 
                 for (auto & [s2, _type] : narAccessor->readDirectory(dir)) {
-                    auto debugPath = dir + s2;
+                    auto debugPath = dir / s2;
 
                     if (narAccessor->lstat(debugPath).type != SourceAccessor::tRegular
                         || !std::regex_match(s2, regex2))

src/libstore/local-fs-store.cc

@@ -28,7 +28,7 @@ struct LocalStoreAccessor : PosixSourceAccessor
         auto [storePath, rest] = store->toStorePath(path.abs());
         if (requireValidPath && !store->isValidPath(storePath))
             throw InvalidPath("path '%1%' is not a valid store path", store->printStorePath(storePath));
-        return CanonPath(store->getRealStoreDir()) + storePath.to_string() + CanonPath(rest);
+        return CanonPath(store->getRealStoreDir()) / storePath.to_string() / CanonPath(rest);
     }
 
     std::optional<Stat> maybeLstat(const CanonPath & path) override

src/libstore/nar-accessor.cc

@@ -277,7 +277,7 @@ json listNar(ref<SourceAccessor> accessor, const CanonPath & path, bool recurse)
             json &res2 = obj["entries"];
             for (const auto & [name, type] : accessor->readDirectory(path)) {
                 if (recurse) {
-                    res2[name] = listNar(accessor, path + name, true);
+                    res2[name] = listNar(accessor, path / name, true);
                 } else
                     res2[name] = json::object();
             }

src/libutil/archive.cc

@@ -77,20 +77,20 @@ void SourceAccessor::dumpPath(
                     std::string name(i.first);
                     size_t pos = i.first.find(caseHackSuffix);
                     if (pos != std::string::npos) {
-                        debug("removing case hack suffix from '%s'", path + i.first);
+                        debug("removing case hack suffix from '%s'", path / i.first);
                         name.erase(pos);
                     }
                     if (!unhacked.emplace(name, i.first).second)
                         throw Error("file name collision in between '%s' and '%s'",
-                            (path + unhacked[name]),
-                            (path + i.first));
+                            (path / unhacked[name]),
+                            (path / i.first));
                 } else
                     unhacked.emplace(i.first, i.first);
 
             for (auto & i : unhacked)
-                if (filter((path + i.first).abs())) {
+                if (filter((path / i.first).abs())) {
                     sink << "entry" << "(" << "name" << i.first << "node";
-                    dump(path + i.second);
+                    dump(path / i.second);
                     sink << ")";
                 }
         }

src/libutil/canon-path.cc

@@ -63,7 +63,7 @@ void CanonPath::extend(const CanonPath & x)
         path += x.abs();
 }
 
-CanonPath CanonPath::operator + (const CanonPath & x) const
+CanonPath CanonPath::operator / (const CanonPath & x) const
 {
     auto res = *this;
     res.extend(x);
@@ -78,7 +78,7 @@ void CanonPath::push(std::string_view c)
     path += c;
 }
 
-CanonPath CanonPath::operator + (std::string_view c) const
+CanonPath CanonPath::operator / (std::string_view c) const
 {
     auto res = *this;
     res.push(c);

src/libutil/canon-path.hh

@@ -190,14 +190,14 @@ public:
     /**
      * Concatenate two paths.
      */
-    CanonPath operator + (const CanonPath & x) const;
+    CanonPath operator / (const CanonPath & x) const;
 
     /**
      * Add a path component to this one. It must not contain any slashes.
      */
     void push(std::string_view c);
 
-    CanonPath operator + (std::string_view c) const;
+    CanonPath operator / (std::string_view c) const;
 
     /**
      * Check whether access to this path is allowed, which is the case

src/libutil/file-system.cc

@@ -25,7 +25,7 @@ Path absPath(PathView path, std::optional<PathView> dir, bool resolveSymlinks)
 {
     std::string scratch;
 
-    if (path[0] != '/') {
+    if (path.empty() || path[0] != '/') {
         // In this case we need to call `canonPath` on a newly-created
         // string. We set `scratch` to that string first, and then set
         // `path` to `scratch`. This ensures the newly-created string

src/libutil/fs-sink.cc

@@ -34,7 +34,7 @@ void copyRecursive(
         sink.createDirectory(to);
         for (auto & [name, _] : accessor.readDirectory(from)) {
             copyRecursive(
-                accessor, from + name,
+                accessor, from / name,
                 sink, to + "/" + name);
             break;
         }

src/libutil/git.cc

@@ -259,7 +259,7 @@ Mode dump(
     {
         Tree entries;
         for (auto & [name, _] : accessor.readDirectory(path)) {
-            auto child = path + name;
+            auto child = path / name;
             if (!filter(child.abs())) continue;
 
             auto entry = hook(child);

src/libutil/source-path.cc

@@ -41,11 +41,11 @@ std::optional<CanonPath> SourcePath::getPhysicalPath() const
 std::string SourcePath::to_string() const
 { return accessor->showPath(path); }
 
-SourcePath SourcePath::operator+(const CanonPath & x) const
-{ return {accessor, path + x}; }
+SourcePath SourcePath::operator / (const CanonPath & x) const
+{ return {accessor, path / x}; }
 
-SourcePath SourcePath::operator+(std::string_view c) const
-{  return {accessor, path + c}; }
+SourcePath SourcePath::operator / (std::string_view c) const
+{ return {accessor, path / c}; }
 
 bool SourcePath::operator==(const SourcePath & x) const
 {

src/libutil/source-path.hh

@@ -89,14 +89,15 @@ struct SourcePath
     /**
      * Append a `CanonPath` to this path.
      */
-    SourcePath operator + (const CanonPath & x) const;
+    SourcePath operator / (const CanonPath & x) const;
 
     /**
      * Append a single component `c` to this path. `c` must not
      * contain a slash. A slash is implicitly added between this path
      * and `c`.
      */
-    SourcePath operator+(std::string_view c) const;
+    SourcePath operator / (std::string_view c) const;
+
     bool operator==(const SourcePath & x) const;
     bool operator!=(const SourcePath & x) const;
     bool operator<(const SourcePath & x) const;

src/nix-env/nix-env.cc

@@ -97,7 +97,7 @@ static bool isNixExpr(const SourcePath & path, struct InputAccessor::Stat & st)
 {
     return
         st.type == InputAccessor::tRegular
-        || (st.type == InputAccessor::tDirectory && (path + "default.nix").resolveSymlinks().pathExists());
+        || (st.type == InputAccessor::tDirectory && (path / "default.nix").resolveSymlinks().pathExists());
 }
 
 
@@ -116,7 +116,7 @@ static void getAllExprs(EvalState & state,
            are implemented using profiles). */
         if (i == "manifest.nix") continue;
 
-        auto path2 = (path + i).resolveSymlinks();
+        auto path2 = (path / i).resolveSymlinks();
 
         InputAccessor::Stat st;
         try {

src/nix/develop.cc

@@ -354,7 +354,7 @@ struct Common : InstallableCommand, MixProfile
         for (auto & i : {"TMP", "TMPDIR", "TEMP", "TEMPDIR"})
             out << fmt("export %s=\"$NIX_BUILD_TOP\"\n", i);
 
-        out << "eval \"$shellHook\"\n";
+        out << "eval \"${shellHook:-}\"\n";
 
         auto script = out.str();
 

src/nix/ls.cc

@@ -72,7 +72,7 @@ struct MixLs : virtual Args, MixJSON
             if (st.type == SourceAccessor::Type::tDirectory && !showDirectory) {
                 auto names = accessor->readDirectory(curPath);
                 for (auto & [name, type] : names)
-                    showFile(curPath + name, relPath + "/" + name);
+                    showFile(curPath / name, relPath + "/" + name);
             } else
                 showFile(curPath, relPath);
         };

src/nix/run.cc

@@ -124,7 +124,7 @@ struct CmdShell : InstallablesCommand, MixEnvironment
             if (true)
                 pathAdditions.push_back(store->printStorePath(path) + "/bin");
 
-            auto propPath = CanonPath(store->printStorePath(path)) + "nix-support" + "propagated-user-env-packages";
+            auto propPath = CanonPath(store->printStorePath(path)) / "nix-support" / "propagated-user-env-packages";
             if (auto st = accessor->maybeLstat(propPath); st && st->type == SourceAccessor::tRegular) {
                 for (auto & p : tokenizeString<Paths>(accessor->readFile(propPath)))
                     todo.push(store->parseStorePath(p));

src/nix/why-depends.cc

@@ -225,7 +225,7 @@ struct CmdWhyDepends : SourceExprCommand, MixOperateOnOptions
                 if (st->type == SourceAccessor::Type::tDirectory) {
                     auto names = accessor->readDirectory(p);
                     for (auto & [name, type] : names)
-                        visitPath(p + name);
+                        visitPath(p / name);
                 }
 
                 else if (st->type == SourceAccessor::Type::tRegular) {

tests/functional/nix-shell.sh

@@ -118,10 +118,10 @@ diff $TEST_ROOT/dev-env{,2}.json
 # Ensure `nix print-dev-env --json` contains variable assignments.
 [[ $(jq -r .variables.arr1.value[2] $TEST_ROOT/dev-env.json) = '3 4' ]]
 
-# Run tests involving `source <(nix print-dev-inv)` in subshells to avoid modifying the current
+# Run tests involving `source <(nix print-dev-env)` in subshells to avoid modifying the current
 # environment.
 
-set +u # FIXME: Make print-dev-env `set -u` compliant (issue #7951)
+set -u
 
 # Ensure `source <(nix print-dev-env)` modifies the environment.
 (

tests/functional/repl.sh

@@ -156,7 +156,7 @@ testReplResponseNoRegex '
 # Same for let expressions
 testReplResponseNoRegex '
 let x = { y = { a = 1; }; inherit x; }; in x
-' '{ x = { ... }; y = { ... }; }'
+' '{ x = «repeated»; y = { ... }; }'
 
 # The :p command should recursively print sets, but prevent infinite recursion
 testReplResponseNoRegex '
@@ -171,4 +171,4 @@ testReplResponseNoRegex '
 # Same for let expressions
 testReplResponseNoRegex '
 :p let x = { y = { a = 1; }; inherit x; }; in x
-' '{ x = { x = «repeated»; y = { a = 1; }; }; y = «repeated»; }'
+' '{ x = «repeated»; y = { a = 1; }; }'

tests/unit/libutil/canon-path.cc

@@ -80,29 +80,29 @@ namespace nix {
         {
             CanonPath p1("a//foo/bar//");
             CanonPath p2("xyzzy/bla");
-            ASSERT_EQ((p1 + p2).abs(), "/a/foo/bar/xyzzy/bla");
+            ASSERT_EQ((p1 / p2).abs(), "/a/foo/bar/xyzzy/bla");
         }
 
         {
             CanonPath p1("/");
             CanonPath p2("/a/b");
-            ASSERT_EQ((p1 + p2).abs(), "/a/b");
+            ASSERT_EQ((p1 / p2).abs(), "/a/b");
         }
 
         {
             CanonPath p1("/a/b");
             CanonPath p2("/");
-            ASSERT_EQ((p1 + p2).abs(), "/a/b");
+            ASSERT_EQ((p1 / p2).abs(), "/a/b");
         }
 
         {
             CanonPath p("/foo/bar");
-            ASSERT_EQ((p + "x").abs(), "/foo/bar/x");
+            ASSERT_EQ((p / "x").abs(), "/foo/bar/x");
         }
 
         {
             CanonPath p("/");
-            ASSERT_EQ((p + "foo" + "bar").abs(), "/foo/bar");
+            ASSERT_EQ((p / "foo" / "bar").abs(), "/foo/bar");
         }
     }