picnoir/fix
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Merge branch 'master' into flake_search

Browse source
This commit is contained in:
tomberek 2021-12-23 15:22:52 -05:00 committed by GitHub
parent b6cc0a704d af553b2090
commit f404189368
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
110 changed files with 13588 additions and 4110 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
.github/PULL_REQUEST_TEMPLATE/pull_request_template.md vendored Normal file
View file

@ -0,0 +1,7 @@
**Release Notes**
Please include relevant [release notes](https://github.com/NixOS/nix/blob/master/doc/manual/src/release-notes/rl-next.md) as needed.
**Testing**
If this issue is a regression or something that should block release, please consider including a test either in the [testsuite](https://github.com/NixOS/nix/tree/master/tests) or as a [hydraJob]( https://github.com/NixOS/nix/blob/master/flake.nix#L396) so that it can be part of the [automatic checks](https://hydra.nixos.org/jobset/nix/master).

2
.version
View file

@ -1 +1 @@
2.5
2.6.0

24
doc/manual/generate-options.nix
View file

@ -8,17 +8,19 @@ concatStrings (map
let option = options.${name}; in
" - `${name}` \n\n"
+ concatStrings (map (s: " ${s}\n") (splitLines option.description)) + "\n\n"
+ " **Default:** " + (
if option.value == "" || option.value == []
then "*empty*"
else if isBool option.value
then (if option.value then "`true`" else "`false`")
else
# n.b. a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is "null" in
# JSON, but that converts to "{ }" here.
(if isAttrs option.value then "`\"\"`"
else "`" + toString option.value + "`")) + "\n\n"
+ (if option.documentDefault
then " **Default:** " + (
if option.value == "" || option.value == []
then "*empty*"
else if isBool option.value
then (if option.value then "`true`" else "`false`")
else
# n.b. a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is "null" in
# JSON, but that converts to "{ }" here.
(if isAttrs option.value then "`\"\"`"
else "`" + toString option.value + "`")) + "\n\n"
else " **Default:** *machine-specific*")
+ (if option.aliases != []
then " **Deprecated alias:** " + (concatStringsSep ", " (map (s: "`${s}`") option.aliases)) + "\n\n"
else "")

1
doc/manual/src/SUMMARY.md.in
View file

@ -72,6 +72,7 @@
- [CLI guideline](contributing/cli-guideline.md)
- [Release Notes](release-notes/release-notes.md)
- [Release X.Y (202?-??-??)](release-notes/rl-next.md)
- [Release 2.5 (2021-12-13)](release-notes/rl-2.5.md)
- [Release 2.4 (2021-11-01)](release-notes/rl-2.4.md)
- [Release 2.3 (2019-09-04)](release-notes/rl-2.3.md)
- [Release 2.2 (2019-01-11)](release-notes/rl-2.2.md)

4
doc/manual/src/advanced-topics/distributed-builds.md
View file

@ -53,8 +53,8 @@ example, the following command allows you to build a derivation for
$ uname
Linux
$ nix build \
'(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
$ nix build --impure \
--expr '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac

24
doc/manual/src/installation/installing-binary.md
View file

@ -119,6 +119,30 @@ this to run the installer, but it may help if you run into trouble:
- update `/etc/synthetic.conf` to direct macOS to create a "synthetic"
empty root directory to mount your volume
- specify mount options for the volume in `/etc/fstab`
- `rw`: read-write
- `noauto`: prevent the system from auto-mounting the volume (so the
LaunchDaemon mentioned below can control mounting it, and to avoid
masking problems with that mounting service).
- `nobrowse`: prevent the Nix Store volume from showing up on your
desktop; also keeps Spotlight from spending resources to index
this volume
<!-- TODO:
- `suid`: honor setuid? surely not? ...
- `owners`: honor file ownership on the volume
For now I'll avoid pretending to understand suid/owners more
than I do. There've been some vague reports of file-ownership
and permission issues, particularly in cloud/VM/headless setups.
My pet theory is that this has something to do with these setups
not having a token that gets delegated to initial/admin accounts
on macOS. See scripts/create-darwin-volume.sh for a little more.
In any case, by Dec 4 2021, it _seems_ like some combination of
suid, owners, and calling diskutil enableOwnership have stopped
new reports from coming in. But I hesitate to celebrate because we
haven't really named and catalogued the behavior, understood what
we're fixing, and validated that all 3 components are essential.
-->
- if you have FileVault enabled
- generate an encryption password
- put it in your system Keychain

4
doc/manual/src/installation/installing-docker.md
View file

@ -16,7 +16,7 @@ nix (Nix) 2.3.12
35ca4ada6e96:/# exit
```
# What is included in Nix' Docker image?
# What is included in Nix's Docker image?
The official Docker image is created using `pkgs.dockerTools.buildLayeredImage`
(and not with `Dockerfile` as it is usual with Docker images). You can still
@ -54,6 +54,6 @@ You can also build a Docker image from source yourself:
```console
$ nix build ./\#hydraJobs.dockerImage.x86_64-linux
$ docker load -i ./result
$ docker load -i ./result/image.tar.gz
$ docker run -ti nix:2.5pre20211105
```

2
doc/manual/src/installation/supported-platforms.md
View file

@ -4,4 +4,4 @@ Nix is currently supported on the following platforms:
- Linux (i686, x86\_64, aarch64).
- macOS (x86\_64).
- macOS (x86\_64, aarch64).

2
doc/manual/src/package-management/basic-package-mgmt.md
View file

@ -40,7 +40,7 @@ $ nix-channel --update
>
> On NixOS, you’re automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g.
> <http://nixos.org/channels/nixos-14.12>). A NixOS channel is identical
> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
> to the Nixpkgs channel, except that it contains only Linux binaries
> and is updated only if a set of regression tests succeed.

16
doc/manual/src/release-notes/rl-2.5.md Normal file
View file

@ -0,0 +1,16 @@
# Release 2.5 (2021-12-13)
* The garbage collector no longer blocks new builds, so the message
`waiting for the big garbage collector lock...` is a thing of the
past.
* Binary cache stores now have a setting `compression-level`.
* `nix develop` now has a flag `--unpack` to run `unpackPhase`.
* Lists can now be compared lexicographically using the `<` operator.
* New built-in function: `builtins.groupBy`, with the same functionality as
Nixpkgs' `lib.groupBy`, but faster.
* `nix repl` now has a `:log` command.

6
doc/manual/src/release-notes/rl-next.md
View file

@ -1,4 +1,5 @@
# Release 2.5 (2021-XX-XX)
# Release X.Y (202?-??-??)
* Binary cache stores now have a setting `compression-level`.
@ -10,3 +11,6 @@
Nixpkgs' `lib.groupBy`, but faster.
* Nix now searches for a flake.nix up until git or filesystem boundary.
* The TOML parser used by `builtins.fromTOML` has been replaced by [a
more compliant one](https://github.com/ToruNiina/toml11).

15
docker.nix
View file

@ -137,11 +137,8 @@ let
name = "root-profile-env";
paths = defaultPkgs;
};
profile = pkgs.buildPackages.runCommand "user-environment" { } ''
mkdir $out
cp -a ${rootEnv}/* $out/
cat > $out/manifest.nix <<EOF
manifest = pkgs.buildPackages.runCommand "manifest.nix" { } ''
cat > $out <<EOF
[
${lib.concatStringsSep "\n" (builtins.map (drv: let
outputs = drv.outputsToInstall or [ "out" ];
@ -161,6 +158,11 @@ let
]
EOF
'';
profile = pkgs.buildPackages.runCommand "user-environment" { } ''
mkdir $out
cp -a ${rootEnv}/* $out/
ln -s ${manifest} $out/manifest.nix
'';
in
pkgs.runCommand "base-system"
{
@ -178,6 +180,9 @@ let
set -x
mkdir -p $out/etc
mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd

7
flake.nix
View file

@ -96,7 +96,7 @@
buildPackages.mdbook
buildPackages.autoconf-archive
buildPackages.autoreconfHook
buildPackages.pkgconfig
buildPackages.pkg-config
# Tests
buildPackages.git
@ -343,7 +343,7 @@
nativeBuildInputs =
[ buildPackages.autoconf-archive
buildPackages.autoreconfHook
buildPackages.pkgconfig
buildPackages.pkg-config
];
buildInputs =
@ -667,6 +667,9 @@
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
});

6
misc/bash/completion.sh
View file

@ -7,13 +7,15 @@ function _complete_nix {
local completion=${line%% *}
if [[ -z $have_type ]]; then
have_type=1
if [[ $completion = filenames ]]; then
if [[ $completion == filenames ]]; then
compopt -o filenames
elif [[ $completion == attrs ]]; then
compopt -o nospace
fi
else
COMPREPLY+=("$completion")
fi
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]}")
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]/#\~/$HOME}")
__ltrim_colon_completions "$cur"
}

1
misc/fish/completion.fish
View file

@ -19,7 +19,6 @@ end
function _nix_accepts_files
set -l response (_nix_complete)
# First line is either filenames or no-filenames.
test $response[1] = 'filenames'
end

5
misc/launchd/org.nixos.nix-daemon.plist.in
View file

@ -25,5 +25,10 @@
<string>/var/log/nix-daemon.log</string>
<key>StandardOutPath</key>
<string>/dev/null</string>
<key>SoftResourceLimits</key>
<dict>
<key>NumberOfFiles</key>
<integer>4096</integer>
</dict>
</dict>
</plist>

21
scripts/create-darwin-volume.sh
View file

@ -440,7 +440,22 @@ add_nix_vol_fstab_line() {
# shellcheck disable=SC1003,SC2026
local escaped_mountpoint="${NIX_ROOT/ /'\\\'040}"
shift
EDITOR="/usr/bin/ex" _sudo "to add nix to fstab" "$@" <<EOF
# wrap `ex` to work around a problem with vim plugins breaking exit codes;
# (see https://github.com/NixOS/nix/issues/5468)
# we'd prefer EDITOR="/usr/bin/ex --noplugin" but vifs doesn't word-split
# the EDITOR env.
#
# TODO: at some point we should switch to `--clean`, but it wasn't added
# until https://github.com/vim/vim/releases/tag/v8.0.1554 while the macOS
# minver 10.12.6 seems to have released with vim 7.4
cat > "$SCRATCH/ex_cleanroom_wrapper" <<EOF
#!/bin/sh
/usr/bin/ex --noplugin "\$@"
EOF
chmod 755 "$SCRATCH/ex_cleanroom_wrapper"
EDITOR="$SCRATCH/ex_cleanroom_wrapper" _sudo "to add nix to fstab" "$@" <<EOF
:a
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,suid,owners
.
@ -631,7 +646,7 @@ EOF
# technically /etc/synthetic.d/nix is supported in Big Sur+
# but handling both takes even more code...
_sudo "to add Nix to /etc/synthetic.conf" \
/usr/bin/ex /etc/synthetic.conf <<EOF
/usr/bin/ex --noplugin /etc/synthetic.conf <<EOF
:a
${NIX_ROOT:1}
.
@ -794,7 +809,7 @@ setup_volume_daemon() {
local volume_uuid="$2"
if ! test_voldaemon; then
task "Configuring LaunchDaemon to mount '$NIX_VOLUME_LABEL'" >&2
_sudo "to install the Nix volume mounter" /usr/bin/ex "$NIX_VOLUME_MOUNTD_DEST" <<EOF
_sudo "to install the Nix volume mounter" /usr/bin/ex --noplugin "$NIX_VOLUME_MOUNTD_DEST" <<EOF
:a
$(generate_mount_daemon "$cmd_type" "$volume_uuid")
.

10
scripts/install-multi-user.sh
View file

@ -377,6 +377,11 @@ cure_artifacts() {
}
validate_starting_assumptions() {
task "Checking for artifacts of previous installs"
cat <<EOF
Before I try to install, I'll check for signs Nix already is or has
been installed on this system.
EOF
if type nix-env 2> /dev/null >&2; then
warning <<EOF
Nix already appears to be installed. This installer may run into issues.
@ -386,6 +391,11 @@ $(uninstall_directions)
EOF
fi
# TODO: I think it would be good for this step to accumulate more
# knowledge of older obsolete artifacts, if there are any.
# We could issue a "reminder" here that the user might want
# to clean them up?
for profile_target in "${PROFILE_TARGETS[@]}"; do
# TODO: I think it would be good to accumulate a list of all
# of the copies so that people don't hit this 2 or 3x in

6
scripts/install-nix-from-closure.sh
View file

@ -38,7 +38,7 @@ fi
# Determine if we could use the multi-user installer or not
if [ "$(uname -s)" = "Linux" ]; then
echo "Note: a multi-user installation is possible. See https://nixos.org/nix/manual/#sect-multi-user-installation" >&2
echo "Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation" >&2
fi
case "$(uname -s)" in
@ -98,7 +98,7 @@ while [ $# -gt 0 ]; do
echo " providing multi-user support and better isolation for local builds."
echo " Both for security and reproducibility, this method is recommended if"
echo " supported on your platform."
echo " See https://nixos.org/nix/manual/#sect-multi-user-installation"
echo " See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation"
echo ""
echo " --no-daemon: Simple, single-user installation that does not require root and is"
echo " trivial to uninstall."
@ -144,7 +144,7 @@ if ! [ -e "$dest" ]; then
fi
if ! [ -w "$dest" ]; then
echo "$0: directory $dest exists, but is not writable by you. This could indicate that another user has already performed a single-user installation of Nix on this system. If you wish to enable multi-user support see https://nixos.org/nix/manual/#ssec-multi-user. If you wish to continue with a single-user install for $USER please run 'chown -R $USER $dest' as root." >&2
echo "$0: directory $dest exists, but is not writable by you. This could indicate that another user has already performed a single-user installation of Nix on this system. If you wish to enable multi-user support see https://nixos.org/manual/nix/stable/installation/multi-user.html. If you wish to continue with a single-user install for $USER please run 'chown -R $USER $dest' as root." >&2
exit 1
fi

18
src/cpptoml/LICENSE
View file

@ -1,18 +0,0 @@
Copyright (c) 2014 Chase Geigle
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

3668
src/cpptoml/cpptoml.h
View file

File diff suppressed because it is too large Load diff

9
src/libcmd/command.cc
View file

@ -73,8 +73,13 @@ ref<Store> EvalCommand::getEvalStore()
ref<EvalState> EvalCommand::getEvalState()
{
if (!evalState)
evalState = std::make_shared<EvalState>(searchPath, getEvalStore(), getStore());
if (!evalState) evalState =
#if HAVE_BOEHMGC
std::allocate_shared<EvalState>(traceable_allocator<EvalState>(),
#else
std::make_shared<EvalState>(
#endif
searchPath, getEvalStore(), getStore());
return ref<EvalState>(evalState);
}

12
src/libcmd/installables.cc
View file

@ -191,7 +191,7 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)
auto sep = prefix_.rfind('.');
std::string searchWord;
if (sep != std::string::npos) {
searchWord = prefix_.substr(sep, std::string::npos);
searchWord = prefix_.substr(sep + 1, std::string::npos);
prefix_ = prefix_.substr(0, sep);
} else {
searchWord = prefix_;
@ -203,6 +203,8 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)
Value v2;
state->autoCallFunction(*autoArgs, v1, v2);
completionType = ctAttrs;
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs) {
std::string name = i.name;
@ -232,7 +234,9 @@ void completeFlakeRefWithFragment(
prefix. */
try {
auto hash = prefix.find('#');
if (hash != std::string::npos) {
if (hash == std::string::npos) {
completeFlakeRef(evalState->store, prefix);
} else {
auto fragment = prefix.substr(hash + 1);
auto flakeRefS = std::string(prefix.substr(0, hash));
// FIXME: do tilde expansion.
@ -248,6 +252,8 @@ void completeFlakeRefWithFragment(
flake. */
attrPathPrefixes.push_back("");
completionType = ctAttrs;
for (auto & attrPathPrefixS : attrPathPrefixes) {
auto attrPathPrefix = parseAttrPath(*evalState, attrPathPrefixS);
auto attrPathS = attrPathPrefixS + std::string(fragment);
@ -285,8 +291,6 @@ void completeFlakeRefWithFragment(
} catch (Error & e) {
warn(e.msg());
}
completeFlakeRef(evalState->store, prefix);
}
void completeFlakeRef(ref<Store> store, std::string_view prefix)

25
src/libexpr/eval.cc
View file

@ -821,8 +821,23 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
Value * EvalState::allocValue()
{
/* We use the boehm batch allocator to speed up allocations of Values (of which there are many).
GC_malloc_many returns a linked list of objects of the given size, where the first word
of each object is also the pointer to the next object in the list. This also means that we
have to explicitly clear the first word of every object we take. */
if (!valueAllocCache) {
valueAllocCache = GC_malloc_many(sizeof(Value));
if (!valueAllocCache) throw std::bad_alloc();
}
/* GC_NEXT is a convenience macro for accessing the first word of an object.
Take the first list item, advance the list to the next item, and clear the next pointer. */
void * p = valueAllocCache;
GC_PTR_STORE_AND_DIRTY(&valueAllocCache, GC_NEXT(p));
GC_NEXT(p) = nullptr;
nrValues++;
auto v = (Value *) allocBytes(sizeof(Value));
auto v = (Value *) p;
return v;
}
@ -1656,7 +1671,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
bool first = !forceString;
ValueType firstType = nString;
for (auto & i : *es) {
for (auto & [i_pos, i] : *es) {
Value vTmp;
i->eval(state, env, vTmp);
@ -1677,19 +1692,19 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
nf = n;
nf += vTmp.fpoint;
} else
throwEvalError(pos, "cannot add %1% to an integer", showType(vTmp));
throwEvalError(i_pos, "cannot add %1% to an integer", showType(vTmp));
} else if (firstType == nFloat) {
if (vTmp.type() == nInt) {
nf += vTmp.integer;
} else if (vTmp.type() == nFloat) {
nf += vTmp.fpoint;
} else
throwEvalError(pos, "cannot add %1% to a float", showType(vTmp));
throwEvalError(i_pos, "cannot add %1% to a float", showType(vTmp));
} else
/* skip canonization of first path, which would only be not
canonized in the first place if it's coming from a ./${foo} type
path */
s << state.coerceToString(pos, vTmp, context, false, firstType == nString, !first);
s << state.coerceToString(i_pos, vTmp, context, false, firstType == nString, !first);
first = false;
}

8
src/libexpr/eval.hh
View file

@ -133,6 +133,9 @@ private:
/* Cache used by prim_match(). */
std::shared_ptr<RegexCache> regexCache;
/* Allocation cache for GC'd Value objects. */
void * valueAllocCache = nullptr;
public:
EvalState(
@ -350,7 +353,10 @@ public:
/* Print statistics. */
void printStats();
void realiseContext(const PathSet & context);
/* Realise the given context, and return a mapping from the placeholders
* used to construct the associated value to their final store path
*/
[[nodiscard]] StringMap realiseContext(const PathSet & context);
private:

8
src/libexpr/flake/config.cc
View file

@ -38,11 +38,11 @@ void ConfigFile::apply()
// FIXME: Move into libutil/config.cc.
std::string valueS;
if (auto s = std::get_if<std::string>(&value))
if (auto* s = std::get_if<std::string>(&value))
valueS = *s;
else if (auto n = std::get_if<int64_t>(&value))
valueS = fmt("%d", n);
else if (auto b = std::get_if<Explicit<bool>>(&value))
else if (auto* n = std::get_if<int64_t>(&value))
valueS = fmt("%d", *n);
else if (auto* b = std::get_if<Explicit<bool>>(&value))
valueS = b->t ? "true" : "false";
else if (auto ss = std::get_if<std::vector<std::string>>(&value))
valueS = concatStringsSep(" ", *ss); // FIXME: evil

10
src/libexpr/flake/flake.cc
View file

@ -155,7 +155,7 @@ static FlakeInput parseFlakeInput(EvalState & state,
if (!attrs.empty())
throw Error("unexpected flake input attribute '%s', at %s", attrs.begin()->first, pos);
if (url)
input.ref = parseFlakeRef(*url, baseDir, true);
input.ref = parseFlakeRef(*url, baseDir, true, input.isFlake);
}
if (!input.follows && !input.ref)
@ -194,8 +194,8 @@ static Flake getFlake(
state, originalRef, allowLookup, flakeCache);
// Guard against symlink attacks.
auto flakeDir = canonPath(sourceInfo.actualPath + "/" + lockedRef.subdir);
auto flakeFile = canonPath(flakeDir + "/flake.nix");
auto flakeDir = canonPath(sourceInfo.actualPath + "/" + lockedRef.subdir, true);
auto flakeFile = canonPath(flakeDir + "/flake.nix", true);
if (!isInDir(flakeFile, sourceInfo.actualPath))
throw Error("'flake.nix' file of flake '%s' escapes from '%s'",
lockedRef, state.store->printStorePath(sourceInfo.storePath));
@ -254,7 +254,7 @@ static Flake getFlake(
else if (setting.value->type() == nInt)
flake.config.settings.insert({setting.name, state.forceInt(*setting.value, *setting.pos)});
else if (setting.value->type() == nBool)
flake.config.settings.insert({setting.name, state.forceBool(*setting.value, *setting.pos)});
flake.config.settings.insert({setting.name, Explicit<bool> { state.forceBool(*setting.value, *setting.pos) }});
else if (setting.value->type() == nList) {
std::vector<std::string> ss;
for (auto elem : setting.value->listItems()) {
@ -570,7 +570,7 @@ LockedFlake lockFlake(
};
// Bring in the current ref for relative path resolution if we have it
auto parentPath = canonPath(flake.sourceInfo->actualPath + "/" + flake.lockedRef.subdir);
auto parentPath = canonPath(flake.sourceInfo->actualPath + "/" + flake.lockedRef.subdir, true);
computeLocks(
flake.inputs, newLockFile.root, {},

75
src/libexpr/flake/flakeref.cc
View file

@ -48,9 +48,12 @@ FlakeRef FlakeRef::resolve(ref<Store> store) const
}
FlakeRef parseFlakeRef(
const std::string & url, const std::optional<Path> & baseDir, bool allowMissing)
const std::string & url,
const std::optional<Path> & baseDir,
bool allowMissing,
bool isFlake)
{
auto [flakeRef, fragment] = parseFlakeRefWithFragment(url, baseDir, allowMissing);
auto [flakeRef, fragment] = parseFlakeRefWithFragment(url, baseDir, allowMissing, isFlake);
if (fragment != "")
throw Error("unexpected fragment '%s' in flake reference '%s'", fragment, url);
return flakeRef;
@ -67,7 +70,10 @@ std::optional<FlakeRef> maybeParseFlakeRef(
}
std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
const std::string & url, const std::optional<Path> & baseDir, bool allowMissing)
const std::string & url,
const std::optional<Path> & baseDir,
bool allowMissing,
bool isFlake)
{
using namespace fetchers;
@ -112,10 +118,9 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
to 'baseDir'). If so, search upward to the root of the
repo (i.e. the directory containing .git). */
path = absPath(path, baseDir, true);
path = absPath(path, baseDir);
if (!S_ISDIR(lstat(path).st_mode))
throw BadURL("path '%s' is not a flake (because it's not a directory)", path);
if (isFlake) {
if (!allowMissing && !pathExists(path + "/flake.nix")){
notice("path '%s' does not contain a 'flake.nix', searching up",path);
@ -138,39 +143,45 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
if (!found)
throw BadURL("could not find a flake.nix file");
}
if (!S_ISDIR(lstat(path).st_mode))
throw BadURL("path '%s' is not a flake (because it's not a directory)", path);
auto flakeRoot = path;
std::string subdir;
if (!allowMissing && !pathExists(path + "/flake.nix"))
throw BadURL("path '%s' is not a flake (because it doesn't contain a 'flake.nix' file)", path);
while (flakeRoot != "/") {
if (pathExists(flakeRoot + "/.git")) {
auto base = std::string("git+file://") + flakeRoot;
auto flakeRoot = path;
std::string subdir;
auto parsedURL = ParsedURL{
.url = base, // FIXME
.base = base,
.scheme = "git+file",
.authority = "",
.path = flakeRoot,
.query = decodeQuery(match[2]),
};
while (flakeRoot != "/") {
if (pathExists(flakeRoot + "/.git")) {
auto base = std::string("git+file://") + flakeRoot;
if (subdir != "") {
if (parsedURL.query.count("dir"))
throw Error("flake URL '%s' has an inconsistent 'dir' parameter", url);
parsedURL.query.insert_or_assign("dir", subdir);
auto parsedURL = ParsedURL{
.url = base, // FIXME
.base = base,
.scheme = "git+file",
.authority = "",
.path = flakeRoot,
.query = decodeQuery(match[2]),
};
if (subdir != "") {
if (parsedURL.query.count("dir"))
throw Error("flake URL '%s' has an inconsistent 'dir' parameter", url);
parsedURL.query.insert_or_assign("dir", subdir);
}
if (pathExists(flakeRoot + "/.git/shallow"))
parsedURL.query.insert_or_assign("shallow", "1");
return std::make_pair(
FlakeRef(Input::fromURL(parsedURL), get(parsedURL.query, "dir").value_or("")),
fragment);
}
if (pathExists(flakeRoot + "/.git/shallow"))
parsedURL.query.insert_or_assign("shallow", "1");
return std::make_pair(
FlakeRef(Input::fromURL(parsedURL), get(parsedURL.query, "dir").value_or("")),
fragment);
subdir = std::string(baseNameOf(flakeRoot)) + (subdir.empty() ? "" : "/" + subdir);
flakeRoot = dirOf(flakeRoot);
}
subdir = std::string(baseNameOf(flakeRoot)) + (subdir.empty() ? "" : "/" + subdir);
flakeRoot = dirOf(flakeRoot);
}
} else {

10
src/libexpr/flake/flakeref.hh
View file

@ -62,13 +62,19 @@ struct FlakeRef
std::ostream & operator << (std::ostream & str, const FlakeRef & flakeRef);
FlakeRef parseFlakeRef(
const std::string & url, const std::optional<Path> & baseDir = {}, bool allowMissing = false);
const std::string & url,
const std::optional<Path> & baseDir = {},
bool allowMissing = false,
bool isFlake = true);
std::optional<FlakeRef> maybeParseFlake(
const std::string & url, const std::optional<Path> & baseDir = {});
std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
const std::string & url, const std::optional<Path> & baseDir = {}, bool allowMissing = false);
const std::string & url,
const std::optional<Path> & baseDir = {},
bool allowMissing = false,
bool isFlake = true);
std::optional<std::pair<FlakeRef, std::string>> maybeParseFlakeRefWithFragment(
const std::string & url, const std::optional<Path> & baseDir = {});

4
src/libexpr/nixexpr.cc
View file

@ -191,7 +191,7 @@ void ExprConcatStrings::show(std::ostream & str) const
str << "(";
for (auto & i : *es) {
if (first) first = false; else str << " + ";
str << *i;
str << i.second;
}
str << ")";
}
@ -439,7 +439,7 @@ void ExprOpNot::bindVars(const StaticEnv & env)
void ExprConcatStrings::bindVars(const StaticEnv & env)
{
for (auto & i : *es)
i->bindVars(env);
i.second->bindVars(env);
}
void ExprPos::bindVars(const StaticEnv & env)

4
src/libexpr/nixexpr.hh
View file

@ -332,8 +332,8 @@ struct ExprConcatStrings : Expr
{
Pos pos;
bool forceString;
vector<Expr *> * es;
ExprConcatStrings(const Pos & pos, bool forceString, vector<Expr *> * es)
vector<std::pair<Pos, Expr *> > * es;
ExprConcatStrings(const Pos & pos, bool forceString, vector<std::pair<Pos, Expr *> > * es)
: pos(pos), forceString(forceString), es(es) { };
COMMON_METHODS
};

38
src/libexpr/parser.y
View file

@ -152,7 +152,7 @@ static void addFormal(const Pos & pos, Formals * formals, const Formal & formal)
}
static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<Expr *> & es)
static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<std::pair<Pos, Expr *> > & es)
{
if (es.empty()) return new ExprString(symbols.create(""));
@ -162,7 +162,7 @@ static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<Ex
bool atStartOfLine = true; /* = seen only whitespace in the current line */
size_t minIndent = 1000000;
size_t curIndent = 0;
for (auto & i : es) {
for (auto & [i_pos, i] : es) {
ExprIndStr * e = dynamic_cast<ExprIndStr *>(i);
if (!e) {
/* Anti-quotations end the current start-of-line whitespace. */
@ -192,12 +192,12 @@ static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<Ex
}
/* Strip spaces from each line. */
vector<Expr *> * es2 = new vector<Expr *>;
vector<std::pair<Pos, Expr *> > * es2 = new vector<std::pair<Pos, Expr *> >;
atStartOfLine = true;
size_t curDropped = 0;
size_t n = es.size();
for (vector<Expr *>::iterator i = es.begin(); i != es.end(); ++i, --n) {
ExprIndStr * e = dynamic_cast<ExprIndStr *>(*i);
for (vector<std::pair<Pos, Expr *> >::iterator i = es.begin(); i != es.end(); ++i, --n) {
ExprIndStr * e = dynamic_cast<ExprIndStr *>(i->second);
if (!e) {
atStartOfLine = false;
curDropped = 0;
@ -234,11 +234,11 @@ static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<Ex
s2 = string(s2, 0, p + 1);
}
es2->push_back(new ExprString(symbols.create(s2)));
es2->emplace_back(i->first, new ExprString(symbols.create(s2)));
}
/* If this is a single string, then don't do a concatenation. */
return es2->size() == 1 && dynamic_cast<ExprString *>((*es2)[0]) ? (*es2)[0] : new ExprConcatStrings(pos, true, es2);
return es2->size() == 1 && dynamic_cast<ExprString *>((*es2)[0].second) ? (*es2)[0].second : new ExprConcatStrings(pos, true, es2);
}
@ -277,7 +277,7 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParseData * data, const char * err
char * path;
char * uri;
std::vector<nix::AttrName> * attrNames;
std::vector<nix::Expr *> * string_parts;
std::vector<std::pair<nix::Pos, nix::Expr *> > * string_parts;
}
%type <e> start expr expr_function expr_if expr_op
@ -364,7 +364,7 @@ expr_op
| expr_op UPDATE expr_op { $$ = new ExprOpUpdate(CUR_POS, $1, $3); }
| expr_op '?' attrpath { $$ = new ExprOpHasAttr($1, *$3); }
| expr_op '+' expr_op
{ $$ = new ExprConcatStrings(CUR_POS, false, new vector<Expr *>({$1, $3})); }
{ $$ = new ExprConcatStrings(CUR_POS, false, new vector<std::pair<Pos, Expr *> >({{makeCurPos(@1, data), $1}, {makeCurPos(@3, data), $3}})); }
| expr_op '-' expr_op { $$ = new ExprCall(CUR_POS, new ExprVar(data->symbols.create("__sub")), {$1, $3}); }
| expr_op '*' expr_op { $$ = new ExprCall(CUR_POS, new ExprVar(data->symbols.create("__mul")), {$1, $3}); }
| expr_op '/' expr_op { $$ = new ExprCall(CUR_POS, new ExprVar(data->symbols.create("__div")), {$1, $3}); }
@ -410,7 +410,7 @@ expr_simple
}
| path_start PATH_END { $$ = $1; }
| path_start string_parts_interpolated PATH_END {
$2->insert($2->begin(), $1);
$2->insert($2->begin(), {makeCurPos(@1, data), $1});
$$ = new ExprConcatStrings(CUR_POS, false, $2);
}
| SPATH {
@ -448,13 +448,13 @@ string_parts
;
string_parts_interpolated
: string_parts_interpolated STR { $$ = $1; $1->push_back($2); }
| string_parts_interpolated DOLLAR_CURLY expr '}' { $$ = $1; $1->push_back($3); }
| DOLLAR_CURLY expr '}' { $$ = new vector<Expr *>; $$->push_back($2); }
: string_parts_interpolated STR { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $2); }
| string_parts_interpolated DOLLAR_CURLY expr '}' { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $3); }
| DOLLAR_CURLY expr '}' { $$ = new vector<std::pair<Pos, Expr *> >; $$->emplace_back(makeCurPos(@1, data), $2); }
| STR DOLLAR_CURLY expr '}' {
$$ = new vector<Expr *>;
$$->push_back($1);
$$->push_back($3);
$$ = new vector<std::pair<Pos, Expr *> >;
$$->emplace_back(makeCurPos(@1, data), $1);
$$->emplace_back(makeCurPos(@2, data), $3);
}
;
@ -473,9 +473,9 @@ path_start
;
ind_string_parts
: ind_string_parts IND_STR { $$ = $1; $1->push_back($2); }
| ind_string_parts DOLLAR_CURLY expr '}' { $$ = $1; $1->push_back($3); }
| { $$ = new vector<Expr *>; }
: ind_string_parts IND_STR { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $2); }
| ind_string_parts DOLLAR_CURLY expr '}' { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $3); }
| { $$ = new vector<std::pair<Pos, Expr *> >; }
;
binds

127
src/libexpr/primops.cc
View file

@ -35,9 +35,10 @@ namespace nix {
InvalidPathError::InvalidPathError(const Path & path) :
EvalError("path '%s' is not valid", path), path(path) {}
void EvalState::realiseContext(const PathSet & context)
StringMap EvalState::realiseContext(const PathSet & context)
{
std::vector<DerivedPath::Built> drvs;
StringMap res;
for (auto & i : context) {
auto [ctxS, outputName] = decodeContext(i);
@ -46,10 +47,12 @@ void EvalState::realiseContext(const PathSet & context)
throw InvalidPathError(store->printStorePath(ctx));
if (!outputName.empty() && ctx.isDerivation()) {
drvs.push_back({ctx, {outputName}});
} else {
res.insert_or_assign(ctxS, ctxS);
}
}
if (drvs.empty()) return;
if (drvs.empty()) return {};
if (!evalSettings.enableImportFromDerivation)
throw Error(
@ -61,19 +64,53 @@ void EvalState::realiseContext(const PathSet & context)
for (auto & d : drvs) buildReqs.emplace_back(DerivedPath { d });
store->buildPaths(buildReqs);
/* Get all the output paths corresponding to the placeholders we had */
for (auto & [drvPath, outputs] : drvs) {
auto outputPaths = store->queryDerivationOutputMap(drvPath);
for (auto & outputName : outputs) {
if (outputPaths.count(outputName) == 0)
throw Error("derivation '%s' does not have an output named '%s'",
store->printStorePath(drvPath), outputName);
res.insert_or_assign(
downstreamPlaceholder(*store, drvPath, outputName),
store->printStorePath(outputPaths.at(outputName))
);
}
}
/* Add the output of this derivations to the allowed
paths. */
if (allowedPaths) {
for (auto & [drvPath, outputs] : drvs) {
auto outputPaths = store->queryDerivationOutputMap(drvPath);
for (auto & outputName : outputs) {
if (outputPaths.count(outputName) == 0)
throw Error("derivation '%s' does not have an output named '%s'",
store->printStorePath(drvPath), outputName);
allowPath(outputPaths.at(outputName));
}
for (auto & [_placeholder, outputPath] : res) {
allowPath(outputPath);
}
}
return res;
}
struct RealisePathFlags {
// Whether to check whether the path is a valid absolute path
bool requireAbsolutePath = true;
// Whether to check that the path is allowed in pure eval mode
bool checkForPureEval = true;
};
static Path realisePath(EvalState & state, const Pos & pos, Value & v, const RealisePathFlags flags = {})
{
PathSet context;
Path path = flags.requireAbsolutePath
? state.coerceToPath(pos, v, context)
: state.coerceToString(pos, v, context, false, false);
StringMap rewrites = state.realiseContext(context);
auto realPath = state.toRealPath(rewriteStrings(path, rewrites), context);
return flags.checkForPureEval
? state.checkSourcePath(realPath)
: realPath;
}
/* Add and attribute to the given attribute map from the output name to
@ -109,11 +146,9 @@ static void mkOutputString(EvalState & state, Value & v,
argument. */
static void import(EvalState & state, const Pos & pos, Value & vPath, Value * vScope, Value & v)
{
PathSet context;
Path path = state.coerceToPath(pos, vPath, context);
Path path;
try {
state.realiseContext(context);
path = realisePath(state, pos, vPath);
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt("cannot import '%1%', since path '%2%' is not valid", path, e.path),
@ -124,8 +159,6 @@ static void import(EvalState & state, const Pos & pos, Value & vPath, Value * vS
throw;
}
Path realPath = state.checkSourcePath(state.toRealPath(path, context));
// FIXME
auto isValidDerivationInStore = [&]() -> std::optional<StorePath> {
if (!state.store->isStorePath(path))
@ -177,7 +210,7 @@ static void import(EvalState & state, const Pos & pos, Value & vPath, Value * vS
else {
if (!vScope)
state.evalFile(realPath, v);
state.evalFile(path, v);
else {
state.forceAttrs(*vScope);
@ -195,8 +228,8 @@ static void import(EvalState & state, const Pos & pos, Value & vPath, Value * vS
// No need to call staticEnv.sort(), because
// args[0]->attrs is already sorted.
printTalkative("evaluating file '%1%'", realPath);
Expr * e = state.parseExprFromFile(resolveExprPath(realPath), staticEnv);
printTalkative("evaluating file '%1%'", path);
Expr * e = state.parseExprFromFile(resolveExprPath(path), staticEnv);
e->eval(state, *env, v);
}
@ -281,22 +314,19 @@ extern "C" typedef void (*ValueInitializer)(EvalState & state, Value & v);
/* Load a ValueInitializer from a DSO and return whatever it initializes */
void prim_importNative(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
Path path = state.coerceToPath(pos, *args[0], context);
Path path;
try {
state.realiseContext(context);
path = realisePath(state, pos, *args[0]);
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt(
"cannot import '%1%', since path '%2%' is not valid",
path, e.path),
.msg = hintfmt("cannot import '%1%', since path '%2%' is not valid", path, e.path),
.errPos = pos
});
} catch (Error & e) {
e.addTrace(pos, "while importing '%s'", path);
throw;
}
path = state.checkSourcePath(path);
string sym = state.forceStringNoCtx(*args[1], pos);
void *handle = dlopen(path.c_str(), RTLD_LAZY | RTLD_LOCAL);
@ -338,7 +368,7 @@ void prim_exec(EvalState & state, const Pos & pos, Value * * args, Value & v)
for (unsigned int i = 1; i < args[0]->listSize(); ++i)
commandArgs.emplace_back(state.coerceToString(pos, *elems[i], context, false, false));
try {
state.realiseContext(context);
auto _ = state.realiseContext(context); // FIXME: Handle CA derivations
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt("cannot execute '%1%', since path '%2%' is not valid",
@ -1349,10 +1379,14 @@ static RegisterPrimOp primop_storePath({
static void prim_pathExists(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
Path path = state.coerceToPath(pos, *args[0], context);
Path path;
try {
state.realiseContext(context);
// We don’t check the path right now, because we don’t want to throw if
// the path isn’t allowed, but just return false
// (and we can’t just catch the exception here because we still want to
// throw if something in the evaluation of `*args[0]` tries to access an
// unauthorized path)
path = realisePath(state, pos, *args[0], { .checkForPureEval = false });
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt(
@ -1426,17 +1460,16 @@ static RegisterPrimOp primop_dirOf({
/* Return the contents of a file as a string. */
static void prim_readFile(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
Path path = state.coerceToPath(pos, *args[0], context);
Path path;
try {
state.realiseContext(context);
path = realisePath(state, pos, *args[0]);
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt("cannot read '%1%', since path '%2%' is not valid", path, e.path),
.errPos = pos
});
}
string s = readFile(state.checkSourcePath(state.toRealPath(path, context)));
string s = readFile(path);
if (s.find((char) 0) != string::npos)
throw Error("the contents of the file '%1%' cannot be represented as a Nix string", path);
mkString(v, s.c_str());
@ -1475,11 +1508,10 @@ static void prim_findFile(EvalState & state, const Pos & pos, Value * * args, Va
pos
);
PathSet context;
string path = state.coerceToString(pos, *i->value, context, false, false);
Path path;
try {
state.realiseContext(context);
path = realisePath(state, pos, *i->value, { .requireAbsolutePath = false });
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt("cannot find '%1%', since path '%2%' is not valid", path, e.path),
@ -1512,15 +1544,14 @@ static void prim_hashFile(EvalState & state, const Pos & pos, Value * * args, Va
.errPos = pos
});
PathSet context;
Path path = state.coerceToPath(pos, *args[1], context);
Path path;
try {
state.realiseContext(context);
path = realisePath(state, pos, *args[1]);
} catch (InvalidPathError & e) {
throw EvalError("cannot read '%s' since path '%s' is not valid, at %s", path, e.path, pos);
}
mkString(v, hashFile(*ht, state.checkSourcePath(state.toRealPath(path, context))).to_string(Base16, false));
mkString(v, hashFile(*ht, path).to_string(Base16, false));
}
static RegisterPrimOp primop_hashFile({
@ -1537,10 +1568,9 @@ static RegisterPrimOp primop_hashFile({
/* Read a directory (without . or ..) */
static void prim_readDir(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet ctx;
Path path = state.coerceToPath(pos, *args[0], ctx);
Path path;
try {
state.realiseContext(ctx);
path = realisePath(state, pos, *args[0]);
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt("cannot read '%1%', since path '%2%' is not valid", path, e.path),
@ -1548,7 +1578,7 @@ static void prim_readDir(EvalState & state, const Pos & pos, Value * * args, Val
});
}
DirEntries entries = readDirectory(state.checkSourcePath(path));
DirEntries entries = readDirectory(path);
state.mkAttrs(v, entries.size());
for (auto & ent : entries) {
@ -1875,7 +1905,8 @@ static void addPath(
try {
// FIXME: handle CA derivation outputs (where path needs to
// be rewritten to the actual output).
state.realiseContext(context);
auto rewrites = state.realiseContext(context);
path = state.toRealPath(rewriteStrings(path, rewrites), context);
StorePathSet refs;

111
src/libexpr/primops/fromTOML.cc
View file

@ -1,86 +1,77 @@
#include "primops.hh"
#include "eval-inline.hh"
#include "../../cpptoml/cpptoml.h"
#include "../../toml11/toml.hpp"
namespace nix {
static void prim_fromTOML(EvalState & state, const Pos & pos, Value * * args, Value & v)
static void prim_fromTOML(EvalState & state, const Pos & pos, Value * * args, Value & val)
{
using namespace cpptoml;
auto toml = state.forceStringNoCtx(*args[0], pos);
std::istringstream tomlStream(toml);
std::function<void(Value &, std::shared_ptr<base>)> visit;
std::function<void(Value &, toml::value)> visit;
visit = [&](Value & v, std::shared_ptr<base> t) {
visit = [&](Value & v, toml::value t) {
if (auto t2 = t->as_table()) {
switch(t.type())
{
case toml::value_t::table:
{
auto table = toml::get<toml::table>(t);
size_t size = 0;
for (auto & i : *t2) { (void) i; size++; }
size_t size = 0;
for (auto & i : table) { (void) i; size++; }
state.mkAttrs(v, size);
state.mkAttrs(v, size);
for (auto & i : *t2) {
auto & v2 = *state.allocAttr(v, state.symbols.create(i.first));
for(auto & elem: table) {
if (auto i2 = i.second->as_table_array()) {
size_t size2 = i2->get().size();
state.mkList(v2, size2);
for (size_t j = 0; j < size2; ++j)
visit(*(v2.listElems()[j] = state.allocValue()), i2->get()[j]);
auto & v2 = *state.allocAttr(v, state.symbols.create(elem.first));
visit(v2, elem.second);
}
}
else
visit(v2, i.second);
}
break;;
case toml::value_t::array:
{
auto array = toml::get<std::vector<toml::value>>(t);
size_t size = array.size();
state.mkList(v, size);
for (size_t i = 0; i < size; ++i)
visit(*(v.listElems()[i] = state.allocValue()), array[i]);
}
break;;
case toml::value_t::boolean:
mkBool(v, toml::get<bool>(t));
break;;
case toml::value_t::integer:
mkInt(v, toml::get<int64_t>(t));
break;;
case toml::value_t::floating:
mkFloat(v, toml::get<NixFloat>(t));
break;;
case toml::value_t::string:
mkString(v, toml::get<std::string>(t));
break;;
case toml::value_t::local_datetime:
case toml::value_t::offset_datetime:
case toml::value_t::local_date:
case toml::value_t::local_time:
// We fail since Nix doesn't have date and time types
throw std::runtime_error("Dates and times are not supported");
break;;
case toml::value_t::empty:
mkNull(v);
break;;
v.attrs->sort();
}
else if (auto t2 = t->as_array()) {
size_t size = t2->get().size();
state.mkList(v, size);
for (size_t i = 0; i < size; ++i)
visit(*(v.listElems()[i] = state.allocValue()), t2->get()[i]);
}
// Handle cases like 'a = [[{ a = true }]]', which IMHO should be
// parsed as a array containing an array containing a table,
// but instead are parsed as an array containing a table array
// containing a table.
else if (auto t2 = t->as_table_array()) {
size_t size = t2->get().size();
state.mkList(v, size);
for (size_t j = 0; j < size; ++j)
visit(*(v.listElems()[j] = state.allocValue()), t2->get()[j]);
}
else if (t->is_value()) {
if (auto val = t->as<int64_t>())
mkInt(v, val->get());
else if (auto val = t->as<NixFloat>())
mkFloat(v, val->get());
else if (auto val = t->as<bool>())
mkBool(v, val->get());
else if (auto val = t->as<std::string>())
mkString(v, val->get());
else
throw EvalError("unsupported value type in TOML");
}
else abort();
};
try {
visit(v, parser(tomlStream).parse());
} catch (std::runtime_error & e) {
visit(val, toml::parse(tomlStream, "fromTOML" /* the "filename" */));
} catch (std::exception & e) { // TODO: toml::syntax_error
throw EvalError({
.msg = hintfmt("while parsing a TOML string: %s", e.what()),
.errPos = pos

1
src/libfetchers/tarball.cc
View file

@ -176,6 +176,7 @@ struct TarballInputScheme : InputScheme
if (!hasSuffix(url.path, ".zip")
&& !hasSuffix(url.path, ".tar")
&& !hasSuffix(url.path, ".tgz")
&& !hasSuffix(url.path, ".tar.gz")
&& !hasSuffix(url.path, ".tar.xz")
&& !hasSuffix(url.path, ".tar.bz2")

12
src/libmain/shared.cc
View file

@ -145,10 +145,10 @@ static void preloadNSS() {
*
* All other platforms are unaffected.
*/
if (dlopen (LIBNSS_DNS_SO, RTLD_NOW) == NULL) {
printMsg(Verbosity::lvlWarn, fmt("Unable to load nss_dns backend"));
}
__nss_configure_lookup ("hosts", "dns");
if (!dlopen(LIBNSS_DNS_SO, RTLD_NOW))
warn("unable to load nss_dns backend");
// FIXME: get hosts entry from nsswitch.conf.
__nss_configure_lookup("hosts", "files dns");
#endif
});
}
@ -427,7 +427,7 @@ RunPager::RunPager()
});
pid.setKillSignal(SIGINT);
stdout = fcntl(STDOUT_FILENO, F_DUPFD_CLOEXEC, 0);
if (dup2(toPager.writeSide.get(), STDOUT_FILENO) == -1)
throw SysError("dupping stdout");
}
@ -438,7 +438,7 @@ RunPager::~RunPager()
try {
if (pid != -1) {
std::cout.flush();
close(STDOUT_FILENO);
dup2(stdout, STDOUT_FILENO);
pid.wait();
}
} catch (...) {

1
src/libmain/shared.hh
View file

@ -88,6 +88,7 @@ public:
private:
Pid pid;
int stdout;
};
extern volatile ::sig_atomic_t blockInt;

36
src/libstore/build/derivation-goal.cc
View file

@ -17,6 +17,7 @@
#include <regex>
#include <queue>
#include <fstream>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
@ -464,7 +465,6 @@ void DerivationGoal::inputsRealised()
Derivation drvResolved { *std::move(attempt) };
auto pathResolved = writeDerivation(worker.store, drvResolved);
resolvedDrv = drvResolved;
auto msg = fmt("Resolved derivation: '%s' -> '%s'",
worker.store.printStorePath(drvPath),
@ -475,9 +475,9 @@ void DerivationGoal::inputsRealised()
worker.store.printStorePath(pathResolved),
});
auto resolvedGoal = worker.makeDerivationGoal(
resolvedDrvGoal = worker.makeDerivationGoal(
pathResolved, wantedOutputs, buildMode);
addWaitee(resolvedGoal);
addWaitee(resolvedDrvGoal);
state = &DerivationGoal::resolvedFinished;
return;
@ -655,7 +655,7 @@ void DerivationGoal::tryLocalBuild() {
throw Error(
"unable to build with a primary store that isn't a local store; "
"either pass a different '--store' or enable remote builds."
"\nhttps://nixos.org/nix/manual/#chap-distributed-builds");
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
}
@ -938,16 +938,17 @@ void DerivationGoal::buildDone()
}
void DerivationGoal::resolvedFinished() {
assert(resolvedDrv);
assert(resolvedDrvGoal);
auto resolvedDrv = *resolvedDrvGoal->drv;
auto resolvedHashes = staticOutputHashes(worker.store, *resolvedDrv);
auto resolvedHashes = staticOutputHashes(worker.store, resolvedDrv);
StorePathSet outputPaths;
// `wantedOutputs` might be empty, which means “all the outputs”
auto realWantedOutputs = wantedOutputs;
if (realWantedOutputs.empty())
realWantedOutputs = resolvedDrv->outputNames();
realWantedOutputs = resolvedDrv.outputNames();
for (auto & wantedOutput : realWantedOutputs) {
assert(initialOutputs.count(wantedOutput) != 0);
@ -979,9 +980,17 @@ void DerivationGoal::resolvedFinished() {
outputPaths
);
// This is potentially a bit fishy in terms of error reporting. Not sure
// how to do it in a cleaner way
amDone(nrFailed == 0 ? ecSuccess : ecFailed, ex);
auto status = [&]() {
auto resolvedResult = resolvedDrvGoal->getResult();
switch (resolvedResult.status) {
case BuildResult::AlreadyValid:
return BuildResult::ResolvesToAlreadyValid;
default:
return resolvedResult.status;
}
}();
done(status);
}
HookReply DerivationGoal::tryBuildHook()
@ -1329,6 +1338,13 @@ void DerivationGoal::done(BuildResult::Status status, std::optional<Error> ex)
}
worker.updateProgress();
auto traceBuiltOutputsFile = getEnv("_NIX_TRACE_BUILT_OUTPUTS").value_or("");
if (traceBuiltOutputsFile != "") {
std::fstream fs;
fs.open(traceBuiltOutputsFile, std::fstream::out);
fs << worker.store.printStorePath(drvPath) << "\t" << result.toString() << std::endl;
}
}

4
src/libstore/build/derivation-goal.hh
View file

@ -50,8 +50,8 @@ struct DerivationGoal : public Goal
/* The path of the derivation. */
StorePath drvPath;
/* The path of the corresponding resolved derivation */
std::optional<BasicDerivation> resolvedDrv;
/* The goal for the corresponding resolved derivation */
std::shared_ptr<DerivationGoal> resolvedDrvGoal;
/* The specific outputs that we need to build. Empty means all of
them. */

11
src/libstore/build/local-derivation-goal.cc
View file

@ -1779,11 +1779,14 @@ void LocalDerivationGoal::runChild()
i686-linux build on an x86_64-linux machine. */
struct utsname utsbuf;
uname(&utsbuf);
if (drv->platform == "i686-linux" &&
(settings.thisSystem == "x86_64-linux" ||
(!strcmp(utsbuf.sysname, "Linux") && !strcmp(utsbuf.machine, "x86_64")))) {
if ((drv->platform == "i686-linux"
&& (settings.thisSystem == "x86_64-linux"
|| (!strcmp(utsbuf.sysname, "Linux") && !strcmp(utsbuf.machine, "x86_64"))))
|| drv->platform == "armv7l-linux"
|| drv->platform == "armv6l-linux")
{
if (personality(PER_LINUX32) == -1)
throw SysError("cannot set i686-linux personality");
throw SysError("cannot set 32-bit personality");
}
/* Impersonate a Linux 2.6 machine to get some determinism in

4
src/libstore/build/worker.cc
View file

@ -281,11 +281,11 @@ void Worker::run(const Goals & _topGoals)
if (getMachines().empty())
throw Error("unable to start any build; either increase '--max-jobs' "
"or enable remote builds."
"\nhttps://nixos.org/nix/manual/#chap-distributed-builds");
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
else
throw Error("unable to start any build; remote machines may not have "
"all required system features."
"\nhttps://nixos.org/nix/manual/#chap-distributed-builds");
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
}
assert(!awake.empty());

5
src/libstore/ca-specific-schema.sql
View file

@ -19,3 +19,8 @@ create table if not exists RealisationsRefs (
foreign key (referrer) references Realisations(id) on delete cascade,
foreign key (realisationReference) references Realisations(id) on delete restrict
);
-- used by QueryRealisationReferences
create index if not exists IndexRealisationsRefs on RealisationsRefs(referrer);
-- used by cascade deletion when ValidPaths is deleted
create index if not exists IndexRealisationsRefsOnOutputPath on Realisations(outputPath);

9
src/libstore/daemon.cc
View file

@ -4,7 +4,6 @@
#include "store-api.hh"
#include "path-with-outputs.hh"
#include "finally.hh"
#include "affinity.hh"
#include "archive.hh"
#include "derivations.hh"
#include "args.hh"
@ -956,12 +955,12 @@ void processConnection(
Finally finally([&]() {
_isInterrupted = false;
prevLogger->log(lvlDebug, fmt("%d operations", opCount));
printMsgUsing(prevLogger, lvlDebug, "%d operations", opCount);
});
if (GET_PROTOCOL_MINOR(clientVersion) >= 14 && readInt(from)) {
auto affinity = readInt(from);
setAffinityTo(affinity);
// Obsolete CPU affinity.
readInt(from);
}
readInt(from); // obsolete reserveSpace
@ -989,6 +988,8 @@ void processConnection(
break;
}
printMsgUsing(prevLogger, lvlDebug, "received daemon op %d", op);
opCount++;
try {

8
src/libstore/filetransfer.cc
View file

@ -544,13 +544,7 @@ struct curlFileTransfer : public FileTransfer
stopWorkerThread();
});
#ifdef __linux__
/* Cause this thread to not share any FS attributes with the main thread,
because this causes setns() in restoreMountNamespace() to fail.
Ideally, this would happen in the std::thread() constructor. */
if (unshare(CLONE_FS) != 0)
throw SysError("unsharing filesystem state in download thread");
#endif
unshareFilesystem();
std::map<CURL *, std::shared_ptr<TransferItem>> items;

25
src/libstore/gc.cc
View file

@ -126,7 +126,17 @@ void LocalStore::addTempRoot(const StorePath & path)
auto socketPath = stateDir.get() + gcSocketPath;
debug("connecting to '%s'", socketPath);
state->fdRootsSocket = createUnixDomainSocket();
nix::connect(state->fdRootsSocket.get(), socketPath);
try {
nix::connect(state->fdRootsSocket.get(), socketPath);
} catch (SysError & e) {
/* The garbage collector may have exited, so we need to
restart. */
if (e.errNo == ECONNREFUSED) {
debug("GC socket connection refused");
state->fdRootsSocket.close();
goto restart;
}
}
}
try {
@ -523,6 +533,8 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
AutoCloseFD fdClient = accept(fdServer.get(), nullptr, nullptr);
if (!fdClient) continue;
debug("GC roots server accepted new client");
/* Process the connection in a separate thread. */
auto fdClient_ = fdClient.get();
std::thread clientThread([&, fdClient = std::move(fdClient)]() {
@ -535,6 +547,12 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
}
});
/* On macOS, accepted sockets inherit the
non-blocking flag from the server socket, so
explicitly make it blocking. */
if (fcntl(fdServer.get(), F_SETFL, fcntl(fdServer.get(), F_GETFL) & ~O_NONBLOCK) == -1)
abort();
while (true) {
try {
auto path = readLine(fdClient.get());
@ -559,7 +577,10 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
} else
printError("received garbage instead of a root from client");
writeFull(fdClient.get(), "1", false);
} catch (Error &) { break; }
} catch (Error & e) {
debug("reading GC root from client: %s", e.msg());
break;
}
}
});

26
src/libstore/globals.hh
View file

@ -21,7 +21,7 @@ struct MaxBuildJobsSetting : public BaseSetting<unsigned int>
const std::string & name,
const std::string & description,
const std::set<std::string> & aliases = {})
: BaseSetting<unsigned int>(def, name, description, aliases)
: BaseSetting<unsigned int>(def, true, name, description, aliases)
{
options->addSetting(this);
}
@ -38,7 +38,7 @@ struct PluginFilesSetting : public BaseSetting<Paths>
const std::string & name,
const std::string & description,
const std::set<std::string> & aliases = {})
: BaseSetting<Paths>(def, name, description, aliases)
: BaseSetting<Paths>(def, true, name, description, aliases)
{
options->addSetting(this);
}
@ -130,7 +130,9 @@ public:
{"build-max-jobs"}};
Setting<unsigned int> buildCores{
this, getDefaultCores(), "cores",
this,
getDefaultCores(),
"cores",
R"(
Sets the value of the `NIX_BUILD_CORES` environment variable in the
invocation of builders. Builders can use this variable at their
@ -141,7 +143,7 @@ public:
command line switch and defaults to `1`. The value `0` means that
the builder should use all available CPU cores in the system.
)",
{"build-cores"}};
{"build-cores"}, false};
/* Read-only mode. Don't copy stuff to the store, don't change
the database. */
@ -583,10 +585,11 @@ public:
platform and generate incompatible code, so you may wish to
cross-check the results of using this option against proper
natively-built versions of your derivations.
)"};
)", {}, false};
Setting<StringSet> systemFeatures{
this, getDefaultSystemFeatures(),
this,
getDefaultSystemFeatures(),
"system-features",
R"(
A set of system “features” supported by this machine, e.g. `kvm`.
@ -602,7 +605,7 @@ public:
This setting by default includes `kvm` if `/dev/kvm` is accessible,
and the pseudo-features `nixos-test`, `benchmark` and `big-parallel`
that are used in Nixpkgs to route builds to specific machines.
)"};
)", {}, false};
Setting<Strings> substituters{
this,
@ -797,6 +800,15 @@ public:
may be useful in certain scenarios (e.g. to spin up containers or
set up userspace network interfaces in tests).
)"};
Setting<StringSet> ignoredAcls{
this, {"security.selinux", "system.nfs4_acl"}, "ignored-acls",
R"(
A list of ACLs that should be ignored, normally Nix attempts to
remove all ACLs from files and directories in the Nix store, but
some ACLs like `security.selinux` or `system.nfs4_acl` can't be
removed even by root. Therefore it's best to just ignore them.
)"};
#endif
Setting<Strings> hashedMirrors{

17
src/libstore/local-store.cc
View file

@ -80,7 +80,7 @@ int getSchema(Path schemaPath)
void migrateCASchema(SQLite& db, Path schemaPath, AutoCloseFD& lockFd)
{
const int nixCASchemaVersion = 2;
const int nixCASchemaVersion = 3;
int curCASchema = getSchema(schemaPath);
if (curCASchema != nixCASchemaVersion) {
if (curCASchema > nixCASchemaVersion) {
@ -131,6 +131,17 @@ void migrateCASchema(SQLite& db, Path schemaPath, AutoCloseFD& lockFd)
txn.commit();
}
if (curCASchema < 3) {
SQLiteTxn txn(db);
// Apply new indices added in this schema update.
db.exec(R"(
-- used by QueryRealisationReferences
create index if not exists IndexRealisationsRefs on RealisationsRefs(referrer);
-- used by cascade deletion when ValidPaths is deleted
create index if not exists IndexRealisationsRefsOnOutputPath on Realisations(outputPath);
)");
txn.commit();
}
writeFile(schemaPath, fmt("%d", nixCASchemaVersion));
lockFile(lockFd.get(), ltRead, true);
}
@ -590,9 +601,7 @@ static void canonicalisePathMetaData_(const Path & path, uid_t fromUid, InodesSe
throw SysError("querying extended attributes of '%s'", path);
for (auto & eaName: tokenizeString<Strings>(std::string(eaBuf.data(), eaSize), std::string("\000", 1))) {
/* Ignore SELinux security labels since these cannot be
removed even by root. */
if (eaName == "security.selinux") continue;
if (settings.ignoredAcls.get().count(eaName)) continue;
if (lremovexattr(path.c_str(), eaName.c_str()) == -1)
throw SysError("removing extended attribute '%s' from '%s'", eaName, path);
}

4
src/libstore/misc.cc
View file

@ -7,6 +7,7 @@
#include "topo-sort.hh"
#include "callback.hh"
#include "closure.hh"
#include "filetransfer.hh"
namespace nix {
@ -100,7 +101,8 @@ void Store::queryMissing(const std::vector<DerivedPath> & targets,
downloadSize_ = narSize_ = 0;
ThreadPool pool;
// FIXME: make async.
ThreadPool pool(fileTransferSettings.httpConnections);
struct State
{

16
src/libstore/remote-store.cc
View file

@ -5,7 +5,6 @@
#include "remote-store.hh"
#include "worker-protocol.hh"
#include "archive.hh"
#include "affinity.hh"
#include "globals.hh"
#include "derivations.hh"
#include "pool.hh"
@ -184,11 +183,8 @@ void RemoteStore::initConnection(Connection & conn)
conn.to << PROTOCOL_VERSION;
if (GET_PROTOCOL_MINOR(conn.daemonVersion) >= 14) {
int cpu = sameMachine() && settings.lockCPU ? lockToCurrentCPU() : -1;
if (cpu != -1)
conn.to << 1 << cpu;
else
conn.to << 0;
// Obsolete CPU affinity.
conn.to << 0;
}
if (GET_PROTOCOL_MINOR(conn.daemonVersion) >= 11)
@ -684,6 +680,14 @@ void RemoteStore::queryRealisationUncached(const DrvOutput & id,
Callback<std::shared_ptr<const Realisation>> callback) noexcept
{
auto conn(getConnection());
if (GET_PROTOCOL_MINOR(conn->daemonVersion) < 27) {
warn("the daemon is too old to support content-addressed derivations, please upgrade it to 2.4");
try {
callback(nullptr);
} catch (...) { return callback.rethrow(); }
}
conn->to << wopQueryRealisation;
conn->to << id.to_string();
conn.processStderr();

2
src/libstore/store-api.cc
View file

@ -1079,7 +1079,7 @@ std::map<StorePath, StorePath> copyPaths(
nrFailed++;
if (!settings.keepGoing)
throw e;
logger->log(lvlError, fmt("could not copy %s: %s", dstStore.printStorePath(storePath), e.what()));
printMsg(lvlError, "could not copy %s: %s", dstStore.printStorePath(storePath), e.what());
showProgress();
return;
}

26
src/libstore/store-api.hh
View file

@ -151,9 +151,33 @@ struct BuildResult
DependencyFailed,
LogLimitExceeded,
NotDeterministic,
ResolvesToAlreadyValid,
} status = MiscFailure;
std::string errorMsg;
std::string toString() const {
auto strStatus = [&]() {
switch (status) {
case Built: return "Built";
case Substituted: return "Substituted";
case AlreadyValid: return "AlreadyValid";
case PermanentFailure: return "PermanentFailure";
case InputRejected: return "InputRejected";
case OutputRejected: return "OutputRejected";
case TransientFailure: return "TransientFailure";
case CachedFailure: return "CachedFailure";
case TimedOut: return "TimedOut";
case MiscFailure: return "MiscFailure";
case DependencyFailed: return "DependencyFailed";
case LogLimitExceeded: return "LogLimitExceeded";
case NotDeterministic: return "NotDeterministic";
case ResolvesToAlreadyValid: return "ResolvesToAlreadyValid";
default: return "Unknown";
};
}();
return strStatus + ((errorMsg == "") ? "" : " : " + errorMsg);
}
/* How many times this build was performed. */
unsigned int timesBuilt = 0;
@ -170,7 +194,7 @@ struct BuildResult
time_t startTime = 0, stopTime = 0;
bool success() {
return status == Built || status == Substituted || status == AlreadyValid;
return status == Built || status == Substituted || status == AlreadyValid || status == ResolvesToAlreadyValid;
}
};

1
src/libutil/abstract-setting-to-json.hh
View file

@ -10,6 +10,7 @@ std::map<std::string, nlohmann::json> BaseSetting<T>::toJSONObject()
auto obj = AbstractSetting::toJSONObject();
obj.emplace("value", value);
obj.emplace("defaultValue", defaultValue);
obj.emplace("documentDefault", documentDefault);
return obj;
}
}

70
src/libutil/affinity.cc
View file

@ -1,70 +0,0 @@
#include "types.hh"
#include "util.hh"
#include "affinity.hh"
#if __linux__
#include <sched.h>
#endif
namespace nix {
#if __linux__
static bool didSaveAffinity = false;
static cpu_set_t savedAffinity;
std::ostream& operator<<(std::ostream &os, const cpu_set_t &cset)
{
auto count = CPU_COUNT(&cset);
for (int i=0; i < count; ++i)
{
os << (CPU_ISSET(i,&cset) ? "1" : "0");
}
return os;
}
#endif
void setAffinityTo(int cpu)
{
#if __linux__
if (sched_getaffinity(0, sizeof(cpu_set_t), &savedAffinity) == -1) return;
didSaveAffinity = true;
debug(format("locking this thread to CPU %1%") % cpu);
cpu_set_t newAffinity;
CPU_ZERO(&newAffinity);
CPU_SET(cpu, &newAffinity);
if (sched_setaffinity(0, sizeof(cpu_set_t), &newAffinity) == -1)
printError("failed to lock thread to CPU %1%", cpu);
#endif
}
int lockToCurrentCPU()
{
#if __linux__
int cpu = sched_getcpu();
if (cpu != -1) setAffinityTo(cpu);
return cpu;
#else
return -1;
#endif
}
void restoreAffinity()
{
#if __linux__
if (!didSaveAffinity) return;
if (sched_setaffinity(0, sizeof(cpu_set_t), &savedAffinity) == -1)
{
std::ostringstream oss;
oss << savedAffinity;
printError("failed to restore CPU affinity %1%", oss.str());
}
#endif
}
}

9
src/libutil/affinity.hh
View file

@ -1,9 +0,0 @@
#pragma once
namespace nix {
void setAffinityTo(int cpu);
int lockToCurrentCPU();
void restoreAffinity();
}

4
src/libutil/args.cc
View file

@ -39,7 +39,7 @@ void Completions::add(std::string completion, std::string description)
bool Completion::operator<(const Completion & other) const
{ return completion < other.completion || (completion == other.completion && description < other.description); }
bool pathCompletions = false;
CompletionType completionType = ctNormal;
std::shared_ptr<Completions> completions;
std::string completionMarker = "___COMPLETE___";
@ -277,7 +277,7 @@ Args::Flag Args::Flag::mkHashTypeOptFlag(std::string && longName, std::optional<
static void _completePath(std::string_view prefix, bool onlyDirs)
{
pathCompletions = true;
completionType = ctFilenames;
glob_t globbuf;
int flags = GLOB_NOESCAPE | GLOB_TILDE;
#ifdef GLOB_ONLYDIR

8
src/libutil/args.hh
View file

@ -237,7 +237,13 @@ public:
void add(std::string completion, std::string description = "");
};
extern std::shared_ptr<Completions> completions;
extern bool pathCompletions;
enum CompletionType {
ctNormal,
ctFilenames,
ctAttrs
};
extern CompletionType completionType;
std::optional<std::string> needsCompletion(std::string_view s);

10
src/libutil/config.hh
View file

@ -232,16 +232,19 @@ protected:
T value;
const T defaultValue;
const bool documentDefault;
public:
BaseSetting(const T & def,
const bool documentDefault,
const std::string & name,
const std::string & description,
const std::set<std::string> & aliases = {})
: AbstractSetting(name, description, aliases)
, value(def)
, defaultValue(def)
, documentDefault(documentDefault)
{ }
operator const T &() const { return value; }
@ -288,8 +291,9 @@ public:
const T & def,
const std::string & name,
const std::string & description,
const std::set<std::string> & aliases = {})
: BaseSetting<T>(def, name, description, aliases)
const std::set<std::string> & aliases = {},
const bool documentDefault = true)
: BaseSetting<T>(def, documentDefault, name, description, aliases)
{
options->addSetting(this);
}
@ -311,7 +315,7 @@ public:
const std::string & name,
const std::string & description,
const std::set<std::string> & aliases = {})
: BaseSetting<Path>(def, name, description, aliases)
: BaseSetting<Path>(def, true, name, description, aliases)
, allowEmpty(allowEmpty)
{
options->addSetting(this);

5
src/libutil/logging.hh
View file

@ -189,13 +189,14 @@ extern Verbosity verbosity; /* suppress msgs > this */
/* Print a string message if the current log level is at least the specified
level. Note that this has to be implemented as a macro to ensure that the
arguments are evaluated lazily. */
#define printMsg(level, args...) \
#define printMsgUsing(loggerParam, level, args...) \
do { \
auto __lvl = level; \
if (__lvl <= nix::verbosity) { \
logger->log(__lvl, fmt(args)); \
loggerParam->log(__lvl, fmt(args)); \
} \
} while (0)
#define printMsg(level, args...) printMsgUsing(logger, level, args)
#define printError(args...) printMsg(lvlError, args)
#define notice(args...) printMsg(lvlNotice, args)

9
src/libutil/tarfile.cc
View file

@ -93,9 +93,16 @@ static void extract_archive(TarArchive & archive, const Path & destDir)
else
archive.check(r);
archive_entry_set_pathname(entry,
archive_entry_copy_pathname(entry,
(destDir + "/" + name).c_str());
// Patch hardlink path
const char *original_hardlink = archive_entry_hardlink(entry);
if (original_hardlink) {
archive_entry_copy_hardlink(entry,
(destDir + "/" + original_hardlink).c_str());
}
archive.check(archive_read_extract(archive.archive, entry, flags));
}

2
src/libutil/tests/config.cc
View file

@ -161,7 +161,7 @@ namespace nix {
Setting<std::string> setting{&config, "", "name-of-the-setting", "description"};
setting.assign("value");
ASSERT_EQ(config.toJSON().dump(), R"#({"name-of-the-setting":{"aliases":[],"defaultValue":"","description":"description\n","value":"value"}})#");
ASSERT_EQ(config.toJSON().dump(), R"#({"name-of-the-setting":{"aliases":[],"defaultValue":"","description":"description\n","documentDefault":true,"value":"value"}})#");
}
TEST(Config, setSettingAlias) {

3
src/libutil/thread-pool.cc
View file

@ -1,13 +1,10 @@
#include "thread-pool.hh"
#include "affinity.hh"
namespace nix {
ThreadPool::ThreadPool(size_t _maxThreads)
: maxThreads(_maxThreads)
{
restoreAffinity(); // FIXME
if (!maxThreads) {
maxThreads = std::thread::hardware_concurrency();
if (!maxThreads) maxThreads = 1;

12
src/libutil/util.cc
View file

@ -1,5 +1,4 @@
#include "util.hh"
#include "affinity.hh"
#include "sync.hh"
#include "finally.hh"
#include "serialise.hh"
@ -1004,7 +1003,6 @@ pid_t startProcess(std::function<void()> fun, const ProcessOptions & options)
if (options.dieWithParent && prctl(PR_SET_PDEATHSIG, SIGKILL) == -1)
throw SysError("setting death signal");
#endif
restoreAffinity();
fun();
} catch (std::exception & e) {
try {
@ -1660,6 +1658,14 @@ void restoreMountNamespace()
#endif
}
void unshareFilesystem()
{
#ifdef __linux__
if (unshare(CLONE_FS) != 0 && errno != EPERM)
throw SysError("unsharing filesystem state in download thread");
#endif
}
void restoreProcessContext(bool restoreMounts)
{
restoreSignals();
@ -1667,8 +1673,6 @@ void restoreProcessContext(bool restoreMounts)
restoreMountNamespace();
}
restoreAffinity();
#if __linux__
if (savedStackSize) {
struct rlimit limit;

7
src/libutil/util.hh
View file

@ -300,7 +300,7 @@ void setStackSize(size_t stackSize);
/* Restore the original inherited Unix process context (such as signal
masks, stack size, CPU affinity). */
masks, stack size). */
void restoreProcessContext(bool restoreMounts = true);
/* Save the current mount namespace. Ignored if called more than
@ -311,6 +311,11 @@ void saveMountNamespace();
if saveMountNamespace() was never called. */
void restoreMountNamespace();
/* Cause this thread to not share any FS attributes with the main
thread, because this causes setns() in restoreMountNamespace() to
fail. */
void unshareFilesystem();
class ExecError : public Error
{

1
src/nix-build/nix-build.cc
View file

@ -14,7 +14,6 @@
#include "local-fs-store.hh"
#include "globals.hh"
#include "derivations.hh"
#include "affinity.hh"
#include "util.hh"
#include "shared.hh"
#include "path-with-outputs.hh"

54
src/nix-channel/nix-channel.cc
View file

@ -88,15 +88,6 @@ static void update(const StringSet & channelNames)
for (const auto & channel : channels) {
auto name = channel.first;
auto url = channel.second;
if (!(channelNames.empty() || channelNames.count(name)))
continue;
// We want to download the url to a file to see if it's a tarball while also checking if we
// got redirected in the process, so that we can grab the various parts of a nix channel
// definition from a consistent location if the redirect changes mid-download.
auto result = fetchers::downloadFile(store, url, std::string(baseNameOf(url)), false);
auto filename = store->toRealPath(result.storePath);
url = result.effectiveUrl;
// If the URL contains a version number, append it to the name
// attribute (so that "nix-env -q" on the channels profile
@ -109,30 +100,43 @@ static void update(const StringSet & channelNames)
std::string extraAttrs;
bool unpacked = false;
if (std::regex_search(filename, std::regex("\\.tar\\.(gz|bz2|xz)$"))) {
runProgram(settings.nixBinDir + "/nix-build", false, { "--no-out-link", "--expr", "import " + unpackChannelPath +
"{ name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; }" });
unpacked = true;
}
if (!(channelNames.empty() || channelNames.count(name))) {
// no need to update this channel, reuse the existing store path
Path symlink = profile + "/" + name;
Path storepath = dirOf(readLink(symlink));
exprs.push_back("f: rec { name = \"" + cname + "\"; type = \"derivation\"; outputs = [\"out\"]; system = \"builtin\"; outPath = builtins.storePath \"" + storepath + "\"; out = { inherit outPath; };}");
} else {
// We want to download the url to a file to see if it's a tarball while also checking if we
// got redirected in the process, so that we can grab the various parts of a nix channel
// definition from a consistent location if the redirect changes mid-download.
auto result = fetchers::downloadFile(store, url, std::string(baseNameOf(url)), false);
auto filename = store->toRealPath(result.storePath);
url = result.effectiveUrl;
if (!unpacked) {
// Download the channel tarball.
try {
filename = store->toRealPath(fetchers::downloadFile(store, url + "/nixexprs.tar.xz", "nixexprs.tar.xz", false).storePath);
} catch (FileTransferError & e) {
filename = store->toRealPath(fetchers::downloadFile(store, url + "/nixexprs.tar.bz2", "nixexprs.tar.bz2", false).storePath);
bool unpacked = false;
if (std::regex_search(filename, std::regex("\\.tar\\.(gz|bz2|xz)$"))) {
runProgram(settings.nixBinDir + "/nix-build", false, { "--no-out-link", "--expr", "import " + unpackChannelPath +
"{ name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; }" });
unpacked = true;
}
}
// Regardless of where it came from, add the expression representing this channel to accumulated expression
exprs.push_back("f: f { name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; " + extraAttrs + " }");
if (!unpacked) {
// Download the channel tarball.
try {
filename = store->toRealPath(fetchers::downloadFile(store, url + "/nixexprs.tar.xz", "nixexprs.tar.xz", false).storePath);
} catch (FileTransferError & e) {
filename = store->toRealPath(fetchers::downloadFile(store, url + "/nixexprs.tar.bz2", "nixexprs.tar.bz2", false).storePath);
}
}
// Regardless of where it came from, add the expression representing this channel to accumulated expression
exprs.push_back("f: f { name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; " + extraAttrs + " }");
}
}
// Unpack the channel tarballs into the Nix store and install them
// into the channels profile.
std::cerr << "unpacking channels...\n";
Strings envArgs{ "--profile", profile, "--file", unpackChannelPath, "--install", "--from-expression" };
Strings envArgs{ "--profile", profile, "--file", unpackChannelPath, "--install", "--remove-all", "--from-expression" };
for (auto & expr : exprs)
envArgs.push_back(std::move(expr));
envArgs.push_back("--quiet");

3
src/nix/app.cc
View file

@ -83,11 +83,14 @@ UnresolvedApp Installable::toApp(EvalState & state)
auto outPath = cursor->getAttr(state.sOutPath)->getString();
auto outputName = cursor->getAttr(state.sOutputName)->getString();
auto name = cursor->getAttr(state.sName)->getString();
auto aPname = cursor->maybeGetAttr("pname");
auto aMeta = cursor->maybeGetAttr("meta");
auto aMainProgram = aMeta ? aMeta->maybeGetAttr("mainProgram") : nullptr;
auto mainProgram =
aMainProgram
? aMainProgram->getString()
: aPname
? aPname->getString()
: DrvName(name).name;
auto program = outPath + "/bin/" + mainProgram;
return UnresolvedApp { App {

1
src/nix/develop.cc
View file

@ -5,7 +5,6 @@
#include "store-api.hh"
#include "path-with-outputs.hh"
#include "derivations.hh"
#include "affinity.hh"
#include "progress-bar.hh"
#include "run.hh"

9
src/nix/main.cc
View file

@ -310,7 +310,14 @@ void mainWrapped(int argc, char * * argv)
Finally printCompletions([&]()
{
if (completions) {
std::cout << (pathCompletions ? "filenames\n" : "no-filenames\n");
switch (completionType) {
case ctNormal:
std::cout << "normal\n"; break;
case ctFilenames:
std::cout << "filenames\n"; break;
case ctAttrs:
std::cout << "attrs\n"; break;
}
for (auto & s : *completions)
std::cout << s.completion << "\t" << s.description << "\n";
}

29
src/nix/repl.cc
View file

@ -28,7 +28,6 @@ extern "C" {
#include "common-eval-args.hh"
#include "get-drvs.hh"
#include "derivations.hh"
#include "affinity.hh"
#include "globals.hh"
#include "command.hh"
#include "finally.hh"
@ -430,7 +429,8 @@ bool NixRepl::processLine(string line)
<< " :s <expr> Build dependencies of derivation, then start nix-shell\n"
<< " :t <expr> Describe result of evaluation\n"
<< " :u <expr> Build derivation, then start nix-shell\n"
<< " :doc <expr> Show documentation of a builtin function\n";
<< " :doc <expr> Show documentation of a builtin function\n"
<< " :log <expr> Show logs for a derivation\n";
}
else if (command == ":a" || command == ":add") {
@ -500,7 +500,7 @@ bool NixRepl::processLine(string line)
runNix("nix-shell", {state->store->printStorePath(drvPath)});
}
else if (command == ":b" || command == ":i" || command == ":s") {
else if (command == ":b" || command == ":i" || command == ":s" || command == ":log") {
Value v;
evalString(arg, v);
StorePath drvPath = getDerivationPath(v);
@ -514,6 +514,27 @@ bool NixRepl::processLine(string line)
logger->cout(" %s -> %s", outputName, state->store->printStorePath(outputPath));
} else if (command == ":i") {
runNix("nix-env", {"-i", drvPathRaw});
} else if (command == ":log") {
settings.readOnlyMode = true;
Finally roModeReset([&]() {
settings.readOnlyMode = false;
});
auto subs = getDefaultSubstituters();
subs.push_front(state->store);
bool foundLog = false;
RunPager pager;
for (auto & sub : subs) {
auto log = sub->getBuildLog(drvPath);
if (log) {
printInfo("got build log for '%s' from '%s'", drvPathRaw, sub->getUri());
logger->writeToStdout(*log);
foundLog = true;
break;
}
}
if (!foundLog) throw Error("build log of '%s' is not available", drvPathRaw);
} else {
runNix("nix-shell", {drvPathRaw});
}
@ -650,6 +671,8 @@ void NixRepl::addVarToScope(const Symbol & name, Value & v)
{
if (displ >= envSize)
throw Error("environment full; cannot add more variables");
if (auto oldVar = staticEnv.find(name); oldVar != staticEnv.vars.end())
staticEnv.vars.erase(oldVar);
staticEnv.vars.emplace_back(name, displ);
staticEnv.sort();
env->values[displ++] = &v;

7
src/nix/repl.md
View file

@ -35,14 +35,17 @@ R""(
nix-repl> emacs.drvPath
"/nix/store/lp0sjrhgg03y2n0l10n70rg0k7hhyz0l-emacs-27.1.drv"
nix-repl> drv = runCommand "hello" { buildInputs = [ hello ]; } "hello > $out"
nix-repl> drv = runCommand "hello" { buildInputs = [ hello ]; } "hello; hello > $out"
nix-repl> :b x
nix-repl> :b drv
this derivation produced the following outputs:
out -> /nix/store/0njwbgwmkwls0w5dv9mpc1pq5fj39q0l-hello
nix-repl> builtins.readFile drv
"Hello, world!\n"
nix-repl> :log drv
Hello, world!
```
# Description

1
src/nix/run.cc
View file

@ -8,7 +8,6 @@
#include "finally.hh"
#include "fs-accessor.hh"
#include "progress-bar.hh"
#include "affinity.hh"
#include "eval.hh"
#if __linux__

13
src/nix/run.md
View file

@ -43,10 +43,15 @@ program specified by the app definition.
If *installable* evaluates to a derivation, it will try to execute the
program `<out>/bin/<name>`, where *out* is the primary output store
path of the derivation and *name* is the `meta.mainProgram` attribute
of the derivation if it exists, and otherwise the name part of the
value of the `name` attribute of the derivation (e.g. if `name` is set
to `hello-1.10`, it will run `$out/bin/hello`).
path of the derivation, and *name* is the first of the following that
exists:
* The `meta.mainProgram` attribute of the derivation.
* The `pname` attribute of the derivation.
* The name part of the value of the `name` attribute of the derivation.
For instance, if `name` is set to `hello-1.10`, `nix run` will run
`$out/bin/hello`.
# Flake output attributes

6
src/nix/shell.md
View file

@ -41,8 +41,8 @@ R""(
# Description
`nix shell` runs a command in an environment in which the `$PATH`
variable provides the specified *installables*. If not command is
specified, it starts the default shell of your user account.
`nix shell` runs a command in an environment in which the `$PATH` variable
provides the specified *installables*. If no command is specified, it starts the
default shell of your user account specified by `$SHELL`.
)""

17
src/nix/why-depends.cc
View file

@ -34,8 +34,21 @@ struct CmdWhyDepends : SourceExprCommand
CmdWhyDepends()
{
expectArg("package", &_package);
expectArg("dependency", &_dependency);
expectArgs({
.label = "package",
.handler = {&_package},
.completer = {[&](size_t, std::string_view prefix) {
completeInstallable(prefix);
}}
});
expectArgs({
.label = "dependency",
.handler = {&_dependency},
.completer = {[&](size_t, std::string_view prefix) {
completeInstallable(prefix);
}}
});
addFlag({
.longName = "all",

21
src/toml11/LICENSE Normal file
View file

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2017 Toru Niina
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

1966
src/toml11/README.md Normal file
View file

File diff suppressed because it is too large Load diff

46
src/toml11/toml.hpp Normal file
View file

@ -0,0 +1,46 @@
/*
* The MIT License (MIT)
*
* Copyright (c) 2017 Toru Niina
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
#ifndef TOML_FOR_MODERN_CPP
#define TOML_FOR_MODERN_CPP
#ifndef __cplusplus
# error "__cplusplus is not defined"
#endif
#if __cplusplus < 201103L && _MSC_VER < 1900
# error "toml11 requires C++11 or later."
#endif
#define TOML11_VERSION_MAJOR 3
#define TOML11_VERSION_MINOR 7
#define TOML11_VERSION_PATCH 0
#include "toml/parser.hpp"
#include "toml/literal.hpp"
#include "toml/serializer.hpp"
#include "toml/get.hpp"
#include "toml/macros.hpp"
#endif// TOML_FOR_MODERN_CPP

64
src/toml11/toml/color.hpp Normal file
View file

@ -0,0 +1,64 @@
#ifndef TOML11_COLOR_HPP
#define TOML11_COLOR_HPP
#include <cstdint>
#include <ostream>
#ifdef TOML11_COLORIZE_ERROR_MESSAGE
#define TOML11_ERROR_MESSAGE_COLORIZED true
#else
#define TOML11_ERROR_MESSAGE_COLORIZED false
#endif
namespace toml
{
// put ANSI escape sequence to ostream
namespace color_ansi
{
namespace detail
{
inline int colorize_index()
{
static const int index = std::ios_base::xalloc();
return index;
}
} // detail
inline std::ostream& colorize(std::ostream& os)
{
// by default, it is zero.
os.iword(detail::colorize_index()) = 1;
return os;
}
inline std::ostream& nocolorize(std::ostream& os)
{
os.iword(detail::colorize_index()) = 0;
return os;
}
inline std::ostream& reset (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[00m";} return os;}
inline std::ostream& bold (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[01m";} return os;}
inline std::ostream& grey (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[30m";} return os;}
inline std::ostream& red (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[31m";} return os;}
inline std::ostream& green (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[32m";} return os;}
inline std::ostream& yellow (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[33m";} return os;}
inline std::ostream& blue (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[34m";} return os;}
inline std::ostream& magenta(std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[35m";} return os;}
inline std::ostream& cyan (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[36m";} return os;}
inline std::ostream& white (std::ostream& os)
{if(os.iword(detail::colorize_index()) == 1) {os << "\033[37m";} return os;}
} // color_ansi
// ANSI escape sequence is the only and default colorization method currently
namespace color = color_ansi;
} // toml
#endif// TOML11_COLOR_HPP

306
src/toml11/toml/combinator.hpp Normal file
View file

@ -0,0 +1,306 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_COMBINATOR_HPP
#define TOML11_COMBINATOR_HPP
#include <cassert>
#include <cctype>
#include <cstdio>
#include <array>
#include <iomanip>
#include <iterator>
#include <limits>
#include <type_traits>
#include "region.hpp"
#include "result.hpp"
#include "traits.hpp"
#include "utility.hpp"
// they scans characters and returns region if it matches to the condition.
// when they fail, it does not change the location.
// in lexer.hpp, these are used.
namespace toml
{
namespace detail
{
// to output character as an error message.
inline std::string show_char(const char c)
{
// It suppresses an error that occurs only in Debug mode of MSVC++ on Windows.
// I'm not completely sure but they check the value of char to be in the
// range [0, 256) and some of the COMPLETELY VALID utf-8 character sometimes
// has negative value (if char has sign). So here it re-interprets c as
// unsigned char through pointer. In general, converting pointer to a
// pointer that has different type cause UB, but `(signed|unsigned)?char`
// are one of the exceptions. Converting pointer only to char and std::byte
// (c++17) are valid.
if(std::isgraph(*reinterpret_cast<unsigned char const*>(std::addressof(c))))
{
return std::string(1, c);
}
else
{
std::array<char, 5> buf;
buf.fill('\0');
const auto r = std::snprintf(
buf.data(), buf.size(), "0x%02x", static_cast<int>(c) & 0xFF);
(void) r; // Unused variable warning
assert(r == static_cast<int>(buf.size()) - 1);
return std::string(buf.data());
}
}
template<char C>
struct character
{
static constexpr char target = C;
static result<region, none_t>
invoke(location& loc)
{
if(loc.iter() == loc.end()) {return none();}
const auto first = loc.iter();
const char c = *(loc.iter());
if(c != target)
{
return none();
}
loc.advance(); // update location
return ok(region(loc, first, loc.iter()));
}
};
template<char C>
constexpr char character<C>::target;
// closed interval [Low, Up]. both Low and Up are included.
template<char Low, char Up>
struct in_range
{
// assuming ascii part of UTF-8...
static_assert(Low <= Up, "lower bound should be less than upper bound.");
static constexpr char upper = Up;
static constexpr char lower = Low;
static result<region, none_t>
invoke(location& loc)
{
if(loc.iter() == loc.end()) {return none();}
const auto first = loc.iter();
const char c = *(loc.iter());
if(c < lower || upper < c)
{
return none();
}
loc.advance();
return ok(region(loc, first, loc.iter()));
}
};
template<char L, char U> constexpr char in_range<L, U>::upper;
template<char L, char U> constexpr char in_range<L, U>::lower;
// keep iterator if `Combinator` matches. otherwise, increment `iter` by 1 char.
// for detecting invalid characters, like control sequences in toml string.
template<typename Combinator>
struct exclude
{
static result<region, none_t>
invoke(location& loc)
{
if(loc.iter() == loc.end()) {return none();}
auto first = loc.iter();
auto rslt = Combinator::invoke(loc);
if(rslt.is_ok())
{
loc.reset(first);
return none();
}
loc.reset(std::next(first)); // XXX maybe loc.advance() is okay but...
return ok(region(loc, first, loc.iter()));
}
};
// increment `iter`, if matches. otherwise, just return empty string.
template<typename Combinator>
struct maybe
{
static result<region, none_t>
invoke(location& loc)
{
const auto rslt = Combinator::invoke(loc);
if(rslt.is_ok())
{
return rslt;
}
return ok(region(loc));
}
};
template<typename ... Ts>
struct sequence;
template<typename Head, typename ... Tail>
struct sequence<Head, Tail...>
{
static result<region, none_t>
invoke(location& loc)
{
const auto first = loc.iter();
auto rslt = Head::invoke(loc);
if(rslt.is_err())
{
loc.reset(first);
return none();
}
return sequence<Tail...>::invoke(loc, std::move(rslt.unwrap()), first);
}
// called from the above function only, recursively.
template<typename Iterator>
static result<region, none_t>
invoke(location& loc, region reg, Iterator first)
{
const auto rslt = Head::invoke(loc);
if(rslt.is_err())
{
loc.reset(first);
return none();
}
reg += rslt.unwrap(); // concat regions
return sequence<Tail...>::invoke(loc, std::move(reg), first);
}
};
template<typename Head>
struct sequence<Head>
{
// would be called from sequence<T ...>::invoke only.
template<typename Iterator>
static result<region, none_t>
invoke(location& loc, region reg, Iterator first)
{
const auto rslt = Head::invoke(loc);
if(rslt.is_err())
{
loc.reset(first);
return none();
}
reg += rslt.unwrap(); // concat regions
return ok(reg);
}
};
template<typename ... Ts>
struct either;
template<typename Head, typename ... Tail>
struct either<Head, Tail...>
{
static result<region, none_t>
invoke(location& loc)
{
const auto rslt = Head::invoke(loc);
if(rslt.is_ok()) {return rslt;}
return either<Tail...>::invoke(loc);
}
};
template<typename Head>
struct either<Head>
{
static result<region, none_t>
invoke(location& loc)
{
return Head::invoke(loc);
}
};
template<typename T, typename N>
struct repeat;
template<std::size_t N> struct exactly{};
template<std::size_t N> struct at_least{};
struct unlimited{};
template<typename T, std::size_t N>
struct repeat<T, exactly<N>>
{
static result<region, none_t>
invoke(location& loc)
{
region retval(loc);
const auto first = loc.iter();
for(std::size_t i=0; i<N; ++i)
{
auto rslt = T::invoke(loc);
if(rslt.is_err())
{
loc.reset(first);
return none();
}
retval += rslt.unwrap();
}
return ok(std::move(retval));
}
};
template<typename T, std::size_t N>
struct repeat<T, at_least<N>>
{
static result<region, none_t>
invoke(location& loc)
{
region retval(loc);
const auto first = loc.iter();
for(std::size_t i=0; i<N; ++i)
{
auto rslt = T::invoke(loc);
if(rslt.is_err())
{
loc.reset(first);
return none();
}
retval += rslt.unwrap();
}
while(true)
{
auto rslt = T::invoke(loc);
if(rslt.is_err())
{
return ok(std::move(retval));
}
retval += rslt.unwrap();
}
}
};
template<typename T>
struct repeat<T, unlimited>
{
static result<region, none_t>
invoke(location& loc)
{
region retval(loc);
while(true)
{
auto rslt = T::invoke(loc);
if(rslt.is_err())
{
return ok(std::move(retval));
}
retval += rslt.unwrap();
}
}
};
} // detail
} // toml
#endif// TOML11_COMBINATOR_HPP

472
src/toml11/toml/comments.hpp Normal file
View file

@ -0,0 +1,472 @@
// Copyright Toru Niina 2019.
// Distributed under the MIT License.
#ifndef TOML11_COMMENTS_HPP
#define TOML11_COMMENTS_HPP
#include <initializer_list>
#include <iterator>
#include <stdexcept>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
#ifdef TOML11_PRESERVE_COMMENTS_BY_DEFAULT
# define TOML11_DEFAULT_COMMENT_STRATEGY ::toml::preserve_comments
#else
# define TOML11_DEFAULT_COMMENT_STRATEGY ::toml::discard_comments
#endif
// This file provides mainly two classes, `preserve_comments` and `discard_comments`.
// Those two are a container that have the same interface as `std::vector<std::string>`
// but bahaves in the opposite way. `preserve_comments` is just the same as
// `std::vector<std::string>` and each `std::string` corresponds to a comment line.
// Conversely, `discard_comments` discards all the strings and ignores everything
// assigned in it. `discard_comments` is always empty and you will encounter an
// error whenever you access to the element.
namespace toml
{
struct discard_comments; // forward decl
// use it in the following way
//
// const toml::basic_value<toml::preserve_comments> data =
// toml::parse<toml::preserve_comments>("example.toml");
//
// the interface is almost the same as std::vector<std::string>.
struct preserve_comments
{
// `container_type` is not provided in discard_comments.
// do not use this inner-type in a generic code.
using container_type = std::vector<std::string>;
using size_type = container_type::size_type;
using difference_type = container_type::difference_type;
using value_type = container_type::value_type;
using reference = container_type::reference;
using const_reference = container_type::const_reference;
using pointer = container_type::pointer;
using const_pointer = container_type::const_pointer;
using iterator = container_type::iterator;
using const_iterator = container_type::const_iterator;
using reverse_iterator = container_type::reverse_iterator;
using const_reverse_iterator = container_type::const_reverse_iterator;
preserve_comments() = default;
~preserve_comments() = default;
preserve_comments(preserve_comments const&) = default;
preserve_comments(preserve_comments &&) = default;
preserve_comments& operator=(preserve_comments const&) = default;
preserve_comments& operator=(preserve_comments &&) = default;
explicit preserve_comments(const std::vector<std::string>& c): comments(c){}
explicit preserve_comments(std::vector<std::string>&& c)
: comments(std::move(c))
{}
preserve_comments& operator=(const std::vector<std::string>& c)
{
comments = c;
return *this;
}
preserve_comments& operator=(std::vector<std::string>&& c)
{
comments = std::move(c);
return *this;
}
explicit preserve_comments(const discard_comments&) {}
explicit preserve_comments(size_type n): comments(n) {}
preserve_comments(size_type n, const std::string& x): comments(n, x) {}
preserve_comments(std::initializer_list<std::string> x): comments(x) {}
template<typename InputIterator>
preserve_comments(InputIterator first, InputIterator last)
: comments(first, last)
{}
template<typename InputIterator>
void assign(InputIterator first, InputIterator last) {comments.assign(first, last);}
void assign(std::initializer_list<std::string> ini) {comments.assign(ini);}
void assign(size_type n, const std::string& val) {comments.assign(n, val);}
// Related to the issue #97.
//
// It is known that `std::vector::insert` and `std::vector::erase` in
// the standard library implementation included in GCC 4.8.5 takes
// `std::vector::iterator` instead of `std::vector::const_iterator`.
// Because of the const-correctness, we cannot convert a `const_iterator` to
// an `iterator`. It causes compilation error in GCC 4.8.5.
#if defined(__GNUC__) && defined(__GNUC_MINOR__) && defined(__GNUC_PATCHLEVEL__) && !defined(__clang__)
# if (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__) <= 40805
# define TOML11_WORKAROUND_GCC_4_8_X_STANDARD_LIBRARY_IMPLEMENTATION
# endif
#endif
#ifdef TOML11_WORKAROUND_GCC_4_8_X_STANDARD_LIBRARY_IMPLEMENTATION
iterator insert(iterator p, const std::string& x)
{
return comments.insert(p, x);
}
iterator insert(iterator p, std::string&& x)
{
return comments.insert(p, std::move(x));
}
void insert(iterator p, size_type n, const std::string& x)
{
return comments.insert(p, n, x);
}
template<typename InputIterator>
void insert(iterator p, InputIterator first, InputIterator last)
{
return comments.insert(p, first, last);
}
void insert(iterator p, std::initializer_list<std::string> ini)
{
return comments.insert(p, ini);
}
template<typename ... Ts>
iterator emplace(iterator p, Ts&& ... args)
{
return comments.emplace(p, std::forward<Ts>(args)...);
}
iterator erase(iterator pos) {return comments.erase(pos);}
iterator erase(iterator first, iterator last)
{
return comments.erase(first, last);
}
#else
iterator insert(const_iterator p, const std::string& x)
{
return comments.insert(p, x);
}
iterator insert(const_iterator p, std::string&& x)
{
return comments.insert(p, std::move(x));
}
iterator insert(const_iterator p, size_type n, const std::string& x)
{
return comments.insert(p, n, x);
}
template<typename InputIterator>
iterator insert(const_iterator p, InputIterator first, InputIterator last)
{
return comments.insert(p, first, last);
}
iterator insert(const_iterator p, std::initializer_list<std::string> ini)
{
return comments.insert(p, ini);
}
template<typename ... Ts>
iterator emplace(const_iterator p, Ts&& ... args)
{
return comments.emplace(p, std::forward<Ts>(args)...);
}
iterator erase(const_iterator pos) {return comments.erase(pos);}
iterator erase(const_iterator first, const_iterator last)
{
return comments.erase(first, last);
}
#endif
void swap(preserve_comments& other) {comments.swap(other.comments);}
void push_back(const std::string& v) {comments.push_back(v);}
void push_back(std::string&& v) {comments.push_back(std::move(v));}
void pop_back() {comments.pop_back();}
template<typename ... Ts>
void emplace_back(Ts&& ... args) {comments.emplace_back(std::forward<Ts>(args)...);}
void clear() {comments.clear();}
size_type size() const noexcept {return comments.size();}
size_type max_size() const noexcept {return comments.max_size();}
size_type capacity() const noexcept {return comments.capacity();}
bool empty() const noexcept {return comments.empty();}
void reserve(size_type n) {comments.reserve(n);}
void resize(size_type n) {comments.resize(n);}
void resize(size_type n, const std::string& c) {comments.resize(n, c);}
void shrink_to_fit() {comments.shrink_to_fit();}
reference operator[](const size_type n) noexcept {return comments[n];}
const_reference operator[](const size_type n) const noexcept {return comments[n];}
reference at(const size_type n) {return comments.at(n);}
const_reference at(const size_type n) const {return comments.at(n);}
reference front() noexcept {return comments.front();}
const_reference front() const noexcept {return comments.front();}
reference back() noexcept {return comments.back();}
const_reference back() const noexcept {return comments.back();}
pointer data() noexcept {return comments.data();}
const_pointer data() const noexcept {return comments.data();}
iterator begin() noexcept {return comments.begin();}
iterator end() noexcept {return comments.end();}
const_iterator begin() const noexcept {return comments.begin();}
const_iterator end() const noexcept {return comments.end();}
const_iterator cbegin() const noexcept {return comments.cbegin();}
const_iterator cend() const noexcept {return comments.cend();}
reverse_iterator rbegin() noexcept {return comments.rbegin();}
reverse_iterator rend() noexcept {return comments.rend();}
const_reverse_iterator rbegin() const noexcept {return comments.rbegin();}
const_reverse_iterator rend() const noexcept {return comments.rend();}
const_reverse_iterator crbegin() const noexcept {return comments.crbegin();}
const_reverse_iterator crend() const noexcept {return comments.crend();}
friend bool operator==(const preserve_comments&, const preserve_comments&);
friend bool operator!=(const preserve_comments&, const preserve_comments&);
friend bool operator< (const preserve_comments&, const preserve_comments&);
friend bool operator<=(const preserve_comments&, const preserve_comments&);
friend bool operator> (const preserve_comments&, const preserve_comments&);
friend bool operator>=(const preserve_comments&, const preserve_comments&);
friend void swap(preserve_comments&, std::vector<std::string>&);
friend void swap(std::vector<std::string>&, preserve_comments&);
private:
container_type comments;
};
inline bool operator==(const preserve_comments& lhs, const preserve_comments& rhs) {return lhs.comments == rhs.comments;}
inline bool operator!=(const preserve_comments& lhs, const preserve_comments& rhs) {return lhs.comments != rhs.comments;}
inline bool operator< (const preserve_comments& lhs, const preserve_comments& rhs) {return lhs.comments < rhs.comments;}
inline bool operator<=(const preserve_comments& lhs, const preserve_comments& rhs) {return lhs.comments <= rhs.comments;}
inline bool operator> (const preserve_comments& lhs, const preserve_comments& rhs) {return lhs.comments > rhs.comments;}
inline bool operator>=(const preserve_comments& lhs, const preserve_comments& rhs) {return lhs.comments >= rhs.comments;}
inline void swap(preserve_comments& lhs, preserve_comments& rhs)
{
lhs.swap(rhs);
return;
}
inline void swap(preserve_comments& lhs, std::vector<std::string>& rhs)
{
lhs.comments.swap(rhs);
return;
}
inline void swap(std::vector<std::string>& lhs, preserve_comments& rhs)
{
lhs.swap(rhs.comments);
return;
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const preserve_comments& com)
{
for(const auto& c : com)
{
os << '#' << c << '\n';
}
return os;
}
namespace detail
{
// To provide the same interface with `preserve_comments`, `discard_comments`
// should have an iterator. But it does not contain anything, so we need to
// add an iterator that points nothing.
//
// It always points null, so DO NOT unwrap this iterator. It always crashes
// your program.
template<typename T, bool is_const>
struct empty_iterator
{
using value_type = T;
using reference_type = typename std::conditional<is_const, T const&, T&>::type;
using pointer_type = typename std::conditional<is_const, T const*, T*>::type;
using difference_type = std::ptrdiff_t;
using iterator_category = std::random_access_iterator_tag;
empty_iterator() = default;
~empty_iterator() = default;
empty_iterator(empty_iterator const&) = default;
empty_iterator(empty_iterator &&) = default;
empty_iterator& operator=(empty_iterator const&) = default;
empty_iterator& operator=(empty_iterator &&) = default;
// DO NOT call these operators.
reference_type operator*() const noexcept {std::terminate();}
pointer_type operator->() const noexcept {return nullptr;}
reference_type operator[](difference_type) const noexcept {return this->operator*();}
// These operators do nothing.
empty_iterator& operator++() noexcept {return *this;}
empty_iterator operator++(int) noexcept {return *this;}
empty_iterator& operator--() noexcept {return *this;}
empty_iterator operator--(int) noexcept {return *this;}
empty_iterator& operator+=(difference_type) noexcept {return *this;}
empty_iterator& operator-=(difference_type) noexcept {return *this;}
empty_iterator operator+(difference_type) const noexcept {return *this;}
empty_iterator operator-(difference_type) const noexcept {return *this;}
};
template<typename T, bool C>
bool operator==(const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return true;}
template<typename T, bool C>
bool operator!=(const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return false;}
template<typename T, bool C>
bool operator< (const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return false;}
template<typename T, bool C>
bool operator<=(const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return true;}
template<typename T, bool C>
bool operator> (const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return false;}
template<typename T, bool C>
bool operator>=(const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return true;}
template<typename T, bool C>
typename empty_iterator<T, C>::difference_type
operator-(const empty_iterator<T, C>&, const empty_iterator<T, C>&) noexcept {return 0;}
template<typename T, bool C>
empty_iterator<T, C>
operator+(typename empty_iterator<T, C>::difference_type, const empty_iterator<T, C>& rhs) noexcept {return rhs;}
template<typename T, bool C>
empty_iterator<T, C>
operator+(const empty_iterator<T, C>& lhs, typename empty_iterator<T, C>::difference_type) noexcept {return lhs;}
} // detail
// The default comment type. It discards all the comments. It requires only one
// byte to contain, so the memory footprint is smaller than preserve_comments.
//
// It just ignores `push_back`, `insert`, `erase`, and any other modifications.
// IT always returns size() == 0, the iterator taken by `begin()` is always the
// same as that of `end()`, and accessing through `operator[]` or iterators
// always causes a segmentation fault. DO NOT access to the element of this.
//
// Why this is chose as the default type is because the last version (2.x.y)
// does not contain any comments in a value. To minimize the impact on the
// efficiency, this is chosen as a default.
//
// To reduce the memory footprint, later we can try empty base optimization (EBO).
struct discard_comments
{
using size_type = std::size_t;
using difference_type = std::ptrdiff_t;
using value_type = std::string;
using reference = std::string&;
using const_reference = std::string const&;
using pointer = std::string*;
using const_pointer = std::string const*;
using iterator = detail::empty_iterator<std::string, false>;
using const_iterator = detail::empty_iterator<std::string, true>;
using reverse_iterator = detail::empty_iterator<std::string, false>;
using const_reverse_iterator = detail::empty_iterator<std::string, true>;
discard_comments() = default;
~discard_comments() = default;
discard_comments(discard_comments const&) = default;
discard_comments(discard_comments &&) = default;
discard_comments& operator=(discard_comments const&) = default;
discard_comments& operator=(discard_comments &&) = default;
explicit discard_comments(const std::vector<std::string>&) noexcept {}
explicit discard_comments(std::vector<std::string>&&) noexcept {}
discard_comments& operator=(const std::vector<std::string>&) noexcept {return *this;}
discard_comments& operator=(std::vector<std::string>&&) noexcept {return *this;}
explicit discard_comments(const preserve_comments&) noexcept {}
explicit discard_comments(size_type) noexcept {}
discard_comments(size_type, const std::string&) noexcept {}
discard_comments(std::initializer_list<std::string>) noexcept {}
template<typename InputIterator>
discard_comments(InputIterator, InputIterator) noexcept {}
template<typename InputIterator>
void assign(InputIterator, InputIterator) noexcept {}
void assign(std::initializer_list<std::string>) noexcept {}
void assign(size_type, const std::string&) noexcept {}
iterator insert(const_iterator, const std::string&) {return iterator{};}
iterator insert(const_iterator, std::string&&) {return iterator{};}
iterator insert(const_iterator, size_type, const std::string&) {return iterator{};}
template<typename InputIterator>
iterator insert(const_iterator, InputIterator, InputIterator) {return iterator{};}
iterator insert(const_iterator, std::initializer_list<std::string>) {return iterator{};}
template<typename ... Ts>
iterator emplace(const_iterator, Ts&& ...) {return iterator{};}
iterator erase(const_iterator) {return iterator{};}
iterator erase(const_iterator, const_iterator) {return iterator{};}
void swap(discard_comments&) {return;}
void push_back(const std::string&) {return;}
void push_back(std::string&& ) {return;}
void pop_back() {return;}
template<typename ... Ts>
void emplace_back(Ts&& ...) {return;}
void clear() {return;}
size_type size() const noexcept {return 0;}
size_type max_size() const noexcept {return 0;}
size_type capacity() const noexcept {return 0;}
bool empty() const noexcept {return true;}
void reserve(size_type) {return;}
void resize(size_type) {return;}
void resize(size_type, const std::string&) {return;}
void shrink_to_fit() {return;}
// DO NOT access to the element of this container. This container is always
// empty, so accessing through operator[], front/back, data causes address
// error.
reference operator[](const size_type) noexcept {return *data();}
const_reference operator[](const size_type) const noexcept {return *data();}
reference at(const size_type) {throw std::out_of_range("toml::discard_comment is always empty.");}
const_reference at(const size_type) const {throw std::out_of_range("toml::discard_comment is always empty.");}
reference front() noexcept {return *data();}
const_reference front() const noexcept {return *data();}
reference back() noexcept {return *data();}
const_reference back() const noexcept {return *data();}
pointer data() noexcept {return nullptr;}
const_pointer data() const noexcept {return nullptr;}
iterator begin() noexcept {return iterator{};}
iterator end() noexcept {return iterator{};}
const_iterator begin() const noexcept {return const_iterator{};}
const_iterator end() const noexcept {return const_iterator{};}
const_iterator cbegin() const noexcept {return const_iterator{};}
const_iterator cend() const noexcept {return const_iterator{};}
reverse_iterator rbegin() noexcept {return iterator{};}
reverse_iterator rend() noexcept {return iterator{};}
const_reverse_iterator rbegin() const noexcept {return const_iterator{};}
const_reverse_iterator rend() const noexcept {return const_iterator{};}
const_reverse_iterator crbegin() const noexcept {return const_iterator{};}
const_reverse_iterator crend() const noexcept {return const_iterator{};}
};
inline bool operator==(const discard_comments&, const discard_comments&) noexcept {return true;}
inline bool operator!=(const discard_comments&, const discard_comments&) noexcept {return false;}
inline bool operator< (const discard_comments&, const discard_comments&) noexcept {return false;}
inline bool operator<=(const discard_comments&, const discard_comments&) noexcept {return true;}
inline bool operator> (const discard_comments&, const discard_comments&) noexcept {return false;}
inline bool operator>=(const discard_comments&, const discard_comments&) noexcept {return true;}
inline void swap(const discard_comments&, const discard_comments&) noexcept {return;}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const discard_comments&)
{
return os;
}
} // toml11
#endif// TOML11_COMMENTS_HPP

631
src/toml11/toml/datetime.hpp Normal file
View file

@ -0,0 +1,631 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_DATETIME_HPP
#define TOML11_DATETIME_HPP
#include <cstdint>
#include <cstdlib>
#include <ctime>
#include <array>
#include <chrono>
#include <iomanip>
#include <ostream>
#include <tuple>
namespace toml
{
// To avoid non-threadsafe std::localtime. In C11 (not C++11!), localtime_s is
// provided in the absolutely same purpose, but C++11 is actually not compatible
// with C11. We need to dispatch the function depending on the OS.
namespace detail
{
// TODO: find more sophisticated way to handle this
#if (defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE >= 1) || defined(_XOPEN_SOURCE) || defined(_BSD_SOURCE) || defined(_SVID_SOURCE) || defined(_POSIX_SOURCE)
inline std::tm localtime_s(const std::time_t* src)
{
std::tm dst;
const auto result = ::localtime_r(src, &dst);
if (!result) { throw std::runtime_error("localtime_r failed."); }
return dst;
}
inline std::tm gmtime_s(const std::time_t* src)
{
std::tm dst;
const auto result = ::gmtime_r(src, &dst);
if (!result) { throw std::runtime_error("gmtime_r failed."); }
return dst;
}
#elif defined(_MSC_VER)
inline std::tm localtime_s(const std::time_t* src)
{
std::tm dst;
const auto result = ::localtime_s(&dst, src);
if (result) { throw std::runtime_error("localtime_s failed."); }
return dst;
}
inline std::tm gmtime_s(const std::time_t* src)
{
std::tm dst;
const auto result = ::gmtime_s(&dst, src);
if (result) { throw std::runtime_error("gmtime_s failed."); }
return dst;
}
#else // fallback. not threadsafe
inline std::tm localtime_s(const std::time_t* src)
{
const auto result = std::localtime(src);
if (!result) { throw std::runtime_error("localtime failed."); }
return *result;
}
inline std::tm gmtime_s(const std::time_t* src)
{
const auto result = std::gmtime(src);
if (!result) { throw std::runtime_error("gmtime failed."); }
return *result;
}
#endif
} // detail
enum class month_t : std::uint8_t
{
Jan = 0,
Feb = 1,
Mar = 2,
Apr = 3,
May = 4,
Jun = 5,
Jul = 6,
Aug = 7,
Sep = 8,
Oct = 9,
Nov = 10,
Dec = 11
};
struct local_date
{
std::int16_t year; // A.D. (like, 2018)
std::uint8_t month; // [0, 11]
std::uint8_t day; // [1, 31]
local_date(int y, month_t m, int d)
: year (static_cast<std::int16_t>(y)),
month(static_cast<std::uint8_t>(m)),
day (static_cast<std::uint8_t>(d))
{}
explicit local_date(const std::tm& t)
: year (static_cast<std::int16_t>(t.tm_year + 1900)),
month(static_cast<std::uint8_t>(t.tm_mon)),
day (static_cast<std::uint8_t>(t.tm_mday))
{}
explicit local_date(const std::chrono::system_clock::time_point& tp)
{
const auto t = std::chrono::system_clock::to_time_t(tp);
const auto time = detail::localtime_s(&t);
*this = local_date(time);
}
explicit local_date(const std::time_t t)
: local_date(std::chrono::system_clock::from_time_t(t))
{}
operator std::chrono::system_clock::time_point() const
{
// std::mktime returns date as local time zone. no conversion needed
std::tm t;
t.tm_sec = 0;
t.tm_min = 0;
t.tm_hour = 0;
t.tm_mday = static_cast<int>(this->day);
t.tm_mon = static_cast<int>(this->month);
t.tm_year = static_cast<int>(this->year) - 1900;
t.tm_wday = 0; // the value will be ignored
t.tm_yday = 0; // the value will be ignored
t.tm_isdst = -1;
return std::chrono::system_clock::from_time_t(std::mktime(&t));
}
operator std::time_t() const
{
return std::chrono::system_clock::to_time_t(
std::chrono::system_clock::time_point(*this));
}
local_date() = default;
~local_date() = default;
local_date(local_date const&) = default;
local_date(local_date&&) = default;
local_date& operator=(local_date const&) = default;
local_date& operator=(local_date&&) = default;
};
inline bool operator==(const local_date& lhs, const local_date& rhs)
{
return std::make_tuple(lhs.year, lhs.month, lhs.day) ==
std::make_tuple(rhs.year, rhs.month, rhs.day);
}
inline bool operator!=(const local_date& lhs, const local_date& rhs)
{
return !(lhs == rhs);
}
inline bool operator< (const local_date& lhs, const local_date& rhs)
{
return std::make_tuple(lhs.year, lhs.month, lhs.day) <
std::make_tuple(rhs.year, rhs.month, rhs.day);
}
inline bool operator<=(const local_date& lhs, const local_date& rhs)
{
return (lhs < rhs) || (lhs == rhs);
}
inline bool operator> (const local_date& lhs, const local_date& rhs)
{
return !(lhs <= rhs);
}
inline bool operator>=(const local_date& lhs, const local_date& rhs)
{
return !(lhs < rhs);
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const local_date& date)
{
os << std::setfill('0') << std::setw(4) << static_cast<int>(date.year ) << '-';
os << std::setfill('0') << std::setw(2) << static_cast<int>(date.month) + 1 << '-';
os << std::setfill('0') << std::setw(2) << static_cast<int>(date.day ) ;
return os;
}
struct local_time
{
std::uint8_t hour; // [0, 23]
std::uint8_t minute; // [0, 59]
std::uint8_t second; // [0, 60]
std::uint16_t millisecond; // [0, 999]
std::uint16_t microsecond; // [0, 999]
std::uint16_t nanosecond; // [0, 999]
local_time(int h, int m, int s,
int ms = 0, int us = 0, int ns = 0)
: hour (static_cast<std::uint8_t>(h)),
minute(static_cast<std::uint8_t>(m)),
second(static_cast<std::uint8_t>(s)),
millisecond(static_cast<std::uint16_t>(ms)),
microsecond(static_cast<std::uint16_t>(us)),
nanosecond (static_cast<std::uint16_t>(ns))
{}
explicit local_time(const std::tm& t)
: hour (static_cast<std::uint8_t>(t.tm_hour)),
minute(static_cast<std::uint8_t>(t.tm_min)),
second(static_cast<std::uint8_t>(t.tm_sec)),
millisecond(0), microsecond(0), nanosecond(0)
{}
template<typename Rep, typename Period>
explicit local_time(const std::chrono::duration<Rep, Period>& t)
{
const auto h = std::chrono::duration_cast<std::chrono::hours>(t);
this->hour = static_cast<std::uint8_t>(h.count());
const auto t2 = t - h;
const auto m = std::chrono::duration_cast<std::chrono::minutes>(t2);
this->minute = static_cast<std::uint8_t>(m.count());
const auto t3 = t2 - m;
const auto s = std::chrono::duration_cast<std::chrono::seconds>(t3);
this->second = static_cast<std::uint8_t>(s.count());
const auto t4 = t3 - s;
const auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(t4);
this->millisecond = static_cast<std::uint16_t>(ms.count());
const auto t5 = t4 - ms;
const auto us = std::chrono::duration_cast<std::chrono::microseconds>(t5);
this->microsecond = static_cast<std::uint16_t>(us.count());
const auto t6 = t5 - us;
const auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(t6);
this->nanosecond = static_cast<std::uint16_t>(ns.count());
}
operator std::chrono::nanoseconds() const
{
return std::chrono::nanoseconds (this->nanosecond) +
std::chrono::microseconds(this->microsecond) +
std::chrono::milliseconds(this->millisecond) +
std::chrono::seconds(this->second) +
std::chrono::minutes(this->minute) +
std::chrono::hours(this->hour);
}
local_time() = default;
~local_time() = default;
local_time(local_time const&) = default;
local_time(local_time&&) = default;
local_time& operator=(local_time const&) = default;
local_time& operator=(local_time&&) = default;
};
inline bool operator==(const local_time& lhs, const local_time& rhs)
{
return std::make_tuple(lhs.hour, lhs.minute, lhs.second, lhs.millisecond, lhs.microsecond, lhs.nanosecond) ==
std::make_tuple(rhs.hour, rhs.minute, rhs.second, rhs.millisecond, rhs.microsecond, rhs.nanosecond);
}
inline bool operator!=(const local_time& lhs, const local_time& rhs)
{
return !(lhs == rhs);
}
inline bool operator< (const local_time& lhs, const local_time& rhs)
{
return std::make_tuple(lhs.hour, lhs.minute, lhs.second, lhs.millisecond, lhs.microsecond, lhs.nanosecond) <
std::make_tuple(rhs.hour, rhs.minute, rhs.second, rhs.millisecond, rhs.microsecond, rhs.nanosecond);
}
inline bool operator<=(const local_time& lhs, const local_time& rhs)
{
return (lhs < rhs) || (lhs == rhs);
}
inline bool operator> (const local_time& lhs, const local_time& rhs)
{
return !(lhs <= rhs);
}
inline bool operator>=(const local_time& lhs, const local_time& rhs)
{
return !(lhs < rhs);
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const local_time& time)
{
os << std::setfill('0') << std::setw(2) << static_cast<int>(time.hour ) << ':';
os << std::setfill('0') << std::setw(2) << static_cast<int>(time.minute) << ':';
os << std::setfill('0') << std::setw(2) << static_cast<int>(time.second);
if(time.millisecond != 0 || time.microsecond != 0 || time.nanosecond != 0)
{
os << '.';
os << std::setfill('0') << std::setw(3) << static_cast<int>(time.millisecond);
if(time.microsecond != 0 || time.nanosecond != 0)
{
os << std::setfill('0') << std::setw(3) << static_cast<int>(time.microsecond);
if(time.nanosecond != 0)
{
os << std::setfill('0') << std::setw(3) << static_cast<int>(time.nanosecond);
}
}
}
return os;
}
struct time_offset
{
std::int8_t hour; // [-12, 12]
std::int8_t minute; // [-59, 59]
time_offset(int h, int m)
: hour (static_cast<std::int8_t>(h)),
minute(static_cast<std::int8_t>(m))
{}
operator std::chrono::minutes() const
{
return std::chrono::minutes(this->minute) +
std::chrono::hours(this->hour);
}
time_offset() = default;
~time_offset() = default;
time_offset(time_offset const&) = default;
time_offset(time_offset&&) = default;
time_offset& operator=(time_offset const&) = default;
time_offset& operator=(time_offset&&) = default;
};
inline bool operator==(const time_offset& lhs, const time_offset& rhs)
{
return std::make_tuple(lhs.hour, lhs.minute) ==
std::make_tuple(rhs.hour, rhs.minute);
}
inline bool operator!=(const time_offset& lhs, const time_offset& rhs)
{
return !(lhs == rhs);
}
inline bool operator< (const time_offset& lhs, const time_offset& rhs)
{
return std::make_tuple(lhs.hour, lhs.minute) <
std::make_tuple(rhs.hour, rhs.minute);
}
inline bool operator<=(const time_offset& lhs, const time_offset& rhs)
{
return (lhs < rhs) || (lhs == rhs);
}
inline bool operator> (const time_offset& lhs, const time_offset& rhs)
{
return !(lhs <= rhs);
}
inline bool operator>=(const time_offset& lhs, const time_offset& rhs)
{
return !(lhs < rhs);
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const time_offset& offset)
{
if(offset.hour == 0 && offset.minute == 0)
{
os << 'Z';
return os;
}
int minute = static_cast<int>(offset.hour) * 60 + offset.minute;
if(minute < 0){os << '-'; minute = std::abs(minute);} else {os << '+';}
os << std::setfill('0') << std::setw(2) << minute / 60 << ':';
os << std::setfill('0') << std::setw(2) << minute % 60;
return os;
}
struct local_datetime
{
local_date date;
local_time time;
local_datetime(local_date d, local_time t): date(d), time(t) {}
explicit local_datetime(const std::tm& t): date(t), time(t){}
explicit local_datetime(const std::chrono::system_clock::time_point& tp)
{
const auto t = std::chrono::system_clock::to_time_t(tp);
std::tm ltime = detail::localtime_s(&t);
this->date = local_date(ltime);
this->time = local_time(ltime);
// std::tm lacks subsecond information, so diff between tp and tm
// can be used to get millisecond & microsecond information.
const auto t_diff = tp -
std::chrono::system_clock::from_time_t(std::mktime(&ltime));
this->time.millisecond = static_cast<std::uint16_t>(
std::chrono::duration_cast<std::chrono::milliseconds>(t_diff).count());
this->time.microsecond = static_cast<std::uint16_t>(
std::chrono::duration_cast<std::chrono::microseconds>(t_diff).count());
this->time.nanosecond = static_cast<std::uint16_t>(
std::chrono::duration_cast<std::chrono::nanoseconds >(t_diff).count());
}
explicit local_datetime(const std::time_t t)
: local_datetime(std::chrono::system_clock::from_time_t(t))
{}
operator std::chrono::system_clock::time_point() const
{
using internal_duration =
typename std::chrono::system_clock::time_point::duration;
// Normally DST begins at A.M. 3 or 4. If we re-use conversion operator
// of local_date and local_time independently, the conversion fails if
// it is the day when DST begins or ends. Since local_date considers the
// time is 00:00 A.M. and local_time does not consider DST because it
// does not have any date information. We need to consider both date and
// time information at the same time to convert it correctly.
std::tm t;
t.tm_sec = static_cast<int>(this->time.second);
t.tm_min = static_cast<int>(this->time.minute);
t.tm_hour = static_cast<int>(this->time.hour);
t.tm_mday = static_cast<int>(this->date.day);
t.tm_mon = static_cast<int>(this->date.month);
t.tm_year = static_cast<int>(this->date.year) - 1900;
t.tm_wday = 0; // the value will be ignored
t.tm_yday = 0; // the value will be ignored
t.tm_isdst = -1;
// std::mktime returns date as local time zone. no conversion needed
auto dt = std::chrono::system_clock::from_time_t(std::mktime(&t));
dt += std::chrono::duration_cast<internal_duration>(
std::chrono::milliseconds(this->time.millisecond) +
std::chrono::microseconds(this->time.microsecond) +
std::chrono::nanoseconds (this->time.nanosecond));
return dt;
}
operator std::time_t() const
{
return std::chrono::system_clock::to_time_t(
std::chrono::system_clock::time_point(*this));
}
local_datetime() = default;
~local_datetime() = default;
local_datetime(local_datetime const&) = default;
local_datetime(local_datetime&&) = default;
local_datetime& operator=(local_datetime const&) = default;
local_datetime& operator=(local_datetime&&) = default;
};
inline bool operator==(const local_datetime& lhs, const local_datetime& rhs)
{
return std::make_tuple(lhs.date, lhs.time) ==
std::make_tuple(rhs.date, rhs.time);
}
inline bool operator!=(const local_datetime& lhs, const local_datetime& rhs)
{
return !(lhs == rhs);
}
inline bool operator< (const local_datetime& lhs, const local_datetime& rhs)
{
return std::make_tuple(lhs.date, lhs.time) <
std::make_tuple(rhs.date, rhs.time);
}
inline bool operator<=(const local_datetime& lhs, const local_datetime& rhs)
{
return (lhs < rhs) || (lhs == rhs);
}
inline bool operator> (const local_datetime& lhs, const local_datetime& rhs)
{
return !(lhs <= rhs);
}
inline bool operator>=(const local_datetime& lhs, const local_datetime& rhs)
{
return !(lhs < rhs);
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const local_datetime& dt)
{
os << dt.date << 'T' << dt.time;
return os;
}
struct offset_datetime
{
local_date date;
local_time time;
time_offset offset;
offset_datetime(local_date d, local_time t, time_offset o)
: date(d), time(t), offset(o)
{}
offset_datetime(const local_datetime& dt, time_offset o)
: date(dt.date), time(dt.time), offset(o)
{}
explicit offset_datetime(const local_datetime& ld)
: date(ld.date), time(ld.time), offset(get_local_offset(nullptr))
// use the current local timezone offset
{}
explicit offset_datetime(const std::chrono::system_clock::time_point& tp)
: offset(0, 0) // use gmtime
{
const auto timet = std::chrono::system_clock::to_time_t(tp);
const auto tm = detail::gmtime_s(&timet);
this->date = local_date(tm);
this->time = local_time(tm);
}
explicit offset_datetime(const std::time_t& t)
: offset(0, 0) // use gmtime
{
const auto tm = detail::gmtime_s(&t);
this->date = local_date(tm);
this->time = local_time(tm);
}
explicit offset_datetime(const std::tm& t)
: offset(0, 0) // assume gmtime
{
this->date = local_date(t);
this->time = local_time(t);
}
operator std::chrono::system_clock::time_point() const
{
// get date-time
using internal_duration =
typename std::chrono::system_clock::time_point::duration;
// first, convert it to local date-time information in the same way as
// local_datetime does. later we will use time_t to adjust time offset.
std::tm t;
t.tm_sec = static_cast<int>(this->time.second);
t.tm_min = static_cast<int>(this->time.minute);
t.tm_hour = static_cast<int>(this->time.hour);
t.tm_mday = static_cast<int>(this->date.day);
t.tm_mon = static_cast<int>(this->date.month);
t.tm_year = static_cast<int>(this->date.year) - 1900;
t.tm_wday = 0; // the value will be ignored
t.tm_yday = 0; // the value will be ignored
t.tm_isdst = -1;
const std::time_t tp_loc = std::mktime(std::addressof(t));
auto tp = std::chrono::system_clock::from_time_t(tp_loc);
tp += std::chrono::duration_cast<internal_duration>(
std::chrono::milliseconds(this->time.millisecond) +
std::chrono::microseconds(this->time.microsecond) +
std::chrono::nanoseconds (this->time.nanosecond));
// Since mktime uses local time zone, it should be corrected.
// `12:00:00+09:00` means `03:00:00Z`. So mktime returns `03:00:00Z` if
// we are in `+09:00` timezone. To represent `12:00:00Z` there, we need
// to add `+09:00` to `03:00:00Z`.
// Here, it uses the time_t converted from date-time info to handle
// daylight saving time.
const auto ofs = get_local_offset(std::addressof(tp_loc));
tp += std::chrono::hours (ofs.hour);
tp += std::chrono::minutes(ofs.minute);
// We got `12:00:00Z` by correcting local timezone applied by mktime.
// Then we will apply the offset. Let's say `12:00:00-08:00` is given.
// And now, we have `12:00:00Z`. `12:00:00-08:00` means `20:00:00Z`.
// So we need to subtract the offset.
tp -= std::chrono::minutes(this->offset);
return tp;
}
operator std::time_t() const
{
return std::chrono::system_clock::to_time_t(
std::chrono::system_clock::time_point(*this));
}
offset_datetime() = default;
~offset_datetime() = default;
offset_datetime(offset_datetime const&) = default;
offset_datetime(offset_datetime&&) = default;
offset_datetime& operator=(offset_datetime const&) = default;
offset_datetime& operator=(offset_datetime&&) = default;
private:
static time_offset get_local_offset(const std::time_t* tp)
{
// get local timezone with the same date-time information as mktime
const auto t = detail::localtime_s(tp);
std::array<char, 6> buf;
const auto result = std::strftime(buf.data(), 6, "%z", &t); // +hhmm\0
if(result != 5)
{
throw std::runtime_error("toml::offset_datetime: cannot obtain "
"timezone information of current env");
}
const int ofs = std::atoi(buf.data());
const int ofs_h = ofs / 100;
const int ofs_m = ofs - (ofs_h * 100);
return time_offset(ofs_h, ofs_m);
}
};
inline bool operator==(const offset_datetime& lhs, const offset_datetime& rhs)
{
return std::make_tuple(lhs.date, lhs.time, lhs.offset) ==
std::make_tuple(rhs.date, rhs.time, rhs.offset);
}
inline bool operator!=(const offset_datetime& lhs, const offset_datetime& rhs)
{
return !(lhs == rhs);
}
inline bool operator< (const offset_datetime& lhs, const offset_datetime& rhs)
{
return std::make_tuple(lhs.date, lhs.time, lhs.offset) <
std::make_tuple(rhs.date, rhs.time, rhs.offset);
}
inline bool operator<=(const offset_datetime& lhs, const offset_datetime& rhs)
{
return (lhs < rhs) || (lhs == rhs);
}
inline bool operator> (const offset_datetime& lhs, const offset_datetime& rhs)
{
return !(lhs <= rhs);
}
inline bool operator>=(const offset_datetime& lhs, const offset_datetime& rhs)
{
return !(lhs < rhs);
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const offset_datetime& dt)
{
os << dt.date << 'T' << dt.time << dt.offset;
return os;
}
}//toml
#endif// TOML11_DATETIME

65
src/toml11/toml/exception.hpp Normal file
View file

@ -0,0 +1,65 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_EXCEPTION_HPP
#define TOML11_EXCEPTION_HPP
#include <stdexcept>
#include <string>
#include "source_location.hpp"
namespace toml
{
struct exception : public std::exception
{
public:
explicit exception(const source_location& loc): loc_(loc) {}
virtual ~exception() noexcept override = default;
virtual const char* what() const noexcept override {return "";}
virtual source_location const& location() const noexcept {return loc_;}
protected:
source_location loc_;
};
struct syntax_error : public toml::exception
{
public:
explicit syntax_error(const std::string& what_arg, const source_location& loc)
: exception(loc), what_(what_arg)
{}
virtual ~syntax_error() noexcept override = default;
virtual const char* what() const noexcept override {return what_.c_str();}
protected:
std::string what_;
};
struct type_error : public toml::exception
{
public:
explicit type_error(const std::string& what_arg, const source_location& loc)
: exception(loc), what_(what_arg)
{}
virtual ~type_error() noexcept override = default;
virtual const char* what() const noexcept override {return what_.c_str();}
protected:
std::string what_;
};
struct internal_error : public toml::exception
{
public:
explicit internal_error(const std::string& what_arg, const source_location& loc)
: exception(loc), what_(what_arg)
{}
virtual ~internal_error() noexcept override = default;
virtual const char* what() const noexcept override {return what_.c_str();}
protected:
std::string what_;
};
} // toml
#endif // TOML_EXCEPTION

19
src/toml11/toml/from.hpp Normal file
View file

@ -0,0 +1,19 @@
// Copyright Toru Niina 2019.
// Distributed under the MIT License.
#ifndef TOML11_FROM_HPP
#define TOML11_FROM_HPP
namespace toml
{
template<typename T>
struct from;
// {
// static T from_toml(const toml::value& v)
// {
// // User-defined conversions ...
// }
// };
} // toml
#endif // TOML11_FROM_HPP

1117
src/toml11/toml/get.hpp Normal file
View file

File diff suppressed because it is too large Load diff

19
src/toml11/toml/into.hpp Normal file
View file

@ -0,0 +1,19 @@
// Copyright Toru Niina 2019.
// Distributed under the MIT License.
#ifndef TOML11_INTO_HPP
#define TOML11_INTO_HPP
namespace toml
{
template<typename T>
struct into;
// {
// static toml::value into_toml(const T& user_defined_type)
// {
// // User-defined conversions ...
// }
// };
} // toml
#endif // TOML11_INTO_HPP

293
src/toml11/toml/lexer.hpp Normal file
View file

@ -0,0 +1,293 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_LEXER_HPP
#define TOML11_LEXER_HPP
#include <istream>
#include <sstream>
#include <stdexcept>
#include <fstream>
#include "combinator.hpp"
namespace toml
{
namespace detail
{
// these scans contents from current location in a container of char
// and extract a region that matches their own pattern.
// to see the implementation of each component, see combinator.hpp.
using lex_wschar = either<character<' '>, character<'\t'>>;
using lex_ws = repeat<lex_wschar, at_least<1>>;
using lex_newline = either<character<'\n'>,
sequence<character<'\r'>, character<'\n'>>>;
using lex_lower = in_range<'a', 'z'>;
using lex_upper = in_range<'A', 'Z'>;
using lex_alpha = either<lex_lower, lex_upper>;
using lex_digit = in_range<'0', '9'>;
using lex_nonzero = in_range<'1', '9'>;
using lex_oct_dig = in_range<'0', '7'>;
using lex_bin_dig = in_range<'0', '1'>;
using lex_hex_dig = either<lex_digit, in_range<'A', 'F'>, in_range<'a', 'f'>>;
using lex_hex_prefix = sequence<character<'0'>, character<'x'>>;
using lex_oct_prefix = sequence<character<'0'>, character<'o'>>;
using lex_bin_prefix = sequence<character<'0'>, character<'b'>>;
using lex_underscore = character<'_'>;
using lex_plus = character<'+'>;
using lex_minus = character<'-'>;
using lex_sign = either<lex_plus, lex_minus>;
// digit | nonzero 1*(digit | _ digit)
using lex_unsigned_dec_int = either<sequence<lex_nonzero, repeat<
either<lex_digit, sequence<lex_underscore, lex_digit>>, at_least<1>>>,
lex_digit>;
// (+|-)? unsigned_dec_int
using lex_dec_int = sequence<maybe<lex_sign>, lex_unsigned_dec_int>;
// hex_prefix hex_dig *(hex_dig | _ hex_dig)
using lex_hex_int = sequence<lex_hex_prefix, sequence<lex_hex_dig, repeat<
either<lex_hex_dig, sequence<lex_underscore, lex_hex_dig>>, unlimited>>>;
// oct_prefix oct_dig *(oct_dig | _ oct_dig)
using lex_oct_int = sequence<lex_oct_prefix, sequence<lex_oct_dig, repeat<
either<lex_oct_dig, sequence<lex_underscore, lex_oct_dig>>, unlimited>>>;
// bin_prefix bin_dig *(bin_dig | _ bin_dig)
using lex_bin_int = sequence<lex_bin_prefix, sequence<lex_bin_dig, repeat<
either<lex_bin_dig, sequence<lex_underscore, lex_bin_dig>>, unlimited>>>;
// (dec_int | hex_int | oct_int | bin_int)
using lex_integer = either<lex_bin_int, lex_oct_int, lex_hex_int, lex_dec_int>;
// ===========================================================================
using lex_inf = sequence<character<'i'>, character<'n'>, character<'f'>>;
using lex_nan = sequence<character<'n'>, character<'a'>, character<'n'>>;
using lex_special_float = sequence<maybe<lex_sign>, either<lex_inf, lex_nan>>;
using lex_zero_prefixable_int = sequence<lex_digit, repeat<either<lex_digit,
sequence<lex_underscore, lex_digit>>, unlimited>>;
using lex_fractional_part = sequence<character<'.'>, lex_zero_prefixable_int>;
using lex_exponent_part = sequence<either<character<'e'>, character<'E'>>,
maybe<lex_sign>, lex_zero_prefixable_int>;
using lex_float = either<lex_special_float,
sequence<lex_dec_int, either<lex_exponent_part,
sequence<lex_fractional_part, maybe<lex_exponent_part>>>>>;
// ===========================================================================
using lex_true = sequence<character<'t'>, character<'r'>,
character<'u'>, character<'e'>>;
using lex_false = sequence<character<'f'>, character<'a'>, character<'l'>,
character<'s'>, character<'e'>>;
using lex_boolean = either<lex_true, lex_false>;
// ===========================================================================
using lex_date_fullyear = repeat<lex_digit, exactly<4>>;
using lex_date_month = repeat<lex_digit, exactly<2>>;
using lex_date_mday = repeat<lex_digit, exactly<2>>;
using lex_time_delim = either<character<'T'>, character<'t'>, character<' '>>;
using lex_time_hour = repeat<lex_digit, exactly<2>>;
using lex_time_minute = repeat<lex_digit, exactly<2>>;
using lex_time_second = repeat<lex_digit, exactly<2>>;
using lex_time_secfrac = sequence<character<'.'>,
repeat<lex_digit, at_least<1>>>;
using lex_time_numoffset = sequence<either<character<'+'>, character<'-'>>,
sequence<lex_time_hour, character<':'>,
lex_time_minute>>;
using lex_time_offset = either<character<'Z'>, character<'z'>,
lex_time_numoffset>;
using lex_partial_time = sequence<lex_time_hour, character<':'>,
lex_time_minute, character<':'>,
lex_time_second, maybe<lex_time_secfrac>>;
using lex_full_date = sequence<lex_date_fullyear, character<'-'>,
lex_date_month, character<'-'>,
lex_date_mday>;
using lex_full_time = sequence<lex_partial_time, lex_time_offset>;
using lex_offset_date_time = sequence<lex_full_date, lex_time_delim, lex_full_time>;
using lex_local_date_time = sequence<lex_full_date, lex_time_delim, lex_partial_time>;
using lex_local_date = lex_full_date;
using lex_local_time = lex_partial_time;
// ===========================================================================
using lex_quotation_mark = character<'"'>;
using lex_basic_unescaped = exclude<either<in_range<0x00, 0x08>, // 0x09 (tab) is allowed
in_range<0x0A, 0x1F>,
character<0x22>, character<0x5C>,
character<0x7F>>>;
using lex_escape = character<'\\'>;
using lex_escape_unicode_short = sequence<character<'u'>,
repeat<lex_hex_dig, exactly<4>>>;
using lex_escape_unicode_long = sequence<character<'U'>,
repeat<lex_hex_dig, exactly<8>>>;
using lex_escape_seq_char = either<character<'"'>, character<'\\'>,
character<'b'>, character<'f'>,
character<'n'>, character<'r'>,
character<'t'>,
lex_escape_unicode_short,
lex_escape_unicode_long
>;
using lex_escaped = sequence<lex_escape, lex_escape_seq_char>;
using lex_basic_char = either<lex_basic_unescaped, lex_escaped>;
using lex_basic_string = sequence<lex_quotation_mark,
repeat<lex_basic_char, unlimited>,
lex_quotation_mark>;
// After toml post-v0.5.0, it is explicitly clarified how quotes in ml-strings
// are allowed to be used.
// After this, the following strings are *explicitly* allowed.
// - One or two `"`s in a multi-line basic string is allowed wherever it is.
// - Three consecutive `"`s in a multi-line basic string is considered as a delimiter.
// - One or two `"`s can appear just before or after the delimiter.
// ```toml
// str4 = """Here are two quotation marks: "". Simple enough."""
// str5 = """Here are three quotation marks: ""\"."""
// str6 = """Here are fifteen quotation marks: ""\"""\"""\"""\"""\"."""
// str7 = """"This," she said, "is just a pointless statement.""""
// ```
// In the current implementation (v3.3.0), it is difficult to parse `str7` in
// the above example. It is difficult to recognize `"` at the end of string body
// collectly. It will be misunderstood as a `"""` delimiter and an additional,
// invalid `"`. Like this:
// ```console
// what(): [error] toml::parse_table: invalid line format
// --> hoge.toml
// |
// 13 | str7 = """"This," she said, "is just a pointless statement.""""
// | ^- expected newline, but got '"'.
// ```
// As a quick workaround for this problem, `lex_ml_basic_string_delim` was
// split into two, `lex_ml_basic_string_open` and `lex_ml_basic_string_close`.
// `lex_ml_basic_string_open` allows only `"""`. `_close` allows 3-5 `"`s.
// In parse_ml_basic_string() function, the trailing `"`s will be attached to
// the string body.
//
using lex_ml_basic_string_delim = repeat<lex_quotation_mark, exactly<3>>;
using lex_ml_basic_string_open = lex_ml_basic_string_delim;
using lex_ml_basic_string_close = sequence<
repeat<lex_quotation_mark, exactly<3>>,
maybe<lex_quotation_mark>, maybe<lex_quotation_mark>
>;
using lex_ml_basic_unescaped = exclude<either<in_range<0x00, 0x08>, // 0x09 is tab
in_range<0x0A, 0x1F>,
character<0x5C>, // backslash
character<0x7F>, // DEL
lex_ml_basic_string_delim>>;
using lex_ml_basic_escaped_newline = sequence<
lex_escape, maybe<lex_ws>, lex_newline,
repeat<either<lex_ws, lex_newline>, unlimited>>;
using lex_ml_basic_char = either<lex_ml_basic_unescaped, lex_escaped>;
using lex_ml_basic_body = repeat<either<lex_ml_basic_char, lex_newline,
lex_ml_basic_escaped_newline>,
unlimited>;
using lex_ml_basic_string = sequence<lex_ml_basic_string_open,
lex_ml_basic_body,
lex_ml_basic_string_close>;
using lex_literal_char = exclude<either<in_range<0x00, 0x08>, in_range<0x0A, 0x1F>,
character<0x7F>, character<0x27>>>;
using lex_apostrophe = character<'\''>;
using lex_literal_string = sequence<lex_apostrophe,
repeat<lex_literal_char, unlimited>,
lex_apostrophe>;
// the same reason as above.
using lex_ml_literal_string_delim = repeat<lex_apostrophe, exactly<3>>;
using lex_ml_literal_string_open = lex_ml_literal_string_delim;
using lex_ml_literal_string_close = sequence<
repeat<lex_apostrophe, exactly<3>>,
maybe<lex_apostrophe>, maybe<lex_apostrophe>
>;
using lex_ml_literal_char = exclude<either<in_range<0x00, 0x08>,
in_range<0x0A, 0x1F>,
character<0x7F>,
lex_ml_literal_string_delim>>;
using lex_ml_literal_body = repeat<either<lex_ml_literal_char, lex_newline>,
unlimited>;
using lex_ml_literal_string = sequence<lex_ml_literal_string_open,
lex_ml_literal_body,
lex_ml_literal_string_close>;
using lex_string = either<lex_ml_basic_string, lex_basic_string,
lex_ml_literal_string, lex_literal_string>;
// ===========================================================================
using lex_dot_sep = sequence<maybe<lex_ws>, character<'.'>, maybe<lex_ws>>;
using lex_unquoted_key = repeat<either<lex_alpha, lex_digit,
character<'-'>, character<'_'>>,
at_least<1>>;
using lex_quoted_key = either<lex_basic_string, lex_literal_string>;
using lex_simple_key = either<lex_unquoted_key, lex_quoted_key>;
using lex_dotted_key = sequence<lex_simple_key,
repeat<sequence<lex_dot_sep, lex_simple_key>,
at_least<1>
>
>;
using lex_key = either<lex_dotted_key, lex_simple_key>;
using lex_keyval_sep = sequence<maybe<lex_ws>,
character<'='>,
maybe<lex_ws>>;
using lex_std_table_open = character<'['>;
using lex_std_table_close = character<']'>;
using lex_std_table = sequence<lex_std_table_open,
maybe<lex_ws>,
lex_key,
maybe<lex_ws>,
lex_std_table_close>;
using lex_array_table_open = sequence<lex_std_table_open, lex_std_table_open>;
using lex_array_table_close = sequence<lex_std_table_close, lex_std_table_close>;
using lex_array_table = sequence<lex_array_table_open,
maybe<lex_ws>,
lex_key,
maybe<lex_ws>,
lex_array_table_close>;
using lex_utf8_1byte = in_range<0x00, 0x7F>;
using lex_utf8_2byte = sequence<
in_range<static_cast<char>(0xC2), static_cast<char>(0xDF)>,
in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>
>;
using lex_utf8_3byte = sequence<either<
sequence<character<static_cast<char>(0xE0)>, in_range<static_cast<char>(0xA0), static_cast<char>(0xBF)>>,
sequence<in_range <static_cast<char>(0xE1), static_cast<char>(0xEC)>, in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>>,
sequence<character<static_cast<char>(0xED)>, in_range<static_cast<char>(0x80), static_cast<char>(0x9F)>>,
sequence<in_range <static_cast<char>(0xEE), static_cast<char>(0xEF)>, in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>>
>, in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>>;
using lex_utf8_4byte = sequence<either<
sequence<character<static_cast<char>(0xF0)>, in_range<static_cast<char>(0x90), static_cast<char>(0xBF)>>,
sequence<in_range <static_cast<char>(0xF1), static_cast<char>(0xF3)>, in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>>,
sequence<character<static_cast<char>(0xF4)>, in_range<static_cast<char>(0x80), static_cast<char>(0x8F)>>
>, in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>,
in_range<static_cast<char>(0x80), static_cast<char>(0xBF)>>;
using lex_utf8_code = either<
lex_utf8_1byte,
lex_utf8_2byte,
lex_utf8_3byte,
lex_utf8_4byte
>;
using lex_comment_start_symbol = character<'#'>;
using lex_non_eol_ascii = either<character<0x09>, in_range<0x20, 0x7E>>;
using lex_comment = sequence<lex_comment_start_symbol, repeat<either<
lex_non_eol_ascii, lex_utf8_2byte, lex_utf8_3byte, lex_utf8_4byte>, unlimited>>;
} // detail
} // toml
#endif // TOML_LEXER_HPP

113
src/toml11/toml/literal.hpp Normal file
View file

@ -0,0 +1,113 @@
// Copyright Toru Niina 2019.
// Distributed under the MIT License.
#ifndef TOML11_LITERAL_HPP
#define TOML11_LITERAL_HPP
#include "parser.hpp"
namespace toml
{
inline namespace literals
{
inline namespace toml_literals
{
// implementation
inline ::toml::basic_value<TOML11_DEFAULT_COMMENT_STRATEGY, std::unordered_map, std::vector>
literal_internal_impl(::toml::detail::location loc)
{
using value_type = ::toml::basic_value<
TOML11_DEFAULT_COMMENT_STRATEGY, std::unordered_map, std::vector>;
// if there are some comments or empty lines, skip them.
using skip_line = ::toml::detail::repeat<toml::detail::sequence<
::toml::detail::maybe<::toml::detail::lex_ws>,
::toml::detail::maybe<::toml::detail::lex_comment>,
::toml::detail::lex_newline
>, ::toml::detail::at_least<1>>;
skip_line::invoke(loc);
// if there are some whitespaces before a value, skip them.
using skip_ws = ::toml::detail::repeat<
::toml::detail::lex_ws, ::toml::detail::at_least<1>>;
skip_ws::invoke(loc);
// to distinguish arrays and tables, first check it is a table or not.
//
// "[1,2,3]"_toml; // this is an array
// "[table]"_toml; // a table that has an empty table named "table" inside.
// "[[1,2,3]]"_toml; // this is an array of arrays
// "[[table]]"_toml; // this is a table that has an array of tables inside.
//
// "[[1]]"_toml; // this can be both... (currently it becomes a table)
// "1 = [{}]"_toml; // this is a table that has an array of table named 1.
// "[[1,]]"_toml; // this is an array of arrays.
// "[[1],]"_toml; // this also.
const auto the_front = loc.iter();
const bool is_table_key = ::toml::detail::lex_std_table::invoke(loc);
loc.reset(the_front);
const bool is_aots_key = ::toml::detail::lex_array_table::invoke(loc);
loc.reset(the_front);
// If it is neither a table-key or a array-of-table-key, it may be a value.
if(!is_table_key && !is_aots_key)
{
if(auto data = ::toml::detail::parse_value<value_type>(loc))
{
return data.unwrap();
}
}
// Note that still it can be a table, because the literal might be something
// like the following.
// ```cpp
// R"( // c++11 raw string literals
// key = "value"
// int = 42
// )"_toml;
// ```
// It is a valid toml file.
// It should be parsed as if we parse a file with this content.
if(auto data = ::toml::detail::parse_toml_file<value_type>(loc))
{
return data.unwrap();
}
else // none of them.
{
throw ::toml::syntax_error(data.unwrap_err(), source_location(loc));
}
}
inline ::toml::basic_value<TOML11_DEFAULT_COMMENT_STRATEGY, std::unordered_map, std::vector>
operator"" _toml(const char* str, std::size_t len)
{
::toml::detail::location loc(
std::string("TOML literal encoded in a C++ code"),
std::vector<char>(str, str + len));
// literal length does not include the null character at the end.
return literal_internal_impl(std::move(loc));
}
// value of __cplusplus in C++2a/20 mode is not fixed yet along compilers.
// So here we use the feature test macro for `char8_t` itself.
#if defined(__cpp_char8_t) && __cpp_char8_t >= 201811L
// value of u8"" literal has been changed from char to char8_t and char8_t is
// NOT compatible to char
inline ::toml::basic_value<TOML11_DEFAULT_COMMENT_STRATEGY, std::unordered_map, std::vector>
operator"" _toml(const char8_t* str, std::size_t len)
{
::toml::detail::location loc(
std::string("TOML literal encoded in a C++ code"),
std::vector<char>(reinterpret_cast<const char*>(str),
reinterpret_cast<const char*>(str) + len));
return literal_internal_impl(std::move(loc));
}
#endif
} // toml_literals
} // literals
} // toml
#endif//TOML11_LITERAL_HPP

121
src/toml11/toml/macros.hpp Normal file
View file

@ -0,0 +1,121 @@
#ifndef TOML11_MACROS_HPP
#define TOML11_MACROS_HPP
#define TOML11_STRINGIZE_AUX(x) #x
#define TOML11_STRINGIZE(x) TOML11_STRINGIZE_AUX(x)
#define TOML11_CONCATENATE_AUX(x, y) x##y
#define TOML11_CONCATENATE(x, y) TOML11_CONCATENATE_AUX(x, y)
// ============================================================================
// TOML11_DEFINE_CONVERSION_NON_INTRUSIVE
#ifndef TOML11_WITHOUT_DEFINE_NON_INTRUSIVE
// ----------------------------------------------------------------------------
// TOML11_ARGS_SIZE
#define TOML11_INDEX_RSEQ() \
32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, \
16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0
#define TOML11_ARGS_SIZE_IMPL(\
ARG1, ARG2, ARG3, ARG4, ARG5, ARG6, ARG7, ARG8, ARG9, ARG10, \
ARG11, ARG12, ARG13, ARG14, ARG15, ARG16, ARG17, ARG18, ARG19, ARG20, \
ARG21, ARG22, ARG23, ARG24, ARG25, ARG26, ARG27, ARG28, ARG29, ARG30, \
ARG31, ARG32, N, ...) N
#define TOML11_ARGS_SIZE_AUX(...) TOML11_ARGS_SIZE_IMPL(__VA_ARGS__)
#define TOML11_ARGS_SIZE(...) TOML11_ARGS_SIZE_AUX(__VA_ARGS__, TOML11_INDEX_RSEQ())
// ----------------------------------------------------------------------------
// TOML11_FOR_EACH_VA_ARGS
#define TOML11_FOR_EACH_VA_ARGS_AUX_1( FUNCTOR, ARG1 ) FUNCTOR(ARG1)
#define TOML11_FOR_EACH_VA_ARGS_AUX_2( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_1( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_3( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_2( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_4( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_3( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_5( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_4( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_6( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_5( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_7( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_6( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_8( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_7( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_9( FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_8( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_10(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_9( FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_11(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_10(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_12(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_11(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_13(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_12(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_14(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_13(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_15(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_14(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_16(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_15(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_17(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_16(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_18(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_17(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_19(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_18(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_20(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_19(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_21(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_20(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_22(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_21(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_23(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_22(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_24(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_23(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_25(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_24(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_26(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_25(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_27(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_26(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_28(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_27(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_29(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_28(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_30(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_29(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_31(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_30(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS_AUX_32(FUNCTOR, ARG1, ...) FUNCTOR(ARG1) TOML11_FOR_EACH_VA_ARGS_AUX_31(FUNCTOR, __VA_ARGS__)
#define TOML11_FOR_EACH_VA_ARGS(FUNCTOR, ...)\
TOML11_CONCATENATE(TOML11_FOR_EACH_VA_ARGS_AUX_, TOML11_ARGS_SIZE(__VA_ARGS__))(FUNCTOR, __VA_ARGS__)
// ----------------------------------------------------------------------------
// TOML11_DEFINE_CONVERSION_NON_INTRUSIVE
// use it in the following way.
// ```cpp
// namespace foo
// {
// struct Foo
// {
// std::string s;
// double d;
// int i;
// };
// } // foo
//
// TOML11_DEFINE_CONVERSION_NON_INTRUSIVE(foo::Foo, s, d, i)
// ```
// And then you can use `toml::find<foo::Foo>(file, "foo");`
//
#define TOML11_FIND_MEMBER_VARIABLE_FROM_VALUE(VAR_NAME)\
obj.VAR_NAME = toml::find<decltype(obj.VAR_NAME)>(v, TOML11_STRINGIZE(VAR_NAME));
#define TOML11_ASSIGN_MEMBER_VARIABLE_TO_VALUE(VAR_NAME)\
v[TOML11_STRINGIZE(VAR_NAME)] = obj.VAR_NAME;
#define TOML11_DEFINE_CONVERSION_NON_INTRUSIVE(NAME, ...)\
namespace toml { \
template<> \
struct from<NAME> \
{ \
template<typename C, template<typename ...> class T, \
template<typename ...> class A> \
static NAME from_toml(const basic_value<C, T, A>& v) \
{ \
NAME obj; \
TOML11_FOR_EACH_VA_ARGS(TOML11_FIND_MEMBER_VARIABLE_FROM_VALUE, __VA_ARGS__) \
return obj; \
} \
}; \
template<> \
struct into<NAME> \
{ \
static value into_toml(const NAME& obj) \
{ \
::toml::value v = ::toml::table{}; \
TOML11_FOR_EACH_VA_ARGS(TOML11_ASSIGN_MEMBER_VARIABLE_TO_VALUE, __VA_ARGS__) \
return v; \
} \
}; \
} /* toml */
#endif// TOML11_WITHOUT_DEFINE_NON_INTRUSIVE
#endif// TOML11_MACROS_HPP

2364
src/toml11/toml/parser.hpp Normal file
View file

File diff suppressed because it is too large Load diff

417
src/toml11/toml/region.hpp Normal file
View file

@ -0,0 +1,417 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_REGION_HPP
#define TOML11_REGION_HPP
#include <memory>
#include <vector>
#include <algorithm>
#include <initializer_list>
#include <iterator>
#include <iomanip>
#include <cassert>
#include "color.hpp"
namespace toml
{
namespace detail
{
// helper function to avoid std::string(0, 'c') or std::string(iter, iter)
template<typename Iterator>
std::string make_string(Iterator first, Iterator last)
{
if(first == last) {return "";}
return std::string(first, last);
}
inline std::string make_string(std::size_t len, char c)
{
if(len == 0) {return "";}
return std::string(len, c);
}
// region_base is a base class of location and region that are defined below.
// it will be used to generate better error messages.
struct region_base
{
region_base() = default;
virtual ~region_base() = default;
region_base(const region_base&) = default;
region_base(region_base&& ) = default;
region_base& operator=(const region_base&) = default;
region_base& operator=(region_base&& ) = default;
virtual bool is_ok() const noexcept {return false;}
virtual char front() const noexcept {return '\0';}
virtual std::string str() const {return std::string("unknown region");}
virtual std::string name() const {return std::string("unknown file");}
virtual std::string line() const {return std::string("unknown line");}
virtual std::string line_num() const {return std::string("?");}
// length of the region
virtual std::size_t size() const noexcept {return 0;}
// number of characters in the line before the region
virtual std::size_t before() const noexcept {return 0;}
// number of characters in the line after the region
virtual std::size_t after() const noexcept {return 0;}
virtual std::vector<std::string> comments() const {return {};}
// ```toml
// # comment_before
// key = "value" # comment_inline
// ```
};
// location represents a position in a container, which contains a file content.
// it can be considered as a region that contains only one character.
//
// it contains pointer to the file content and iterator that points the current
// location.
struct location final : public region_base
{
using const_iterator = typename std::vector<char>::const_iterator;
using difference_type = typename const_iterator::difference_type;
using source_ptr = std::shared_ptr<const std::vector<char>>;
location(std::string source_name, std::vector<char> cont)
: source_(std::make_shared<std::vector<char>>(std::move(cont))),
line_number_(1), source_name_(std::move(source_name)), iter_(source_->cbegin())
{}
location(std::string source_name, const std::string& cont)
: source_(std::make_shared<std::vector<char>>(cont.begin(), cont.end())),
line_number_(1), source_name_(std::move(source_name)), iter_(source_->cbegin())
{}
location(const location&) = default;
location(location&&) = default;
location& operator=(const location&) = default;
location& operator=(location&&) = default;
~location() = default;
bool is_ok() const noexcept override {return static_cast<bool>(source_);}
char front() const noexcept override {return *iter_;}
// this const prohibits codes like `++(loc.iter())`.
const const_iterator iter() const noexcept {return iter_;}
const_iterator begin() const noexcept {return source_->cbegin();}
const_iterator end() const noexcept {return source_->cend();}
// XXX `location::line_num()` used to be implemented using `std::count` to
// count a number of '\n'. But with a long toml file (typically, 10k lines),
// it becomes intolerably slow because each time it generates error messages,
// it counts '\n' from thousands of characters. To workaround it, I decided
// to introduce `location::line_number_` member variable and synchronize it
// to the location changes the point to look. So an overload of `iter()`
// which returns mutable reference is removed and `advance()`, `retrace()`
// and `reset()` is added.
void advance(difference_type n = 1) noexcept
{
this->line_number_ += static_cast<std::size_t>(
std::count(this->iter_, std::next(this->iter_, n), '\n'));
this->iter_ += n;
return;
}
void retrace(difference_type n = 1) noexcept
{
this->line_number_ -= static_cast<std::size_t>(
std::count(std::prev(this->iter_, n), this->iter_, '\n'));
this->iter_ -= n;
return;
}
void reset(const_iterator rollback) noexcept
{
// since c++11, std::distance works in both ways for random-access
// iterators and returns a negative value if `first > last`.
if(0 <= std::distance(rollback, this->iter_)) // rollback < iter
{
this->line_number_ -= static_cast<std::size_t>(
std::count(rollback, this->iter_, '\n'));
}
else // iter < rollback [[unlikely]]
{
this->line_number_ += static_cast<std::size_t>(
std::count(this->iter_, rollback, '\n'));
}
this->iter_ = rollback;
return;
}
std::string str() const override {return make_string(1, *this->iter());}
std::string name() const override {return source_name_;}
std::string line_num() const override
{
return std::to_string(this->line_number_);
}
std::string line() const override
{
return make_string(this->line_begin(), this->line_end());
}
const_iterator line_begin() const noexcept
{
using reverse_iterator = std::reverse_iterator<const_iterator>;
return std::find(reverse_iterator(this->iter()),
reverse_iterator(this->begin()), '\n').base();
}
const_iterator line_end() const noexcept
{
return std::find(this->iter(), this->end(), '\n');
}
// location is always points a character. so the size is 1.
std::size_t size() const noexcept override
{
return 1u;
}
std::size_t before() const noexcept override
{
const auto sz = std::distance(this->line_begin(), this->iter());
assert(sz >= 0);
return static_cast<std::size_t>(sz);
}
std::size_t after() const noexcept override
{
const auto sz = std::distance(this->iter(), this->line_end());
assert(sz >= 0);
return static_cast<std::size_t>(sz);
}
source_ptr const& source() const& noexcept {return source_;}
source_ptr&& source() && noexcept {return std::move(source_);}
private:
source_ptr source_;
std::size_t line_number_;
std::string source_name_;
const_iterator iter_;
};
// region represents a range in a container, which contains a file content.
//
// it contains pointer to the file content and iterator that points the first
// and last location.
struct region final : public region_base
{
using const_iterator = typename std::vector<char>::const_iterator;
using source_ptr = std::shared_ptr<const std::vector<char>>;
// delete default constructor. source_ never be null.
region() = delete;
explicit region(const location& loc)
: source_(loc.source()), source_name_(loc.name()),
first_(loc.iter()), last_(loc.iter())
{}
explicit region(location&& loc)
: source_(loc.source()), source_name_(loc.name()),
first_(loc.iter()), last_(loc.iter())
{}
region(const location& loc, const_iterator f, const_iterator l)
: source_(loc.source()), source_name_(loc.name()), first_(f), last_(l)
{}
region(location&& loc, const_iterator f, const_iterator l)
: source_(loc.source()), source_name_(loc.name()), first_(f), last_(l)
{}
region(const region&) = default;
region(region&&) = default;
region& operator=(const region&) = default;
region& operator=(region&&) = default;
~region() = default;
region& operator+=(const region& other)
{
// different regions cannot be concatenated
assert(this->begin() == other.begin() && this->end() == other.end() &&
this->last_ == other.first_);
this->last_ = other.last_;
return *this;
}
bool is_ok() const noexcept override {return static_cast<bool>(source_);}
char front() const noexcept override {return *first_;}
std::string str() const override {return make_string(first_, last_);}
std::string line() const override
{
if(this->contain_newline())
{
return make_string(this->line_begin(),
std::find(this->line_begin(), this->last(), '\n'));
}
return make_string(this->line_begin(), this->line_end());
}
std::string line_num() const override
{
return std::to_string(1 + std::count(this->begin(), this->first(), '\n'));
}
std::size_t size() const noexcept override
{
const auto sz = std::distance(first_, last_);
assert(sz >= 0);
return static_cast<std::size_t>(sz);
}
std::size_t before() const noexcept override
{
const auto sz = std::distance(this->line_begin(), this->first());
assert(sz >= 0);
return static_cast<std::size_t>(sz);
}
std::size_t after() const noexcept override
{
const auto sz = std::distance(this->last(), this->line_end());
assert(sz >= 0);
return static_cast<std::size_t>(sz);
}
bool contain_newline() const noexcept
{
return std::find(this->first(), this->last(), '\n') != this->last();
}
const_iterator line_begin() const noexcept
{
using reverse_iterator = std::reverse_iterator<const_iterator>;
return std::find(reverse_iterator(this->first()),
reverse_iterator(this->begin()), '\n').base();
}
const_iterator line_end() const noexcept
{
return std::find(this->last(), this->end(), '\n');
}
const_iterator begin() const noexcept {return source_->cbegin();}
const_iterator end() const noexcept {return source_->cend();}
const_iterator first() const noexcept {return first_;}
const_iterator last() const noexcept {return last_;}
source_ptr const& source() const& noexcept {return source_;}
source_ptr&& source() && noexcept {return std::move(source_);}
std::string name() const override {return source_name_;}
std::vector<std::string> comments() const override
{
// assuming the current region (`*this`) points a value.
// ```toml
// a = "value"
// ^^^^^^^- this region
// ```
using rev_iter = std::reverse_iterator<const_iterator>;
std::vector<std::string> com{};
{
// find comments just before the current region.
// ```toml
// # this should be collected.
// # this also.
// a = value # not this.
// ```
// # this is a comment for `a`, not array elements.
// a = [1, 2, 3, 4, 5]
if(this->first() == std::find_if(this->line_begin(), this->first(),
[](const char c) noexcept -> bool {return c == '[' || c == '{';}))
{
auto iter = this->line_begin(); // points the first character
while(iter != this->begin())
{
iter = std::prev(iter);
// range [line_start, iter) represents the previous line
const auto line_start = std::find(
rev_iter(iter), rev_iter(this->begin()), '\n').base();
const auto comment_found = std::find(line_start, iter, '#');
if(comment_found == iter)
{
break; // comment not found.
}
// exclude the following case.
// > a = "foo" # comment // <-- this is not a comment for b but a.
// > b = "current value"
if(std::all_of(line_start, comment_found,
[](const char c) noexcept -> bool {
return c == ' ' || c == '\t';
}))
{
// unwrap the first '#' by std::next.
auto s = make_string(std::next(comment_found), iter);
if(!s.empty() && s.back() == '\r') {s.pop_back();}
com.push_back(std::move(s));
}
else
{
break;
}
iter = line_start;
}
}
}
if(com.size() > 1)
{
std::reverse(com.begin(), com.end());
}
{
// find comments just after the current region.
// ```toml
// # not this.
// a = value # this one.
// a = [ # not this (technically difficult)
//
// ] # and this.
// ```
// The reason why it's difficult is that it requires parsing in the
// following case.
// ```toml
// a = [ 10 # this comment is for `10`. not for `a` but `a[0]`.
// # ...
// ] # this is apparently a comment for a.
//
// b = [
// 3.14 ] # there is no way to add a comment to `3.14` currently.
//
// c = [
// 3.14 # do this if you need a comment here.
// ]
// ```
const auto comment_found =
std::find(this->last(), this->line_end(), '#');
if(comment_found != this->line_end()) // '#' found
{
// table = {key = "value"} # what is this for?
// the above comment is not for "value", but {key="value"}.
if(comment_found == std::find_if(this->last(), comment_found,
[](const char c) noexcept -> bool {
return !(c == ' ' || c == '\t' || c == ',');
}))
{
// unwrap the first '#' by std::next.
auto s = make_string(std::next(comment_found), this->line_end());
if(!s.empty() && s.back() == '\r') {s.pop_back();}
com.push_back(std::move(s));
}
}
}
return com;
}
private:
source_ptr source_;
std::string source_name_;
const_iterator first_, last_;
};
} // detail
} // toml
#endif// TOML11_REGION_H

717
src/toml11/toml/result.hpp Normal file
View file

@ -0,0 +1,717 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_RESULT_HPP
#define TOML11_RESULT_HPP
#include "traits.hpp"
#include <type_traits>
#include <stdexcept>
#include <utility>
#include <new>
#include <string>
#include <sstream>
#include <cassert>
namespace toml
{
template<typename T>
struct success
{
using value_type = T;
value_type value;
explicit success(const value_type& v)
noexcept(std::is_nothrow_copy_constructible<value_type>::value)
: value(v)
{}
explicit success(value_type&& v)
noexcept(std::is_nothrow_move_constructible<value_type>::value)
: value(std::move(v))
{}
template<typename U>
explicit success(U&& v): value(std::forward<U>(v)) {}
template<typename U>
explicit success(const success<U>& v): value(v.value) {}
template<typename U>
explicit success(success<U>&& v): value(std::move(v.value)) {}
~success() = default;
success(const success&) = default;
success(success&&) = default;
success& operator=(const success&) = default;
success& operator=(success&&) = default;
};
template<typename T>
struct failure
{
using value_type = T;
value_type value;
explicit failure(const value_type& v)
noexcept(std::is_nothrow_copy_constructible<value_type>::value)
: value(v)
{}
explicit failure(value_type&& v)
noexcept(std::is_nothrow_move_constructible<value_type>::value)
: value(std::move(v))
{}
template<typename U>
explicit failure(U&& v): value(std::forward<U>(v)) {}
template<typename U>
explicit failure(const failure<U>& v): value(v.value) {}
template<typename U>
explicit failure(failure<U>&& v): value(std::move(v.value)) {}
~failure() = default;
failure(const failure&) = default;
failure(failure&&) = default;
failure& operator=(const failure&) = default;
failure& operator=(failure&&) = default;
};
template<typename T>
success<typename std::remove_cv<typename std::remove_reference<T>::type>::type>
ok(T&& v)
{
return success<
typename std::remove_cv<typename std::remove_reference<T>::type>::type
>(std::forward<T>(v));
}
template<typename T>
failure<typename std::remove_cv<typename std::remove_reference<T>::type>::type>
err(T&& v)
{
return failure<
typename std::remove_cv<typename std::remove_reference<T>::type>::type
>(std::forward<T>(v));
}
inline success<std::string> ok(const char* literal)
{
return success<std::string>(std::string(literal));
}
inline failure<std::string> err(const char* literal)
{
return failure<std::string>(std::string(literal));
}
template<typename T, typename E>
struct result
{
using value_type = T;
using error_type = E;
using success_type = success<value_type>;
using failure_type = failure<error_type>;
result(const success_type& s): is_ok_(true)
{
auto tmp = ::new(std::addressof(this->succ)) success_type(s);
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
result(const failure_type& f): is_ok_(false)
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(f);
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
result(success_type&& s): is_ok_(true)
{
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(s));
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
result(failure_type&& f): is_ok_(false)
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(f));
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
template<typename U>
result(const success<U>& s): is_ok_(true)
{
auto tmp = ::new(std::addressof(this->succ)) success_type(s.value);
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
template<typename U>
result(const failure<U>& f): is_ok_(false)
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(f.value);
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
template<typename U>
result(success<U>&& s): is_ok_(true)
{
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(s.value));
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
template<typename U>
result(failure<U>&& f): is_ok_(false)
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(f.value));
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
result& operator=(const success_type& s)
{
this->cleanup();
this->is_ok_ = true;
auto tmp = ::new(std::addressof(this->succ)) success_type(s);
assert(tmp == std::addressof(this->succ));
(void)tmp;
return *this;
}
result& operator=(const failure_type& f)
{
this->cleanup();
this->is_ok_ = false;
auto tmp = ::new(std::addressof(this->fail)) failure_type(f);
assert(tmp == std::addressof(this->fail));
(void)tmp;
return *this;
}
result& operator=(success_type&& s)
{
this->cleanup();
this->is_ok_ = true;
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(s));
assert(tmp == std::addressof(this->succ));
(void)tmp;
return *this;
}
result& operator=(failure_type&& f)
{
this->cleanup();
this->is_ok_ = false;
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(f));
assert(tmp == std::addressof(this->fail));
(void)tmp;
return *this;
}
template<typename U>
result& operator=(const success<U>& s)
{
this->cleanup();
this->is_ok_ = true;
auto tmp = ::new(std::addressof(this->succ)) success_type(s.value);
assert(tmp == std::addressof(this->succ));
(void)tmp;
return *this;
}
template<typename U>
result& operator=(const failure<U>& f)
{
this->cleanup();
this->is_ok_ = false;
auto tmp = ::new(std::addressof(this->fail)) failure_type(f.value);
assert(tmp == std::addressof(this->fail));
(void)tmp;
return *this;
}
template<typename U>
result& operator=(success<U>&& s)
{
this->cleanup();
this->is_ok_ = true;
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(s.value));
assert(tmp == std::addressof(this->succ));
(void)tmp;
return *this;
}
template<typename U>
result& operator=(failure<U>&& f)
{
this->cleanup();
this->is_ok_ = false;
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(f.value));
assert(tmp == std::addressof(this->fail));
(void)tmp;
return *this;
}
~result() noexcept {this->cleanup();}
result(const result& other): is_ok_(other.is_ok())
{
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(other.as_ok());
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(other.as_err());
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
}
result(result&& other): is_ok_(other.is_ok())
{
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(other.as_ok()));
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(other.as_err()));
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
}
template<typename U, typename F>
result(const result<U, F>& other): is_ok_(other.is_ok())
{
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(other.as_ok());
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(other.as_err());
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
}
template<typename U, typename F>
result(result<U, F>&& other): is_ok_(other.is_ok())
{
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(other.as_ok()));
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(other.as_err()));
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
}
result& operator=(const result& other)
{
this->cleanup();
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(other.as_ok());
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(other.as_err());
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
is_ok_ = other.is_ok();
return *this;
}
result& operator=(result&& other)
{
this->cleanup();
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(other.as_ok()));
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(other.as_err()));
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
is_ok_ = other.is_ok();
return *this;
}
template<typename U, typename F>
result& operator=(const result<U, F>& other)
{
this->cleanup();
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(other.as_ok());
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(other.as_err());
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
is_ok_ = other.is_ok();
return *this;
}
template<typename U, typename F>
result& operator=(result<U, F>&& other)
{
this->cleanup();
if(other.is_ok())
{
auto tmp = ::new(std::addressof(this->succ)) success_type(std::move(other.as_ok()));
assert(tmp == std::addressof(this->succ));
(void)tmp;
}
else
{
auto tmp = ::new(std::addressof(this->fail)) failure_type(std::move(other.as_err()));
assert(tmp == std::addressof(this->fail));
(void)tmp;
}
is_ok_ = other.is_ok();
return *this;
}
bool is_ok() const noexcept {return is_ok_;}
bool is_err() const noexcept {return !is_ok_;}
operator bool() const noexcept {return is_ok_;}
value_type& unwrap() &
{
if(is_err())
{
throw std::runtime_error("toml::result: bad unwrap: " +
format_error(this->as_err()));
}
return this->succ.value;
}
value_type const& unwrap() const&
{
if(is_err())
{
throw std::runtime_error("toml::result: bad unwrap: " +
format_error(this->as_err()));
}
return this->succ.value;
}
value_type&& unwrap() &&
{
if(is_err())
{
throw std::runtime_error("toml::result: bad unwrap: " +
format_error(this->as_err()));
}
return std::move(this->succ.value);
}
value_type& unwrap_or(value_type& opt) &
{
if(is_err()) {return opt;}
return this->succ.value;
}
value_type const& unwrap_or(value_type const& opt) const&
{
if(is_err()) {return opt;}
return this->succ.value;
}
value_type unwrap_or(value_type opt) &&
{
if(is_err()) {return opt;}
return this->succ.value;
}
error_type& unwrap_err() &
{
if(is_ok()) {throw std::runtime_error("toml::result: bad unwrap_err");}
return this->fail.value;
}
error_type const& unwrap_err() const&
{
if(is_ok()) {throw std::runtime_error("toml::result: bad unwrap_err");}
return this->fail.value;
}
error_type&& unwrap_err() &&
{
if(is_ok()) {throw std::runtime_error("toml::result: bad unwrap_err");}
return std::move(this->fail.value);
}
value_type& as_ok() & noexcept {return this->succ.value;}
value_type const& as_ok() const& noexcept {return this->succ.value;}
value_type&& as_ok() && noexcept {return std::move(this->succ.value);}
error_type& as_err() & noexcept {return this->fail.value;}
error_type const& as_err() const& noexcept {return this->fail.value;}
error_type&& as_err() && noexcept {return std::move(this->fail.value);}
// prerequisities
// F: T -> U
// retval: result<U, E>
template<typename F>
result<detail::return_type_of_t<F, value_type&>, error_type>
map(F&& f) &
{
if(this->is_ok()){return ok(f(this->as_ok()));}
return err(this->as_err());
}
template<typename F>
result<detail::return_type_of_t<F, value_type const&>, error_type>
map(F&& f) const&
{
if(this->is_ok()){return ok(f(this->as_ok()));}
return err(this->as_err());
}
template<typename F>
result<detail::return_type_of_t<F, value_type &&>, error_type>
map(F&& f) &&
{
if(this->is_ok()){return ok(f(std::move(this->as_ok())));}
return err(std::move(this->as_err()));
}
// prerequisities
// F: E -> F
// retval: result<T, F>
template<typename F>
result<value_type, detail::return_type_of_t<F, error_type&>>
map_err(F&& f) &
{
if(this->is_err()){return err(f(this->as_err()));}
return ok(this->as_ok());
}
template<typename F>
result<value_type, detail::return_type_of_t<F, error_type const&>>
map_err(F&& f) const&
{
if(this->is_err()){return err(f(this->as_err()));}
return ok(this->as_ok());
}
template<typename F>
result<value_type, detail::return_type_of_t<F, error_type&&>>
map_err(F&& f) &&
{
if(this->is_err()){return err(f(std::move(this->as_err())));}
return ok(std::move(this->as_ok()));
}
// prerequisities
// F: T -> U
// retval: U
template<typename F, typename U>
detail::return_type_of_t<F, value_type&>
map_or_else(F&& f, U&& opt) &
{
if(this->is_err()){return std::forward<U>(opt);}
return f(this->as_ok());
}
template<typename F, typename U>
detail::return_type_of_t<F, value_type const&>
map_or_else(F&& f, U&& opt) const&
{
if(this->is_err()){return std::forward<U>(opt);}
return f(this->as_ok());
}
template<typename F, typename U>
detail::return_type_of_t<F, value_type&&>
map_or_else(F&& f, U&& opt) &&
{
if(this->is_err()){return std::forward<U>(opt);}
return f(std::move(this->as_ok()));
}
// prerequisities
// F: E -> U
// retval: U
template<typename F, typename U>
detail::return_type_of_t<F, error_type&>
map_err_or_else(F&& f, U&& opt) &
{
if(this->is_ok()){return std::forward<U>(opt);}
return f(this->as_err());
}
template<typename F, typename U>
detail::return_type_of_t<F, error_type const&>
map_err_or_else(F&& f, U&& opt) const&
{
if(this->is_ok()){return std::forward<U>(opt);}
return f(this->as_err());
}
template<typename F, typename U>
detail::return_type_of_t<F, error_type&&>
map_err_or_else(F&& f, U&& opt) &&
{
if(this->is_ok()){return std::forward<U>(opt);}
return f(std::move(this->as_err()));
}
// prerequisities:
// F: func T -> U
// toml::err(error_type) should be convertible to U.
// normally, type U is another result<S, F> and E is convertible to F
template<typename F>
detail::return_type_of_t<F, value_type&>
and_then(F&& f) &
{
if(this->is_ok()){return f(this->as_ok());}
return err(this->as_err());
}
template<typename F>
detail::return_type_of_t<F, value_type const&>
and_then(F&& f) const&
{
if(this->is_ok()){return f(this->as_ok());}
return err(this->as_err());
}
template<typename F>
detail::return_type_of_t<F, value_type&&>
and_then(F&& f) &&
{
if(this->is_ok()){return f(std::move(this->as_ok()));}
return err(std::move(this->as_err()));
}
// prerequisities:
// F: func E -> U
// toml::ok(value_type) should be convertible to U.
// normally, type U is another result<S, F> and T is convertible to S
template<typename F>
detail::return_type_of_t<F, error_type&>
or_else(F&& f) &
{
if(this->is_err()){return f(this->as_err());}
return ok(this->as_ok());
}
template<typename F>
detail::return_type_of_t<F, error_type const&>
or_else(F&& f) const&
{
if(this->is_err()){return f(this->as_err());}
return ok(this->as_ok());
}
template<typename F>
detail::return_type_of_t<F, error_type&&>
or_else(F&& f) &&
{
if(this->is_err()){return f(std::move(this->as_err()));}
return ok(std::move(this->as_ok()));
}
// if *this is error, returns *this. otherwise, returns other.
result and_other(const result& other) const&
{
return this->is_err() ? *this : other;
}
result and_other(result&& other) &&
{
return this->is_err() ? std::move(*this) : std::move(other);
}
// if *this is okay, returns *this. otherwise, returns other.
result or_other(const result& other) const&
{
return this->is_ok() ? *this : other;
}
result or_other(result&& other) &&
{
return this->is_ok() ? std::move(*this) : std::move(other);
}
void swap(result<T, E>& other)
{
result<T, E> tmp(std::move(*this));
*this = std::move(other);
other = std::move(tmp);
return ;
}
private:
static std::string format_error(std::exception const& excpt)
{
return std::string(excpt.what());
}
template<typename U, typename std::enable_if<!std::is_base_of<
std::exception, U>::value, std::nullptr_t>::type = nullptr>
static std::string format_error(U const& others)
{
std::ostringstream oss; oss << others;
return oss.str();
}
void cleanup() noexcept
{
if(this->is_ok_) {this->succ.~success_type();}
else {this->fail.~failure_type();}
return;
}
private:
bool is_ok_;
union
{
success_type succ;
failure_type fail;
};
};
template<typename T, typename E>
void swap(result<T, E>& lhs, result<T, E>& rhs)
{
lhs.swap(rhs);
return;
}
// this might be confusing because it eagerly evaluated, while in the other
// cases operator && and || are short-circuited.
//
// template<typename T, typename E>
// inline result<T, E>
// operator&&(const result<T, E>& lhs, const result<T, E>& rhs) noexcept
// {
// return lhs.is_ok() ? rhs : lhs;
// }
//
// template<typename T, typename E>
// inline result<T, E>
// operator||(const result<T, E>& lhs, const result<T, E>& rhs) noexcept
// {
// return lhs.is_ok() ? lhs : rhs;
// }
// ----------------------------------------------------------------------------
// re-use result<T, E> as a optional<T> with none_t
namespace detail
{
struct none_t {};
inline bool operator==(const none_t&, const none_t&) noexcept {return true;}
inline bool operator!=(const none_t&, const none_t&) noexcept {return false;}
inline bool operator< (const none_t&, const none_t&) noexcept {return false;}
inline bool operator<=(const none_t&, const none_t&) noexcept {return true;}
inline bool operator> (const none_t&, const none_t&) noexcept {return false;}
inline bool operator>=(const none_t&, const none_t&) noexcept {return true;}
template<typename charT, typename traitsT>
std::basic_ostream<charT, traitsT>&
operator<<(std::basic_ostream<charT, traitsT>& os, const none_t&)
{
os << "none";
return os;
}
inline failure<none_t> none() noexcept {return failure<none_t>{none_t{}};}
} // detail
} // toml11
#endif// TOML11_RESULT_H

922
src/toml11/toml/serializer.hpp Normal file
View file

@ -0,0 +1,922 @@
// Copyright Toru Niina 2019.
// Distributed under the MIT License.
#ifndef TOML11_SERIALIZER_HPP
#define TOML11_SERIALIZER_HPP
#include <cmath>
#include <cstdio>
#include <limits>
#include "lexer.hpp"
#include "value.hpp"
namespace toml
{
// This function serialize a key. It checks a string is a bare key and
// escapes special characters if the string is not compatible to a bare key.
// ```cpp
// std::string k("non.bare.key"); // the key itself includes `.`s.
// std::string formatted = toml::format_key(k);
// assert(formatted == "\"non.bare.key\"");
// ```
//
// This function is exposed to make it easy to write a user-defined serializer.
// Since toml restricts characters available in a bare key, generally a string
// should be escaped. But checking whether a string needs to be surrounded by
// a `"` and escaping some special character is boring.
template<typename charT, typename traits, typename Alloc>
std::basic_string<charT, traits, Alloc>
format_key(const std::basic_string<charT, traits, Alloc>& k)
{
if(k.empty())
{
return std::string("\"\"");
}
// check the key can be a bare (unquoted) key
detail::location loc(k, std::vector<char>(k.begin(), k.end()));
detail::lex_unquoted_key::invoke(loc);
if(loc.iter() == loc.end())
{
return k; // all the tokens are consumed. the key is unquoted-key.
}
//if it includes special characters, then format it in a "quoted" key.
std::basic_string<charT, traits, Alloc> serialized("\"");
for(const char c : k)
{
switch(c)
{
case '\\': {serialized += "\\\\"; break;}
case '\"': {serialized += "\\\""; break;}
case '\b': {serialized += "\\b"; break;}
case '\t': {serialized += "\\t"; break;}
case '\f': {serialized += "\\f"; break;}
case '\n': {serialized += "\\n"; break;}
case '\r': {serialized += "\\r"; break;}
default : {serialized += c; break;}
}
}
serialized += "\"";
return serialized;
}
template<typename charT, typename traits, typename Alloc>
std::basic_string<charT, traits, Alloc>
format_keys(const std::vector<std::basic_string<charT, traits, Alloc>>& keys)
{
if(keys.empty())
{
return std::string("\"\"");
}
std::basic_string<charT, traits, Alloc> serialized;
for(const auto& ky : keys)
{
serialized += format_key(ky);
serialized += charT('.');
}
serialized.pop_back(); // remove the last dot '.'
return serialized;
}
template<typename Value>
struct serializer
{
static_assert(detail::is_basic_value<Value>::value,
"toml::serializer is for toml::value and its variants, "
"toml::basic_value<...>.");
using value_type = Value;
using key_type = typename value_type::key_type ;
using comment_type = typename value_type::comment_type ;
using boolean_type = typename value_type::boolean_type ;
using integer_type = typename value_type::integer_type ;
using floating_type = typename value_type::floating_type ;
using string_type = typename value_type::string_type ;
using local_time_type = typename value_type::local_time_type ;
using local_date_type = typename value_type::local_date_type ;
using local_datetime_type = typename value_type::local_datetime_type ;
using offset_datetime_type = typename value_type::offset_datetime_type;
using array_type = typename value_type::array_type ;
using table_type = typename value_type::table_type ;
serializer(const std::size_t w = 80u,
const int float_prec = std::numeric_limits<toml::floating>::max_digits10,
const bool can_be_inlined = false,
const bool no_comment = false,
std::vector<toml::key> ks = {},
const bool value_has_comment = false)
: can_be_inlined_(can_be_inlined), no_comment_(no_comment),
value_has_comment_(value_has_comment && !no_comment),
float_prec_(float_prec), width_(w), keys_(std::move(ks))
{}
~serializer() = default;
std::string operator()(const boolean_type& b) const
{
return b ? "true" : "false";
}
std::string operator()(const integer_type i) const
{
return std::to_string(i);
}
std::string operator()(const floating_type f) const
{
if(std::isnan(f))
{
if(std::signbit(f))
{
return std::string("-nan");
}
else
{
return std::string("nan");
}
}
else if(!std::isfinite(f))
{
if(std::signbit(f))
{
return std::string("-inf");
}
else
{
return std::string("inf");
}
}
const auto fmt = "%.*g";
const auto bsz = std::snprintf(nullptr, 0, fmt, this->float_prec_, f);
// +1 for null character(\0)
std::vector<char> buf(static_cast<std::size_t>(bsz + 1), '\0');
std::snprintf(buf.data(), buf.size(), fmt, this->float_prec_, f);
std::string token(buf.begin(), std::prev(buf.end()));
if(!token.empty() && token.back() == '.') // 1. => 1.0
{
token += '0';
}
const auto e = std::find_if(
token.cbegin(), token.cend(), [](const char c) noexcept -> bool {
return c == 'e' || c == 'E';
});
const auto has_exponent = (token.cend() != e);
const auto has_fraction = (token.cend() != std::find(
token.cbegin(), token.cend(), '.'));
if(!has_exponent && !has_fraction)
{
// the resulting value does not have any float specific part!
token += ".0";
}
return token;
}
std::string operator()(const string_type& s) const
{
if(s.kind == string_t::basic)
{
if((std::find(s.str.cbegin(), s.str.cend(), '\n') != s.str.cend() ||
std::find(s.str.cbegin(), s.str.cend(), '\"') != s.str.cend()) &&
this->width_ != (std::numeric_limits<std::size_t>::max)())
{
// if linefeed or double-quote is contained,
// make it multiline basic string.
const auto escaped = this->escape_ml_basic_string(s.str);
std::string open("\"\"\"");
std::string close("\"\"\"");
if(escaped.find('\n') != std::string::npos ||
this->width_ < escaped.size() + 6)
{
// if the string body contains newline or is enough long,
// add newlines after and before delimiters.
open += "\n";
close = std::string("\\\n") + close;
}
return open + escaped + close;
}
// no linefeed. try to make it oneline-string.
std::string oneline = this->escape_basic_string(s.str);
if(oneline.size() + 2 < width_ || width_ < 2)
{
const std::string quote("\"");
return quote + oneline + quote;
}
// the line is too long compared to the specified width.
// split it into multiple lines.
std::string token("\"\"\"\n");
while(!oneline.empty())
{
if(oneline.size() < width_)
{
token += oneline;
oneline.clear();
}
else if(oneline.at(width_-2) == '\\')
{
token += oneline.substr(0, width_-2);
token += "\\\n";
oneline.erase(0, width_-2);
}
else
{
token += oneline.substr(0, width_-1);
token += "\\\n";
oneline.erase(0, width_-1);
}
}
return token + std::string("\\\n\"\"\"");
}
else // the string `s` is literal-string.
{
if(std::find(s.str.cbegin(), s.str.cend(), '\n') != s.str.cend() ||
std::find(s.str.cbegin(), s.str.cend(), '\'') != s.str.cend() )
{
std::string open("'''");
if(this->width_ + 6 < s.str.size())
{
open += '\n'; // the first newline is ignored by TOML spec
}
const std::string close("'''");
return open + s.str + close;
}
else
{
const std::string quote("'");
return quote + s.str + quote;
}
}
}
std::string operator()(const local_date_type& d) const
{
std::ostringstream oss;
oss << d;
return oss.str();
}
std::string operator()(const local_time_type& t) const
{
std::ostringstream oss;
oss << t;
return oss.str();
}
std::string operator()(const local_datetime_type& dt) const
{
std::ostringstream oss;
oss << dt;
return oss.str();
}
std::string operator()(const offset_datetime_type& odt) const
{
std::ostringstream oss;
oss << odt;
return oss.str();
}
std::string operator()(const array_type& v) const
{
if(v.empty())
{
return std::string("[]");
}
if(this->is_array_of_tables(v))
{
return make_array_of_tables(v);
}
// not an array of tables. normal array.
// first, try to make it inline if none of the elements have a comment.
if( ! this->has_comment_inside(v))
{
const auto inl = this->make_inline_array(v);
if(inl.size() < this->width_ &&
std::find(inl.cbegin(), inl.cend(), '\n') == inl.cend())
{
return inl;
}
}
// if the length exceeds this->width_, print multiline array.
// key = [
// # ...
// 42,
// ...
// ]
std::string token;
std::string current_line;
token += "[\n";
for(const auto& item : v)
{
if( ! item.comments().empty() && !no_comment_)
{
// if comment exists, the element must be the only element in the line.
// e.g. the following is not allowed.
// ```toml
// array = [
// # comment for what?
// 1, 2, 3, 4, 5
// ]
// ```
if(!current_line.empty())
{
if(current_line.back() != '\n')
{
current_line += '\n';
}
token += current_line;
current_line.clear();
}
for(const auto& c : item.comments())
{
token += '#';
token += c;
token += '\n';
}
token += toml::visit(*this, item);
if(!token.empty() && token.back() == '\n') {token.pop_back();}
token += ",\n";
continue;
}
std::string next_elem;
if(item.is_table())
{
serializer ser(*this);
ser.can_be_inlined_ = true;
ser.width_ = (std::numeric_limits<std::size_t>::max)();
next_elem += toml::visit(ser, item);
}
else
{
next_elem += toml::visit(*this, item);
}
// comma before newline.
if(!next_elem.empty() && next_elem.back() == '\n') {next_elem.pop_back();}
// if current line does not exceeds the width limit, continue.
if(current_line.size() + next_elem.size() + 1 < this->width_)
{
current_line += next_elem;
current_line += ',';
}
else if(current_line.empty())
{
// if current line was empty, force put the next_elem because
// next_elem is not splittable
token += next_elem;
token += ",\n";
// current_line is kept empty
}
else // reset current_line
{
assert(current_line.back() == ',');
token += current_line;
token += '\n';
current_line = next_elem;
current_line += ',';
}
}
if(!current_line.empty())
{
if(!current_line.empty() && current_line.back() != '\n')
{
current_line += '\n';
}
token += current_line;
}
token += "]\n";
return token;
}
// templatize for any table-like container
std::string operator()(const table_type& v) const
{
// if an element has a comment, then it can't be inlined.
// table = {# how can we write a comment for this? key = "value"}
if(this->can_be_inlined_ && !(this->has_comment_inside(v)))
{
std::string token;
if(!this->keys_.empty())
{
token += format_key(this->keys_.back());
token += " = ";
}
token += this->make_inline_table(v);
if(token.size() < this->width_ &&
token.end() == std::find(token.begin(), token.end(), '\n'))
{
return token;
}
}
std::string token;
if(!keys_.empty())
{
token += '[';
token += format_keys(keys_);
token += "]\n";
}
token += this->make_multiline_table(v);
return token;
}
private:
std::string escape_basic_string(const std::string& s) const
{
//XXX assuming `s` is a valid utf-8 sequence.
std::string retval;
for(const char c : s)
{
switch(c)
{
case '\\': {retval += "\\\\"; break;}
case '\"': {retval += "\\\""; break;}
case '\b': {retval += "\\b"; break;}
case '\t': {retval += "\\t"; break;}
case '\f': {retval += "\\f"; break;}
case '\n': {retval += "\\n"; break;}
case '\r': {retval += "\\r"; break;}
default :
{
if((0x00 <= c && c <= 0x08) || (0x0A <= c && c <= 0x1F) || c == 0x7F)
{
retval += "\\u00";
retval += char(48 + (c / 16));
retval += char((c % 16 < 10 ? 48 : 55) + (c % 16));
}
else
{
retval += c;
}
}
}
}
return retval;
}
std::string escape_ml_basic_string(const std::string& s) const
{
std::string retval;
for(auto i=s.cbegin(), e=s.cend(); i!=e; ++i)
{
switch(*i)
{
case '\\': {retval += "\\\\"; break;}
// One or two consecutive "s are allowed.
// Later we will check there are no three consecutive "s.
// case '\"': {retval += "\\\""; break;}
case '\b': {retval += "\\b"; break;}
case '\t': {retval += "\\t"; break;}
case '\f': {retval += "\\f"; break;}
case '\n': {retval += "\n"; break;}
case '\r':
{
if(std::next(i) != e && *std::next(i) == '\n')
{
retval += "\r\n";
++i;
}
else
{
retval += "\\r";
}
break;
}
default :
{
const auto c = *i;
if((0x00 <= c && c <= 0x08) || (0x0A <= c && c <= 0x1F) || c == 0x7F)
{
retval += "\\u00";
retval += char(48 + (c / 16));
retval += char((c % 16 < 10 ? 48 : 55) + (c % 16));
}
else
{
retval += c;
}
}
}
}
// Only 1 or 2 consecutive `"`s are allowed in multiline basic string.
// 3 consecutive `"`s are considered as a closing delimiter.
// We need to check if there are 3 or more consecutive `"`s and insert
// backslash to break them down into several short `"`s like the `str6`
// in the following example.
// ```toml
// str4 = """Here are two quotation marks: "". Simple enough."""
// # str5 = """Here are three quotation marks: """.""" # INVALID
// str5 = """Here are three quotation marks: ""\"."""
// str6 = """Here are fifteen quotation marks: ""\"""\"""\"""\"""\"."""
// ```
auto found_3_quotes = retval.find("\"\"\"");
while(found_3_quotes != std::string::npos)
{
retval.replace(found_3_quotes, 3, "\"\"\\\"");
found_3_quotes = retval.find("\"\"\"");
}
return retval;
}
// if an element of a table or an array has a comment, it cannot be inlined.
bool has_comment_inside(const array_type& a) const noexcept
{
// if no_comment is set, comments would not be written.
if(this->no_comment_) {return false;}
for(const auto& v : a)
{
if(!v.comments().empty()) {return true;}
}
return false;
}
bool has_comment_inside(const table_type& t) const noexcept
{
// if no_comment is set, comments would not be written.
if(this->no_comment_) {return false;}
for(const auto& kv : t)
{
if(!kv.second.comments().empty()) {return true;}
}
return false;
}
std::string make_inline_array(const array_type& v) const
{
assert(!has_comment_inside(v));
std::string token;
token += '[';
bool is_first = true;
for(const auto& item : v)
{
if(is_first) {is_first = false;} else {token += ',';}
token += visit(serializer(
(std::numeric_limits<std::size_t>::max)(), this->float_prec_,
/* inlined */ true, /*no comment*/ false, /*keys*/ {},
/*has_comment*/ !item.comments().empty()), item);
}
token += ']';
return token;
}
std::string make_inline_table(const table_type& v) const
{
assert(!has_comment_inside(v));
assert(this->can_be_inlined_);
std::string token;
token += '{';
bool is_first = true;
for(const auto& kv : v)
{
// in inline tables, trailing comma is not allowed (toml-lang #569).
if(is_first) {is_first = false;} else {token += ',';}
token += format_key(kv.first);
token += '=';
token += visit(serializer(
(std::numeric_limits<std::size_t>::max)(), this->float_prec_,
/* inlined */ true, /*no comment*/ false, /*keys*/ {},
/*has_comment*/ !kv.second.comments().empty()), kv.second);
}
token += '}';
return token;
}
std::string make_multiline_table(const table_type& v) const
{
std::string token;
// print non-table elements first.
// ```toml
// [foo] # a table we're writing now here
// key = "value" # <- non-table element, "key"
// # ...
// [foo.bar] # <- table element, "bar"
// ```
// because after printing [foo.bar], the remaining non-table values will
// be assigned into [foo.bar], not [foo]. Those values should be printed
// earlier.
for(const auto& kv : v)
{
if(kv.second.is_table() || is_array_of_tables(kv.second))
{
continue;
}
token += write_comments(kv.second);
const auto key_and_sep = format_key(kv.first) + " = ";
const auto residual_width = (this->width_ > key_and_sep.size()) ?
this->width_ - key_and_sep.size() : 0;
token += key_and_sep;
token += visit(serializer(residual_width, this->float_prec_,
/*can be inlined*/ true, /*no comment*/ false, /*keys*/ {},
/*has_comment*/ !kv.second.comments().empty()), kv.second);
if(token.back() != '\n')
{
token += '\n';
}
}
// normal tables / array of tables
// after multiline table appeared, the other tables cannot be inline
// because the table would be assigned into the table.
// [foo]
// ...
// bar = {...} # <- bar will be a member of [foo].
bool multiline_table_printed = false;
for(const auto& kv : v)
{
if(!kv.second.is_table() && !is_array_of_tables(kv.second))
{
continue; // other stuff are already serialized. skip them.
}
std::vector<toml::key> ks(this->keys_);
ks.push_back(kv.first);
auto tmp = visit(serializer(this->width_, this->float_prec_,
!multiline_table_printed, this->no_comment_, ks,
/*has_comment*/ !kv.second.comments().empty()), kv.second);
// If it is the first time to print a multi-line table, it would be
// helpful to separate normal key-value pair and subtables by a
// newline.
// (this checks if the current key-value pair contains newlines.
// but it is not perfect because multi-line string can also contain
// a newline. in such a case, an empty line will be written) TODO
if((!multiline_table_printed) &&
std::find(tmp.cbegin(), tmp.cend(), '\n') != tmp.cend())
{
multiline_table_printed = true;
token += '\n'; // separate key-value pairs and subtables
token += write_comments(kv.second);
token += tmp;
// care about recursive tables (all tables in each level prints
// newline and there will be a full of newlines)
if(tmp.substr(tmp.size() - 2, 2) != "\n\n" &&
tmp.substr(tmp.size() - 4, 4) != "\r\n\r\n" )
{
token += '\n';
}
}
else
{
token += write_comments(kv.second);
token += tmp;
token += '\n';
}
}
return token;
}
std::string make_array_of_tables(const array_type& v) const
{
// if it's not inlined, we need to add `[[table.key]]`.
// but if it can be inlined, we can format it as the following.
// ```
// table.key = [
// {...},
// # comment
// {...},
// ]
// ```
// This function checks if inlinization is possible or not, and then
// format the array-of-tables in a proper way.
//
// Note about comments:
//
// If the array itself has a comment (value_has_comment_ == true), we
// should try to make it inline.
// ```toml
// # comment about array
// array = [
// # comment about table element
// {of = "table"}
// ]
// ```
// If it is formatted as a multiline table, the two comments becomes
// indistinguishable.
// ```toml
// # comment about array
// # comment about table element
// [[array]]
// of = "table"
// ```
// So we need to try to make it inline, and it force-inlines regardless
// of the line width limit.
// It may fail if the element of a table has comment. In that case,
// the array-of-tables will be formatted as a multiline table.
if(this->can_be_inlined_ || this->value_has_comment_)
{
std::string token;
if(!keys_.empty())
{
token += format_key(keys_.back());
token += " = ";
}
bool failed = false;
token += "[\n";
for(const auto& item : v)
{
// if an element of the table has a comment, the table
// cannot be inlined.
if(this->has_comment_inside(item.as_table()))
{
failed = true;
break;
}
// write comments for the table itself
token += write_comments(item);
const auto t = this->make_inline_table(item.as_table());
if(t.size() + 1 > width_ || // +1 for the last comma {...},
std::find(t.cbegin(), t.cend(), '\n') != t.cend())
{
// if the value itself has a comment, ignore the line width limit
if( ! this->value_has_comment_)
{
failed = true;
break;
}
}
token += t;
token += ",\n";
}
if( ! failed)
{
token += "]\n";
return token;
}
// if failed, serialize them as [[array.of.tables]].
}
std::string token;
for(const auto& item : v)
{
token += write_comments(item);
token += "[[";
token += format_keys(keys_);
token += "]]\n";
token += this->make_multiline_table(item.as_table());
}
return token;
}
std::string write_comments(const value_type& v) const
{
std::string retval;
if(this->no_comment_) {return retval;}
for(const auto& c : v.comments())
{
retval += '#';
retval += c;
retval += '\n';
}
return retval;
}
bool is_array_of_tables(const value_type& v) const
{
if(!v.is_array() || v.as_array().empty()) {return false;}
return is_array_of_tables(v.as_array());
}
bool is_array_of_tables(const array_type& v) const
{
// Since TOML v0.5.0, heterogeneous arrays are allowed. So we need to
// check all the element in an array to check if the array is an array
// of tables.
return std::all_of(v.begin(), v.end(), [](const value_type& elem) {
return elem.is_table();
});
}
private:
bool can_be_inlined_;
bool no_comment_;
bool value_has_comment_;
int float_prec_;
std::size_t width_;
std::vector<toml::key> keys_;
};
template<typename C,
template<typename ...> class M, template<typename ...> class V>
std::string
format(const basic_value<C, M, V>& v, std::size_t w = 80u,
int fprec = std::numeric_limits<toml::floating>::max_digits10,
bool no_comment = false, bool force_inline = false)
{
using value_type = basic_value<C, M, V>;
// if value is a table, it is considered to be a root object.
// the root object can't be an inline table.
if(v.is_table())
{
std::ostringstream oss;
if(!v.comments().empty())
{
oss << v.comments();
oss << '\n'; // to split the file comment from the first element
}
const auto serialized = visit(serializer<value_type>(w, fprec, false, no_comment), v);
oss << serialized;
return oss.str();
}
return visit(serializer<value_type>(w, fprec, force_inline), v);
}
namespace detail
{
template<typename charT, typename traits>
int comment_index(std::basic_ostream<charT, traits>&)
{
static const int index = std::ios_base::xalloc();
return index;
}
} // detail
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
nocomment(std::basic_ostream<charT, traits>& os)
{
// by default, it is zero. and by default, it shows comments.
os.iword(detail::comment_index(os)) = 1;
return os;
}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
showcomment(std::basic_ostream<charT, traits>& os)
{
// by default, it is zero. and by default, it shows comments.
os.iword(detail::comment_index(os)) = 0;
return os;
}
template<typename charT, typename traits, typename C,
template<typename ...> class M, template<typename ...> class V>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const basic_value<C, M, V>& v)
{
using value_type = basic_value<C, M, V>;
// get status of std::setw().
const auto w = static_cast<std::size_t>(os.width());
const int fprec = static_cast<int>(os.precision());
os.width(0);
// by default, iword is initialized by 0. And by default, toml11 outputs
// comments. So `0` means showcomment. 1 means nocommnet.
const bool no_comment = (1 == os.iword(detail::comment_index(os)));
if(!no_comment && v.is_table() && !v.comments().empty())
{
os << v.comments();
os << '\n'; // to split the file comment from the first element
}
// the root object can't be an inline table. so pass `false`.
const auto serialized = visit(serializer<value_type>(w, fprec, no_comment, false), v);
os << serialized;
// if v is a non-table value, and has only one comment, then
// put a comment just after a value. in the following way.
//
// ```toml
// key = "value" # comment.
// ```
//
// Since the top-level toml object is a table, one who want to put a
// non-table toml value must use this in a following way.
//
// ```cpp
// toml::value v;
// std::cout << "user-defined-key = " << v << std::endl;
// ```
//
// In this case, it is impossible to put comments before key-value pair.
// The only way to preserve comments is to put all of them after a value.
if(!no_comment && !v.is_table() && !v.comments().empty())
{
os << " #";
for(const auto& c : v.comments()) {os << c;}
}
return os;
}
} // toml
#endif// TOML11_SERIALIZER_HPP

233
src/toml11/toml/source_location.hpp Normal file
View file

@ -0,0 +1,233 @@
// Copyright Toru Niina 2019.
// Distributed under the MIT License.
#ifndef TOML11_SOURCE_LOCATION_HPP
#define TOML11_SOURCE_LOCATION_HPP
#include <cstdint>
#include <sstream>
#include "region.hpp"
namespace toml
{
// A struct to contain location in a toml file.
// The interface imitates std::experimental::source_location,
// but not completely the same.
//
// It would be constructed by toml::value. It can be used to generate
// user-defined error messages.
//
// - std::uint_least32_t line() const noexcept
// - returns the line number where the region is on.
// - std::uint_least32_t column() const noexcept
// - returns the column number where the region starts.
// - std::uint_least32_t region() const noexcept
// - returns the size of the region.
//
// +-- line() +-- region of interest (region() == 9)
// v .---+---.
// 12 | value = "foo bar"
// ^
// +-- column()
//
// - std::string const& file_name() const noexcept;
// - name of the file.
// - std::string const& line_str() const noexcept;
// - the whole line that contains the region of interest.
//
struct source_location
{
public:
source_location()
: line_num_(1), column_num_(1), region_size_(1),
file_name_("unknown file"), line_str_("")
{}
explicit source_location(const detail::region_base* reg)
: line_num_(1), column_num_(1), region_size_(1),
file_name_("unknown file"), line_str_("")
{
if(reg)
{
if(reg->line_num() != detail::region_base().line_num())
{
line_num_ = static_cast<std::uint_least32_t>(
std::stoul(reg->line_num()));
}
column_num_ = static_cast<std::uint_least32_t>(reg->before() + 1);
region_size_ = static_cast<std::uint_least32_t>(reg->size());
file_name_ = reg->name();
line_str_ = reg->line();
}
}
explicit source_location(const detail::region& reg)
: line_num_(static_cast<std::uint_least32_t>(std::stoul(reg.line_num()))),
column_num_(static_cast<std::uint_least32_t>(reg.before() + 1)),
region_size_(static_cast<std::uint_least32_t>(reg.size())),
file_name_(reg.name()),
line_str_ (reg.line())
{}
explicit source_location(const detail::location& loc)
: line_num_(static_cast<std::uint_least32_t>(std::stoul(loc.line_num()))),
column_num_(static_cast<std::uint_least32_t>(loc.before() + 1)),
region_size_(static_cast<std::uint_least32_t>(loc.size())),
file_name_(loc.name()),
line_str_ (loc.line())
{}
~source_location() = default;
source_location(source_location const&) = default;
source_location(source_location &&) = default;
source_location& operator=(source_location const&) = default;
source_location& operator=(source_location &&) = default;
std::uint_least32_t line() const noexcept {return line_num_;}
std::uint_least32_t column() const noexcept {return column_num_;}
std::uint_least32_t region() const noexcept {return region_size_;}
std::string const& file_name() const noexcept {return file_name_;}
std::string const& line_str() const noexcept {return line_str_;}
private:
std::uint_least32_t line_num_;
std::uint_least32_t column_num_;
std::uint_least32_t region_size_;
std::string file_name_;
std::string line_str_;
};
namespace detail
{
// internal error message generation.
inline std::string format_underline(const std::string& message,
const std::vector<std::pair<source_location, std::string>>& loc_com,
const std::vector<std::string>& helps = {},
const bool colorize = TOML11_ERROR_MESSAGE_COLORIZED)
{
std::size_t line_num_width = 0;
for(const auto& lc : loc_com)
{
std::uint_least32_t line = lc.first.line();
std::size_t digit = 0;
while(line != 0)
{
line /= 10;
digit += 1;
}
line_num_width = (std::max)(line_num_width, digit);
}
// 1 is the minimum width
line_num_width = std::max<std::size_t>(line_num_width, 1);
std::ostringstream retval;
if(colorize)
{
retval << color::colorize; // turn on ANSI color
}
// XXX
// Here, before `colorize` support, it does not output `[error]` prefix
// automatically. So some user may output it manually and this change may
// duplicate the prefix. To avoid it, check the first 7 characters and
// if it is "[error]", it removes that part from the message shown.
if(message.size() > 7 && message.substr(0, 7) == "[error]")
{
retval << color::bold << color::red << "[error]" << color::reset
<< color::bold << message.substr(7) << color::reset << '\n';
}
else
{
retval << color::bold << color::red << "[error] " << color::reset
<< color::bold << message << color::reset << '\n';
}
const auto format_one_location = [line_num_width]
(std::ostringstream& oss,
const source_location& loc, const std::string& comment) -> void
{
oss << ' ' << color::bold << color::blue
<< std::setw(static_cast<int>(line_num_width))
<< std::right << loc.line() << " | " << color::reset
<< loc.line_str() << '\n';
oss << make_string(line_num_width + 1, ' ')
<< color::bold << color::blue << " | " << color::reset
<< make_string(loc.column()-1 /*1-origin*/, ' ');
if(loc.region() == 1)
{
// invalid
// ^------
oss << color::bold << color::red << "^---" << color::reset;
}
else
{
// invalid
// ~~~~~~~
const auto underline_len = (std::min)(
static_cast<std::size_t>(loc.region()), loc.line_str().size());
oss << color::bold << color::red
<< make_string(underline_len, '~') << color::reset;
}
oss << ' ';
oss << comment;
return;
};
assert(!loc_com.empty());
// --> example.toml
// |
retval << color::bold << color::blue << " --> " << color::reset
<< loc_com.front().first.file_name() << '\n';
retval << make_string(line_num_width + 1, ' ')
<< color::bold << color::blue << " |\n" << color::reset;
// 1 | key value
// | ^--- missing =
format_one_location(retval, loc_com.front().first, loc_com.front().second);
// process the rest of the locations
for(std::size_t i=1; i<loc_com.size(); ++i)
{
const auto& prev = loc_com.at(i-1);
const auto& curr = loc_com.at(i);
retval << '\n';
// if the filenames are the same, print "..."
if(prev.first.file_name() == curr.first.file_name())
{
retval << color::bold << color::blue << " ...\n" << color::reset;
}
else // if filename differs, print " --> filename.toml" again
{
retval << color::bold << color::blue << " --> " << color::reset
<< curr.first.file_name() << '\n';
retval << make_string(line_num_width + 1, ' ')
<< color::bold << color::blue << " |\n" << color::reset;
}
format_one_location(retval, curr.first, curr.second);
}
if(!helps.empty())
{
retval << '\n';
retval << make_string(line_num_width + 1, ' ');
retval << color::bold << color::blue << " |" << color::reset;
for(const auto& help : helps)
{
retval << color::bold << "\nHint: " << color::reset;
retval << help;
}
}
return retval.str();
}
} // detail
} // toml
#endif// TOML11_SOURCE_LOCATION_HPP

43
src/toml11/toml/storage.hpp Normal file
View file

@ -0,0 +1,43 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_STORAGE_HPP
#define TOML11_STORAGE_HPP
#include "utility.hpp"
namespace toml
{
namespace detail
{
// this contains pointer and deep-copy the content if copied.
// to avoid recursive pointer.
template<typename T>
struct storage
{
using value_type = T;
explicit storage(value_type const& v): ptr(toml::make_unique<T>(v)) {}
explicit storage(value_type&& v): ptr(toml::make_unique<T>(std::move(v))) {}
~storage() = default;
storage(const storage& rhs): ptr(toml::make_unique<T>(*rhs.ptr)) {}
storage& operator=(const storage& rhs)
{
this->ptr = toml::make_unique<T>(*rhs.ptr);
return *this;
}
storage(storage&&) = default;
storage& operator=(storage&&) = default;
bool is_ok() const noexcept {return static_cast<bool>(ptr);}
value_type& value() & noexcept {return *ptr;}
value_type const& value() const& noexcept {return *ptr;}
value_type&& value() && noexcept {return std::move(*ptr);}
private:
std::unique_ptr<value_type> ptr;
};
} // detail
} // toml
#endif// TOML11_STORAGE_HPP

225
src/toml11/toml/string.hpp Normal file
View file

@ -0,0 +1,225 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_STRING_HPP
#define TOML11_STRING_HPP
#include <cstdint>
#include <algorithm>
#include <string>
#if __cplusplus >= 201703L
#if __has_include(<string_view>)
#define TOML11_USING_STRING_VIEW 1
#include <string_view>
#endif
#endif
namespace toml
{
enum class string_t : std::uint8_t
{
basic = 0,
literal = 1,
};
struct string
{
string() = default;
~string() = default;
string(const string& s) = default;
string(string&& s) = default;
string& operator=(const string& s) = default;
string& operator=(string&& s) = default;
string(const std::string& s): kind(string_t::basic), str(s){}
string(const std::string& s, string_t k): kind(k), str(s){}
string(const char* s): kind(string_t::basic), str(s){}
string(const char* s, string_t k): kind(k), str(s){}
string(std::string&& s): kind(string_t::basic), str(std::move(s)){}
string(std::string&& s, string_t k): kind(k), str(std::move(s)){}
string& operator=(const std::string& s)
{kind = string_t::basic; str = s; return *this;}
string& operator=(std::string&& s)
{kind = string_t::basic; str = std::move(s); return *this;}
operator std::string& () & noexcept {return str;}
operator std::string const& () const& noexcept {return str;}
operator std::string&& () && noexcept {return std::move(str);}
string& operator+=(const char* rhs) {str += rhs; return *this;}
string& operator+=(const char rhs) {str += rhs; return *this;}
string& operator+=(const std::string& rhs) {str += rhs; return *this;}
string& operator+=(const string& rhs) {str += rhs.str; return *this;}
#if defined(TOML11_USING_STRING_VIEW) && TOML11_USING_STRING_VIEW>0
explicit string(std::string_view s): kind(string_t::basic), str(s){}
string(std::string_view s, string_t k): kind(k), str(s){}
string& operator=(std::string_view s)
{kind = string_t::basic; str = s; return *this;}
explicit operator std::string_view() const noexcept
{return std::string_view(str);}
string& operator+=(const std::string_view& rhs) {str += rhs; return *this;}
#endif
string_t kind;
std::string str;
};
inline bool operator==(const string& lhs, const string& rhs)
{
return lhs.kind == rhs.kind && lhs.str == rhs.str;
}
inline bool operator!=(const string& lhs, const string& rhs)
{
return !(lhs == rhs);
}
inline bool operator<(const string& lhs, const string& rhs)
{
return (lhs.kind == rhs.kind) ? (lhs.str < rhs.str) : (lhs.kind < rhs.kind);
}
inline bool operator>(const string& lhs, const string& rhs)
{
return rhs < lhs;
}
inline bool operator<=(const string& lhs, const string& rhs)
{
return !(rhs < lhs);
}
inline bool operator>=(const string& lhs, const string& rhs)
{
return !(lhs < rhs);
}
inline bool
operator==(const string& lhs, const std::string& rhs) {return lhs.str == rhs;}
inline bool
operator!=(const string& lhs, const std::string& rhs) {return lhs.str != rhs;}
inline bool
operator< (const string& lhs, const std::string& rhs) {return lhs.str < rhs;}
inline bool
operator> (const string& lhs, const std::string& rhs) {return lhs.str > rhs;}
inline bool
operator<=(const string& lhs, const std::string& rhs) {return lhs.str <= rhs;}
inline bool
operator>=(const string& lhs, const std::string& rhs) {return lhs.str >= rhs;}
inline bool
operator==(const std::string& lhs, const string& rhs) {return lhs == rhs.str;}
inline bool
operator!=(const std::string& lhs, const string& rhs) {return lhs != rhs.str;}
inline bool
operator< (const std::string& lhs, const string& rhs) {return lhs < rhs.str;}
inline bool
operator> (const std::string& lhs, const string& rhs) {return lhs > rhs.str;}
inline bool
operator<=(const std::string& lhs, const string& rhs) {return lhs <= rhs.str;}
inline bool
operator>=(const std::string& lhs, const string& rhs) {return lhs >= rhs.str;}
inline bool
operator==(const string& lhs, const char* rhs) {return lhs.str == std::string(rhs);}
inline bool
operator!=(const string& lhs, const char* rhs) {return lhs.str != std::string(rhs);}
inline bool
operator< (const string& lhs, const char* rhs) {return lhs.str < std::string(rhs);}
inline bool
operator> (const string& lhs, const char* rhs) {return lhs.str > std::string(rhs);}
inline bool
operator<=(const string& lhs, const char* rhs) {return lhs.str <= std::string(rhs);}
inline bool
operator>=(const string& lhs, const char* rhs) {return lhs.str >= std::string(rhs);}
inline bool
operator==(const char* lhs, const string& rhs) {return std::string(lhs) == rhs.str;}
inline bool
operator!=(const char* lhs, const string& rhs) {return std::string(lhs) != rhs.str;}
inline bool
operator< (const char* lhs, const string& rhs) {return std::string(lhs) < rhs.str;}
inline bool
operator> (const char* lhs, const string& rhs) {return std::string(lhs) > rhs.str;}
inline bool
operator<=(const char* lhs, const string& rhs) {return std::string(lhs) <= rhs.str;}
inline bool
operator>=(const char* lhs, const string& rhs) {return std::string(lhs) >= rhs.str;}
template<typename charT, typename traits>
std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, const string& s)
{
if(s.kind == string_t::basic)
{
if(std::find(s.str.cbegin(), s.str.cend(), '\n') != s.str.cend())
{
// it contains newline. make it multiline string.
os << "\"\"\"\n";
for(auto i=s.str.cbegin(), e=s.str.cend(); i!=e; ++i)
{
switch(*i)
{
case '\\': {os << "\\\\"; break;}
case '\"': {os << "\\\""; break;}
case '\b': {os << "\\b"; break;}
case '\t': {os << "\\t"; break;}
case '\f': {os << "\\f"; break;}
case '\n': {os << '\n'; break;}
case '\r':
{
// since it is a multiline string,
// CRLF is not needed to be escaped.
if(std::next(i) != e && *std::next(i) == '\n')
{
os << "\r\n";
++i;
}
else
{
os << "\\r";
}
break;
}
default: {os << *i; break;}
}
}
os << "\\\n\"\"\"";
return os;
}
// no newline. make it inline.
os << "\"";
for(const auto c : s.str)
{
switch(c)
{
case '\\': {os << "\\\\"; break;}
case '\"': {os << "\\\""; break;}
case '\b': {os << "\\b"; break;}
case '\t': {os << "\\t"; break;}
case '\f': {os << "\\f"; break;}
case '\n': {os << "\\n"; break;}
case '\r': {os << "\\r"; break;}
default : {os << c; break;}
}
}
os << "\"";
return os;
}
// the string `s` is literal-string.
if(std::find(s.str.cbegin(), s.str.cend(), '\n') != s.str.cend() ||
std::find(s.str.cbegin(), s.str.cend(), '\'') != s.str.cend() )
{
// contains newline or single quote. make it multiline.
os << "'''\n" << s.str << "'''";
return os;
}
// normal literal string
os << '\'' << s.str << '\'';
return os;
}
} // toml
#endif// TOML11_STRING_H

327
src/toml11/toml/traits.hpp Normal file
View file

@ -0,0 +1,327 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_TRAITS_HPP
#define TOML11_TRAITS_HPP
#include "from.hpp"
#include "into.hpp"
#include <chrono>
#include <forward_list>
#include <string>
#include <tuple>
#include <type_traits>
#include <utility>
#if __cplusplus >= 201703L
#if __has_include(<string_view>)
#include <string_view>
#endif // has_include(<string_view>)
#endif // cplusplus >= C++17
namespace toml
{
template<typename C, template<typename ...> class T, template<typename ...> class A>
class basic_value;
namespace detail
{
// ---------------------------------------------------------------------------
// check whether type T is a kind of container/map class
struct has_iterator_impl
{
template<typename T> static std::true_type check(typename T::iterator*);
template<typename T> static std::false_type check(...);
};
struct has_value_type_impl
{
template<typename T> static std::true_type check(typename T::value_type*);
template<typename T> static std::false_type check(...);
};
struct has_key_type_impl
{
template<typename T> static std::true_type check(typename T::key_type*);
template<typename T> static std::false_type check(...);
};
struct has_mapped_type_impl
{
template<typename T> static std::true_type check(typename T::mapped_type*);
template<typename T> static std::false_type check(...);
};
struct has_reserve_method_impl
{
template<typename T> static std::false_type check(...);
template<typename T> static std::true_type check(
decltype(std::declval<T>().reserve(std::declval<std::size_t>()))*);
};
struct has_push_back_method_impl
{
template<typename T> static std::false_type check(...);
template<typename T> static std::true_type check(
decltype(std::declval<T>().push_back(std::declval<typename T::value_type>()))*);
};
struct is_comparable_impl
{
template<typename T> static std::false_type check(...);
template<typename T> static std::true_type check(
decltype(std::declval<T>() < std::declval<T>())*);
};
struct has_from_toml_method_impl
{
template<typename T, typename C,
template<typename ...> class Tb, template<typename ...> class A>
static std::true_type check(
decltype(std::declval<T>().from_toml(
std::declval<::toml::basic_value<C, Tb, A>>()))*);
template<typename T, typename C,
template<typename ...> class Tb, template<typename ...> class A>
static std::false_type check(...);
};
struct has_into_toml_method_impl
{
template<typename T>
static std::true_type check(decltype(std::declval<T>().into_toml())*);
template<typename T>
static std::false_type check(...);
};
struct has_specialized_from_impl
{
template<typename T>
static std::false_type check(...);
template<typename T, std::size_t S = sizeof(::toml::from<T>)>
static std::true_type check(::toml::from<T>*);
};
struct has_specialized_into_impl
{
template<typename T>
static std::false_type check(...);
template<typename T, std::size_t S = sizeof(::toml::into<T>)>
static std::true_type check(::toml::from<T>*);
};
/// Intel C++ compiler can not use decltype in parent class declaration, here
/// is a hack to work around it. https://stackoverflow.com/a/23953090/4692076
#ifdef __INTEL_COMPILER
#define decltype(...) std::enable_if<true, decltype(__VA_ARGS__)>::type
#endif
template<typename T>
struct has_iterator : decltype(has_iterator_impl::check<T>(nullptr)){};
template<typename T>
struct has_value_type : decltype(has_value_type_impl::check<T>(nullptr)){};
template<typename T>
struct has_key_type : decltype(has_key_type_impl::check<T>(nullptr)){};
template<typename T>
struct has_mapped_type : decltype(has_mapped_type_impl::check<T>(nullptr)){};
template<typename T>
struct has_reserve_method : decltype(has_reserve_method_impl::check<T>(nullptr)){};
template<typename T>
struct has_push_back_method : decltype(has_push_back_method_impl::check<T>(nullptr)){};
template<typename T>
struct is_comparable : decltype(is_comparable_impl::check<T>(nullptr)){};
template<typename T, typename C,
template<typename ...> class Tb, template<typename ...> class A>
struct has_from_toml_method
: decltype(has_from_toml_method_impl::check<T, C, Tb, A>(nullptr)){};
template<typename T>
struct has_into_toml_method
: decltype(has_into_toml_method_impl::check<T>(nullptr)){};
template<typename T>
struct has_specialized_from : decltype(has_specialized_from_impl::check<T>(nullptr)){};
template<typename T>
struct has_specialized_into : decltype(has_specialized_into_impl::check<T>(nullptr)){};
#ifdef __INTEL_COMPILER
#undef decltype
#endif
// ---------------------------------------------------------------------------
// C++17 and/or/not
#if __cplusplus >= 201703L
using std::conjunction;
using std::disjunction;
using std::negation;
#else
template<typename ...> struct conjunction : std::true_type{};
template<typename T> struct conjunction<T> : T{};
template<typename T, typename ... Ts>
struct conjunction<T, Ts...> :
std::conditional<static_cast<bool>(T::value), conjunction<Ts...>, T>::type
{};
template<typename ...> struct disjunction : std::false_type{};
template<typename T> struct disjunction<T> : T {};
template<typename T, typename ... Ts>
struct disjunction<T, Ts...> :
std::conditional<static_cast<bool>(T::value), T, disjunction<Ts...>>::type
{};
template<typename T>
struct negation : std::integral_constant<bool, !static_cast<bool>(T::value)>{};
#endif
// ---------------------------------------------------------------------------
// type checkers
template<typename T> struct is_std_pair : std::false_type{};
template<typename T1, typename T2>
struct is_std_pair<std::pair<T1, T2>> : std::true_type{};
template<typename T> struct is_std_tuple : std::false_type{};
template<typename ... Ts>
struct is_std_tuple<std::tuple<Ts...>> : std::true_type{};
template<typename T> struct is_std_forward_list : std::false_type{};
template<typename T>
struct is_std_forward_list<std::forward_list<T>> : std::true_type{};
template<typename T> struct is_chrono_duration: std::false_type{};
template<typename Rep, typename Period>
struct is_chrono_duration<std::chrono::duration<Rep, Period>>: std::true_type{};
template<typename T>
struct is_map : conjunction< // map satisfies all the following conditions
has_iterator<T>, // has T::iterator
has_value_type<T>, // has T::value_type
has_key_type<T>, // has T::key_type
has_mapped_type<T> // has T::mapped_type
>{};
template<typename T> struct is_map<T&> : is_map<T>{};
template<typename T> struct is_map<T const&> : is_map<T>{};
template<typename T> struct is_map<T volatile&> : is_map<T>{};
template<typename T> struct is_map<T const volatile&> : is_map<T>{};
template<typename T>
struct is_container : conjunction<
negation<is_map<T>>, // not a map
negation<std::is_same<T, std::string>>, // not a std::string
#if __cplusplus >= 201703L
#if __has_include(<string_view>)
negation<std::is_same<T, std::string_view>>, // not a std::string_view
#endif // has_include(<string_view>)
#endif
has_iterator<T>, // has T::iterator
has_value_type<T> // has T::value_type
>{};
template<typename T> struct is_container<T&> : is_container<T>{};
template<typename T> struct is_container<T const&> : is_container<T>{};
template<typename T> struct is_container<T volatile&> : is_container<T>{};
template<typename T> struct is_container<T const volatile&> : is_container<T>{};
template<typename T>
struct is_basic_value: std::false_type{};
template<typename T> struct is_basic_value<T&> : is_basic_value<T>{};
template<typename T> struct is_basic_value<T const&> : is_basic_value<T>{};
template<typename T> struct is_basic_value<T volatile&> : is_basic_value<T>{};
template<typename T> struct is_basic_value<T const volatile&> : is_basic_value<T>{};
template<typename C, template<typename ...> class M, template<typename ...> class V>
struct is_basic_value<::toml::basic_value<C, M, V>>: std::true_type{};
// ---------------------------------------------------------------------------
// C++14 index_sequence
#if __cplusplus >= 201402L
using std::index_sequence;
using std::make_index_sequence;
#else
template<std::size_t ... Ns> struct index_sequence{};
template<typename IS, std::size_t N> struct push_back_index_sequence{};
template<std::size_t N, std::size_t ... Ns>
struct push_back_index_sequence<index_sequence<Ns...>, N>
{
typedef index_sequence<Ns..., N> type;
};
template<std::size_t N>
struct index_sequence_maker
{
typedef typename push_back_index_sequence<
typename index_sequence_maker<N-1>::type, N>::type type;
};
template<>
struct index_sequence_maker<0>
{
typedef index_sequence<0> type;
};
template<std::size_t N>
using make_index_sequence = typename index_sequence_maker<N-1>::type;
#endif // __cplusplus >= 2014
// ---------------------------------------------------------------------------
// C++14 enable_if_t
#if __cplusplus >= 201402L
using std::enable_if_t;
#else
template<bool B, typename T>
using enable_if_t = typename std::enable_if<B, T>::type;
#endif // __cplusplus >= 2014
// ---------------------------------------------------------------------------
// return_type_of_t
#if __cplusplus >= 201703L && defined(__cpp_lib_is_invocable) && __cpp_lib_is_invocable>=201703
template<typename F, typename ... Args>
using return_type_of_t = std::invoke_result_t<F, Args...>;
#else
// result_of is deprecated after C++17
template<typename F, typename ... Args>
using return_type_of_t = typename std::result_of<F(Args...)>::type;
#endif
// ---------------------------------------------------------------------------
// is_string_literal
//
// to use this, pass `typename remove_reference<T>::type` to T.
template<typename T>
struct is_string_literal:
disjunction<
std::is_same<const char*, T>,
conjunction<
std::is_array<T>,
std::is_same<const char, typename std::remove_extent<T>::type>
>
>{};
// ---------------------------------------------------------------------------
// C++20 remove_cvref_t
template<typename T>
struct remove_cvref
{
using type = typename std::remove_cv<
typename std::remove_reference<T>::type>::type;
};
template<typename T>
using remove_cvref_t = typename remove_cvref<T>::type;
}// detail
}//toml
#endif // TOML_TRAITS

173
src/toml11/toml/types.hpp Normal file
View file

@ -0,0 +1,173 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_TYPES_HPP
#define TOML11_TYPES_HPP
#include <unordered_map>
#include <vector>
#include "comments.hpp"
#include "datetime.hpp"
#include "string.hpp"
#include "traits.hpp"
namespace toml
{
template<typename Comment, // discard/preserve_comment
template<typename ...> class Table, // map-like class
template<typename ...> class Array> // vector-like class
class basic_value;
using character = char;
using key = std::string;
#if !defined(__clang__) && defined(__GNUC__) && __GNUC__ <= 4
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wshadow"
#endif
using boolean = bool;
using integer = std::int64_t;
using floating = double; // "float" is a keyword, cannot use it here.
// the following stuffs are structs defined here, so aliases are not needed.
// - string
// - offset_datetime
// - offset_datetime
// - local_datetime
// - local_date
// - local_time
#if defined(__GNUC__) && !defined(__clang__)
# pragma GCC diagnostic pop
#endif
// default toml::value and default array/table. these are defined after defining
// basic_value itself.
// using value = basic_value<discard_comments, std::unordered_map, std::vector>;
// using array = typename value::array_type;
// using table = typename value::table_type;
// to avoid warnings about `value_t::integer` is "shadowing" toml::integer in
// GCC -Wshadow=global.
#if defined(__GNUC__) && !defined(__clang__)
# pragma GCC diagnostic push
# if 7 <= __GNUC__
# pragma GCC diagnostic ignored "-Wshadow=global"
# else // gcc-6 or older
# pragma GCC diagnostic ignored "-Wshadow"
# endif
#endif
enum class value_t : std::uint8_t
{
empty = 0,
boolean = 1,
integer = 2,
floating = 3,
string = 4,
offset_datetime = 5,
local_datetime = 6,
local_date = 7,
local_time = 8,
array = 9,
table = 10,
};
#if defined(__GNUC__) && !defined(__clang__)
# pragma GCC diagnostic pop
#endif
template<typename charT, typename traits>
inline std::basic_ostream<charT, traits>&
operator<<(std::basic_ostream<charT, traits>& os, value_t t)
{
switch(t)
{
case value_t::boolean : os << "boolean"; return os;
case value_t::integer : os << "integer"; return os;
case value_t::floating : os << "floating"; return os;
case value_t::string : os << "string"; return os;
case value_t::offset_datetime : os << "offset_datetime"; return os;
case value_t::local_datetime : os << "local_datetime"; return os;
case value_t::local_date : os << "local_date"; return os;
case value_t::local_time : os << "local_time"; return os;
case value_t::array : os << "array"; return os;
case value_t::table : os << "table"; return os;
case value_t::empty : os << "empty"; return os;
default : os << "unknown"; return os;
}
}
template<typename charT = char,
typename traits = std::char_traits<charT>,
typename alloc = std::allocator<charT>>
inline std::basic_string<charT, traits, alloc> stringize(value_t t)
{
std::basic_ostringstream<charT, traits, alloc> oss;
oss << t;
return oss.str();
}
namespace detail
{
// helper to define a type that represents a value_t value.
template<value_t V>
using value_t_constant = std::integral_constant<value_t, V>;
// meta-function that convertes from value_t to the exact toml type that corresponds to.
// It takes toml::basic_value type because array and table types depend on it.
template<value_t t, typename Value> struct enum_to_type {using type = void ;};
template<typename Value> struct enum_to_type<value_t::empty , Value>{using type = void ;};
template<typename Value> struct enum_to_type<value_t::boolean , Value>{using type = boolean ;};
template<typename Value> struct enum_to_type<value_t::integer , Value>{using type = integer ;};
template<typename Value> struct enum_to_type<value_t::floating , Value>{using type = floating ;};
template<typename Value> struct enum_to_type<value_t::string , Value>{using type = string ;};
template<typename Value> struct enum_to_type<value_t::offset_datetime, Value>{using type = offset_datetime ;};
template<typename Value> struct enum_to_type<value_t::local_datetime , Value>{using type = local_datetime ;};
template<typename Value> struct enum_to_type<value_t::local_date , Value>{using type = local_date ;};
template<typename Value> struct enum_to_type<value_t::local_time , Value>{using type = local_time ;};
template<typename Value> struct enum_to_type<value_t::array , Value>{using type = typename Value::array_type;};
template<typename Value> struct enum_to_type<value_t::table , Value>{using type = typename Value::table_type;};
// meta-function that converts from an exact toml type to the enum that corresponds to.
template<typename T, typename Value>
struct type_to_enum : std::conditional<
std::is_same<T, typename Value::array_type>::value, // if T == array_type,
value_t_constant<value_t::array>, // then value_t::array
typename std::conditional< // else...
std::is_same<T, typename Value::table_type>::value, // if T == table_type
value_t_constant<value_t::table>, // then value_t::table
value_t_constant<value_t::empty> // else value_t::empty
>::type
>::type {};
template<typename Value> struct type_to_enum<boolean , Value>: value_t_constant<value_t::boolean > {};
template<typename Value> struct type_to_enum<integer , Value>: value_t_constant<value_t::integer > {};
template<typename Value> struct type_to_enum<floating , Value>: value_t_constant<value_t::floating > {};
template<typename Value> struct type_to_enum<string , Value>: value_t_constant<value_t::string > {};
template<typename Value> struct type_to_enum<offset_datetime, Value>: value_t_constant<value_t::offset_datetime> {};
template<typename Value> struct type_to_enum<local_datetime , Value>: value_t_constant<value_t::local_datetime > {};
template<typename Value> struct type_to_enum<local_date , Value>: value_t_constant<value_t::local_date > {};
template<typename Value> struct type_to_enum<local_time , Value>: value_t_constant<value_t::local_time > {};
// meta-function that checks the type T is the same as one of the toml::* types.
template<typename T, typename Value>
struct is_exact_toml_type : disjunction<
std::is_same<T, boolean >,
std::is_same<T, integer >,
std::is_same<T, floating >,
std::is_same<T, string >,
std::is_same<T, offset_datetime>,
std::is_same<T, local_datetime >,
std::is_same<T, local_date >,
std::is_same<T, local_time >,
std::is_same<T, typename Value::array_type>,
std::is_same<T, typename Value::table_type>
>{};
template<typename T, typename V> struct is_exact_toml_type<T&, V> : is_exact_toml_type<T, V>{};
template<typename T, typename V> struct is_exact_toml_type<T const&, V> : is_exact_toml_type<T, V>{};
template<typename T, typename V> struct is_exact_toml_type<T volatile&, V> : is_exact_toml_type<T, V>{};
template<typename T, typename V> struct is_exact_toml_type<T const volatile&, V>: is_exact_toml_type<T, V>{};
} // detail
} // toml
#endif// TOML11_TYPES_H

149
src/toml11/toml/utility.hpp Normal file
View file

@ -0,0 +1,149 @@
// Copyright Toru Niina 2017.
// Distributed under the MIT License.
#ifndef TOML11_UTILITY_HPP
#define TOML11_UTILITY_HPP
#include <memory>
#include <sstream>
#include <utility>
#include "traits.hpp"
#if __cplusplus >= 201402L
# define TOML11_MARK_AS_DEPRECATED(msg) [[deprecated(msg)]]
#elif defined(__GNUC__)
# define TOML11_MARK_AS_DEPRECATED(msg) __attribute__((deprecated(msg)))
#elif defined(_MSC_VER)
# define TOML11_MARK_AS_DEPRECATED(msg) __declspec(deprecated(msg))
#else
# define TOML11_MARK_AS_DEPRECATED
#endif
namespace toml
{
#if __cplusplus >= 201402L
using std::make_unique;
#else
template<typename T, typename ... Ts>
inline std::unique_ptr<T> make_unique(Ts&& ... args)
{
return std::unique_ptr<T>(new T(std::forward<Ts>(args)...));
}
#endif // __cplusplus >= 2014
namespace detail
{
template<typename Container>
void try_reserve_impl(Container& container, std::size_t N, std::true_type)
{
container.reserve(N);
return;
}
template<typename Container>
void try_reserve_impl(Container&, std::size_t, std::false_type) noexcept
{
return;
}
} // detail
template<typename Container>
void try_reserve(Container& container, std::size_t N)
{
if(N <= container.size()) {return;}
detail::try_reserve_impl(container, N, detail::has_reserve_method<Container>{});
return;
}
namespace detail
{
inline std::string concat_to_string_impl(std::ostringstream& oss)
{
return oss.str();
}
template<typename T, typename ... Ts>
std::string concat_to_string_impl(std::ostringstream& oss, T&& head, Ts&& ... tail)
{
oss << std::forward<T>(head);
return concat_to_string_impl(oss, std::forward<Ts>(tail) ... );
}
} // detail
template<typename ... Ts>
std::string concat_to_string(Ts&& ... args)
{
std::ostringstream oss;
oss << std::boolalpha << std::fixed;
return detail::concat_to_string_impl(oss, std::forward<Ts>(args) ...);
}
template<typename T>
T from_string(const std::string& str, T opt)
{
T v(opt);
std::istringstream iss(str);
iss >> v;
return v;
}
namespace detail
{
#if __cplusplus >= 201402L
template<typename T>
decltype(auto) last_one(T&& tail) noexcept
{
return std::forward<T>(tail);
}
template<typename T, typename ... Ts>
decltype(auto) last_one(T&& /*head*/, Ts&& ... tail) noexcept
{
return last_one(std::forward<Ts>(tail)...);
}
#else // C++11
// The following code
// ```cpp
// 1 | template<typename T, typename ... Ts>
// 2 | auto last_one(T&& /*head*/, Ts&& ... tail)
// 3 | -> decltype(last_one(std::forward<Ts>(tail)...))
// 4 | {
// 5 | return last_one(std::forward<Ts>(tail)...);
// 6 | }
// ```
// does not work because the function `last_one(...)` is not yet defined at
// line #3, so `decltype()` cannot deduce the type returned from `last_one`.
// So we need to determine return type in a different way, like a meta func.
template<typename T, typename ... Ts>
struct last_one_in_pack
{
using type = typename last_one_in_pack<Ts...>::type;
};
template<typename T>
struct last_one_in_pack<T>
{
using type = T;
};
template<typename ... Ts>
using last_one_in_pack_t = typename last_one_in_pack<Ts...>::type;
template<typename T>
T&& last_one(T&& tail) noexcept
{
return std::forward<T>(tail);
}
template<typename T, typename ... Ts>
enable_if_t<(sizeof...(Ts) > 0), last_one_in_pack_t<Ts&& ...>>
last_one(T&& /*head*/, Ts&& ... tail)
{
return last_one(std::forward<Ts>(tail)...);
}
#endif
} // detail
}// toml
#endif // TOML11_UTILITY

2035
src/toml11/toml/value.hpp Normal file
View file

File diff suppressed because it is too large Load diff

6
tests/ca/import-derivation.sh Normal file
View file

@ -0,0 +1,6 @@
source common.sh
export NIX_TESTS_CA_BY_DEFAULT=1
cd .. && source import-derivation.sh

Some files were not shown because too many files have changed in this diff Show more