Merge how-to section on S3 buckets into S3 store docs (#7972)
Rather than having a misc tutorial page in the grab-bag "package management" section, this information should just be part of the S3 store docs. --------- Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
This commit is contained in:
Valentin Gagarin
GitHub
parent
c9528d2081
commit
cd680bd53d
|
|
@ -24,7 +24,6 @@
|
|||
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
|
||||
- [Copying Closures via SSH](package-management/copy-closure.md)
|
||||
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
|
||||
- [Serving a Nix store via S3](package-management/s3-substituter.md)
|
||||
- [Nix Language](language/index.md)
|
||||
- [Data Types](language/values.md)
|
||||
- [Language Constructs](language/constructs.md)
|
||||
|
|
|
|||
|
|
@ -17,9 +17,8 @@ the build loop.
|
|||
|
||||
# Prerequisites
|
||||
|
||||
This tutorial assumes you have [configured an S3-compatible binary
|
||||
cache](../package-management/s3-substituter.md), and that the `root`
|
||||
user's default AWS profile can upload to the bucket.
|
||||
This tutorial assumes you have configured an [S3-compatible binary cache](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store) as a [substituter](../command-ref/conf-file.md#conf-substituters),
|
||||
and that the `root` user's default AWS profile can upload to the bucket.
|
||||
|
||||
# Set up a Signing Key
|
||||
|
||||
|
|
|
|||
|
|
@ -1,115 +0,0 @@
|
|||
# Serving a Nix store via S3
|
||||
|
||||
Nix has [built-in support](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store)
|
||||
for storing and fetching store paths from
|
||||
Amazon S3 and S3-compatible services. This uses the same *binary*
|
||||
cache mechanism that Nix usually uses to fetch prebuilt binaries from
|
||||
[cache.nixos.org](https://cache.nixos.org/).
|
||||
|
||||
In this example we will use the bucket named `example-nix-cache`.
|
||||
|
||||
## Anonymous Reads to your S3-compatible binary cache
|
||||
|
||||
If your binary cache is publicly accessible and does not require
|
||||
authentication, the simplest and easiest way to use Nix with your S3
|
||||
compatible binary cache is to use the HTTP URL for that cache.
|
||||
|
||||
For AWS S3 the binary cache URL for example bucket will be exactly
|
||||
<https://example-nix-cache.s3.amazonaws.com> or
|
||||
<s3://example-nix-cache>. For S3 compatible binary caches, consult that
|
||||
cache's documentation.
|
||||
|
||||
Your bucket will need the following bucket policy:
|
||||
|
||||
```json
|
||||
{
|
||||
"Id": "DirectReads",
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "AllowDirectReads",
|
||||
"Action": [
|
||||
"s3:GetObject",
|
||||
"s3:GetBucketLocation"
|
||||
],
|
||||
"Effect": "Allow",
|
||||
"Resource": [
|
||||
"arn:aws:s3:::example-nix-cache",
|
||||
"arn:aws:s3:::example-nix-cache/*"
|
||||
],
|
||||
"Principal": "*"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Authenticated Reads to your S3 binary cache
|
||||
|
||||
For AWS S3 the binary cache URL for example bucket will be exactly
|
||||
<s3://example-nix-cache>.
|
||||
|
||||
Nix will use the [default credential provider
|
||||
chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
|
||||
for authenticating requests to Amazon S3.
|
||||
|
||||
Nix supports authenticated reads from Amazon S3 and S3 compatible binary
|
||||
caches.
|
||||
|
||||
Your bucket will need a bucket policy allowing the desired users to
|
||||
perform the `s3:GetObject` and `s3:GetBucketLocation` action on all
|
||||
objects in the bucket. The [anonymous policy given
|
||||
above](#anonymous-reads-to-your-s3-compatible-binary-cache) can be
|
||||
updated to have a restricted `Principal` to support this.
|
||||
|
||||
## Authenticated Writes to your S3-compatible binary cache
|
||||
|
||||
Nix support fully supports writing to Amazon S3 and S3 compatible
|
||||
buckets. The binary cache URL for our example bucket will be
|
||||
<s3://example-nix-cache>.
|
||||
|
||||
Nix will use the [default credential provider
|
||||
chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
|
||||
for authenticating requests to Amazon S3.
|
||||
|
||||
Your account will need the following IAM policy to upload to the cache:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "UploadToCache",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:AbortMultipartUpload",
|
||||
"s3:GetBucketLocation",
|
||||
"s3:GetObject",
|
||||
"s3:ListBucket",
|
||||
"s3:ListBucketMultipartUploads",
|
||||
"s3:ListMultipartUploadParts",
|
||||
"s3:PutObject"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:s3:::example-nix-cache",
|
||||
"arn:aws:s3:::example-nix-cache/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
To upload with a specific credential profile for Amazon S3:
|
||||
|
||||
```console
|
||||
$ nix copy nixpkgs.hello \
|
||||
--to 's3://example-nix-cache?profile=cache-upload®ion=eu-west-2'
|
||||
```
|
||||
|
||||
To upload to an S3-compatible binary cache:
|
||||
|
||||
```console
|
||||
$ nix copy nixpkgs.hello --to \
|
||||
's3://example-nix-cache?profile=cache-upload&scheme=https&endpoint=minio.example.com'
|
||||
```
|
||||
|
|
@ -2,7 +2,103 @@ R"(
|
|||
|
||||
**Store URL format**: `s3://`*bucket-name*
|
||||
|
||||
This store allows reading and writing a binary cache stored in an AWS
|
||||
S3 bucket.
|
||||
This store allows reading and writing a binary cache stored in an AWS S3 (or S3-compatible service) bucket.
|
||||
This store shares many idioms with the [HTTP Binary Cache Store](#http-binary-cache-store).
|
||||
|
||||
For AWS S3, the binary cache URL for a bucket named `example-nix-cache` will be exactly <s3://example-nix-cache>.
|
||||
For S3 compatible binary caches, consult that cache's documentation.
|
||||
|
||||
### Anonymous reads to your S3-compatible binary cache
|
||||
|
||||
> If your binary cache is publicly accessible and does not require authentication,
|
||||
> it is simplest to use the [HTTP Binary Cache Store] rather than S3 Binary Cache Store with
|
||||
> <https://example-nix-cache.s3.amazonaws.com> instead of <s3://example-nix-cache>.
|
||||
|
||||
Your bucket will need a
|
||||
[bucket policy](https://docs.aws.amazon.com/AmazonS3/v1/userguide/bucket-policies.html)
|
||||
like the following to be accessible:
|
||||
|
||||
```json
|
||||
{
|
||||
"Id": "DirectReads",
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "AllowDirectReads",
|
||||
"Action": [
|
||||
"s3:GetObject",
|
||||
"s3:GetBucketLocation"
|
||||
],
|
||||
"Effect": "Allow",
|
||||
"Resource": [
|
||||
"arn:aws:s3:::example-nix-cache",
|
||||
"arn:aws:s3:::example-nix-cache/*"
|
||||
],
|
||||
"Principal": "*"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
Nix will use the
|
||||
[default credential provider chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
|
||||
for authenticating requests to Amazon S3.
|
||||
|
||||
Note that this means Nix will read environment variables and files with different idioms than with Nix's own settings, as implemented by the AWS SDK.
|
||||
Consult the documentation linked above for further details.
|
||||
|
||||
### Authenticated reads to your S3 binary cache
|
||||
|
||||
Your bucket will need a bucket policy allowing the desired users to perform the `s3:GetObject` and `s3:GetBucketLocation` action on all objects in the bucket.
|
||||
The [anonymous policy given above](#anonymous-reads-to-your-s3-compatible-binary-cache) can be updated to have a restricted `Principal` to support this.
|
||||
|
||||
### Authenticated writes to your S3-compatible binary cache
|
||||
|
||||
Your account will need an IAM policy to support uploading to the bucket:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "UploadToCache",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:AbortMultipartUpload",
|
||||
"s3:GetBucketLocation",
|
||||
"s3:GetObject",
|
||||
"s3:ListBucket",
|
||||
"s3:ListBucketMultipartUploads",
|
||||
"s3:ListMultipartUploadParts",
|
||||
"s3:PutObject"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:s3:::example-nix-cache",
|
||||
"arn:aws:s3:::example-nix-cache/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
With bucket policies and authentication set up as described above, uploading works via [`nix copy`](@docroot@/command-ref/new-cli/nix3-copy.md) (experimental).
|
||||
|
||||
- To upload with a specific credential profile for Amazon S3:
|
||||
|
||||
```console
|
||||
$ nix copy nixpkgs.hello \
|
||||
--to 's3://example-nix-cache?profile=cache-upload®ion=eu-west-2'
|
||||
```
|
||||
|
||||
- To upload to an S3-compatible binary cache:
|
||||
|
||||
```console
|
||||
$ nix copy nixpkgs.hello --to \
|
||||
's3://example-nix-cache?profile=cache-upload&scheme=https&endpoint=minio.example.com'
|
||||
```
|
||||
|
||||
)"
|
||||
|
|
|
|||
Loading…
Reference in New Issue