Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(self-hosted): external storage #10884

Merged
86 changes: 86 additions & 0 deletions develop-docs/self-hosted/external-storage.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: Self Hosted External Storage
sidebar_title: External Storage
sidebar_order: 90
---

In some cases, storing Sentry data on-disk is not really something people can do. Sometimes, it's better if they can offload it into some bucket storage (like AWS S3 or Google Cloud Storage).
aldy505 marked this conversation as resolved.
Show resolved Hide resolved

<Alert title="Note" level="info">
After changing configuration files, re-run the <code>./install.sh</code> script, to rebuild and restart the containers. See the <Link to="/self-hosted/#configuration">configuration section</Link> for more information.
</Alert>

## Sentry

The Sentry service has a abstraction called "filestore" that handles storing attachment, sourcemap (release artifacts), and replays. Filestore configuration for Sentry should be configured on the `sentry/config.yml` file.
aldy505 marked this conversation as resolved.
Show resolved Hide resolved

### Google Cloud Storage backend

The configuration for GCS backend is pointed to `sentry.filestore.gcs.GoogleCloudStorage`. You will need to set `GOOGLE_APPLICATION_CREDENTIALS` environment variable. For more information, refer to the [Google Cloud documentation for setting up authentication](https://cloud.google.com/storage/docs/reference/libraries#setting_up_authentication).

```yaml
filestore.backend: "gcs"
filestore.options:
bucket_name: "..."
```

### S3 backend

<Alert title="Warning" level="warning">
Although S3 support is available, it is not thoroughly tested and being used by Sentry SaaS internally. Therefore, it is not something that Sentry folks will offer very good support for it.
aldy505 marked this conversation as resolved.
Show resolved Hide resolved
</Alert>

The configuration for S3-compatible backend is pointed to `sentry.filestore.s3.S3Boto3Storage`.

```yaml
filestore.backend: 's3'
filestore.options:
bucket_acl: 'private'
default_acl: 'private'
access_key: '<REDACTED>'
secret_key: '<REDACTED>'
bucket_name: 'my-bucket'
region_name: 'auto'
endpoint_url: 'https://<REDACTED>' # If you're not using AWS.
addressing_style: 'path' # For regular AWS S3, use "auto" or "virtual". For other S3-compatible API like MinIO or Ceph, use "path".
signature_version: 's3v4'
```

Refer to [botocore configuration](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) for valid configuration values.

## Vroom

Vroom is the service that handles profiling. By default the data for profiling is saved on local filesystem. On self-hosted deployment, this should be done by overriding the `SENTRY_BUCKET_PROFILES` environment variable. It's also possible that additional environment variables should be added, depending on the backend of choice.
aldy505 marked this conversation as resolved.
Show resolved Hide resolved

### Google Cloud Storage backend

You will need to set `GOOGLE_APPLICATION_CREDENTIALS` environment variable. For more information, refer to the [Google Cloud documentation for setting up authentication](https://cloud.google.com/storage/docs/reference/libraries#setting_up_authentication).

```bash
gs://my-bucket
```

### S3 backend
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see two S3 backend sections here. Should these be combined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. It's for different services, should resides on a different bucket.


<Alert title="Note" level="warning">
Although S3 support is available, it is not thoroughly tested and being used by Sentry SaaS internally. Therefore, it is not something that Sentry folks will offer very good support for it.
aldy505 marked this conversation as resolved.
Show resolved Hide resolved
</Alert>

```bash
# For regular AWS S3
s3://my-bucket?awssdk=v1&region=us-west-1&endpoint=amazonaws.com

# For other S3-compatible APIs
s3://my-bucket?awssdk=v1&region=any-region&endpoint=minio.yourcompany.com&s3ForcePathStyle=true&disableSSL
```

Additional environment variables should be provided:
- `AWS_ACCESS_KEY=foobar`
- `AWS_SECRET_KEY=foobar`
- `AWS_SESSION_TOKEN=foobar` (optional)

Further explanation on the query string options:
- `region`: The AWS region for requests.
- `endpoint`: The endpoint URL (hostname only or fully qualified URI).
- `disableSSL`: A value of "true" disables SSL when sending requests.
- `s3ForcePathStyle`: A value of "true" forces the request to use path-style addressing.
Loading