Skip to content

Commit

Permalink
Switch cache directories to EmptyDir by default
Browse files Browse the repository at this point in the history
When using PVCs for cache directories, each pod requires two PVCs. This easily leads to exhaustion of the maximum number of mounted volumes per node (see [Node-specific Volume Limits](https://kubernetes.io/docs/concepts/storage/storage-limits/)).

Closes #103
  • Loading branch information
Stefan Bethke committed Oct 17, 2023
1 parent 0822a7c commit 4ec2124
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 3 deletions.
9 changes: 9 additions & 0 deletions docs/custom-resource.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ properties have suitable defaults.
| `licenseSecrets.RLSLicense` | String | `license-rls` | Name of the secret containing a `license.zip` entry with the appropriate file contents |
| `siteMappings` | array || Mappings between DNS names and site segments, see below |
| `with` | object || Optional special components and configurations |
| `with.cachesAsPvc` | boolean | false | Use Persistent Volume Claims when creating various cache directories, instead of EmptyDirs. |
| `with.databases` | boolean | false | Create both a MariaDB and MongoDB server, and schemas and secrets for all components that require them |
| `with.databasesOverride` | object || If `with.databases` is `true`, override the creation for specific kinds. |
| `with.databasesOverride.`*kind* | boolean | true | When set to `false`, do not create database and secrets for *kind*. If set to true, or the entry is missing, do create them. |
Expand All @@ -143,6 +144,14 @@ properties have suitable defaults.

## Enabling Convenience Options `with`

### Persistent Caches `with.cachesAsPvc`

By default, cache directories in UAPI components are created
using [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir)s.

Set this to `true` to instead use Persistent Volume Claims. This allows the caches to persist across pod restarts. Note
however, that this requires a large number of PVCs, which might exceed the limit on your nodes.

### Local database servers `with.databases`

When `with.databases` is enabled, the operator will add a MariaDB and a MongoDB server to the components, and create
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,16 +137,29 @@ public List<Volume> getVolumes() {
.withEmptyDir(new EmptyDirVolumeSource())
.build()
));
if (!getCmcc().getSpec().getWith().getCachesAsPvc()) {
volumes.addAll(List.of(
new VolumeBuilder()
.withName(PVC_TRANSFORMED_BLOBCACHE)
.withEmptyDir(new EmptyDirVolumeSource())
.build(),
new VolumeBuilder()
.withName(PVC_UAPI_BLOBCACHE)
.withEmptyDir(new EmptyDirVolumeSource())
.build()
));
}
return volumes;
}

@Override
public List<PersistentVolumeClaim> getVolumeClaims() {
List<PersistentVolumeClaim> claims = super.getVolumeClaims();

claims.add(getPersistentVolumeClaim(PVC_TRANSFORMED_BLOBCACHE, getVolumeSize(ComponentSpec.VolumeSize::getTransformedBlobCache)));
claims.add(getPersistentVolumeClaim(PVC_UAPI_BLOBCACHE, getVolumeSize(ComponentSpec.VolumeSize::getUapiBlobCache)));

if (getCmcc().getSpec().getWith().getCachesAsPvc()) {
claims.add(getPersistentVolumeClaim(PVC_TRANSFORMED_BLOBCACHE, getVolumeSize(ComponentSpec.VolumeSize::getTransformedBlobCache)));
claims.add(getPersistentVolumeClaim(PVC_UAPI_BLOBCACHE, getVolumeSize(ComponentSpec.VolumeSize::getUapiBlobCache)));
}
return claims;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ public class WithOptions {
@JsonPropertyDescription("Import content, users, themes and workflows")
Boolean contentImport = true;

@JsonPropertyDescription("Create cache directories as Persistent Volume Claims")
Boolean cachesAsPvc = false;

@JsonPropertyDescription("Create databases and secrets for CoreMedia")
Boolean databases = false;

Expand Down

0 comments on commit 4ec2124

Please sign in to comment.