Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Must specify a schema config when using S3 block storage #176

Open
RyanW8 opened this issue Sep 3, 2020 · 1 comment
Open

Must specify a schema config when using S3 block storage #176

RyanW8 opened this issue Sep 3, 2020 · 1 comment

Comments

@RyanW8
Copy link

RyanW8 commented Sep 3, 2020

With the below configuration I am unable to generate the kubernetes manifests. I get this error evaluating jsonnet: RUNTIME ERROR: must specify a schema config. My understanding was that when using s3 block storage it wasn't required to specify a schema?

local cortex = import "cortex/cortex.libsonnet";

  _config+:: {
    namespace: "cortex",
    blocks_storage: {
      backend: 's3',
      tsdb: {
        dir: '/data/cortex-tsdb-ingester',
        ship_interval: '1m',
        block_ranges_period: [ '2h' ],
        retention_period: '3h'
      },
      bucket_store: {
        sync_dir: '/data/cortex-tsdb-querier'
      },
      s3: {
        bucket_name: 'cortex-tsdb',
        endpoint: 'minio-minio-sre-s3-minio.minio.svc.cluster.local:9000',
        secret_access_key: 'REDACTED',
        access_key_id: 'n4monitoring'
      }
    },
    ingester: {
      statefulset_replicas: 3,
      max_transfer_retries: 0,
      lifecycler: {
        join_after: '0s',
        final_sleep: '0s',
        num_tokens: 512,
        ring: {
          replication_factor: 1,
          kvstore: {
            store: 'etcd',
            prefix: 'collectors/',
            etcd: {
              endpoints: [
                'cortex-etcd.cortex.svc.cluster.local:2379'
              ]
            }
          }
        }
      }
    },
    storage: {
      engine: 'blocks'
    },
    store_gateway: {
      sharding_enabled: true,
      sharding_ring: {
        kvstore: {
          store: 'etcd',
          etcd: {
            endpoints: [
              'cortex-etcd.cortex.svc.cluster.local:2379'
            ]
          }
        }
      }
    }
  },
}
@RyanW8 RyanW8 changed the title Must specify a Schema config when using S3 block storage Must specify a schema config when using S3 block storage Sep 3, 2020
@pstibrany
Copy link
Member

Hi, _config isn't just plan translation of Cortex configuration YAML file. It has its own structure, see https://github.com/grafana/cortex-jsonnet/blob/master/cortex/config.libsonnet. To enable blocks storage, you need to use storage_engine: 'blocks' inside _config object (done automatically if you also import "cortex/tsdb.libsonnet" and add it to the config:

local cortex = import 'cortex/cortex.libsonnet';
local tsdb = import 'cortex/tsdb.libsonnet';

cortex + tsdb {
  _config+:: {
    cluster: 'cluster-name',
    namespace: 'cortex',

    blocks_storage_bucket_name: 'bucket_name',
    cortex_store_gateway_data_disk_size: '10Gi',
  },
}

Another example (using chunks storage) is at https://github.com/grafana/cortex-jsonnet/blob/master/cortex/cortex-manifests.jsonnet.example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants