Skip to content

Latest commit

 

History

History
84 lines (66 loc) · 11.7 KB

RUNNING.md

File metadata and controls

84 lines (66 loc) · 11.7 KB
Raw

Running Shell-operator

Configuration

Run in a cluster

  • Build an image from ghcr.io/flant/shell-operator:latest (or use a specific tag).
  • Copy your hooks to /hooks directory.
  • Apply RBAC manifests.
  • Apply Pod or Deployment manifest with the built image.

More detailed explanation is available in README, also see installing with Helm in the example.

Run outside of a cluster

  • Setup kube context,
  • Prepare hooks directory,
  • Run shell-operator with context and path to hooks directory.

It is not recommended for production but can be useful while debugging hooks. A scenario can be like this:

# Start local cluster
kind create cluster

# Start Shell-operator from outside the cluster
shell-operator start --hooks-dir $(pwd)/hooks --tmp-dir $(pwd)/tmp --log-type color

Environment variables and flags

You can configure the operator with the following environment variables and cli flags:

CLI flag Env-Variable name Default Description
--hooks-dir SHELL_OPERATOR_HOOKS_DIR "" A path to a hooks file structure
--tmp-dir SHELL_OPERATOR_TMP_DIR "/tmp/shell-operator" A path to store temporary files with data for hooks
--listen-address SHELL_OPERATOR_LISTEN_ADDRESS "0.0.0.0" Address to use for HTTP serving.
--listen-port SHELL_OPERATOR_LISTEN_PORT "9115" Port to use for HTTP serving.
--prometheus-metrics-prefix SHELL_OPERATOR_PROMETHEUS_METRICS_PREFIX "shell_operator_" A prefix for metrics names.
--kube-context KUBE_CONTEXT "" The name of the kubeconfig context to use. (as a --context flag of kubectl)
--kube-config KUBE_CONFIG "" Path to the kubeconfig file. (as a $KUBECONFIG for kubectl)
--kube-client-qps KUBE_CLIENT_QPS 5 QPS for rate limiter of k8s.io/client-go
--kube-client-burst KUBE_CLIENT_BURST 10 burst for rate limiter of k8s.io/client-go
--object-patcher-kube-client-timeout OBJECT_PATCHER_KUBE_CLIENT_TIMEOUT 10s timeout for object patcher's requests to the Kubernetes API server
--jq-library-path JQ_LIBRARY_PATH "" Prepend directory to the search list for jq modules (works as jq -L).
n/a JQ_EXEC "" Set to yes to use jq as executable — it is more for developing purposes.
--log-level LOG_LEVEL "info" Logging level: debug, info, error.
--log-type LOG_TYPE "text" Logging formatter type: json, text or color.
--log-no-time LOG_NO_TIME false Disable timestamp logging if flag is present. Useful when output is redirected to logging system that already adds timestamps.
--log-proxy-hook-json LOG_PROXY_HOOK_JSON false Delegate hook stdout/ stderr JSON logging to the hooks and act as a proxy that adds some extra fields before just printing the output. NOTE: It ignores LOG_TYPE for the output of the hooks; expects JSON lines to stdout/ stderr from the hooks
--debug-keep-tmp-files DEBUG_KEEP_TMP_FILES "no" Set to yes to keep files in $SHELL_OPERATOR_TMP_DIR for debugging purposes. Note that it can generate many files.
--debug-unix-socket DEBUG_UNIX_SOCKET "/var/run/shell-operator/debug.socket" Path to the unix socket file for debugging purposes.
--validating-webhook-configuration-name VALIDATING_WEBHOOK_CONFIGURATION_NAME "shell-operator-hooks" A name of a ValidatingWebhookConfiguration resource.
--validating-webhook-service-name VALIDATING_WEBHOOK_SERVICE_NAME "shell-operator-validating-svc" A name of a service used in ValidatingWebhookConfiguration.
--validating-webhook-server-cert VALIDATING_WEBHOOK_SERVER_CERT "/validating-certs/tls.crt" A path to a server certificate for service used in ValidatingWebhookConfiguration.
--validating-webhook-server-key VALIDATING_WEBHOOK_SERVER_KEY "/validating-certs/tls.key" A path to a server private key for service used in ValidatingWebhookConfiguration.
--validating-webhook-ca VALIDATING_WEBHOOK_CA "/validating-certs/ca.crt" A path to a ca certificate for ValidatingWebhookConfiguration.
--validating-webhook-client-ca VALIDATING_WEBHOOK_CLIENT_CA [] A path to a server certificate for ValidatingWebhookConfiguration.
--conversion-webhook-service-name CONVERSION_WEBHOOK_SERVICE_NAME "shell-operator-conversion-svc" A name of a service for clientConfig in CRD.
--conversion-webhook-server-cert CONVERSION_WEBHOOK_SERVER_CERT "/conversion-certs/tls.crt" A path to a server certificate for clientConfig in CRD.
--conversion-webhook-server-key CONVERSION_WEBHOOK_SERVER_KEY "/conversion-certs/tls.key" A path to a server private key for clientConfig in CRD.
--conversion-webhook-ca CONVERSION_WEBHOOK_CA "/conversion-certs/ca.crt" A path to a ca certificate for clientConfig in CRD.
--conversion-webhook-client-ca CONVERSION_WEBHOOK_CLIENT_CA [] A path to a server certificate for CRD.spec.conversion.webhook.

Notes on JSON log proxying

  • JSON log proxying (see above --log-proxy-hook-json) gives a lot of control to the hooks, which might want to use their own logger or different fields or log level
  • It is incompatible with the other log flags in regard to filtering or configuring logging for the hooks. shell-operator will always expect valid json lines and output them regardless of the other flags
  • The log lines from the hooks will be enhanced with these top-level fields, from shell-operator before being printed: 'hook', 'binding', 'event', 'task', 'queue'
  • Configure hooks to use the msg, time and level fields for consistency with the logs coming from shell-operator. This, however, is not enforced.

Debug

The following tools for debugging and fine-tuning of Shell-operator and hooks are available:

  • Analysis of logs of a Shell-operator’s pod (enter kubectl logs -f po/POD_NAME in terminal),
  • The environment variable can be set to LOG_LEVEL=debug to include the detailed debugging information into logs,
  • You can view the contents of the working queues with cli command from inside a Pod: 
    kubectl exec -ti po/shell-operator /bin/bash
shell-operator queue list