Skip to content

Latest commit

 

History

History
140 lines (124 loc) · 23.4 KB

README.md

File metadata and controls

140 lines (124 loc) · 23.4 KB

Infrastructure monitoring

This module provides EKS cluster monitoring with the following resources:

  • AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
  • Logs with AWS for FluentBit
  • Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
  • Installs FluxCD to perform GitOps sync of a Git Repo to EKS Cluster. We will use this later for creating Grafana Dashboards and AWS datasources to Amazon Managed Grafana.
  • Installs External Secrets Operator to retrieve and Sync the Grafana API keys from AWS SSM Parameter Store.
  • Amazon Managed Grafana Dashboard and data source
  • Alerts and recording rules with AWS Managed Service for Prometheus

This module makes use of the open source kube-prometheus-stack

See examples using this Terraform modules in the Amazon EKS section of this documentation

Requirements

Name Version
terraform >= 1.1.0
aws >= 5.0.0
helm >= 2.4.1
kubectl >= 2.0.3
kubernetes >= 2.10

Providers

Name Version
aws >= 5.0.0
helm >= 2.4.1
kubectl >= 2.0.3

Modules

Name Source Version
external_secrets ./add-ons/external-secrets n/a
fluentbit_logs ./add-ons/aws-for-fluentbit n/a
helm_addon github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon v4.32.1
istio_monitoring ./patterns/istio n/a
java_monitoring ./patterns/java n/a
nginx_monitoring ./patterns/nginx n/a
operator ./add-ons/adot-operator n/a

Resources

Name Type
aws_prometheus_rule_group_namespace.alerting_rules resource
aws_prometheus_rule_group_namespace.recording_rules resource
helm_release.fluxcd resource
helm_release.grafana_operator resource
helm_release.kube_state_metrics resource
helm_release.prometheus_node_exporter resource
kubectl_manifest.adothealth_monitoring_dashboards resource
kubectl_manifest.api_server_dashboards resource
kubectl_manifest.flux_gitrepository resource
kubectl_manifest.flux_kustomization resource
kubectl_manifest.kubeproxy_monitoring_dashboard resource
aws_caller_identity.current data source
aws_eks_cluster.eks_cluster data source
aws_partition.current data source
aws_region.current data source

Inputs

Name Description Type Default Required
adot_loglevel Verbosity level for ADOT collector logs. This accepts (detailed|normal|basic), see https://aws-otel.github.io/docs/components/misc-exporters for mor infos. string "normal" no
adothealth_monitoring_config Config object for ADOT health monitoring 
object({
    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    dashboards = object({
      health = string
    })
  })
 null no
apiserver_monitoring_config Config object for API server monitoring 
object({
    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    dashboards = object({
      basic           = string
      advanced        = string
      troubleshooting = string
    })
  })
 null no
custom_metrics_config Configuration object to enable custom metrics collection 
map(object({
    enableBasicAuth       = bool
    path                  = string
    basicAuthUsername     = string
    basicAuthPassword     = string
    ports                 = string
    droppedSeriesPrefixes = string
  }))
 null no
eks_cluster_id EKS Cluster Id string n/a yes
enable_adotcollector_metrics Enables collection of ADOT collector metrics bool true no
enable_alerting_rules Enables or disables Managed Prometheus alerting rules bool true no
enable_amazon_eks_adot Enables the ADOT Operator on the EKS Cluster bool true no
enable_apiserver_monitoring Enable EKS kube-apiserver monitoring, alerting and dashboards bool true no
enable_cert_manager Allow reusing an existing installation of cert-manager bool true no
enable_custom_metrics Allows additional metrics collection for config elements in the custom_metrics_config config object. Automatic dashboards are not included bool false no
enable_dashboards Enables or disables curated dashboards bool true no
enable_external_secrets Installs External Secrets to EKS Cluster bool true no
enable_fluxcd Enables or disables FluxCD. Disabling this might affect some data in the dashboards bool true no
enable_grafana_operator Deploys Grafana Operator to EKS Cluster bool true no
enable_istio Enable ISTIO workloads monitoring, alerting and default dashboards bool false no
enable_java Enable Java workloads monitoring, alerting and default dashboards bool false no
enable_kube_state_metrics Enables or disables Kube State metrics exporter. Disabling this might affect some data in the dashboards bool true no
enable_logs Using AWS For FluentBit to collect cluster and application logs to Amazon CloudWatch bool true no
enable_nginx Enable NGINX workloads monitoring, alerting and default dashboards bool false no
enable_node_exporter Enables or disables Node exporter. Disabling this might affect some data in the dashboards bool true no
enable_recording_rules Enables or disables Managed Prometheus recording rules bool true no
enable_tracing Enables tracing with OTLP traces receiver to X-Ray bool true no
flux_config FluxCD configuration 
object({
    create_namespace   = bool
    k8s_namespace      = string
    helm_chart_name    = string
    helm_chart_version = string
    helm_release_name  = string
    helm_repo_url      = string
    helm_settings      = map(string)
    helm_values        = map(any)
  })
 
{
  "create_namespace": true,
  "helm_chart_name": "flux2",
  "helm_chart_version": "2.12.2",
  "helm_release_name": "observability-fluxcd-addon",
  "helm_repo_url": "https://fluxcd-community.github.io/helm-charts",
  "helm_settings": {},
  "helm_values": {},
  "k8s_namespace": "flux-system"
}
 no
flux_gitrepository_branch Flux GitRepository Branch string "v0.3.2" no
flux_gitrepository_name Flux GitRepository name string "aws-observability-accelerator" no
flux_gitrepository_url Flux GitRepository URL string "https://github.com/aws-observability/aws-observability-accelerator" no
flux_kustomization_name Flux Kustomization name string "grafana-dashboards-infrastructure" no
flux_kustomization_path Flux Kustomization Path string "./artifacts/grafana-operator-manifests/eks/infrastructure" no
go_config Grafana Operator configuration 
object({
    create_namespace   = bool
    helm_chart         = string
    helm_name          = string
    k8s_namespace      = string
    helm_release_name  = string
    helm_chart_version = string
  })
 
{
  "create_namespace": true,
  "helm_chart": "oci://ghcr.io/grafana-operator/helm-charts/grafana-operator",
  "helm_chart_version": "v5.5.2",
  "helm_name": "grafana-operator",
  "helm_release_name": "grafana-operator",
  "k8s_namespace": "grafana-operator"
}
 no
grafana_api_key Grafana API key for the Amazon Managed Grafana workspace. Required if enable_external_secrets = true string "" no
grafana_cluster_dashboard_url Dashboard URL for Cluster Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/cluster.json" no
grafana_kubelet_dashboard_url Dashboard URL for Kubelet Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json" no
grafana_kubeproxy_dashboard_url Dashboard URL for kube-proxy Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/kube-proxy/kube-proxy.json" no
grafana_namespace_workloads_dashboard_url Dashboard URL for Namespace Workloads Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json" no
grafana_node_exporter_dashboard_url Dashboard URL for Node Exporter Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json" no
grafana_nodes_dashboard_url Dashboard URL for Nodes Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodes.json" no
grafana_url Endpoint URL of Amazon Managed Grafana workspace. Required if enable_grafana_operator = true string "" no
grafana_workloads_dashboard_url Dashboard URL for Workloads Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/workloads.json" no
helm_config Helm Config for Prometheus any {} no
irsa_iam_additional_policies IAM additional policies for IRSA roles list(string) [] no
irsa_iam_permissions_boundary IAM permissions boundary for IRSA roles string null no
irsa_iam_role_name IAM role name for IRSA roles string "" no
irsa_iam_role_path IAM role path for IRSA roles string "/" no
istio_config Configuration object for ISTIO monitoring 
object({
    enable_alerting_rules  = bool
    enable_recording_rules = bool
    enable_dashboards      = bool
    scrape_sample_limit    = number

    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    managed_prometheus_workspace_id = string
    prometheus_metrics_endpoint     = string

    dashboards = object({
      cp          = string
      mesh        = string
      performance = string
      service     = string
    })
  })
 null no
java_config Configuration object for Java/JMX monitoring 
object({
    enable_alerting_rules  = bool
    enable_recording_rules = bool
    enable_dashboards      = bool
    scrape_sample_limit    = number


    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    grafana_dashboard_url = string

    prometheus_metrics_endpoint = string
  })
 null no
ksm_config Kube State metrics configuration 
object({
    create_namespace   = bool
    k8s_namespace      = string
    helm_chart_name    = string
    helm_chart_version = string
    helm_release_name  = string
    helm_repo_url      = string
    helm_settings      = map(string)
    helm_values        = map(any)

    scrape_interval = string
    scrape_timeout  = string
  })
 
{
  "create_namespace": true,
  "helm_chart_name": "kube-state-metrics",
  "helm_chart_version": "5.15.2",
  "helm_release_name": "kube-state-metrics",
  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",
  "helm_settings": {},
  "helm_values": {},
  "k8s_namespace": "kube-system",
  "scrape_interval": "60s",
  "scrape_timeout": "15s"
}
 no
kubeproxy_monitoring_config Config object for kube-proxy monitoring 
object({
    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    dashboards = object({
      default = string
    })
  })
 null no
logs_config Configuration object for logs collection 
object({
    cw_log_retention_days = number
  })
 
{
  "cw_log_retention_days": 90
}
 no
managed_prometheus_cross_account_role Amazon Managed Prometheus Workspace's Account Role Arn string "" no
managed_prometheus_workspace_endpoint Amazon Managed Prometheus Workspace Endpoint string "" no
managed_prometheus_workspace_id Amazon Managed Prometheus Workspace ID string null no
managed_prometheus_workspace_region Amazon Managed Prometheus Workspace's Region string null no
ne_config Node exporter configuration 
object({
    create_namespace   = bool
    k8s_namespace      = string
    helm_chart_name    = string
    helm_chart_version = string
    helm_release_name  = string
    helm_repo_url      = string
    helm_settings      = map(string)
    helm_values        = map(any)

    scrape_interval = string
    scrape_timeout  = string
  })
 
{
  "create_namespace": true,
  "helm_chart_name": "prometheus-node-exporter",
  "helm_chart_version": "4.24.0",
  "helm_release_name": "prometheus-node-exporter",
  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",
  "helm_settings": {},
  "helm_values": {},
  "k8s_namespace": "prometheus-node-exporter",
  "scrape_interval": "60s",
  "scrape_timeout": "60s"
}
 no
nginx_config Configuration object for NGINX monitoring 
object({
    enable_alerting_rules  = bool
    enable_recording_rules = bool
    enable_dashboards      = bool
    scrape_sample_limit    = number

    flux_gitrepository_name   = string
    flux_gitrepository_url    = string
    flux_gitrepository_branch = string
    flux_kustomization_name   = string
    flux_kustomization_path   = string

    grafana_dashboard_url = string

    prometheus_metrics_endpoint = string
  })
 null no
prometheus_config Controls default values such as scrape interval, timeouts and ports globally 
object({
    global_scrape_interval = string
    global_scrape_timeout  = string
  })
 
{
  "global_scrape_interval": "120s",
  "global_scrape_timeout": "15s"
}
 no
tags Additional tags (e.g. map('BusinessUnit,XYZ) map(string) {} no
target_secret_name Target secret in Kubernetes to store the Grafana API Key Secret string "grafana-admin-credentials" no
target_secret_namespace Target namespace of secret in Kubernetes to store the Grafana API Key Secret string "grafana-operator" no
tracing_config Configuration object for traces collection to AWS X-Ray 
object({
    otlp_grpc_endpoint = string
    otlp_http_endpoint = string
    send_batch_size    = number
    timeout            = string
  })
 
{
  "otlp_grpc_endpoint": "0.0.0.0:4317",
  "otlp_http_endpoint": "0.0.0.0:4318",
  "send_batch_size": 50,
  "timeout": "30s"
}
 no

Outputs

Name Description
adot_irsa_arn IRSA Arn for ADOT
eks_cluster_id EKS Cluster Id
eks_cluster_version EKS Cluster version