Skip to content

Latest commit

 

History

History
 
 

Infrastructure monitoring

This module provides EKS cluster monitoring with the following resources:

  • AWS Distro For OpenTelemetry Operator and Collector for Metrics and Traces
  • Logs with AWS for FluentBit
  • Installs Grafana Operator to add AWS data sources and create Grafana Dashboards to Amazon Managed Grafana.
  • Installs FluxCD to perform GitOps sync of a Git Repo to EKS Cluster. We will use this later for creating Grafana Dashboards and AWS datasources to Amazon Managed Grafana.
  • Installs External Secrets Operator to retrieve and Sync the Grafana API keys from AWS SSM Parameter Store.
  • Amazon Managed Grafana Dashboard and data source
  • Alerts and recording rules with AWS Managed Service for Prometheus

This module makes use of the open source kube-prometheus-stack

See examples using this Terraform modules in the Amazon EKS section of this documentation

Requirements

Name Version
terraform >= 1.1.0
aws >= 5.0.0
helm >= 2.4.1
kubectl >= 2.0.3
kubernetes >= 2.10

Providers

Name Version
aws >= 5.0.0
helm >= 2.4.1
kubectl >= 2.0.3

Modules

Name Source Version
external_secrets ./add-ons/external-secrets n/a
fluentbit_logs ./add-ons/aws-for-fluentbit n/a
helm_addon github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon v4.32.1
istio_monitoring ./patterns/istio n/a
java_monitoring ./patterns/java n/a
nginx_monitoring ./patterns/nginx n/a
operator ./add-ons/adot-operator n/a

Resources

Name Type
aws_prometheus_rule_group_namespace.alerting_rules resource
aws_prometheus_rule_group_namespace.recording_rules resource
helm_release.fluxcd resource
helm_release.grafana_operator resource
helm_release.kube_state_metrics resource
helm_release.prometheus_node_exporter resource
kubectl_manifest.adothealth_monitoring_dashboards resource
kubectl_manifest.api_server_dashboards resource
kubectl_manifest.flux_gitrepository resource
kubectl_manifest.flux_kustomization resource
kubectl_manifest.kubeproxy_monitoring_dashboard resource
aws_caller_identity.current data source
aws_eks_cluster.eks_cluster data source
aws_partition.current data source
aws_region.current data source

Inputs

Name Description Type Default Required
adot_loglevel Verbosity level for ADOT collector logs. This accepts (detailed|normal|basic), see https://aws-otel.github.io/docs/components/misc-exporters for mor infos. string "normal" no
adothealth_monitoring_config Config object for ADOT health monitoring
object({
flux_gitrepository_name = string
flux_gitrepository_url = string
flux_gitrepository_branch = string
flux_kustomization_name = string
flux_kustomization_path = string

dashboards = object({
health = string
})
})
null no
apiserver_monitoring_config Config object for API server monitoring
object({
flux_gitrepository_name = string
flux_gitrepository_url = string
flux_gitrepository_branch = string
flux_kustomization_name = string
flux_kustomization_path = string

dashboards = object({
basic = string
advanced = string
troubleshooting = string
})
})
null no
custom_metrics_config Configuration object to enable custom metrics collection
map(object({
enableBasicAuth = bool
path = string
basicAuthUsername = string
basicAuthPassword = string
ports = string
droppedSeriesPrefixes = string
}))
null no
eks_cluster_id EKS Cluster Id string n/a yes
enable_adotcollector_metrics Enables collection of ADOT collector metrics bool true no
enable_alerting_rules Enables or disables Managed Prometheus alerting rules bool true no
enable_amazon_eks_adot Enables the ADOT Operator on the EKS Cluster bool true no
enable_apiserver_monitoring Enable EKS kube-apiserver monitoring, alerting and dashboards bool true no
enable_cert_manager Allow reusing an existing installation of cert-manager bool true no
enable_custom_metrics Allows additional metrics collection for config elements in the custom_metrics_config config object. Automatic dashboards are not included bool false no
enable_dashboards Enables or disables curated dashboards bool true no
enable_external_secrets Installs External Secrets to EKS Cluster bool true no
enable_fluxcd Enables or disables FluxCD. Disabling this might affect some data in the dashboards bool true no
enable_grafana_operator Deploys Grafana Operator to EKS Cluster bool true no
enable_istio Enable ISTIO workloads monitoring, alerting and default dashboards bool false no
enable_java Enable Java workloads monitoring, alerting and default dashboards bool false no
enable_kube_state_metrics Enables or disables Kube State metrics exporter. Disabling this might affect some data in the dashboards bool true no
enable_logs Using AWS For FluentBit to collect cluster and application logs to Amazon CloudWatch bool true no
enable_nginx Enable NGINX workloads monitoring, alerting and default dashboards bool false no
enable_node_exporter Enables or disables Node exporter. Disabling this might affect some data in the dashboards bool true no
enable_recording_rules Enables or disables Managed Prometheus recording rules bool true no
enable_tracing Enables tracing with OTLP traces receiver to X-Ray bool true no
flux_config FluxCD configuration
object({
create_namespace = bool
k8s_namespace = string
helm_chart_name = string
helm_chart_version = string
helm_release_name = string
helm_repo_url = string
helm_settings = map(string)
helm_values = map(any)
})
{
"create_namespace": true,
"helm_chart_name": "flux2",
"helm_chart_version": "2.12.2",
"helm_release_name": "observability-fluxcd-addon",
"helm_repo_url": "https://fluxcd-community.github.io/helm-charts",
"helm_settings": {},
"helm_values": {},
"k8s_namespace": "flux-system"
}
no
flux_gitrepository_branch Flux GitRepository Branch string "v0.3.2" no
flux_gitrepository_name Flux GitRepository name string "aws-observability-accelerator" no
flux_gitrepository_url Flux GitRepository URL string "https://github.com/aws-observability/aws-observability-accelerator" no
flux_kustomization_name Flux Kustomization name string "grafana-dashboards-infrastructure" no
flux_kustomization_path Flux Kustomization Path string "./artifacts/grafana-operator-manifests/eks/infrastructure" no
go_config Grafana Operator configuration
object({
create_namespace = bool
helm_chart = string
helm_name = string
k8s_namespace = string
helm_release_name = string
helm_chart_version = string
})
{
"create_namespace": true,
"helm_chart": "oci://ghcr.io/grafana-operator/helm-charts/grafana-operator",
"helm_chart_version": "v5.5.2",
"helm_name": "grafana-operator",
"helm_release_name": "grafana-operator",
"k8s_namespace": "grafana-operator"
}
no
grafana_api_key Grafana API key for the Amazon Managed Grafana workspace. Required if enable_external_secrets = true string "" no
grafana_cluster_dashboard_url Dashboard URL for Cluster Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/cluster.json" no
grafana_kubelet_dashboard_url Dashboard URL for Kubelet Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json" no
grafana_kubeproxy_dashboard_url Dashboard URL for kube-proxy Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/kube-proxy/kube-proxy.json" no
grafana_namespace_workloads_dashboard_url Dashboard URL for Namespace Workloads Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json" no
grafana_node_exporter_dashboard_url Dashboard URL for Node Exporter Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json" no
grafana_nodes_dashboard_url Dashboard URL for Nodes Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/nodes.json" no
grafana_url Endpoint URL of Amazon Managed Grafana workspace. Required if enable_grafana_operator = true string "" no
grafana_workloads_dashboard_url Dashboard URL for Workloads Grafana Dashboard JSON string "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/v0.2.0/artifacts/grafana-dashboards/eks/infrastructure/workloads.json" no
helm_config Helm Config for Prometheus any {} no
irsa_iam_additional_policies IAM additional policies for IRSA roles list(string) [] no
irsa_iam_permissions_boundary IAM permissions boundary for IRSA roles string null no
irsa_iam_role_name IAM role name for IRSA roles string "" no
irsa_iam_role_path IAM role path for IRSA roles string "/" no
istio_config Configuration object for ISTIO monitoring
object({
enable_alerting_rules = bool
enable_recording_rules = bool
enable_dashboards = bool
scrape_sample_limit = number

flux_gitrepository_name = string
flux_gitrepository_url = string
flux_gitrepository_branch = string
flux_kustomization_name = string
flux_kustomization_path = string

managed_prometheus_workspace_id = string
prometheus_metrics_endpoint = string

dashboards = object({
cp = string
mesh = string
performance = string
service = string
})
})
null no
java_config Configuration object for Java/JMX monitoring
object({
enable_alerting_rules = bool
enable_recording_rules = bool
enable_dashboards = bool
scrape_sample_limit = number


flux_gitrepository_name = string
flux_gitrepository_url = string
flux_gitrepository_branch = string
flux_kustomization_name = string
flux_kustomization_path = string

grafana_dashboard_url = string

prometheus_metrics_endpoint = string
})
null no
ksm_config Kube State metrics configuration
object({
create_namespace = bool
k8s_namespace = string
helm_chart_name = string
helm_chart_version = string
helm_release_name = string
helm_repo_url = string
helm_settings = map(string)
helm_values = map(any)

scrape_interval = string
scrape_timeout = string
})
{
"create_namespace": true,
"helm_chart_name": "kube-state-metrics",
"helm_chart_version": "5.15.2",
"helm_release_name": "kube-state-metrics",
"helm_repo_url": "https://prometheus-community.github.io/helm-charts",
"helm_settings": {},
"helm_values": {},
"k8s_namespace": "kube-system",
"scrape_interval": "60s",
"scrape_timeout": "15s"
}
no
kubeproxy_monitoring_config Config object for kube-proxy monitoring
object({
flux_gitrepository_name = string
flux_gitrepository_url = string
flux_gitrepository_branch = string
flux_kustomization_name = string
flux_kustomization_path = string

dashboards = object({
default = string
})
})
null no
logs_config Configuration object for logs collection
object({
cw_log_retention_days = number
})
{
"cw_log_retention_days": 90
}
no
managed_prometheus_cross_account_role Amazon Managed Prometheus Workspace's Account Role Arn string "" no
managed_prometheus_workspace_endpoint Amazon Managed Prometheus Workspace Endpoint string "" no
managed_prometheus_workspace_id Amazon Managed Prometheus Workspace ID string null no
managed_prometheus_workspace_region Amazon Managed Prometheus Workspace's Region string null no
ne_config Node exporter configuration
object({
create_namespace = bool
k8s_namespace = string
helm_chart_name = string
helm_chart_version = string
helm_release_name = string
helm_repo_url = string
helm_settings = map(string)
helm_values = map(any)

scrape_interval = string
scrape_timeout = string
})
{
"create_namespace": true,
"helm_chart_name": "prometheus-node-exporter",
"helm_chart_version": "4.24.0",
"helm_release_name": "prometheus-node-exporter",
"helm_repo_url": "https://prometheus-community.github.io/helm-charts",
"helm_settings": {},
"helm_values": {},
"k8s_namespace": "prometheus-node-exporter",
"scrape_interval": "60s",
"scrape_timeout": "60s"
}
no
nginx_config Configuration object for NGINX monitoring
object({
enable_alerting_rules = bool
enable_recording_rules = bool
enable_dashboards = bool
scrape_sample_limit = number

flux_gitrepository_name = string
flux_gitrepository_url = string
flux_gitrepository_branch = string
flux_kustomization_name = string
flux_kustomization_path = string

grafana_dashboard_url = string

prometheus_metrics_endpoint = string
})
null no
prometheus_config Controls default values such as scrape interval, timeouts and ports globally
object({
global_scrape_interval = string
global_scrape_timeout = string
})
{
"global_scrape_interval": "120s",
"global_scrape_timeout": "15s"
}
no
tags Additional tags (e.g. map('BusinessUnit,XYZ) map(string) {} no
target_secret_name Target secret in Kubernetes to store the Grafana API Key Secret string "grafana-admin-credentials" no
target_secret_namespace Target namespace of secret in Kubernetes to store the Grafana API Key Secret string "grafana-operator" no
tracing_config Configuration object for traces collection to AWS X-Ray
object({
otlp_grpc_endpoint = string
otlp_http_endpoint = string
send_batch_size = number
timeout = string
})
{
"otlp_grpc_endpoint": "0.0.0.0:4317",
"otlp_http_endpoint": "0.0.0.0:4318",
"send_batch_size": 50,
"timeout": "30s"
}
no

Outputs

Name Description
adot_irsa_arn IRSA Arn for ADOT
eks_cluster_id EKS Cluster Id
eks_cluster_version EKS Cluster version