Skip to content

studyproject: Prometheus & Grafana to observe an application

Notifications You must be signed in to change notification settings

DamarisLa/clc3-prometheusgrafana

Repository files navigation

Google Cloud with Services already running

Use PowerShell, so you don't need to copy paste the names of the pods.

  1. Log in from your command line with

    gcloud auth login
  2. Connect: at https://console.cloud.google.com/kubernetes click on the 3 dots and select connect, copy the command-line statement and run it in your PowerShell

  3. Check if the node is running

    kubectl get nodes
  4. Check if pods are running with

    kubectl get pods --namespace monitoring
  5. Forward the port of prometheus-server

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=prometheus-server" --output jsonpath='{.items[0].metadata.name}') 8080:9090

    and open localhost:8080

  6. Open a new PowerShell and forward the port of alertmanager

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=alertmanager" --output jsonpath='{.items[0].metadata.name}') 8081:9093

    and open localhost:8081

  7. Open a new PowerShell and forward the port of grafana

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=grafana" --output jsonpath='{.items[0].metadata.name}') 8082:3000

    and open localhost:8082

  8. Login with
    Username: admin
    Password: admin
    Skip setting new password

  9. Open a new PowerShell and forward the port of Application1

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=application1" --output jsonpath='{.items[0].metadata.name}') 8083:8000

    and open for example localhost:8083/delay/2 for a 2 seconds delay

  10. Open a new PowerShell and forward the port of Application2

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=application2" --output jsonpath='{.items[0].metadata.name}') 8084:8001

    and open localhost:8084/ -> the Service should be Unavailable

  11. Open a new PowerShell and open CLI for Redis

    kubectl exec -i -t $(kubectl get pod --namespace monitoring --selector="app=redis" --output jsonpath='{.items[0].metadata.name}') -n monitoring --container redis -- redis-cli
  12. Open a new PowerShell and forward the port of Redis

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=redis" --output jsonpath='{.items[0].metadata.name}') 8085:9121

    and open localhost:8085/metrics

Overview of the ports (localhost:xxxx):

  • 8080 -> prometheus-server
  • 8081 -> alertmanager
  • 8082 -> grafana
  • 8083 -> Application1 (= Sleep)
  • 8084 -> Application2 (= Not available)
  • 8085 -> Redis

First setup with Google Cloud

Prerequisites

  1. Account in Google Cloud and start a trial version
  2. Install Google Cloud SDK Run the command:
    gcloud version
  3. If not done before: run the kubectl installation command
    gcloud components intall kubectl

Setup Cluster

  1. Log in from your command line with

    gcloud auth login
  2. Open the Kubernetes Engine Overview

  3. Click on Create Cluster and select "Standard: you manage your cluster"

  4. Create a cluster with the default settings except for the Number of nodes where you only need 1:

    K8s Cluster in GKE K8s Cluster in GKE K8s Cluster in GKE

    Standard machine type is Nodes

    Changed the Machine type to e2-standard-4 so CPU should be no problem in future Nodes

  5. As soon as your cluster is ready, click on Connect and copy and paste this command into your terminal

  6. Now your kubectl (i.e., the Kubernetes command-line tool) should be configured for your cluster. In order to verify this, execute the command:

    kubectl get nodes
  7. Change to the directory in the git folder and run the commands for Setup Prometheus from the setup_prometheus_example.md

  8. Check if it is running with

    kubectl get pods --namespace monitoring
  9. Forward the port

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=prometheus-server" --output jsonpath='{.items[0].metadata.name}') 8080:9090
  10. Now open the link localhost:8080 in your browser.

  11. Continue the setup_prometheus_example.md with Setup State Metrics and Setup Alert manager

  12. Open a new PowerShell and forward the port of alertmanager

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=alertmanager" --output jsonpath='{.items[0].metadata.name}') 8081:9093

    and open localhost:8081

  13. Continue the setup_prometheus_example.md with Setup Grafana. For setup the Grafana Dashboards see grafana_dashboard.md

  14. Open a new PowerShell and forward the port of grafana

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=grafana" --output jsonpath='{.items[0].metadata.name}') 8082:3000

    and open localhost:8082

  15. Login with
    Username: admin
    Password: admin
    Skip setting new password

  16. Continue the setup_prometheus_example.md with Setup Node Exporter and Setup Application1 (Sleep)

  17. Open a new PowerShell and forward the port of Application1

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=application1" --output jsonpath='{.items[0].metadata.name}') 8083:8000

    and open for example localhost:8083/delay/2 for a 2 seconds delay

  18. Continue the setup_prometheus_example.md with Setup Application2 (Not available)

  19. Open a new PowerShell and forward the port of Application2

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=application2" --output jsonpath='{.items[0].metadata.name}') 8084:8001

    and open localhost:8084/ -> the Service should be Unavailable

  20. Continue the setup_prometheus_example.md with Setup Redis

  21. Open a new PowerShell and forward the port of Redis

    kubectl port-forward --namespace monitoring $(kubectl get pod --namespace monitoring --selector="app=redis" --output jsonpath='{.items[0].metadata.name}') 8085:9121

    and open localhost:8085/metrics


For updating a Deployment.yaml run for example

kubectl edit deployment alertmanager --namespace monitoring

For deleting (= restarting) -> change [POD-NAME]

kubectl delete pod $(kubectl get pod --namespace monitoring --selector="app=[POD-NAME]" --output jsonpath='{.items[0].metadata.name}') -n monitoring

About

studyproject: Prometheus & Grafana to observe an application

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published