From eb77ba6449a53d78ceb49f589b184e18fe8259aa Mon Sep 17 00:00:00 2001 From: Jonathan Funk Date: Mon, 20 Nov 2023 07:56:06 +0000 Subject: [PATCH] tweak redis and certmanager issues --- codebundles/k8s-certmanager-healthcheck/README.md | 1 + codebundles/k8s-certmanager-healthcheck/runbook.robot | 6 ++++-- codebundles/k8s-redis-healthcheck/runbook.robot | 7 +++++-- 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/codebundles/k8s-certmanager-healthcheck/README.md b/codebundles/k8s-certmanager-healthcheck/README.md index ed9ec37d..3dd41c86 100644 --- a/codebundles/k8s-certmanager-healthcheck/README.md +++ b/codebundles/k8s-certmanager-healthcheck/README.md @@ -20,3 +20,4 @@ The TaskSet requires initialization to import necessary secrets, services, and u ## TODO - [ ] Add additional documentation. +- [ ] Add cert-manager operator namespace as config field and genrules \ No newline at end of file diff --git a/codebundles/k8s-certmanager-healthcheck/runbook.robot b/codebundles/k8s-certmanager-healthcheck/runbook.robot index 11ef3ab7..46512d8e 100644 --- a/codebundles/k8s-certmanager-healthcheck/runbook.robot +++ b/codebundles/k8s-certmanager-healthcheck/runbook.robot @@ -16,7 +16,7 @@ Suite Setup Suite Initialization *** Tasks *** -Get Namespace Certificate Summary +Get Namespace `${NAMESPACE}` Certificate Summary [Documentation] Gets a list of certmanager certificates and summarize their information for review. [Tags] tls certificates kubernetes objects expiration summary certmanager ${cert_info}= RW.CLI.Run Cli @@ -31,12 +31,13 @@ Get Namespace Certificate Summary ... set_issue_actual=Certificates were found in the namespace ${NAMESPACE} that are past their renewal time and not renewed ... set_issue_title=Found certificates due for renewal in namespace ${NAMESPACE} that are not renewing ... set_issue_details=CertManager certificates not renewing: "$_stdout" - investigate CertManager. + ... set_issue_next_steps=Check the health of the `cert-manager` deployment in the `cert-manager` namespace\nInspect `cert-manager` deployment logs for renewal errors in `cert-manager` namespace ... _line__raise_issue_if_contains=Namespace RW.Core.Add Pre To Report Certificate Information:\n${cert_info.stdout} ${history}= RW.CLI.Pop Shell History RW.Core.Add Pre To Report Commands Used: ${history} -Find Failed Certificate Requests and Identify Issues +Find Failed Certificate Requests and Identify Issues In Namespace `${NAMESPACE}` [Documentation] Gets a list of failed certmanager certificates and summarize their issues. [Tags] tls certificates kubernetes objects failed certificaterequest certmanager ${failed_certificaterequests}= RW.CLI.Run Cli @@ -51,6 +52,7 @@ Find Failed Certificate Requests and Identify Issues ... set_issue_actual=Certificates are not ready in ${NAMESPACE} ... set_issue_title=Found failed certificates in namespace ${NAMESPACE} ... set_issue_details=CertManager certificates failed: "$_stdout" - investigate Issuers or ClusterIssuers. + ... set_issue_next_steps=Check the health of the `cert-manager` deployment in `cert-manager` namespace\nInspect `cert-manager` deployment logs for renewal errors in `cert-manager` namespace ... _line__raise_issue_if_contains=- RW.Core.Add Pre To Report Certificate Information:\n${failed_certificaterequests.stdout} ${history}= RW.CLI.Pop Shell History diff --git a/codebundles/k8s-redis-healthcheck/runbook.robot b/codebundles/k8s-redis-healthcheck/runbook.robot index 06feae9a..6ebb52b2 100644 --- a/codebundles/k8s-redis-healthcheck/runbook.robot +++ b/codebundles/k8s-redis-healthcheck/runbook.robot @@ -14,8 +14,9 @@ Library Collections Suite Setup Suite Initialization + *** Tasks *** -Ping Redis Workload +Ping `${DEPLOYMENT_NAME}` Redis Workload [Documentation] Verifies that a PING can be peformed against the redis workload. [Tags] redis cli ping pong alive probe ready ${rsp}= RW.CLI.Run Cli @@ -30,12 +31,13 @@ Ping Redis Workload ... set_issue_actual=The Redis workload was unable to properly repond to the PING request ... set_issue_title=Redis PING Failed In Namespace ${NAMESPACE} For Redis Deployment ${DEPLOYMENT_NAME} ... set_issue_details=Found $_line in namespace ${NAMESPACE}\nCheck if the redis workload is healthy and available. Attempt to run a 'redis-cli PING' directly on the workload and verify the response which should be PONG. + ... set_issue_next_steps=Check PVC health status in namespace ${NAMESPACE} ... _line__raise_issue_if_ncontains=PONG RW.Core.Add Pre To Report Redis Response:\n${rsp.stdout} ${history}= RW.CLI.Pop Shell History RW.Core.Add Pre To Report Commands Used: ${history} -Verify Redis Read Write Operation +Verify `${DEPLOYMENT_NAME}` Redis Read Write Operation [Documentation] Attempts to perform a write and read operation on the redis workload, checking that a key can be set, incremented, and read from. [Tags] redis cli increment health check read write ${set_op}= RW.CLI.Run Cli @@ -60,6 +62,7 @@ Verify Redis Read Write Operation ... set_issue_actual=The redis workload failed to increment the key as expected. ... set_issue_title=Redis Read Write Operation Failure In Namespace ${NAMESPACE} For Redis Deployment ${DEPLOYMENT_NAME} ... set_issue_details=Found $_line in namespace ${NAMESPACE}\nCheck the PVC that the redis workload depends on and verify it's healthy. Try use 'redis-cli INCR ${REDIS_HEALTHCHECK_KEY}' yourself on the workload. + ... set_issue_next_steps=Check PVC health status in namespace ${NAMESPACE} ... _line__raise_issue_if_neq=1 RW.Core.Add Pre To Report Redis Response For Key ${REDIS_HEALTHCHECK_KEY}:${get_op.stdout} ${history}= RW.CLI.Pop Shell History