refs sparkfabrik-innovation-team/board#3029: Create application error log group error

[FabrizioCafolla]

PR Type

Enhancement

---

Description

• Introduced a new log group for application errors
• Added configuration to capture and filter application error logs
• Implemented filtering for logs with status codes 4xx and 5xx
• Set up CloudWatch Logs output for the new application-errors log group
• Updated existing filters to include the new application-errors tag
• Maintained consistency with existing log processing patterns

---

Changes walkthrough 📝

	Relevant files
Enhancement	values.yml.tftpl Add application errors logging configuration                          files/values.yml.tftpl Added new INPUT section for application-errors log group Updated FILTER sections to include application-errors Added new FILTER for retrieving logs with status codes 4xx and 5xx Added new OUTPUT section for application-errors to CloudWatch Logs +38/-4

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[sparkfabrik-ai-bot]

PR Reviewer Guide 🔍

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Key issues to review Performance Concern The new input configuration for application errors might lead to duplicate log processing, as it's reading from the same path as the existing application logs input. Potential Oversight The grep filter for application errors is using a regex that might not catch all error scenarios, as it's only checking the status code in the log_processed.level field.

[sparkfabrik-ai-bot]

PR Code Suggestions ✨

Category	Suggestion	Score
Enhancement	Expand the regex pattern to include more error and warning log levels The regex pattern for filtering error logs might be too restrictive. Consider including all log levels that indicate errors or warnings. files/values.yml.tftpl [175-178] [FILTER] Name grep Match application-errors.* - Regex log_processed.level [45][0-9]{2} + Regex log_processed.level (ERROR|WARN|[45][0-9]{2}) Suggestion importance[1-10]: 8 Why: This suggestion addresses a potential oversight in error logging, which could lead to missing important log entries. It's a valuable improvement for monitoring and debugging.	8
Maintainability	Extract common configuration options into a separate include file to reduce duplication Consider using a variable for the common configuration options in the [INPUT] sections to reduce duplication and improve maintainability. files/values.yml.tftpl [76-91] [INPUT] - Name tail + @INCLUDE input_common_config Tag application-errors.* Exclude_Path ${join(", ", formatlist("/var/log/containers/%s*.log", distinct(additional_exclude_from_application_log_group)))} Path /var/log/containers/*.log - Docker_Mode On - Docker_Mode_Flush 5 - Docker_Mode_Parser custom_json - Parser custom_json - DB /var/fluent-bit/state/flb_container.db - Mem_Buf_Limit 50MB - Skip_Long_Lines On - Refresh_Interval 10 - Rotate_Wait 30 - storage.type filesystem - Read_from_Head $${READ_FROM_HEAD} Suggestion importance[1-10]: 7 Why: The suggestion correctly identifies code duplication and proposes a valid solution to improve maintainability, but it's not addressing a critical issue.	7
Performance	Add buffer configuration to the CloudWatch Logs output for improved performance and reliability Consider adding a buffer configuration for the CloudWatch Logs output to improve performance and reliability. files/values.yml.tftpl [196-205] [OUTPUT] Name cloudwatch_logs Match application-errors.* region $${AWS_REGION} log_group_name /aws/containerinsights/$${CLUSTER_NAME}/application-errors log_stream_prefix $${HOST_NAME}- log_format json auto_create_group true log_retention_days ${application_log_retention_days} #extra_user_agent container-insights + buffer_size 5M + queue_size 10000 + retry_limit 5 Suggestion importance[1-10]: 6 Why: The suggestion proposes a valid performance improvement, but it's not addressing a critical issue. The benefit depends on the specific use case and load.	6