Migrate new Jenkins and QA tools to the new QA repository

[QU3B1M]

Main issue
#5556

Description

Migrate the new Jenkins code developed for https://github.com/wazuh/wazuh-jenkins/issues/5927 and the new QA tools made on #4495 and on https://github.com/wazuh/wazuh-jenkins/issues/6359 to the new wazuh-qa repository respecting the defined structure: #5475 (comment)

• Main issue PR: https://github.com/wazuh/wazuh-qa-automation/pull/20

Tasks

• Migrate JobFlow (ex Workflow engine) tool - https://github.com/wazuh/wazuh-qa-automation/pull/9
• Migrate Provision module - https://github.com/wazuh/wazuh-qa-automation/pull/10
• Migrate release procedures - https://github.com/wazuh/wazuh-qa-automation/pull/15
• Migrate new Jenkins's code - https://github.com/wazuh/wazuh-qa-automation/pull/16
• Migrate Tests module and DTT workflows - https://github.com/wazuh/wazuh-qa-automation/pull/17
  • CSV with the migrated DTT coverage: Coverage.xlsx
• Identify and list the possible improvements to each migrated module - Migrate new Jenkins and QA tools to the new QA repository #5557 (comment)

Important

Once all the tasks' PRs are approved and merged, review and merge the main migrations branch PR: https://github.com/wazuh/wazuh-qa-automation/pull/20

[QU3B1M]

Update report

Last Update: 05/08/2024 - 09:38p.m.

• Migrate JobFlow's code to new QA repository - https://github.com/wazuh/wazuh-qa-automation/pull/9
• Migrate Provision from wazuh-qa to new QA repository - https://github.com/wazuh/wazuh-qa-automation/pull/10
  • Error found on JobFlow and Provision setup.py, the installation works but not as it should, probably the packaging method must be updated to correctly install these packages. Solved
  • Hotfix JobFlow replacing setup.py with pyproject.toml
• Migrate New Jenkins to new QA Repository - https://github.com/wazuh/wazuh-qa-automation/pull/13
  • A bug was found, analyzing the cause and a possible fix
  • Commit history must be reviewed and fixed (there are extra commits)
• Migrate release procedures - https://github.com/wazuh/wazuh-qa-automation/pull/15
• Migrate the DTT test module and workflows - (WIP)

[QU3B1M]

Improvements detected

While working on the migration, some improvements were detected and referenced in this comment.

Release Procedures

Medium priority improvements:

• tools/release/src/**/docker/Dockerfile_tests: Each module's docker test image.
  We should re-evaluate if it's necessary to run each module's unit tests on a docker image, noting that it consumes time and may not add much value to the development itself.
• .github/workflows: Main GHA workflows directory
  • Currently, all the Release Procedures modules' Unit Tests are being executed on GHA and running on Docker, which means that we build a Docker image and run it on each GHA, and it is resources-consuming.
  • All the GHA Workflows can be simplified in just one workflows that runs all the UTs (but first, it would be great if we get rid of the Docker build)

---

Testing

High-priority improvements:

• tools/testing/src/testing/testing.py: Main testing module file
  • The module's parameters are strongly attached to DTT, it must be more generic to re-use it on different suites.
  • It expects the target and dependencies inventories to be on a specific path with a desired format, so it won't detect the target system correctly if the inventory is in a custom directory.
• tools/testing/src/testing/playbooks/test.yml Test execution playbook
  The test path is generated "dynamically" but it depends completely on the DTT format
• tests/test_functional/test_system/test_deployability/workflows: DTT workflows directory
  • The workflows are outdated, and its using the legacy allocator path, it must be updated to use them.
  • Wazuh version on the Workflows is hardcoded using an outdated value.
  • Add parametrization on the Workflows where possible.
• tools/testing/tests: Testing tool Unit Tests directory
  There are no Unit Tests developed, we must create a set of tests to ensure its correct functionality
  Related issues:
  • DTT2 - Iteration 3 - Unit test - Unit tests for each module #4993

Medium priority improvements:

• tests/test_functional/test_system/test_deployability/tests: Main DTT tests directory
  The test helpers have a lot of duplicated code, and there are several wrongly handled conditional flows. We could include unit tests for the helpers to validate its correct behavior

---

JobFlow

Medium priority improvements:

• tools/jobflow/README.md: Main documentation file of the JobFlow tool
  Currently, the documentation is DTT1-specific, it does not help the user to understand what the tool does, and how it can be used for different cases.

Low priority improvements:

• tools/jobflow/tests: Unit Tests suite
  This suite uses Python's default test runner UnitTests utilities and pytest as the runner, it could be improved by only using pytest and its utilities, which is recommended.
  Related issues:
  • DTT2 - Iteration 3 - Unit test - Unit tests for each module #4993
• tools/jobflow/examples: Workflows examples directory
  We must add more Workflows as examples of different use cases.

Other improvements could be found here: #5044

---

Provision

Medium priority improvements:

• tools/provision/README.md: Main documentation file of the Provision tool
  This README is strongly related to the JobFlow, it could be improved detailing mainly the standalone execution and having the JobFlow workflow as a secondary use.

Low-priority improvements:

• tools/provision/src/provision/playbooks/wazuh: Wazuh-related playbooks
  This directory could be re-structured, probably the non install/uninstall actions (register, services, set repo) could be separated on a different folder as these actions are not specifically related to an installation type.
• tools/provision/tests: Unit Tests suite
  This suite uses Python's default test runner UnitTests utilities and pytest as the runner, it could be improved by only using pytest and its utilities, which is recommended.
  Related issues:
  • DTT2 - Iteration 3 - Unit test - Unit tests for each module #4993

---

Jenkins

Low-priority improvements:

• tools/jenkins/tests: tool's Unit Tests directory
  Must implement more Unit Tests for this tool
• tools/jenkins/src/job-builder: Jenkins pipelines as a code
  Add a README.md detailing this folder, it could be implemented here or in the main Jenkins src directory (tools/jenkins/src), in the second case, it should also includes a summary about bootstrap

---

[damarisg]

GJ!