Merge branch 'master' of github.com:wazuh/wazuh-qa into enhancement/3…

…199-enable-windows-vd-e2e
wazuh · Oct 6, 2023 · e94b796 · e94b796
2 parents e851e62 + 23bbc80
commit e94b796
Show file tree

Hide file tree

Showing 172 changed files with 4,413 additions and 86,265 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,10 +7,28 @@ All notable changes to this project will be documented in this file.
 Wazuh commit: TBD \
 Release report: TBD
 
+### Added
+
+- Add system test for global group hash ([#4015](https://github.com/wazuh/wazuh-qa/pull/4015)) \- (Tests)
+- Add tests for new FIM audit buffer option. ([#4485](https://githubhttps://github.com/wazuh/wazuh-qa/pull/4497#pullrequestreview-1654748331.com/wazuh/wazuh-qa/pull/4485)) \- (Framework + tests)
+- Add tests for merged.mg file generation. ([#4129](https://github.com/wazuh/wazuh-qa/pull/4129)) \- (Tests)
+- Added tests for checking agent status upon ungraceful closure.([#4146](https://github.com/wazuh/wazuh-qa/pull/4146)) \- (Tests)
+- Agent syncronization testing after group deleting ([#4143](https://github.com/wazuh/wazuh-qa/pull/4143)) \- (Tests)
+
 ### Changed
 
 - Enable Windows Vulnerability Detector E2E. ([#4251](https://github.com/wazuh/wazuh-qa/pull/4251)) (Tests)
+- Update certifi library due to a vulnerability. ([#4486](https://github.com/wazuh/wazuh-qa/pull/4486)) \- (Tests)
+- Enable Ubuntu Vulnerability Detector E2E. ([#4252](https://github.com/wazuh/wazuh-qa/pull/4252)) \- (Tests)
 - Update _wazuh_db_ schema database version ([#4353](https://github.com/wazuh/wazuh-qa/pull/4353)) \- (Tests)
+- Update the JSON schema with the required fields for the output content of the migration tool ([#4375](https://github.com/wazuh/wazuh-qa/pull/4375)) \- (Tests)
+- Update framework known flaws file ([#4443](https://github.com/wazuh/wazuh-qa/pull/4443)) \- (Tests)
+- Align migration tool system tests to the tool's new output directory structure ([#4561](https://github.com/wazuh/wazuh-qa/pull/4561)) \- (Tests)
+
+## [4.7.1] - TBD
+
+Wazuh commit: TBD \
+Release report: TBD
 
 ## [4.7.0] - TBD
 
@@ -19,15 +37,18 @@ Release report: TBD
 
 ### Added
 
-- Add callbacks and IT tests for Integratord options tag. ([#4108](https://github.com/wazuh/wazuh-qa/pull/4108)) \- (Framework + tests)
+- Add callbacks and IT tests for Integratord options tag. ([#4166](https://github.com/wazuh/wazuh-qa/pull/4166)) \- (Framework + tests)
+- Add vulnerability Detector integration tests for Amazon Linux 2023 ([#4482](https://github.com/wazuh/wazuh-qa/pull/4482)) \- (Core)
 
 ### Changed
 
 - Add option to run some logcollector tests isolated (without a manager) [#4226](https://github.com/wazuh/wazuh-qa/pull/4226) \- (Tests + Framework)
-- Update code analysis and dependencies known flaws. [#3319](https://github.com/wazuh/wazuh-qa/pull/4083) \- (Tests)
-- Update schema database version([#3879](https://github.com/wazuh/wazuh-qa/pull/3879)) \- (Tests)
+- Update code analysis and dependencies known flaws. [#4083](https://github.com/wazuh/wazuh-qa/pull/4083) \- (Tests)
+- Update _wazuh_db_ schema database version([#4405](https://github.com/wazuh/wazuh-qa/pull/4405)) \- (Tests)
+- Update framework known flaws file ([#4313](https://github.com/wazuh/wazuh-qa/pull/4313)) \- (Tests)
 
 ### Fixed
+- Fix registry wildcards path ([#4400](https://github.com/wazuh/wazuh-qa/pull/4400)) \- (Tests)
 - Update `get_test_cases_data` function so it handles fim_mode parameter ([#4185](https://github.com/wazuh/wazuh-qa/pull/4185)) \- (Framework)
 - Fix warnings in the rids tests([#4151](https://github.com/wazuh/wazuh-qa/pull/4151)) \- (Framework + Tests)
 
@@ -38,6 +59,10 @@ Release report: TBD
 
 ### Added
 
+- Add EC2 information for system tests. ([#4536](https://github.com/wazuh/wazuh-qa/pull/4536)) \- (Documentation)
+- Add Debian Bookworm VDT IT support. ([#4463](https://github.com/wazuh/wazuh-qa/pull/4463)) \- (Tests)
+- Add new test cases for the `discard_regex` functionality of `CloudWatchLogs` and `Inspector` services. ([#4278](https://github.com/wazuh/wazuh-qa/pull/4278)) \- (Tests)
+- Add Windows location wildcards tests ([#4263](https://github.com/wazuh/wazuh-qa/pull/4263)) \- (Tests + Framework)
 - New 'SCA' test suite and framework. ([#3566](https://github.com/wazuh/wazuh-qa/pull/3566)) \- (Framework + Tests)
 - Add integration tests for AWS module. ([#3911](https://github.com/wazuh/wazuh-qa/pull/3911)) \- (Framework + Tests + Documentation)
 - Add tests for msu patches with no associated CVE . ([#4009](https://github.com/wazuh/wazuh-qa/pull/4009)) \- (Framework + Tests)
@@ -50,9 +75,16 @@ Release report: TBD
 - Add 'Force reconnect' feature to agent_simulator tool. ([#3111](https://github.com/wazuh/wazuh-qa/pull/3111)) \- (Tools)
 - Add new module to support migration tool. ([#3837](https://github.com/wazuh/wazuh-qa/pull/3837))
 - Add IT tests FIM registry monitoring using wildcards. ([#4270](https://github.com/wazuh/wazuh-qa/pull/4270)) \- (Framework + Tests)
+- Update schema database version ([#4128](https://github.com/wazuh/wazuh-qa/pull/4128)) \- (Tests)
+- Update framework known flaws files ([#4380](https://github.com/wazuh/wazuh-qa/pull/4380)) \- (Tests)
 
 ### Changed
 
+- `Agentless_cluster` system tests timeout changed in order to reduce EC2 requirements ([#4534](https://github.com/wazuh/wazuh-qa/pull/4534)) \- (Tests)
+- Skip `test_authd_ssl_options` cases that use TLS 1.1 causing errors on several OpenSSL versions. ([#4229](https://github.com/wazuh/wazuh-qa/pull/4229)) \- (Tests)
+- Update database version ([#4467](https://github.com/wazuh/wazuh-qa/pull/4467)) \- (Tests)
+- Remove versionStartIncluding from NVD custom feed ([#4441](https://github.com/wazuh/wazuh-qa/pull/4441)) \- (Tests)
+- Updated syscollector wmodules prefix ([#4384](https://github.com/wazuh/wazuh-qa/pull/4384)) \- (Framework)
 - Replace embedded python invocations with generic `python3`. ([#4186](https://github.com/wazuh/wazuh-qa/pull/4186)) - (Tests)
 - Fix FIM test_large_changes test suite ([#3948](https://github.com/wazuh/wazuh-qa/pull/3948)) \- (Tests)
 - Update `get_test_cases_data` function so it handles fim_mode parameter ([#4185](https://github.com/wazuh/wazuh-qa/pull/4185)) \- (Framework)
@@ -74,26 +106,82 @@ Release report: TBD
 - Use correct version format in agent_simulator tool ([#3198](https://github.com/wazuh/wazuh-qa/pull/3198)) \- (Tools)
 - Upgrade PyYAML to 6.0.1. ([#4326](https://github.com/wazuh/wazuh-qa/pull/4326)) \- (Framework)
 - Update schema database version ([#4128](https://github.com/wazuh/wazuh-qa/pull/4128)) \- (Tests)
+- Update framework known flaws files ([#4380](https://github.com/wazuh/wazuh-qa/pull/4380)) \- (Tests)
 
 ### Fixed
 
+- Fix invalid AR conf in integration tests ([#4521](https://github.com/wazuh/wazuh-qa/pull/4521)) \- (Tests)
+- Fix an error in AR library and test ([#4511](https://github.com/wazuh/wazuh-qa/pull/4511)) \- (Framework + Tests)
+- Fix provisioned pytest failure fixed ([#4520](https://github.com/wazuh/wazuh-qa/pull/4520)) \- (Framework)
+- Fix FIM framework to validate path in event correctly ([#4390](https://github.com/wazuh/wazuh-qa/pull/4390)) \- (Framework)
+- Fix an error related to logs format in reliability test ([#4387](https://github.com/wazuh/wazuh-qa/pull/4387)) \- (Tests)
 - Fix boto3 version requirement for legacy OS ([#4150](https://github.com/wazuh/wazuh-qa/pull/4150)) \- (Framework)
 - Fix cases yaml of the analysisd windows registry IT ([#4149](https://github.com/wazuh/wazuh-qa/pull/4149)) \- (Tests)
 - Fix a bug in on Migration tool's library ([#4106](https://github.com/wazuh/wazuh-qa/pull/4106)) \- (Framework)
 - Fix imports and add windows support for test_report_changes_and_diff IT ([#3548](https://github.com/wazuh/wazuh-qa/issues/3548)) \- (Framework + Tests)
 - Fix a regex error in the FIM integration tests ([#3061](https://github.com/wazuh/wazuh-qa/issues/3061)) \- (Framework + Tests)
 - Fix an error in the cluster performance tests related to CSV parser ([#2999](https://github.com/wazuh/wazuh-qa/pull/2999)) \- (Framework + Tests)
 - Fix bug in the framework on migration tool ([#4027](https://github.com/wazuh/wazuh-qa/pull/4027)) \- (Framework)
+- Fix test cluster / integrity sync system test and configuration to avoid flaky behavior ([#4406](https://github.com/wazuh/wazuh-qa/pull/4406)) \- (Tests)
 
-## [4.5.1] - TBD
+## [4.5.3] - 28/09/2023
 
-Wazuh commit: TBD \
-Release report: TBD
+Wazuh commit: https://github.com/wazuh/wazuh/commit/9087982b0c4ae0180bcdcd214a2b243e75cd8416 \
+Release report: https://github.com/wazuh/wazuh/issues/19111
 
-## [4.5.0] - TBD
+### Changed
 
-Wazuh commit: TBD \
-Release report: TBD
+- Update the cluster master logs reliability test to run with python 3.7 [#4445](https://github.com/wazuh/wazuh-qa/pull/4478) \- (Tests)
+- Update ITs URL for SUSE OVAL ([#4496](https://github.com/wazuh/wazuh-qa/pull/4496/))
+
+### Fixed
+
+- Update the request method used to call the login API endpoint. ([#4492](https://github.com/wazuh/wazuh-qa/pull/4492)) \- (Tests)
+- Enhancing the handling of authd and remoted simulators in case of restart failures ([#Wazuh-jenkins#3487](https://github.com/wazuh/wazuh-qa/pull/4205)) \- (Tests)
+- Fix py dependency version to install for Windows after the change to Python 3.11([#4523](https://github.com/wazuh/wazuh-qa/pull/4523)) \- (Framework)
+
+## [4.5.2] - 06-08-2023
+
+Wazuh commit: https://github.com/wazuh/wazuh/commit/2efea7428ad34bce8ea0bd32d56b5faccad114a6 \
+Release report: https://github.com/wazuh/wazuh/issues/18794
+
+### Changed
+
+- Update ITs URL for Debian OVAL ([#4491](https://github.com/wazuh/wazuh-qa/pull/4491)) \- (Tests)
+- Syscollector package inventory deltas fix ([#4483](https://github.com/wazuh/wazuh-qa/pull/4483)) \- (Tests)
+- Update schema sys_programs table ([#4451](https://github.com/wazuh/wazuh-qa/pull/4451)) \- (Tests)
+- Update enrollment logs in system test ([#4442](https://github.com/wazuh/wazuh-qa/pull/4442)) \- (Tests)
+- Fix one_manager_agent environment provisioning by packages for system tests ([#4438](https://github.com/wazuh/wazuh-qa/pull/4438)) \- (Framework)
+- Update framework known flaws files ([#4379](https://github.com/wazuh/wazuh-qa/pull/4379)) \- (Tests)
+
+### Fixed
+
+- Minor fixes in the `tests_python_flaws.py` scan ([#4439](https://github.com/wazuh/wazuh-qa/pull/4439)) \- (Tests)
+
+## [4.5.1] - 24-08-2023
+
+Wazuh commit: https://github.com/wazuh/wazuh/commit/731cdf39a430d2fb6fa02f3721624e07f887b02f
+Release report: https://github.com/wazuh/wazuh/issues/18475
+
+### Added
+
+- Add an integration test to check the wazuh-analysisd's decoder parser ([#4286](https://github.com/wazuh/wazuh-qa/pull/4286)) \- (Tests)
+
+### Changed
+
+- Update python integration test dependencies in the README ([#4427](https://github.com/wazuh/wazuh-qa/pull/4427)) \- (Documentation)
+- Update vulnerability detector IT outdated URLs ([#4428](https://github.com/wazuh/wazuh-qa/pull/4428)) \- (Tests)
+
+## [4.5.0] - 11-08-2023
+
+Wazuh commit: https://github.com/wazuh/wazuh/commit/f6aba151d08ef065dfc1bdc9b8885c3d4f618fca
+Release report: https://github.com/wazuh/wazuh/issues/18235
+
+### Changed
+
+- Delete `update_from_year` from system and E2E tests configuration ([#4372](https://github.com/wazuh/wazuh-qa/pull/4372)) \- (Tests)
+- Upgrade PyYAML to 6.0.1. ([#4326](https://github.com/wazuh/wazuh-qa/pull/4326)) \- (Framework)
+- Change Vulnerability Detector ITs to support the development of the NVD 2.0 refactor. ([#4327](https://github.com/wazuh/wazuh-qa/pull/4327)) \- (Tests)
 
 ## [4.4.5] - 10-07-2023
 
@@ -175,8 +263,6 @@ Release report: https://github.com/wazuh/wazuh/issues/15504
 - Adapt binary performance module to wazuh-cluster script renaming ([#3944](https://github.com/wazuh/wazuh-qa/pull/3944)) \- (Framework)
 - Add an option to store logs in system tests ([#2445](https://github.com/wazuh/wazuh-qa/pull/2445)) \- (Framework + Tests)
 - Add new test to check cpe_helper.json file ([#3731](https://github.com/wazuh/wazuh-qa/pull/3731))
-- Add new tests analysid handling of invalid/empty rule signature IDs ([#3649]
-(https://github.com/wazuh/wazuh-qa/pull/3649)) \- (Framework + Tests)
 - Add integration test to check statistics format ([#3813](https://github.com/wazuh/wazuh-qa/pull/3813)) \- (Framework + Tests)
 - Add new test to check vulnerable packages with triaged null([#3587](https://github.com/wazuh/wazuh-qa/pull/3587)) \- (Framework + Tests)
 - Add new tests analysid handling of invalid/empty rule signature IDs ([#3649](https://github.com/wazuh/wazuh-qa/pull/3649)) \- (Framework + Tests)

diff --git a/deps/wazuh_testing/wazuh_testing/__init__.py b/deps/wazuh_testing/wazuh_testing/__init__.py
@@ -71,7 +71,11 @@
 ANALYSISD_STATE = os.path.join(WAZUH_PATH, 'var', 'run', 'wazuh-analysisd.state')
 
 # Timeouts
+
+T_025 = 0.25
+T_1 = 1
 T_2 = 2
+T_3 = 3
 T_5 = 5
 T_10 = 10
 T_20 = 20

diff --git a/deps/wazuh_testing/wazuh_testing/cluster.py b/deps/wazuh_testing/wazuh_testing/cluster.py
@@ -26,7 +26,7 @@ def callback_detect_master_serving(line):
 
 
 def callback_detect_worker_connected(line):
-    match = re.match(r'.*Sucessfully connected to master', line)
+    match = re.match(r'.*Successfully connected to master', line)
     if match:
         return line
 

diff --git a/deps/wazuh_testing/wazuh_testing/db_interface/cve_db.py b/deps/wazuh_testing/wazuh_testing/db_interface/cve_db.py
@@ -212,7 +212,7 @@ def get_metadata_timestamp(provider_os):
     return result[0]
 
 
-def get_nvd_metadata_timestamp(year):
+def get_nvd_metadata_timestamp():
     """Get the NVD timestamp data for a specific year from nvd_metadata table.
 
     Args:
@@ -221,7 +221,7 @@ def get_nvd_metadata_timestamp(year):
     Returns:
         str: Timestamp data. (example: 2022-03-03T03:00:01-05:00)
     """
-    query_string = f"SELECT timestamp FROM nvd_metadata WHERE year={year}"
+    query_string = f"SELECT timestamp FROM nvd_metadata"
     result = get_sqlite_query_result(CVE_DB_PATH, query_string)
 
     if len(result) == 0:

diff --git a/deps/wazuh_testing/wazuh_testing/execd.py b/deps/wazuh_testing/wazuh_testing/execd.py
@@ -1,5 +1,6 @@
 import os
 import platform
+import re
 
 from wazuh_testing.tools import LOG_FILE_PATH, WAZUH_PATH
 from wazuh_testing.tools.file import truncate_file
@@ -16,14 +17,38 @@ def clean_logs():
 
 def wait_ended_message_line(line):
     """Callback function to wait for the Ended Active Response message."""
-    return True if "Ended" in line else None
+    regex = r'.*active-response\/bin\/\S+: Ended$'
+    match = re.match(regex, line)
+
+    return None if not match else line
 
 
 def wait_received_message_line(line):
     """Callback function to wait for the Received Active Response message."""
-    return True if "DEBUG: Received message: " in line else None
+    regex = r'.*DEBUG: Received message: .+'
+    match = re.match(regex, line)
+
+    return None if not match else line
 
 
 def wait_start_message_line(line):
     """Callback function to wait for the Starting Active Response message."""
-    return True if "Starting" in line else None
+    regex = r'.*active-response\/bin\/\S+: Starting$'
+    match = re.match(regex, line)
+
+    return None if not match else line
+
+
+def wait_firewall_drop_msg(line):
+    """Callback function to wait for a JSON message with the AR command.
+
+    Args:
+        line (str): String containing message.
+
+    Returns:
+        match.group(1): First capturing group which is the JSON message.
+    """
+    regex = r'.*active-response\/bin\/firewall-drop: (.+)'
+    match = re.match(regex, line)
+
+    return None if not match else match.group(1)
diff --git a/deps/wazuh_testing/wazuh_testing/mocking/__init__.py b/deps/wazuh_testing/wazuh_testing/mocking/__init__.py
@@ -76,6 +76,14 @@
                   'config_sum': '', 'merged_sum': '', 'manager_host': 'alas2022',  'node_name': 'node01',
                   'date_add': '1645538646', 'last_keepalive': '253402300799',  'sync_status': 'synced',
                   'connection_status': 'active'},
+    'ALAS_2023': {'hostname': 'alas2023', 'architecture': 'x86_64', 'os_name': 'Amazon Linux', 'os_version': '2023',
+                  'os_codename': '', 'os_major': '2023', 'os_minor': '', 'os_patch': '', 'os_build': '',
+                  'os_platform': 'amzn', 'sysname': 'Linux', 'release': '6.2.0-26.111.amzn2023.x86_64',
+                  'version': 'Wazuh v4.4.0', 'os_release': '', 'checksum': '1693284466493410477', 'name': 'alas2023',
+                  'ip': '127.0.0.1', 'register_ip': '127.0.0.1', 'internal_key': '', 'os_arch': 'x86_64',
+                  'config_sum': '', 'merged_sum': '', 'manager_host': 'alas2023',  'node_name': 'node01',
+                  'date_add': '1645538646', 'last_keepalive': '253402300799',  'sync_status': 'synced',
+                  'connection_status': 'active'},
     'RHEL8': {'os_name': 'CentOS Linux', 'os_major': '8', 'os_minor': '1', 'os_platform': 'centos',
               'name': 'centos8', 'connection_status': 'active'},
     'RHEL7': {'os_name': 'CentOS Linux', 'os_major': '7', 'os_minor': '1', 'os_platform': 'centos', 'os_version': '7.0',
@@ -101,6 +109,15 @@
                'name': 'Ubuntu-xenial'},
     'TRUSTY': {'os_name': 'Ubuntu', 'os_major': '14', 'os_minor': '04', 'os_platform': 'ubuntu',
                'name': 'Ubuntu-trusty'},
+    'BOOKWORM': {'hostname': 'bookworm', 'architecture': 'x86_64', 'os_name': 'Debian GNU/Linux', 'os_version': '12',
+                 'os_codename': 'bookworm', 'os_major': '12', 'os_minor': '', 'os_patch': '', 'os_build': '',
+                 'os_platform': 'debian', 'sysname': 'Linux', 'release': '6.1.0-10-amd64', 'version': 'Wazuh v4.5.0',
+                 'os_release': '', 'checksum': '1692739317269125720', 'name': 'bookworm', 'ip': '127.0.0.1',
+                 'register_ip': '127.0.0.1', 'internal_key': '',
+                 'os_uname': 'Linux |bookworm |6.1.0-10-amd64 |#1 SMP Debian 6.1.38-1 (2023-07-14) |x86_64',
+                 'os_arch': 'x86_64', 'config_sum': '', 'merged_sum': '', 'manager_host': 'bookworm',
+                 'node_name': 'node01', 'date_add': '1645537986', 'last_keepalive': '253402300799',
+                 'sync_status': 'synced', 'connection_status': 'active'},
     'BULLSEYE': {'hostname': 'bullseye', 'architecture': 'x86_64', 'os_name': 'Debian GNU/Linux', 'os_version': '11',
                  'os_codename': 'bullseye', 'os_major': '11', 'os_minor': '', 'os_patch': '', 'os_build': '',
                  'os_platform': 'debian', 'sysname': 'Linux', 'release': '5.10.0-10-amd64', 'version': 'Wazuh v4.3.0',
@@ -132,7 +149,15 @@
                'os_uname': 'Linux |localhost|5.3.18-22-default |#1 SMP Wed Jun 3 12:16:43 UTC 2020 (720aeba)|x86_64',
                'os_arch': 'x64', 'config_sum': '', 'merged_sum': '', 'manager_host': 'localhost.localdomain',
                'node_name': 'node01', 'date_add': '1652381429', 'last_keepalive': '253402300799',
-               'sync_status': 'synced', 'connection_status': 'active'}
+               'sync_status': 'synced', 'connection_status': 'active'},
+    'AlmaLinux-8': {'hostname': 'localhost', 'architecture': 'x64', 'os_name': 'AlmaLinux', 'os_version': '8',
+                    'os_codename': '', 'os_major': '8', 'os_minor': '', 'os_patch': '', 'os_build': '',
+                    'os_platform': 'almalinux', 'sysname': 'Linux', 'release': '5.3.18-22-default',
+                    'version': 'Wazuh v4.4.0', 'os_release': '', 'checksum': '1652388661375945607',
+                    'name': 'ALMALINUX8', 'ip': '127.0.0.1', 'register_ip': 'any', 'internal_key': '',
+                    'os_arch': 'x64', 'config_sum': '', 'merged_sum': '', 'manager_host': 'localhost.localdomain',
+                    'node_name': 'node01', 'date_add': '1652381429', 'last_keepalive': '253402300799',
+                    'sync_status': 'synced', 'connection_status': 'active'}
 }