SCAP report and remediation #1441

lhellebr · 2024-06-24T15:07:51Z

Required by SatelliteQE/robottelo#15491

Also, added limit of rows to read because sometimes you don't need them all and it takes a while to read.

lhellebr · 2024-06-24T15:10:26Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_host.py::test_positive_end_to_end

lhellebr · 2024-06-24T15:10:43Z

PRT to check whether limit has caused regression

lhellebr · 2024-06-25T09:32:52Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_host.py::test_positive_end_to_end

Satellite-QE · 2024-06-25T09:54:11Z

PRT Result

Build Number: 308
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_host.py::test_positive_end_to_end --external-logging
Test Result : ================== 1 failed, 9 warnings in 681.62s (0:11:21) ===================

lhellebr · 2024-06-25T12:52:51Z

This test wasn't a good choice, it fails in master on stream as well. I'll pick another one.

lhellebr · 2024-06-25T12:56:11Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_activationkey.py::test_positive_end_to_end_crud

Satellite-QE · 2024-06-25T13:13:33Z

PRT Result

Build Number: 309
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_activationkey.py::test_positive_end_to_end_crud --external-logging
Test Result : ================== 1 failed, 10 warnings in 691.84s (0:11:31) ==================

lhellebr · 2024-06-25T13:25:40Z

Ok, this is a true positive, do not merge and I will look into it

lhellebr · 2024-06-25T14:49:54Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_activationkey.py::test_positive_end_to_end_crud

Satellite-QE · 2024-06-25T15:18:27Z

PRT Result

Build Number: 310
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/ui/test_activationkey.py::test_positive_end_to_end_crud --external-logging
Test Result : ================== 1 passed, 12 warnings in 757.32s (0:12:37) ==================

lhellebr · 2024-06-25T15:27:37Z

Fixed, pls review

vijaysawant

Need to update doc string. Otherwise looks good.

airgun/entities/oscapreport.py

airgun/views/common.py

lhellebr · 2024-07-09T12:45:50Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_host.py::test_positive_end_to_end tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation

Satellite-QE · 2024-07-09T13:16:45Z

PRT Result

Build Number: 311
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/ui/test_host.py::test_positive_end_to_end tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation --external-logging
Test Result : ================= 2 passed, 130 warnings in 1480.32s (0:24:40) =================

lhellebr · 2024-07-09T13:24:09Z

@vijaysawant comments addressed

lhellebr · 2024-07-10T13:40:51Z

And the "remove PRT" failure is not this PR's fault

lhellebr · 2024-07-15T09:59:50Z

omkarkhatavkar · 2024-07-16T11:15:21Z

airgun/entities/oscapreport.py

+        view.wait_displayed()
+        self.browser.plugin.ensure_page_safe()
+        wait_for(lambda: view.title.is_displayed, timeout=10, delay=1)
+        view.fill({'select_remediation_method.snippet': 'Ansible'})


Ansible is a default value? you could expect some more options there ?

That's what this function does as per docstring: Remediate the failed rule using automatic remediation through Ansible.
It surely can be changed to allow for different methods. On one hand, it's not a goal of this function/test to test various methods of remote execution. On the other hand, there is a wizard specifically for remediation that lets you choose so IMO this will be a good addition, later.

omkarkhatavkar · 2024-07-16T11:16:30Z

airgun/entities/oscapreport.py

+            clicking on the link in Reported At column of
+            SCAP Report list
+
+        :param search_string:


what is this search string ? is this any type of entity?

Unfortunately a link to the correct arf report is in the column Reported At and you get there by clicking the link saying e.g. 1 day ago... so search actually searches by a (not visible) id in the link.
You call it like this:

session.oscapreport.remediate(f'id={arf_id}', title)

omkarkhatavkar · 2024-07-16T11:19:49Z

airgun/widgets.py

@@ -1929,11 +1929,56 @@ def has_rows(self):
            return False
        return True

-    def read(self):
+    def read_limited(self, limit):


@lhellebr nice addition!

omkarkhatavkar · 2024-07-16T11:22:59Z

airgun/views/common.py


        :return: A :py:class:`dict` of ``widget_name: widget_read_value``
            where the values are retrieved using the :py:meth:`Widget.read`.
        """
        if widget_names is None:
+            if limit is not None:
+                raise NotImplementedError("You must specify widgets to be able to specify limit")


s/widgets/widget_names is that right ?

Well... you specify widgets by their names

omkarkhatavkar

ACK, non-blocking comments

lhellebr · 2024-08-20T13:01:38Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_host.py::test_positive_end_to_end tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation

Satellite-QE · 2024-08-20T13:27:40Z

PRT Result

Build Number: 363
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_host.py::test_positive_end_to_end tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation --external-logging
Test Result : ============ 1 failed, 1 passed, 111 warnings in 1204.78s (0:20:04) ============

lhellebr · 2024-08-20T13:52:24Z

So... after this hasn't been merged for 2 MONTHS, there is a seemingly unrelated issue that I will have to investigate. Does it ring a bell?

robottelo.exceptions.CLIReturnCodeError: CLIReturnCodeError(status=70, stderr='Could not update the host:\n  Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Environment with ID \\"74cd9541b1d4a00c20d1e5df73d5eb95\\" could not be found.","requestUuid":"01bd3816-957a-4d12-9c11-fa0953beceab"} (PUT /candlepin/consumers/c3c5db21-f366-48d7-9f60-25f6fa2d914e)\n', msg='Command "host update" finished with status 70\nstderr contains:\nCould not update the host:\n  Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Environment with ID \\"74cd9541b1d4a00c20d1e5df73d5eb95\\" could not be found.","requestUuid":"01bd3816-957a-4d12-9c11-fa0953beceab"} (PUT /candlepin/consumers/c3c5db21-f366-48d7-9f60-25f6fa2d914e)\n'

lhellebr · 2024-08-26T15:39:29Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_host.py::test_positive_end_to_end tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation
robottelo: 16070

Satellite-QE · 2024-08-26T16:10:28Z

PRT Result

Build Number: 376
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_host.py::test_positive_end_to_end tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation --external-logging
Test Result : ============ 1 failed, 1 passed, 126 warnings in 1337.22s (0:22:17) ============

lhellebr · 2024-08-26T16:15:53Z

The failure in test_positive_end_to_end is NOT a regression caused by this PR. The same failure appears in the last 6.16 automation.

lhellebr · 2024-08-26T16:26:50Z

trigger: test-robottelo
pytest: tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation
robottelo: 16070

Satellite-QE · 2024-08-26T16:55:29Z

PRT Result

Build Number: 378
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation --external-logging
Test Result : ================= 1 passed, 16 warnings in 1323.01s (0:22:03) ==================

lhellebr · 2024-08-27T08:16:19Z

I'm gonna self-merge in 48 hours unless someone comments here.

lhellebr · 2024-08-27T08:23:20Z

trigger: test-robottelo
pytest: tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation tests/foreman/ui/test_activationkey.py::test_positive_end_to_end_crud
robottelo: 16070

Satellite-QE · 2024-08-27T09:42:35Z

PRT Result

Build Number: 380
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/longrun/test_oscap.py::test_positive_oscap_remediation tests/foreman/ui/test_activationkey.py::test_positive_end_to_end_crud --external-logging
Test Result : ================= 2 passed, 110 warnings in 4061.85s (1:07:41) =================

lhellebr · 2024-08-27T13:55:33Z

We can't merge due to Remove the PRT label failure. I don't think it's caused by this PR. How can I fix it? @SatelliteQE/airgun-tier-2-reviewers ?

* SCAP report and remediation * Fix missing attribute error when called without limit * Comments addressed

Satellite-QE assigned lhellebr Jun 24, 2024

lhellebr mentioned this pull request Jun 24, 2024

Add a scap remediation feature test SatelliteQE/robottelo#15491

Merged

lhellebr added the No-CherryPick PR doesnt need CherryPick to previous branches label Jun 24, 2024

Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label Jun 25, 2024

lhellebr added the do not merge label Jun 25, 2024

lhellebr added 2 commits June 25, 2024 16:10

SCAP report and remediation

b1ab301

Fix missing attribute error when called without limit

48cf185

lhellebr force-pushed the scap_report branch from 9a36c8a to 48cf185 Compare June 25, 2024 14:49

lhellebr removed the do not merge label Jun 25, 2024

Satellite-QE added PRT-Passed Indicates that latest PRT run is passed for the PR and removed PRT-Failed Indicates that latest PRT run is failed for the PR labels Jun 25, 2024

lhellebr requested review from a team June 25, 2024 15:27

vijaysawant requested changes Jun 27, 2024

View reviewed changes

airgun/entities/oscapreport.py Outdated Show resolved Hide resolved

airgun/entities/oscapreport.py Outdated Show resolved Hide resolved

airgun/views/common.py Show resolved Hide resolved

airgun/views/common.py Show resolved Hide resolved

Comments addressed

51385d5

lhellebr requested a review from vijaysawant July 9, 2024 13:23

lhellebr requested a review from a team July 15, 2024 09:25

omkarkhatavkar reviewed Jul 16, 2024

View reviewed changes

omkarkhatavkar approved these changes Jul 16, 2024

View reviewed changes

sambible approved these changes Jul 24, 2024

View reviewed changes

sambible removed the PRT-Passed Indicates that latest PRT run is passed for the PR label Jul 24, 2024

Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label Aug 20, 2024

Satellite-QE added PRT-Passed Indicates that latest PRT run is passed for the PR and removed PRT-Failed Indicates that latest PRT run is failed for the PR labels Aug 26, 2024

lhellebr requested a review from a team August 27, 2024 08:15

pondrejk approved these changes Aug 27, 2024

View reviewed changes

omkarkhatavkar merged commit 73239e4 into SatelliteQE:master Aug 28, 2024
11 of 12 checks passed

lhellebr added a commit that referenced this pull request Sep 2, 2024

SCAP report and remediation (#1441)

fe46eab

* SCAP report and remediation * Fix missing attribute error when called without limit * Comments addressed

SCAP report and remediation #1441

SCAP report and remediation #1441

Conversation

lhellebr commented Jun 24, 2024

lhellebr commented Jun 24, 2024

lhellebr commented Jun 24, 2024

lhellebr commented Jun 25, 2024

Satellite-QE commented Jun 25, 2024

lhellebr commented Jun 25, 2024

lhellebr commented Jun 25, 2024

Satellite-QE commented Jun 25, 2024

lhellebr commented Jun 25, 2024

lhellebr commented Jun 25, 2024

Satellite-QE commented Jun 25, 2024

lhellebr commented Jun 25, 2024

vijaysawant left a comment

Choose a reason for hiding this comment

lhellebr commented Jul 9, 2024

Satellite-QE commented Jul 9, 2024

lhellebr commented Jul 9, 2024

lhellebr commented Jul 10, 2024 • edited Loading

lhellebr commented Jul 15, 2024

omkarkhatavkar Jul 16, 2024

Choose a reason for hiding this comment

lhellebr Aug 20, 2024

Choose a reason for hiding this comment

omkarkhatavkar Jul 16, 2024

Choose a reason for hiding this comment

lhellebr Aug 20, 2024

Choose a reason for hiding this comment

omkarkhatavkar Jul 16, 2024

Choose a reason for hiding this comment

omkarkhatavkar Jul 16, 2024

Choose a reason for hiding this comment

lhellebr Aug 20, 2024

Choose a reason for hiding this comment

omkarkhatavkar left a comment

Choose a reason for hiding this comment

lhellebr commented Aug 20, 2024

Satellite-QE commented Aug 20, 2024

lhellebr commented Aug 20, 2024

lhellebr commented Aug 26, 2024

Satellite-QE commented Aug 26, 2024

lhellebr commented Aug 26, 2024

lhellebr commented Aug 26, 2024

Satellite-QE commented Aug 26, 2024

lhellebr commented Aug 27, 2024

lhellebr commented Aug 27, 2024

Satellite-QE commented Aug 27, 2024

lhellebr commented Aug 27, 2024

lhellebr commented Jul 10, 2024 •

edited

Loading