Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stabiliser test_delete_prior_action #4480

Merged
merged 1 commit into from
Jul 30, 2024
Merged

Conversation

francoisfreitag
Copy link
Contributor

🤔 Pourquoi ?

Exemple d’échec:

========================================================================================== FAILURES ===========================================================================================
_______________________________________________________________________________ test_delete_prior_action[True] ________________________________________________________________________________
[gw0] linux -- Python 3.11.9 /home/freitafr/dev/itou3/.venv/bin/python3.11

client = <tests.utils.test.ItouClient object at 0x6f9f78b33ed0>
snapshot = '''
<ul class="list-step" hx-swap-oob="true" id="transition_logs_11111111-1111-1111-1111-111111111111">

...11+00:00">Le 10 décembre 2023 à 11:11</time>
            <span>Nouvelle candidature</span>
        </li>

</ul>
'''
with_geiq_diagnosis = True

    @pytest.mark.ignore_unknown_variable_template_error("final_hr", "with_oob_state_update")
    @pytest.mark.parametrize("with_geiq_diagnosis", [True, False])
    @freeze_time("2023-12-12 13:37:00", tz_offset=-1)
    def test_delete_prior_action(client, snapshot, with_geiq_diagnosis):
        job_application = JobApplicationFactory(
            for_snapshot=True,
            to_company__kind=CompanyKind.GEIQ,
            state=job_applications_enums.JobApplicationState.PROCESSING,
            created_at=datetime.datetime(2023, 12, 10, 10, 11, 11, tzinfo=datetime.UTC),
        )
        prior_action1 = PriorActionFactory(
            job_application=job_application, action=job_applications_enums.Prequalification.AFPR
        )
        prior_action2 = PriorActionFactory(
            job_application=job_application, action=job_applications_enums.Prequalification.AFPR
        )
        user = job_application.to_company.members.first()
        if with_geiq_diagnosis:
            GEIQEligibilityDiagnosisFactory(
                job_seeker=job_application.job_seeker,
                author_geiq=job_application.to_company,
                author=user,
                author_kind=AuthorKind.GEIQ,
            )
        # Create transition logs
        job_application.move_to_prior_to_hire(user=user)
        delete_prior_action1_url = reverse(
            "apply:delete_prior_action",
            kwargs={"job_application_id": job_application.pk, "prior_action_id": prior_action1.pk},
        )
        delete_prior_action2_url = reverse(
            "apply:delete_prior_action",
            kwargs={"job_application_id": job_application.pk, "prior_action_id": prior_action2.pk},
        )
        client.force_login(user)
        details_url = reverse("apply:details_for_company", kwargs={"job_application_id": job_application.pk})
        response = client.get(details_url)
        simulated_page = parse_response_to_soup(response, selector="#main")

        # Delete first action
        response = client.post(delete_prior_action1_url, data={})
        assert response.status_code == 200
        update_page_with_htmx(
            simulated_page,
            f"form[hx-post='{delete_prior_action1_url}']",
            response,
        )
        job_application.refresh_from_db()
        assert job_application.prior_actions.count() == 1
        assert job_application.state.is_prior_to_hire

        # Check that a fresh reload gets us in the same state
        response = client.get(details_url)
        assertSoupEqual(parse_response_to_soup(response, selector="#main"), simulated_page)

        # Delete second action
        response = client.post(delete_prior_action2_url, data={})
        assert response.status_code == 200
        soup = parse_response_to_soup(response, selector=f"#transition_logs_{job_application.pk}")
>       assert str(soup) == snapshot
E       assert [+ received] == [- snapshot]
E           '''
E                   ...
E                           <span>
E         -                     Passé en "Action préalable à l’embauche"
E         +                     Passé en "Candidature à l'étude"
E                               par John DOE
E                             ......
E
E         ...Full output truncated (6 lines hidden), use '-vv' to show

tests/www/apply/test_process.py:3088: AssertionError
------------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------------
Installed 63 object(s) from 2 fixture(s)
------------------------------------------------------------------------------------ Captured stderr setup ------------------------------------------------------------------------------------
{"message": "HTTP Request: HEAD http://localhost:9000/minio/health/live \"HTTP/1.1 200 OK\"", "logger.name": "httpx", "logger.thread_name": "MainThread", "logger.method_name": "_send_single_request", "date": "2024-07-29T14:38:34.150386+00:00", "status": "INFO"}
{"message": "Management command itou.files.management.commands.configure_bucket succeeded in 0.18 seconds", "logger.name": "itou.files.management.commands.configure_bucket", "logger.thread_name": "MainThread", "logger.method_name": "_log_command_result", "date": "2024-07-29T14:38:34.317345+00:00", "status": "INFO", "duration": 175229059}
------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------
INFO     httpx:_client.py:1013 HTTP Request: HEAD http://localhost:9000/minio/health/live "HTTP/1.1 200 OK"
INFO     itou.files.management.commands.configure_bucket:command.py:9 Management command itou.files.management.commands.configure_bucket succeeded in 0.18 seconds
------------------------------------------------------------------------------------ Captured stderr call -------------------------------------------------------------------------------------
{"message": "<JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.move_to_prior_to_hire (processing -> prior_to_hire)", "logger.name": "xworkflows.transitions", "logger.thread_name": "MainThread", "logger.method_name": "log_transition", "date": "2023-12-12T13:37:00+00:00", "status": "INFO"}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:details_for_company", "http.method": "GET", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "8d5418d4-f543-4d09-ae16-b1572549b4ec", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/127/delete", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/127/delete", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:delete_prior_action", "http.method": "POST", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "a0560c45-ed13-49e6-b491-6c30998c0270", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:details_for_company", "http.method": "GET", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "894e3432-328f-44e7-9647-e05bff16dead", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
{"message": "<JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.cancel_prior_to_hire (prior_to_hire -> processing)", "logger.name": "xworkflows.transitions", "logger.thread_name": "MainThread", "logger.method_name": "log_transition", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:delete_prior_action", "http.method": "POST", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "0b3628d3-2cda-45e6-80ca-bd2402f56352", "usr.id": 12361, "usr.organization_id": 3551}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:delete_prior_action", "http.method": "POST", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "0b3628d3-2cda-45e6-80ca-bd2402f56352", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
-------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------
INFO     xworkflows.transitions:base.py:866 <JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.move_to_prior_to_hire (processing -> prior_to_hire)
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
INFO     xworkflows.transitions:base.py:866 <JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.cancel_prior_to_hire (prior_to_hire -> processing)
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
=================================================================================== short test summary info ===================================================================================
FAILED tests/www/apply/test_process.py::test_delete_prior_action[True] - assert [+ received] == [- snapshot]

🍰 Comment ?

Changeant la date des changements d’états des transitions de candidature.

🏝️ Comment tester

Lancer le test dans une tight loop.

@francoisfreitag francoisfreitag requested a review from rsebille July 29, 2024 16:00
Copy link
Contributor

@rsebille rsebille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Le problème sous-jacent ça serais pas plutôt dû à un "problème" de tri ? Soir révélé par la fixture make_unordered_queries_randomly_ordered ou parce qu'il manque une condition en plus de la date ?

@francoisfreitag
Copy link
Contributor Author

Je vois mal comment deux transitions d’état sur la même candidature peuvent se passer exactement au même moment ? Et au pire, leur affichage de manière indéterminée n’est pas un souci si elles ont eu lieu exactement en même temps.

Copy link
Contributor

@rsebille rsebille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, je vois ta logique :).
J'ai pris le truc en mode dans le cas limite où plusieurs transitions ont le même horodatage alors leur ordre est indéterminé donc la solution est de déterminer l'ordre pour que ça "corrige" partout et tout le temps, mais c'est peut-être mon coté flemmard qui parle 😁.

tests/www/apply/test_process.py Outdated Show resolved Hide resolved
Example failure:

```
========================================================================================== FAILURES ===========================================================================================
_______________________________________________________________________________ test_delete_prior_action[True] ________________________________________________________________________________
[gw0] linux -- Python 3.11.9 /home/freitafr/dev/itou3/.venv/bin/python3.11

client = <tests.utils.test.ItouClient object at 0x6f9f78b33ed0>
snapshot = '''
<ul class="list-step" hx-swap-oob="true" id="transition_logs_11111111-1111-1111-1111-111111111111">

...11+00:00">Le 10 décembre 2023 à 11:11</time>
            <span>Nouvelle candidature</span>
        </li>

</ul>
'''
with_geiq_diagnosis = True

    @pytest.mark.ignore_unknown_variable_template_error("final_hr", "with_oob_state_update")
    @pytest.mark.parametrize("with_geiq_diagnosis", [True, False])
    @freeze_time("2023-12-12 13:37:00", tz_offset=-1)
    def test_delete_prior_action(client, snapshot, with_geiq_diagnosis):
        job_application = JobApplicationFactory(
            for_snapshot=True,
            to_company__kind=CompanyKind.GEIQ,
            state=job_applications_enums.JobApplicationState.PROCESSING,
            created_at=datetime.datetime(2023, 12, 10, 10, 11, 11, tzinfo=datetime.UTC),
        )
        prior_action1 = PriorActionFactory(
            job_application=job_application, action=job_applications_enums.Prequalification.AFPR
        )
        prior_action2 = PriorActionFactory(
            job_application=job_application, action=job_applications_enums.Prequalification.AFPR
        )
        user = job_application.to_company.members.first()
        if with_geiq_diagnosis:
            GEIQEligibilityDiagnosisFactory(
                job_seeker=job_application.job_seeker,
                author_geiq=job_application.to_company,
                author=user,
                author_kind=AuthorKind.GEIQ,
            )
        # Create transition logs
        job_application.move_to_prior_to_hire(user=user)
        delete_prior_action1_url = reverse(
            "apply:delete_prior_action",
            kwargs={"job_application_id": job_application.pk, "prior_action_id": prior_action1.pk},
        )
        delete_prior_action2_url = reverse(
            "apply:delete_prior_action",
            kwargs={"job_application_id": job_application.pk, "prior_action_id": prior_action2.pk},
        )
        client.force_login(user)
        details_url = reverse("apply:details_for_company", kwargs={"job_application_id": job_application.pk})
        response = client.get(details_url)
        simulated_page = parse_response_to_soup(response, selector="#main")

        # Delete first action
        response = client.post(delete_prior_action1_url, data={})
        assert response.status_code == 200
        update_page_with_htmx(
            simulated_page,
            f"form[hx-post='{delete_prior_action1_url}']",
            response,
        )
        job_application.refresh_from_db()
        assert job_application.prior_actions.count() == 1
        assert job_application.state.is_prior_to_hire

        # Check that a fresh reload gets us in the same state
        response = client.get(details_url)
        assertSoupEqual(parse_response_to_soup(response, selector="#main"), simulated_page)

        # Delete second action
        response = client.post(delete_prior_action2_url, data={})
        assert response.status_code == 200
        soup = parse_response_to_soup(response, selector=f"#transition_logs_{job_application.pk}")
>       assert str(soup) == snapshot
E       assert [+ received] == [- snapshot]
E           '''
E                   ...
E                           <span>
E         -                     Passé en "Action préalable à l’embauche"
E         +                     Passé en "Candidature à l'étude"
E                               par John DOE
E                             ......
E
E         ...Full output truncated (6 lines hidden), use '-vv' to show

tests/www/apply/test_process.py:3088: AssertionError
------------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------------
Installed 63 object(s) from 2 fixture(s)
------------------------------------------------------------------------------------ Captured stderr setup ------------------------------------------------------------------------------------
{"message": "HTTP Request: HEAD http://localhost:9000/minio/health/live \"HTTP/1.1 200 OK\"", "logger.name": "httpx", "logger.thread_name": "MainThread", "logger.method_name": "_send_single_request", "date": "2024-07-29T14:38:34.150386+00:00", "status": "INFO"}
{"message": "Management command itou.files.management.commands.configure_bucket succeeded in 0.18 seconds", "logger.name": "itou.files.management.commands.configure_bucket", "logger.thread_name": "MainThread", "logger.method_name": "_log_command_result", "date": "2024-07-29T14:38:34.317345+00:00", "status": "INFO", "duration": 175229059}
------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------
INFO     httpx:_client.py:1013 HTTP Request: HEAD http://localhost:9000/minio/health/live "HTTP/1.1 200 OK"
INFO     itou.files.management.commands.configure_bucket:command.py:9 Management command itou.files.management.commands.configure_bucket succeeded in 0.18 seconds
------------------------------------------------------------------------------------ Captured stderr call -------------------------------------------------------------------------------------
{"message": "<JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.move_to_prior_to_hire (processing -> prior_to_hire)", "logger.name": "xworkflows.transitions", "logger.thread_name": "MainThread", "logger.method_name": "log_transition", "date": "2023-12-12T13:37:00+00:00", "status": "INFO"}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:details_for_company", "http.method": "GET", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "8d5418d4-f543-4d09-ae16-b1572549b4ec", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/127/delete", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/127/delete", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:delete_prior_action", "http.method": "POST", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "a0560c45-ed13-49e6-b491-6c30998c0270", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/details", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:details_for_company", "http.method": "GET", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "894e3432-328f-44e7-9647-e05bff16dead", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
{"message": "<JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.cancel_prior_to_hire (prior_to_hire -> processing)", "logger.name": "xworkflows.transitions", "logger.thread_name": "MainThread", "logger.method_name": "log_transition", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:delete_prior_action", "http.method": "POST", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "0b3628d3-2cda-45e6-80ca-bd2402f56352", "usr.id": 12361, "usr.organization_id": 3551}
{"message": "HTTP 200 OK", "logger.name": "django_datadog_logger.middleware.request_log", "logger.thread_name": "MainThread", "logger.method_name": "log_response", "date": "2023-12-12T13:37:00+00:00", "status": "INFO", "network.client.ip": "127.0.0.1", "http.url": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.host": "testserver", "http.url_details.port": null, "http.url_details.path": "/apply/11111111-1111-1111-1111-111111111111/siae/prior-action/128/delete", "http.url_details.queryString": {}, "http.url_details.scheme": "http", "http.url_details.view_name": "apply:delete_prior_action", "http.method": "POST", "http.accept": null, "http.referer": null, "http.useragent": null, "http.request_version": null, "http.request_id": "0b3628d3-2cda-45e6-80ca-bd2402f56352", "usr.id": 12361, "duration": 0.0, "http.status_code": 200, "usr.organization_id": 3551}
-------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------
INFO     xworkflows.transitions:base.py:866 <JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.move_to_prior_to_hire (processing -> prior_to_hire)
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
INFO     xworkflows.transitions:base.py:866 <JobApplication: 11111111-1111-1111-1111-111111111111> performed transition JobApplicationWorkflow.cancel_prior_to_hire (prior_to_hire -> processing)
INFO     django_datadog_logger.middleware.request_log:request_log.py:46 HTTP 200 OK
=================================================================================== short test summary info ===================================================================================
FAILED tests/www/apply/test_process.py::test_delete_prior_action[True] - assert [+ received] == [- snapshot]
```
@francoisfreitag francoisfreitag force-pushed the ff/flaky-prior-action branch from 96d00d0 to d5e4034 Compare July 30, 2024 10:18
@francoisfreitag francoisfreitag added this pull request to the merge queue Jul 30, 2024
Merged via the queue into master with commit 782ed15 Jul 30, 2024
10 of 11 checks passed
@francoisfreitag francoisfreitag deleted the ff/flaky-prior-action branch July 30, 2024 10:34
@francoisfreitag francoisfreitag added the no-changelog Ne doit pas figurer dans le journal des changements. label Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-changelog Ne doit pas figurer dans le journal des changements.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants