feat: Commerce Coordinator step in retirement pipeline

.github/workflows/units-test-scripts-user-retirement.yml

@@ -26,9 +26,11 @@ jobs:
 
       - name: Install dependencies
         run: |
+          sudo apt-get update && sudo apt-get install libxmlsec1-dev
           python -m pip install --upgrade pip
           pip install -r scripts/user_retirement/requirements/testing.txt
 
       - name: Run pytest
         run: |
+          export DJANGO_SETTINGS_MODULE=lms.envs.test
           pytest scripts/user_retirement

lms/envs/common.py

@@ -4996,6 +4996,8 @@ def _make_locale_paths(settings):  # pylint: disable=missing-function-docstring
     'COMPLETE',
 ]
 
+EXTRA_SERVICES_TO_RETIRE_FROM = []
+
 USERNAME_REPLACEMENT_WORKER = "REPLACE WITH VALID USERNAME"
 
 ############## Settings for Microfrontends  #########################

lms/envs/test.py

@@ -662,6 +662,14 @@
 CSRF_TRUSTED_ORIGINS = ['.example.com']
 CSRF_TRUSTED_ORIGINS_WITH_SCHEME = ['https://*.example.com']
 
+EXTRA_SERVICES_TO_RETIRE_FROM = [
+    {
+        'name': 'MOCK_SERVICE',
+        'service_base_url': 'http://fake_service_base_url',
+        'retirement_url_path': 'fake_retirement_url_path'
+    }
+]
+
 # values are already updated above with default CSRF_TRUSTED_ORIGINS values but in
 # case of new django version these values will override.
 if django.VERSION[0] >= 4:  # for greater than django 3.2 use with schemes.

scripts/user_retirement/docs/implementation_overview.rst

@@ -11,8 +11,8 @@ these services. As a consequence, to remove a user's PII, you must be able
 to request each service containing PII to remove, delete, or unlink the
 data for that user in that service.
 
-In the user retirement feature, a centralized process (the *driver* scripts) 
-orchestrates all of these requests. For information about how to configure the 
+In the user retirement feature, a centralized process (the *driver* scripts)
+orchestrates all of these requests. For information about how to configure the
 driver scripts, see :ref:`driver-setup`.
 
 ****************************
@@ -46,6 +46,27 @@ table of the states themselves (the ``RetirementState`` model), rather than
 hard-coding the states.  This was done because we cannot predict all the
 possible states required by all members of the Open edX community.
 
+The workflow also allows the inclusion of additional services that may need to be
+retired alongside the core services. These extra services can now be configured within
+the Django settings under the ``EXTRA_SERVICES_TO_RETIRE_FROM`` setting. This setting allows
+the platform to handle the retirement of PII in custom or internal services that are not
+part of the standard Open edX installation but are crucial for specific deployments.
+
+Here is an example of how the ``EXTRA_SERVICES_TO_RETIRE_FROM`` setting can be configured:
+
+.. code-block:: python
+
+     EXTRA_SERVICES_TO_RETIRE_FROM = [
+        {
+            'name': 'MOCK_SERVICE',
+            'service_base_url': 'http://fake_service_base_url',
+            'retirement_url_path': 'fake_retirement_url_path'
+        }
+     ]
+
+This setting defines a list of services, each with a ``name``, ``service_base_url``, and ``retirement_url_path``,
+allowing the user retirement process to include these additional services.
+
 This example state diagram outlines the pathways users follow throughout the
 workflow:
 

scripts/user_retirement/docs/service_setup.rst

@@ -36,13 +36,13 @@ defining *derived* settings specific to Open edX. Read more about it in
      - ``'retired.invalid'``
      - The domain part of hashed emails. Used in ``RETIRED_EMAIL_FMT``.
    * - RETIRED_USERNAME_FMT
-     - ``lambda settings: 
+     - ``lambda settings:
        settings.RETIRED_USERNAME_PREFIX + '{}'``
      - The username field for a retired user gets transformed into this format,
        where ``{}`` is replaced with the hash of their username.
    * - RETIRED_EMAIL_FMT
-     - ``lambda settings: 
-       settings.RETIRED_EMAIL_PREFIX + '{}@' + 
+     - ``lambda settings:
+       settings.RETIRED_EMAIL_PREFIX + '{}@' +
        settings.RETIRED_EMAIL_DOMAIN``
      - The email field for a retired user gets transformed into this format, where
        ``{}`` is replaced with the hash of their email.
@@ -60,6 +60,27 @@ defining *derived* settings specific to Open edX. Read more about it in
    * - FEATURES['ENABLE_ACCOUNT_DELETION']
      - True
      - Whether to display the "Delete My Account" section the account settings page.
+   * - EXTRA_SERVICES_TO_RETIRE_FROM
+     - None
+     - A list of additional services from which user data should be retired. Each entry in the list should be a dictionary with the following keys:
+
+       - ``name``: The name of the service.
+       - ``service_base_url``: The base URL of the service's API.
+       - ``retirement_url_path``: The API path for the user retirement endpoint.
+
+       This setting allows the retirement process to interact with external services not covered by default. For example:
+
+       .. code-block:: python
+
+         EXTRA_SERVICES_TO_RETIRE_FROM = [
+             {
+                 'name': 'MOCK_SERVICE',
+                 'service_base_url': 'http://fake_service_base_url',
+                 'retirement_url_path': 'fake_retirement_url_path'
+             }
+         ]
+
+       By default, this setting is empty, meaning no additional services are included. This can be overridden in `common.py <https://github.com/openedx/edx-platform/blob/master/lms/envs/common.py>`_.
 
 
 =================
@@ -79,10 +100,10 @@ state at the beginning, and ``COMPLETED``, ``ERRORED``, and ``ABORTED`` states
 at the end of the list.  Also, for every ``RETIRING_foo`` state, there must be
 a corresponding ``foo_COMPLETE`` state.
 
-Override these states if you need to add any states.  Typically, these 
+Override these states if you need to add any states.  Typically, these
 settings are set in ``lms.yml``.
 
-After you have defined any custom states, populate the states table with the 
+After you have defined any custom states, populate the states table with the
 following management command:
 
 .. code-block:: bash
@@ -120,7 +141,7 @@ Retirement Service User
 
 The user retirement driver scripts authenticate with the LMS and IDAs as the
 retirement service user with oauth client credentials.  Therefore, to use the
-driver scripts, you must create a retirement service user, and generate a DOT 
+driver scripts, you must create a retirement service user, and generate a DOT
 application and client credentials, as in the following command.
 
 .. code-block:: bash
@@ -162,7 +183,7 @@ that relate to user retirement.
      - ``/admin/user_api/retirementstate/``
      - Represents the table of states defined in ``RETIREMENT_STATES`` and
        populated with ``populate_retirement_states``.
-   * - User Retirement Requests 
+   * - User Retirement Requests
      - ``/admin/user_api/userretirementrequest/``
      - Represents the table that tracks the user IDs of every learner who
        has ever requested account deletion. This table is primarily used for
@@ -173,7 +194,7 @@ that relate to user retirement.
 
 In special cases where you may need to manually intervene with the pipeline,
 you can use the User Retirement Statuses management page to change the
-state for an individual user.  For more information about how to handle these 
+state for an individual user.  For more information about how to handle these
 cases, see :ref:`handling-special-cases`.
 
 .. include:: ../../../../links/links.rst

scripts/user_retirement/pytest.ini

@@ -0,0 +1,3 @@
+[pytest]
+
+DJANGO_SETTINGS_MODULE = lms.envs.test

scripts/user_retirement/requirements/base.in

@@ -1,4 +1,5 @@
 -c ../../../requirements/constraints.txt
+-r ../../../requirements/edx/base.txt
 
 boto3
 click