Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: exporter sanitizing content to delete #1796

Merged

Conversation

alex-sheehan-edx
Copy link
Contributor

Merge checklist:

  • Any new requirements are in the right place (do not manually modify the requirements/*.txt files)
    • base.in if needed in production but edx-platform doesn't install it
    • test-master.in if edx-platform pins it, with a matching version
    • make upgrade && make requirements have been run to regenerate requirements
  • make static has been run to update webpack bundling if any static content was updated
  • ./manage.py makemigrations has been run
    • Checkout the Database Migration Confluence page for helpful tips on creating migrations.
    • Note: This must be run if you modified any models.
      • It may or may not make a migration depending on exactly what you modified, but it should still be run.
    • This should be run from either a venv with all the lms/edx-enterprise requirements installed or if you checked out edx-enterprise into the src directory used by lms, you can run this command through an lms shell.
      • It would be ./manage.py lms makemigrations in the shell.
  • Version bumped
  • Changelog record added
  • Translations updated (see docs/internationalization.rst but also this isn't blocking for merge atm)

Post merge:

  • Tag pushed and a new version released
    • Note: Assets will be added automatically. You just need to provide a tag (should match your version number) and title and description.
  • After versioned build finishes in GitHub Actions, verify version has been pushed to PyPI
    • Each step in the release build has a condition flag that checks if the rest of the steps are done and if so will deploy to PyPi.
      (so basically once your build finishes, after maybe a minute you should see the new version in PyPi automatically (on refresh))
  • PR created in edx-platform to upgrade dependencies (including edx-enterprise)
    • This must be done after the version is visible in PyPi as make upgrade in edx-platform will look for the latest version in PyPi.
    • Note: the edx-enterprise constraint in edx-platform must also be bumped to the latest version in PyPi.

@alex-sheehan-edx alex-sheehan-edx requested a review from a team July 7, 2023 16:19
@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch 2 times, most recently from 9c73c48 to 3c49e07 Compare July 10, 2023 21:58
@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch from 3c49e07 to 7901820 Compare July 10, 2023 22:09
@alex-sheehan-edx alex-sheehan-edx changed the title fix: catching orphaned records without catalog uuids fix: exporter sanitizing content to delete Jul 10, 2023
@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch 3 times, most recently from 51e9757 to a920697 Compare July 11, 2023 00:15
Copy link
Contributor

@katrinan029 katrinan029 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you for walking through these changes with me.

@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch 5 times, most recently from bc76259 to c2b498b Compare July 12, 2023 16:41
Copy link
Contributor

@johnnagro johnnagro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the approach looks good. i would recommend a few tweaks. i'd love to know if we can factor out that need to hold back on the content_last_modified timestamp.

)

transformer_for_action = (
getattr(self, f'transform_for_action_{content_metadata_type}_{edx_data_schema_key}', None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick question: In what instance will this be used? Where is this attribute defined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention's to go into the channel specific content metadata exporter and use the defined mappings to tailor and sanitize the code (see https://github.com/openedx/edx-enterprise/blob/master/integrated_channels/sap_success_factors/exporters/content_metadata.py). There we have two different types of mapping methods, transform_{attribute} and transform_{content_type}_{attribute} because some external platforms care about and want to distinguish between metadata depending on what type of content it is, ie SAP wants different titles for course runs than for course and programs. In this case we're supporting the old implementation while also allowing for the caller to specify more detail/actions. transform_{content_type}_{attribute} isn't currently being used by any channel yet but I wanted to add the functionality as we know channels do this content specific mapping already and we may want to support it in an upgrade in the future.

@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch from c2b498b to c88e335 Compare July 12, 2023 19:53
@@ -19,16 +19,6 @@ def __init__(self, enterprise_configuration, client=SAPSuccessFactorsAPIClient):
client=client
)

def _transmit_action(self, content_metadata_item_map, client_method, action_name):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇

@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch 3 times, most recently from 5207bf9 to 9fc81cb Compare July 13, 2023 14:14
@alex-sheehan-edx alex-sheehan-edx force-pushed the asheehan-edx/more-orphaned-content-detection branch from 9fc81cb to ab55127 Compare July 13, 2023 15:05
@alex-sheehan-edx alex-sheehan-edx merged commit 3755e5c into master Jul 13, 2023
6 checks passed
@alex-sheehan-edx alex-sheehan-edx deleted the asheehan-edx/more-orphaned-content-detection branch July 13, 2023 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants