-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: adds sortable fields to studio content search index [FC-0059] #35103
feat: adds sortable fields to studio content search index [FC-0059] #35103
Conversation
* new datetime fields added to library component serializer: created, modified, and last_published * new datetime fields are stored in Meilisearch as POSIX timestamps for sorting * Meilisearch search ranking updated to allow sorting by display_name or these new datetime fields * Updates tests to verify these new properties and client settings
Thanks for the pull request, @pomegranited! What's next?Please work through the following steps to get your changes ready for engineering review: 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. 🔘 Let us know that your PR is ready for review:Who will review my changes?This repository is currently maintained by Where can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:
When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
* Instead of using `latest.modified` (which always queries the database), use `draft.latest` * However, we have to increase the query count by 1 query per block, because the published_at date can only be found in the publish log, which must be fetched individually for each block.
@@ -441,7 +441,7 @@ def test_build_library_object_tree(self) -> None: | |||
""" | |||
Test if we can export a library | |||
""" | |||
with self.assertNumQueries(8): | |||
with self.assertNumQueries(11): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My initial changes to LibraryXBlockMetadata.from_component
brought this query count up to 14, because it used component.versioning.latest
.
Using component.versioning.draft
instead brought the query count down to 11, because the authoring API doesn't give us a more efficient way to fetch the published_at
date than from the last_publish_log
.
Welcome any advice on how to avoid the increased query count here!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like an @ormsbee question if I'm correctly understanding that it's about efficiently fetching modified dates from learning core's authoring
API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pomegranited Great work on this! I've tested it out with the frontend and it's working quite well. I just left a comment the issue with updating the index when updating the component in the library. I shared an idea, hopefully it might help solve it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 👍
Thank you for your work @pomegranited!
- I tested this using the instructions from feat: adds sort widget to search manager and library component page [FC-0059] frontend-app-authoring#1147
- I read through the code
-
I checked for accessibility issues - Includes documentation
Note: While testing this, I found an issue (not related) updating my index. I create a PR to circumvent this:
@pomegranited 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future. |
2U Release Notice: This PR has been deployed to the edX production environment. |
1 similar comment
2U Release Notice: This PR has been deployed to the edX production environment. |
return cls( | ||
usage_key=LibraryUsageLocatorV2( | ||
library_key, | ||
component.component_type.name, | ||
component.local_key, | ||
), | ||
display_name=component.versioning.draft.title, | ||
created=component.created, | ||
modified=component.versioning.draft.created, | ||
last_published=None if last_publish_log is None else last_publish_log.published_at, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @ormsbee 👋
I added this code to record the published date for library components in the Studio search index. However, it doesn't work, seemingly because components don't get publish records recorded when they are published -- only the parent library does.
However, there isn't a date available on the component.versioning.published
ComponentVersion -- the published.created
matches the draft.modified
date after publishing, and doesn't represent the actual date published.
What would be the best way to address this issue?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cf #35195
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't remember if there's a good helper for it at the moment, but I think you can get the datetime when a Component was published by following these relations:
Component.publishable_entity -> PublishableEntity.published -> Published.publish_log_record -> PublishLogRecord.publish_log -> PublishLog. published_at
@pomegranited I think one of the new tests might be flaky: https://github.com/openedx/frontend-app-course-authoring/actions/runs/10166343467/job/28116281335 Might need to wrap the failing line in a |
Description
This PR adds new datetime fields to the serialized library block metadata:
created
,modified
, andlast_published
, and adds support for sorting search results by these dates or the block title/display_name
.Also adds
last_published
to the list of fields we can filter by, so that e.g. sorting by "Recently Published" can also filter out unpublished content.This change should not impact any current users, as content libraries v2 are not yet ready for production use.
Supporting information
Part of openedx/frontend-app-authoring#1038
Testing instructions
See openedx/frontend-app-authoring#1147 for full testing instructions.
Deadline
ASAP -- to meet project deadlines
Other information
These changes can be deployed by running the management command: