Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Small refactor to attempt reconciliation code re-use between Azure & AWS integrations #47427

Merged
merged 8 commits into from
Oct 15, 2024

Conversation

mvbrock
Copy link
Contributor

@mvbrock mvbrock commented Oct 10, 2024

This PR simplifies some of the TAG AWS synchronization resource reconciliation for re-use with Azure integration. Specifically it introduces an wrapping function which wraps the original AWS item into the AWSResource type.

The eventual goal is to provide some simple upsert/delete reconciliation functionality that can be used across AWS, Azure, GCP, etc. While this PR does not directly accomplish this goal, it reduces some golang lines, and simplifies the reconciliation towards a basic process:

  1. If there are no new items, add all old items to deleted resources and return
  2. If there are no old items, add all new items to upserted resources and return
  3. Map old and new items by a unique resource key
  4. Append new/modified items to the upserted resources, using the resource key and an equality check
  5. Append removed items to the deleted resources, using the resource key
  6. Return the upserted and deleted resources

Copy link

The PR changelog entry failed validation: Changelog entry not found in the PR body. Please add a "no-changelog" label to the PR, or changelog lines starting with changelog: followed by the changelog entries for the PR.

@mvbrock mvbrock added the no-changelog Indicates that a PR does not require a changelog entry label Oct 10, 2024
@mvbrock mvbrock requested a review from jakule October 10, 2024 14:45
@mvbrock
Copy link
Contributor Author

mvbrock commented Oct 11, 2024

@tigrato @jakule ping

lib/srv/discovery/fetchers/aws-sync/reconcile.go Outdated Show resolved Hide resolved
lib/srv/discovery/fetchers/aws-sync/reconcile.go Outdated Show resolved Hide resolved
lib/srv/discovery/fetchers/aws-sync/reconcile.go Outdated Show resolved Hide resolved
Copy link
Contributor

@tigrato tigrato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve the PR because it simplifies the code but I don't think we should use this for Aws/Azure as some of the fields are AWS only and should be kept that way.
Feel free to merge once you address comments

@mvbrock
Copy link
Contributor Author

mvbrock commented Oct 15, 2024

I don't think we should use this for Aws/Azure as some of the fields are AWS only and should be kept that way.

Agree--I didn't mean to say we should re-use fields or anything like that. I originally had a reusable approach with an additional generic parameter and a few more functions passed as parameters, but it was ugly/complex. If I can't figure out a nice/simple way to make this reusable, then at least there's less lines to copy/borrow for Azure.

@mvbrock mvbrock enabled auto-merge October 15, 2024 15:39
Copy link
Collaborator

@r0mant r0mant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot.

@mvbrock mvbrock added this pull request to the merge queue Oct 15, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 15, 2024
@mvbrock mvbrock added this pull request to the merge queue Oct 15, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 15, 2024
@mvbrock mvbrock added this pull request to the merge queue Oct 15, 2024
Merged via the queue into master with commit 7f27431 Oct 15, 2024
39 checks passed
@mvbrock mvbrock deleted the mvbrock/pre-azure-reconciliation branch October 15, 2024 21:51
mvbrock added a commit that referenced this pull request Oct 16, 2024
…AWS integrations (#47427)

* A little refactoring to attempt resuing reconciliation code between Azure & AWS integrations

* Use "Wrap" instead of "Insert"

* Removing my overly complex generic reconciler for now

* A little more cleanup

* PR feedback

* Linting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discovery no-changelog Indicates that a PR does not require a changelog entry size/sm
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants