Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Stage connector batch processing sync state per instance #573

Open
lpalashevski opened this issue Jul 13, 2021 · 0 comments
Open

Data Stage connector batch processing sync state per instance #573

lpalashevski opened this issue Jul 13, 2021 · 0 comments

Comments

@lpalashevski
Copy link
Contributor

Current connector implementation stores batch processing state (lastSyncDate) based on the unique key (governance rule name) that is composed of the configuration parameters for given proxy instance.

Example:

"Job metadata will be synced through Egeria (JOB_LEVEL) for projects: [MY_ETL_PROJECT]"

This does not allow uses (for advanced deployment) to run and sync multiple independed proxy instances.

One possible improvement will be use metadataCollectionId from the proxy repository as key (or append it to the existing composite key / governance rule name).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant