Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ansible collections import error #1987

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

hao-yu
Copy link

@hao-yu hao-yu commented Oct 3, 2024

Avoid "duplicate key value violates unique" error
when importing ansible collections.

fixes #1986

Avoid "duplicate key value violates unique" error
when importing ansible collections.

fixes pulp#1986
Comment on lines +124 to +130
instance = super().get_instance(instance_loader, row)
if not instance and row["is_highest"] == "1":
CollectionVersion.objects.filter(
collection_id=row["collection"], is_highest=True
).update(is_highest=False)

return instance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh damn it! is_highest really needs to go away.
Thank you for finding this.
I think however, we cannot be sure that when importing a highest collection version, that there is no higher one in the system already. Am I right, that you don't actually care about the is_highest flag? And this seems to fix the issue with the database constraint?
I guess we could also just lift the constraint, if we realize we want to accept inconsistent data here.

@mdellweg
Copy link
Member

mdellweg commented Oct 7, 2024

Speaking with the ansible folks, they don't seem to care about that field anymore either.
That means we can safely remove it.
The question here remains to be "Do you need a backportable version of a fix?"

@mdellweg
Copy link
Member

mdellweg commented Oct 8, 2024

Hey, can you have a try if this magically solves the issue for you too?
#1992

@hao-yu
Copy link
Author

hao-yu commented Oct 14, 2024

If I understand correctly, this fix will not be merged because PR #1992 removed the is_highest. However, #1992 will not be backported to the older versions.

@mdellweg
Copy link
Member

Yeah, I'm thinking backporting a fix that removes the field from the api's is probably a bad thing to do.
About your fix (that runs one additional sql update query per impoted row,) my fear is that it slows down import dramatically. Can we somehow just not read that column on import? (Or always set it to False?)
That'd be a fix we can add to the supported branches.
On which branches do you want to see a fix?

@hao-yu
Copy link
Author

hao-yu commented Oct 24, 2024

About your fix (that runs one additional sql update query per impoted row,) my fear is that it slows down import dramatically.

Not really. As far as I see, it will run only one additional sql update query per collection.

Can we somehow just not read that column on import? (Or always set it to False?)

I think it is ok if we are sure we don't need it? I am not exactly sure what is depending on this flag. I didn't check if Satellite needs it or not.

On which branches do you want to see a fix?

Satellite 6.15 (pulp ansible 0.20.8) and Satellite 6.16 (pulp ansible 0.21.8 i think) ?

@mdellweg
Copy link
Member

Satellite 6.15 (pulp ansible 0.20.8) and Satellite 6.16 (pulp ansible 0.21.8 i think) ?

Thank you. (So 0.20 and 0.21 branches...)

@mdellweg
Copy link
Member

This is my proposal for a fix. WDYT?
Would you mind trying that one too?
#1999

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failed to import ansible collections with "duplicate key value violates unique" error
2 participants