Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First draft of the git integration #90

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions accepted/00000-git-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
- Feature Name: Git integration
- Start Date: 2024-04-23

# Summary
[summary]: #summary

This is a feature which will add a git integration to Uyuni without the need to use gitfs. It would be possible to sync formulas and playbooks and use them with an existent infrastructure.

# Motivation
[motivation]: #motivation

- Why are we doing this?
The git integration is not existent or it the depends from gitfs which is unstable

- What use cases does it support?
For all users which have a lot of self-written salt formulas and ansible-playbooks.
For all users that have multiple git repositories and branches which would like to deploy based on environment(env for ansible should be taken from the inventory file)

- What is the expected outcome?
Use salt formulas and ansible playbooks which are stored on git repositories
Use them against clients managed by Uyuni


# Detailed design
[design]: #detailed-design

It should be designed on four parts:

The first part would be done by a container that is responsible to sync the git repositories and sort it out per type of the provisioning tool.
- If that is a ansible playbook it would be scanned(find $path for a role infrastructure)
- If that is a salt formula it should be done by the states delivered by the repository
- Path structure:
- Ansible: $local/ansible/roles, $local/ansible/collections, $local/ansible/inventory, $local/ansible/config
- Salt: $local/salt/states
- All the data would be saved in a volume which would be shared with the suma container - e.g. $provisioning
- A customer could add multiple git repositories and multiple branches - it would make easier to deploy formulas / playbooks based in a environment, like LCM do with repositories and promotion

The second part it would be done for a md5/sha256sum check. If possible, also a scan that could be done for a container with e.g. clamav to check if there any viroses / trojans being imported into the clients being managed by Uyuni.

The third part will be dependent from the the first two parts. If everything runs right that would be processed either by the salt master from Uyuni or the ansible control node - so here will probably smart to get rid of the manual configuration from the ansible playbooks and inventories and parse the configuration which will be delivered by the shared volume.

The fourth part would be a normalization from this content from the user and commit it to a different repository that would be used by Uyuni. The motivation here is to have a standard path infrastructure that makes easy the onboarding on Uyuni.


# Drawbacks
[drawbacks]: #drawbacks

Why should we **not** do this?

* obscure corner cases - Users which do not have any qualíty control on their provisioning tools, importing from malicious playbooks / states from a public git repository
* will it impact performance? No, it will accelerate the adoption from Uyuni for users with and existent provisioning structure
* what other parts of the product will be affected? Salt-Master
* will the solution be hard to maintain in the future? No, it would not. It should be delivered as a module that anytime could be activated. If the customer does not have nothing in a git repository he will not activate this feature. It also not be possible to activate without git(github, gitlab....),git_repo, branch and auth_details. Public repositories on github will not be accepted as a security measure(to avoid malicious formulas / playbooks into Uyuni and the linux infrastructure).
If a public repository will be accepted, it should appear a warning about it.

# Alternatives
[alternatives]: #alternatives

- What other designs/options have been considered? No.
- What is the impact of not doing this? In a modern approach on almost every playbook / salt state is in a git repository and the automation is needed to manage tons of linux clients

# Unresolved questions
[unresolved]: #unresolved-questions

- What are the unknowns?
- What can happen if Murphy's law holds true? Deactivate the module and fix it.
63 changes: 63 additions & 0 deletions accepted/00102-git-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
- Feature Name: Git integration
- Start Date: 2024-04-23

# Summary
[summary]: #summary

This is a feature which will add a git integration to Uyuni without the need to use gitfs. It would be possible to sync formulas and playbooks and use them with an existent infrastructure.

# Motivation

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think it would play nicely with EDA (uyuni-project/uyuni#7801)

[motivation]: #motivation

- Why are we doing this?
The git integration is not existent or it the depends from gitfs which is unstable

- What use cases does it support?
For all users which have a lot of self-written salt formulas and ansible-playbooks.

- What is the expected outcome?
Use salt formulas and ansible playbooks which are stored on git repositories
Use them against clients managed by Uyuni


# Detailed design

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I think is missing: Such a solution should include that customers can add N git repositories. It is pretty unlikely that everything is in one large mono-repo.

[design]: #detailed-design

It should be designed on four parts:

The first part would be done by a container that is responsible to sync the git repositories and sort it out per type of the provisioning tool.
- If that is a ansible playbook it would be scanned(find $path for a role infrastructure)
- If that is a salt formula it should be done by the states delivered by the repository
- Path structure:
- Ansible: $local/ansible/roles, $local/ansible/inventory, $local/ansible/config

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should also cover collections? $local/ansible/collections

- Salt: $local/salt/states, $local/ansible/config

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would someone have a ansible config in a salt formula repo?

- All the data would be saved in a volume which would be shared with the suma container - e.g. $provisioning

The second part it would be done for a md5/sha256sum check. If possible, also a scan that could be done for a container with e.g. clamav to check if there any viroses / trojans being imported into the clients being managed by Uyuni.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The checksum part sounds like a good candidate to add (optional) ansible-sign support: https://ansible.readthedocs.io/projects/sign/en/latest/.
Not sure about the clamav part though. This seems uncommon when synching repo content.


The third part will be dependent from the the first two parts. If everything runs right that would be processed either by the salt master from Uyuni or the ansible control node - so here will probably smart to get rid of the manual configuration from the ansible playbooks and inventories and parse the configuration which will be delivered by the shared volume.

The fourth part would be a normalization from this content from the user and commit it to a different repository that would be used by Uyuni. The motivation here is to have a standard path infrastructure that makes easy the onboarding on Uyuni.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's not Uyuni's goal to be an AWX clone. But to be honest, this problem was already kind of solved so the logic how to sync from git could follow a similar approach as described in https://ansible.readthedocs.io/projects/awx/en/latest/userguide/projects.html#scm-types-git-and-subversion



# Drawbacks
[drawbacks]: #drawbacks

Why should we **not** do this?

* obscure corner cases - Users which do not have any qualíty control on their provisioning tools, importing from malicious playbooks / states from a public git repository

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be fair, those would also be the case outside Uyuni then and is not in control of the project. I wouldn't list it as "Why should we not do this".

* will it impact performance? No, it will accelerate the adoption from Uyuni for users with and existent provisioning structure

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will it impact performance: yes
Git sync takes additional time, this is some sort of performance impact. What you describe is the Customer benefit. I agree that this can and will outweighed the slight performance impact.

* what other parts of the product will be affected? Salt-Master

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly sure how Salt-Master is impacted? Doesn't it, at the end, look exactly the same for Salt because you plan to sync into a local folder/shared volume? Salt doesn't really care where the formulas or playbooks come from right?

* will the solution be hard to maintain in the future? No, it would not. It should be delivered as a module that anytime could be activated. If the customer does not have nothing in a git repository he will not activate this feature. It also not be possible to activate without git(github, gitlab....),git_repo, branch and auth_details. Public repositories on github will not be accepted as a security measure.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Public repositories on github will not be accepted as a security measure. disagree with that. There is no reason to force customers to authenticate against a private repo. Public repositories are fine and a valid use-case.


# Alternatives
[alternatives]: #alternatives

- What other designs/options have been considered? No.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say, there are not much other options to consider in this specific case. The idea is to allow customers to manage salt formulas / ansible playbooks in git repositories that are then used by Uyuni. Implementation details can be further fleshed out but the overall idea is pretty straight forward.

- What is the impact of not doing this? It will make more difficult for teams to adopt the product - in a modern approach on almost everym playbook / salt state is in a private repository and the automation is needed to manage tons of linux clients

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not, they would likely continue to use the other Uyuni features but keep the Automation part separate from it. So the proposed feature will drive adoption of Uyuni for Automation tasks, but unlikely that someone decide to use Uyuni only because of this. It's not a black/white thing.


# Unresolved questions
[unresolved]: #unresolved-questions

- What are the unknowns?
- What can happen if Murphy's law holds true? Deactivate the module and fix it.