Introduce proper formatting on python ,bash and yaml files #2774

hansinikarunarathne · 2024-06-28T06:04:26Z

Pull Request Template for Kubeflow manifests Issues

Please include a summary of changes and the related issue.
List any dependencies that are required for this change.
Please delete the options that are not relevant.
The following checklist will help you to satisfy the requirements.

✏️ A brief description of the changes

I add black GitHub action to format the python files
I used yamllint to format the YAML files. Created a GitHub pipeline to automate finding format errors in YAML files.
I used shellcheck to format the bash files. Created a Github pipeline out of it

🐛 If this PR is related to an issue, please put the link of the issue here.

#2773

✅ Contributor checklist

All the commits have been signed-off (To pass the DCO check)
Submit the Contributor License Agreements (To pass the cla/google check)

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

juliusvonkohout · 2024-06-30T09:43:19Z

Please rebase to master to fix the conflict with trivy_scan.sh and please only scan files that are changed in pull requests and are in the folders /common and /example for now.

juliusvonkohout · 2024-06-30T09:49:54Z

The error messages should also guide developers on how to fix it in their branch to pass the test.

juliusvonkohout · 2024-07-01T16:59:41Z

I think you can fix the python and bash scripts in this PR.

hansinikarunarathne · 2024-07-01T17:06:05Z

I think you can fix the python and bash scripts in this PR.

did not understand what you meant?

juliusvonkohout · 2024-07-01T17:17:01Z

I think you can fix the python and bash scripts in this PR.

did not understand what you meant?

There are a lot of warnings about bash and python files. Most of them are maintained directly in this repository. So you can just fix them with the formatting/linting changes and add them here to this PR. Everything in /hack should be fixed for example.

hansinikarunarathne · 2024-07-01T18:15:08Z

I think you can fix the python and bash scripts in this PR.

did not understand what you meant?

There are a lot of warnings about bash and python files. Most of them are maintained directly in this repository. So you can just fix them with the formatting/linting changes and add them here to this PR. Everything in /hack should be fixed for example.

Okay I will do that. And Do I need to run this action only in the files change during the PR ?

juliusvonkohout · 2024-07-01T19:17:54Z

Yes we should only check files that are changed in the PR. So PR by PR stuff will be cleaned up.

And I think we should exclude /apps in general.

Do you have something to check markdown as well?

hansinikarunarathne · 2024-07-01T22:16:48Z

Yes we should only check files that are changed in the PR. So PR by PR stuff will be cleaned up.

And I think we should exclude /apps in general.

Do you have something to check markdown as well?

I di the requested changes. After the check, I will add guidelines to find respective errors in the workflow. Can you give me update regarding the state ? @juliusvonkohout

There are linters for markdowns as well. markdown_linters

apps/kfp-tekton/upstream/base/installs/multi-user/pipelines-profile-controller/run_tests.sh

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

… sh files to address the formattings according to shellcheck Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

…matter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

juliusvonkohout · 2024-07-09T14:51:46Z

/lgtm
/approve

we have to test this in practice

google-oss-prow · 2024-07-09T14:51:52Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…2774) * Add balck github action to format python files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Formatted python files from the balck formatter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add github action to format yaml files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add step YAML Formatting Guidelines to yaml_formatter.yaml file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made chnages to run next steps although the previous step fail Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * switch steps Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * added shellCheck for bash formatting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Changed code only to lint yaml files inside the common and example folder Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Changed main to master Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Run yamllint on files which is chnaged in PR only Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Changed 'Proper formatting on python files' github workflow only to run for python files in common and example folder Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add shellcheckrc file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Disable SC1017,SC2086,SC2070 rules when shellcheck and did changes in sh files to address the formattings according to shellcheck Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made chnages in sh files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made chnages in sh files to address formattings Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made chnages in sh files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * disable SC2046 rule Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * disable SC2006 and SC2155 rule Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Change shellcheck only to run for PR chnaged files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Address a issue with bash_formatter.yaml Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Changed origin/main to origin/master Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Formatted python files from the balck formatter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add github action to format yaml files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add step YAML Formatting Guidelines to yaml_formatter.yaml file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made chnages to run next steps although the previous step fail Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * switch steps Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add shellcheckrc file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Disable SC1017,SC2086,SC2070 rules when shellcheck and did changes in sh files to address the formattings according to shellcheck Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made chnages in sh files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Change shellcheck only to run for PR chnaged files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Ensure the full history is fetched Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Ensure the full history is fetched in bash_formatter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * fix SC2148 (error) Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * remove commented lines in bash_formatter.yaml Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * remove formatting chnages done to app folder content Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add guidlines to how to format python files according to black formatter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add guidlines to how to format bash files according to shellcheck formatter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

google-oss-prow bot added the size/S label Jun 28, 2024

google-oss-prow bot requested review from juliusvonkohout and kimwnasptd June 28, 2024 06:04

hansinikarunarathne closed this Jun 28, 2024

hansinikarunarathne force-pushed the feature-workflow-to-formatting-python-and-yml-files branch from 82f6b06 to 787cb98 Compare June 28, 2024 06:07

google-oss-prow bot added size/XS and removed size/S labels Jun 28, 2024

hansinikarunarathne reopened this Jun 28, 2024

google-oss-prow bot added size/S size/XXL and removed size/XS size/S labels Jun 28, 2024

hansinikarunarathne force-pushed the feature-workflow-to-formatting-python-and-yml-files branch from a3e448d to 6456723 Compare July 1, 2024 08:42

hansinikarunarathne force-pushed the feature-workflow-to-formatting-python-and-yml-files branch 2 times, most recently from ce6f35e to a3ea293 Compare July 1, 2024 21:57

juliusvonkohout self-assigned this Jul 2, 2024

juliusvonkohout reviewed Jul 2, 2024

View reviewed changes

apps/kfp-tekton/upstream/base/installs/multi-user/pipelines-profile-controller/run_tests.sh Outdated Show resolved Hide resolved

google-oss-prow bot added size/XL and removed size/XXL labels Jul 7, 2024

hansinikarunarathne requested a review from juliusvonkohout July 7, 2024 08:28

Add balck github action to format python files

590a0ee

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

hansinikarunarathne added 22 commits July 7, 2024 14:02

made chnages in sh files

58ee8cd

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

disable SC2046 rule

399e0a3

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

disable SC2006 and SC2155 rule

3c4c985

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Change shellcheck only to run for PR chnaged files

35b242f

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Address a issue with bash_formatter.yaml

bc5fd0b

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Changed origin/main to origin/master

cb67c39

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Formatted python files from the balck formatter

77e2c25

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Add github action to format yaml files

8f252bb

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Add step YAML Formatting Guidelines to yaml_formatter.yaml file

045b504

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

made chnages to run next steps although the previous step fail

19881d4

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

switch steps

04e1858

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Add shellcheckrc file

8ee988d

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Disable SC1017,SC2086,SC2070 rules when shellcheck and did changes in…

0f27d46

… sh files to address the formattings according to shellcheck Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

made chnages in sh files

5e5e4c4

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Change shellcheck only to run for PR chnaged files

859ced5

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Ensure the full history is fetched

107db8e

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Ensure the full history is fetched in bash_formatter

64bd2d9

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

fix SC2148 (error)

0226a0e

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

remove commented lines in bash_formatter.yaml

a4bc63f

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

remove formatting chnages done to app folder content

2730efc

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Add guidlines to how to format python files according to black formatter

5e7dfb5

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

Add guidlines to how to format bash files according to shellcheck for…

9ae743f

…matter Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

hansinikarunarathne force-pushed the feature-workflow-to-formatting-python-and-yml-files branch from ee3415d to 9ae743f Compare July 7, 2024 08:33

google-oss-prow bot added the lgtm label Jul 9, 2024

google-oss-prow bot added the approved label Jul 9, 2024

google-oss-prow bot merged commit 9b22cdc into kubeflow:master Jul 9, 2024
8 checks passed

juliusvonkohout linked an issue Jul 10, 2024 that may be closed by this pull request

Enforce proper formatting on python, bash and yaml files #2773

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce proper formatting on python ,bash and yaml files #2774

Introduce proper formatting on python ,bash and yaml files #2774

hansinikarunarathne commented Jun 28, 2024 •

edited

Loading

juliusvonkohout commented Jun 30, 2024

juliusvonkohout commented Jun 30, 2024

juliusvonkohout commented Jul 1, 2024

hansinikarunarathne commented Jul 1, 2024

juliusvonkohout commented Jul 1, 2024 •

edited

Loading

hansinikarunarathne commented Jul 1, 2024

juliusvonkohout commented Jul 1, 2024 •

edited

Loading

hansinikarunarathne commented Jul 1, 2024 •

edited

Loading

juliusvonkohout commented Jul 9, 2024

google-oss-prow bot commented Jul 9, 2024

Introduce proper formatting on python ,bash and yaml files #2774

Introduce proper formatting on python ,bash and yaml files #2774

Conversation

hansinikarunarathne commented Jun 28, 2024 • edited Loading

Pull Request Template for Kubeflow manifests Issues

✏️ A brief description of the changes

🐛 If this PR is related to an issue, please put the link of the issue here.

✅ Contributor checklist

juliusvonkohout commented Jun 30, 2024

juliusvonkohout commented Jun 30, 2024

juliusvonkohout commented Jul 1, 2024

hansinikarunarathne commented Jul 1, 2024

juliusvonkohout commented Jul 1, 2024 • edited Loading

hansinikarunarathne commented Jul 1, 2024

juliusvonkohout commented Jul 1, 2024 • edited Loading

hansinikarunarathne commented Jul 1, 2024 • edited Loading

juliusvonkohout commented Jul 9, 2024

google-oss-prow bot commented Jul 9, 2024

hansinikarunarathne commented Jun 28, 2024 •

edited

Loading

juliusvonkohout commented Jul 1, 2024 •

edited

Loading

juliusvonkohout commented Jul 1, 2024 •

edited

Loading

hansinikarunarathne commented Jul 1, 2024 •

edited

Loading