Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(backend): mount EmptyDir volumes for launcher write locations #10857

Merged
merged 1 commit into from
Jun 6, 2024

Conversation

HumairAK
Copy link
Collaborator

@HumairAK HumairAK commented May 31, 2024

This is a shared pr with @gregsheremeta, picking this pr up from here due to the expressed urgency by the community to have this issue prioritized.

credit to @gregsheremeta for the changes, I just fixed the test cases and rebased.

Fixes: #5673
Fixes: #7345


Description of your changes:

Comments by @gregsheremeta from the original PR:

Launcher writes input artifacts to root paths /gcs, /minio, and /s3. These paths are not accessible by non-root users by default, which is problematic in locked-down Kubernetes installations and/or OpenShift. #5673 (comment)
Mount an EmptyDir scratch volume to these paths to work around this.
Additionally, /.local and /.cache are written to by pip, so add EmptyDir mounts for those too.

Checklist:

Launcher writes input artifacts to root paths /gcs, /minio, and /s3.
These paths are not accessible by non-root users by default, which is
problematic in locked-down Kubernetes installations and/or OpenShift.
/gcs is currently a contract for KFP v2 python component wrappers, so
the path cannot be changed.

Mount an EmptyDir scratch volume to these paths to work around this.

Additionally, /.local and /.cache are written to by pip, so add
EmptyDir mounts for those too.

Fixes: kubeflow#5673
Fixes: kubeflow#7345
Signed-off-by: Humair Khan <HumairAK@users.noreply.github.com>
Copy link

@HumairAK: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
kubeflow-pipeline-upgrade-test 2c23995 link false /test kubeflow-pipeline-upgrade-test
kubeflow-pipeline-e2e-test 2c23995 link false /test kubeflow-pipeline-e2e-test
kubeflow-pipelines-samples-v2 2c23995 link false /test kubeflow-pipelines-samples-v2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Copy link
Contributor

@hbelmiro hbelmiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@droctothorpe
Copy link
Contributor

/lgtm Thanks for expediting this!

Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chensun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 65839ce into kubeflow:master Jun 6, 2024
5 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants