-
Notifications
You must be signed in to change notification settings - Fork 885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
minimize the RBAC permissions for the pull mode cluster #5793
base: master
Are you sure you want to change the base?
Conversation
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
f69f61d
to
2d45bc6
Compare
Local validation: $ karmadactl register 172.18.0.6:32543 --token ak0bng.1mde96apjyi98dwz --discovery-token-ca-cert-hash sha256:c31ef70196f7b28545526c94a3c62ce74ad9c476971a818e16f49396f50b9430 --kubeconfig ~/.kube/member4.config --context member4 --karmada-agent-image docker.io/karmada/karmada-agent:latest --cluster-name member4 -v 4
I1107 20:23:59.964757 3098581 register.go:331] Registering cluster. cluster name: member4
I1107 20:23:59.964796 3098581 register.go:332] Registering cluster. cluster namespace: karmada-cluster
[preflight] Running pre-flight checks
I1107 20:23:59.964814 3098581 register.go:491] Validating the existence of file /etc/karmada/bootstrap-karmada-agent.conf
I1107 20:23:59.964827 3098581 register.go:491] Validating the existence of file /etc/karmada/karmada-agent.conf
I1107 20:23:59.964836 3098581 register.go:491] Validating the existence of file /etc/karmada/pki/ca.crt
[preflight] All pre-flight checks were passed
[karmada-agent-start] Waiting to perform the TLS Bootstrap
I1107 20:23:59.976730 3098581 register.go:1155] [discovery] Created cluster-info discovery client, requesting info from "172.18.0.6:32543"
I1107 20:23:59.988138 3098581 register.go:1193] [discovery] Requesting info from "172.18.0.6:32543" again to validate TLS against the pinned public key
I1107 20:23:59.997904 3098581 register.go:1210] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "172.18.0.6:32543"
I1107 20:23:59.997940 3098581 register.go:506] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I1107 20:23:59.997958 3098581 register.go:517] [discovery] writing bootstrap karmada-agent config file at /etc/karmada/bootstrap-karmada-agent.conf
I1107 20:23:59.998535 3098581 register.go:526] [discovery] writing CA certificate at /etc/karmada/pki/ca.crt
I1107 20:24:00.461989 3098581 register.go:866] Waiting for the client certificate agent-rbac-generator-member4wcq4v to be issued
I1107 20:24:01.465729 3098581 register.go:874] Signing certificate of csr agent-rbac-generator-member4wcq4v successfully
[karmada-agent-start] Waiting to check cluster exists
[karmada-agent-start] Assign the necessary RBAC permissions to the agent
[karmada-agent-start] Waiting to construct karmada-agent kubeconfig
I1107 20:24:03.665759 3098581 register.go:866] Waiting for the client certificate member4-nn9h2 to be issued
I1107 20:24:04.669051 3098581 register.go:874] Signing certificate of csr member4-nn9h2 successfully
I1107 20:24:04.669084 3098581 register.go:908] writing bootstrap karmada-agent config file at /etc/karmada/karmada-agent.conf
[karmada-agent-start] Waiting the necessary secret and RBAC
I1107 20:24:04.680320 3098581 idempotency.go:68] Secret karmada-system/karmada-kubeconfig has been created or updated.
I1107 20:24:04.683534 3098581 idempotency.go:188] ClusterRole karmada-agent has been created or updated.
I1107 20:24:04.691452 3098581 idempotency.go:212] ClusterRolebinding karmada-agent has been created or updated.
[karmada-agent-start] Waiting karmada-agent Deployment
cluster(member4) is joined successfully
$ kubectl get cluster
NAME VERSION MODE READY AGE
member1 v1.31.0 Push True 3d10h
member2 v1.31.0 Push True 6d6h
member3 v1.31.0 Pull True 32h
member4 v1.31.0 Pull True 34s
$ karmadactl apply -f samples/nginx/deployment.yaml --cluster member4
deployment.apps/nginx created
propagationpolicy.policy.karmada.io/nginx-6d7f8d5f5b unchanged
$ kubectl --kubeconfig ~/.kube/member4.config get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 2/2 2 2 11s
$ karmadactl unregister member4 --cluster-kubeconfig ~/.kube/member4.config -v=4
I1107 20:26:53.269394 3100053 unregister.go:186] Unregistering cluster. cluster name: member4
I1107 20:26:53.269597 3100053 unregister.go:187] Unregistering cluster. karmada-agent deployed in namespace: karmada-system
I1107 20:26:53.269608 3100053 unregister.go:188] Unregistering cluster. member cluster secrets stored in namespace: karmada-cluster
I1107 20:26:56.314363 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:26:57.317515 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:26:58.310547 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:26:59.310182 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:00.309665 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:01.314053 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:02.311988 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:03.309908 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:04.310080 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:05.309887 3100053 cluster.go:64] Waiting for the cluster object member4 to be deleted
I1107 20:27:06.312824 3100053 unregister.go:333] Successfully delete cluster object (member4) from control plane.
W1107 20:27:06.312863 3100053 unregister.go:342] The RBAC resources on the control plane need to be manually cleaned up, including the following resources:
ClusterRole/system:karmada:member4:agent
ClusterRoleBinding/system:karmada:member4:agent
Role/karmada-cluster/system:karmada:member4:agent-secret
Role/karmada-es-member4/system:karmada:member4:agent-work
RoleBinding/karmada-cluster/system:karmada:member4:agent-secret
RoleBinding/karmada-es-member4/system:karmada:member4:agent-work
I1107 20:27:06.316944 3100053 unregister.go:366] Successfully delete resource ({ServiceAccount karmada-member4 karmada-cluster}) from member cluster (member4).
I1107 20:27:06.320891 3100053 unregister.go:366] Successfully delete resource ({ClusterRole karmada-controller-manager:karmada-member4 }) from member cluster (member4).
I1107 20:27:06.323882 3100053 unregister.go:366] Successfully delete resource ({ClusterRoleBinding karmada-controller-manager:karmada-member4 }) from member cluster (member4).
I1107 20:27:06.325671 3100053 unregister.go:366] Successfully delete resource ({Secret karmada-member4 karmada-cluster}) from member cluster (member4).
I1107 20:27:06.328792 3100053 unregister.go:366] Successfully delete resource ({ServiceAccount karmada-impersonator karmada-cluster}) from member cluster (member4).
I1107 20:27:06.332187 3100053 unregister.go:366] Successfully delete resource ({Secret karmada-impersonator karmada-cluster}) from member cluster (member4).
I1107 20:27:06.335267 3100053 unregister.go:366] Successfully delete resource ({Namespace karmada-cluster }) from member cluster (member4).
I1107 20:27:06.339246 3100053 unregister.go:366] Successfully delete resource ({Deployment karmada-agent karmada-system}) from member cluster (member4).
I1107 20:27:06.342516 3100053 unregister.go:366] Successfully delete resource ({ServiceAccount karmada-agent-sa karmada-system}) from member cluster (member4).
I1107 20:27:06.346048 3100053 unregister.go:366] Successfully delete resource ({ClusterRole karmada-agent }) from member cluster (member4).
I1107 20:27:06.350296 3100053 unregister.go:366] Successfully delete resource ({ClusterRoleBinding karmada-agent }) from member cluster (member4).
I1107 20:27:06.354519 3100053 unregister.go:366] Successfully delete resource ({Secret karmada-kubeconfig karmada-system}) from member cluster (member4).
I1107 20:27:06.354622 3100053 unregister.go:382] Successfully delete local file (/etc/karmada/karmada-agent.conf) in current node.
I1107 20:27:06.354666 3100053 unregister.go:382] Successfully delete local file (/etc/karmada/pki/ca.crt) in current node. |
2d45bc6
to
afd687a
Compare
Signed-off-by: zhzhuang-zju <m17799853869@163.com>
afd687a
to
5f781cd
Compare
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #5793 +/- ##
==========================================
+ Coverage 42.41% 42.48% +0.06%
==========================================
Files 656 657 +1
Lines 55884 56178 +294
==========================================
+ Hits 23705 23867 +162
- Misses 30659 30786 +127
- Partials 1520 1525 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
What type of PR is this?
/kind feature
What this PR does / why we need it:
Currently, pull-mode clusters interact with
karmada-apiserver
on the control plane via a mountedkarmada-kubeconfig
, where the certificate has Organization assystem:nodes
and CN assystem:node:{cluser_name}
, so karmada -agent has the RBAC permissions on the control plane of a user with Group assystem:nodes
and username assystem:node:{cluser_name}
.Since the control plane binds the clusterrole to Group
system:nodes
, all karmada-agents actually have the same set of RBAC permissions.The clusterrole bound to Group
system:nodes
has too many permissions, e.g. secret permissions, cluster permissions.So in order to minimise RBAC permissions, each karmada-agent needs to be assigned separate RBAC permissions
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Installation methods involved:
Does this PR introduce a user-facing change?: