Add kubelet instance configuration to configure CRI socket for each node

[HirazawaUi]

Is this a BUG REPORT or FEATURE REQUEST?

FEATURE REQUEST

Anything else we need to know?

This issue wants to remove container runtime interface (CRI) socket annotations from Node objects in Kubernetes and no longer added --container-runtime-endpoint args in kubeadm-flags.env ( --container-runtime-endpoint args deprecated in kubelet), this annotation and args are used to specify the CRI socket endpoint on each node that the kubelet uses to communicate with the container runtime.

Add kubelet-instance-config.yml locally to store container runtime interface (CRI) sockets, and in kubeadm init, join and upgrade, override the global kubelet configuration through kubelet-instance-config.yml.

More detailed design at: https://hackmd.io/@r2Yq-PVoRcK2OaGypnwPMg/rJFChACCp/edit

Tasks

Beta Give feedback

No tasks being tracked yet.

---

edit by neolit123:

1.32 alpha

• KEP:
  • kubeadm: Add kubelet instance configuration to configure CRI socket for each node enhancements#4658
  • Add kubelet instance configuration to configure CRI socket for each node #3042
• k/k PRs:
  • Add kubelet instance configuration to configure CRI socket for each node enhancements#4654
• e2e tests
  • TODO
• k/website changes:
  • kubeadm: add 1.32 documentation website#48408

[HirazawaUi]

/cc @neolit123 @pacoxu

[neolit123]

@HirazawaUi we need a KEP for this, ket's see what @pacoxu can comment about the previous work done on this.

@pacoxu can you please post links to your previous KEP work.?
also didn't we already have a tracking issue here in k/kubeadm?

maybe we can close KEP PRs and let @HirazawaUi to take over.

[neolit123]

also are you willing to work on this @HirazawaUi
there are a lot of things to update around this change across multiple releases. and i don't think me or anyone else will have a lot of time for it.

• KEP design how is it going to work exactly, do we need a feature gate
• k8s.io docs update
• k/k code update
• do we need a e2e test in k/kubeadm

too many questions

[HirazawaUi]

also are you willing to work on this @HirazawaUi there are a lot of things to update around this change across multiple releases. and i don't think me or anyone else will have a lot of time for it.

• KEP design how is it going to work exactly, do we need a feature gate
• k8s.io docs update
• k/k code update
• do we need a e2e test in k/kubeadm

too many questions

Yes, I'll try to finish it.

[HirazawaUi]

@pacoxu What do you think about this? I would love to hear your comments :)

[pacoxu]

The KEP

• kubeadm: not use CRI socket annotation on the Node object  enhancements#3929
• kubeadm: init kep for no cri socket annotation enhancements#3930

My current proposal would be something like kubernetes/enhancements#3930 (comment)

i meant a generic patches approach where a local config file overrides the global config downloaded from the cm. not the --patches feature.

this local file must be stored somewhere. perhaps in the same dir as config.yaml, but called config-instance.yaml. IMO it has a number of tricky aspects that need to be covered in the design doc for init, join, upgrade.

Above is @neolit123 your proposal in comment here https://github.com/kubernetes/enhancements/pull/3930/files#r1177682829.

I prefer this solution.

BTW, /var/lib/kubelet/kubeadm-flags.env will only have --container-runtime-endpoint later. I would like to make this cri-socket-annotation and runtime endpoint configuration to config-instance.yaml under /var/lib/kubelet.

[HirazawaUi]

Thanks, I have missed this comment, I will resurrect kep as soon as possible.

[neolit123]

note if this ever becomes part of kubeadm, we can close this ticket:
#1924

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[neolit123]

/remove-lifecycle stale

[neolit123]

@HirazawaUi i added some TODOs a tracking in the issue description here.
please keep it updated with links - ....

[HirazawaUi]

@HirazawaUi i added some TODOs a tracking in the issue description here. please keep it updated with links - ....

OK, thank you so much for your help.