Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] kbcli kubeblocks upgrade 0.5.3-beta.7-->0.6.0-alpha.22 Failed #3957

Closed
linghan-hub opened this issue Jun 26, 2023 · 3 comments
Closed

[BUG] kbcli kubeblocks upgrade 0.5.3-beta.7-->0.6.0-alpha.22 Failed #3957

linghan-hub opened this issue Jun 26, 2023 · 3 comments
Assignees
@ldming
Labels
bug kind/bug Something isn't working Stale
Milestone
Release 0.6.0

Comments

@linghan-hub
Copy link
Collaborator

linghan-hub commented Jun 26, 2023 •
edited
Loading 

kbcli version
Kubernetes: v1.23.8+k3s1
KubeBlocks: 0.5.3-beta.7
kbcli: 0.5.3-beta.7

kbcli kb upgrade --version=0.6.0-alpha.22
Current KubeBlocks version {0.5.3-beta.7 v1.23.8+k3s1 0.5.3-beta.7}.
Kubernetes version 1.23.8+k3s1
kbcli version 0.5.3-beta.7
Add and update repo kubeblocks                     OK
Upgrading KubeBlocks to 0.6.0-alpha.22             FAIL
error: timed out waiting for the condition

kbcli kb upgrade --version=0.6.0-alpha.22
Current version 0.6.0-alpha.22 is the same as the upgraded version, no need to upgrade.

kbcli version
Kubernetes: v1.23.8+k3s1
KubeBlocks: 0.6.0-alpha.22
kbcli: 0.5.3-beta.7

kubectl get pods -A
NAMESPACE     NAME                                                     READY   STATUS     RESTARTS   AGE
kube-system   coredns-d76bd69b-m7lcp                                   1/1     Running    0          14m
kb-system     kubeblocks-865c748ccc-nqh2w                              1/1     Running    0          14m
kb-system     kb-addon-alertmanager-webhook-adaptor-8698656c95-f4mj9   2/2     Running    0          13m
kb-system     kb-addon-snapshot-controller-649f8b9949-6dr9h            1/1     Running    0          12m
kb-system     kb-addon-prometheus-alertmanager-5dd8b8dd64-9nlx7        2/2     Running    0          13m
kb-system     kb-addon-grafana-579dff9688-kg2tb                        3/3     Running    0          13m
kb-system     kb-addon-prometheus-server-58658f468-5vdq7               2/2     Running    0          13m
kb-system     kb-addon-csi-hostpath-driver-0                           8/8     Running    0          12m
default       mycluster-mysql-0                                        4/4     Running    0          10m
kb-system     kubeblocks-84485f7b5d-dj4tb                              0/1     Init:0/1   0          6m36s

helm list -A
NAME                                 	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART                             	APP VERSION
kb-addon-alertmanager-webhook-adaptor	kb-system	1       	2023-06-26 11:08:52.381169046 +0000 UTC	deployed	alertmanager-webhook-adaptor-0.1.4	0.1.4
kb-addon-apecloud-mysql              	kb-system	2       	2023-06-26 11:15:20.170122086 +0000 UTC	deployed	apecloud-mysql-0.6.0-alpha.22     	8.0.30
kb-addon-csi-hostpath-driver         	kb-system	2       	2023-06-26 11:15:20.204454836 +0000 UTC	deployed	csi-hostpath-driver-0.6.0-alpha.22	v1.11.0
kb-addon-grafana                     	kb-system	1       	2023-06-26 11:08:53.75513288 +0000 UTC 	deployed	grafana-6.43.5                    	9.2.4
kb-addon-mongodb                     	kb-system	2       	2023-06-26 11:15:20.243442795 +0000 UTC	deployed	mongodb-0.6.0-alpha.22            	5.0.14
kb-addon-postgresql                  	kb-system	2       	2023-06-26 11:15:21.01241717 +0000 UTC 	deployed	postgresql-0.6.0-alpha.22         	14.7.2
kb-addon-prometheus                  	kb-system	1       	2023-06-26 11:08:52.791268962 +0000 UTC	deployed	prometheus-15.16.1                	2.39.1
kb-addon-redis                       	kb-system	2       	2023-06-26 11:15:20.64766917 +0000 UTC 	deployed	redis-0.6.0-alpha.22              	7.0.6
kb-addon-snapshot-controller         	kb-system	1       	2023-06-26 11:08:56.595577298 +0000 UTC	deployed	snapshot-controller-1.7.2         	v6.2.1
kubeblocks                           	kb-system	2       	2023-06-26 19:15:15.17043 +0800 CST    	failed  	kubeblocks-0.6.0-alpha.22         	0.6.0-alpha.22

k describe pod kubeblocks-84485f7b5d-dj4tb -n kb-system
Name:             kubeblocks-84485f7b5d-dj4tb
Namespace:        kb-system
Priority:         0
Service Account:  kubeblocks
Node:             k3d-kb-playground-server-0/172.24.0.3
Start Time:       Mon, 26 Jun 2023 19:15:18 +0800
Labels:           app.kubernetes.io/instance=kubeblocks
                  app.kubernetes.io/name=kubeblocks
                  pod-template-hash=84485f7b5d
Annotations:      <none>
Status:           Pending
IP:               10.42.0.25
IPs:
  IP:           10.42.0.25
Controlled By:  ReplicaSet/kubeblocks-84485f7b5d
Init Containers:
  tools:
    Container ID:  containerd://8e57fa90f7e523e83fa38948a9bd67cff33a20ce76f805b5b7c2ac03d2807378
    Image:         registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks-tools:0.6.0-alpha.22
    Image ID:      registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks-tools@sha256:8702295a6aadb3a801b74dd3f1f246b4d723c0a4127fcf969a1fd65cb92669a5
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/true
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 26 Jun 2023 19:23:28 +0800
      Finished:     Mon, 26 Jun 2023 19:23:28 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2rw5j (ro)
Containers:
  manager:
    Container ID:
    Image:         registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks:0.6.0-alpha.22
    Image ID:
    Ports:         9443/TCP, 8081/TCP, 8080/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      --health-probe-bind-address=:8081
      --metrics-bind-address=:8080
      --leader-elect
      --zap-devel=false
      --zap-time-encoding=iso8601
      --zap-encoder=console
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Liveness:       http-get http://:health/healthz delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:      http-get http://:health/readyz delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment:
      CM_NAMESPACE:                    kb-system
      CM_AFFINITY:                     {"nodeAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"preference":{"matchExpressions":[{"key":"kb-controller","operator":"In","values":["true"]}]},"weight":100}]}}
      CM_TOLERATIONS:                  [{"effect":"NoSchedule","key":"kb-controller","operator":"Equal","value":"true"}]
      KUBEBLOCKS_IMAGE_PULL_POLICY:    IfNotPresent
      KUBEBLOCKS_TOOLS_IMAGE:          registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks-tools:0.6.0-alpha.22
      KUBEBLOCKS_SERVICEACCOUNT_NAME:  kubeblocks
      VOLUMESNAPSHOT:                  true
      ADDON_JOB_TTL:
      ADDON_JOB_IMAGE_PULL_POLICY:     IfNotPresent
      KUBEBLOCKS_ADDON_SA_NAME:        kubeblocks-addon-installer
    Mounts:
      /etc/kubeblocks from manager-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2rw5j (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  manager-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      kubeblocks-manager-config
    Optional:  false
  kube-api-access-2rw5j:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 kb-controller=true:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  8m46s  default-scheduler  Successfully assigned kb-system/kubeblocks-84485f7b5d-dj4tb to k3d-kb-playground-server-0
  Normal  Pulling    8m47s  kubelet            Pulling image "registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks-tools:0.6.0-alpha.22"
  Normal  Pulled     38s    kubelet            Successfully pulled image "registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks-tools:0.6.0-alpha.22" in 8m9.173266847s
  Normal  Created    37s    kubelet            Created container tools
  Normal  Started    37s    kubelet            Started container tools
  Normal  Pulling    37s    kubelet            Pulling image "registry.cn-hangzhou.aliyuncs.com/apecloud/kubeblocks:0.6.0-alpha.22"
@linghan-hub linghan-hub added the kind/bug Something isn't working label Jun 26, 2023
@linghan-hub linghan-hub added this to the Release 0.6.0 milestone Jun 26, 2023
@ldming
Copy link
Collaborator

ldming commented Jun 26, 2023

Need more information, kubectl get pods -A, helm list -A.
And better to use a new version kbcli (such as 0.6.0-alpha.22) to upgrade.

@github-actions
Copy link

This issue has been marked as stale because it has been open for 30 days with no activity

@github-actions github-actions bot added the Stale label Jul 31, 2023
@ldming
Copy link
Collaborator

ldming commented Aug 7, 2023

It has been verified that upgrading from 0.5.3 to 0.6.0-beta.* is problem-free.

@ldming ldming closed this as completed Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ldming ldming

Labels
bug kind/bug Something isn't working Stale
Projects
None yet
Milestone
Release 0.6.0
Development

No branches or pull requests
3 participants
@ldming @linghan-hub @apecloud-bot