Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] mysql cluster stop to start sometimes role is none #3951

Closed
JashBook opened this issue Jun 26, 2023 · 1 comment
Closed

[BUG] mysql cluster stop to start sometimes role is none #3951

JashBook opened this issue Jun 26, 2023 · 1 comment
Assignees
@xuriwuyun
Labels
bug kind/bug Something isn't working
Milestone
Release 0.6.0

Comments

@JashBook
Copy link
Collaborator

JashBook commented Jun 26, 2023 •
edited
Loading

Describe the bug
mysql cluster stop to start sometimes role is none.

kbcli version
Kubernetes: v1.23.15
KubeBlocks: 0.5.3-beta.7
kbcli: 0.5.3-beta.7

To Reproduce
Steps to reproduce the behavior:

  1. create cluster
  2. stop to start
  3. See error
    start
    kbcli cluster start cluster-dxnbrl
    kbcli cluster list cluster-dxnbrl --show-labels 
NAME             NAMESPACE   CLUSTER-DEFINITION   VERSION           TERMINATION-POLICY   STATUS     CREATED-TIME                 LABELS                                                                                                                                            
cluster-dxnbrl   default     apecloud-mysql       ac-mysql-8.0.30   Delete               Abnormal   Jun 26,2023 13:01 UTC+0800   app.kubernetes.io/instance=cluster-dxnbrl,clusterdefinition.kubeblocks.io/name=apecloud-mysql,clusterversion.kubeblocks.io/name=ac-mysql-8.0.30

kbcli cluster list-instances cluster-dxnbrl 

NAME                     NAMESPACE   CLUSTER          COMPONENT   STATUS    ROLE       ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE    NODE                    CREATED-TIME                 
cluster-dxnbrl-mysql-0   default     cluster-dxnbrl   mysql       Running   leader     ReadWrite    <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-1   default     cluster-dxnbrl   mysql       Running   <none>     <none>       <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-2   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800

kbcli cluster list-ops cluster-dxnbrl --status all 

NAME                         TYPE    CLUSTER          COMPONENT   STATUS   PROGRESS   CREATED-TIME                 
cluster-dxnbrl-start-fb554   Start   cluster-dxnbrl   mysql       Failed   3/3        Jun 26,2023 13:05 UTC+0800

hscale
kbcli cluster hscale cluster-dxnbrl --components mysql --replicas 4
kbcli cluster list cluster-dxnbrl --show-labels

NAME             NAMESPACE   CLUSTER-DEFINITION   VERSION           TERMINATION-POLICY   STATUS     CREATED-TIME                 LABELS                                                                                                                                            
cluster-dxnbrl   default     apecloud-mysql       ac-mysql-8.0.30   Delete               Abnormal   Jun 26,2023 13:01 UTC+0800   app.kubernetes.io/instance=cluster-dxnbrl,clusterdefinition.kubeblocks.io/name=apecloud-mysql,clusterversion.kubeblocks.io/name=ac-mysql-8.0.30

kbcli cluster list-instances cluster-dxnbrl --namespace default 

NAME                     NAMESPACE   CLUSTER          COMPONENT   STATUS    ROLE       ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE    NODE                    CREATED-TIME                 
cluster-dxnbrl-mysql-0   default     cluster-dxnbrl   mysql       Running   leader     ReadWrite    <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-1   default     cluster-dxnbrl   mysql       Running   <none>     <none>       <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-2   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-3   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0               data:1Gi   minikube/192.168.49.2   Jun 26,2023 13:11 UTC+0800

kbcli cluster list-ops cluster-dxnbrl --status all 

NAME                                     TYPE                CLUSTER          COMPONENT   STATUS   PROGRESS   CREATED-TIME                 
cluster-dxnbrl-horizontalscaling-jlcvj   HorizontalScaling   cluster-dxnbrl   mysql       Failed   0/1        Jun 26,2023 13:10 UTC+0800

vscale
kbcli cluster vscale cluster-dxnbrl --components mysql --cpu 200m --memory 0.6Gi
kbcli cluster list cluster-dxnbrl --show-labels 

NAME             NAMESPACE   CLUSTER-DEFINITION   VERSION           TERMINATION-POLICY   STATUS            CREATED-TIME                 LABELS                                                                                                                                            
cluster-dxnbrl   default     apecloud-mysql       ac-mysql-8.0.30   Delete               Abnormal   Jun 26,2023 13:01 UTC+0800   app.kubernetes.io/instance=cluster-dxnbrl,clusterdefinition.kubeblocks.io/name=apecloud-mysql,clusterversion.kubeblocks.io/name=ac-mysql-8.0.30

kbcli cluster list-instances cluster-dxnbrl 

NAME                     NAMESPACE   CLUSTER          COMPONENT   STATUS    ROLE       ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)           STORAGE    NODE                    CREATED-TIME                 
cluster-dxnbrl-mysql-0   default     cluster-dxnbrl   mysql       Running   leader     ReadWrite    <none>   100m / 0             512Mi / 0                       data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-1   default     cluster-dxnbrl   mysql       Running   <none>     <none>       <none>   200m / 200m          644245094400m / 644245094400m   data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:17 UTC+0800   
cluster-dxnbrl-mysql-2   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0                       data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-3   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0                       data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:11 UTC+0800

kbcli cluster list-ops cluster-dxnbrl --status all 

NAME                                   TYPE              CLUSTER          COMPONENT   STATUS   PROGRESS   CREATED-TIME                 
cluster-dxnbrl-verticalscaling-d622p   VerticalScaling   cluster-dxnbrl   mysql       Failed   1/4        Jun 26,2023 13:17 UTC+0800

restart
kbcli cluster restart cluster-dxnbrl --components mysql
kbcli cluster list cluster-dxnbrl --show-labels 

NAME             NAMESPACE   CLUSTER-DEFINITION   VERSION           TERMINATION-POLICY   STATUS       CREATED-TIME                 LABELS                                                                                                                                            
cluster-dxnbrl   default     apecloud-mysql       ac-mysql-8.0.30   Delete               Abnormal   Jun 26,2023 13:01 UTC+0800   app.kubernetes.io/instance=cluster-dxnbrl,clusterdefinition.kubeblocks.io/name=apecloud-mysql,clusterversion.kubeblocks.io/name=ac-mysql-8.0.30

kbcli cluster list-instances cluster-dxnbr 

NAME                     NAMESPACE   CLUSTER          COMPONENT   STATUS    ROLE       ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)           STORAGE    NODE                    CREATED-TIME                 
cluster-dxnbrl-mysql-0   default     cluster-dxnbrl   mysql       Running   leader     ReadWrite    <none>   100m / 0             512Mi / 0                       data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-1   default     cluster-dxnbrl   mysql       Running   <none>     <none>       <none>   200m / 200m          644245094400m / 644245094400m   data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:22 UTC+0800   
cluster-dxnbrl-mysql-2   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0                       data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:05 UTC+0800   
cluster-dxnbrl-mysql-3   default     cluster-dxnbrl   mysql       Running   follower   Readonly     <none>   100m / 0             512Mi / 0                       data:2Gi   minikube/192.168.49.2   Jun 26,2023 13:11 UTC+0800

kbcli cluster list-ops cluster-dxnbrl --status all 

NAME                           TYPE      CLUSTER          COMPONENT   STATUS   PROGRESS   CREATED-TIME                 
cluster-dxnbrl-restart-q46nc   Restart   cluster-dxnbrl   mysql       Failed   1/4        Jun 26,2023 13:22 UTC+0800

Expected behavior
mysql cluster stop to start cluster cluster always ok.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.
@JashBook JashBook added the kind/bug Something isn't working label Jun 26, 2023
@JashBook JashBook added this to the Release 0.6.0 milestone Jun 26, 2023
@JashBook
Copy link
Collaborator Author

JashBook commented Aug 1, 2023

The new version did not reproduce.

@JashBook JashBook closed this as completed Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xuriwuyun xuriwuyun

Labels
bug kind/bug Something isn't working
Projects
None yet
Milestone
Release 0.6.0
Development

No branches or pull requests
3 participants
@xuriwuyun @JashBook @apecloud-bot