You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm encountering an issue with configuring Kafka in KRaft mode using the Ansible role for Confluent Platform. Despite specifying KRaft mode in my hosts.yml file, the generated server.properties file still includes ZooKeeper-related settings.
After running the Ansible playbook with the above configuration, the server.properties file on Kafka brokers still contains ZooKeeper-related configurations.
Error that i get after running ansible:
TASK [confluent.platform.kafka_controller : Check Kafka Metadata Quorum]
fatal: [kafka-1]: FAILED! => {"changed": false, "cmd": "/usr/bin/kafka-metadata-quorum --bootstrap-server kafka-1:9091 --command-config /etc/controller/client.properties describe --replication\n", "delta": "0:01:01.149704", "end": "2024-08-19 10:20:01.372824", "msg": "non-zero return code", "rc": 1, "start": "2024-08-19 10:19:00.223120", "stderr": "[2024-08-19 10:19:01,068] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)\n[2024-08-19 10:19:01,172] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)\n[2024-08-19 10:19:01,274] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)\n[2024-08-19 10:19:01,576] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)\n[2024-08-19 10:19:02,079] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)\n[2024-08-19 10:20:00,070] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)\norg.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeMetadataQuorum\njava.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeMetadataQuorum\n\tat java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)\n\tat java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)\n\tat org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)\n\tat org.apache.kafka.tools.MetadataQuorumCommand.handleDescribeReplication(MetadataQuorumCommand.java:158)\n\tat org.apache.kafka.tools.MetadataQuorumCommand.execute(MetadataQuorumCommand.java:106)\n\tat org.apache.kafka.tools.MetadataQuorumCommand.mainNoExit(MetadataQuorumCommand.java:62)\n\tat org.apache.kafka.tools.MetadataQuorumCommand.main(MetadataQuorumCommand.java:57)\nCaused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeMetadataQuorum", "stderr_lines": ["[2024-08-19 10:19:01,068] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-1/172.16.104.41:9091) could not be established. Node may not be available. ((org.apache.kafka.clients.NetworkClient)", "org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeMetadataQuorum", "java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeMetadataQuorum", "\tat java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)", "\tat java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)", "\tat org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)", "\tat org.apache.kafka.tools.MetadataQuorumCommand.handleDescribeReplication(MetadataQuorumCommand.java:158)", "\tat org.apache.kafka.tools.MetadataQuorumCommand.execute(MetadataQuorumCommand.java:106)", "\tat org.apache.kafka.tools.MetadataQuorumCommand.mainNoExit(MetadataQuorumCommand.java:62)", "\tat org.apache.kafka.tools.MetadataQuorumCommand.main(MetadataQuorumCommand.java:57)", "Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeMetadataQuorum"], "stdout": "", "stdout_lines": []}
And here is confluent-server service logs:
Aug 05 17:28:34 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:34 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:34 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:34 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:34 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:34 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:35,693] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:35,694] WARN Session 0x0 for server localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:35 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:36,795] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:36,797] WARN Session 0x0 for server localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:36 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:37,898] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:37,899] WARN Session 0x0 for server localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:37 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:39,000] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:39,002] WARN Session 0x0 for server localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:39 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:40,103] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:40,105] WARN Session 0x0 for server localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:40 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:41,206] INFO Opening socket connection to server localhost/[0:0:0:0:0:0:0:1]:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:41,207] WARN Session 0x0 for server localhost/[0:0:0:0:0:0:0:1]:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:41 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:42,308] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:42,309] WARN Session 0x0 for server localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: java.net.ConnectException: Connection refused
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1293)
Aug 05 17:28:42 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:42,662] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,410] INFO Opening socket connection to server localhost/[0:0:0:0:0:0:0:1]:2181. (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,514] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,515] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,516] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,518] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:258)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:216)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:254)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:116)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:2853)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zk.KafkaZkClient$.createZkClient(KafkaZkClient.scala:2959)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:1065)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.server.KafkaServer.startup(KafkaServer.scala:355)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.Kafka$.main(Kafka.scala:112)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.Kafka.main(Kafka.scala)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,519] INFO shutting down (kafka.server.KafkaServer)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,537] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,538] INFO shut down completed (kafka.server.KafkaServer)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,538] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:258)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:216)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:254)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:116)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:2853)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.zk.KafkaZkClient$.createZkClient(KafkaZkClient.scala:2959)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:1065)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.server.KafkaServer.startup(KafkaServer.scala:355)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.Kafka$.main(Kafka.scala:112)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: at kafka.Kafka.main(Kafka.scala)
Aug 05 17:28:43 kafka-1 kafka-server-start[105343]: [2024-08-05 17:28:43,539] INFO shutting down (kafka.server.KafkaServer)
Aug 05 17:28:43 kafka-1 systemd[1]: confluent-server.service: Main process exited, code=exited, status=1/FAILURE
Aug 05 17:28:43 kafka-1 systemd[1]: confluent-server.service: Failed with result 'exit-code'.
Aug 05 17:28:43 kafka-1 systemd[1]: confluent-server.service: Consumed 3.602s CPU time.
The text was updated successfully, but these errors were encountered:
I'm encountering an issue with configuring Kafka in KRaft mode using the Ansible role for Confluent Platform. Despite specifying KRaft mode in my hosts.yml file, the generated server.properties file still includes ZooKeeper-related settings.
Debian Version: 12.6
cp-ansible: 7.7.0-post
hosts.yml:
After running the Ansible playbook with the above configuration, the server.properties file on Kafka brokers still contains ZooKeeper-related configurations.
Error that i get after running ansible:
And here is confluent-server service logs:
The text was updated successfully, but these errors were encountered: