-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Kafka Message loss when Kafkaserver goes down #4099
Replies: 1 comment · 6 replies
-
Is there an error associated with that delivery result? There should be |
Beta Was this translation helpful? Give feedback.
All reactions
-
Then the message is not lost, it failed to deliver because the topic or partition doesn't/no-longer exists - if this happens when a broker goes down it is typically because the topic is not properly replicated. Use a replication factor of 3. |
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
Thanks for quick reply @edenhill. I agree with you on replication factor to be 3, But for some reason we are using single kafka server and Zookeeper. How do we overcome with this message loss. I understand that by having Not sure for what reason message is not able to deliver. Is there any setting is available for delivery timeout. kindly give a thought on this. Below is the another message on the same scenario, sometimes Exception message says And also I observed that Kafka server is up after 1 minute then the message is lost. if kafka is up before 1 min. then message is able to produce it. |
Beta Was this translation helpful? Give feedback.
All reactions
-
@edenhill Appreciate your inputs to understand the above behaviour further |
Beta Was this translation helpful? Give feedback.
All reactions
-
@edenhill and @mhowlett ` ` |
Beta Was this translation helpful? Give feedback.
All reactions
-
Could you please provide thoughts on this. Appreciate your time on this. |
Beta Was this translation helpful? Give feedback.
-
I have a scenario that when Kafka server or Zookeeper both goes down, but the producer application trying to send a message to Kafka broker.
When the Kafka broker or Zookeeper comes online, messages is getting lost.
Below are my configuration to reproduce
I have a scenario that when Kafka server or Zookeeper both goes down, but the producer application trying to send a message to Kafka broker.
When the Kafka broker or Zookeeper comes online, messages is getting lost and i see the producer delivery result offset has Offset: -1001 and Partition [Any]
I used the above mentioned setting to produce the message using Confluent kafka nuget package version 1.8.2 from .Net code.
Appreciate your inputs in advance @edenhill
Beta Was this translation helpful? Give feedback.
All reactions