Skip to content

Commit

Permalink
Updated custom_size_based_buffer.rb
Browse files Browse the repository at this point in the history
Added exponential backoff to retries. Removed max_retries.
  • Loading branch information
MonishkaDas committed Oct 21, 2024
1 parent ca981ba commit 07fc531
Showing 1 changed file with 4 additions and 10 deletions.
14 changes: 4 additions & 10 deletions lib/logstash/outputs/kusto/custom_size_based_buffer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,7 @@ def buffer_flush(options = {})
end

items_flushed = 0
max_retries = 5
retries = 0
backoff_interval = 1

begin
outgoing_items = []
Expand Down Expand Up @@ -103,14 +102,9 @@ def buffer_flush(options = {})
rescue => e
@buffer_config[:logger].error("Flush failed: #{e.message}")
@buffer_config[:logger].error(e.backtrace.join("\n"))
retries += 1
if retries <= max_retries
sleep 1
retry
else
@buffer_config[:logger].error("Max retries reached. Data loss may occur.")
raise e
end
sleep backoff_interval
backoff_interval = [backoff_interval * 2, 60].min # Exponential backoff with a max interval of 60 seconds
retry
end

@buffer_state[:last_flush] = Time.now.to_i
Expand Down

0 comments on commit 07fc531

Please sign in to comment.