Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Synchronize access to @client_threads in LogStash::Outputs::Tcp #2

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Synchronize access to @client_threads in LogStash::Outputs::Tcp #2

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 17 additions & 8 deletions lib/logstash/outputs/tcp.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,23 +75,32 @@ def register
@logger.info("Starting tcp output listener", :address => "#{@host}:#{@port}")
@server_socket = TCPServer.new(@host, @port)
@client_threads = []
@client_threads_lock = Mutex.new

@accept_thread = Thread.new(@server_socket) do |server_socket|
loop do
client_thread = Thread.start(server_socket.accept) do |client_socket|
client = Client.new(client_socket, @logger)
Thread.current[:client] = client
client.run
@client_threads_lock.synchronize do
@client_threads << Thread.start(server_socket.accept) do |client_socket|
begin
client = Client.new(client_socket, @logger)
Thread.current[:client] = client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of doing a "long" lock, I'd suggest keeping the client_thread var usage and simply do a short lock to update the @client_threads ivar like that

@client_threads_lock.synchronize do
  @client_threads << client_thread
end

client.run
ensure
@client_threads_lock.synchronize do
@client_threads.delete(Thread.current)
end
end
end
end
@client_threads << client_thread
end
end

@codec.on_event do |payload|
@client_threads.each do |client_thread|
client_thread[:client].write(payload)
# dup @client_threads to avoid holding the lock while writing to clients
threads = @client_threads_lock.synchronize { @client_threads.dup }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a good idea but thinking about it, what is not thread-safe here is the array updating and not the ivar setting. in this case, the code that is executed the most is this on_event and if we can avoid synchronizing and array dupping that would be best since this is being done for all event.

maybe we could instead move all this in the client_treat stuff where instead of directly adding in the @client_threads array we simply set a new array with the new client thread? with something like this:

@client_threads = @client_threads + [client_thread]

since setting an ivar is threadsafe, then in the on_event we could just do a @client_threads.each do ... which would avoid all locking. WDYT?

threads.each do |thread|
thread[:client].write(payload)
end
@client_threads.reject! {|t| !t.alive? }
end
else
client_socket = nil
Expand Down