Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filebeat: improve debug logging in udp/tcp inputs #34060

Closed
efd6 opened this issue Dec 15, 2022 · 7 comments · Fixed by #41059
Closed

filebeat: improve debug logging in udp/tcp inputs #34060

efd6 opened this issue Dec 15, 2022 · 7 comments · Fixed by #41059
Assignees
Labels
Team:Security-Deployment and Devices Deployment and Devices Team in Security Solution

Comments

@efd6
Copy link
Contributor

efd6 commented Dec 15, 2022

While working on a customer issue in filebeat relating to the udp input I noticed that this input does very little debug logging. Examples of this are that not all transitions are logged during execution and details of the address/port are not logged making it difficult to differentiate what is logged when more than one port or address is being listened on.

It would be good to improve this situation and it may be worth looking over the other inputs to check how well useful information is being logged in them as well.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 15, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Dec 15, 2022
@botelastic
Copy link

botelastic bot commented Dec 15, 2023

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Dec 15, 2023
@efd6 efd6 removed the Stalled label Dec 17, 2023
@andrewkroh andrewkroh changed the title filebeat: improve debug logging in inputs filebeat: improve debug logging in udp/tcp inputs Jan 31, 2024
@norrietaylor norrietaylor added the Team:Security-Deployment and Devices Deployment and Devices Team in Security Solution label Jan 31, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

@aleksmaus
Copy link
Member

aleksmaus commented Sep 30, 2024

Examples of this are that not all transitions are logged during execution and details of the address/port are not logged making it difficult to differentiate what is logged when more than one port or address is being listened on.

This seems like a pretty old ticket.
Just tested with TCP input, and it includes the the source and destination addresses and ports now:

{"log.level":"debug","@timestamp":"2024-09-30T15:40:28.017-0400","log.logger":"tcp","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/inputsource/common/streaming.(*Listener).handleConnection","file.name":"streaming/listener.go","file.line":194},"message":"New client connection.","service.name":"filebeat","address":"localhost:32001","remote_address":"127.0.0.1:49748","active_clients":1,"ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-09-30T15:40:28.018-0400","log.logger":"tcp","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/inputsource/common/streaming.(*Listener).handleConnection.func2","file.name":"streaming/listener.go","file.line":197},"message":"Client disconnected.","service.name":"filebeat","address":"localhost:32001","remote_address":"127.0.0.1:49748","active_clients":0,"ecs.version":"1.6.0"}

What else is missing? @efd6

@efd6
Copy link
Contributor Author

efd6 commented Sep 30, 2024

I think that I was probably thinking of in here and in here. Could also include in here. Basically logging the network metadata and the data len.

@aleksmaus
Copy link
Member

Something like this then:

{"log.level":"debug","@timestamp":"2024-09-30T20:41:02.152-0400","log.logger":"input.tcp","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/tcp.(*server).Run.func1","file.name":"tcp/input.go","file.line":112},"message":"Data received","service.name":"filebeat","id":"FEAD914EAB357583","host":"localhost:32001","bytes":7,"remote_address":"127.0.0.1:52756","truncated":false,"ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-09-30T20:42:39.869-0400","log.logger":"input.udp","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/udp.(*server).Run.func1","file.name":"udp/input.go","file.line":102},"message":"Data received","service.name":"filebeat","id":"5CF62520E29C9FC3","host":"localhost:32001","bytes":8,"remote_address":"127.0.0.1:52667","truncated":false,"ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-09-30T20:45:22.175-0400","log.logger":"input.unix","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/unix.(*server).Run.func1","file.name":"unix/input.go","file.line":105},"message":"Data received","service.name":"filebeat","id":"A41E47DC48C253F1","path":"/Users/amaus/filebeat.sock","bytes":7,"ecs.version":"1.6.0"}

@efd6
Copy link
Contributor Author

efd6 commented Oct 1, 2024

Yeah, I think that's good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Security-Deployment and Devices Deployment and Devices Team in Security Solution
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants