Skip to content
This repository has been archived by the owner on Mar 22, 2024. It is now read-only.

Commit

Permalink
add udp rcv buff config option
Browse files Browse the repository at this point in the history
  • Loading branch information
robcowart committed Jul 26, 2018
1 parent 754dba3 commit 00473b8
Show file tree
Hide file tree
Showing 9 changed files with 18 additions and 0 deletions.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,10 +143,13 @@ Environment Variable | Description | Default Value
--- | --- | ---
ELASTIFLOW_NETFLOW_UDP_WORKERS | The number of Netflow input threads | 4
ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE | The number of unprocessed Netflow UDP packets the input can buffer | 4096
ELASTIFLOW_NETFLOW_UDP_RCV_BUFF | The socket receive buffer size (bytes) for Netflow | 33554432
ELASTIFLOW_SFLOW_UDP_WORKERS | The number of sFlow input threads | 4
ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE | The number of unprocessed sFlow UDP packets the input can buffer | 4096
ELASTIFLOW_SFLOW_UDP_RCV_BUFF | The socket receive buffer size (bytes) for sFlow | 33554432
ELASTIFLOW_IPFIX_UDP_WORKERS | The number of IPFIX input threads | 4
ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE | The number of unprocessed IPFIX UDP packets the input can buffer | 4096
ELASTIFLOW_IPFIX_UDP_RCV_BUFF | The socket receive buffer size (bytes) for IPFIX | 33554432

> WARNING! Increasing `queue_size` will increase heap_usage. Make sure have configured JVM heap appropriately as specified in the [Requirements](#requirements)
Expand Down Expand Up @@ -343,6 +346,7 @@ ELASTIFLOW_NETFLOW_IPV6_HOST | The IP address on which to listen for Netflow mes
ELASTIFLOW_NETFLOW_IPV6_PORT | The UDP port on which to listen for Netflow messages | 52055
ELASTIFLOW_NETFLOW_UDP_WORKERS | The number of Netflow input threads | 4
ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE | The number of unprocessed Netflow UDP packets the input can buffer | 4096
ELASTIFLOW_NETFLOW_UDP_RCV_BUFF | The socket receive buffer size (bytes) for Netflow | 33554432
ELASTIFLOW_NETFLOW_LASTSW_TIMESTAMP | Enable/Disable setting `@timestamp` with the value of netflow.last_switched | false
ELASTIFLOW_NETFLOW_TZ | The timezone of netflow.last_switched | UTC
ELASTIFLOW_SFLOW_IPV4_HOST | The IP address on which to listen for sFlow messages | 0.0.0.0
Expand All @@ -351,6 +355,7 @@ ELASTIFLOW_SFLOW_IPV6_HOST | The IP address on which to listen for sFlow message
ELASTIFLOW_SFLOW_IPV6_PORT | The UDP port on which to listen for sFlow messages | 56343
ELASTIFLOW_SFLOW_UDP_WORKERS | The number of sFlow input threads | 4
ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE | The number of unprocessed sFlow UDP packets the input can buffer | 4096
ELASTIFLOW_SFLOW_UDP_RCV_BUFF | The socket receive buffer size (bytes) for sFlow | 33554432
ELASTIFLOW_IPFIX_TCP_IPV4_HOST | The IP address on which to listen for IPFIX messages via TCP | 0.0.0.0
ELASTIFLOW_IPFIX_TCP_IPV4_PORT | The port on which to listen for IPFIX messages via TCP | 4739
ELASTIFLOW_IPFIX_UDP_IPV4_HOST | The IP address on which to listen for IPFIX messages via UDP | 0.0.0.0
Expand All @@ -361,6 +366,7 @@ ELASTIFLOW_IPFIX_UDP_IPV6_HOST | The IP address on which to listen for IPFIX mes
ELASTIFLOW_IPFIX_UDP_IPV6_PORT | The port on which to listen for IPFIX messages via UDP | 54739
ELASTIFLOW_IPFIX_UDP_WORKERS | The number of IPFIX input threads | 4
ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE | The number of unprocessed IPFIX UDP packets the input can buffer | 4096
ELASTIFLOW_IPFIX_UDP_RCV_BUFF | The socket receive buffer size (bytes) for IPFIX | 33554432

# Recommended Setting for timepicker:quickRanges
I recommend configuring `timepicker:quickRanges` for the setting below. The result will look like this:
Expand Down
3 changes: 3 additions & 0 deletions logstash.service.d/elastiflow.conf
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Environment="ELASTIFLOW_NETFLOW_IPV6_PORT=52055"
# Netflow - UDP input options
Environment="ELASTIFLOW_NETFLOW_UDP_WORKERS=4"
Environment="ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE=4096"
Environment="ELASTIFLOW_NETFLOW_UDP_RCV_BUFF=33554432"
# Netflow timestamp options
Environment="ELASTIFLOW_NETFLOW_LASTSW_TIMESTAMP=false"
Environment="ELASTIFLOW_NETFLOW_TZ=UTC"
Expand All @@ -78,6 +79,7 @@ Environment="ELASTIFLOW_SFLOW_IPV6_PORT=56343"
# sFlow - UDP input options
Environment="ELASTIFLOW_SFLOW_UDP_WORKERS=4"
Environment="ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE=4096"
Environment="ELASTIFLOW_SFLOW_UDP_RCV_BUFF=33554432"


# IPFIX - IPv4
Expand All @@ -93,3 +95,4 @@ Environment="ELASTIFLOW_IPFIX_UDP_IPV6_PORT=54739"
# IPFIX - UDP input options
Environment="ELASTIFLOW_IPFIX_UDP_WORKERS=4"
Environment="ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE=4096"
Environment="ELASTIFLOW_IPFIX_UDP_RCV_BUFF=33554432"
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ input {
port => "${ELASTIFLOW_IPFIX_UDP_IPV4_PORT:4739}"
workers => "${ELASTIFLOW_IPFIX_UDP_WORKERS:4}"
queue_size => "${ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE:2048}"
receive_buffer_bytes => "${ELASTIFLOW_IPFIX_UDP_RCV_BUFF:33554432}"
codec => netflow {
versions => [5,9,10]
target => "ipfix"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ input {
port => "${ELASTIFLOW_IPFIX_UDP_IPV6_PORT:54739}"
workers => "${ELASTIFLOW_IPFIX_UDP_WORKERS:4}"
queue_size => "${ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE:2048}"
receive_buffer_bytes => "${ELASTIFLOW_IPFIX_UDP_RCV_BUFF:33554432}"
codec => netflow {
versions => [5,9,10]
target => "ipfix"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ input {
port => "${ELASTIFLOW_NETFLOW_IPV4_PORT:2055}"
workers => "${ELASTIFLOW_NETFLOW_UDP_WORKERS:4}"
queue_size => "${ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE:2048}"
receive_buffer_bytes => "${ELASTIFLOW_NETFLOW_UDP_RCV_BUFF:33554432}"
codec => netflow {
versions => [5,9,10]
#netflow_definitions => "${ELASTIFLOW_DEFINITION_PATH:/etc/logstash/elastiflow/definitions}/netflow.yml"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ input {
port => "${ELASTIFLOW_NETFLOW_IPV6_PORT:52055}"
workers => "${ELASTIFLOW_NETFLOW_UDP_WORKERS:4}"
queue_size => "${ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE:2048}"
receive_buffer_bytes => "${ELASTIFLOW_NETFLOW_UDP_RCV_BUFF:33554432}"
codec => netflow {
versions => [5,9,10]
#netflow_definitions => "${ELASTIFLOW_DEFINITION_PATH:/etc/logstash/elastiflow/definitions}/netflow.yml"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ input {
port => "${ELASTIFLOW_SFLOW_IPV4_PORT:6343}"
workers => "${ELASTIFLOW_SFLOW_UDP_WORKERS:4}"
queue_size => "${ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE:2048}"
receive_buffer_bytes => "${ELASTIFLOW_SFLOW_UDP_RCV_BUFF:33554432}"
codec => sflow { }
type => "sflow"
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ input {
port => "${ELASTIFLOW_SFLOW_IPV6_PORT:56343}"
workers => "${ELASTIFLOW_SFLOW_UDP_WORKERS:4}"
queue_size => "${ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE:2048}"
receive_buffer_bytes => "${ELASTIFLOW_SFLOW_UDP_RCV_BUFF:33554432}"
codec => sflow { }
type => "sflow"
}
Expand Down
3 changes: 3 additions & 0 deletions profile.d/elastiflow.sh
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ export ELASTIFLOW_NETFLOW_IPV6_PORT=52055
# Netflow - UDP input options
export ELASTIFLOW_NETFLOW_UDP_WORKERS=4
export ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE=4096
export ELASTIFLOW_NETFLOW_UDP_RCV_BUFF=33554432
# Netflow timestamp options
export ELASTIFLOW_NETFLOW_LASTSW_TIMESTAMP=false
export ELASTIFLOW_NETFLOW_TZ=UTC
Expand All @@ -77,6 +78,7 @@ export ELASTIFLOW_SFLOW_IPV6_PORT=56343
# sFlow - UDP input options
export ELASTIFLOW_SFLOW_UDP_WORKERS=4
export ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE=4096
export ELASTIFLOW_SFLOW_UDP_RCV_BUFF=33554432


# IPFIX - IPv4
Expand All @@ -92,3 +94,4 @@ export ELASTIFLOW_IPFIX_UDP_IPV6_PORT=54739
# IPFIX - UDP input options
export ELASTIFLOW_IPFIX_UDP_WORKERS=4
export ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE=4096
export ELASTIFLOW_IPFIX_UDP_RCV_BUFF=33554432

0 comments on commit 00473b8

Please sign in to comment.