Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[filebeat][input] - Websocket Input with CEL engine #37774

Merged
merged 39 commits into from
Feb 12, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
0de28ef
initial commit
ShourieG Nov 15, 2023
58a7e23
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Nov 30, 2023
1b58923
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Dec 21, 2023
1b8980e
working version of websocket input with cel engine
ShourieG Jan 29, 2024
ee9cce6
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Jan 29, 2024
9819004
updated go mod and NOTICE
ShourieG Jan 29, 2024
f2056a0
added regex support to cel engine, added all metrics params to releva…
ShourieG Jan 30, 2024
de1b707
removed unused config from input struct
ShourieG Jan 30, 2024
c0dddc6
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Jan 30, 2024
1f6da1a
added redactor
ShourieG Jan 30, 2024
85ad3e6
removed cel references in logs
ShourieG Jan 30, 2024
8e7ad35
addressed draft PR suggestions and added more metrics
ShourieG Feb 1, 2024
4768d4a
added tests
ShourieG Feb 5, 2024
8aaed71
added retry function
ShourieG Feb 5, 2024
d8bc064
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 5, 2024
9347ff6
updated changelog
ShourieG Feb 5, 2024
f7b2d19
updated tests
ShourieG Feb 5, 2024
b618501
addressed PR suggestions, removed auto retry mechanism for the moment
ShourieG Feb 6, 2024
1e5b205
addressed PR suggestions, removed auto retry mechanism for the moment
ShourieG Feb 6, 2024
e96e474
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 6, 2024
e9aa0b3
added documentation and updated codeowners
ShourieG Feb 6, 2024
2b1f90f
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 6, 2024
a71731b
updated experimental tags
ShourieG Feb 6, 2024
2ae755b
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 6, 2024
129c5bd
added a new test, cleaned up some code and logic
ShourieG Feb 6, 2024
21cdf6f
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 6, 2024
82b54bc
added cursor condition check test and updated filebeat-options asciidoc
ShourieG Feb 8, 2024
581a210
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 8, 2024
c550da9
added auth tests, removed api-key config and updated it to custom aut…
ShourieG Feb 8, 2024
3d6f070
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 8, 2024
8a61509
addressed PR suggestions and added config tests
ShourieG Feb 9, 2024
6d00e55
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 9, 2024
c301b16
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 9, 2024
d60d73b
Update x-pack/filebeat/docs/inputs/input-websocket.asciidoc
ShourieG Feb 12, 2024
b3a7e20
Update x-pack/filebeat/docs/inputs/input-websocket.asciidoc
ShourieG Feb 12, 2024
341ad87
Update x-pack/filebeat/docs/inputs/input-websocket.asciidoc
ShourieG Feb 12, 2024
dd850d2
added debug log for cel state before cel eval, updated the docs accor…
ShourieG Feb 12, 2024
57a5dac
Merge remote-tracking branch 'upstream/main' into feature/websocket
ShourieG Feb 12, 2024
c68ad52
updated URL config docs
ShourieG Feb 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ CHANGELOG*
/x-pack/filebeat/input/lumberjack/ @elastic/security-service-integrations
/x-pack/filebeat/input/netflow/ @elastic/sec-deployment-and-devices
/x-pack/filebeat/input/o365audit/ @elastic/security-service-integrations
/x-pack/filebeat/input/websocket/ @elastic/security-service-integrations
/x-pack/filebeat/module/activemq @elastic/obs-infraobs-integrations
/x-pack/filebeat/module/aws @elastic/obs-cloud-monitoring
/x-pack/filebeat/module/awsfargate @elastic/obs-cloud-monitoring
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d
- Relax TCP/UDP metric polling expectations to improve metric collection. {pull}37714[37714]
- Add support for PEM-based Okta auth in HTTPJSON. {pull}37772[37772]
- Prevent complete loss of long request trace data. {issue}37826[37826] {pull}37836[37836]
- Added experimental version of the Websocket Input. {pull}37774[37774]
- Add support for PEM-based Okta auth in CEL. {pull}37813[37813]

*Auditbeat*
Expand Down
64 changes: 32 additions & 32 deletions NOTICE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18447,6 +18447,38 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/gorilla/websocket
Version: v1.4.2
Licence type (autodetected): BSD-2-Clause
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/gorilla/websocket@v1.4.2/LICENSE:

Copyright (c) 2013 The Gorilla WebSocket Authors. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/h2non/filetype
Version: v1.1.1
Expand Down Expand Up @@ -41594,38 +41626,6 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/gorilla/websocket
Version: v1.4.2
Licence type (autodetected): BSD-2-Clause
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/gorilla/websocket@v1.4.2/LICENSE:

Copyright (c) 2013 The Gorilla WebSocket Authors. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/hashicorp/cronexpr
Version: v1.1.0
Expand Down
3 changes: 3 additions & 0 deletions filebeat/docs/filebeat-options.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ You can configure {beatname_uc} to use the following inputs:
* <<{beatname_lc}-input-tcp>>
* <<{beatname_lc}-input-udp>>
* <<{beatname_lc}-input-gcs>>
* <<{beatname_lc}-input-websocket>>

include::multiline.asciidoc[]

Expand Down Expand Up @@ -145,3 +146,5 @@ include::inputs/input-udp.asciidoc[]
include::inputs/input-unix.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-gcs.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-websocket.asciidoc[]
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ require (
github.com/googleapis/gax-go/v2 v2.12.0
github.com/gorilla/handlers v1.5.1
github.com/gorilla/mux v1.8.0
github.com/gorilla/websocket v1.4.2
github.com/icholy/digest v0.1.22
github.com/lestrrat-go/jwx/v2 v2.0.19
github.com/otiai10/copy v1.12.0
Expand Down Expand Up @@ -299,7 +300,6 @@ require (
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.4 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/hashicorp/cronexpr v1.1.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
Expand Down
Loading
Loading