Skip to content

Commit

Permalink
Merge pull request #4 from billmurrin/documentation
Browse files Browse the repository at this point in the history
Updated README to support v1.1.0
  • Loading branch information
billmurrin authored Apr 18, 2017
2 parents 1505a54 + 7aa3536 commit 5c5448e
Showing 1 changed file with 25 additions and 7 deletions.
32 changes: 25 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,37 +22,55 @@ Function Breakdown

Function | Description
-------- | -----------
slookup(stream, srcField, dstField, rtnField, timeRange) : String | Conduct a lookup in a remote stream and return a field value based on a matching source field. Similar to VLOOKUP in Excel
slookup(stream, srcField, dstField, rtnField, timeRange, sortOrder) : String | Conduct a lookup in a remote stream and return a field value based on a matching source field. Similar to VLOOKUP in Excel

Parameter | Type | Required | Description
--------- | ---- | -------- | -----------
stream | String | Y | The stream to look up the source field.
srcField | String | Y | The source field. The value to query for in the remote stream.
dstField | String | Y | The destination field that will be queried against.
rtnField | String | Y | The field to return if the query is successful.
timeRange | String | Y | Relative Time Range
timeRange | String | Y | Relative Time Range (Seconds)
sortOrder | String | Y | Timestamp sort order either "asc" or "desc".

Use Case and Rule Example
---

Below is a rule that was created to lookup the value of an IP address in order enrich Windows Event Log messages coming from WinLogBeat.
Below are example rules that were created to lookup the value of an IP address in order enrich Windows Event Log messages coming from WinLogBeat.

In this use case, the remote stream named Systems with stream_id 58aba0cb3cbe8205e76c6145 contains system information (IP, MAC, ComputerName). This could be a dump of Directory Service Computer Objects, a listing of NBTScan results, etc.

The slookup function constructs a search query using the value of winlogbeat_computer_name on the computer_name field (computer_name:VALUE_OF_FIELD). If the search is successful, the ip_address field is returned. The returned value can then be added to the current stream message in the pipeline.

The sortOrder parameter instructs the function to either return the oldest match (ascending), or the newest match (descending) if multiple records are found during the query.

```
rule "IP Lookup"
rule "IP Lookup - Ascending"
when
has_field("winlogbeat_computer_name")
then
//StreamID, Source Field, Destination Field, Return Field, Relative Time
let system_info = slookup("58aba0cb3cbe8205e76c6145", "winlogbeat_computer_name", "computer_name", "ip_address", "14400");
//StreamID, Source Field, Destination Field, Return Field, Relative Time, Ascending SortOrder
let system_info = slookup("58aba0cb3cbe8205e76c6145", "winlogbeat_computer_name", "computer_name", "ip_address", "14400", "asc");
set_field("ip_address", to_ip(system_info));
end
```

This function has only been tested in a limited setting. Its performance impact on large remote streams and very large relative data timeframes, remains unknown.
```
rule "IP Lookup - Descending"
when
has_field("winlogbeat_computer_name")
then
//StreamID, Source Field, Destination Field, Return Field, Relative Time, Descending SortOrder
let system_info = slookup("58aba0cb3cbe8205e76c6145", "winlogbeat_computer_name", "computer_name", "ip_address", "14400", "desc");
set_field("ip_address", to_ip(system_info));
end
```

Additional Info
---
This function has been tested in a limited setting. Its performance impact on very large remote streams and very large relative data timeframes, remains unknown.

If you experience an ingestion slow-down enriching a large volume of data, you can attempt increasing *processbuffer_processors* in the graylog server.conf file.

More information about writing a Graylog2 processor pipeline function.
https://www.graylog.org/blog/71-writing-your-own-graylog-processing-pipeline-functions

0 comments on commit 5c5448e

Please sign in to comment.