From 712e3fc2624f9f9c14478a27598c39912e0556fa Mon Sep 17 00:00:00 2001 From: Matrix86 Date: Wed, 31 Jan 2024 21:33:53 +0000 Subject: [PATCH] Merge pull request #10 from Matrix86/feat/ratelimiter ratelimit filter 289a2841a467c66f043d767082ac896332c2612b --- doc/configuration/index.html | 4 +- doc/feeders/apt/index.html | 4 +- doc/feeders/file/index.html | 4 +- doc/feeders/folder/index.html | 4 +- doc/feeders/imap/index.html | 4 +- doc/feeders/index.html | 4 +- doc/feeders/rss/index.html | 4 +- doc/feeders/slack/index.html | 4 +- doc/feeders/timer/index.html | 4 +- doc/feeders/twitter/index.html | 4 +- doc/feeders/web/index.html | 4 +- doc/filters/cache/index.html | 4 +- doc/filters/changed/index.html | 4 +- doc/filters/echo/index.html | 4 +- doc/filters/elasticsearch/index.html | 4 +- doc/filters/file/index.html | 4 +- doc/filters/format/index.html | 4 +- doc/filters/hash/index.html | 4 +- doc/filters/html/index.html | 4 +- doc/filters/http/index.html | 4 +- doc/filters/index.html | 4 +- doc/filters/index.xml | 3 +- doc/filters/js/basics/index.html | 4 +- doc/filters/js/entrypoint/index.html | 4 +- doc/filters/js/index.html | 4 +- doc/filters/js/packages/index.html | 4 +- doc/filters/json/index.html | 4 +- doc/filters/mail/index.html | 4 +- doc/filters/mimetype/index.html | 4 +- doc/filters/override/index.html | 4 +- doc/filters/pdf/index.html | 4 +- doc/filters/random/index.html | 4 +- doc/filters/ratelimit/index.html | 107 +++++++++++++++++++++++++++ doc/filters/slack/index.html | 4 +- doc/filters/striptag/index.html | 4 +- doc/filters/system/index.html | 4 +- doc/filters/text/index.html | 4 +- doc/filters/url/index.html | 4 +- doc/filters/xls/index.html | 4 +- doc/index.html | 4 +- doc/installation/build/index.html | 4 +- doc/installation/docker/index.html | 4 +- doc/installation/index.html | 4 +- doc/rules/definition/index.html | 4 +- doc/rules/index.html | 4 +- doc/rules/syntax/index.html | 4 +- index.json | 2 +- index.xml | 3 +- sitemap.xml | 2 +- 49 files changed, 201 insertions(+), 92 deletions(-) create mode 100644 doc/filters/ratelimit/index.html diff --git a/doc/configuration/index.html b/doc/configuration/index.html index 5e41a25..607e924 100644 --- a/doc/configuration/index.html +++ b/doc/configuration/index.html @@ -80,7 +80,7 @@ Feeders Filters

Configuration

Driplane needs a yaml file to work. An example can be found here.

The configuration file could have several sections, but the mandatory one is the general one. It specifies all the paths needed by driplane to find js files, templates and rules.

It contains the paths of rules, templates and javascript plugins.

1
 2
 3
diff --git a/doc/feeders/apt/index.html b/doc/feeders/apt/index.html
index 41653c2..e745c31 100644
--- a/doc/feeders/apt/index.html
+++ b/doc/feeders/apt/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
Apt feeder
This feeder can create the stream starting from an apt repository. It supports also flat repositories.
It is possible to specify the frequency of the quering and receive a message every time a new package is published.
Parameters
ParameterTypeDefaultDescription
urlSTRINGemptyURL of the apt repo
freqDURATION60show often the feed should be parsed
suiteSTRING“stable”suite of the repo to keep under control
archSTRINGempty"architecture of the repo, if empty the first arch returned by the Release file will be used
indexSTRINGemptyURL of the Packages file (it overrides the url parameter)
insecureBOOLfalseallow repository with insecure certificates
... | <apt: url="http://apt.modmyi.com/dists/stable/Release", insecure="true", freq="3h"> | ...
Output
Text
The main field of the Message will contain the filename of the package and all the other field will be present in extra fields.
Extra
List of the supported field that will be returned as extra field.
Name
Filename
Size
MD5sum
SHA1
SHA256
DescriptionMD5
Depends
InstalledSize
Package
Architecture
Version
Section
Maintainer
Homepage
Description
Tag
Author
Name
Examples
What's on this Page
File feeder
This feeder can create the stream starting from a file. Like the tail -f command it opens the specified file and propagates a data message if a line is being added to the file.
Parameters
ParameterTypeDefaultDescription
filenameSTRINGemptythe path of the file that it has to keep track
toendBOOL“false”the feeder will start to create data messages only for new added lines
... | <file: filename="path/of/the/file.txt", toend="false"> | ...
Output
Text
The main field of the Message will contain the new read line.
Extra
NameDescription
file_namethe name of the read file
Examples
What's on this Page
Folder feeder
This feeder can create the stream of fsnotify events for a given folder or for cloud platform storage like Amazon S3,
Google Drive and Dropbox.
The feeder use cloudwatcher to keep track of changes on the chosen directory.
Parameters
ParameterTypeDefaultDescription
nameSTRINGemptythe path of the folder that it has to keep track
typeSTRING“local”the type of service to use local, dropbox, gdrive, s3 or git
freqDURATION2show often the directory should be checked for updates
<folder: name="/tmp", type="local"> | ...
Output
Text
The main field of the Message will contain the filename while the op extra the type of event.
Extra
NameDescription
optype of event: FileCreated, FileChanged, FileDeleted, TagsChanged
sizefor some events you can find the size of the file that triggered the event
Examples
What's on this Page
Imap
This feeder creates a stream starting from emails received on the account read by an IMAP client. It is possible to define how often the email account should be checked.
Every time the email inbox is parsed a Message is sent down the lane.
Parameters
ParameterTypeDefaultDescription
hostSTRINGemptyHost of the IMAP server
portSTRINGemptyPort of the IMAP server
usernameSTRINGemptyUsername of the account
passwordSTRINGemptyPassword of the account
mailboxSTRING“INBOX”Name of the mailbox to read
freqDURATION“1m”how often the email account should be checked
start_from_beginningBOOL“true”if “true” it reads all the emails in the mailbox from the beginning
get_attachmentsBOOL“false”if “true” it reads also the attachments
... | <imap: host="imap.gmail.com", port="993", username="test@gmail.com", password="xxxxx", get_attachments="true", freq="30m"> ...
Output
Text
The main field of the Message will contain the email’s subject.
Extra
NameDescription
fromList of the senders in the form email@mail.com Name
toList of the recipients in the form email@mail.com Name
reply_toList of address in the “Reply-To” header
in_reply_toParent Message-id
ccList of the CC Header Addresses in the form email@mail.com Name
bccList of the BCC Header Addresses in the form email@mail.com Name
senderMessage sender
message_idMessage-Id of the current email
dateMessage Date
subjectSubject of the email
is_attachmentIt is “true” if has the following 2 fields
attachment_filenameName of the attachment
attachment_bodyBinary content of the attachment
Not all the Extra field could be filled. If the relative tag is not present on the feed it will be empty.
Examples
What's on this Page

\ No newline at end of file
diff --git a/doc/feeders/rss/index.html b/doc/feeders/rss/index.html
index 63e573c..de5d418 100644
--- a/doc/feeders/rss/index.html
+++ b/doc/feeders/rss/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
RSS
This feeder creates a stream starting from a feed RSS, ATOM or JSON.
It is based on gofeed so you can refer to it for more info and supported formats.
Parameters
ParameterTypeDefaultDescription
urlSTRINGemptyURL of the feed
freqDURATION60show often the feed should be parsed
start_from_beginningBOOL“false”if “true” it starts to parse the feed from the beginning (the first time it will ignore the pubdate field of the feed)
ignore_pubdateBOOL“false”if “true” it ignores the pubdate and it returns all the feed content every time
... | <rss: url="https://example.rss", freq="5s", start_from_beginning="false"> | ...
Output
Text
The main field of the Message will contain the item.Title string of the gofeed.Item struct.
Extra
NameDescription
feed_titletitle of the feed (feed.Title)
feed_feedlinkfeed url (feed.FeedLink)
feed_updatedtime of the last update (feed.Updated)
feed_publisheddate of publication (feed.Published)
feed_authorauthor in the form name <e-mail> (feed.Author.Name)
feed_languagelanguage of the feed (feed.Language)
feed_copyrightcopyright (feed.Copyright)
feed_generatorgenerator used to create the feed (feed.Generator)
In addition to the feed tags, the Extra will also contain the item’s fields. Since they could be different from feed to feed and it is possible to configure custom tag, you will find all them in the extra with their name.
Not all the Extra field could be filled. If the relative tag is not present on the feed it will be empty.
Examples
What's on this Page
Slack
This feeder creates a stream using the Slack Events API.
In order to use this feeder, you need to create a Slack App, define its Bot Token scopes and finally enable the Event Subscriptions.
The feeder will start a webserver to receive the events, so it should be reachable by slack. If you are behind a NAT or Firewall, you can enable localtunnel.
Based on slack-go/slack
Parameters
ParameterTypeDefaultDescription
bot_tokenSTRINGemptytoken Bot (starting with xoxb-*)
app_tokenSTRINGemptyApp token (starting with xapp-*)
verification_tokenSTRINGemptytoken to verify the requests (you can find it in the Basic information section)
addrSTRING“:3000”address and port in the form IP:PORT
socket_modeBOOLfalseenable the Socket Mode
lt_enableBOOLfalseenable localtunnel for the server
lt_baseurlSTRING“https://localtunnel.me”URL to the Proxy server
lt_subdomainSTRINGemptyspecify a subdomain to use with localtunnel
eventsSTRING*eventscomma separated event list to handle
ignore_botBOOLtrueif true the bot will ignore mentions and messages created by another bot
*events = “app_mention”,“app_home_opened”,“app_uninstalled”,“grid_migration_finished”,“grid_migration_started”,“link_shared”,“message”,“member_joined_channel”,“member_left_channel”,“pin_added”,“pin_removed”,“reaction_added”,“reaction_removed”,“tokens_revoked”,“file_shared”
... | <slack: token="xoxb-xxx", verification_token="xxxx", lt_enable="true", lt_subdomain="domaintesttt"> | ...
Output
The messages propagated by this feeder could contain different info and they depend from the type of the event it received.
Every propagated Message will have a type extra field with the name of the events (see the above list).
Event app_mention
The bot has been mentioned by someone.
The main field of the Message contain the text of the event.
NameDescription
typeit contain app_mention for this event
useruser who triggered the events
texttext of the message
timestamptimestamp
threadtimestamptimestamp of the thread
channelchannel name
eventtimestamptimestamp of the event
userteamfilled when a message comes from a channel that is shared between workspaces
sourceteamfilled when a message comes from a channel that is shared between workspaces
botidfilled out when a bot triggers the app_mention event
Event app_home_opened
User clicked into your App Home.
The main field of the Message contains the type name of the event.
NameDescription
typeit contain app_home_opened for this event
useruser who triggered the events
channelchannel name
eventtimestamptimestamp of the event
tabfilled when a message comes from a channel that is shared between workspaces
Event app_uninstalled
Your Slack app was uninstalled.
The main field of the Message contains the type name of the event.
NameDescription
typeit contain app_uninstalled for this event
Event grid_migration_finished
An enterprise grid migration has finished on this workspace.
The main field of the Message contains the type name of the event.
NameDescription
typeit contain grid_migration_finished for this event
enterpriseidenterprise id
Event grid_migration_started
An enterprise grid migration has started on this workspace.
The main field of the Message contains the type name of the event.
NameDescription
typeit contain grid_migration_started for this event
enterpriseidenterprise id
A message was posted containing one or more links relevant to your application.
The main field of the Message contains the shared URL.
NameDescription
typeit contain link_shared for this event
timestamptimestamp of the message
threadtimestamptimestamp of the thread
domaindomain of the shared link
linkshared link
Event file_shared
A message was posted containing one or more links relevant to your application.
The main field of the Message contains the text of the message used to share the file.
NameDescription
idid of the file upload
created
timestamptimestamp of the event
namename of the file
mimetypemimetype of the file
filetypeextension of the file
prettytype
useruser who triggered the event
sizesize of the file
urlprivatedownloadurl to download the file
imageexifrotation
originalwwidth of the file if it is an image
originalhheight of the file if it is an imange
permalink
permalinkpublic
Event message
A message was posted as direct message or in a channel/group.
The main field of the Message contains the text of the message.
NameDescription
typeit contain message for this event
timestamptimestamp of the message
threadtimestamptimestamp of the thread
botidfilled if the message has been sent by a bot
channelchannel id
channeltypetype of the channel (im, group, mpim, channel)
clientmsgidid of the message
subtypesub type of the message
texttext of the message
useruser who sent the message
usernamefilled if it is a bot_message
userteamfilled when the message comes from a channel that is shared between workspaces
sourceteamfilled when the message comes from a channel that is shared between workspaces
Event member_joined_channel
An user just joined in a channel or it has been invited.
The main field of the Message contains the type of the message.
NameDescription
typeit contain member_joined_channel for this event
useruser who has been invited
channelchannel id
channeltypetype of the channel (im, group, mpim, channel)
inviteruser who invited
teamfilled when the message comes from a channel that is shared between workspaces
Event member_left_channel
An user just left the channel.
The main field of the Message contains the type of the message.
NameDescription
typeit contain member_left_channel for this event
useruser who has been invited
channelchannel id
channeltypetype of the channel (im, group, mpim, channel)
teamfilled when the message comes from a channel that is shared between workspaces
Event pin_added / pin_removed
A pin has been added or removed.
The main field of the Message contains the type of the message.
NameDescription
typeit contain pin_added or pin_removed for this event
useruser who triggered the event
channelchannel id
eventtimestamptype of the channel (im, group, mpim, channel)
haspins
itemit contains the object slackevents.Item that can be used from the slack filter
Event reaction_added/reaction_removed
A member has added an emoji reaction to an item.
The main field of the Message contains the type of the message.
NameDescription
typeit contain reaction_added or reaction_removed for this event
useruser who triggered the event
itemuser
eventtimestamptimestamp of the event
itemit contains the object slackevents.Item that can be used from the slack filter
Examples
What's on this Page
Timer feeder
This feeder trigger a pipeline every time the timer is fired.
Parameters
ParameterTypeDefaultDescription
freqDURATION60sThe intervals (in duration) on how often to execute the pipeline
<timer: freq="30s"> | ...
Output
Text
The main field of the Message will contain time in rfc3339 format.
Extra
NameDescription
rfc3339time in rfc3339 format
timestamptime in Unix timestamp
Examples
What's on this Page
Twitter
This feeder creates a stream from tweets. It is possible to define the keywords, or the users to follow.
Based on go-twitter
Parameters
ParameterTypeDefaultDescription
bearerTokenSTRINGemptyTwitter Auth
keywordsSTRINGemptycomma separated keywords that should match on the tweets
usersSTRINGemptycomma separated users list
rulesSTRINGemptyset multiple custom rules separated by the char
languagesSTRINGemptyfilter by language (comma separated languages)
disable_retweetBOOL“false”don’t include retweets in the stream
disable_quotedBOOL“false”don’t include quoted tweets in the stream
... | <twitter: users="goofy, mickeymouse",keywords="movie, cartoon", rules="rule1:movie OR cartoon|rule2:mickey mouse OR pluto"> | ...
Output
Text
The main field of the Message will contain:
  • the text of the tweet if it is a normal tweet;
  • the text of the retweeted message if it is a retweet;
  • the text of the quote if it is a quote tweet;
Extra
NameDescription
linklink to the tweet
languagelanguage used for the tweet
usernameauthor of the tweet
author_idID of the author of the tweet
quoted“true” if the tweet is a quoted tweet, “false” otherwise
retweet“true” if the tweet is a retweeted tweet, “false” otherwise
response“true” if the tweet is a response for another tweet, “false” otherwise
reply_for_userit contains the userID if the tweet is a reply for a user
original_linklink of the original tweet if it is a retweet or quoted tweet
original_usernameusername of the tweet linked to the current one
original_namename of the author of the tweet linked to the current one
original_texttext of the tweet linked to the current one
original_useridID of the original author of the tweet if it is a retweet or a quote
matched_ruleslist of the matched rules (tags) comma separated
In some cases the extra field could be empty.
Examples
What's on this Page
Web
This feeder creates a stream starting from a web page. It is possible to define how often the page should be downloaded and parsed.
Every time the page is parsed a Message is sent down the lane.
Parameters
ParameterTypeDefaultDescription
urlSTRINGemptyURL of the web page
freqDURATION60show often the page should be parsed
text_onlyBOOL“false”if “true” it removes all the tags from the page
methodSTRING“GET”HTTP method to use on the requests
headersJSONemptyHeaders to use in the request
dataJSONemptyPOST fields to send with the requests (it’s not possible to use in combination with rawData)
rawDataSTRINGemptyraw body of the requests (it’s not possible to use in combination with data)
statusSTRINGemptythe filter will propagate the Message only if the returned status is this
cookiesSTRINGemptyPath of the JSON file containing the cookies to use
... | <web: url="https://example.com", freq="30m", status="200", cookies="/path/to/exported.json"> | ...
Output
Text
The main field of the Message will contain the HTML source or the text of the website if the text_only parameter is set to true.
Extra
NameDescription
urlURL of the web page
titlemeta tag title
descriptionmeta tag description
imagemeta tag image
sitenamemeta tag sitename
Not all the Extra field could be filled. If the relative tag is not present on the feed it will be empty.
Examples
What's on this Page
Cache
This filter introduces a simple cache mechanism in the rule. It is a TTL based cache and it can have a local visibility (cache is only visible to the current filter) or a global visibility (cache shared across ALL the rules).
If the target of the Message as input has been cached before, and his TTL is not expired, it will be dropped and not propagated to the next filter.
Otherwise if the target of the Message is new to the cache, it is inserted in it and propagated to the next filter.
Parameters
ParameterTypeDefaultDescription
targetSTRING“main”the field of the Message that should be used for the filter (it could be main or and extra field)
refresh_on_getBOOL“true”the TTL is refreshed if the key has been looked up
ttlDURATION24hhow long after the key will be deleted
sync_timeDURATION5mhow often the sync on file should be called
nameSTRING""named global cache: this cache will be available for ALL the rules also for different files
globalBOOL“false”make this cache global: like for the name param this cache is available to all the rules with the name “global”
fileSTRING""enable cache persistence. It loads and writes the cache from a file
ignore_first_runBOOL“false”if true the messages that arrives to the cache with the firstRun flag enabled will be send to the next filter
... | cache(ttl="24h", global="true") | ...
Output
The output is not being changed. This filter can only stop or not the propagation of the Message.
Examples
What's on this Page
Changed
This filter is similar to the cache. It can only stop the propagation of the Message across the lane, but only if the target of the received Message is different from the previous one.
Parameters
ParameterTypeDefaultDescription
targetSTRING“main”the field of the Message that should be used for the filter (it could be main or and extra field)
... | changed(target="original_author") | ...
Output
The output is not being changed. This filter can only stop or not the propagation of the Message.
Examples
What's on this Page
Echo
This filter prints the Message on the logs. It is mostly used to debug the rules.
Parameters
ParameterTypeDefaultDescription
extraBOOL“false”print also all the extra fields
... | echo(extra="false") | ...
Output
The output is not being changed. This filter prints the received Message and send it to the next filter in the rule.
Examples
What's on this Page
ElasticSearch
This filter writes on ElasticSearch all the message it receives as input and return as output the document ID.
You can also specify what message’s field it should use as input.
Parameters
ParameterTypeDefaultDescription
targetSTRING“main”the field of the Message that should be used for filter’s output (it could be main or an extra field)
... | elasticsearch(target="input_field") | ...
Output
The main field will contain the docID returned by ElasticSearch.
Examples
What's on this Page
File
This filter get as input the path of a local file, read it and return the content back to the pipeline.
Parameters
ParameterTypeDefaultDescription
targetSTRING“main”the field of the Message that should be used for filter’s output (it could be main or an extra field)
... | file(target="file_content") | ...
Output
The output will contain the content of the read file. Using the target parameter you can specify the output field’s name.
Examples
What's on this Page
Format
This filter is used to format the received Message. It is based on
the Golang templates and it can load templates from the template_path
directory specified in the configuration file.
Parameters
ParameterTypeDefaultDescription
typeSTRING“text”specify the type of template to use : "text" or "html"
templateSTRINGemptya template could be specified directly here, instead of load it from file
fileSTRINGemptyload the template from file
targetSTRING“main”the field of the Message that should be used for the filter (it could be main or an extra field)
In the template is allowed to use all the fields of the received Message: main or extra.
... | format(type="html", template="main : {{.main}} extra : {{.file_name}}") | ...
Output
The new formatted text is sent to the next filter in the main field of the Message if the target parameter is not specified.
The extra fields do not undergo changes.
Examples
What's on this Page
Hash
This filter is used to search or extract hashes from a Message.
Supported types of hashes are:
  • MD5
  • SHA1
  • SHA256
  • SHA512
Parameters
ParameterTypeDefaultDescription
targetSTRING“main”the field of the Message that should be used for the filter (it could be main or an extra field)
extractBOOL“false”if "true" the main field of the output Message will be the extracted hash
md5BOOL“true”if "false" md5 hashes will be ignored
sha1BOOL“true”if "false" sha1 hashes will be ignored
sha256BOOL“true”if "false" sha256 hashes will be ignored
sha512BOOL“true”if "false" sha512 hashes will be ignored
... | hash(target="description", extract="true") | ...
Output
If the extract parameter is “false”, the received Message will be propagated only if at least one hash is found in it.
Otherwise, if extract is “true” and the Message contains one or more hashes, the main field of the propagated Message will contain only the extracted hash.
If the extract parameter is “true” and the message is propagated, a new extra field is created: fulltext will contain the original target string.
If the targeted field contains multiple hashes, the filter will create and propagate multiple messages, one for each hash.
Examples
What's on this Page
Html
This filter is used to extract information from an HTML page received through the Message.
Like jQuery for JS, we can set a selector to find in the page, extract text from the tags and the html content (we are using goquery library under the hood).
Parameters
ParameterTypeDefaultDescription
targetSTRING“main”the field of the Message that should be used for the filter (it could be main or an extra field)
selectorSTRING""the selector to find in the HTML page
getSTRING“html”what do we want to retrieve on the tags found in the selected one: html, text, attr
attrSTRING""if get is attr you can define what attr name it should extract
... | html(selector=".link", get="attr", attr="href") | ...
Output
The filter will generate one or more Messages. It is possible to use more than 1 time this filter.
The field fulltext will contain the original target string.
Examples
What's on this Page
HTTP
This filter allows you to send HTTP requests. When a Message arrives to the filter, we can decide if use the main field of the Message as URL on the request, or its content for the HTTP data.
This behaviour can be handled with the parameters.
Parameters
ParameterTypeDefaultDescription
urlSTRINGemptyURL of the web page. It is possible use the Golang templates to use fields of the Message
download_toSTRINGemptypath of where to download the file. It is possible use the Golang templates to use fields of the Message
text_onlyBOOL“false”if “true” it removes all the tags from the body response
methodSTRING“GET”HTTP method to use on the request
headersJSONemptyHeaders to use in the request
dataJSONemptyPOST fields to send with the request (it’s not possible to use in combination with rawData)
rawDataSTRINGemptyraw body of the request (it’s not possible to use in combination with data)
statusSTRINGemptythe filter will propagate the Message only if the returned status has the specified value
cookiesSTRINGemptyPath of the JSON file containing the cookies to use
... | http(url="{{ .main }}", cookies="exported.json", headers="{\"Content-type\": \"application/json\"}") | ...
Output
If the request was successful, the output Message will have the main field set to the HTTP body response. If the status is set, and the response http status is different from it, the Message will be dropped.
The Message is dropped if the request is failed.
Examples
What's on this Page

\ No newline at end of file
diff --git a/doc/filters/index.xml b/doc/filters/index.xml
index b8713e8..cf45ff0 100644
--- a/doc/filters/index.xml
+++ b/doc/filters/index.xml
@@ -5,7 +5,8 @@ Parameters Parameter Type Default Description target STRING &ldquo;main&
 Output The output will contain the content of the read file.Cachehttps://matrix86.github.io/driplane/doc/filters/cache/Wed, 16 Sep 2020 22:38:02 +0200https://matrix86.github.io/driplane/doc/filters/cache/Cache This filter introduces a simple cache mechanism in the rule. It is a TTL based cache and it can have a local visibility (cache is only visible to the current filter) or a global visibility (cache shared across ALL the rules).
 If the target of the Message as input has been cached before, and his TTL is not expired, it will be dropped and not propagated to the next filter.JShttps://matrix86.github.io/driplane/doc/filters/js/Wed, 16 Sep 2020 22:38:02 +0200https://matrix86.github.io/driplane/doc/filters/js/XLShttps://matrix86.github.io/driplane/doc/filters/xls/Thu, 16 Nov 2023 20:08:45 +0200https://matrix86.github.io/driplane/doc/filters/xls/XLS This filter allows you to extract all the rows from a Excel file.
 Based on the qax-os/excelize library.
-Parameters Parameter Type Default Description target STRING &ldquo;main&rdquo; the field of the Message that should be used for the filter (it could be the main or and extra field) filename STRING empty the filename of the XLS file to parse (supports Golang templates) .Jsonhttps://matrix86.github.io/driplane/doc/filters/json/Sun, 29 Jan 2023 18:57:01 +0200https://matrix86.github.io/driplane/doc/filters/json/Json This filter is used to extract information from a JSON doc received through the Message.
+Parameters Parameter Type Default Description target STRING &ldquo;main&rdquo; the field of the Message that should be used for the filter (it could be the main or and extra field) filename STRING empty the filename of the XLS file to parse (supports Golang templates) .RateLimithttps://matrix86.github.io/driplane/doc/filters/ratelimit/Tue, 31 Jan 2023 18:50:23 +0200https://matrix86.github.io/driplane/doc/filters/ratelimit/RateLimit This filter allows you to set a rate limit on the messages that can go through it. So for example if we don&rsquo;t want to limit the number of messages in a pipe to 5 messages per second we just to set the parameter rate to 5.
+Parameters Parameter Type Default Description rate STRING &ldquo;0&rdquo; how many event per second you want to have as rate limiter .Jsonhttps://matrix86.github.io/driplane/doc/filters/json/Sun, 29 Jan 2023 18:57:01 +0200https://matrix86.github.io/driplane/doc/filters/json/Json This filter is used to extract information from a JSON doc received through the Message.
 It is possible to use XPath query for JSON specifying a selector to search in the doc and extract data (it uses jsonquery).
 Parameters Parameter Type Default Description target STRING &ldquo;main&rdquo; the field of the Message that should be used for the filter (it could be main or an extra field) selector STRING &quot;&quot; the selector to find the data in the JSON .Htmlhttps://matrix86.github.io/driplane/doc/filters/html/Tue, 19 Jul 2022 23:38:02 +0200https://matrix86.github.io/driplane/doc/filters/html/Html This filter is used to extract information from an HTML page received through the Message.
 Like jQuery for JS, we can set a selector to find in the page, extract text from the tags and the html content (we are using goquery library under the hood).
diff --git a/doc/filters/js/basics/index.html b/doc/filters/js/basics/index.html
index cb06062..f24e587 100644
--- a/doc/filters/js/basics/index.html
+++ b/doc/filters/js/basics/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
JS
This filter allows to extend the basic driplane’s filter, defining Javascript scripts. It is based on islazy/plugin which in turn is based on robertkrimen/otto.
Defining a JS file with our custom logic, it is possible create a complex filter.
Parameters
ParameterTypeDefaultDescription
pathSTRINGemptypath of the Javascript file (it can contains multiple functions)
functionSTRINGemptyname of the function in the JS file to call when a Message is received
... | js(path="script.js", function="MyFunction") | ...
Output
The output Message of this filter depends on the return value of the JS function itself.
Examples
What's on this Page
Entrypoint
The JS file consists of one or more functions, and they can be used in different filters or in combination between them.
The function must respect some constraints:
  • the function’s name contained in the function parameter of the filter has to start with a capital letter;
  • the function prototype must 3 variables;
  • the function must return a JS object with at least the filtered field;
Function’s prototype
The function’s name specified in the function parameter of the filter, receives 3 input parameters:
  • main: it is a string with the content of the field main of the input Message;
  • extra: a JS object that can be seen like an associative array, containing the extra fields of the input Message;
  • params: a JS object like the previous, but it contains the configurations from the custom and the general sections.
So for example we can define a function like the follow:
1
+
  • XLS
  • RateLimit
  • Json
  • Html
  • Striptag
  • Slack
  • Changed
  • Echo
  • Format
  • Hash
  • Http
  • Mail
  • MIME type
  • Override
  • Pdf
  • Random
  • System
  • Text
  • Url
    • Entrypoint
    The JS file consists of one or more functions, and they can be used in different filters or in combination between them.
    The function must respect some constraints:
    • the function’s name contained in the function parameter of the filter has to start with a capital letter;
    • the function prototype must 3 variables;
    • the function must return a JS object with at least the filtered field;
    Function’s prototype
    The function’s name specified in the function parameter of the filter, receives 3 input parameters:
    • main: it is a string with the content of the field main of the input Message;
    • extra: a JS object that can be seen like an associative array, containing the extra fields of the input Message;
    • params: a JS object like the previous, but it contains the configurations from the custom and the general sections.
    So for example we can define a function like the follow:
    1
 2
 3
 
    function Entry(main, extra, params) {
diff --git a/doc/filters/js/index.html b/doc/filters/js/index.html
index c6e6a93..e81bd77 100644
--- a/doc/filters/js/index.html
+++ b/doc/filters/js/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    
\ No newline at end of file
diff --git a/doc/filters/js/packages/index.html b/doc/filters/js/packages/index.html
index 439bf97..23a6367 100644
--- a/doc/filters/js/packages/index.html
+++ b/doc/filters/js/packages/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    Packages
    Since the JS doesn’t have all the useful functions to perform operations like on file manipulation or http request (see otto docs for more info), driplane maps and export to the JS environment some Go functions as packages.
    File
    This package contains functions for manipulating files.
    Return
    The return value will be a JS object containing 2 fields:
    NameTypeDescription
    StatusBOOLif true the operation was successful
    ErrorSTRINGif status is false it contains the reason of the failure
    Functions
    PrototypeDescription
    file.Move(src string, dest string)Move the file src to the new position dest
    file.Copy(src string, dest string)Copy the file src to the position dest
    file.Truncate(filename string, size int)set or adjust the file called filename by size bytes
    file.Delete(filename string)Remove the file called filename
    file.Exists(filename string)Return Status = true if filename exists on disk
    file.AppendString(filename string, text string)Append the string text to the file filename
    Log
    This package contains functions for writing strings on the logs.
    Return
    It doesn’t have a return value.
    Functions
    PrototypeDescription
    log.Info(format string, …)Write an Info line on the log
    log.Error(src string, …)Write an Error line on the log
    log.Debug(src string, …)Write a Debug line on the log
    Strings
    This package contains functions for manipulating strings.
    Return
    The return value will be a JS object containing 2 fields:
    NameTypeDescription
    StatusBOOLif true the operation was successful
    ErrorSTRINGif status is false it contains the reason of the failure
    Functions
    PrototypeDescription
    strings.StartsWith(str string, substr string)Status = true if str start with substr
    Util
    This package contains miscellaneous functions.
    Return
    The return value will be a JS object containing 2 fields:
    NameTypeDescription
    StatusBOOLif true the operation was successful
    ErrorSTRINGif status is false it contains the reason of the failure
    ValueSTRINGthis string contains the returned value
    Functions
    PrototypeDescription
    util.Sleep(seconds int)wait for seconds seconds
    util.Getenv(name string)get the value of the name environment variables, if it exists, and return it on the Value field of the returned value
    util.Md5File(filename string)calculate the MD5 hash of the filename and return it on the Value field
    Http
    This package contains functions to perform HTTP requests.
    Return
    The return value will be a JS object containing 4 fields:
    NameTypeDescription
    StatusBOOLif true the operation was successful
    ErrorSTRINGif status is false it contains the reason of the failure
    ResponseOBJECTit returns the http.Response object
    BodySTRINGit contains the body of the response converted in string
    Functions
    PrototypeDescription
    http.Request(method string, uri string, headers interface{}, data interface{})
    http.Get(url string, headers map[string]string)
    http.Post(url string, headers map[string]string, data interface{})
    http.DownloadFile(filepath string, method string, uri string, headers interface{}, data interface{})
    http.UploadFile(filename string, fieldname string, method string, uri string, headers interface{}, data interface{})
    Cache
    This package contains functions for add and get values from the global cache.
    Return
    The return value will be a JS object containing the following fields:
    NameTypeDescription
    StatusBOOLif true the operation was successful
    ErrorSTRINGif status is false it contains the reason of the failure
    ValueSTRINGif status is true it contains the resulting value
    Functions
    PrototypeDescription
    cache.Put(key string, value string, ttl int64)add the value in the cache using the key key and it will be deleted after ttl seconds
    cache.Get(key string)get the value stored in the cache with the key key
    What's on this Page
    Json
    This filter is used to extract information from a JSON doc received through the Message.
    It is possible to use XPath query for JSON specifying a selector to search in the doc and extract data (it uses jsonquery).
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be main or an extra field)
    selectorSTRING""the selector to find the data in the JSON
    ... | json(selector="id", target="doc") | ...
    Output
    The filter will generate one or more Messages. It is possible to use more than 1 time this filter.
    The field fulltext will contain the original target string.
    Examples
    What's on this Page
    Mail
    This filter allows you to send an e-mail. When a Message arrives to the filter, we can decide if use the main field of the Message as URL on the request, or its content for the HTTP data.
    This behaviour can be handled with the parameters.
    Parameters
    ParameterTypeDefaultDescription
    bodySTRINGemptythe body of the e-mail (supports Golang templates)
    usernameSTRINGemptyusername for the host authentication
    passwordSTRINGemptypassword for the host authentication
    hostSTRINGemptyhost server used to send the e-mail
    portSTRINGemptyport of the host server
    fromAddrSTRINGemptysource e-mail address
    fromNameSTRINGemptysource name address
    toSTRINGemptydestination e-mail address (supports multi-destination, comma separated)
    subjectSTRINGemptysubject field of the e-mail to send
    use_authBOOL“false”if “true” the sendmail server will receive the credentials specified in username and password fields
    ... | http(url="{{ .main }}", cookies="exported.json", headers="{\"Content-type\": \"application/json\"}") | ...
    Every default’s value can be set in the configuration, creating a section with the name of theFilter/Feeder.
    more info
    Output
    The input Message is always propagated to the next filter without changes.
    Examples
    What's on this Page
    MIME type
    This filter allows you to detect the MIME type of a file and its extension.
    Based on the gabriel-vasile/mimetype library.
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be the main or and extra field)
    filenameSTRINGemptythe filename of the file to detect (supports Golang templates)
    ... | mime(target="{{ .extra_field }}") | ...
    The filename field override the target. They are mutually exclusive, so you can specify only one of them.
    Output
    The propagated Message will contain the mimetype’s string in the main field and the extension in the extra field mimetype_ext.
    Examples
    What's on this Page
    Override
    This filter allows you to change a field of a Message, before sending it to the next filter. Template can be used.
    Parameters
    ParameterTypeDefaultDescription
    nameSTRINGemptyname of the field to change (supports Golang templates)
    valueSTRINGemptynew value to assign to the Message’s field specified (supports Golang templates)
    ... | override(name="description", value="{{ .title }}") | ...
    Output
    The propagated Message will be identical to the original, with only the specified field changed.
    Examples
    What's on this Page
    Pdf
    This filter allows you to extract plain text from a PDF file.
    Based on the ledongthuc/pdf library.
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be the main or and extra field)
    filenameSTRINGemptythe filename of the PDF file to parse (supports Golang templates)
    ... | pdf(target="{{ .extra_field }}") | ...
    The filename field override the target. They are mutually exclusive, so you can specify only one of them.
    Output
    The propagated Message contains the plain text of the input PDF file (fulltext will be set to the file name received as input).
    Examples
    What's on this Page
    Random
    This filter is used to inject an extra field with a random number in the propagated Message.
    Parameters
    ParameterTypeDefaultDescription
    outputSTRING“main”the field of the propagated Message that will contain the random number
    minSTRING“0”the min value of the extracted number is min
    maxSTRING“999999”the max value of the extracted number is max
    ... | random(output="random_field", min="9", max="90") | ...
    Output
    The output Message will be equal to the input, but it will also include the new random field.
    Examples
    What's on this Page
    Driplane
    RateLimit
    RateLimit
    RateLimit
    This filter allows you to set a rate limit on the messages that can go through it. So for example if we don’t want to limit the number of messages in a pipe to 5 messages per second we just to set the parameter rate to 5.
    Parameters
    ParameterTypeDefaultDescription
    rateSTRING“0”how many event per second you want to have as rate limiter
    ... | ratelimit(rate="5") | ...
    Output
    The filter will slow down the rate of the messages as specified on the parameter rate.
    Examples
    What's on this Page
    
\ No newline at end of file
diff --git a/doc/filters/slack/index.html b/doc/filters/slack/index.html
index 99e51c2..b2df491 100644
--- a/doc/filters/slack/index.html
+++ b/doc/filters/slack/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    Slack
    This filter allows you to send files, messages and download files from Slack.
    It can be used alone or with the Slack Feeder to create a simple Slack bot for events API.
    Parameters
    The following parameters are required from this filter:
    ParameterTypeDefaultDescription
    actionSTRING“send_message”action to perform: “send_message”, “send_file”, “download_file”, “user_info”
    tokenSTRING""Slack bot Token
    Each action can use different parameters:
    action = send_message
    ParameterTypeDefaultDescription
    toSTRING""channel ID or User ID that should receive the message (supports Golang templates)
    targetSTRING“main”if text is not used you can choose which field of Message to use as text
    textSTRING""you can define the text of the message to send (supports Golang templates)
    blocksBOOLfalseif true you can use the Slack template blocks (block builder)
    action = send_file
    ParameterTypeDefaultDescription
    toSTRING""channel ID or User ID that should receive the message (supports Golang templates)
    targetSTRING“main”if filename is not specified you can choose which field of Message has to be used as file content to send
    filenameSTRING""path of the file to send (supports Golang templates)
    action = download_file
    ParameterTypeDefaultDescription
    urlSTRING""it should contain the Slack private url for the file download (supports Golang templates)
    targetSTRING“urlprivate”if url is not specified you can choose which field of Message contains the Slack private url
    filenameSTRING""path of where to save the downloaded file. If not specified the file content will be inserted in the main field (supports Golang templates)
    action = user_info
    ParameterTypeDefaultDescription
    targetSTRING“user”this field has to contain the USERID
    In the output Message you can find all the user information returned by Slack:
    	user_id                
+
  • XLS
  • RateLimit
  • Json
  • Html
  • Striptag
  • Slack
  • Changed
  • Echo
  • Format
  • Hash
  • Http
  • Mail
  • MIME type
  • Override
  • Pdf
  • Random
  • System
  • Text
  • Url
    • Slack
    This filter allows you to send files, messages and download files from Slack.
    It can be used alone or with the Slack Feeder to create a simple Slack bot for events API.
    Parameters
    The following parameters are required from this filter:
    ParameterTypeDefaultDescription
    actionSTRING“send_message”action to perform: “send_message”, “send_file”, “download_file”, “user_info”
    tokenSTRING""Slack bot Token
    Each action can use different parameters:
    action = send_message
    ParameterTypeDefaultDescription
    toSTRING""channel ID or User ID that should receive the message (supports Golang templates)
    targetSTRING“main”if text is not used you can choose which field of Message to use as text
    textSTRING""you can define the text of the message to send (supports Golang templates)
    blocksBOOLfalseif true you can use the Slack template blocks (block builder)
    action = send_file
    ParameterTypeDefaultDescription
    toSTRING""channel ID or User ID that should receive the message (supports Golang templates)
    targetSTRING“main”if filename is not specified you can choose which field of Message has to be used as file content to send
    filenameSTRING""path of the file to send (supports Golang templates)
    action = download_file
    ParameterTypeDefaultDescription
    urlSTRING""it should contain the Slack private url for the file download (supports Golang templates)
    targetSTRING“urlprivate”if url is not specified you can choose which field of Message contains the Slack private url
    filenameSTRING""path of where to save the downloaded file. If not specified the file content will be inserted in the main field (supports Golang templates)
    action = user_info
    ParameterTypeDefaultDescription
    targetSTRING“user”this field has to contain the USERID
    In the output Message you can find all the user information returned by Slack:
    	user_id                
 	user_teamid            
 	user_name              
 	user_deleted           
diff --git a/doc/filters/striptag/index.html b/doc/filters/striptag/index.html
index 267507c..c57c7b8 100644
--- a/doc/filters/striptag/index.html
+++ b/doc/filters/striptag/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    StripTag
    This filter is used to remove all the HTML tags from a string.
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be main or an extra field)
    ... | striptag(target="main") | ...
    Output
    The main of the output Message will have the text extracted from the target config, stripped by all the HTML tags.
    A new extra field is created: fulltext will contain the original target string.
    Examples
    What's on this Page
    System
    This filter allows you to exec a command on the host machine. The received Message can be used to create the command to launch.
    It supports Golang templates (only text.Template).
    Parameters
    ParameterTypeDefaultDescription
    cmdSTRINGemptycommand line to exec for each received Message (supports Golang templates)
    ... | system(cmd="echo '{{ .author }} wrote {{ .main }}' >> logs.txt") | ...
    Output
    The propagated Message will contain the output of the command if it is provided, and it is not failed.
    Examples
    What's on this Page
    Text
    This filter searches or extracts strings from the received Message. It can be used with a regular expression or a simple string.
    If the string is found, the condition is matched and the Message is propagated to the next filter.
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be main or and extra field)
    regexpBOOLfalsethe pattern field is a regular expression
    extractBOOL“false”if “true” the main field of the propagated Message will contain the extracted string (it can be used only if regexp parameter is set true)
    patternSTRINGemptyspecifies the pattern that should be matched on the Message to check the condition
    ... | text(target="description", pattern="(#[^\\s]+)", regexp="true", extract="true") | ...
    Output
    If the extract parameter is “false”, the received Message will be propagated only if the specified pattern is matched in the target field of the Message.
    Otherwise if extract is “true” (only regexp can be used in this case), and one or more strings matches with the pattern, the main field of the propagated Message will contain only the matched string.
    If the extract parameter is “true” and the message is propagated, a new extra field is created: fulltext will contain the original target string.
    If the targeted field contains multiple matches, the filter will create and propagate multiple Messages, one for each matched string.
    Examples
    What's on this Page
    Url
    This filter is used to search or extract URLs from a Message.
    Currently supported types of URLs are:
    • http/s
    • ftp
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be main or an extra field)
    httpBOOL“true”if “false”, http scheme urls are ignored
    httpsBOOL“true”if “false”, https scheme urls are ignored
    ftpBOOL“true”if “false”, ftp scheme urls are ignored
    extractBOOL“true”if “true”, the main field of the propagated Message will contain the found URL
    ... | url(extract="true") | ...
    Output
    If the extract parameter is “false”, the received Message will be propagated only if at least one URL is found in it.
    Otherwise, if extract is “true” and the Message contains one or more URLs, the main field of the propagated Message will contain only the extracted URLs.
    If the extract parameter is “true” and the message is propagated, a new extra field is created: fulltext will contain the original target string.
    If the targeted field contains multiple URLs, the filter will create and propagate multiple messages, one for each URL.
    Examples
    What's on this Page
    XLS
    This filter allows you to extract all the rows from a Excel file.
    Based on the qax-os/excelize library.
    Parameters
    ParameterTypeDefaultDescription
    targetSTRING“main”the field of the Message that should be used for the filter (it could be the main or and extra field)
    filenameSTRINGemptythe filename of the XLS file to parse (supports Golang templates)
    ... | xls(target="{{ .extra_field }}") | ...
    The filename field override the target. They are mutually exclusive, so you can specify only one of them.
    Output
    The filter produces one Message for each row of the XLS file.
    Examples
    What's on this Page
    Introduction
    Driplane allows you to create an automatic alerting system or start automated tasks triggered by events.
    You can keep under control a stream source as Twitter, a file, a RSS feed or a website.
    It includes a mini language that allows you to define the filtering pipelines (the rules), and it is extensible thanks to an integrated JS plugin system.
    With driplane you can create several rules starting from one or more streams, filter the content in the pipeline and launch tasks or send alerts if some event occurred.
    The documentation can be found HERE
    How it works
    The user can define one or more rules. Usually a rule contains a source (feeder), which takes care of getting the information and sending updates (Message) through the pipeline, and one or more filters.
    The filters' job is to choose whether to propagate or not the Message to the next filter in the pipeline relying on a condition, or change the Message received before to propagate it. The Message will be propagated only if it verifies the condition.
    Use cases
    Using driplane it is possible to:
    • keep track of keywords or users on Twitter, receive the new tweets or quoted tweets from them, search for URLs or particular strings in them and send a Telegram or a Slack message through their webhooks.
    • keep track of a RSS feed or a website, and download and store on file all the new changes to them.
    • keep track of changes on a file, and launch alert if a particular condition is verified.
    The rules and the JS plugins allow you to create very complex custom logics.
    Usage
    Usage of ./bin/driplane:
   -config string
     	Set configuration file.
diff --git a/doc/installation/build/index.html b/doc/installation/build/index.html
index 0d7ec54..f5c18a8 100644
--- a/doc/installation/build/index.html
+++ b/doc/installation/build/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    Build and install
    Launching the follow command go will take care of the downloading, the dependency resolving, the building and finally of the installation of the binary in the $GOBIN directory.
    What's on this Page
    Docker
    driplane is containerized using a really lightweight Linux distribution called Alpine Linux.
    To pull the latest image version:
    To run it:
    where the config directory contains the config.yaml file, the rule directory, the js directory and the templates directory.
    Link to the Docker repository
    What's on this Page
    
\ No newline at end of file
diff --git a/doc/rules/definition/index.html b/doc/rules/definition/index.html
index 48e5292..29d6596 100644
--- a/doc/rules/definition/index.html
+++ b/doc/rules/definition/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    First look
    Driplane include a simple language to define where to get the stream and what operation to exec, or how to filter the data.
    In the rule could be defined 3 type of nodes:
    • FEEDER : it is the node responsible for creating of a stream of data (it reads every changes on a file, gets tweets from Twitter, etc..)
    • FILTER : it receives data from a feeder or another filter and checks some conditions on them or makes some changes on them. It sends the data to the next filter ONLY if the condition is verified.
    • RULE CALL : every rule has a name, so you can define a rule with a preset feeder or a pipe of filters and connect them to another filter/feeder.
    It is possible to define custom parameters for feeders or filters. Each one of them has a different type of parameters that can change their behaviour and you can find a list of them in the related section.
    What's on this Page
    
\ No newline at end of file
diff --git a/doc/rules/syntax/index.html b/doc/rules/syntax/index.html
index c781c45..92182d7 100644
--- a/doc/rules/syntax/index.html
+++ b/doc/rules/syntax/index.html
@@ -80,7 +80,7 @@
 Feeders
 Filters
 
    Syntax
    The syntax of the driplane’s rules is very simple. As for the BASH every filter’s output is sent to the next filter’s input concatenating the two of them with the | character.
    All the rules have to start with a name and end with a ; char.
    Import
    Using the directive import it is possible to include one or more files and their rules in another one. All the rules defined in an included file
    are available for the file that imports them.
    To use this directive the path of the file can be absolute or relative to the rules directory specified in the configuration:
    Example:
    #import "file_to_import.rule"
    Rule Name and Rule Call
    Each rule has to start with an identifier follow by =>. This identifier identifies rule name and it could be used in another rule to concatenate 2 rules together.
    This name must be unique in the rules.
    Example:
    IDENTIFIER => ... ;
    It can be included in another rule (rule call) prepending the @ to it.
    Example:
    IDENTIFIER1 => ... ;
    IDENTIFIER2 => ... | @IDENTIFIER1 | ... ;
    Feeder
    The feeder creates the stream, so they don’t accept inputs. For this reason, they can be positioned ONLY to the beginning of a rule.
    The feeder definition starts with a < char followed by an identifier. That’s the type of the feeder we want to use.
    After the type we found a : char followed by a list of parameters comma separated and a >.
    The parameters are in the form of key/value where the value is between double quotes key="value".
    Example:
    IDENTIFIER => <FEEDER_TYPE: param1="value1", param2="value2"> | ... ;
    Filter
    The filters are the main operators of a rule, because they decide if a data is interesting and perform operations.
    The definition of a filter start with his name and it is followed by parameters contained between ( and ).
    According to the settings a Filter can change his behaviour and can modify the data passing through it.
    Example:
    IDENTIFIER => ... | FILTER_TYPE( param1="value1", ... ) | ... ;
    The operator NOT ! can be used on filter to negate his result (propagate the data if the condition is not verified).
    It has to be put before the filter definition: !FILTER_TYPE(...)
    All the parameters have to be enclosed in quotes.
    JSON requires double quotes to encode strings, so in order to define a JSON string you need to escape the quotes \".
    Data message and Extra
    The data stream in driplane is based on text and the basic object that is part of it is the Message.
    The Message is an object that contains the text that needs to be filtered and extra.
    The main string is identified as text in the filters, whereas the extra data are identified by a key.
    There are fixed extra, created from driplane itself and other extras related to a feeder or filter.
    NameDescription
    source_feederthe name of the feeder creates this Message
    rule_namethe name of the rule that contains this filter
    What's on this Page