Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document log processors and component templates #4219

Open
mdbirnstiehl opened this issue Sep 4, 2024 · 0 comments
Open

Document log processors and component templates #4219

mdbirnstiehl opened this issue Sep 4, 2024 · 0 comments
Assignees

Comments

@mdbirnstiehl
Copy link
Contributor

mdbirnstiehl commented Sep 4, 2024

We have a list of logs component templates that aren't documented anywhere, aside from logs@custom. We should mention all the defaults, when they are applied and how users can make use of them if they don't follow the naming conventions. These include the following:

logs@mappings: general mappings for log data streams that include disabling automatic date detection from string fields and specifying mappings for data_stream ECS fields.
logs@settings: general settings for log data streams including the following:

The default lifecycle policy that rolls over when the primary shard reaches 50 GB or after 30 days.
The default pipeline uses the ingest timestamp if there is no specified @timestamp and places a hook for the logs@custom pipeline. If a logs@custom pipeline is installed, it’s applied to logs ingested into this data stream.
Sets the ignore_malformed flag to true. When ingesting a large batch of log data, a single malformed field like an IP address can cause the entire batch to fail. When set to true, malformed fields with a mapping type that supports this flag are still processed.
ecs@mappings: dynamic templates that automatically ensure your data stream mappings comply with the Elastic Common Schema (ECS).

@mdbirnstiehl mdbirnstiehl self-assigned this Sep 4, 2024
@mdbirnstiehl mdbirnstiehl changed the title Document log processors Document log processors and component templates Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant