Filebeat elasticsearch module uses datastream constant_keyword Fields

[mag-mkorn]

In version 8.x the filebeat elasticsearch module writes data to the following fields:

• data_stream.dataset
• data_stream.namespace
• data_stream.type

These fields are mapped with type constant_keyword, which results in wrong information for subsequent events as filebeat does not use dedicated datastreams per input, like the agent does.

AFAIK these fields should not be used by beats.

The beats test files can be used as example:
https://github.com/elastic/beats/blob/main/filebeat/module/elasticsearch/deprecation/test/es_deprecation-json.800.log-expected.json

As a workaround I changed the mapping in the template to keyword.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[mag-mkorn]

Still relevant in 8.6.2.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[mag-mkorn]

Still relevant.