[receiver/datadog] Span Obfuscation

[lopes-felipe]

Description

Datadog Agent's span processor has an obfuscation pipeline that—among other things—obfuscates sensitive data, cleans up span resource names, and extracts particular components into specific span metadata.

This PR implements the same behavior into the datadogreceiver, but disabled by default for retro compatibility with existing users.

Link to tracking issue

Fixes

Testing

Documentation

[MovieStoreGuy]

@lopes-felipe,

Have you checked to see if any of the existing processors would provide this functionality?

I am hesitant to allow adding this into the receiver when it could already be achieved with existing functionality. There is also the added concern that this component is effectively bundling the datadog agent into the collector which doesn't sound ideal.

[lopes-felipe]

@lopes-felipe,

Have you checked to see if any of the existing processors would provide this functionality?

I am hesitant to allow adding this into the receiver when it could already be achieved with existing functionality. There is also the added concern that this component is effectively bundling the datadog agent into the collector which doesn't sound ideal.

Hey @MovieStoreGuy,

Thank you for the review. Yes, I have checked all the existing processors, and none of them have the capability to clean or/and obfuscate the span data in such a context-aware/structured way as it's done here.

For instance, like this simple test demonstrates, it takes a SQL query with comments, aliases, and repetitions (ref), and turn it into a flat, un-aliased, un-commented, deduplicated string (ref), something that is just not possible with any other processor currently available.

I would be more than happy to propose something like this for an existing processor such as Span Processor though, but I guess we could first prove the value of such a focused processing approach as this in a more narrowed-scope component such as Datadog Receiver here.

Finally, regarding bundling the Datadog Agent into the Collector, please correct me if I'm wrong here, but I don't think this particular feature implementation is moving this (very reasonable) concern any further, considering what has already taken place by the Datadog Exporter, and for the very own Datadog Receiver. Still, again, I might be missing something here.

Thank you again for the thoughtful review :)