Native Pubsub support

[owenhaynes]

Request

Allow for native Pubsub support to fetch logs and process

Use case

I know we currently have gcplog, the issue with this it expects log messages to be the GCP Log Format.

We currently abuse this with getting non-gcp logs into Alloy, so a lot of the metadata is unpopulated, it does not complain. I am not sure what issues it is having under the hood by doing this, and if changes later on could break this behaviour.

It would more ideal if we could have just a native receiver for pubsub which could fetch other JSON payloads(other formats could be supported), and not have any of the gcp log processing logic.

It could even make sense to make the gcp-log processing into its own node, but that maybe more of a breaking change.

[ptodev]

Hello! Would the OTel Google Pubsub Receiver resolve your issue? We could port it over as an otelcol component.

[owenhaynes]

yes that would work

[ptodev]

I'd be happy for us to incorporate it. I just don't know if (and when) the Alloy dev team would have time to work on it. In the meantime, if anyone is interested in submitting a PR then I'd be happy to review it. This could also be a community component similar to the Splunk exporter.

[owenhaynes]

If I get time over the next few weeks, I will give this a shot, does not look to difficult to integrate.