[JMX Metric Gatherer] Formalize relationship to the OpenTelemetry Collector

[djaglowski]

This issue is related to 6750 (collector-contrib) and follows from a preliminary discussion in today's Java SIG.

The goal of this issue is to formalize that the JMX Metric Gatherer is intended only for use as a subcomponent of the OpenTelemetry collector, OR that it must be maintained as a standalone tool intended for use in other contexts as well.

The motivation for this is as follows: If the JMX Metric Gatherer is explicitly a subcomponent of the OpenTelemetry Collector, then this reduces the scope of maintenance obligations for both the Collector and the JMX Metric Gatherer. The interface between the two can be changed arbitrarily, so long as the Collector is able to manage the JMX Metric Gatherer as necessary for its use cases. Formalizing this relationship would mean that the two components can be very tightly coupled, likely leading to improved security.

I want to be clear that it is not my intent to invalidate anyone else's use cases for the JMX Metric Gatherer. Rather, I am asking whether there are any users besides the Collector because it is currently not clear.

[djaglowski]

cc: @rmfitzpatrick, @dehaansa, @Mrod1598, @trask

Please also tag anyone else who you think can help us make the right decision on this.

[trask]

@anuraaga found a Go library for fetching JMX metrics while we were discussing this yesterday: https://github.com/newrelic/nri-jmx

I like the idea of the JMX metric "scraper" being integrated tightly into the collector, modeling it after prometheus scraping (implemented internally either as a JVM subprocess or a rewrite in Go).

[dehaansa]

I dug into it a bit and nri-jmx does appear to start a java subprocess under the hood, so I don't think it would resolve the concerns about the JMX receiver subprocess.

[rmfitzpatrick]

Some planning/origin notes for the JMX metric gatherer are in the community issue and its linked PRs: open-telemetry/community#449. The library wasn't intended to solely be for the corresponding collector receiver but given the nature of the collector and reporting in otlp their integration is likely unavoidable for a number of nontrivial uses. The collector component is mainly to simplify the metric gatherer's use for those who don't want to configure two executables to be able to obtain metrics from a single target service when configuring one can do.

Though #5 hasn't been internally prioritized by my group at Splunk, and I haven't had the cycles to flesh out a spike for it on my own, it's something that I can see being a more idealized interface for the metric gatherer and a way to invite more community-provided target mbean scraping definitions (potentially one that could better justify it as a standalone offering).

[carlosalberto]

@djaglowski So it looks like the JMX gatherer is supposed to also exist as a standalone component. What's the follow up from here? Happy to help if anything else is needed.

[djaglowski]

Thanks for the feedback everyone. As @carlosalberto mentioned, it's been established that the JMX Metric Gatherer is expected to run as a standalone executable. Based on this, I'm closing this issue.

I will formulate a proposal to address collector-contrib#6750 with this understanding in mind. If it appears necessary to make changes to the JMX Metric Gatherer in order that the collector's jmxreceiver can to run it securely, I will open new issues to propose them.