Skip to content

Commit

Permalink
make Protobuf optional (#1190)
Browse files Browse the repository at this point in the history
* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* make protobuf optional

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* plugin management

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* smoke test

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* smoke test

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* smoke test

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* smoke test

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* release automation

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* cache delegate

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* annotations should be provided

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* fix rebase

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* fix rebase

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* checkstyle

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

* add docs

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

---------

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
  • Loading branch information
zeitlinger authored Nov 22, 2024
1 parent 03609eb commit 66f4dc8
Show file tree
Hide file tree
Showing 57 changed files with 987 additions and 342 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ jobs:
PROTO_GENERATION: true
REQUIRE_PROTO_UP_TO_DATE: true
run: |
echo "Java version: $(java -version) in $(which java), Maven version: $(mvn -v)"
echo "JAVA_HOME: $JAVA_HOME"
mvn -v
./mvnw clean install
./mvnw javadoc:javadoc -P javadoc # just to check if javadoc is generated
21 changes: 21 additions & 0 deletions .github/workflows/native-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: GraalVM Native Tests

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
native-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK
uses: actions/setup-java@v4
with:
java-version: '21'
distribution: graalvm
cache: 'maven'
- name: Run the Maven verify phase
run: ./scripts/run-native-tests.sh
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
cache: 'maven'

- name: Build with Maven
run: mvn -B package -P release -Dmaven.test.skip=true
run: ./scripts/build-release.sh ${{ github.ref_name }}

- name: Set up Apache Maven Central
uses: actions/setup-java@v4
Expand Down
10 changes: 0 additions & 10 deletions RELEASING.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,3 @@
## Update Version

In a new PR, update the version in `pom.xml` using

```shell
mvn versions:set -DnewVersion=<VERSION>
```

Commit the changes and open a PR.

## Create a Release

1. Go to https://github.com/prometheus/client_java/releases
Expand Down
27 changes: 23 additions & 4 deletions docs/content/exporters/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,35 @@ All exporters the following exposition formats:

Moreover, gzip encoding is supported for each of these formats.

Scraping with a Prometheus server
---------------------------------
## Scraping with a Prometheus server

The Prometheus server sends an `Accept` header to specify which format is requested. By default, the Prometheus server will scrape OpenMetrics text format with gzip encoding. If the Prometheus server is started with `--enable-feature=native-histograms`, it will scrape Prometheus protobuf format instead.

Viewing with a Web Browser
--------------------------
## Viewing with a Web Browser

If you view the `/metrics` endpoint with your Web browser you will see Prometheus text format. For quick debugging of the other formats, exporters provide a `debug` URL parameter:

* `/metrics?debug=openmetrics`: View OpenMetrics text format.
* `/metrics?debug=text`: View Prometheus text format.
* `/metrics?debug=prometheus-protobuf`: View a text representation of the Prometheus protobuf format.

## Exclude protobuf exposition format

You can exclude the protobuf exposition format by including the
`prometheus-metrics-exposition-textformats` module and excluding the
`prometheus-metrics-exposition-formats` module in your build file.

For example, in Maven:

```xml
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exporter-httpserver</artifactId>
<exclusions>
<exclusion>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
</exclusion>
</exclusions>
</dependency>
```
8 changes: 8 additions & 0 deletions docs/content/getting-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ implementation 'io.prometheus:prometheus-metrics-exporter-httpserver:$version'

There are alternative exporters as well, for example if you are using a Servlet container like Tomcat or Undertow you might want to use `prometheus-exporter-servlet-jakarta` rather than a standalone HTTP server.

{{< hint type=note >}}

If you do not use the protobuf exposition format, you can
[exclude](../../exporters/formats#exclude-protobuf-exposition-format)
it from the dependencies.

{{< /hint >}}

# Dependency management

A Bill of Material
Expand Down
10 changes: 4 additions & 6 deletions docs/content/internals/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,23 @@ The illustration below shows the internal architecture of the Prometheus Java cl

![Internal architecture of the Prometheus Java client library](/client_java/images/model.png)

prometheus-metrics-core
-----------------------
## prometheus-metrics-core

This is the user facing metrics library, implementing the core metric types, like [Counter](/client_java/api/io/prometheus/metrics/core/metrics/Counter.html), [Gauge](/client_java/api/io/prometheus/metrics/core/metrics/Gauge.html) [Histogram](/client_java/api/io/prometheus/metrics/core/metrics/Histogram.html), and so on.

All metric types implement the [Collector](/client_java/api/io/prometheus/metrics/model/registry/Collector.html) interface, i.e. they provide a [collect()](/client_java/api/io/prometheus/metrics/model/registry/Collector.html#collect()) method to produce snapshots.

prometheus-metrics-model
------------------------
## prometheus-metrics-model

The model is an internal library, implementing read-only immutable snapshots. These snapshots are returned by the [Collector.collect()](/client_java/api/io/prometheus/metrics/model/registry/Collector.html#collect()) method.

There is no need for users to use `prometheus-metrics-model` directly. Users should use the API provided by `prometheus-metrics-core`, which includes the core metrics as well as callback metrics.

However, maintainers of 3rd party metrics libraries might want to use `prometheus-metrics-model` if they want to add Prometheus exposition formats to their metrics library.

exporters and exposition formats
--------------------------------
## Exporters and exposition formats

The `prometheus-metrics-exposition-formats` module converts snapshots to Prometheus exposition formats, like text format, OpenMetrics text format, or Prometheus protobuf format.

The exporters like `prometheus-metrics-exporter-httpserver` or `prometheus-metrics-exporter-servlet-jakarta` use this to convert snapshots into the right format depending on the `Accept` header in the scrape request.

12 changes: 10 additions & 2 deletions integration-tests/it-common/pom.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
Expand All @@ -11,11 +12,18 @@
<artifactId>it-common</artifactId>

<name>Integration Tests - Common Utilities</name>
<url>http://github.com/prometheus/client_java</url>
<description>
Common utilities for integration tests
</description>

<dependencies>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>

<build>
<testResources>
<testResource>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
package io.prometheus.client.it.common;

import static java.nio.charset.StandardCharsets.UTF_8;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.fail;

import io.prometheus.metrics.expositionformats.generated.com_google_protobuf_4_28_3.Metrics;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.zip.GZIPInputStream;
import org.apache.commons.io.IOUtils;
import org.junit.jupiter.api.AfterEach;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.GenericContainer;

public abstract class ExporterTest {
private final GenericContainer<?> sampleAppContainer;
private final Volume sampleAppVolume;
protected final String sampleApp;

public ExporterTest(String sampleApp) throws IOException, URISyntaxException {
this.sampleApp = sampleApp;
this.sampleAppVolume =
Volume.create("it-exporter")
.copy("../../it-" + sampleApp + "/target/" + sampleApp + ".jar");
this.sampleAppContainer =
new GenericContainer<>("openjdk:17")
.withFileSystemBind(sampleAppVolume.getHostPath(), "/app", BindMode.READ_ONLY)
.withWorkingDirectory("/app")
.withLogConsumer(LogConsumer.withPrefix(sampleApp))
.withExposedPorts(9400);
}

// @BeforeEach?
protected void start() {
start("success");
}

protected void start(String outcome) {
sampleAppContainer
.withCommand("java", "-jar", "/app/" + sampleApp + ".jar", "9400", outcome)
.start();
}

@AfterEach
public void tearDown() throws IOException {
sampleAppContainer.stop();
sampleAppVolume.remove();
}

public static void assertContentType(String expected, String actual) {
if (!expected.replace(" ", "").equals(actual)) {
assertThat(actual).isEqualTo(expected);
}
}

protected Response scrape(String method, String queryString, String... requestHeaders)
throws IOException {
return scrape(
method,
new URL(
"http://localhost:"
+ sampleAppContainer.getMappedPort(9400)
+ "/metrics?"
+ queryString),
requestHeaders);
}

public static Response scrape(String method, URL url, String... requestHeaders)
throws IOException {
long timeoutMillis = TimeUnit.SECONDS.toMillis(5);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod(method);
for (int i = 0; i < requestHeaders.length; i += 2) {
con.setRequestProperty(requestHeaders[i], requestHeaders[i + 1]);
}
long start = System.currentTimeMillis();
Exception exception = null;
while (System.currentTimeMillis() - start < timeoutMillis) {
try {
if (con.getResponseCode() == 200) {
return new Response(
con.getResponseCode(),
con.getHeaderFields(),
IOUtils.toByteArray(con.getInputStream()));
} else {
return new Response(
con.getResponseCode(),
con.getHeaderFields(),
IOUtils.toByteArray(con.getErrorStream()));
}
} catch (Exception e) {
exception = e;
try {
Thread.sleep(100);
} catch (InterruptedException ignored) {
// ignore
}
}
}
if (exception != null) {
exception.printStackTrace();
}
fail("timeout while getting metrics from " + url);
return null; // will not happen
}

public static class Response {
public final int status;
private final Map<String, String> headers;
public final byte[] body;

private Response(int status, Map<String, List<String>> headers, byte[] body) {
this.status = status;
this.headers = new HashMap<>(headers.size());
this.body = body;
for (Map.Entry<String, List<String>> entry : headers.entrySet()) {
if (entry.getKey()
!= null) { // HttpUrlConnection uses pseudo key "null" for the status line
this.headers.put(entry.getKey().toLowerCase(Locale.ROOT), entry.getValue().get(0));
}
}
}

public String getHeader(String name) {
// HTTP headers are case-insensitive
return headers.get(name.toLowerCase(Locale.ROOT));
}

public String stringBody() {
return new String(body, UTF_8);
}

public String gzipBody() throws IOException {
return new String(
IOUtils.toByteArray(new GZIPInputStream(new ByteArrayInputStream(body))), UTF_8);
}

public List<Metrics.MetricFamily> protoBody() throws IOException {
List<Metrics.MetricFamily> metrics = new ArrayList<>();
InputStream in = new ByteArrayInputStream(body);
while (in.available() > 0) {
metrics.add(Metrics.MetricFamily.parseDelimitedFrom(in));
}
return metrics;
}
}
}
69 changes: 69 additions & 0 deletions integration-tests/it-exporter/it-exporter-no-protobuf/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.prometheus</groupId>
<artifactId>it-exporter</artifactId>
<version>1.3.3</version>
</parent>

<artifactId>it-exporter-no-protobuf</artifactId>

<name>Integration Tests - HTTPServer Exporter Sample - no protobuf</name>
<description>
HTTPServer Sample for the Exporter Integration Test without protobuf
</description>

<dependencies>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exporter-httpserver</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-core</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>

<build>
<finalName>exporter-no-protobuf</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>io.prometheus.metrics.it.exporter.httpserver.HTTPServerSample</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Loading

0 comments on commit 66f4dc8

Please sign in to comment.