Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the Exponential Histogram Aggregator. #4245

Merged
merged 34 commits into from
Aug 4, 2023
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6862bf7
Adds Exponential Histograms aggregator
MadVikingGod May 31, 2023
46f1825
Added aggregation to the pipeline.
MadVikingGod Jun 8, 2023
bd8ac1b
Add no allocation if cap is available.
MadVikingGod Jun 16, 2023
bdd0602
Expand tests
MadVikingGod Jun 20, 2023
354b1fa
Fix lint
MadVikingGod Jun 20, 2023
181d036
Fix 64 bit math on 386 platform.
MadVikingGod Jun 21, 2023
c92b2b7
Fix tests to work in go 1.19.
MadVikingGod Jun 21, 2023
2d514e0
Merge branch 'main' into mvg/exp-hist-agg
MadVikingGod Jun 21, 2023
54f540c
Merge branch 'main' into mvg/exp-hist-agg
pellared Jun 22, 2023
9b1b8d0
fix codespell
MadVikingGod Jun 21, 2023
b41b1e4
Add example
MadVikingGod Jun 22, 2023
978afbb
Update sdk/metric/aggregation/aggregation.go
MadVikingGod Jun 23, 2023
8b6f044
Merge branch 'main' into mvg/exp-hist-agg
hanyuancheung Jun 26, 2023
89a65d5
Update sdk/metric/aggregation/aggregation.go
hanyuancheung Jun 26, 2023
41757d1
Update sdk/metric/aggregation/aggregation.go
hanyuancheung Jun 26, 2023
9536e35
Changelog
MadVikingGod Jun 26, 2023
d0139f3
Merge branch 'main' into mvg/exp-hist-agg
hanyuancheung Jun 27, 2023
734f2ac
Merge branch 'main' into mvg/exp-hist-agg
MadVikingGod Jul 5, 2023
07b9cc7
Fix move
MadVikingGod Jul 5, 2023
6a68057
Merge branch 'main' into mvg/exp-hist-agg
hanyuancheung Jul 11, 2023
a4ad0b0
Address feedback from the PR.
MadVikingGod Jul 7, 2023
c0f282b
Merge remote-tracking branch 'upstream/main' into mvg/exp-hist-agg
MadVikingGod Jul 17, 2023
8ca255e
Update expo histo to new aggregator format.
MadVikingGod Jul 17, 2023
18b5f91
Fix lint
MadVikingGod Jul 17, 2023
cb3693f
Remove Zero Threshold from config of expo histograms
MadVikingGod Jul 19, 2023
9b55b27
Merge remote-tracking branch 'upstream/main' into mvg/exp-hist-agg
MadVikingGod Jul 19, 2023
66c44df
Merge remote-tracking branch 'upstream/main' into mvg/exp-hist-agg
MadVikingGod Jul 20, 2023
581c7d6
Remove DefaultExponentialHistogram()
MadVikingGod Jul 20, 2023
14aee40
Refactor GetBin, and address PR Feedback
MadVikingGod Aug 2, 2023
4244780
Merge remote-tracking branch 'upstream/main' into mvg/exp-hist-agg
MadVikingGod Aug 4, 2023
dfc13fd
Address PR feedback
MadVikingGod Aug 4, 2023
abdcaa4
Fix comment in wrong location
MadVikingGod Aug 4, 2023
4bb46fa
Fix misapplied PR feedback
MadVikingGod Aug 4, 2023
e626be7
Fix codespell
MadVikingGod Aug 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

- Add `ManualReader` struct in `go.opentelemetry.io/otel/sdk/metric`. (#4244)
- Add `PeriodicReader` struct in `go.opentelemetry.io/otel/sdk/metric`. (#4244)
- Add support for Exponential Histogram Aggregations. A Histogram can be configured as an Exponential Histogram using a view with the `go.opentelemetry.io/otel/sdk/metric/aggregation.ExponentialHistogram` as the aggregation. (#4245)
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
- Add `Exporter` struct in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. (#4272)
- Add `Exporter` struct in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#4272)

Expand Down
62 changes: 62 additions & 0 deletions sdk/metric/aggregation/aggregation.go
Original file line number Diff line number Diff line change
Expand Up @@ -164,3 +164,65 @@ func (h ExplicitBucketHistogram) Copy() Aggregation {
NoMinMax: h.NoMinMax,
}
}

// ExponentialHistogram is an aggregation that summarizes a set of
// measurements as an histogram with buckets widths that grow exponentially.
MadVikingGod marked this conversation as resolved.
Show resolved Hide resolved
type ExponentialHistogram struct {
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
// MaxSize is the maximum number of buckets to use for the histogram.
MaxSize int
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
// MaxScale is the maximum resolution scale to use for the histogram.
MaxScale int
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
// ZeroThreshold sets the minimum value blow which will be recorded as zero.
ZeroThreshold float64
MadVikingGod marked this conversation as resolved.
Show resolved Hide resolved

// NoMinMax indicates whether to not record the min and max of the
// distribution. By default, these extrema are recorded.
//
// Recording these extrema for cumulative data is expected to have little
// value, they will represent the entire life of the instrument instead of
// just the current collection cycle. It is recommended to set this to true
// for that type of data to avoid computing the low-value extrema.
NoMinMax bool
}

// DefaultExponentialHistogram returns the default Exponential Histogram aggregation.
func DefaultExponentialHistogram() ExponentialHistogram {
return ExponentialHistogram{
MaxSize: 160,
MaxScale: 20,
ZeroThreshold: 0.0,
}
}
MadVikingGod marked this conversation as resolved.
Show resolved Hide resolved

var _ Aggregation = ExponentialHistogram{}

// private attempts to ensure no user-defined Aggregation is allowed. The
// OTel specification does not allow user-defined Aggregation currently.
func (e ExponentialHistogram) private() {}

// Copy returns a deep copy of the Aggregation.
func (e ExponentialHistogram) Copy() Aggregation {
return e
}

const (
expoMaxScale = 20
expoMinScale = -10
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
)

// errExpoHist is returned by misconfigured ExplicitBucketHistograms.
var errExpoHist = fmt.Errorf("%w: explicit bucket histogram", errAgg)
MadVikingGod marked this conversation as resolved.
Show resolved Hide resolved

// Err returns an error for any misconfigured Aggregation.
func (e ExponentialHistogram) Err() error {
if e.MaxScale < expoMinScale {
MrAlias marked this conversation as resolved.
Show resolved Hide resolved
return fmt.Errorf("%w: max size %d is less than minimum scale %d", errExpoHist, e.MaxSize, expoMinScale)
}
if e.MaxScale > expoMaxScale {
return fmt.Errorf("%w: max size %d is greater than maximum scale %d", errExpoHist, e.MaxSize, expoMaxScale)
}
if e.MaxSize <= 0 {
return fmt.Errorf("%w: max size %d is less than or equal to zero", errExpoHist, e.MaxSize)
}
return nil
}
30 changes: 30 additions & 0 deletions sdk/metric/aggregation/aggregation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,36 @@ func TestAggregationErr(t *testing.T) {
Boundaries: []float64{0, 1, 2, 1, 3, 4},
}.Err(), errAgg)
})

t.Run("ExponentialHistogramOperation", func(t *testing.T) {
assert.NoError(t, DefaultExponentialHistogram().Err())

assert.NoError(t, ExponentialHistogram{
MaxSize: 1,
NoMinMax: true,
}.Err())

assert.NoError(t, ExponentialHistogram{
MaxSize: 1024,
MaxScale: -3,
ZeroThreshold: 1.0,
}.Err())
})

t.Run("InvalidExponentialHistogramOperation", func(t *testing.T) {
// MazSize must be greater than 0
assert.ErrorIs(t, ExponentialHistogram{}.Err(), errAgg)

// MaxScale Must be <=20 and >= -10
assert.ErrorIs(t, ExponentialHistogram{
MaxSize: 1,
MaxScale: 30,
}.Err(), errAgg)
assert.ErrorIs(t, ExponentialHistogram{
MaxSize: 1,
MaxScale: -20,
}.Err(), errAgg)
})
}

func TestExplicitBucketHistogramDeepCopy(t *testing.T) {
Expand Down
19 changes: 19 additions & 0 deletions sdk/metric/internal/aggregate/aggregator_example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,23 @@ func (p *meter) Int64Histogram(string, ...metric.Int64HistogramOption) (metric.I
return hist, nil
}

func (p *meter) Int64ExponentialHistogram(string, ...metric.Int64HistogramOption) (metric.Int64Histogram, error) {
// This is an example of how a meter would create an aggregator for a new
// histogram. At this point the provider would determine the aggregation
// and temporality to used based on the Reader and View configuration.
// Assume here these are determined to be a delta explicit-bucket
// histogram.

aggregator := NewDeltaExponentialHistogram[int64](aggregation.DefaultExponentialHistogram())
hist := inst{aggregateFunc: aggregator.Aggregate}

p.aggregations = append(p.aggregations, aggregator.Aggregation())

fmt.Printf("using %T aggregator for histogram\n", aggregator)

return hist, nil
}

// inst is a generalized int64 synchronous counter, up-down counter, and
// histogram used for demonstration purposes only.
type inst struct {
Expand All @@ -103,9 +120,11 @@ func Example() {
_, _ = m.Int64Counter("counter example")
_, _ = m.Int64UpDownCounter("up-down counter example")
_, _ = m.Int64Histogram("histogram example")
_, _ = m.Int64ExponentialHistogram("exponential histogram example")

// Output:
// using *aggregate.cumulativeSum[int64] aggregator for counter
// using *aggregate.lastValue[int64] aggregator for up-down counter
// using *aggregate.deltaHistogram[int64] aggregator for histogram
// using *aggregate.deltaExponentialHistogram[int64] aggregator for histogram
}
Loading