Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VCS metrics from Github receiver #1383

Open
wants to merge 38 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
44a1c81
[cicd] add first cicd metric to registry, created cicd-metrics doc
christophe-kamphaus-jemmic Sep 2, 2024
700412a
[cicd] link to cicd doc from parent readme
christophe-kamphaus-jemmic Sep 2, 2024
4d8e3a9
[cicd] fix typo vsc -> vcs
christophe-kamphaus-jemmic Sep 3, 2024
9bd4a95
[cicd] fix toc
christophe-kamphaus-jemmic Sep 3, 2024
592483f
[cicd] use {change} as metric unit
christophe-kamphaus-jemmic Sep 3, 2024
238c764
[cicd] formatting and rename metrics section
christophe-kamphaus-jemmic Sep 3, 2024
ef91dfa
[cicd] add remaining metrics from githubreceiver
christophe-kamphaus-jemmic Sep 3, 2024
f712312
[cicd] fix markdown code quotation
christophe-kamphaus-jemmic Sep 3, 2024
8c2e8a7
[cicd] add changelog entry
christophe-kamphaus-jemmic Sep 3, 2024
a148a4e
[cicd] add closed state to vcs.change.state
christophe-kamphaus-jemmic Sep 6, 2024
347b15b
[cicd] improved brief for vcs.change.state
christophe-kamphaus-jemmic Sep 6, 2024
afaa1f5
[cicd] update vcs.change.state briefs
christophe-kamphaus-jemmic Sep 15, 2024
5a9ffe7
[cicd] combine vcs.change.time_* metrics into one histogram metric
christophe-kamphaus-jemmic Sep 15, 2024
2da96a5
[cicd] update vcs.repository.ref.count brief to follow vcs.repository…
christophe-kamphaus-jemmic Sep 15, 2024
c232041
[cicd] fix metric and table mismatch
christophe-kamphaus-jemmic Sep 15, 2024
aab9f90
[cicd] change instrument from gauge to updowncounter
christophe-kamphaus-jemmic Sep 15, 2024
41543cd
[cicd] replace vcs.repository.name with vcs.repository.url.full in ci…
christophe-kamphaus-jemmic Sep 15, 2024
7f35251
Merge branch 'main' into 1372-vcs-metrics
christophe-kamphaus-jemmic Sep 17, 2024
a5eff73
[cicd] apply changes from #1411 to docs/cicd/cicd-metrics.md
christophe-kamphaus-jemmic Sep 17, 2024
7107911
[cicd] fix yamllint
christophe-kamphaus-jemmic Sep 17, 2024
15fe1fb
Merge branch 'main' into 1372-vcs-metrics
christophe-kamphaus-jemmic Oct 2, 2024
30c8fdb
Merge branch 'main' into 1372-vcs-metrics
christophe-kamphaus-jemmic Oct 3, 2024
344eeed
[cicd] Add /docs/cicd/ to CODEOWNERS
christophe-kamphaus-jemmic Oct 3, 2024
cfcdaf1
[cicd] rename metric and improve description
christophe-kamphaus-jemmic Oct 3, 2024
e174ef3
[cicd] add (PR/MR) to all vcs metric briefs relating to changes
christophe-kamphaus-jemmic Oct 3, 2024
7d114d3
[cicd] rephrase examples in brief to be inclusive
christophe-kamphaus-jemmic Oct 3, 2024
b92cc08
[cicd] make repository count an updowncounter
christophe-kamphaus-jemmic Oct 3, 2024
84b2b6a
[cicd] add metric requirement levels
christophe-kamphaus-jemmic Oct 3, 2024
163e498
[cicd] add wip state to vcs.change.state
christophe-kamphaus-jemmic Oct 3, 2024
d2ce62f
[cicd] add changelist to briefs for change
christophe-kamphaus-jemmic Oct 3, 2024
536a2a7
[cicd] rename vcs.repository.change.* to vcs.change.*
christophe-kamphaus-jemmic Oct 3, 2024
cfffb4f
[cicd] rename vcs.repository.ref.* to vcs.ref.*
christophe-kamphaus-jemmic Oct 3, 2024
0e02a47
[cicd] follow procedure for renaming attributes
christophe-kamphaus-jemmic Oct 3, 2024
1a89934
Merge branch 'main' into 1372-vcs-metrics
christophe-kamphaus-jemmic Oct 10, 2024
b5c4660
[cicd] change instrument type from "gauge histogram" back to gauge
christophe-kamphaus-jemmic Oct 10, 2024
dafdad7
[cicd] combine lines added/removed and revisions ahead/behind metrics
christophe-kamphaus-jemmic Oct 10, 2024
3e122e8
[cicd] fix minor brief issues
christophe-kamphaus-jemmic Oct 10, 2024
513ae8b
Merge branch 'main' into 1372-vcs-metrics
christophe-kamphaus-jemmic Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .chloggen/1372-vcs-metrics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: breaking

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: vcs

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add the VCS metrics from the GitHub Receiver

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [1372]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
- The VCS attribute `vcs.change.state` has been added to the registry.
- `vcs.repository.change.*` attributes have been deprecated and moved to `vcs.change.*`.
- `vcs.repository.ref.*` attributes have been deprecated and moved to `vcs.ref.*`.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@
/model/user/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-security-approvers

# CICD semantic conventions
/docs/cicd/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-cicd-approvers
/model/artifact/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-cicd-approvers
/model/cicd/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-cicd-approvers
/model/code/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-cicd-approvers
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ The benefit to using Semantic Conventions is in following a common naming scheme
Semantic Conventions are defined for the following areas:

* **[General](general/README.md): General Semantic Conventions**.
* [CICD](cicd/cicd-metrics.md): Semantic Conventions for CICD systems.
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
* [Cloud Providers](cloud-providers/README.md): Semantic Conventions for cloud providers libraries.
* [CloudEvents](cloudevents/README.md): Semantic Conventions for the CloudEvents specification.
* [Database](database/README.md): Semantic Conventions for database operations.
Expand Down
64 changes: 56 additions & 8 deletions docs/attributes-registry/vcs.md

Large diffs are not rendered by default.

404 changes: 404 additions & 0 deletions docs/cicd/cicd-metrics.md

Large diffs are not rendered by default.

61 changes: 61 additions & 0 deletions model/vcs/deprecated/registry-deprecated.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
groups:
- id: registry.vcs.deprecated
type: attribute_group
brief: >
"Describes deprecated vcs attributes."
attributes:
- id: vcs.repository.ref.name
type: string
stability: experimental
deprecated: 'Deprecated, use `vcs.ref.name` instead.'
brief: >
Deprecated, use `vcs.ref.name` instead.
examples: ["my-feature-branch", "tag-1-test"]
- id: vcs.repository.ref.type
type:
members:
- id: branch
value: branch
brief: "[branch](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddefbranchabranch)"
stability: experimental
- id: tag
value: tag
brief: "[tag](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftagatag)"
stability: experimental
stability: experimental
deprecated: 'Deprecated, use `vcs.ref.type` instead.'
brief: >
Deprecated, use `vcs.ref.type` instead.
examples: ["branch", "tag"]
- id: vcs.repository.ref.revision
type: string
stability: experimental
deprecated: 'Deprecated, use `vcs.ref.revision` instead.'
brief: >
Deprecated, use `vcs.ref.revision` instead.
examples:
[
"9d59409acf479dfa0df1aa568182e43e43df8bbe28d60fcf2bc52e30068802cc",
"main",
"123",
"HEAD",
]
- id: vcs.repository.change.title
type: string
stability: experimental
deprecated: 'Deprecated, use `vcs.change.title` instead.'
brief: >
Deprecated, use `vcs.change.title` instead.
examples:
[
"Fixes broken thing",
"feat: add my new feature",
"[chore] update dependency",
]
- id: vcs.repository.change.id
type: string
stability: experimental
deprecated: 'Deprecated, use `vcs.change.id` instead.'
brief: >
Deprecated, use `vcs.change.id` instead.
examples: ["123"]
147 changes: 147 additions & 0 deletions model/vcs/metrics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
groups:
- id: metric.vcs.change.count
type: metric
metric_name: vcs.change.count
brief: 'The number of changes (pull requests/merge requests/changelists) in a repository, categorized by their state (e.g. open or merged)'
instrument: updowncounter
unit: "{change}"
stability: experimental
attributes:
- ref: vcs.change.state
requirement_level: required
- ref: vcs.repository.url.full
requirement_level: required
- id: metric.vcs.change.duration
type: metric
metric_name: vcs.change.duration
brief: 'The time duration a change (pull request/merge request/changelist) has been in a given state.'
instrument: gauge
unit: "s"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- ref: vcs.change.state
requirement_level: required
- id: metric.vcs.change.time_to_approval
type: metric
metric_name: vcs.change.time_to_approval
brief: 'The amount of time it took a change (pull request/merge request/changelist) to go from open to approved'
instrument: gauge
unit: "s"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- id: metric.vcs.repository.count
type: metric
metric_name: vcs.repository.count
brief: 'The number of repositories in an organization'
instrument: updowncounter
unit: "{repository}"
stability: experimental
attributes: []
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
- id: metric.vcs.ref.count
type: metric
metric_name: vcs.ref.count
brief: 'The number of refs of type branch or tag in a repository'
instrument: updowncounter
unit: "{ref}"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.type
requirement_level: required
- id: metric.vcs.change.lines_delta
type: metric
metric_name: vcs.change.lines_delta
brief: 'The number of lines added/removed in a change relative to the target branch of the change'
instrument: gauge
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be a histogram?

Copy link
Contributor

@jmacd jmacd Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you feel about the claim, here which says that we can form a histogram out of any fundamental instrument type? For "lines_delta" specifically, it sounds like an Counter-- if you were to aggregate a number of refs or repositories together, would you want to sum the lines added and removed.

Is there an attribute to distinguish "add" from "remove" in this situation? If you were to aggregate-away this attribute, would you expect to have a single number that is added+removed?

unit: "{line}"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- ref: vcs.ref.type
requirement_level: required
- ref: vcs.line_change.type
requirement_level: required
- id: metric.vcs.ref.lines_delta
type: metric
metric_name: vcs.ref.lines_delta
brief: 'The number of lines added/removed in a ref (branch) relative to the previous day'
instrument: gauge
unit: "{line}"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- ref: vcs.ref.type
requirement_level: required
- ref: vcs.line_change.type
requirement_level: required
- id: metric.vcs.change.revisions_delta
type: metric
metric_name: vcs.change.revisions_delta
brief: 'The number of revisions (commits) a change is ahead/behind the target branch of the change'
instrument: gauge
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider using histogram vcs.repository.ref.revisions.delta (vcs.ref.revisions.lag, etc) and make ahead/behind an attribute.

Hopefully at some point negative measurements will be supported - open-telemetry/opentelemetry-specification#4155 (comment) and life would be even easier - no additional attribute will be necessary.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, I have changed back any histograms to gauge.
Let's discuss on Monday how to proceed regarding instrument type.

unit: "{revision}"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- ref: vcs.ref.type
requirement_level: required
- ref: vcs.revision_delta.type
requirement_level: required
- id: metric.vcs.ref.revisions_delta
type: metric
metric_name: vcs.ref.revisions_delta
brief: 'The number of revisions (commits) a ref (branch) is ahead/behind the default branch (trunk)'
instrument: gauge
unit: "{revision}"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- ref: vcs.ref.type
requirement_level: required
- ref: vcs.revision_delta.type
requirement_level: required
- id: metric.vcs.ref.time
type: metric
metric_name: vcs.ref.time
brief: 'Time a ref (branch) created from the default branch (trunk) has existed. The `ref.type` attribute will always be `branch`'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we instead report creation time for each branch? Then people can derive other things including deltas from it. It could also be useful for tags. E.g. vcs.ref.created|creation_time

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adrielp what's your opinion?

instrument: gauge
unit: "s"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
- ref: vcs.ref.name
requirement_level: required
- ref: vcs.ref.type
requirement_level: required
- id: metric.vcs.repository.contributor.count
type: metric
metric_name: vcs.repository.contributor.count
brief: 'The number of unique contributors to a repository'
instrument: gauge
lmolkova marked this conversation as resolved.
Show resolved Hide resolved
unit: "{contributor}"
stability: experimental
attributes:
- ref: vcs.repository.url.full
requirement_level: required
69 changes: 62 additions & 7 deletions model/vcs/registry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@ groups:
"https://github.com/opentelemetry/open-telemetry-collector-contrib",
"https://gitlab.com/my-org/my-project/my-projects-project/repo",
]
- id: vcs.repository.ref.name
- id: vcs.ref.name
type: string
stability: experimental
brief: >
The name of the
[reference](https://git-scm.com/docs/gitglossary#def_ref) such as
**branch** or **tag** in the repository.
examples: ["my-feature-branch", "tag-1-test"]
- id: vcs.repository.ref.type
- id: vcs.ref.type
type:
members:
- id: branch
Expand All @@ -40,7 +40,7 @@ groups:
brief: >
The type of the [reference](https://git-scm.com/docs/gitglossary#def_ref) in the repository.
examples: ["branch", "tag"]
- id: vcs.repository.ref.revision
- id: vcs.ref.revision
type: string
stability: experimental
brief: >
Expand All @@ -63,11 +63,41 @@ groups:
"123",
"HEAD",
]
- id: vcs.repository.change.title
- id: vcs.revision_delta.type
type:
members:
- id: behind
value: behind
brief: "How many revisions the change is behind the target ref."
stability: experimental
- id: ahead
value: ahead
brief: "How many revisions the change is ahead of the target ref."
stability: experimental
stability: experimental
brief: >
The type of revision comparison.
examples: ["ahead", "behind"]
- id: vcs.line_change.type
type:
members:
- id: added
value: added
brief: "How many lines were added."
stability: experimental
- id: removed
value: removed
brief: "How many lines were removed."
stability: experimental
stability: experimental
brief: >
The type of line change being measured on a branch or change.
examples: ["added", "removed"]
- id: vcs.change.title
type: string
stability: experimental
brief: >
The human readable title of the change (pull request/merge request).
The human readable title of the change (pull request/merge request/changelist).
This title is often a brief summary of the change and may get merged
in to a ref as the commit summary.
examples:
Expand All @@ -76,10 +106,35 @@ groups:
"feat: add my new feature",
"[chore] update dependency",
]
- id: vcs.repository.change.id
- id: vcs.change.id
type: string
stability: experimental
brief: >
The ID of the change (pull request/merge request) if applicable. This
The ID of the change (pull request/merge request/changelist) if applicable. This
is usually a unique (within repository) identifier generated by the VCS system.
examples: ["123"]
- id: vcs.change.state
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
type:
members:
- id: open
value: open
brief: "Open means the change is currently active and under review. It hasn't been merged into the target branch yet, and it's still possible to make changes or add comments."
stability: experimental
- id: wip
value: wip
brief: "WIP (work-in-progress, draft) means the change is still in progress and not yet ready for a full review. It might still undergo significant changes."
stability: experimental
- id: closed
value: closed
brief: >-
Closed means the merge request has been closed without merging. This can happen for various reasons, such as the changes being deemed unnecessary,
the issue being resolved in another way, or the author deciding to withdraw the request.
stability: experimental
- id: merged
value: merged
brief: "Merged indicates that the change has been successfully integrated into the target codebase."
stability: experimental
stability: experimental
brief: >
The state of the change (pull request/merge request/changelist).
examples: ["open", "closed", "merged"]
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
10 changes: 10 additions & 0 deletions schema-next.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@ file_format: 1.1.0
schema_url: https://opentelemetry.io/schemas/next
versions:
next:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1383
- rename_attributes:
attribute_map:
vcs.repository.change.id: vcs.change.id
vcs.repository.change.title: vcs.change.title
vcs.repository.ref.name: vcs.ref.name
vcs.repository.ref.revision: vcs.ref.revision
vcs.repository.ref.type: vcs.ref.type
1.28.0:
metrics:
changes:
Expand Down