✨ feat: implement initial benchmark tests

[OchiengEd]

Description

This PR is for implementing benchmarking tests which will be used in CI to evaluate change in performance / compute resource utilization as new features are implemented.

Reviewer Checklist

• API Go Documentation
• Tests: Unit Tests (and E2E Tests, if appropriate)
• Comprehensive Commit Messages
• Links to related GitHub Issue(s)

Relates to #920

[netlify]

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	1075eb3
🔍 Latest deploy log	https://app.netlify.com/sites/olmv1/deploys/66b52ff7e13e21000831fa97
😎 Deploy Preview	https://deploy-preview-1071--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 75.23%. Comparing base (6864b0c) to head (1075eb3).
Report is 168 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/rukpak/source/image_registry.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1071      +/-   ##
==========================================
- Coverage   75.28%   75.23%   -0.06%     
==========================================
  Files          35       35              
  Lines        1914     1914              
==========================================
- Hits         1441     1440       -1     
- Misses        330      331       +1     
  Partials      143      143              

Flag	Coverage Δ
e2e	57.41% <0.00%> (-0.06%)	⬇️
unit	50.78% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joelanford]

Broadly, I would suggest that we focus benchmarks on areas of code that we think may be particularly prone to CPU time and allocation.

Do we have any indication that the contentmanager, tokengetter, or schemebuilder are particularly problematic? If not, I'm concerned that these benchmarks could become a source of noise and toil.

In particular, I think the four best places to add benchmarks are:

• The catalog cache and/or client.
• The Resolver interface implementation.
• The image unpacker.
• The registry+v1 to helm chart conversion.

In my observation, these seem to be the largest contributors to the duration of a Reconcile call, so even performance regressions in these areas likely have a bigger impact that other areas of the codebase.

[OchiengEd]

In my observation, these seem to be the largest contributors to the duration of a Reconcile

Totally agree. I was intent on going through the code base and identifying where we have more CPU / memory intensive tasks. However, to start working on GH actions, I wanted to have something in place.

[OchiengEd]

Still needs a step to compare the change in performance as a result of the PR .

In addition, what do we think would be a good threshold for a computing performance degradation

[OchiengEd]

This bundle image will also need to be changed to something that we expect to stay around longer.

[OchiengEd]

While reviewing, consider:

1. What should be threshold for an increase in computational resource utilization?
2. And should we error when above threshold is exceeded?
3. Should we add a comment with the benchmark results?
4. Informational logs in the function being benchmarked will break the benchmark format. As a workaround, should we remove the comments altogether or suppress them when benchmarks are running?

[OchiengEd]

During a meeting on the benchmarking of operator-controller functions held on 08/13/2024, the following action items were identified:

• Committing a golden benchmark to the repository and comparing benchmarks when new PRs are created to the golden / reference benchmark
• Set the threshold as low as possible, preferrably zero, and alert when there is any measurable change
• Switch the analysis of the benchmark data to use Golang tools / modules from Pandas which is written in Python
• Fail / Alert when a benchmark is removed from the golden benchmark or when a benchmark which does not exist in the Golden benchmark is added to a PR

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.